Prior node selection for scheduling workflows in a heterogeneous system

Hidehiro Kanemitsu, Masaki Hanada, Hidenori Nakazato

    Research output: Contribution to journalArticle

    2 Citations (Scopus)

    Abstract

    Many workflow scheduling algorithms for heterogeneous systems have been developed to satisfy multiple requirements such as minimizing schedule length while maximizing throughput. In particular, in list-based scheduling approaches, the schedule length depends on the given nodes as well as the task allocation and ordering policies. This is because the scheduling priority is derived by averaging the execution time and communication time of the given nodes. If the set of nodes can be adjusted before the scheduling tasks, a small schedule length can be achieved. In this paper, we propose a prior node selection algorithm, called lower bound based candidate node selection (LBCNS) to select a subset of given nodes to minimize the schedule length while fairly scheduling each job. Our proposal has two approaches: (i) LBCNS_DEFAULT, which considers the job characteristics and each node's performance, and (ii) priority-based LBCNS, which additionally takes each scheduling priority into account for a dedicated task scheduling algorithm. The experimental results of extensive simulations show that LBCNS_DEFAULT has the best fairness for scheduling multiple workflow jobs, while priority-based LBCNS achieves the minimum schedule length with the highest efficiency for a single workflow job and multiple workflow jobs.

    Original languageEnglish
    Pages (from-to)155-177
    Number of pages23
    JournalJournal of Parallel and Distributed Computing
    Volume109
    DOIs
    Publication statusPublished - 2017 Nov 1

    Fingerprint

    Heterogeneous Systems
    Work Flow
    Scheduling
    Vertex of a graph
    Schedule
    Scheduling algorithms
    Lower bound
    Task Scheduling
    Scheduling Algorithm
    Throughput
    Ordering Policy
    Task Allocation
    Job Scheduling
    Communication
    Fairness
    Execution Time
    High Efficiency
    Averaging
    Minimise

    Keywords

    • DAG
    • Heterogeneous system
    • Node grouping
    • Processor grouping
    • Task scheduling
    • Workflow scheduling

    ASJC Scopus subject areas

    • Theoretical Computer Science
    • Software
    • Hardware and Architecture
    • Computer Networks and Communications
    • Artificial Intelligence

    Cite this

    Prior node selection for scheduling workflows in a heterogeneous system. / Kanemitsu, Hidehiro; Hanada, Masaki; Nakazato, Hidenori.

    In: Journal of Parallel and Distributed Computing, Vol. 109, 01.11.2017, p. 155-177.

    Research output: Contribution to journalArticle

    @article{9e55f2902b264debb4a72dff55ba3313,
    title = "Prior node selection for scheduling workflows in a heterogeneous system",
    abstract = "Many workflow scheduling algorithms for heterogeneous systems have been developed to satisfy multiple requirements such as minimizing schedule length while maximizing throughput. In particular, in list-based scheduling approaches, the schedule length depends on the given nodes as well as the task allocation and ordering policies. This is because the scheduling priority is derived by averaging the execution time and communication time of the given nodes. If the set of nodes can be adjusted before the scheduling tasks, a small schedule length can be achieved. In this paper, we propose a prior node selection algorithm, called lower bound based candidate node selection (LBCNS) to select a subset of given nodes to minimize the schedule length while fairly scheduling each job. Our proposal has two approaches: (i) LBCNS_DEFAULT, which considers the job characteristics and each node's performance, and (ii) priority-based LBCNS, which additionally takes each scheduling priority into account for a dedicated task scheduling algorithm. The experimental results of extensive simulations show that LBCNS_DEFAULT has the best fairness for scheduling multiple workflow jobs, while priority-based LBCNS achieves the minimum schedule length with the highest efficiency for a single workflow job and multiple workflow jobs.",
    keywords = "DAG, Heterogeneous system, Node grouping, Processor grouping, Task scheduling, Workflow scheduling",
    author = "Hidehiro Kanemitsu and Masaki Hanada and Hidenori Nakazato",
    year = "2017",
    month = "11",
    day = "1",
    doi = "10.1016/j.jpdc.2017.06.005",
    language = "English",
    volume = "109",
    pages = "155--177",
    journal = "Journal of Parallel and Distributed Computing",
    issn = "0743-7315",
    publisher = "Academic Press Inc.",

    }

    TY - JOUR

    T1 - Prior node selection for scheduling workflows in a heterogeneous system

    AU - Kanemitsu, Hidehiro

    AU - Hanada, Masaki

    AU - Nakazato, Hidenori

    PY - 2017/11/1

    Y1 - 2017/11/1

    N2 - Many workflow scheduling algorithms for heterogeneous systems have been developed to satisfy multiple requirements such as minimizing schedule length while maximizing throughput. In particular, in list-based scheduling approaches, the schedule length depends on the given nodes as well as the task allocation and ordering policies. This is because the scheduling priority is derived by averaging the execution time and communication time of the given nodes. If the set of nodes can be adjusted before the scheduling tasks, a small schedule length can be achieved. In this paper, we propose a prior node selection algorithm, called lower bound based candidate node selection (LBCNS) to select a subset of given nodes to minimize the schedule length while fairly scheduling each job. Our proposal has two approaches: (i) LBCNS_DEFAULT, which considers the job characteristics and each node's performance, and (ii) priority-based LBCNS, which additionally takes each scheduling priority into account for a dedicated task scheduling algorithm. The experimental results of extensive simulations show that LBCNS_DEFAULT has the best fairness for scheduling multiple workflow jobs, while priority-based LBCNS achieves the minimum schedule length with the highest efficiency for a single workflow job and multiple workflow jobs.

    AB - Many workflow scheduling algorithms for heterogeneous systems have been developed to satisfy multiple requirements such as minimizing schedule length while maximizing throughput. In particular, in list-based scheduling approaches, the schedule length depends on the given nodes as well as the task allocation and ordering policies. This is because the scheduling priority is derived by averaging the execution time and communication time of the given nodes. If the set of nodes can be adjusted before the scheduling tasks, a small schedule length can be achieved. In this paper, we propose a prior node selection algorithm, called lower bound based candidate node selection (LBCNS) to select a subset of given nodes to minimize the schedule length while fairly scheduling each job. Our proposal has two approaches: (i) LBCNS_DEFAULT, which considers the job characteristics and each node's performance, and (ii) priority-based LBCNS, which additionally takes each scheduling priority into account for a dedicated task scheduling algorithm. The experimental results of extensive simulations show that LBCNS_DEFAULT has the best fairness for scheduling multiple workflow jobs, while priority-based LBCNS achieves the minimum schedule length with the highest efficiency for a single workflow job and multiple workflow jobs.

    KW - DAG

    KW - Heterogeneous system

    KW - Node grouping

    KW - Processor grouping

    KW - Task scheduling

    KW - Workflow scheduling

    UR - http://www.scopus.com/inward/record.url?scp=85021653967&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85021653967&partnerID=8YFLogxK

    U2 - 10.1016/j.jpdc.2017.06.005

    DO - 10.1016/j.jpdc.2017.06.005

    M3 - Article

    AN - SCOPUS:85021653967

    VL - 109

    SP - 155

    EP - 177

    JO - Journal of Parallel and Distributed Computing

    JF - Journal of Parallel and Distributed Computing

    SN - 0743-7315

    ER -