A heuristic scheduling approach to hybrid Makepsan problem for data intensive computing

Wei Liang, Yonghua Xiong, Min Wu, Qun Jin

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    1 Citation (Scopus)

    Abstract

    Data intensive computing (DIC) offers an attractive option for business to remotely execute applications and load the computing resources from cloud in a streaming way. A key challenge in such environment is to increase the utilization of cloud cluster for the high throughput processing. One way of achieving this goal is to optimize the execution of computing jobs on the cluster. We observe that the order in which these jobs are executed can have a significant impact on their overall completion time (makespan). Our goal is to design a job scheduler that minimizes the makespan. In this study, a new formalization is introduced to present each job as a pair of disk processing and network transmitting two-stage durations. Due to the streaming processing feature, the two-stage operations are executed in an overlap manner and may lead to both one-stage and two-stage scheduling situations. A novel heuristic scheduling strategy is proposed for this hybrid scheduling problem, and the performance of the method is confirmed by the experimental evaluation.

    Original languageEnglish
    Title of host publicationProceedings - 2015 IEEE International Conference on Smart City, SmartCity 2015, Held Jointly with 8th IEEE International Conference on Social Computing and Networking, SocialCom 2015, 5th IEEE International Conference on Sustainable Computing and Communications, SustainCom 2015, 2015 International Conference on Big Data Intelligence and Computing, DataCom 2015, 5th International Symposium on Cloud and Service Computing, SC2 2015
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    Pages932-937
    Number of pages6
    ISBN (Electronic)9781509018932
    DOIs
    Publication statusPublished - 2016 May 2
    EventIEEE International Conference on Smart City, SmartCity 2015 - Chengdu, China
    Duration: 2015 Dec 192015 Dec 21

    Other

    OtherIEEE International Conference on Smart City, SmartCity 2015
    CountryChina
    CityChengdu
    Period15/12/1915/12/21

    Fingerprint

    heuristics
    scheduling
    Scheduling
    Heuristics
    Streaming
    Computing
    Processing
    Completion Time
    Scheduler
    Formalization
    Experimental Evaluation
    High Throughput
    Scheduling Problem
    Overlap
    formalization
    resource
    Optimise
    Throughput
    Minimise
    Resources

    Keywords

    • Data Intensive Computing
    • Makespan
    • NP
    • Scheduling

    ASJC Scopus subject areas

    • Information Systems
    • Media Technology
    • Computer Science Applications
    • Signal Processing
    • Computer Networks and Communications
    • Modelling and Simulation
    • Sociology and Political Science
    • Urban Studies

    Cite this

    Liang, W., Xiong, Y., Wu, M., & Jin, Q. (2016). A heuristic scheduling approach to hybrid Makepsan problem for data intensive computing. In Proceedings - 2015 IEEE International Conference on Smart City, SmartCity 2015, Held Jointly with 8th IEEE International Conference on Social Computing and Networking, SocialCom 2015, 5th IEEE International Conference on Sustainable Computing and Communications, SustainCom 2015, 2015 International Conference on Big Data Intelligence and Computing, DataCom 2015, 5th International Symposium on Cloud and Service Computing, SC2 2015 (pp. 932-937). [7463845] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SmartCity.2015.190

    A heuristic scheduling approach to hybrid Makepsan problem for data intensive computing. / Liang, Wei; Xiong, Yonghua; Wu, Min; Jin, Qun.

    Proceedings - 2015 IEEE International Conference on Smart City, SmartCity 2015, Held Jointly with 8th IEEE International Conference on Social Computing and Networking, SocialCom 2015, 5th IEEE International Conference on Sustainable Computing and Communications, SustainCom 2015, 2015 International Conference on Big Data Intelligence and Computing, DataCom 2015, 5th International Symposium on Cloud and Service Computing, SC2 2015. Institute of Electrical and Electronics Engineers Inc., 2016. p. 932-937 7463845.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Liang, W, Xiong, Y, Wu, M & Jin, Q 2016, A heuristic scheduling approach to hybrid Makepsan problem for data intensive computing. in Proceedings - 2015 IEEE International Conference on Smart City, SmartCity 2015, Held Jointly with 8th IEEE International Conference on Social Computing and Networking, SocialCom 2015, 5th IEEE International Conference on Sustainable Computing and Communications, SustainCom 2015, 2015 International Conference on Big Data Intelligence and Computing, DataCom 2015, 5th International Symposium on Cloud and Service Computing, SC2 2015., 7463845, Institute of Electrical and Electronics Engineers Inc., pp. 932-937, IEEE International Conference on Smart City, SmartCity 2015, Chengdu, China, 15/12/19. https://doi.org/10.1109/SmartCity.2015.190
    Liang W, Xiong Y, Wu M, Jin Q. A heuristic scheduling approach to hybrid Makepsan problem for data intensive computing. In Proceedings - 2015 IEEE International Conference on Smart City, SmartCity 2015, Held Jointly with 8th IEEE International Conference on Social Computing and Networking, SocialCom 2015, 5th IEEE International Conference on Sustainable Computing and Communications, SustainCom 2015, 2015 International Conference on Big Data Intelligence and Computing, DataCom 2015, 5th International Symposium on Cloud and Service Computing, SC2 2015. Institute of Electrical and Electronics Engineers Inc. 2016. p. 932-937. 7463845 https://doi.org/10.1109/SmartCity.2015.190
    Liang, Wei ; Xiong, Yonghua ; Wu, Min ; Jin, Qun. / A heuristic scheduling approach to hybrid Makepsan problem for data intensive computing. Proceedings - 2015 IEEE International Conference on Smart City, SmartCity 2015, Held Jointly with 8th IEEE International Conference on Social Computing and Networking, SocialCom 2015, 5th IEEE International Conference on Sustainable Computing and Communications, SustainCom 2015, 2015 International Conference on Big Data Intelligence and Computing, DataCom 2015, 5th International Symposium on Cloud and Service Computing, SC2 2015. Institute of Electrical and Electronics Engineers Inc., 2016. pp. 932-937
    @inproceedings{645c74f1e8fe4f05a1b5b8ea8fbf64ab,
    title = "A heuristic scheduling approach to hybrid Makepsan problem for data intensive computing",
    abstract = "Data intensive computing (DIC) offers an attractive option for business to remotely execute applications and load the computing resources from cloud in a streaming way. A key challenge in such environment is to increase the utilization of cloud cluster for the high throughput processing. One way of achieving this goal is to optimize the execution of computing jobs on the cluster. We observe that the order in which these jobs are executed can have a significant impact on their overall completion time (makespan). Our goal is to design a job scheduler that minimizes the makespan. In this study, a new formalization is introduced to present each job as a pair of disk processing and network transmitting two-stage durations. Due to the streaming processing feature, the two-stage operations are executed in an overlap manner and may lead to both one-stage and two-stage scheduling situations. A novel heuristic scheduling strategy is proposed for this hybrid scheduling problem, and the performance of the method is confirmed by the experimental evaluation.",
    keywords = "Data Intensive Computing, Makespan, NP, Scheduling",
    author = "Wei Liang and Yonghua Xiong and Min Wu and Qun Jin",
    year = "2016",
    month = "5",
    day = "2",
    doi = "10.1109/SmartCity.2015.190",
    language = "English",
    pages = "932--937",
    booktitle = "Proceedings - 2015 IEEE International Conference on Smart City, SmartCity 2015, Held Jointly with 8th IEEE International Conference on Social Computing and Networking, SocialCom 2015, 5th IEEE International Conference on Sustainable Computing and Communications, SustainCom 2015, 2015 International Conference on Big Data Intelligence and Computing, DataCom 2015, 5th International Symposium on Cloud and Service Computing, SC2 2015",
    publisher = "Institute of Electrical and Electronics Engineers Inc.",
    address = "United States",

    }

    TY - GEN

    T1 - A heuristic scheduling approach to hybrid Makepsan problem for data intensive computing

    AU - Liang, Wei

    AU - Xiong, Yonghua

    AU - Wu, Min

    AU - Jin, Qun

    PY - 2016/5/2

    Y1 - 2016/5/2

    N2 - Data intensive computing (DIC) offers an attractive option for business to remotely execute applications and load the computing resources from cloud in a streaming way. A key challenge in such environment is to increase the utilization of cloud cluster for the high throughput processing. One way of achieving this goal is to optimize the execution of computing jobs on the cluster. We observe that the order in which these jobs are executed can have a significant impact on their overall completion time (makespan). Our goal is to design a job scheduler that minimizes the makespan. In this study, a new formalization is introduced to present each job as a pair of disk processing and network transmitting two-stage durations. Due to the streaming processing feature, the two-stage operations are executed in an overlap manner and may lead to both one-stage and two-stage scheduling situations. A novel heuristic scheduling strategy is proposed for this hybrid scheduling problem, and the performance of the method is confirmed by the experimental evaluation.

    AB - Data intensive computing (DIC) offers an attractive option for business to remotely execute applications and load the computing resources from cloud in a streaming way. A key challenge in such environment is to increase the utilization of cloud cluster for the high throughput processing. One way of achieving this goal is to optimize the execution of computing jobs on the cluster. We observe that the order in which these jobs are executed can have a significant impact on their overall completion time (makespan). Our goal is to design a job scheduler that minimizes the makespan. In this study, a new formalization is introduced to present each job as a pair of disk processing and network transmitting two-stage durations. Due to the streaming processing feature, the two-stage operations are executed in an overlap manner and may lead to both one-stage and two-stage scheduling situations. A novel heuristic scheduling strategy is proposed for this hybrid scheduling problem, and the performance of the method is confirmed by the experimental evaluation.

    KW - Data Intensive Computing

    KW - Makespan

    KW - NP

    KW - Scheduling

    UR - http://www.scopus.com/inward/record.url?scp=84973889235&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=84973889235&partnerID=8YFLogxK

    U2 - 10.1109/SmartCity.2015.190

    DO - 10.1109/SmartCity.2015.190

    M3 - Conference contribution

    AN - SCOPUS:84973889235

    SP - 932

    EP - 937

    BT - Proceedings - 2015 IEEE International Conference on Smart City, SmartCity 2015, Held Jointly with 8th IEEE International Conference on Social Computing and Networking, SocialCom 2015, 5th IEEE International Conference on Sustainable Computing and Communications, SustainCom 2015, 2015 International Conference on Big Data Intelligence and Computing, DataCom 2015, 5th International Symposium on Cloud and Service Computing, SC2 2015

    PB - Institute of Electrical and Electronics Engineers Inc.

    ER -