Dynamic scheduling for speculative execution to improve MapReduce performance in heterogeneous environment

Hyungjae Jung, Hidenori Nakazato

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    4 Citations (Scopus)

    Abstract

    MapReduce framework allows users to quickly develop big-data applications and process big-data effectively. However, unexpected malfunction may be found in cloud environment because a distributed system consists of several hardware, and this malfunction often causes delay of overall processing. MapReduce framework provides Speculative Execution (SE). SE reduces delay in a homogeneous environment by assigning delayed tasks to additional nodes. As cloud computing prevails, cloud computing environment is moving from homogeneous to heterogeneous. Original SE is not perfect and sometimes produces inefficient result in a heterogeneous environment. This paper proposes Dynamic Scheduling for Speculative Execution (DSSE) which enhances performance in a heterogeneous environment by improving existing SE. DSSE prevents wasted SE since it calculates processing capability of each node more objectively and precisely. DSSE has reduced entire processing time approximately 10% compared to original SE. Success rate of SE was 100%.

    Original languageEnglish
    Title of host publicationProceedings 2014 IEEE 34th International Conference on Distributed Computing Systems Workshops, ICDCSW 2014
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    Pages119-124
    Number of pages6
    Volume30-June-2014
    ISBN (Electronic)9781479941810
    DOIs
    Publication statusPublished - 2014 Aug 29
    Event2014 IEEE 34th International Conference on Distributed Computing Systems Workshops, ICDCSW 2014 - Madrid, Spain
    Duration: 2014 Jun 302014 Jul 3

    Other

    Other2014 IEEE 34th International Conference on Distributed Computing Systems Workshops, ICDCSW 2014
    CountrySpain
    CityMadrid
    Period14/6/3014/7/3

    Fingerprint

    Scheduling
    Cloud computing
    Processing
    Hardware
    Big data

    Keywords

    • Cloud Computing
    • DSSE
    • Heterogeneous environment
    • MapReduce
    • Speculative Execution

    ASJC Scopus subject areas

    • Computer Networks and Communications
    • Hardware and Architecture
    • Software

    Cite this

    Jung, H., & Nakazato, H. (2014). Dynamic scheduling for speculative execution to improve MapReduce performance in heterogeneous environment. In Proceedings 2014 IEEE 34th International Conference on Distributed Computing Systems Workshops, ICDCSW 2014 (Vol. 30-June-2014, pp. 119-124). [6888850] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICDCSW.2014.23

    Dynamic scheduling for speculative execution to improve MapReduce performance in heterogeneous environment. / Jung, Hyungjae; Nakazato, Hidenori.

    Proceedings 2014 IEEE 34th International Conference on Distributed Computing Systems Workshops, ICDCSW 2014. Vol. 30-June-2014 Institute of Electrical and Electronics Engineers Inc., 2014. p. 119-124 6888850.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Jung, H & Nakazato, H 2014, Dynamic scheduling for speculative execution to improve MapReduce performance in heterogeneous environment. in Proceedings 2014 IEEE 34th International Conference on Distributed Computing Systems Workshops, ICDCSW 2014. vol. 30-June-2014, 6888850, Institute of Electrical and Electronics Engineers Inc., pp. 119-124, 2014 IEEE 34th International Conference on Distributed Computing Systems Workshops, ICDCSW 2014, Madrid, Spain, 14/6/30. https://doi.org/10.1109/ICDCSW.2014.23
    Jung H, Nakazato H. Dynamic scheduling for speculative execution to improve MapReduce performance in heterogeneous environment. In Proceedings 2014 IEEE 34th International Conference on Distributed Computing Systems Workshops, ICDCSW 2014. Vol. 30-June-2014. Institute of Electrical and Electronics Engineers Inc. 2014. p. 119-124. 6888850 https://doi.org/10.1109/ICDCSW.2014.23
    Jung, Hyungjae ; Nakazato, Hidenori. / Dynamic scheduling for speculative execution to improve MapReduce performance in heterogeneous environment. Proceedings 2014 IEEE 34th International Conference on Distributed Computing Systems Workshops, ICDCSW 2014. Vol. 30-June-2014 Institute of Electrical and Electronics Engineers Inc., 2014. pp. 119-124
    @inproceedings{10d8f44f97a94d5fb2c56b9ea1185615,
    title = "Dynamic scheduling for speculative execution to improve MapReduce performance in heterogeneous environment",
    abstract = "MapReduce framework allows users to quickly develop big-data applications and process big-data effectively. However, unexpected malfunction may be found in cloud environment because a distributed system consists of several hardware, and this malfunction often causes delay of overall processing. MapReduce framework provides Speculative Execution (SE). SE reduces delay in a homogeneous environment by assigning delayed tasks to additional nodes. As cloud computing prevails, cloud computing environment is moving from homogeneous to heterogeneous. Original SE is not perfect and sometimes produces inefficient result in a heterogeneous environment. This paper proposes Dynamic Scheduling for Speculative Execution (DSSE) which enhances performance in a heterogeneous environment by improving existing SE. DSSE prevents wasted SE since it calculates processing capability of each node more objectively and precisely. DSSE has reduced entire processing time approximately 10{\%} compared to original SE. Success rate of SE was 100{\%}.",
    keywords = "Cloud Computing, DSSE, Heterogeneous environment, MapReduce, Speculative Execution",
    author = "Hyungjae Jung and Hidenori Nakazato",
    year = "2014",
    month = "8",
    day = "29",
    doi = "10.1109/ICDCSW.2014.23",
    language = "English",
    volume = "30-June-2014",
    pages = "119--124",
    booktitle = "Proceedings 2014 IEEE 34th International Conference on Distributed Computing Systems Workshops, ICDCSW 2014",
    publisher = "Institute of Electrical and Electronics Engineers Inc.",
    address = "United States",

    }

    TY - GEN

    T1 - Dynamic scheduling for speculative execution to improve MapReduce performance in heterogeneous environment

    AU - Jung, Hyungjae

    AU - Nakazato, Hidenori

    PY - 2014/8/29

    Y1 - 2014/8/29

    N2 - MapReduce framework allows users to quickly develop big-data applications and process big-data effectively. However, unexpected malfunction may be found in cloud environment because a distributed system consists of several hardware, and this malfunction often causes delay of overall processing. MapReduce framework provides Speculative Execution (SE). SE reduces delay in a homogeneous environment by assigning delayed tasks to additional nodes. As cloud computing prevails, cloud computing environment is moving from homogeneous to heterogeneous. Original SE is not perfect and sometimes produces inefficient result in a heterogeneous environment. This paper proposes Dynamic Scheduling for Speculative Execution (DSSE) which enhances performance in a heterogeneous environment by improving existing SE. DSSE prevents wasted SE since it calculates processing capability of each node more objectively and precisely. DSSE has reduced entire processing time approximately 10% compared to original SE. Success rate of SE was 100%.

    AB - MapReduce framework allows users to quickly develop big-data applications and process big-data effectively. However, unexpected malfunction may be found in cloud environment because a distributed system consists of several hardware, and this malfunction often causes delay of overall processing. MapReduce framework provides Speculative Execution (SE). SE reduces delay in a homogeneous environment by assigning delayed tasks to additional nodes. As cloud computing prevails, cloud computing environment is moving from homogeneous to heterogeneous. Original SE is not perfect and sometimes produces inefficient result in a heterogeneous environment. This paper proposes Dynamic Scheduling for Speculative Execution (DSSE) which enhances performance in a heterogeneous environment by improving existing SE. DSSE prevents wasted SE since it calculates processing capability of each node more objectively and precisely. DSSE has reduced entire processing time approximately 10% compared to original SE. Success rate of SE was 100%.

    KW - Cloud Computing

    KW - DSSE

    KW - Heterogeneous environment

    KW - MapReduce

    KW - Speculative Execution

    UR - http://www.scopus.com/inward/record.url?scp=84988329621&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=84988329621&partnerID=8YFLogxK

    U2 - 10.1109/ICDCSW.2014.23

    DO - 10.1109/ICDCSW.2014.23

    M3 - Conference contribution

    VL - 30-June-2014

    SP - 119

    EP - 124

    BT - Proceedings 2014 IEEE 34th International Conference on Distributed Computing Systems Workshops, ICDCSW 2014

    PB - Institute of Electrical and Electronics Engineers Inc.

    ER -