Dynamic scheduling for speculative execution to improve MapReduce performance in heterogeneous environment

Hyungjae Jung, Hidenori Nakazato

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    4 Citations (Scopus)

    Abstract

    MapReduce framework allows users to quickly develop big-data applications and process big-data effectively. However, unexpected malfunction may be found in cloud environment because a distributed system consists of several hardware, and this malfunction often causes delay of overall processing. MapReduce framework provides Speculative Execution (SE). SE reduces delay in a homogeneous environment by assigning delayed tasks to additional nodes. As cloud computing prevails, cloud computing environment is moving from homogeneous to heterogeneous. Original SE is not perfect and sometimes produces inefficient result in a heterogeneous environment. This paper proposes Dynamic Scheduling for Speculative Execution (DSSE) which enhances performance in a heterogeneous environment by improving existing SE. DSSE prevents wasted SE since it calculates processing capability of each node more objectively and precisely. DSSE has reduced entire processing time approximately 10% compared to original SE. Success rate of SE was 100%.

    Original languageEnglish
    Title of host publicationProceedings 2014 IEEE 34th International Conference on Distributed Computing Systems Workshops, ICDCSW 2014
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    Pages119-124
    Number of pages6
    Volume30-June-2014
    ISBN (Electronic)9781479941810
    DOIs
    Publication statusPublished - 2014 Aug 29
    Event2014 IEEE 34th International Conference on Distributed Computing Systems Workshops, ICDCSW 2014 - Madrid, Spain
    Duration: 2014 Jun 302014 Jul 3

    Other

    Other2014 IEEE 34th International Conference on Distributed Computing Systems Workshops, ICDCSW 2014
    CountrySpain
    CityMadrid
    Period14/6/3014/7/3

    Keywords

    • Cloud Computing
    • DSSE
    • Heterogeneous environment
    • MapReduce
    • Speculative Execution

    ASJC Scopus subject areas

    • Computer Networks and Communications
    • Hardware and Architecture
    • Software

    Fingerprint Dive into the research topics of 'Dynamic scheduling for speculative execution to improve MapReduce performance in heterogeneous environment'. Together they form a unique fingerprint.

  • Cite this

    Jung, H., & Nakazato, H. (2014). Dynamic scheduling for speculative execution to improve MapReduce performance in heterogeneous environment. In Proceedings 2014 IEEE 34th International Conference on Distributed Computing Systems Workshops, ICDCSW 2014 (Vol. 30-June-2014, pp. 119-124). [6888850] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICDCSW.2014.23