On the optimal number of computational resources in MapReduce

Htway Htway Hlaing, Hidehiro Kanemitsu, Tatsuo Nakajima, Hidenori Nakazato

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Big data computing in the cloud needs faster processing and better resource provisioning. MapReduce is the framework for computing large scale datasets in cloud environments. Optimization of resource requirement for each job to satisfy a specific objective in MapReduce is an open problem. Many factors, e.g., system side information and requirements of each client must be considered to estimate the appropriate amount of resources. This paper presents a mathematical model for the optimal number of map tasks in MapReduce resource provisioning. This model is to estimate the optimal number of the mappers based on the resource specification and the size of the dataset.

Original languageEnglish
Title of host publicationCloud Computing – CLOUD 2019 - 12th International Conference, Held as Part of the Services Conference Federation, SCF 2019, Proceedings
EditorsQingyang Wang, Liang-Jie Zhang, Dilma Da Silva
PublisherSpringer-Verlag
Pages240-252
Number of pages13
ISBN (Print)9783030235017
DOIs
Publication statusPublished - 2019 Jan 1
Event12th International Conference on Cloud Computing, CLOUD 2019 held as part of the Services Conference Federation, SCF 2019 - San Diego, United States
Duration: 2019 Jun 252019 Jun 30

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11513 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference12th International Conference on Cloud Computing, CLOUD 2019 held as part of the Services Conference Federation, SCF 2019
CountryUnited States
CitySan Diego
Period19/6/2519/6/30

    Fingerprint

Keywords

  • Big data
  • Cloud computing
  • Resource provisioning

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Hlaing, H. H., Kanemitsu, H., Nakajima, T., & Nakazato, H. (2019). On the optimal number of computational resources in MapReduce. In Q. Wang, L-J. Zhang, & D. Da Silva (Eds.), Cloud Computing – CLOUD 2019 - 12th International Conference, Held as Part of the Services Conference Federation, SCF 2019, Proceedings (pp. 240-252). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11513 LNCS). Springer-Verlag. https://doi.org/10.1007/978-3-030-23502-4_17