MCMalloc: A scalable memory allocator for multithreaded applications on a many-core shared-memory machine

Akira Umayabara, Hayato Yamana

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    1 Citation (Scopus)

    Abstract

    In the big data era, multithreaded processing on a many-core machine, whose core number is still increasing, has become essential to parallelize the execution of big data applications, besides distributed computing. In such a machine, malloc-intensive applications cannot scale due to lock contentions among threads, which becomes worse as the number of threads increases. To solve the problem, we propose a new method to reduce lock contentions by batch malloc, pseudo free, and fine-grained data-locking. Experimental result shows 4.72 times speed-up in comparison with JEmalloc which is the fastest memory allocator among previous ones.

    Original languageEnglish
    Title of host publicationProceedings - 2017 IEEE International Conference on Big Data, Big Data 2017
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    Pages4846-4848
    Number of pages3
    Volume2018-January
    ISBN (Electronic)9781538627143
    DOIs
    Publication statusPublished - 2018 Jan 12
    Event5th IEEE International Conference on Big Data, Big Data 2017 - Boston, United States
    Duration: 2017 Dec 112017 Dec 14

    Other

    Other5th IEEE International Conference on Big Data, Big Data 2017
    CountryUnited States
    CityBoston
    Period17/12/1117/12/14

    Fingerprint

    Many-core
    Shared Memory
    Contention
    Data storage equipment
    Thread
    Distributed computer systems
    Locking
    Distributed Computing
    Batch
    Speedup
    Processing
    Experimental Results
    Big data
    Distributed computing

    Keywords

    • many-core
    • Memory Allocator

    ASJC Scopus subject areas

    • Computer Networks and Communications
    • Hardware and Architecture
    • Information Systems
    • Information Systems and Management
    • Control and Optimization

    Cite this

    Umayabara, A., & Yamana, H. (2018). MCMalloc: A scalable memory allocator for multithreaded applications on a many-core shared-memory machine. In Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017 (Vol. 2018-January, pp. 4846-4848). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BigData.2017.8258563

    MCMalloc : A scalable memory allocator for multithreaded applications on a many-core shared-memory machine. / Umayabara, Akira; Yamana, Hayato.

    Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017. Vol. 2018-January Institute of Electrical and Electronics Engineers Inc., 2018. p. 4846-4848.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Umayabara, A & Yamana, H 2018, MCMalloc: A scalable memory allocator for multithreaded applications on a many-core shared-memory machine. in Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017. vol. 2018-January, Institute of Electrical and Electronics Engineers Inc., pp. 4846-4848, 5th IEEE International Conference on Big Data, Big Data 2017, Boston, United States, 17/12/11. https://doi.org/10.1109/BigData.2017.8258563
    Umayabara A, Yamana H. MCMalloc: A scalable memory allocator for multithreaded applications on a many-core shared-memory machine. In Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017. Vol. 2018-January. Institute of Electrical and Electronics Engineers Inc. 2018. p. 4846-4848 https://doi.org/10.1109/BigData.2017.8258563
    Umayabara, Akira ; Yamana, Hayato. / MCMalloc : A scalable memory allocator for multithreaded applications on a many-core shared-memory machine. Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017. Vol. 2018-January Institute of Electrical and Electronics Engineers Inc., 2018. pp. 4846-4848
    @inproceedings{97d56768f6d446c489d0d9fb46470a2c,
    title = "MCMalloc: A scalable memory allocator for multithreaded applications on a many-core shared-memory machine",
    abstract = "In the big data era, multithreaded processing on a many-core machine, whose core number is still increasing, has become essential to parallelize the execution of big data applications, besides distributed computing. In such a machine, malloc-intensive applications cannot scale due to lock contentions among threads, which becomes worse as the number of threads increases. To solve the problem, we propose a new method to reduce lock contentions by batch malloc, pseudo free, and fine-grained data-locking. Experimental result shows 4.72 times speed-up in comparison with JEmalloc which is the fastest memory allocator among previous ones.",
    keywords = "many-core, Memory Allocator",
    author = "Akira Umayabara and Hayato Yamana",
    year = "2018",
    month = "1",
    day = "12",
    doi = "10.1109/BigData.2017.8258563",
    language = "English",
    volume = "2018-January",
    pages = "4846--4848",
    booktitle = "Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017",
    publisher = "Institute of Electrical and Electronics Engineers Inc.",

    }

    TY - GEN

    T1 - MCMalloc

    T2 - A scalable memory allocator for multithreaded applications on a many-core shared-memory machine

    AU - Umayabara, Akira

    AU - Yamana, Hayato

    PY - 2018/1/12

    Y1 - 2018/1/12

    N2 - In the big data era, multithreaded processing on a many-core machine, whose core number is still increasing, has become essential to parallelize the execution of big data applications, besides distributed computing. In such a machine, malloc-intensive applications cannot scale due to lock contentions among threads, which becomes worse as the number of threads increases. To solve the problem, we propose a new method to reduce lock contentions by batch malloc, pseudo free, and fine-grained data-locking. Experimental result shows 4.72 times speed-up in comparison with JEmalloc which is the fastest memory allocator among previous ones.

    AB - In the big data era, multithreaded processing on a many-core machine, whose core number is still increasing, has become essential to parallelize the execution of big data applications, besides distributed computing. In such a machine, malloc-intensive applications cannot scale due to lock contentions among threads, which becomes worse as the number of threads increases. To solve the problem, we propose a new method to reduce lock contentions by batch malloc, pseudo free, and fine-grained data-locking. Experimental result shows 4.72 times speed-up in comparison with JEmalloc which is the fastest memory allocator among previous ones.

    KW - many-core

    KW - Memory Allocator

    UR - http://www.scopus.com/inward/record.url?scp=85047777895&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85047777895&partnerID=8YFLogxK

    U2 - 10.1109/BigData.2017.8258563

    DO - 10.1109/BigData.2017.8258563

    M3 - Conference contribution

    AN - SCOPUS:85047777895

    VL - 2018-January

    SP - 4846

    EP - 4848

    BT - Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017

    PB - Institute of Electrical and Electronics Engineers Inc.

    ER -