MEC: Network optimized multi-stage erasure coding for scalable storage systems

Hiroaki Akutsu, Takahiro Yamamoto, Kazunori Ueda, Hideo Saito

    研究成果: Conference contribution

    抜粋

    In scalable storage systems, there are two kinds of methods for data redundancy: mirroring and parity. Each has its pros and cons. Mirroring creates a large amount of redundancy data, resulting in less usable space. Write performance degrades proportionally to the redundancy level due to an increase in communication. Parity-based methods partition data into multiple pieces, add parity information, and distribute the pieces of data and parity information. Parity-based methods are not often used with memory class media that are faster than the network, because distributing the data across servers results in low read performance. This research aims to establish an efficient data protection method that can be applied to fast, memory class media. We propose a new parity-based method called Multi-stage Erasure Coding (MEC), which creates two different erasure codes: one at the data transmission source server, and the other at the destination server. We show that our method reduces the space required to achieve redundancy while achieving high performance by making the amount of write communication independent of the redundancy level. We built a prototype program using MEC on a commodity cluster server. We show that compared with conventional parity-based methods with redundancy level 2, read I/O throughput is over one order of magnitude higher thanks to local reads and that write I/O throughput is almost the same due to network bottleneck.

    元の言語English
    ホスト出版物のタイトルProceedings - 2017 IEEE 22nd Pacific Rim International Symposium on Dependable Computing, PRDC 2017
    出版者IEEE Computer Society
    ページ292-300
    ページ数9
    ISBN(電子版)9781509056514
    DOI
    出版物ステータスPublished - 2017 5 5
    イベント22nd IEEE Pacific Rim International Symposium on Dependable Computing, PRDC 2017 - Christchurch, New Zealand
    継続期間: 2017 1 222017 1 25

    Other

    Other22nd IEEE Pacific Rim International Symposium on Dependable Computing, PRDC 2017
    New Zealand
    Christchurch
    期間17/1/2217/1/25

    ASJC Scopus subject areas

    • Computational Theory and Mathematics
    • Computer Science Applications
    • Hardware and Architecture
    • Software

    フィンガープリント MEC: Network optimized multi-stage erasure coding for scalable storage systems' の研究トピックを掘り下げます。これらはともに一意のフィンガープリントを構成します。

  • これを引用

    Akutsu, H., Yamamoto, T., Ueda, K., & Saito, H. (2017). MEC: Network optimized multi-stage erasure coding for scalable storage systems. : Proceedings - 2017 IEEE 22nd Pacific Rim International Symposium on Dependable Computing, PRDC 2017 (pp. 292-300). [7920634] IEEE Computer Society. https://doi.org/10.1109/PRDC.2017.54