The challenge of eliminating storage bottlenecks in distributed systems

Takanori Ueda, Yu Hirate, Hayato Yamana

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    One of the most difficult problems in distributed systems is load-balancing. Even if we take care of load-balancing, heavily-loaded nodes often occur while there are still lightly-loaded nodes that have idle memory and idle CPU power. Our idea is to exploit this idle memory and idle CPU power to improve the storage performance of heavily-loaded nodes. Idle memory can be used for caching file data and idle CPU power can be used for extracting file access patterns from file access logs. File access patterns are valuable sources for optimizing a cache strategy. Our project goal is to improve the overall performance of distributed systems by improving storage access performance. This paper gives an overview of this project idea and reports the current status of the project. In addition, we show benchmark results from our prototype cache extension system, which is implemented in Linux Kernel 2.6. The DBT-3 (TPC-H) benchmark results show that our system can increase computer speed by a factor of 6.68.

    Original languageEnglish
    Title of host publicationProceedings - 1st International Workshop on Software Technologies for Future Dependable Distributed Systems, STFSSD 2009
    Pages49-53
    Number of pages5
    DOIs
    Publication statusPublished - 2009
    Event1st International Workshop on Software Technologies for Future Dependable Distributed Systems, STFSSD 2009 - Tokyo
    Duration: 2009 Mar 172009 Mar 18

    Other

    Other1st International Workshop on Software Technologies for Future Dependable Distributed Systems, STFSSD 2009
    CityTokyo
    Period09/3/1709/3/18

    Fingerprint

    Program processors
    Data storage equipment
    Resource allocation
    Linux

    Keywords

    • Access pattern
    • Access pattern mining
    • Buffer caching
    • Distributed systems
    • Replacement algorithm

    ASJC Scopus subject areas

    • Hardware and Architecture
    • Information Systems

    Cite this

    Ueda, T., Hirate, Y., & Yamana, H. (2009). The challenge of eliminating storage bottlenecks in distributed systems. In Proceedings - 1st International Workshop on Software Technologies for Future Dependable Distributed Systems, STFSSD 2009 (pp. 49-53). [4804571] https://doi.org/10.1109/STFSSD.2009.46

    The challenge of eliminating storage bottlenecks in distributed systems. / Ueda, Takanori; Hirate, Yu; Yamana, Hayato.

    Proceedings - 1st International Workshop on Software Technologies for Future Dependable Distributed Systems, STFSSD 2009. 2009. p. 49-53 4804571.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Ueda, T, Hirate, Y & Yamana, H 2009, The challenge of eliminating storage bottlenecks in distributed systems. in Proceedings - 1st International Workshop on Software Technologies for Future Dependable Distributed Systems, STFSSD 2009., 4804571, pp. 49-53, 1st International Workshop on Software Technologies for Future Dependable Distributed Systems, STFSSD 2009, Tokyo, 09/3/17. https://doi.org/10.1109/STFSSD.2009.46
    Ueda T, Hirate Y, Yamana H. The challenge of eliminating storage bottlenecks in distributed systems. In Proceedings - 1st International Workshop on Software Technologies for Future Dependable Distributed Systems, STFSSD 2009. 2009. p. 49-53. 4804571 https://doi.org/10.1109/STFSSD.2009.46
    Ueda, Takanori ; Hirate, Yu ; Yamana, Hayato. / The challenge of eliminating storage bottlenecks in distributed systems. Proceedings - 1st International Workshop on Software Technologies for Future Dependable Distributed Systems, STFSSD 2009. 2009. pp. 49-53
    @inproceedings{4210a71bec4648439756650fcfad9a8d,
    title = "The challenge of eliminating storage bottlenecks in distributed systems",
    abstract = "One of the most difficult problems in distributed systems is load-balancing. Even if we take care of load-balancing, heavily-loaded nodes often occur while there are still lightly-loaded nodes that have idle memory and idle CPU power. Our idea is to exploit this idle memory and idle CPU power to improve the storage performance of heavily-loaded nodes. Idle memory can be used for caching file data and idle CPU power can be used for extracting file access patterns from file access logs. File access patterns are valuable sources for optimizing a cache strategy. Our project goal is to improve the overall performance of distributed systems by improving storage access performance. This paper gives an overview of this project idea and reports the current status of the project. In addition, we show benchmark results from our prototype cache extension system, which is implemented in Linux Kernel 2.6. The DBT-3 (TPC-H) benchmark results show that our system can increase computer speed by a factor of 6.68.",
    keywords = "Access pattern, Access pattern mining, Buffer caching, Distributed systems, Replacement algorithm",
    author = "Takanori Ueda and Yu Hirate and Hayato Yamana",
    year = "2009",
    doi = "10.1109/STFSSD.2009.46",
    language = "English",
    isbn = "9780769535722",
    pages = "49--53",
    booktitle = "Proceedings - 1st International Workshop on Software Technologies for Future Dependable Distributed Systems, STFSSD 2009",

    }

    TY - GEN

    T1 - The challenge of eliminating storage bottlenecks in distributed systems

    AU - Ueda, Takanori

    AU - Hirate, Yu

    AU - Yamana, Hayato

    PY - 2009

    Y1 - 2009

    N2 - One of the most difficult problems in distributed systems is load-balancing. Even if we take care of load-balancing, heavily-loaded nodes often occur while there are still lightly-loaded nodes that have idle memory and idle CPU power. Our idea is to exploit this idle memory and idle CPU power to improve the storage performance of heavily-loaded nodes. Idle memory can be used for caching file data and idle CPU power can be used for extracting file access patterns from file access logs. File access patterns are valuable sources for optimizing a cache strategy. Our project goal is to improve the overall performance of distributed systems by improving storage access performance. This paper gives an overview of this project idea and reports the current status of the project. In addition, we show benchmark results from our prototype cache extension system, which is implemented in Linux Kernel 2.6. The DBT-3 (TPC-H) benchmark results show that our system can increase computer speed by a factor of 6.68.

    AB - One of the most difficult problems in distributed systems is load-balancing. Even if we take care of load-balancing, heavily-loaded nodes often occur while there are still lightly-loaded nodes that have idle memory and idle CPU power. Our idea is to exploit this idle memory and idle CPU power to improve the storage performance of heavily-loaded nodes. Idle memory can be used for caching file data and idle CPU power can be used for extracting file access patterns from file access logs. File access patterns are valuable sources for optimizing a cache strategy. Our project goal is to improve the overall performance of distributed systems by improving storage access performance. This paper gives an overview of this project idea and reports the current status of the project. In addition, we show benchmark results from our prototype cache extension system, which is implemented in Linux Kernel 2.6. The DBT-3 (TPC-H) benchmark results show that our system can increase computer speed by a factor of 6.68.

    KW - Access pattern

    KW - Access pattern mining

    KW - Buffer caching

    KW - Distributed systems

    KW - Replacement algorithm

    UR - http://www.scopus.com/inward/record.url?scp=84880422401&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=84880422401&partnerID=8YFLogxK

    U2 - 10.1109/STFSSD.2009.46

    DO - 10.1109/STFSSD.2009.46

    M3 - Conference contribution

    SN - 9780769535722

    SP - 49

    EP - 53

    BT - Proceedings - 1st International Workshop on Software Technologies for Future Dependable Distributed Systems, STFSSD 2009

    ER -