The challenge of eliminating storage bottlenecks in distributed systems

Takanori Ueda, Yu Hirate, Hayato Yamana

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

One of the most difficult problems in distributed systems is load-balancing. Even if we take care of load-balancing, heavily-loaded nodes often occur while there are still lightly-loaded nodes that have idle memory and idle CPU power. Our idea is to exploit this idle memory and idle CPU power to improve the storage performance of heavily-loaded nodes. Idle memory can be used for caching file data and idle CPU power can be used for extracting file access patterns from file access logs. File access patterns are valuable sources for optimizing a cache strategy. Our project goal is to improve the overall performance of distributed systems by improving storage access performance. This paper gives an overview of this project idea and reports the current status of the project. In addition, we show benchmark results from our prototype cache extension system, which is implemented in Linux Kernel 2.6. The DBT-3 (TPC-H) benchmark results show that our system can increase computer speed by a factor of 6.68.

Original languageEnglish
Title of host publicationProceedings - 1st International Workshop on Software Technologies for Future Dependable Distributed Systems, STFSSD 2009
Pages49-53
Number of pages5
DOIs
Publication statusPublished - 2009 Dec 1
Event1st International Workshop on Software Technologies for Future Dependable Distributed Systems, STFSSD 2009 - Tokyo, Japan
Duration: 2009 Mar 172009 Mar 18

Publication series

NameProceedings - 1st International Workshop on Software Technologies for Future Dependable Distributed Systems, STFSSD 2009

Conference

Conference1st International Workshop on Software Technologies for Future Dependable Distributed Systems, STFSSD 2009
CountryJapan
CityTokyo
Period09/3/1709/3/18

    Fingerprint

Keywords

  • Access pattern
  • Access pattern mining
  • Buffer caching
  • Distributed systems
  • Replacement algorithm

ASJC Scopus subject areas

  • Hardware and Architecture
  • Information Systems

Cite this

Ueda, T., Hirate, Y., & Yamana, H. (2009). The challenge of eliminating storage bottlenecks in distributed systems. In Proceedings - 1st International Workshop on Software Technologies for Future Dependable Distributed Systems, STFSSD 2009 (pp. 49-53). [4804571] (Proceedings - 1st International Workshop on Software Technologies for Future Dependable Distributed Systems, STFSSD 2009). https://doi.org/10.1109/STFSSD.2009.46