Making big data intelligent storable at the edge: Storage resource intelligent orchestration

Fuli Qiao, Mianxiong Dong, Kaoru Ota, Siyi Liao, Jun Wu, Jianhua Li

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Network edge equipment has generated a large amount of fast- growing data, which has placed a heavy burden on the collaboration of heterogeneous networks. Due to the diversity of edge computing application scenarios, many new requirements are advocated for unified data storage management, such as latency and processing efficiency. Traditional centralized cloud storage can no longer meet the on- demand of edge computing in the case of a surge in data volume. Therefore, a unified storage architecture is required for the current improvements in computational offloading schemes and storage optimization algorithms. To solve these challenges and make data intelligent collaborative storable, this paper proposes a novel unified storage architecture for big data in the edge-cloud, which supports edge services in order to extend Hadoop at the edge. The functions of the edge nodes are proposed to synchronize the edge nodes of the same neighborhood and store data dynamically via Q- learning based on popularity, in order to mitigate network load pressure and improve the efficiency of edge services. An intelligent scheme that impacts the quality of service (QoS) through data marginal storage is proposed to improve the resource scheduling and to the distribution of storage space. Simulation results demonstrate the merits and efficiency of the proposed intelligent architecture is superior to the comparison schemes.

Original languageEnglish
Title of host publication2019 IEEE Global Communications Conference, GLOBECOM 2019 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728109626
DOIs
Publication statusPublished - 2019 Dec
Externally publishedYes
Event2019 IEEE Global Communications Conference, GLOBECOM 2019 - Waikoloa, United States
Duration: 2019 Dec 92019 Dec 13

Publication series

Name2019 IEEE Global Communications Conference, GLOBECOM 2019 - Proceedings

Conference

Conference2019 IEEE Global Communications Conference, GLOBECOM 2019
Country/TerritoryUnited States
CityWaikoloa
Period19/12/919/12/13

Keywords

  • Intelligent storage architecture
  • Machine learning
  • Mobile edge computing
  • Unified edgecloud

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Hardware and Architecture
  • Information Systems
  • Signal Processing
  • Information Systems and Management
  • Safety, Risk, Reliability and Quality
  • Media Technology
  • Health Informatics

Fingerprint

Dive into the research topics of 'Making big data intelligent storable at the edge: Storage resource intelligent orchestration'. Together they form a unique fingerprint.

Cite this