Maximum Data-resolution Efficiency for Fog-Computing Supported Spatial Big Data Processing in Disaster Scenarios

Junbo Wang, Michael Conrad Meyer, Yilang Wu, Yu Wang

Research output: Contribution to journalArticle

Abstract

Spatial big data analysis is very important in disaster scenarios to understand distribution patterns of situations, e.g., people's movements, people's requirements, resource shortage situations, and so on. In a general case, spatial big data is generated from distributed sensing devices and analyzed in a centralized way, e.g. a cloud center with high-performance computing resources. However, data transmission from sensing devices to cloud centers always takes a long time, especially in disaster scenarios with an unstable network. Fog computing is a promising technique to solve the above problem by offloading data processing tasks from the cloud to nearby computation devices. But data resolution also decreases after local processing in the fog nodes. It is necessary to investigate the optimal task distribution solutions to efficiently use computation resources in the fog layer. In this paper, we take the above research problem, and study fog-computing supported spatial big data processing. We analyze the process for spatial clustering, which is a typical category for spatial data analysis, and propose an architecture to integrate data processing into fog computing. We formalize a problem to maximize the data-resolution efficiency by considering data resolution and delay. We further propose core algorithms to enable spatial clustering in a fog-computing environment and implement the above algorithms in a real system. We have performed both simulations and experiments on a real Twitter dataset collected when Kumamoto-city suffered an earthquake. Through the simulations and the experiments, we have determined that the proposed solution significantly outperforms the other solutions.

Original languageEnglish
JournalIEEE Transactions on Parallel and Distributed Systems
DOIs
Publication statusAccepted/In press - 2019 Jan 1
Externally publishedYes

Fingerprint

Fog
Disasters
Data communication systems
Earthquakes
Experiments
Big data
Processing

Keywords

  • Big Data
  • Clustering algorithms
  • Data Resolution
  • Disaster
  • Distributed databases
  • Edge computing
  • Fog Computing
  • Spatial Big Data Analytics
  • Spatial Clustering
  • Spatial databases
  • Spatial resolution

ASJC Scopus subject areas

  • Signal Processing
  • Hardware and Architecture
  • Computational Theory and Mathematics

Cite this

@article{d4538ed6cb9b4f64834673b249da5568,
title = "Maximum Data-resolution Efficiency for Fog-Computing Supported Spatial Big Data Processing in Disaster Scenarios",
abstract = "Spatial big data analysis is very important in disaster scenarios to understand distribution patterns of situations, e.g., people's movements, people's requirements, resource shortage situations, and so on. In a general case, spatial big data is generated from distributed sensing devices and analyzed in a centralized way, e.g. a cloud center with high-performance computing resources. However, data transmission from sensing devices to cloud centers always takes a long time, especially in disaster scenarios with an unstable network. Fog computing is a promising technique to solve the above problem by offloading data processing tasks from the cloud to nearby computation devices. But data resolution also decreases after local processing in the fog nodes. It is necessary to investigate the optimal task distribution solutions to efficiently use computation resources in the fog layer. In this paper, we take the above research problem, and study fog-computing supported spatial big data processing. We analyze the process for spatial clustering, which is a typical category for spatial data analysis, and propose an architecture to integrate data processing into fog computing. We formalize a problem to maximize the data-resolution efficiency by considering data resolution and delay. We further propose core algorithms to enable spatial clustering in a fog-computing environment and implement the above algorithms in a real system. We have performed both simulations and experiments on a real Twitter dataset collected when Kumamoto-city suffered an earthquake. Through the simulations and the experiments, we have determined that the proposed solution significantly outperforms the other solutions.",
keywords = "Big Data, Clustering algorithms, Data Resolution, Disaster, Distributed databases, Edge computing, Fog Computing, Spatial Big Data Analytics, Spatial Clustering, Spatial databases, Spatial resolution",
author = "Junbo Wang and Meyer, {Michael Conrad} and Yilang Wu and Yu Wang",
year = "2019",
month = "1",
day = "1",
doi = "10.1109/TPDS.2019.2896143",
language = "English",
journal = "IEEE Transactions on Parallel and Distributed Systems",
issn = "1045-9219",
publisher = "IEEE Computer Society",

}

TY - JOUR

T1 - Maximum Data-resolution Efficiency for Fog-Computing Supported Spatial Big Data Processing in Disaster Scenarios

AU - Wang, Junbo

AU - Meyer, Michael Conrad

AU - Wu, Yilang

AU - Wang, Yu

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Spatial big data analysis is very important in disaster scenarios to understand distribution patterns of situations, e.g., people's movements, people's requirements, resource shortage situations, and so on. In a general case, spatial big data is generated from distributed sensing devices and analyzed in a centralized way, e.g. a cloud center with high-performance computing resources. However, data transmission from sensing devices to cloud centers always takes a long time, especially in disaster scenarios with an unstable network. Fog computing is a promising technique to solve the above problem by offloading data processing tasks from the cloud to nearby computation devices. But data resolution also decreases after local processing in the fog nodes. It is necessary to investigate the optimal task distribution solutions to efficiently use computation resources in the fog layer. In this paper, we take the above research problem, and study fog-computing supported spatial big data processing. We analyze the process for spatial clustering, which is a typical category for spatial data analysis, and propose an architecture to integrate data processing into fog computing. We formalize a problem to maximize the data-resolution efficiency by considering data resolution and delay. We further propose core algorithms to enable spatial clustering in a fog-computing environment and implement the above algorithms in a real system. We have performed both simulations and experiments on a real Twitter dataset collected when Kumamoto-city suffered an earthquake. Through the simulations and the experiments, we have determined that the proposed solution significantly outperforms the other solutions.

AB - Spatial big data analysis is very important in disaster scenarios to understand distribution patterns of situations, e.g., people's movements, people's requirements, resource shortage situations, and so on. In a general case, spatial big data is generated from distributed sensing devices and analyzed in a centralized way, e.g. a cloud center with high-performance computing resources. However, data transmission from sensing devices to cloud centers always takes a long time, especially in disaster scenarios with an unstable network. Fog computing is a promising technique to solve the above problem by offloading data processing tasks from the cloud to nearby computation devices. But data resolution also decreases after local processing in the fog nodes. It is necessary to investigate the optimal task distribution solutions to efficiently use computation resources in the fog layer. In this paper, we take the above research problem, and study fog-computing supported spatial big data processing. We analyze the process for spatial clustering, which is a typical category for spatial data analysis, and propose an architecture to integrate data processing into fog computing. We formalize a problem to maximize the data-resolution efficiency by considering data resolution and delay. We further propose core algorithms to enable spatial clustering in a fog-computing environment and implement the above algorithms in a real system. We have performed both simulations and experiments on a real Twitter dataset collected when Kumamoto-city suffered an earthquake. Through the simulations and the experiments, we have determined that the proposed solution significantly outperforms the other solutions.

KW - Big Data

KW - Clustering algorithms

KW - Data Resolution

KW - Disaster

KW - Distributed databases

KW - Edge computing

KW - Fog Computing

KW - Spatial Big Data Analytics

KW - Spatial Clustering

KW - Spatial databases

KW - Spatial resolution

UR - http://www.scopus.com/inward/record.url?scp=85061041675&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85061041675&partnerID=8YFLogxK

U2 - 10.1109/TPDS.2019.2896143

DO - 10.1109/TPDS.2019.2896143

M3 - Article

JO - IEEE Transactions on Parallel and Distributed Systems

JF - IEEE Transactions on Parallel and Distributed Systems

SN - 1045-9219

ER -