Intelligent Small Object Detection for Digital Twin in Smart Manufacturing with Industrial Cyber-Physical Systems

Xiaokang Zhou, Xuesong Xu*, Wei Liang, Zhi Zeng, Shohei Shimizu, Laurence T. Yang, Qun Jin

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

28 Citations (Scopus)

Abstract

Recently, along with several technological advancements in cyber-physical systems, the revolution of Industry 4.0 has brought in an emerging concept named digital twin (DT), which shows its potential to break the barrier between the physical and cyber space in smart manufacturing. However, it is still difficult to analyze and estimate the real-time structural and environmental parameters in terms of their dynamic changes in digital twinning, especially when facing detection tasks of multiple small objects from a large-scale scene with complex contexts in modern manufacturing environments. In this article, we focus on a small object detection model for DT, aiming to realize the dynamic synchronization between a physical manufacturing system and its virtual representation. Three significant elements, including equipment, product, and operator, are considered as the basic environmental parameters to represent and estimate the dynamic characteristics and real-time changes in building a generic DT system of smart manufacturing workshop. A hybrid deep neural network model, based on the integration of MobileNetv2, YOLOv4, and Openpose, is constructed to identify the real-time status from physical manufacturing environment to virtual space. A learning algorithm is then developed to realize the efficient multitype small object detection based on the feature integration and fusion from both shallow and deep layers, in order to facilitate the modeling, monitoring, and optimizing of the whole manufacturing process in the DT system. Experiments and evaluations conducted in three different use cases demonstrate the effectiveness and usefulness of our proposed method, which can achieve a higher detection accuracy for DT in smart manufacturing.

Original languageEnglish
Pages (from-to)1377-1386
Number of pages10
JournalIEEE Transactions on Industrial Informatics
Volume18
Issue number2
DOIs
Publication statusPublished - 2022 Feb 1

Keywords

  • Deep neural network
  • digital twin
  • industrial cyber-physical systems (CPS)
  • object detection
  • posture recognition

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Information Systems
  • Computer Science Applications
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Intelligent Small Object Detection for Digital Twin in Smart Manufacturing with Industrial Cyber-Physical Systems'. Together they form a unique fingerprint.

Cite this