Multi-task and multi-level detection neural network based real-time 3D pose estimation

Dingli Luo, Songlin Du, Takeshi Ikenaga

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

3D pose estimation is a core step for human-computer interaction and human action recognition. However, time-sensitive applications like virtual reality also need this task to achieve real-time speed. This paper proposes a multitask and multi-level neural network architecture with a highspeed friendly 3D human pose representation. Based on this, we build a real-time multi-person 3D pose estimation system with a single RGB image as input. The network estimates 3D poses from the input image directly by the multi-task design and keeps both accuracy and speed by the multi-level detection design. By evaluation, we show our system achieves the 21 fps on RTX 2080 with only 33 mm accuracy lose compared with related works. We also provide network visualization to prove our network work as we design. This work shows the possibility for a single RGB image based 3D pose estimation system to achieve real-time speed, which is a basement for building a low-cost 3D motion capture system.

Original languageEnglish
Title of host publication2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1427-1434
Number of pages8
ISBN (Electronic)9781728132488
DOIs
Publication statusPublished - 2019 Nov
Event2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2019 - Lanzhou, China
Duration: 2019 Nov 182019 Nov 21

Publication series

Name2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2019

Conference

Conference2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2019
CountryChina
CityLanzhou
Period19/11/1819/11/21

ASJC Scopus subject areas

  • Information Systems

Fingerprint Dive into the research topics of 'Multi-task and multi-level detection neural network based real-time 3D pose estimation'. Together they form a unique fingerprint.

Cite this