Image information assistance neural network for videopose3d-based monocular 3D pose estimation

Hao Wang*, Dingli Luo, Takeshi Ikenaga

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

3D pose estimation based on a monocular camera can be applied to various fields such as human-computer interaction and human action recognition. As a two-stage 3D pose estimator, VideoPose3D achieves state-of-the-art accuracy. However, because of the limitation of two-stage processing, image information is partially lost in the process of mapping 2D poses to 3D space, which results in limited final accuracy. This paper proposes an image-assisting pose estimation model and a back-projection based offset generating module. The image-assisting pose estimation model consists of a 2D pose processing branch and an image processing branch. Image information is processed to generate an offset to refine the intermediate 3D pose produced by the 2D pose processing network. The back-projection based offset generating module projects the intermediate 3D poses to 2D space and calculates the error between the projection and input 2D pose. With the error combining with extracted image feature, the neural network generates an offset to decrease the error. By evaluation, the accuracy on each action of Human3.6M dataset gets an average improvement of 0.9 mm over the VideoPose3D baseline.

Original languageEnglish
Title of host publicationProceedings of MVA 2021 - 17th International Conference on Machine Vision Applications
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9784901122207
DOIs
Publication statusPublished - 2021 Jul 25
Event17th International Conference on Machine Vision Applications, MVA 2021 - Aichi, Japan
Duration: 2021 Jul 252021 Jul 27

Publication series

NameProceedings of MVA 2021 - 17th International Conference on Machine Vision Applications

Conference

Conference17th International Conference on Machine Vision Applications, MVA 2021
Country/TerritoryJapan
CityAichi
Period21/7/2521/7/27

ASJC Scopus subject areas

  • Computer Science Applications
  • Computer Vision and Pattern Recognition
  • Signal Processing

Fingerprint

Dive into the research topics of 'Image information assistance neural network for videopose3d-based monocular 3D pose estimation'. Together they form a unique fingerprint.

Cite this