Hetero complementary networks with hard-wired condensing binarization for high frame rate and ultra-low delay dual-hand tracking

Peiqi Zhang, Dingli Luo, Songlin Du, Takeshi Ikenaga

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

High frame rate, ultra-low delay yet accurate hand tracking system provides a seamless and intuitive interface for Human Computer Interaction (HCI). Tracking multi-person's dual-hand from monocular RGB camera is challenging for hand's variant image feature. Although many CNN based trackers have been proposed on general hardware, they cannot address this challenge with ultra-high speed. This paper proposes: (A) Hetero complementary networks for ultra-high speed dual-hand tracking, where the quick primary result from an FPGA network is intermittently combined with delayed accurate result from a GPU network. (B) Hard-wired condensing binarization for ultrahigh speed network implementation on FPGA. The network is able to be directly mapped as hardware resource because complex computation is condensed into binary layers. The proposed method achieves 69.8% accuracy on test sequences, which is only 4.7% lower compared with the general method. Meanwhile, the estimated FPGA resource utilization is tremendously reduced to 54.7% on the target platform. This work shows the potential to track multi-person's dual-hand at millisecond-level speed.

Original languageEnglish
Title of host publicationProceedings - 2020 13th International Conference on Human System Interaction, HSI 2020
PublisherIEEE Computer Society
Pages82-87
Number of pages6
ISBN (Electronic)9781728173924
DOIs
Publication statusPublished - 2020 Jun
Event13th International Conference on Human System Interaction, HSI 2020 - Tokyo, Japan
Duration: 2020 Jun 62020 Jun 8

Publication series

NameInternational Conference on Human System Interaction, HSI
Volume2020-June
ISSN (Print)2158-2246
ISSN (Electronic)2158-2254

Conference

Conference13th International Conference on Human System Interaction, HSI 2020
CountryJapan
CityTokyo
Period20/6/620/6/8

Keywords

  • Binary neural network
  • Hetero architecture
  • High frame rate ultra-low delay
  • Multi-person dual-hand tracking

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Software

Fingerprint Dive into the research topics of 'Hetero complementary networks with hard-wired condensing binarization for high frame rate and ultra-low delay dual-hand tracking'. Together they form a unique fingerprint.

Cite this