JointFlow: A Foot Motion Tracking Model Combining Pose Estimation Model with Optical Flow

Hiroaki Hayashi, Hirofumi Aoki, Mitsuhiro Kamezaki, Kan Shimazaki, Kunitomo Aoki, Shigeki Sugano

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Understanding driver's foot behavior helps reduce pedal-related traffic accidents. Conventional systems introduced wearable devices to track drivers' foot motion and tracking drivers' feet using non-contact sensors is still challenging. Moreover, to better understand driver's foot behavior, it is required to stably track drivers' small foot motion and at the joint level. To cope with this problem, we develop a novel foot motion tracking system, referred to as 'JointFlow.' This system integrates a pose estimation model, i.e., OpenPose, with Optical Flow. At first, a keypoint region-based convolutional neural network (keypoint R-CNN) is trained to detect the joints of the foot. At the same time, the Lucas-Kanade algorithm of Optical Flow is used to calculate the motion of each foot joint between consecutive frames. We implemented and evaluated the system using a real-world driving dataset of 50 drivers. The evaluation result shows that JointFlow could track both small and large foot motion. By comparing with the conventional pose estimation model, we could confirm that JointFlow tracked small foot motion more stable.

Original languageEnglish
Title of host publication2022 IEEE 25th International Conference on Intelligent Transportation Systems, ITSC 2022
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages2875-2881
Number of pages7
ISBN (Electronic)9781665468800
DOIs
Publication statusPublished - 2022
Event25th IEEE International Conference on Intelligent Transportation Systems, ITSC 2022 - Macau, China
Duration: 2022 Oct 82022 Oct 12

Publication series

NameIEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC
Volume2022-October

Conference

Conference25th IEEE International Conference on Intelligent Transportation Systems, ITSC 2022
Country/TerritoryChina
CityMacau
Period22/10/822/10/12

ASJC Scopus subject areas

  • Automotive Engineering
  • Mechanical Engineering
  • Computer Science Applications

Fingerprint

Dive into the research topics of 'JointFlow: A Foot Motion Tracking Model Combining Pose Estimation Model with Optical Flow'. Together they form a unique fingerprint.

Cite this