Dynamic hand gesture recognition for robot arm teaching based on improved LRCN model

Kaixiang Luan, Takafumi Matsumaru

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this research, we focus on finding a new method of human-robot interaction in industrial environment. A vision-based dynamic hand gestures recognition system has been proposed for robot arm picking task. 8 dynamic hand gestures are captured for this task with a 100fps high speed camera. Based on the LRCN model, we combine the MobileNets (V2) and LSTM for this task, the MobileNets (V2) for extracting the image features and recognize the gestures, then, Long Short-Term Memory (LSTM) architecture for interpreting the features across time steps. Around 100 samples are taken for each gesture for training at first, then, the samples are augmented to 200 samples per gesture by data augmentation. Result shows that the model is able to learn the gestures varying in duration and complexity and gestures can be recognized in 88ms with 90.62% accuracy in the experiment on our hand gesture dataset.

Original languageEnglish
Title of host publicationIEEE International Conference on Robotics and Biomimetics, ROBIO 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1269-1274
Number of pages6
ISBN (Electronic)9781728163215
DOIs
Publication statusPublished - 2019 Dec
Event2019 IEEE International Conference on Robotics and Biomimetics, ROBIO 2019 - Dali, China
Duration: 2019 Dec 62019 Dec 8

Publication series

NameIEEE International Conference on Robotics and Biomimetics, ROBIO 2019

Conference

Conference2019 IEEE International Conference on Robotics and Biomimetics, ROBIO 2019
CountryChina
CityDali
Period19/12/619/12/8

Keywords

  • Deep Learning
  • Gesture recognition
  • LSTM
  • Robot teaching
  • Robotics picking

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Science Applications
  • Hardware and Architecture
  • Mechanical Engineering
  • Control and Optimization

Fingerprint Dive into the research topics of 'Dynamic hand gesture recognition for robot arm teaching based on improved LRCN model'. Together they form a unique fingerprint.

  • Cite this

    Luan, K., & Matsumaru, T. (2019). Dynamic hand gesture recognition for robot arm teaching based on improved LRCN model. In IEEE International Conference on Robotics and Biomimetics, ROBIO 2019 (pp. 1269-1274). [8961787] (IEEE International Conference on Robotics and Biomimetics, ROBIO 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ROBIO49542.2019.8961787