Multimodal integration learning of object manipulation behaviors using deep neural networks

Kuniaki Noda, Hiroaki Arie, Yuki Suga, Testuya Ogata

Research output: Chapter in Book/Report/Conference proceedingConference contribution

10 Citations (Scopus)

Abstract

This paper presents a novel computational approach for modeling and generating multiple object manipulation behaviors by a humanoid robot. The contribution of this paper is that deep learning methods are applied not only for multimodal sensor fusion but also for sensory-motor coordination. More specifically, a time-delay deep neural network is applied for modeling multiple behavior patterns represented with multi-dimensional visuomotor temporal sequences. By using the efficient training performance of Hessian-free optimization, the proposed mechanism successfully models six different object manipulation behaviors in a single network. The generalization capability of the learning mechanism enables the acquired model to perform the functions of cross-modal memory retrieval and temporal sequence prediction. The experimental results show that the motion patterns for object manipulation behaviors are successfully generated from the corresponding image sequence, and vice versa. Moreover, the temporal sequence prediction enables the robot to interactively switch multiple behaviors in accordance with changes in the displayed objects.

Original languageEnglish
Title of host publicationIROS 2013
Subtitle of host publicationNew Horizon, Conference Digest - 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems
Pages1728-1733
Number of pages6
DOIs
Publication statusPublished - 2013 Dec 1
Event2013 26th IEEE/RSJ International Conference on Intelligent Robots and Systems: New Horizon, IROS 2013 - Tokyo, Japan
Duration: 2013 Nov 32013 Nov 8

Publication series

NameIEEE International Conference on Intelligent Robots and Systems
ISSN (Print)2153-0858
ISSN (Electronic)2153-0866

Other

Other2013 26th IEEE/RSJ International Conference on Intelligent Robots and Systems: New Horizon, IROS 2013
CountryJapan
CityTokyo
Period13/11/313/11/8

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Software
  • Computer Vision and Pattern Recognition
  • Computer Science Applications

Fingerprint Dive into the research topics of 'Multimodal integration learning of object manipulation behaviors using deep neural networks'. Together they form a unique fingerprint.

  • Cite this

    Noda, K., Arie, H., Suga, Y., & Ogata, T. (2013). Multimodal integration learning of object manipulation behaviors using deep neural networks. In IROS 2013: New Horizon, Conference Digest - 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 1728-1733). [6696582] (IEEE International Conference on Intelligent Robots and Systems). https://doi.org/10.1109/IROS.2013.6696582