Intersensory causality modeling using deep neural networks

Kuniaki Noda, Hiroaki Arie, Yuki Suga, Tetsuya Ogata

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

Our brain is known to enhance perceptual precision and reduce ambiguity about sensory environment by integrating multiple sources of sensory information acquired from different modalities, such as vision, auditory and somatic sensation. From an engineering perspective, building a computational model that replicates this ability to integrate multimodal information and to self-organize the causal dependency among them, represents one of the central challenges in robotics. In this study, we propose such a model based on a deep learning framework and we evaluate the proposed model by conducting a bell ring task using a small humanoid robot. Our experimental results demonstrate that (1) the cross-modal memory retrieval function of the proposed method succeeds in generating visual sequence from the corresponding sound and bell ring motion, and (2) the proposed method leads to accurate causal dependencies among the sensory-motor sequence.

Original languageEnglish
Title of host publicationProceedings - 2013 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2013
Pages1995-2000
Number of pages6
DOIs
Publication statusPublished - 2013 Dec 1
Event2013 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2013 - Manchester, United Kingdom
Duration: 2013 Oct 132013 Oct 16

Publication series

NameProceedings - 2013 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2013

Other

Other2013 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2013
CountryUnited Kingdom
CityManchester
Period13/10/1313/10/16

Keywords

  • Deep learning
  • Multimodal integration
  • Robotics
  • Temporal sequence learning

ASJC Scopus subject areas

  • Human-Computer Interaction

Fingerprint Dive into the research topics of 'Intersensory causality modeling using deep neural networks'. Together they form a unique fingerprint.

Cite this