Inter-modality mapping in robot with recurrent neural network

Tetsuya Ogata, Shun Nishide, Hideki Kozima, Kazunori Komatani, Hiroshi G. Okuno

Research output: Contribution to journalArticle

17 Citations (Scopus)

Abstract

A system for mapping between different sensory modalities was developed for a robot system to enable it to generate motions expressing auditory signals and sounds generated by object movement. A recurrent neural network model with parametric bias, which has good generalization ability, is used as a learning model. Since the correspondences between auditory signals and visual signals are too numerous to memorize, the ability to generalize is indispensable. This system was implemented in the "Keepon" robot, and the robot was shown horizontal reciprocating or rotating motions with the sound of friction and falling or overturning motion with the sound of collision by manipulating a box object. Keepon behaved appropriately not only from learned events but also from unknown events and generated various sounds in accordance with observed motions.

Original languageEnglish
Pages (from-to)1560-1569
Number of pages10
JournalPattern Recognition Letters
Volume31
Issue number12
DOIs
Publication statusPublished - 2010 Sep 1
Externally publishedYes

Fingerprint

Recurrent neural networks
Acoustic waves
Robots
Friction

Keywords

  • Dynamical systems
  • Generalization
  • Inter-modal mapping
  • Recurrent neural network with parametric bias

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence
  • Computer Vision and Pattern Recognition
  • Signal Processing

Cite this

Inter-modality mapping in robot with recurrent neural network. / Ogata, Tetsuya; Nishide, Shun; Kozima, Hideki; Komatani, Kazunori; Okuno, Hiroshi G.

In: Pattern Recognition Letters, Vol. 31, No. 12, 01.09.2010, p. 1560-1569.

Research output: Contribution to journalArticle

Ogata, Tetsuya ; Nishide, Shun ; Kozima, Hideki ; Komatani, Kazunori ; Okuno, Hiroshi G. / Inter-modality mapping in robot with recurrent neural network. In: Pattern Recognition Letters. 2010 ; Vol. 31, No. 12. pp. 1560-1569.
@article{267e93b992574330ae7a07751d2466dd,
title = "Inter-modality mapping in robot with recurrent neural network",
abstract = "A system for mapping between different sensory modalities was developed for a robot system to enable it to generate motions expressing auditory signals and sounds generated by object movement. A recurrent neural network model with parametric bias, which has good generalization ability, is used as a learning model. Since the correspondences between auditory signals and visual signals are too numerous to memorize, the ability to generalize is indispensable. This system was implemented in the {"}Keepon{"} robot, and the robot was shown horizontal reciprocating or rotating motions with the sound of friction and falling or overturning motion with the sound of collision by manipulating a box object. Keepon behaved appropriately not only from learned events but also from unknown events and generated various sounds in accordance with observed motions.",
keywords = "Dynamical systems, Generalization, Inter-modal mapping, Recurrent neural network with parametric bias",
author = "Tetsuya Ogata and Shun Nishide and Hideki Kozima and Kazunori Komatani and Okuno, {Hiroshi G.}",
year = "2010",
month = "9",
day = "1",
doi = "10.1016/j.patrec.2010.05.002",
language = "English",
volume = "31",
pages = "1560--1569",
journal = "Pattern Recognition Letters",
issn = "0167-8655",
publisher = "Elsevier",
number = "12",

}

TY - JOUR

T1 - Inter-modality mapping in robot with recurrent neural network

AU - Ogata, Tetsuya

AU - Nishide, Shun

AU - Kozima, Hideki

AU - Komatani, Kazunori

AU - Okuno, Hiroshi G.

PY - 2010/9/1

Y1 - 2010/9/1

N2 - A system for mapping between different sensory modalities was developed for a robot system to enable it to generate motions expressing auditory signals and sounds generated by object movement. A recurrent neural network model with parametric bias, which has good generalization ability, is used as a learning model. Since the correspondences between auditory signals and visual signals are too numerous to memorize, the ability to generalize is indispensable. This system was implemented in the "Keepon" robot, and the robot was shown horizontal reciprocating or rotating motions with the sound of friction and falling or overturning motion with the sound of collision by manipulating a box object. Keepon behaved appropriately not only from learned events but also from unknown events and generated various sounds in accordance with observed motions.

AB - A system for mapping between different sensory modalities was developed for a robot system to enable it to generate motions expressing auditory signals and sounds generated by object movement. A recurrent neural network model with parametric bias, which has good generalization ability, is used as a learning model. Since the correspondences between auditory signals and visual signals are too numerous to memorize, the ability to generalize is indispensable. This system was implemented in the "Keepon" robot, and the robot was shown horizontal reciprocating or rotating motions with the sound of friction and falling or overturning motion with the sound of collision by manipulating a box object. Keepon behaved appropriately not only from learned events but also from unknown events and generated various sounds in accordance with observed motions.

KW - Dynamical systems

KW - Generalization

KW - Inter-modal mapping

KW - Recurrent neural network with parametric bias

UR - http://www.scopus.com/inward/record.url?scp=77955554135&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77955554135&partnerID=8YFLogxK

U2 - 10.1016/j.patrec.2010.05.002

DO - 10.1016/j.patrec.2010.05.002

M3 - Article

VL - 31

SP - 1560

EP - 1569

JO - Pattern Recognition Letters

JF - Pattern Recognition Letters

SN - 0167-8655

IS - 12

ER -