Motion from sound: Intermodal neural network mapping

Tetsuya Ogata, Hiroshi G. Okuno, Hideki Kozima

Research output: Contribution to journalArticle

Abstract

A technological method has been developed for intermodal mapping to generate robot motion from various sounds as well as to generate sounds from motions. The procedure consists of two phases, first the learning phase in which it observes some events together with associated sounds and then memorizes those sounds along with the motions of the sound source. Second phase is the interacting phase in which the robot receives limited sensory information from a single modality as input and associates this with different modality and expresses it. The recurrent-neural-network model with parametric bias (RNNPB) is applied that uses the current state-vector as input for outputting the next state-vector. The RNNPB model can self-organize the values that encode the input dynamics into special parametric-bias modes to reproduce he multimodal sensory flow.

Original languageEnglish
Article number4475863
Pages (from-to)76-78
Number of pages3
JournalIEEE Intelligent Systems
Volume23
Issue number2
DOIs
Publication statusPublished - 2008 Mar
Externally publishedYes

    Fingerprint

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Electrical and Electronic Engineering
  • Artificial Intelligence

Cite this