Two-way translation of compound sentences and arm motions by recurrent neural networks

Tetsuya Ogata, Masamitsu Murase, Jim Tani, Kazunori Komatani, Hiroshi G. Okuno

Research output: Chapter in Book/Report/Conference proceedingConference contribution

32 Citations (Scopus)

Abstract

We present a connectionist model that combines motions and language based on the behavioral experiences of a real robot. Two models of recurrent neural network with parametric bias (RNNPB) were trained using motion sequences and linguistic sequences. These sequences were combined using their respective parameters so that the robot could handle many-to-many relationships between motion sequences and linguistic sequences. Motion sequences were articulated into some primitives corresponding to given linguistic sequences using the prediction error of the RNNPB model. The experimental task in which a humanoid robot moved its arm on a table demonstrated that the robot could generate a motion sequence corresponding to given linguistic sequence even if the motions or sequences were not included in the training data, and vice versa.

Original languageEnglish
Title of host publicationIEEE International Conference on Intelligent Robots and Systems
Pages1858-1863
Number of pages6
DOIs
Publication statusPublished - 2007
Externally publishedYes
Event2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2007 - San Diego, CA
Duration: 2007 Oct 292007 Nov 2

Other

Other2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2007
CitySan Diego, CA
Period07/10/2907/11/2

Fingerprint

Recurrent neural networks
Linguistics
Robots

ASJC Scopus subject areas

  • Control and Systems Engineering

Cite this

Ogata, T., Murase, M., Tani, J., Komatani, K., & Okuno, H. G. (2007). Two-way translation of compound sentences and arm motions by recurrent neural networks. In IEEE International Conference on Intelligent Robots and Systems (pp. 1858-1863). [4399265] https://doi.org/10.1109/IROS.2007.4399265

Two-way translation of compound sentences and arm motions by recurrent neural networks. / Ogata, Tetsuya; Murase, Masamitsu; Tani, Jim; Komatani, Kazunori; Okuno, Hiroshi G.

IEEE International Conference on Intelligent Robots and Systems. 2007. p. 1858-1863 4399265.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Ogata, T, Murase, M, Tani, J, Komatani, K & Okuno, HG 2007, Two-way translation of compound sentences and arm motions by recurrent neural networks. in IEEE International Conference on Intelligent Robots and Systems., 4399265, pp. 1858-1863, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2007, San Diego, CA, 07/10/29. https://doi.org/10.1109/IROS.2007.4399265
Ogata T, Murase M, Tani J, Komatani K, Okuno HG. Two-way translation of compound sentences and arm motions by recurrent neural networks. In IEEE International Conference on Intelligent Robots and Systems. 2007. p. 1858-1863. 4399265 https://doi.org/10.1109/IROS.2007.4399265
Ogata, Tetsuya ; Murase, Masamitsu ; Tani, Jim ; Komatani, Kazunori ; Okuno, Hiroshi G. / Two-way translation of compound sentences and arm motions by recurrent neural networks. IEEE International Conference on Intelligent Robots and Systems. 2007. pp. 1858-1863
@inproceedings{d4ca7fab8d3e40368ff7f7ffd7b8b93d,
title = "Two-way translation of compound sentences and arm motions by recurrent neural networks",
abstract = "We present a connectionist model that combines motions and language based on the behavioral experiences of a real robot. Two models of recurrent neural network with parametric bias (RNNPB) were trained using motion sequences and linguistic sequences. These sequences were combined using their respective parameters so that the robot could handle many-to-many relationships between motion sequences and linguistic sequences. Motion sequences were articulated into some primitives corresponding to given linguistic sequences using the prediction error of the RNNPB model. The experimental task in which a humanoid robot moved its arm on a table demonstrated that the robot could generate a motion sequence corresponding to given linguistic sequence even if the motions or sequences were not included in the training data, and vice versa.",
author = "Tetsuya Ogata and Masamitsu Murase and Jim Tani and Kazunori Komatani and Okuno, {Hiroshi G.}",
year = "2007",
doi = "10.1109/IROS.2007.4399265",
language = "English",
isbn = "1424409128",
pages = "1858--1863",
booktitle = "IEEE International Conference on Intelligent Robots and Systems",

}

TY - GEN

T1 - Two-way translation of compound sentences and arm motions by recurrent neural networks

AU - Ogata, Tetsuya

AU - Murase, Masamitsu

AU - Tani, Jim

AU - Komatani, Kazunori

AU - Okuno, Hiroshi G.

PY - 2007

Y1 - 2007

N2 - We present a connectionist model that combines motions and language based on the behavioral experiences of a real robot. Two models of recurrent neural network with parametric bias (RNNPB) were trained using motion sequences and linguistic sequences. These sequences were combined using their respective parameters so that the robot could handle many-to-many relationships between motion sequences and linguistic sequences. Motion sequences were articulated into some primitives corresponding to given linguistic sequences using the prediction error of the RNNPB model. The experimental task in which a humanoid robot moved its arm on a table demonstrated that the robot could generate a motion sequence corresponding to given linguistic sequence even if the motions or sequences were not included in the training data, and vice versa.

AB - We present a connectionist model that combines motions and language based on the behavioral experiences of a real robot. Two models of recurrent neural network with parametric bias (RNNPB) were trained using motion sequences and linguistic sequences. These sequences were combined using their respective parameters so that the robot could handle many-to-many relationships between motion sequences and linguistic sequences. Motion sequences were articulated into some primitives corresponding to given linguistic sequences using the prediction error of the RNNPB model. The experimental task in which a humanoid robot moved its arm on a table demonstrated that the robot could generate a motion sequence corresponding to given linguistic sequence even if the motions or sequences were not included in the training data, and vice versa.

UR - http://www.scopus.com/inward/record.url?scp=51349107869&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=51349107869&partnerID=8YFLogxK

U2 - 10.1109/IROS.2007.4399265

DO - 10.1109/IROS.2007.4399265

M3 - Conference contribution

AN - SCOPUS:51349107869

SN - 1424409128

SN - 9781424409129

SP - 1858

EP - 1863

BT - IEEE International Conference on Intelligent Robots and Systems

ER -