Two-way translation of compound sentences and arm motions by recurrent neural networks

Tetsuya Ogata; Masamitsu Murase; Jim Tani; Kazunori Komatani; Hiroshi G. Okuno

doi:10.1109/IROS.2007.4399265

Two-way translation of compound sentences and arm motions by recurrent neural networks

Tetsuya Ogata^*, Masamitsu Murase, Jim Tani, Kazunori Komatani, Hiroshi G. Okuno

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

45 Citations (Scopus)

Abstract

We present a connectionist model that combines motions and language based on the behavioral experiences of a real robot. Two models of recurrent neural network with parametric bias (RNNPB) were trained using motion sequences and linguistic sequences. These sequences were combined using their respective parameters so that the robot could handle many-to-many relationships between motion sequences and linguistic sequences. Motion sequences were articulated into some primitives corresponding to given linguistic sequences using the prediction error of the RNNPB model. The experimental task in which a humanoid robot moved its arm on a table demonstrated that the robot could generate a motion sequence corresponding to given linguistic sequence even if the motions or sequences were not included in the training data, and vice versa.

Original language	English
Title of host publication	Proceedings of the 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2007
Pages	1858-1863
Number of pages	6
DOIs	https://doi.org/10.1109/IROS.2007.4399265
Publication status	Published - 2007 Dec 1
Externally published	Yes
Event	2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2007 - San Diego, CA, United States Duration: 2007 Oct 29 → 2007 Nov 2

Publication series

Name	IEEE International Conference on Intelligent Robots and Systems

Conference

Conference	2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2007
Country/Territory	United States
City	San Diego, CA
Period	07/10/29 → 07/11/2

ASJC Scopus subject areas

Control and Systems Engineering
Software
Computer Vision and Pattern Recognition
Computer Science Applications

Access to Document

10.1109/IROS.2007.4399265

Cite this

Ogata, T., Murase, M., Tani, J., Komatani, K., & Okuno, H. G. (2007). Two-way translation of compound sentences and arm motions by recurrent neural networks. In Proceedings of the 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2007 (pp. 1858-1863). [4399265] (IEEE International Conference on Intelligent Robots and Systems). https://doi.org/10.1109/IROS.2007.4399265

Two-way translation of compound sentences and arm motions by recurrent neural networks. / Ogata, Tetsuya; Murase, Masamitsu; Tani, Jim et al.

Proceedings of the 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2007. 2007. p. 1858-1863 4399265 (IEEE International Conference on Intelligent Robots and Systems).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Ogata, T, Murase, M, Tani, J, Komatani, K & Okuno, HG 2007, Two-way translation of compound sentences and arm motions by recurrent neural networks. in Proceedings of the 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2007., 4399265, IEEE International Conference on Intelligent Robots and Systems, pp. 1858-1863, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2007, San Diego, CA, United States, 07/10/29. https://doi.org/10.1109/IROS.2007.4399265

Ogata T, Murase M, Tani J, Komatani K, Okuno HG. Two-way translation of compound sentences and arm motions by recurrent neural networks. In Proceedings of the 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2007. 2007. p. 1858-1863. 4399265. (IEEE International Conference on Intelligent Robots and Systems). doi: 10.1109/IROS.2007.4399265

@inproceedings{d4ca7fab8d3e40368ff7f7ffd7b8b93d,

title = "Two-way translation of compound sentences and arm motions by recurrent neural networks",

abstract = "We present a connectionist model that combines motions and language based on the behavioral experiences of a real robot. Two models of recurrent neural network with parametric bias (RNNPB) were trained using motion sequences and linguistic sequences. These sequences were combined using their respective parameters so that the robot could handle many-to-many relationships between motion sequences and linguistic sequences. Motion sequences were articulated into some primitives corresponding to given linguistic sequences using the prediction error of the RNNPB model. The experimental task in which a humanoid robot moved its arm on a table demonstrated that the robot could generate a motion sequence corresponding to given linguistic sequence even if the motions or sequences were not included in the training data, and vice versa.",

author = "Tetsuya Ogata and Masamitsu Murase and Jim Tani and Kazunori Komatani and Okuno, {Hiroshi G.}",

year = "2007",

month = dec,

day = "1",

doi = "10.1109/IROS.2007.4399265",

language = "English",

isbn = "1424409128",

series = "IEEE International Conference on Intelligent Robots and Systems",

pages = "1858--1863",

booktitle = "Proceedings of the 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2007",

note = "2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2007 ; Conference date: 29-10-2007 Through 02-11-2007",

}

TY - GEN

T1 - Two-way translation of compound sentences and arm motions by recurrent neural networks

AU - Ogata, Tetsuya

AU - Murase, Masamitsu

AU - Tani, Jim

AU - Komatani, Kazunori

AU - Okuno, Hiroshi G.

PY - 2007/12/1

Y1 - 2007/12/1

N2 - We present a connectionist model that combines motions and language based on the behavioral experiences of a real robot. Two models of recurrent neural network with parametric bias (RNNPB) were trained using motion sequences and linguistic sequences. These sequences were combined using their respective parameters so that the robot could handle many-to-many relationships between motion sequences and linguistic sequences. Motion sequences were articulated into some primitives corresponding to given linguistic sequences using the prediction error of the RNNPB model. The experimental task in which a humanoid robot moved its arm on a table demonstrated that the robot could generate a motion sequence corresponding to given linguistic sequence even if the motions or sequences were not included in the training data, and vice versa.

AB - We present a connectionist model that combines motions and language based on the behavioral experiences of a real robot. Two models of recurrent neural network with parametric bias (RNNPB) were trained using motion sequences and linguistic sequences. These sequences were combined using their respective parameters so that the robot could handle many-to-many relationships between motion sequences and linguistic sequences. Motion sequences were articulated into some primitives corresponding to given linguistic sequences using the prediction error of the RNNPB model. The experimental task in which a humanoid robot moved its arm on a table demonstrated that the robot could generate a motion sequence corresponding to given linguistic sequence even if the motions or sequences were not included in the training data, and vice versa.

UR - http://www.scopus.com/inward/record.url?scp=51349107869&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=51349107869&partnerID=8YFLogxK

U2 - 10.1109/IROS.2007.4399265

DO - 10.1109/IROS.2007.4399265

M3 - Conference contribution

AN - SCOPUS:51349107869

SN - 1424409128

SN - 9781424409129

T3 - IEEE International Conference on Intelligent Robots and Systems

SP - 1858

EP - 1863

BT - Proceedings of the 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2007

T2 - 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2007

Y2 - 29 October 2007 through 2 November 2007

ER -

Two-way translation of compound sentences and arm motions by recurrent neural networks

Abstract

Publication series

Conference

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this