Paired recurrent autoencoders for bidirectional translation between robot actions and linguistic descriptions

Tatsuro Yamada, Hiroyuki Matsunaga, Tetsuya Ogata

研究成果: Article

9 引用 (Scopus)

抜粋

We propose a novel deep learning framework for bidirectional translation between robot actions and their linguistic descriptions. Our model consists of two recurrent autoencoders (RAEs). One RAE learns to encode action sequences as fixed-dimensional vectors in a way that allows the sequences to be reproduced from the vectors by its decoder. The other RAE learns to encode descriptions in a similar way. In the learning process, in addition to reproduction losses, we create another loss function whereby the representations of an action and its corresponding description approach each other in the latent vector space. Across the shared representation, the trained model can produce a linguistic description given a robot action. The model is also able to generate an appropriate action by receiving a linguistic instruction, conditioned on the current visual input. Visualization of the latent representations shows that the robot actions are embedded in a semantically compositional way in the vector space by being learned jointly with descriptions.

元の言語English
記事番号8403309
ページ(範囲)3441-3448
ページ数8
ジャーナルIEEE Robotics and Automation Letters
3
発行部数4
DOI
出版物ステータスPublished - 2018 10

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Biomedical Engineering
  • Human-Computer Interaction
  • Mechanical Engineering
  • Computer Vision and Pattern Recognition
  • Computer Science Applications
  • Control and Optimization
  • Artificial Intelligence

フィンガープリント Paired recurrent autoencoders for bidirectional translation between robot actions and linguistic descriptions' の研究トピックを掘り下げます。これらはともに一意のフィンガープリントを構成します。

  • これを引用