Toward abstraction from multi-modal data: Empirical studies on multiple time-scale recurrent models

Junpei Zhong, Angelo Cangelosi, Tetsuya Ogata

研究成果: Conference contribution

9 被引用数 (Scopus)

抄録

The abstraction tasks are challenging for multi-modal sequences as they require a deeper semantic understanding and a novel text generation for the data. Although the recurrent neural networks (RNN) can be used to model the context of the time-sequences, in most cases the long-term dependencies of multi-modal data make the back-propagation through time training of RNN tend to vanish in the time domain. Recently, inspired from Multiple Time-scale Recurrent Neural Network (MTRNN) [1], an extension of Gated Recurrent Unit (GRU), called Multiple Time-scale Gated Recurrent Unit (MTGRU), has been proposed [2] to learn the long-term dependencies in natural language processing. Particularly it is also able to accomplish the abstraction task for paragraphs given that the time constants are well defined. In this paper, we compare the MTRNN and MTGRU in terms of its learning performances as well as their abstraction representation on higher level (with a slower neural activation). This was done by conducting two studies based on a smaller dataset (two-dimension time sequences from non-linear functions) and a relatively large data-set (43-dimension time sequences from iCub manipulation tasks with multi-modal data). We conclude that gated recurrent mechanisms may be necessary for learning long-term dependencies in large dimension multi-modal data-sets (e.g. learning of robot manipulation), even when natural language commands was not involved. But for smaller learning tasks with simple time-sequences, generic version of recurrent models, such as MTRNN, were sufficient to accomplish the abstraction task.

本文言語English
ホスト出版物のタイトル2017 International Joint Conference on Neural Networks, IJCNN 2017 - Proceedings
出版社Institute of Electrical and Electronics Engineers Inc.
ページ3625-3632
ページ数8
ISBN(電子版)9781509061815
DOI
出版ステータスPublished - 2017 6 30
外部発表はい
イベント2017 International Joint Conference on Neural Networks, IJCNN 2017 - Anchorage, United States
継続期間: 2017 5 142017 5 19

出版物シリーズ

名前Proceedings of the International Joint Conference on Neural Networks
2017-May

Other

Other2017 International Joint Conference on Neural Networks, IJCNN 2017
国/地域United States
CityAnchorage
Period17/5/1417/5/19

ASJC Scopus subject areas

  • ソフトウェア
  • 人工知能

フィンガープリント

「Toward abstraction from multi-modal data: Empirical studies on multiple time-scale recurrent models」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル