Reinforcement learning of a continuous motor sequence with hidden states

Hiroaki Arie*, Tetsuya Ogata, Jun Tani, Shigeki Sugano


研究成果: Article査読

13 被引用数 (Scopus)


Reinforcement learning is the scheme for unsupervised learning in which robots are expected to acquire behavior skills through self-explorations based on reward signals. There are some difficulties, however, in applying conventional reinforcement learning algorithms to motion control tasks of a robot because most algorithms are concerned with discrete state space and based on the assumption of complete observability of the state. Real-world environments often have partial observablility; therefore, robots have to estimate the unobservable hidden states. This paper proposes a method to solve these two problems by combining the reinforcement learning algorithm and a learning algorithm for a continuous time recurrent neural network (CTRNN). The CTRNN can learn spatio-temporal structures in a continuous time and space domain, and can preserve the contextual flow by a self-organizing appropriate internal memory structure. This enables the robot to deal with the hidden state problem. We carried out an experiment on the pendulum swing-up task without rotational speed information. As a result, this task is accomplished in several hundred trials using the proposed algorithm. In addition, it is shown that the information about the rotational speed of the pendulum, which is considered as a hidden state, is estimated and encoded on the activation of a context neuron.

ジャーナルAdvanced Robotics
出版ステータスPublished - 2007 10月 1

ASJC Scopus subject areas

  • ソフトウェア
  • 制御およびシステム工学
  • 人間とコンピュータの相互作用
  • ハードウェアとアーキテクチャ
  • コンピュータ サイエンスの応用


「Reinforcement learning of a continuous motor sequence with hidden states」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。