Reinforcement learning algorithm with CTRNN in continuous action space

Hiroaki Arie; Jun Namikawa; Tetsuya Ogata; Jun Tani; Shigeki Sugano

Reinforcement learning algorithm with CTRNN in continuous action space

Hiroaki Arie^*, Jun Namikawa, Tetsuya Ogata, Jun Tani, Shigeki Sugano

^*Corresponding author for this work

School of Creative Science and Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Citations (Scopus)

Abstract

There are some difficulties in applying traditional reinforcement learning algorithms to motion control tasks of robot. Because most algorithms are concerned with discrete actions and based on the assumption of complete observability of the state. This paper deals with these two problems by combining the reinforcement learning algorithm and CTRNN learning algorithm. We carried out an experiment on the pendulum swing-up task without rotational speed information. It is shown that the information about the rotational speed, which is considered as a hidden state, is estimated and encoded on the activation of a context neuron. As a result, this task is accomplished in several hundred trials using the proposed algorithm.

Original language	English
Title of host publication	Neural Information Processing - 13th International Conference, ICONIP 2006, Proceedings
Publisher	Springer Verlag
Pages	387-396
Number of pages	10
ISBN (Print)	3540464794, 9783540464792
Publication status	Published - 2006 Jan 1
Event	13th International Conference on Neural Information Processing, ICONIP 2006 - Hong Kong, China Duration: 2006 Oct 3 → 2006 Oct 6

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	4232 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	13th International Conference on Neural Information Processing, ICONIP 2006
Country/Territory	China
City	Hong Kong
Period	06/10/3 → 06/10/6

ASJC Scopus subject areas

Theoretical Computer Science
Computer Science(all)

Cite this

Arie, H., Namikawa, J., Ogata, T., Tani, J., & Sugano, S. (2006). Reinforcement learning algorithm with CTRNN in continuous action space. In Neural Information Processing - 13th International Conference, ICONIP 2006, Proceedings (pp. 387-396). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4232 LNCS). Springer Verlag.

Reinforcement learning algorithm with CTRNN in continuous action space. / Arie, Hiroaki; Namikawa, Jun; Ogata, Tetsuya; Tani, Jun; Sugano, Shigeki.

Neural Information Processing - 13th International Conference, ICONIP 2006, Proceedings. Springer Verlag, 2006. p. 387-396 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4232 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Arie, H, Namikawa, J, Ogata, T, Tani, J & Sugano, S 2006, Reinforcement learning algorithm with CTRNN in continuous action space. in Neural Information Processing - 13th International Conference, ICONIP 2006, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 4232 LNCS, Springer Verlag, pp. 387-396, 13th International Conference on Neural Information Processing, ICONIP 2006, Hong Kong, China, 06/10/3.

Arie H, Namikawa J, Ogata T, Tani J, Sugano S. Reinforcement learning algorithm with CTRNN in continuous action space. In Neural Information Processing - 13th International Conference, ICONIP 2006, Proceedings. Springer Verlag. 2006. p. 387-396. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

Arie, Hiroaki ; Namikawa, Jun ; Ogata, Tetsuya ; Tani, Jun ; Sugano, Shigeki. / Reinforcement learning algorithm with CTRNN in continuous action space. Neural Information Processing - 13th International Conference, ICONIP 2006, Proceedings. Springer Verlag, 2006. pp. 387-396 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{2a0eff6c5c224679ae603ab0d588766a,

title = "Reinforcement learning algorithm with CTRNN in continuous action space",

abstract = "There are some difficulties in applying traditional reinforcement learning algorithms to motion control tasks of robot. Because most algorithms are concerned with discrete actions and based on the assumption of complete observability of the state. This paper deals with these two problems by combining the reinforcement learning algorithm and CTRNN learning algorithm. We carried out an experiment on the pendulum swing-up task without rotational speed information. It is shown that the information about the rotational speed, which is considered as a hidden state, is estimated and encoded on the activation of a context neuron. As a result, this task is accomplished in several hundred trials using the proposed algorithm.",

author = "Hiroaki Arie and Jun Namikawa and Tetsuya Ogata and Jun Tani and Shigeki Sugano",

year = "2006",

month = jan,

day = "1",

language = "English",

isbn = "3540464794",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "387--396",

booktitle = "Neural Information Processing - 13th International Conference, ICONIP 2006, Proceedings",

note = "13th International Conference on Neural Information Processing, ICONIP 2006 ; Conference date: 03-10-2006 Through 06-10-2006",

}

TY - GEN

T1 - Reinforcement learning algorithm with CTRNN in continuous action space

AU - Arie, Hiroaki

AU - Namikawa, Jun

AU - Ogata, Tetsuya

AU - Tani, Jun

AU - Sugano, Shigeki

PY - 2006/1/1

Y1 - 2006/1/1

N2 - There are some difficulties in applying traditional reinforcement learning algorithms to motion control tasks of robot. Because most algorithms are concerned with discrete actions and based on the assumption of complete observability of the state. This paper deals with these two problems by combining the reinforcement learning algorithm and CTRNN learning algorithm. We carried out an experiment on the pendulum swing-up task without rotational speed information. It is shown that the information about the rotational speed, which is considered as a hidden state, is estimated and encoded on the activation of a context neuron. As a result, this task is accomplished in several hundred trials using the proposed algorithm.

AB - There are some difficulties in applying traditional reinforcement learning algorithms to motion control tasks of robot. Because most algorithms are concerned with discrete actions and based on the assumption of complete observability of the state. This paper deals with these two problems by combining the reinforcement learning algorithm and CTRNN learning algorithm. We carried out an experiment on the pendulum swing-up task without rotational speed information. It is shown that the information about the rotational speed, which is considered as a hidden state, is estimated and encoded on the activation of a context neuron. As a result, this task is accomplished in several hundred trials using the proposed algorithm.

UR - http://www.scopus.com/inward/record.url?scp=33750590179&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33750590179&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:33750590179

SN - 3540464794

SN - 9783540464792

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 387

EP - 396

BT - Neural Information Processing - 13th International Conference, ICONIP 2006, Proceedings

PB - Springer Verlag

T2 - 13th International Conference on Neural Information Processing, ICONIP 2006

Y2 - 3 October 2006 through 6 October 2006

ER -

Reinforcement learning algorithm with CTRNN in continuous action space

Abstract

Publication series

Conference

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this