Reinforcement learning algorithm with CTRNN in continuous action space

Hiroaki Arie, Jun Namikawa, Tetsuya Ogata, Jun Tani, Shigeki Sugano

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    1 Citation (Scopus)

    Abstract

    There are some difficulties in applying traditional reinforcement learning algorithms to motion control tasks of robot. Because most algorithms are concerned with discrete actions and based on the assumption of complete observability of the state. This paper deals with these two problems by combining the reinforcement learning algorithm and CTRNN learning algorithm. We carried out an experiment on the pendulum swing-up task without rotational speed information. It is shown that the information about the rotational speed, which is considered as a hidden state, is estimated and encoded on the activation of a context neuron. As a result, this task is accomplished in several hundred trials using the proposed algorithm.

    Original languageEnglish
    Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    Pages387-396
    Number of pages10
    Volume4232 LNCS
    Publication statusPublished - 2006
    Event13th International Conference on Neural Information Processing, ICONIP 2006 - Hong Kong
    Duration: 2006 Oct 32006 Oct 6

    Publication series

    NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    Volume4232 LNCS
    ISSN (Print)03029743
    ISSN (Electronic)16113349

    Other

    Other13th International Conference on Neural Information Processing, ICONIP 2006
    CityHong Kong
    Period06/10/306/10/6

    Fingerprint

    Reinforcement learning
    Reinforcement Learning
    Learning algorithms
    Learning Algorithm
    Learning
    Motion Control
    Observability
    Motion control
    Pendulum
    Pendulums
    Neurons
    Activation
    Neuron
    Robot
    Chemical activation
    Robots
    Experiment
    Reinforcement (Psychology)
    Experiments

    ASJC Scopus subject areas

    • Computer Science(all)
    • Biochemistry, Genetics and Molecular Biology(all)
    • Theoretical Computer Science

    Cite this

    Arie, H., Namikawa, J., Ogata, T., Tani, J., & Sugano, S. (2006). Reinforcement learning algorithm with CTRNN in continuous action space. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 4232 LNCS, pp. 387-396). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4232 LNCS).

    Reinforcement learning algorithm with CTRNN in continuous action space. / Arie, Hiroaki; Namikawa, Jun; Ogata, Tetsuya; Tani, Jun; Sugano, Shigeki.

    Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 4232 LNCS 2006. p. 387-396 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4232 LNCS).

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Arie, H, Namikawa, J, Ogata, T, Tani, J & Sugano, S 2006, Reinforcement learning algorithm with CTRNN in continuous action space. in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). vol. 4232 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 4232 LNCS, pp. 387-396, 13th International Conference on Neural Information Processing, ICONIP 2006, Hong Kong, 06/10/3.
    Arie H, Namikawa J, Ogata T, Tani J, Sugano S. Reinforcement learning algorithm with CTRNN in continuous action space. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 4232 LNCS. 2006. p. 387-396. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
    Arie, Hiroaki ; Namikawa, Jun ; Ogata, Tetsuya ; Tani, Jun ; Sugano, Shigeki. / Reinforcement learning algorithm with CTRNN in continuous action space. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 4232 LNCS 2006. pp. 387-396 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
    @inproceedings{2a0eff6c5c224679ae603ab0d588766a,
    title = "Reinforcement learning algorithm with CTRNN in continuous action space",
    abstract = "There are some difficulties in applying traditional reinforcement learning algorithms to motion control tasks of robot. Because most algorithms are concerned with discrete actions and based on the assumption of complete observability of the state. This paper deals with these two problems by combining the reinforcement learning algorithm and CTRNN learning algorithm. We carried out an experiment on the pendulum swing-up task without rotational speed information. It is shown that the information about the rotational speed, which is considered as a hidden state, is estimated and encoded on the activation of a context neuron. As a result, this task is accomplished in several hundred trials using the proposed algorithm.",
    author = "Hiroaki Arie and Jun Namikawa and Tetsuya Ogata and Jun Tani and Shigeki Sugano",
    year = "2006",
    language = "English",
    isbn = "3540464794",
    volume = "4232 LNCS",
    series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
    pages = "387--396",
    booktitle = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

    }

    TY - GEN

    T1 - Reinforcement learning algorithm with CTRNN in continuous action space

    AU - Arie, Hiroaki

    AU - Namikawa, Jun

    AU - Ogata, Tetsuya

    AU - Tani, Jun

    AU - Sugano, Shigeki

    PY - 2006

    Y1 - 2006

    N2 - There are some difficulties in applying traditional reinforcement learning algorithms to motion control tasks of robot. Because most algorithms are concerned with discrete actions and based on the assumption of complete observability of the state. This paper deals with these two problems by combining the reinforcement learning algorithm and CTRNN learning algorithm. We carried out an experiment on the pendulum swing-up task without rotational speed information. It is shown that the information about the rotational speed, which is considered as a hidden state, is estimated and encoded on the activation of a context neuron. As a result, this task is accomplished in several hundred trials using the proposed algorithm.

    AB - There are some difficulties in applying traditional reinforcement learning algorithms to motion control tasks of robot. Because most algorithms are concerned with discrete actions and based on the assumption of complete observability of the state. This paper deals with these two problems by combining the reinforcement learning algorithm and CTRNN learning algorithm. We carried out an experiment on the pendulum swing-up task without rotational speed information. It is shown that the information about the rotational speed, which is considered as a hidden state, is estimated and encoded on the activation of a context neuron. As a result, this task is accomplished in several hundred trials using the proposed algorithm.

    UR - http://www.scopus.com/inward/record.url?scp=33750590179&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=33750590179&partnerID=8YFLogxK

    M3 - Conference contribution

    AN - SCOPUS:33750590179

    SN - 3540464794

    SN - 9783540464792

    VL - 4232 LNCS

    T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

    SP - 387

    EP - 396

    BT - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

    ER -