Learning to control a joint driven double inverted pendulum using nested actor/critic algorithm

N. Kobori, K. Suzuki, P. Hartono, S. Hashimoto

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    2 Citations (Scopus)

    Abstract

    In recent years, 'Reinforcement Learning' which can acquire reflective and adaptive actions, is becoming the center of attention as a learning method for robotics control. However, there are many unsolved problems that have to be cleared in order to put the method into practical use. One of the problems is the handling of the state space and the action space. Many algorithms of existing reinforcement learning deal with discrete state space and action space. When the unit of search space is rough, a subtle control cannot be achieved (imperfect perception). On the contrary, when the unit of search space is too fine, searching space is enlarged accordingly and the stable convergence of learning cannot be obtained (curse of dimensionality). In this paper, we propose a nested actor/critic algorithm that can deal with the continuous state and action space. The method proposed in this paper inserts a child actor/critic into the actor part of parent actor/critic algorithm. We examined the proposed algorithm for a stable control problem in both simulation and prototype model of a joint-driven double inverted pendulum.

    Original languageEnglish
    Title of host publicationICONIP 2002 - Proceedings of the 9th International Conference on Neural Information Processing: Computational Intelligence for the E-Age
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    Pages2610-2614
    Number of pages5
    Volume5
    ISBN (Electronic)9810475241, 9789810475246
    DOIs
    Publication statusPublished - 2002
    Event9th International Conference on Neural Information Processing, ICONIP 2002 - Singapore, Singapore
    Duration: 2002 Nov 182002 Nov 22

    Other

    Other9th International Conference on Neural Information Processing, ICONIP 2002
    CountrySingapore
    CitySingapore
    Period02/11/1802/11/22

    Fingerprint

    Pendulums
    Reinforcement learning
    Robotics

    ASJC Scopus subject areas

    • Computer Networks and Communications
    • Information Systems
    • Signal Processing

    Cite this

    Kobori, N., Suzuki, K., Hartono, P., & Hashimoto, S. (2002). Learning to control a joint driven double inverted pendulum using nested actor/critic algorithm. In ICONIP 2002 - Proceedings of the 9th International Conference on Neural Information Processing: Computational Intelligence for the E-Age (Vol. 5, pp. 2610-2614). [1201968] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICONIP.2002.1201968

    Learning to control a joint driven double inverted pendulum using nested actor/critic algorithm. / Kobori, N.; Suzuki, K.; Hartono, P.; Hashimoto, S.

    ICONIP 2002 - Proceedings of the 9th International Conference on Neural Information Processing: Computational Intelligence for the E-Age. Vol. 5 Institute of Electrical and Electronics Engineers Inc., 2002. p. 2610-2614 1201968.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Kobori, N, Suzuki, K, Hartono, P & Hashimoto, S 2002, Learning to control a joint driven double inverted pendulum using nested actor/critic algorithm. in ICONIP 2002 - Proceedings of the 9th International Conference on Neural Information Processing: Computational Intelligence for the E-Age. vol. 5, 1201968, Institute of Electrical and Electronics Engineers Inc., pp. 2610-2614, 9th International Conference on Neural Information Processing, ICONIP 2002, Singapore, Singapore, 02/11/18. https://doi.org/10.1109/ICONIP.2002.1201968
    Kobori N, Suzuki K, Hartono P, Hashimoto S. Learning to control a joint driven double inverted pendulum using nested actor/critic algorithm. In ICONIP 2002 - Proceedings of the 9th International Conference on Neural Information Processing: Computational Intelligence for the E-Age. Vol. 5. Institute of Electrical and Electronics Engineers Inc. 2002. p. 2610-2614. 1201968 https://doi.org/10.1109/ICONIP.2002.1201968
    Kobori, N. ; Suzuki, K. ; Hartono, P. ; Hashimoto, S. / Learning to control a joint driven double inverted pendulum using nested actor/critic algorithm. ICONIP 2002 - Proceedings of the 9th International Conference on Neural Information Processing: Computational Intelligence for the E-Age. Vol. 5 Institute of Electrical and Electronics Engineers Inc., 2002. pp. 2610-2614
    @inproceedings{8ad6f4d83cd549758ba43b329773f2df,
    title = "Learning to control a joint driven double inverted pendulum using nested actor/critic algorithm",
    abstract = "In recent years, 'Reinforcement Learning' which can acquire reflective and adaptive actions, is becoming the center of attention as a learning method for robotics control. However, there are many unsolved problems that have to be cleared in order to put the method into practical use. One of the problems is the handling of the state space and the action space. Many algorithms of existing reinforcement learning deal with discrete state space and action space. When the unit of search space is rough, a subtle control cannot be achieved (imperfect perception). On the contrary, when the unit of search space is too fine, searching space is enlarged accordingly and the stable convergence of learning cannot be obtained (curse of dimensionality). In this paper, we propose a nested actor/critic algorithm that can deal with the continuous state and action space. The method proposed in this paper inserts a child actor/critic into the actor part of parent actor/critic algorithm. We examined the proposed algorithm for a stable control problem in both simulation and prototype model of a joint-driven double inverted pendulum.",
    author = "N. Kobori and K. Suzuki and P. Hartono and S. Hashimoto",
    year = "2002",
    doi = "10.1109/ICONIP.2002.1201968",
    language = "English",
    volume = "5",
    pages = "2610--2614",
    booktitle = "ICONIP 2002 - Proceedings of the 9th International Conference on Neural Information Processing: Computational Intelligence for the E-Age",
    publisher = "Institute of Electrical and Electronics Engineers Inc.",
    address = "United States",

    }

    TY - GEN

    T1 - Learning to control a joint driven double inverted pendulum using nested actor/critic algorithm

    AU - Kobori, N.

    AU - Suzuki, K.

    AU - Hartono, P.

    AU - Hashimoto, S.

    PY - 2002

    Y1 - 2002

    N2 - In recent years, 'Reinforcement Learning' which can acquire reflective and adaptive actions, is becoming the center of attention as a learning method for robotics control. However, there are many unsolved problems that have to be cleared in order to put the method into practical use. One of the problems is the handling of the state space and the action space. Many algorithms of existing reinforcement learning deal with discrete state space and action space. When the unit of search space is rough, a subtle control cannot be achieved (imperfect perception). On the contrary, when the unit of search space is too fine, searching space is enlarged accordingly and the stable convergence of learning cannot be obtained (curse of dimensionality). In this paper, we propose a nested actor/critic algorithm that can deal with the continuous state and action space. The method proposed in this paper inserts a child actor/critic into the actor part of parent actor/critic algorithm. We examined the proposed algorithm for a stable control problem in both simulation and prototype model of a joint-driven double inverted pendulum.

    AB - In recent years, 'Reinforcement Learning' which can acquire reflective and adaptive actions, is becoming the center of attention as a learning method for robotics control. However, there are many unsolved problems that have to be cleared in order to put the method into practical use. One of the problems is the handling of the state space and the action space. Many algorithms of existing reinforcement learning deal with discrete state space and action space. When the unit of search space is rough, a subtle control cannot be achieved (imperfect perception). On the contrary, when the unit of search space is too fine, searching space is enlarged accordingly and the stable convergence of learning cannot be obtained (curse of dimensionality). In this paper, we propose a nested actor/critic algorithm that can deal with the continuous state and action space. The method proposed in this paper inserts a child actor/critic into the actor part of parent actor/critic algorithm. We examined the proposed algorithm for a stable control problem in both simulation and prototype model of a joint-driven double inverted pendulum.

    UR - http://www.scopus.com/inward/record.url?scp=18444383994&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=18444383994&partnerID=8YFLogxK

    U2 - 10.1109/ICONIP.2002.1201968

    DO - 10.1109/ICONIP.2002.1201968

    M3 - Conference contribution

    AN - SCOPUS:18444383994

    VL - 5

    SP - 2610

    EP - 2614

    BT - ICONIP 2002 - Proceedings of the 9th International Conference on Neural Information Processing: Computational Intelligence for the E-Age

    PB - Institute of Electrical and Electronics Engineers Inc.

    ER -