A sample efficient model-based deep reinforcement learning algorithm with experience replay for robot manipulation

Cheng Zhang, Liang Ma*, Alexander Schmitz

*この研究の対応する著者

研究成果: Article査読

抄録

For robot manipulation, reinforcement learning has provided an effective end to end approach in controlling the complicated dynamic system. Model-free reinforcement learning methods ignore the model of system dynamics and are limited to simple behavior control. By contrast, model-based methods can quickly reach optimal trajectory planning by building a dynamic system model. However, it is not easy to build an accurate and efficient system model with high generalization ability, especially when facing complex dynamic system and various manipulation tasks. Furthermore, when the rewards provided by the environment are sparse, the agent will also lose effective guidance and fail to optimize the policy efficiently, which results in considerably decreased sample efficiency. In this paper, a model-based deep reinforcement learning algorithm, in which a deep neural network model is utilized to simulate the system dynamics, is designed for robot manipulation. The proposed deep neural network model is robust enough to deal with complex control tasks and possesses the generalization ability. Moreover, a curiosity-based experience replay method is incorporated to solve the sparse reward problem and improve the sample efficiency in reinforcement learning. The agent who manipulates a robotic hand, will be encouraged to explore optimal trajectories according to the failure experience. Simulation experiment results show great effectiveness of the proposed method. Various manipulation tasks are achieved successfully in such a complex dynamic system and the sample efficiency gets improved even in a sparse reward environment, as the learning time gets reduced considerably.

本文言語English
ページ(範囲)217-228
ページ数12
ジャーナルInternational Journal of Intelligent Robotics and Applications
4
2
DOI
出版ステータスPublished - 2020 6 1

ASJC Scopus subject areas

  • コンピュータ サイエンスの応用
  • 人工知能

フィンガープリント

「A sample efficient model-based deep reinforcement learning algorithm with experience replay for robot manipulation」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル