A continuous estimation of distribution algorithm by evolving graph structures using reinforcement learning

Xianneng Li*, Bing Li, Shingo Mabu, Kotaro Hirasawa

*この研究の対応する著者

研究成果: Conference contribution

2 被引用数 (Scopus)

抄録

A novel graph-based Estimation of Distribution Algorithm (EDA) named Probabilistic Model Building Genetic Network Programming (PMBGNP) has been proposed. Inspired by classical EDAs, PMBGNP memorizes the current best individuals and uses them to estimate a distribution for the generation of the new population. However, PMBGNP can evolve compact programs by representing its solutions as graph structures. Therefore, it can solve a range of problems different from conventional ones in EDA literature, such as data mining and Reinforcement Learning (RL) problems. This paper extends PMBGNP from discrete to continuous search space, which is named PMBGNP-AC. Besides evolving the node connections to determine the optimal graph structures using conventional PMBGNP, Gaussian distribution is used for the distribution of continuous variables of nodes. The mean value μ and standard deviation σ are constructed like those of classical continuous Population-based incremental learning (PBILc). However, a RL technique, i.e., Actor-Critic (AC), is designed to update the parameters (μ and σ). AC allows us to calculate the Temporal-Difference (TD) error to evaluate whether the selection of the continuous value is better or worse than expected. This scalar reinforcement signal can decide whether the tendency to select this continuous value should be strengthened or weakened, allowing us to determine the shape of the probability density functions of the Gaussian distribution. The proposed algorithm is applied to a RL problem, i.e., autonomous robot control, where the robot's wheel speeds and sensor values are continuous. The experimental results show the superiority of PMBGNP-AC comparing with the conventional algorithms.

本文言語English
ホスト出版物のタイトル2012 IEEE Congress on Evolutionary Computation, CEC 2012
DOI
出版ステータスPublished - 2012
イベント2012 IEEE Congress on Evolutionary Computation, CEC 2012 - Brisbane, QLD
継続期間: 2012 6 102012 6 15

Other

Other2012 IEEE Congress on Evolutionary Computation, CEC 2012
CityBrisbane, QLD
Period12/6/1012/6/15

ASJC Scopus subject areas

  • 計算理論と計算数学
  • 理論的コンピュータサイエンス

フィンガープリント

「A continuous estimation of distribution algorithm by evolving graph structures using reinforcement learning」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル