Genetic network programming with actor-critic and its application

A new graph-based evolutionary algorithm named "Genetic Network Programming, GNP" was proposed. GNP represents its solutions as graph structures, which can improve the expression ability and performance. And then, GNP with Reinforcement Learning (GNP with RL) has been proposed in order to search for solutions efficiently. GNP with RL can use the current information (state and reward) and change its programs during task execution, so it has an advantage over the evolution based algorithms in case much information can be obtained during task execution. In this paper, GNP with Actor-Critic (GNP-AC) which is a new type of GNP with RL is proposed. Originally, GNP deals with discrete information (ex. right, left, etc.), but GNP with AC aims to deal with continuous information (ex. the sensor value is "32"). The proposed method is applied to the controller of the Khepera simulator and its performance is evaluated.