CMA-TWEANN: Efficient optimization of neural networks via self-adaptation and seamless augmentation

Hirotaka Moriguchi, Shinichi Honiden

Research output: Chapter in Book/Report/Conference proceedingConference contribution

11 Citations (Scopus)

Abstract

Neuroevolutionary algorithms are successful methods for optimizing neural networks, especially for learning a neural policy (controller) in reinforcement learning tasks. Their significant advantage over gradient-based algorithms is the capability to search network topology as well as connection weights. However, state-of-the-art topology evolving methods are known to be inefficient compared to weight evolving methods with an appropriately hand-tuned topology. This paper introduces a novel efficient algorithm called CMA-TWEANN for evolving both topology and weights. Its high efficiency is achieved by introducing efficient topological mutation operators and integrating a state-of-the-art function optimization algorithm for weight optimization. Experiments on benchmark reinforcement learning tasks demonstrate that CMA-TWEANN solves tasks significantly faster than existing topology evolving methods. Furthermore, it outperforms weight evolving techniques even when they are equipped with a hand-tuned topology. Additional experiments reveal how and why CMA-TWEANN is the best performing weight evolving method.

Original languageEnglish
Title of host publicationGECCO'12 - Proceedings of the 14th International Conference on Genetic and Evolutionary Computation
Pages903-910
Number of pages8
DOIs
Publication statusPublished - 2012 Aug 13
Externally publishedYes
Event14th International Conference on Genetic and Evolutionary Computation, GECCO'12 - Philadelphia, PA, United States
Duration: 2012 Jul 72012 Jul 11

Other

Other14th International Conference on Genetic and Evolutionary Computation, GECCO'12
CountryUnited States
CityPhiladelphia, PA
Period12/7/712/7/11

Fingerprint

Self-adaptation
Augmentation
Topology
Neural Networks
Neural networks
Optimization
Reinforcement learning
Reinforcement Learning
Function Optimization
Network Topology
High Efficiency
Experiment
Mathematical operators
Optimization Algorithm
Mutation
Efficient Algorithms
Experiments
Benchmark
Gradient
Controller

Keywords

  • neuroevolution
  • reinforcement leanring
  • tweann

ASJC Scopus subject areas

  • Computational Theory and Mathematics
  • Applied Mathematics

Cite this

Moriguchi, H., & Honiden, S. (2012). CMA-TWEANN: Efficient optimization of neural networks via self-adaptation and seamless augmentation. In GECCO'12 - Proceedings of the 14th International Conference on Genetic and Evolutionary Computation (pp. 903-910) https://doi.org/10.1145/2330163.2330288

CMA-TWEANN : Efficient optimization of neural networks via self-adaptation and seamless augmentation. / Moriguchi, Hirotaka; Honiden, Shinichi.

GECCO'12 - Proceedings of the 14th International Conference on Genetic and Evolutionary Computation. 2012. p. 903-910.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Moriguchi, H & Honiden, S 2012, CMA-TWEANN: Efficient optimization of neural networks via self-adaptation and seamless augmentation. in GECCO'12 - Proceedings of the 14th International Conference on Genetic and Evolutionary Computation. pp. 903-910, 14th International Conference on Genetic and Evolutionary Computation, GECCO'12, Philadelphia, PA, United States, 12/7/7. https://doi.org/10.1145/2330163.2330288
Moriguchi H, Honiden S. CMA-TWEANN: Efficient optimization of neural networks via self-adaptation and seamless augmentation. In GECCO'12 - Proceedings of the 14th International Conference on Genetic and Evolutionary Computation. 2012. p. 903-910 https://doi.org/10.1145/2330163.2330288
Moriguchi, Hirotaka ; Honiden, Shinichi. / CMA-TWEANN : Efficient optimization of neural networks via self-adaptation and seamless augmentation. GECCO'12 - Proceedings of the 14th International Conference on Genetic and Evolutionary Computation. 2012. pp. 903-910
@inproceedings{db0503215323465888e28f61b832d9c8,
title = "CMA-TWEANN: Efficient optimization of neural networks via self-adaptation and seamless augmentation",
abstract = "Neuroevolutionary algorithms are successful methods for optimizing neural networks, especially for learning a neural policy (controller) in reinforcement learning tasks. Their significant advantage over gradient-based algorithms is the capability to search network topology as well as connection weights. However, state-of-the-art topology evolving methods are known to be inefficient compared to weight evolving methods with an appropriately hand-tuned topology. This paper introduces a novel efficient algorithm called CMA-TWEANN for evolving both topology and weights. Its high efficiency is achieved by introducing efficient topological mutation operators and integrating a state-of-the-art function optimization algorithm for weight optimization. Experiments on benchmark reinforcement learning tasks demonstrate that CMA-TWEANN solves tasks significantly faster than existing topology evolving methods. Furthermore, it outperforms weight evolving techniques even when they are equipped with a hand-tuned topology. Additional experiments reveal how and why CMA-TWEANN is the best performing weight evolving method.",
keywords = "neuroevolution, reinforcement leanring, tweann",
author = "Hirotaka Moriguchi and Shinichi Honiden",
year = "2012",
month = "8",
day = "13",
doi = "10.1145/2330163.2330288",
language = "English",
isbn = "9781450311779",
pages = "903--910",
booktitle = "GECCO'12 - Proceedings of the 14th International Conference on Genetic and Evolutionary Computation",

}

TY - GEN

T1 - CMA-TWEANN

T2 - Efficient optimization of neural networks via self-adaptation and seamless augmentation

AU - Moriguchi, Hirotaka

AU - Honiden, Shinichi

PY - 2012/8/13

Y1 - 2012/8/13

N2 - Neuroevolutionary algorithms are successful methods for optimizing neural networks, especially for learning a neural policy (controller) in reinforcement learning tasks. Their significant advantage over gradient-based algorithms is the capability to search network topology as well as connection weights. However, state-of-the-art topology evolving methods are known to be inefficient compared to weight evolving methods with an appropriately hand-tuned topology. This paper introduces a novel efficient algorithm called CMA-TWEANN for evolving both topology and weights. Its high efficiency is achieved by introducing efficient topological mutation operators and integrating a state-of-the-art function optimization algorithm for weight optimization. Experiments on benchmark reinforcement learning tasks demonstrate that CMA-TWEANN solves tasks significantly faster than existing topology evolving methods. Furthermore, it outperforms weight evolving techniques even when they are equipped with a hand-tuned topology. Additional experiments reveal how and why CMA-TWEANN is the best performing weight evolving method.

AB - Neuroevolutionary algorithms are successful methods for optimizing neural networks, especially for learning a neural policy (controller) in reinforcement learning tasks. Their significant advantage over gradient-based algorithms is the capability to search network topology as well as connection weights. However, state-of-the-art topology evolving methods are known to be inefficient compared to weight evolving methods with an appropriately hand-tuned topology. This paper introduces a novel efficient algorithm called CMA-TWEANN for evolving both topology and weights. Its high efficiency is achieved by introducing efficient topological mutation operators and integrating a state-of-the-art function optimization algorithm for weight optimization. Experiments on benchmark reinforcement learning tasks demonstrate that CMA-TWEANN solves tasks significantly faster than existing topology evolving methods. Furthermore, it outperforms weight evolving techniques even when they are equipped with a hand-tuned topology. Additional experiments reveal how and why CMA-TWEANN is the best performing weight evolving method.

KW - neuroevolution

KW - reinforcement leanring

KW - tweann

UR - http://www.scopus.com/inward/record.url?scp=84864666853&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84864666853&partnerID=8YFLogxK

U2 - 10.1145/2330163.2330288

DO - 10.1145/2330163.2330288

M3 - Conference contribution

AN - SCOPUS:84864666853

SN - 9781450311779

SP - 903

EP - 910

BT - GECCO'12 - Proceedings of the 14th International Conference on Genetic and Evolutionary Computation

ER -