Dynamics of the adaptive natural gradient descent method for soft committee machines

Masato Inoue, Hyeyoung Park, Masato Okada

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

The learning efficiency of a simplified version of adaptive natural gradient descent (ANGD) for soft committee machines was evaluated. Statistical-mechanical techniques, which extract order parameters and make the stochastic learning dynamics converge towards deterministic at the large limit of the input dimension N [1,2], were employed. ANGD was found to perform as well as natural gradient descent (NGD). The key condition affecting the learning plateau in ANGD were also revealed.

Original languageEnglish
Article number056120
Pages (from-to)056120-1-056120-14
JournalPhysical Review E - Statistical, Nonlinear, and Soft Matter Physics
Volume69
Issue number5 1
Publication statusPublished - 2004 May 1
Externally publishedYes

ASJC Scopus subject areas

  • Statistical and Nonlinear Physics
  • Statistics and Probability
  • Condensed Matter Physics

Fingerprint Dive into the research topics of 'Dynamics of the adaptive natural gradient descent method for soft committee machines'. Together they form a unique fingerprint.

  • Cite this