Dynamics of the adaptive natural gradient descent method for soft committee machines

Masato Inoue, Hyeyoung Park, Masato Okada

Research output: Contribution to journalArticle

4 Citations (Scopus)


The learning efficiency of a simplified version of adaptive natural gradient descent (ANGD) for soft committee machines was evaluated. Statistical-mechanical techniques, which extract order parameters and make the stochastic learning dynamics converge towards deterministic at the large limit of the input dimension N [1,2], were employed. ANGD was found to perform as well as natural gradient descent (NGD). The key condition affecting the learning plateau in ANGD were also revealed.

Original languageEnglish
Article number056120
Pages (from-to)056120-1-056120-14
JournalPhysical Review E - Statistical, Nonlinear, and Soft Matter Physics
Issue number5 1
Publication statusPublished - 2004 May 1


ASJC Scopus subject areas

  • Statistical and Nonlinear Physics
  • Statistics and Probability
  • Condensed Matter Physics

Cite this