The learning efficiency of a simplified version of adaptive natural gradient descent (ANGD) for soft committee machines was evaluated. Statistical-mechanical techniques, which extract order parameters and make the stochastic learning dynamics converge towards deterministic at the large limit of the input dimension N [1,2], were employed. ANGD was found to perform as well as natural gradient descent (NGD). The key condition affecting the learning plateau in ANGD were also revealed.
|ジャーナル||Physical Review E - Statistical, Nonlinear, and Soft Matter Physics|
|出版物ステータス||Published - 2004 5 1|
ASJC Scopus subject areas
- Statistical and Nonlinear Physics
- Statistics and Probability
- Condensed Matter Physics