Slow dynamics due to singularities of hierarchical learning machines

Hyeyoung Park*, Masato Inoue, Masato Okada

*この研究の対応する著者

研究成果: Article査読

10 被引用数 (Scopus)

抄録

Recently, slow dynamics in learning of neural networks has been known to be closely related to singularities, which exist in parameter spaces of hierarchical learning models. To show the influence of singular structure on learning dynamics, we take statistical mechanical approaches and investigate online-learning dynamics under various learning scenario with different relationship between optimum and singularities. From the investigation, we found a quasi-plateau phenomenon which differs from the well known plateau. The quasi-plateau and plateau become extremely serious when an optimal point is in a neighborhood of a singularity. The quasi-plateau and plateau disappear in the natural gradient learning, which takes singular structures into account and uses Riemannian measure for the parameter space.

本文言語English
ページ(範囲)275-279
ページ数5
ジャーナルProgress of Theoretical Physics Supplement
157
DOI
出版ステータスPublished - 2005
外部発表はい

ASJC Scopus subject areas

  • 物理学および天文学(その他)

フィンガープリント

「Slow dynamics due to singularities of hierarchical learning machines」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル