Statistical analysis of learning dynamics

Noboru Murata, Shun Ichi Amari

Research output: Contribution to journalArticle

26 Citations (Scopus)

Abstract

Learning is a flexible and effective means of extracting the stochastic structure of the environment. It provides an effective method for blind separation and deconvolution in signal processing. Two different types of learning are used, namely batch learning and on-line learning. The batch learning procedure uses all the training examples repeatedly so that its performance is compared to the statistical estimation procedure. On-line learning is more dynamical, updating the current estimate by observing a new datum one by one. On-line learning is slow in general but works well in the changing environment. The present paper gives a unified framework of statistical analysis for batch and on-line learning. The topics include the asymptotic learning curve, generalization error and training error, over-fitting and over-training, efficiency of learning, and an adaptive method of determining learning rate.

Original languageEnglish
Pages (from-to)3-28
Number of pages26
JournalSignal Processing
Volume74
Issue number1
DOIs
Publication statusPublished - 1999 Apr
Externally publishedYes

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Statistical analysis of learning dynamics'. Together they form a unique fingerprint.

  • Cite this