Statistical analysis of learning dynamics

Noboru Murata, Shun Ichi Amari

Research output: Contribution to journalArticle

26 Citations (Scopus)

Abstract

Learning is a flexible and effective means of extracting the stochastic structure of the environment. It provides an effective method for blind separation and deconvolution in signal processing. Two different types of learning are used, namely batch learning and on-line learning. The batch learning procedure uses all the training examples repeatedly so that its performance is compared to the statistical estimation procedure. On-line learning is more dynamical, updating the current estimate by observing a new datum one by one. On-line learning is slow in general but works well in the changing environment. The present paper gives a unified framework of statistical analysis for batch and on-line learning. The topics include the asymptotic learning curve, generalization error and training error, over-fitting and over-training, efficiency of learning, and an adaptive method of determining learning rate.

Original languageEnglish
Pages (from-to)3-28
Number of pages26
JournalSignal Processing
Volume74
Issue number1
DOIs
Publication statusPublished - 1999 Apr
Externally publishedYes

Fingerprint

Statistical methods
Deconvolution
Signal processing

ASJC Scopus subject areas

  • Signal Processing
  • Electrical and Electronic Engineering

Cite this

Statistical analysis of learning dynamics. / Murata, Noboru; Amari, Shun Ichi.

In: Signal Processing, Vol. 74, No. 1, 04.1999, p. 3-28.

Research output: Contribution to journalArticle

Murata, Noboru ; Amari, Shun Ichi. / Statistical analysis of learning dynamics. In: Signal Processing. 1999 ; Vol. 74, No. 1. pp. 3-28.
@article{204a90ae28f040ae948cdbf1b6642078,
title = "Statistical analysis of learning dynamics",
abstract = "Learning is a flexible and effective means of extracting the stochastic structure of the environment. It provides an effective method for blind separation and deconvolution in signal processing. Two different types of learning are used, namely batch learning and on-line learning. The batch learning procedure uses all the training examples repeatedly so that its performance is compared to the statistical estimation procedure. On-line learning is more dynamical, updating the current estimate by observing a new datum one by one. On-line learning is slow in general but works well in the changing environment. The present paper gives a unified framework of statistical analysis for batch and on-line learning. The topics include the asymptotic learning curve, generalization error and training error, over-fitting and over-training, efficiency of learning, and an adaptive method of determining learning rate.",
author = "Noboru Murata and Amari, {Shun Ichi}",
year = "1999",
month = "4",
doi = "10.1016/S0165-1684(98)00206-0",
language = "English",
volume = "74",
pages = "3--28",
journal = "Signal Processing",
issn = "0165-1684",
publisher = "Elsevier",
number = "1",

}

TY - JOUR

T1 - Statistical analysis of learning dynamics

AU - Murata, Noboru

AU - Amari, Shun Ichi

PY - 1999/4

Y1 - 1999/4

N2 - Learning is a flexible and effective means of extracting the stochastic structure of the environment. It provides an effective method for blind separation and deconvolution in signal processing. Two different types of learning are used, namely batch learning and on-line learning. The batch learning procedure uses all the training examples repeatedly so that its performance is compared to the statistical estimation procedure. On-line learning is more dynamical, updating the current estimate by observing a new datum one by one. On-line learning is slow in general but works well in the changing environment. The present paper gives a unified framework of statistical analysis for batch and on-line learning. The topics include the asymptotic learning curve, generalization error and training error, over-fitting and over-training, efficiency of learning, and an adaptive method of determining learning rate.

AB - Learning is a flexible and effective means of extracting the stochastic structure of the environment. It provides an effective method for blind separation and deconvolution in signal processing. Two different types of learning are used, namely batch learning and on-line learning. The batch learning procedure uses all the training examples repeatedly so that its performance is compared to the statistical estimation procedure. On-line learning is more dynamical, updating the current estimate by observing a new datum one by one. On-line learning is slow in general but works well in the changing environment. The present paper gives a unified framework of statistical analysis for batch and on-line learning. The topics include the asymptotic learning curve, generalization error and training error, over-fitting and over-training, efficiency of learning, and an adaptive method of determining learning rate.

UR - http://www.scopus.com/inward/record.url?scp=0032629928&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0032629928&partnerID=8YFLogxK

U2 - 10.1016/S0165-1684(98)00206-0

DO - 10.1016/S0165-1684(98)00206-0

M3 - Article

AN - SCOPUS:0032629928

VL - 74

SP - 3

EP - 28

JO - Signal Processing

JF - Signal Processing

SN - 0165-1684

IS - 1

ER -