Statistical analysis of regularization constant from bayes, MDL and NIC Points of view

Shun Ichi Amari, Noboru Murata

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

In order to avoid overfitting in neural learning, a regularization term is added to the loss function to be minimized. It is naturMly derived from the Bayesian standpoint. The present paper studies how to determine the regularization constant from the points of view of the empirical Bayes approach, the maximum description length (MDL) approach, and the network information criterion (NIC) approach. The asymptotic statistical analysis is given to elucidate their differences. These approaches are tightly connected with the method of model selection. The superiority of the NIC is shown from this analysis.

Original languageEnglish
Title of host publicationBiological and Artificial Computation
Subtitle of host publicationFrom Neuroscience to Technology - International Work-Conference on Artificial and Natural Neural Networks, IWANN 1997, Proceedings
PublisherSpringer Verlag
Pages284-293
Number of pages10
ISBN (Print)3540630473, 9783540630470
Publication statusPublished - 1997 Jan 1
Externally publishedYes
Event4th International Work-Conference on Artificial and Natural Neural Networks, IWANN 1997 - Lanzarote, Canary Islands, Spain
Duration: 1997 Jun 41997 Jun 6

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume1240 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference4th International Work-Conference on Artificial and Natural Neural Networks, IWANN 1997
CountrySpain
CityLanzarote, Canary Islands
Period97/6/497/6/6

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Statistical analysis of regularization constant from bayes, MDL and NIC Points of view'. Together they form a unique fingerprint.

  • Cite this

    Amari, S. I., & Murata, N. (1997). Statistical analysis of regularization constant from bayes, MDL and NIC Points of view. In Biological and Artificial Computation: From Neuroscience to Technology - International Work-Conference on Artificial and Natural Neural Networks, IWANN 1997, Proceedings (pp. 284-293). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 1240 LNCS). Springer Verlag.