Network information criterion - determining the number of hidden units for an artificial neural network model

Noboru Murata, Shuji Yoshizawa, Shun ichi Amari

Research output: Contribution to journalArticle

395 Citations (Scopus)

Abstract

The problem of model selection, or determination of the number of hidden units, can be approached statistically, by generalizing Akaike's information criterion (AIC) to be applicable to unfaithful (i.e., unrealizable) models with general loss criteria including regularization terms. The relation between the training error and the generalization error is studied in terms of the number of the training examples and the complexity of a network which reduces to the number of parameters in the ordinary statistical theory of the AIC. This relation leads to a new Network Information Criterion (NIC) which is useful for selecting the optimal network model based on a given training set.

Original languageEnglish
Pages (from-to)865-872
Number of pages8
JournalIEEE Transactions on Neural Networks
Volume5
Issue number6
DOIs
Publication statusPublished - 1994 Nov
Externally publishedYes

Fingerprint

Information Criterion
Neural Network Model
Artificial Neural Network
Akaike Information Criterion
Neural networks
Unit
Generalization Error
Model Selection
Network Model
Regularization
Model-based
Term
Training
Model

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computational Theory and Mathematics
  • Hardware and Architecture
  • Control and Systems Engineering
  • Electrical and Electronic Engineering
  • Theoretical Computer Science

Cite this

Network information criterion - determining the number of hidden units for an artificial neural network model. / Murata, Noboru; Yoshizawa, Shuji; Amari, Shun ichi.

In: IEEE Transactions on Neural Networks, Vol. 5, No. 6, 11.1994, p. 865-872.

Research output: Contribution to journalArticle

@article{091edff2e73846739c93e5ac2847bab0,
title = "Network information criterion - determining the number of hidden units for an artificial neural network model",
abstract = "The problem of model selection, or determination of the number of hidden units, can be approached statistically, by generalizing Akaike's information criterion (AIC) to be applicable to unfaithful (i.e., unrealizable) models with general loss criteria including regularization terms. The relation between the training error and the generalization error is studied in terms of the number of the training examples and the complexity of a network which reduces to the number of parameters in the ordinary statistical theory of the AIC. This relation leads to a new Network Information Criterion (NIC) which is useful for selecting the optimal network model based on a given training set.",
author = "Noboru Murata and Shuji Yoshizawa and Amari, {Shun ichi}",
year = "1994",
month = "11",
doi = "10.1109/72.329683",
language = "English",
volume = "5",
pages = "865--872",
journal = "IEEE Transactions on Neural Networks and Learning Systems",
issn = "2162-237X",
publisher = "IEEE Computational Intelligence Society",
number = "6",

}

TY - JOUR

T1 - Network information criterion - determining the number of hidden units for an artificial neural network model

AU - Murata, Noboru

AU - Yoshizawa, Shuji

AU - Amari, Shun ichi

PY - 1994/11

Y1 - 1994/11

N2 - The problem of model selection, or determination of the number of hidden units, can be approached statistically, by generalizing Akaike's information criterion (AIC) to be applicable to unfaithful (i.e., unrealizable) models with general loss criteria including regularization terms. The relation between the training error and the generalization error is studied in terms of the number of the training examples and the complexity of a network which reduces to the number of parameters in the ordinary statistical theory of the AIC. This relation leads to a new Network Information Criterion (NIC) which is useful for selecting the optimal network model based on a given training set.

AB - The problem of model selection, or determination of the number of hidden units, can be approached statistically, by generalizing Akaike's information criterion (AIC) to be applicable to unfaithful (i.e., unrealizable) models with general loss criteria including regularization terms. The relation between the training error and the generalization error is studied in terms of the number of the training examples and the complexity of a network which reduces to the number of parameters in the ordinary statistical theory of the AIC. This relation leads to a new Network Information Criterion (NIC) which is useful for selecting the optimal network model based on a given training set.

UR - http://www.scopus.com/inward/record.url?scp=0028544395&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0028544395&partnerID=8YFLogxK

U2 - 10.1109/72.329683

DO - 10.1109/72.329683

M3 - Article

AN - SCOPUS:0028544395

VL - 5

SP - 865

EP - 872

JO - IEEE Transactions on Neural Networks and Learning Systems

JF - IEEE Transactions on Neural Networks and Learning Systems

SN - 2162-237X

IS - 6

ER -