Approximation bounds of three-layered neural networks-A theorem on an integral transform with ridge functions

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

Neural networks have attracted attention due to their capability to perform nonlinear function approximation. In this paper, in order to better understand this capability, a new theorem on an integral transform was derived by applying ridge functions to neural networks. From the theorem, it is possible to obtain approximation bounds which clarify the quantitative relationship between the function approximation accuracy and the number of nodes in the hidden layer. The theorem indicates that the approximation accuracy depends on the smoothness of the target function. Furthermore, the theorem also shows that this type of approximation method differs from usual methods and is able to escape the so-called "curse of dimensionality," in which the approximation accuracy depends strongly of the input dimension of the function and deteriorates exponentially.

Original languageEnglish
Pages (from-to)23-30
Number of pages8
JournalElectronics and Communications in Japan, Part III: Fundamental Electronic Science (English translation of Denshi Tsushin Gakkai Ronbunshi)
Volume79
Issue number3
Publication statusPublished - 1996 Mar
Externally publishedYes

Fingerprint

Neural networks

Keywords

  • Curse of dimensionality
  • Integral transform
  • Neural networks
  • Nonlinear function approximation
  • Ridge functions

ASJC Scopus subject areas

  • Electrical and Electronic Engineering

Cite this

@article{1cd288509a31490586c7e9123eaca895,
title = "Approximation bounds of three-layered neural networks-A theorem on an integral transform with ridge functions",
abstract = "Neural networks have attracted attention due to their capability to perform nonlinear function approximation. In this paper, in order to better understand this capability, a new theorem on an integral transform was derived by applying ridge functions to neural networks. From the theorem, it is possible to obtain approximation bounds which clarify the quantitative relationship between the function approximation accuracy and the number of nodes in the hidden layer. The theorem indicates that the approximation accuracy depends on the smoothness of the target function. Furthermore, the theorem also shows that this type of approximation method differs from usual methods and is able to escape the so-called {"}curse of dimensionality,{"} in which the approximation accuracy depends strongly of the input dimension of the function and deteriorates exponentially.",
keywords = "Curse of dimensionality, Integral transform, Neural networks, Nonlinear function approximation, Ridge functions",
author = "Noboru Murata",
year = "1996",
month = "3",
language = "English",
volume = "79",
pages = "23--30",
journal = "Electronics and Communications in Japan, Part III: Fundamental Electronic Science (English translation of Denshi Tsushin Gakkai Ronbunshi)",
issn = "1042-0967",
publisher = "John Wiley and Sons Inc.",
number = "3",

}

TY - JOUR

T1 - Approximation bounds of three-layered neural networks-A theorem on an integral transform with ridge functions

AU - Murata, Noboru

PY - 1996/3

Y1 - 1996/3

N2 - Neural networks have attracted attention due to their capability to perform nonlinear function approximation. In this paper, in order to better understand this capability, a new theorem on an integral transform was derived by applying ridge functions to neural networks. From the theorem, it is possible to obtain approximation bounds which clarify the quantitative relationship between the function approximation accuracy and the number of nodes in the hidden layer. The theorem indicates that the approximation accuracy depends on the smoothness of the target function. Furthermore, the theorem also shows that this type of approximation method differs from usual methods and is able to escape the so-called "curse of dimensionality," in which the approximation accuracy depends strongly of the input dimension of the function and deteriorates exponentially.

AB - Neural networks have attracted attention due to their capability to perform nonlinear function approximation. In this paper, in order to better understand this capability, a new theorem on an integral transform was derived by applying ridge functions to neural networks. From the theorem, it is possible to obtain approximation bounds which clarify the quantitative relationship between the function approximation accuracy and the number of nodes in the hidden layer. The theorem indicates that the approximation accuracy depends on the smoothness of the target function. Furthermore, the theorem also shows that this type of approximation method differs from usual methods and is able to escape the so-called "curse of dimensionality," in which the approximation accuracy depends strongly of the input dimension of the function and deteriorates exponentially.

KW - Curse of dimensionality

KW - Integral transform

KW - Neural networks

KW - Nonlinear function approximation

KW - Ridge functions

UR - http://www.scopus.com/inward/record.url?scp=0030102235&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0030102235&partnerID=8YFLogxK

M3 - Article

VL - 79

SP - 23

EP - 30

JO - Electronics and Communications in Japan, Part III: Fundamental Electronic Science (English translation of Denshi Tsushin Gakkai Ronbunshi)

JF - Electronics and Communications in Japan, Part III: Fundamental Electronic Science (English translation of Denshi Tsushin Gakkai Ronbunshi)

SN - 1042-0967

IS - 3

ER -