Sampling hidden parameters from oracle distribution

Sho Sonoda, Noboru Murata

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

A new sampling learning method for neural networks is proposed. Derived from an integral representation of neural networks, an oracle probability distribution of hidden parameters is introduced. In general rigorous sampling from the oracle distribution holds numerical difficulty, a linear-time sampling algorithm is also developed. Numerical experiments showed that when hidden parameters were initialized by the oracle distribution, following backpropagation converged faster to better parameters than when parameters were initialized by a normal distribution.

Original languageEnglish
Title of host publicationArtificial Neural Networks and Machine Learning, ICANN 2014 - 24th International Conference on Artificial Neural Networks, Proceedings
PublisherSpringer Verlag
Pages539-546
Number of pages8
ISBN (Print)9783319111780
DOIs
Publication statusPublished - 2014
Event24th International Conference on Artificial Neural Networks, ICANN 2014 - Hamburg, Germany
Duration: 2014 Sep 152014 Sep 19

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume8681 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other24th International Conference on Artificial Neural Networks, ICANN 2014
Country/TerritoryGermany
CityHamburg
Period14/9/1514/9/19

Keywords

  • Integral representation
  • backpropagation
  • neural networks
  • oracle distribution
  • sampling learning
  • weight initialization

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint

Dive into the research topics of 'Sampling hidden parameters from oracle distribution'. Together they form a unique fingerprint.

Cite this