Sampling hidden parameters from oracle distribution

Sho Sonoda, Noboru Murata

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    3 Citations (Scopus)

    Abstract

    A new sampling learning method for neural networks is proposed. Derived from an integral representation of neural networks, an oracle probability distribution of hidden parameters is introduced. In general rigorous sampling from the oracle distribution holds numerical difficulty, a linear-time sampling algorithm is also developed. Numerical experiments showed that when hidden parameters were initialized by the oracle distribution, following backpropagation converged faster to better parameters than when parameters were initialized by a normal distribution.

    Original languageEnglish
    Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    PublisherSpringer Verlag
    Pages539-546
    Number of pages8
    Volume8681 LNCS
    ISBN (Print)9783319111780
    DOIs
    Publication statusPublished - 2014
    Event24th International Conference on Artificial Neural Networks, ICANN 2014 - Hamburg, Germany
    Duration: 2014 Sep 152014 Sep 19

    Publication series

    NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    Volume8681 LNCS
    ISSN (Print)03029743
    ISSN (Electronic)16113349

    Other

    Other24th International Conference on Artificial Neural Networks, ICANN 2014
    CountryGermany
    CityHamburg
    Period14/9/1514/9/19

    Keywords

    • backpropagation
    • Integral representation
    • neural networks
    • oracle distribution
    • sampling learning
    • weight initialization

    ASJC Scopus subject areas

    • Computer Science(all)
    • Theoretical Computer Science

    Fingerprint Dive into the research topics of 'Sampling hidden parameters from oracle distribution'. Together they form a unique fingerprint.

  • Cite this

    Sonoda, S., & Murata, N. (2014). Sampling hidden parameters from oracle distribution. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 8681 LNCS, pp. 539-546). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8681 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-11179-7_68