An entropy estimator based on polynomial regression with poisson error structure

Hideitsu Hino, Shotaro Akaho, Noboru Murata

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    A method for estimating Shannon differential entropy is proposed based on the second order expansion of the probability mass around the inspection point with respect to the distance from the point. Polynomial regression with Poisson error structure is utilized to estimate the values of density function. The density estimates at every given data points are averaged to obtain entropy estimators. The proposed estimator is shown to perform well through numerical experiments for various probability distributions.

    Original languageEnglish
    Title of host publicationNeural Information Processing - 23rd International Conference, ICONIP 2016, Proceedings
    PublisherSpringer Verlag
    Pages11-19
    Number of pages9
    Volume9948 LNCS
    ISBN (Print)9783319466712
    DOIs
    Publication statusPublished - 2016
    Event23rd International Conference on Neural Information Processing, ICONIP 2016 - Kyoto, Japan
    Duration: 2016 Oct 162016 Oct 21

    Publication series

    NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    Volume9948 LNCS
    ISSN (Print)03029743
    ISSN (Electronic)16113349

    Other

    Other23rd International Conference on Neural Information Processing, ICONIP 2016
    CountryJapan
    CityKyoto
    Period16/10/1616/10/21

    Fingerprint

    Polynomial Regression
    Siméon Denis Poisson
    Entropy
    Polynomials
    Estimator
    Probability distributions
    Probability density function
    Density Estimates
    Inspection
    Density Function
    Probability Distribution
    Numerical Experiment
    Experiments
    Estimate

    Keywords

    • Density estimation
    • Entropy
    • Poisson error structure
    • Regression

    ASJC Scopus subject areas

    • Theoretical Computer Science
    • Computer Science(all)

    Cite this

    Hino, H., Akaho, S., & Murata, N. (2016). An entropy estimator based on polynomial regression with poisson error structure. In Neural Information Processing - 23rd International Conference, ICONIP 2016, Proceedings (Vol. 9948 LNCS, pp. 11-19). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9948 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-46672-9_2

    An entropy estimator based on polynomial regression with poisson error structure. / Hino, Hideitsu; Akaho, Shotaro; Murata, Noboru.

    Neural Information Processing - 23rd International Conference, ICONIP 2016, Proceedings. Vol. 9948 LNCS Springer Verlag, 2016. p. 11-19 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9948 LNCS).

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Hino, H, Akaho, S & Murata, N 2016, An entropy estimator based on polynomial regression with poisson error structure. in Neural Information Processing - 23rd International Conference, ICONIP 2016, Proceedings. vol. 9948 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9948 LNCS, Springer Verlag, pp. 11-19, 23rd International Conference on Neural Information Processing, ICONIP 2016, Kyoto, Japan, 16/10/16. https://doi.org/10.1007/978-3-319-46672-9_2
    Hino H, Akaho S, Murata N. An entropy estimator based on polynomial regression with poisson error structure. In Neural Information Processing - 23rd International Conference, ICONIP 2016, Proceedings. Vol. 9948 LNCS. Springer Verlag. 2016. p. 11-19. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-46672-9_2
    Hino, Hideitsu ; Akaho, Shotaro ; Murata, Noboru. / An entropy estimator based on polynomial regression with poisson error structure. Neural Information Processing - 23rd International Conference, ICONIP 2016, Proceedings. Vol. 9948 LNCS Springer Verlag, 2016. pp. 11-19 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
    @inproceedings{a7f097a8164949e89b8c211b0546c851,
    title = "An entropy estimator based on polynomial regression with poisson error structure",
    abstract = "A method for estimating Shannon differential entropy is proposed based on the second order expansion of the probability mass around the inspection point with respect to the distance from the point. Polynomial regression with Poisson error structure is utilized to estimate the values of density function. The density estimates at every given data points are averaged to obtain entropy estimators. The proposed estimator is shown to perform well through numerical experiments for various probability distributions.",
    keywords = "Density estimation, Entropy, Poisson error structure, Regression",
    author = "Hideitsu Hino and Shotaro Akaho and Noboru Murata",
    year = "2016",
    doi = "10.1007/978-3-319-46672-9_2",
    language = "English",
    isbn = "9783319466712",
    volume = "9948 LNCS",
    series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
    publisher = "Springer Verlag",
    pages = "11--19",
    booktitle = "Neural Information Processing - 23rd International Conference, ICONIP 2016, Proceedings",
    address = "Germany",

    }

    TY - GEN

    T1 - An entropy estimator based on polynomial regression with poisson error structure

    AU - Hino, Hideitsu

    AU - Akaho, Shotaro

    AU - Murata, Noboru

    PY - 2016

    Y1 - 2016

    N2 - A method for estimating Shannon differential entropy is proposed based on the second order expansion of the probability mass around the inspection point with respect to the distance from the point. Polynomial regression with Poisson error structure is utilized to estimate the values of density function. The density estimates at every given data points are averaged to obtain entropy estimators. The proposed estimator is shown to perform well through numerical experiments for various probability distributions.

    AB - A method for estimating Shannon differential entropy is proposed based on the second order expansion of the probability mass around the inspection point with respect to the distance from the point. Polynomial regression with Poisson error structure is utilized to estimate the values of density function. The density estimates at every given data points are averaged to obtain entropy estimators. The proposed estimator is shown to perform well through numerical experiments for various probability distributions.

    KW - Density estimation

    KW - Entropy

    KW - Poisson error structure

    KW - Regression

    UR - http://www.scopus.com/inward/record.url?scp=84992533118&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=84992533118&partnerID=8YFLogxK

    U2 - 10.1007/978-3-319-46672-9_2

    DO - 10.1007/978-3-319-46672-9_2

    M3 - Conference contribution

    AN - SCOPUS:84992533118

    SN - 9783319466712

    VL - 9948 LNCS

    T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

    SP - 11

    EP - 19

    BT - Neural Information Processing - 23rd International Conference, ICONIP 2016, Proceedings

    PB - Springer Verlag

    ER -