### Abstract

A method for estimating Shannon differential entropy is proposed based on the second order expansion of the probability mass around the inspection point with respect to the distance from the point. Polynomial regression with Poisson error structure is utilized to estimate the values of density function. The density estimates at every given data points are averaged to obtain entropy estimators. The proposed estimator is shown to perform well through numerical experiments for various probability distributions.

Original language | English |
---|---|

Title of host publication | Neural Information Processing - 23rd International Conference, ICONIP 2016, Proceedings |

Publisher | Springer Verlag |

Pages | 11-19 |

Number of pages | 9 |

Volume | 9948 LNCS |

ISBN (Print) | 9783319466712 |

DOIs | |

Publication status | Published - 2016 |

Event | 23rd International Conference on Neural Information Processing, ICONIP 2016 - Kyoto, Japan Duration: 2016 Oct 16 → 2016 Oct 21 |

### Publication series

Name | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
---|---|

Volume | 9948 LNCS |

ISSN (Print) | 03029743 |

ISSN (Electronic) | 16113349 |

### Other

Other | 23rd International Conference on Neural Information Processing, ICONIP 2016 |
---|---|

Country | Japan |

City | Kyoto |

Period | 16/10/16 → 16/10/21 |

### Fingerprint

### Keywords

- Density estimation
- Entropy
- Poisson error structure
- Regression

### ASJC Scopus subject areas

- Theoretical Computer Science
- Computer Science(all)

### Cite this

*Neural Information Processing - 23rd International Conference, ICONIP 2016, Proceedings*(Vol. 9948 LNCS, pp. 11-19). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9948 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-46672-9_2

**An entropy estimator based on polynomial regression with poisson error structure.** / Hino, Hideitsu; Akaho, Shotaro; Murata, Noboru.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

*Neural Information Processing - 23rd International Conference, ICONIP 2016, Proceedings.*vol. 9948 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9948 LNCS, Springer Verlag, pp. 11-19, 23rd International Conference on Neural Information Processing, ICONIP 2016, Kyoto, Japan, 16/10/16. https://doi.org/10.1007/978-3-319-46672-9_2

}

TY - GEN

T1 - An entropy estimator based on polynomial regression with poisson error structure

AU - Hino, Hideitsu

AU - Akaho, Shotaro

AU - Murata, Noboru

PY - 2016

Y1 - 2016

N2 - A method for estimating Shannon differential entropy is proposed based on the second order expansion of the probability mass around the inspection point with respect to the distance from the point. Polynomial regression with Poisson error structure is utilized to estimate the values of density function. The density estimates at every given data points are averaged to obtain entropy estimators. The proposed estimator is shown to perform well through numerical experiments for various probability distributions.

AB - A method for estimating Shannon differential entropy is proposed based on the second order expansion of the probability mass around the inspection point with respect to the distance from the point. Polynomial regression with Poisson error structure is utilized to estimate the values of density function. The density estimates at every given data points are averaged to obtain entropy estimators. The proposed estimator is shown to perform well through numerical experiments for various probability distributions.

KW - Density estimation

KW - Entropy

KW - Poisson error structure

KW - Regression

UR - http://www.scopus.com/inward/record.url?scp=84992533118&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84992533118&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-46672-9_2

DO - 10.1007/978-3-319-46672-9_2

M3 - Conference contribution

AN - SCOPUS:84992533118

SN - 9783319466712

VL - 9948 LNCS

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 11

EP - 19

BT - Neural Information Processing - 23rd International Conference, ICONIP 2016, Proceedings

PB - Springer Verlag

ER -