### Abstract

Abstract Estimators for differential entropy are proposed. The estimators are based on the second order expansion of the probability mass around the inspection point with respect to the distance from the point. Simple linear regression is utilized to estimate the values of density function and its second derivative at a point. After estimating the values of the probability density function at each of the given sample points, by taking the empirical average of the negative logarithm of the density estimates, two entropy estimators are derived. Other entropy estimators which directly estimate entropy by linear regression, are also proposed. The proposed four estimators are shown to perform well through numerical experiments for various probability distributions.

Original language | English |
---|---|

Article number | 6063 |

Pages (from-to) | 72-84 |

Number of pages | 13 |

Journal | Computational Statistics and Data Analysis |

Volume | 89 |

DOIs | |

Publication status | Published - 2015 |

### Fingerprint

### Keywords

- Entropy estimation
- Non-parametric
- Simple linear regression

### ASJC Scopus subject areas

- Computational Mathematics
- Computational Theory and Mathematics
- Statistics and Probability
- Applied Mathematics

### Cite this

*Computational Statistics and Data Analysis*,

*89*, 72-84. [6063]. https://doi.org/10.1016/j.csda.2015.03.011

**Non-parametric entropy estimators based on simple linear regression.** / Hino, Hideitsu; Koshijima, Kensuke; Murata, Noboru.

Research output: Contribution to journal › Article

*Computational Statistics and Data Analysis*, vol. 89, 6063, pp. 72-84. https://doi.org/10.1016/j.csda.2015.03.011

}

TY - JOUR

T1 - Non-parametric entropy estimators based on simple linear regression

AU - Hino, Hideitsu

AU - Koshijima, Kensuke

AU - Murata, Noboru

PY - 2015

Y1 - 2015

N2 - Abstract Estimators for differential entropy are proposed. The estimators are based on the second order expansion of the probability mass around the inspection point with respect to the distance from the point. Simple linear regression is utilized to estimate the values of density function and its second derivative at a point. After estimating the values of the probability density function at each of the given sample points, by taking the empirical average of the negative logarithm of the density estimates, two entropy estimators are derived. Other entropy estimators which directly estimate entropy by linear regression, are also proposed. The proposed four estimators are shown to perform well through numerical experiments for various probability distributions.

AB - Abstract Estimators for differential entropy are proposed. The estimators are based on the second order expansion of the probability mass around the inspection point with respect to the distance from the point. Simple linear regression is utilized to estimate the values of density function and its second derivative at a point. After estimating the values of the probability density function at each of the given sample points, by taking the empirical average of the negative logarithm of the density estimates, two entropy estimators are derived. Other entropy estimators which directly estimate entropy by linear regression, are also proposed. The proposed four estimators are shown to perform well through numerical experiments for various probability distributions.

KW - Entropy estimation

KW - Non-parametric

KW - Simple linear regression

UR - http://www.scopus.com/inward/record.url?scp=84926286540&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84926286540&partnerID=8YFLogxK

U2 - 10.1016/j.csda.2015.03.011

DO - 10.1016/j.csda.2015.03.011

M3 - Article

VL - 89

SP - 72

EP - 84

JO - Computational Statistics and Data Analysis

JF - Computational Statistics and Data Analysis

SN - 0167-9473

M1 - 6063

ER -