TY - JOUR
T1 - Non-parametric entropy estimators based on simple linear regression
AU - Hino, Hideitsu
AU - Koshijima, Kensuke
AU - Murata, Noboru
PY - 2015
Y1 - 2015
N2 - Abstract Estimators for differential entropy are proposed. The estimators are based on the second order expansion of the probability mass around the inspection point with respect to the distance from the point. Simple linear regression is utilized to estimate the values of density function and its second derivative at a point. After estimating the values of the probability density function at each of the given sample points, by taking the empirical average of the negative logarithm of the density estimates, two entropy estimators are derived. Other entropy estimators which directly estimate entropy by linear regression, are also proposed. The proposed four estimators are shown to perform well through numerical experiments for various probability distributions.
AB - Abstract Estimators for differential entropy are proposed. The estimators are based on the second order expansion of the probability mass around the inspection point with respect to the distance from the point. Simple linear regression is utilized to estimate the values of density function and its second derivative at a point. After estimating the values of the probability density function at each of the given sample points, by taking the empirical average of the negative logarithm of the density estimates, two entropy estimators are derived. Other entropy estimators which directly estimate entropy by linear regression, are also proposed. The proposed four estimators are shown to perform well through numerical experiments for various probability distributions.
KW - Entropy estimation
KW - Non-parametric
KW - Simple linear regression
UR - http://www.scopus.com/inward/record.url?scp=84926286540&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84926286540&partnerID=8YFLogxK
U2 - 10.1016/j.csda.2015.03.011
DO - 10.1016/j.csda.2015.03.011
M3 - Article
AN - SCOPUS:84926286540
VL - 89
SP - 72
EP - 84
JO - Computational Statistics and Data Analysis
JF - Computational Statistics and Data Analysis
SN - 0167-9473
M1 - 6063
ER -