{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T04:30:50Z","timestamp":1760243450702,"version":"build-2065373602"},"reference-count":41,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2013,5,6]],"date-time":"2013-05-06T00:00:00Z","timestamp":1367798400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/3.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>The use of a metric to assess distance between probability densities is an important practical problem. In this work, a particular metric induced by an \u03b1-divergence is studied. The Hellinger metric can be interpreted as a particular case within the framework of generalized Tsallis divergences and entropies. The nonparametric Parzen\u2019s density estimator emerges as a natural candidate to estimate the underlying probability density function, since it may account for data from different groups, or experiments with distinct instrumental precisions, i.e., non-independent and identically distributed (non-i.i.d.) data. However, the information theoretic derived metric of the nonparametric Parzen\u2019s density estimator displays infinite variance, limiting the direct use of resampling estimators. Based on measure theory, we present a change of measure to build a finite variance density allowing the use of resampling estimators. In order to counteract the poor scaling with dimension, we propose a new nonparametric two-stage robust resampling estimator of Hellinger\u2019s metric error bounds for heterocedastic data. The approach presents very promising results allowing the use of different covariances for different clusters with impact on the distance evaluation.<\/jats:p>","DOI":"10.3390\/e15051609","type":"journal-article","created":{"date-parts":[[2013,5,6]],"date-time":"2013-05-06T12:49:36Z","timestamp":1367844576000},"page":"1609-1623","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["A Novel Nonparametric Distance Estimator for Densities with Error Bounds"],"prefix":"10.3390","volume":"15","author":[{"given":"Alexandre","family":"Carvalho","sequence":"first","affiliation":[{"name":"Instituto de Engenharia Mec\u00e2nica e Gest\u00e3o Industrial, Faculdade de Engenharia, Universidade do Porto, Rua Dr. Roberto Frias s\/n, 4200-465 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7603-6526","authenticated-orcid":false,"given":"Jo\u00e3o","family":"Tavares","sequence":"additional","affiliation":[{"name":"Instituto de Engenharia Mec\u00e2nica e Gest\u00e3o Industrial, Faculdade de Engenharia, Universidade do Porto, Rua Dr. Roberto Frias s\/n, 4200-465 Porto, Portugal"}]},{"given":"Jose","family":"Principe","sequence":"additional","affiliation":[{"name":"Computational Neuro Engineering Laboratory, University of Florida, EB451 Engineering Building, University of Florida, Gainesville, FL 32611, USA"}]}],"member":"1968","published-online":{"date-parts":[[2013,5,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Hellinger, E. (1909). Neue Begr\u00fcndung der Theorie quadratischer Formen von unendlichvielen Ver\u00e4nderlichen. Crelle, 210\u2013271.","DOI":"10.1515\/crll.1909.136.210"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1111\/j.2517-6161.1966.tb00626.x","article-title":"A general class of coefficients of divergence of one distribution from another","volume":"28","author":"Ali","year":"1966","journal-title":"J. R. Stat. Soc. Series B."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1016\/0378-3758(95)00034-8","article-title":"Entropy, divergence and distance measures with econometric applications","volume":"49","author":"Ullah","year":"1996","journal-title":"J. Stat. Plan Inferace"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Principe, J.C. (2010). Information Theoretic Learning Renyi's Entropy and Kernel Perspectives, Springer.","DOI":"10.1007\/978-1-4419-1570-2"},{"key":"ref_5","unstructured":"Pardo, L. (2005). Statistical Inference Based on Divergence Measures, Chapman and Hall\/CRC."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1214\/aoms\/1177729694","article-title":"On Information and Sufficiency","volume":"22","author":"Kullback","year":"1951","journal-title":"Ann. Math. Stat."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"200","DOI":"10.1016\/0041-5553(67)90040-7","article-title":"The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming","volume":"7","author":"Bregman","year":"1967","journal-title":"USSR Comput. Math. Math. Phys."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1","DOI":"10.2307\/1402679","article-title":"Fisher and inverse probability","volume":"42","author":"Jeffreys","year":"1974","journal-title":"Int. Stat. Rev."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"589","DOI":"10.1109\/TIT.1985.1057082","article-title":"Cross entropy, dissimilarity measures, and characterizations of quadratic entropy","volume":"31","author":"Rao","year":"1985","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1109\/18.61115","article-title":"Divergence measures based on the Shannon Entropy","volume":"37","author":"Lin","year":"1991","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1023\/A:1022214326758","article-title":"(h, \u03a6)-entropy differential metric","volume":"42","author":"Morales","year":"1997","journal-title":"Appl. Math."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Seth, S., and Principe, J.C. (2008, January 31\u2013April). Compressed signal reconstruction using the correntropy induced metric. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, Las Vegas, NV, USA.","DOI":"10.1109\/ICASSP.2008.4518492"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1602","DOI":"10.1109\/18.850703","article-title":"Some inequalities for information divergence and Related measures of discrimination","volume":"46","author":"Topsoe","year":"2000","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"4394","DOI":"10.1109\/TIT.2006.881731","article-title":"On divergences and informations in statistics and information theory","volume":"52","author":"Liese","year":"2006","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/S0378-3758(02)00267-7","article-title":"Hypothesis testing: A model selection approach","volume":"108","author":"Cubedo","year":"2002","journal-title":"J. Stat. Plan. Inference"},{"key":"ref_16","unstructured":"Puga, A.T. (2007, January 13\u201315). Non-parametric Hellinger\u2019s Metric. Proceedings of CMNE\/CILANCE 2007, Porto, Portugal."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"535","DOI":"10.1002\/j.1538-7305.1928.tb01236.x","article-title":"Transmission of information","volume":"7","author":"Hartley","year":"1928","journal-title":"Bell Syst. Tech. J."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J."},{"key":"ref_19","unstructured":"R\u00e9nyi, A. (1961). On Measures of Entropy and Information, Fourth Berkeley Symposium on Math. Statist. and Prob, University of California."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Gell-Mann, M., and Tsallis, C. (2004). Nonextensive Entropy: Interdisciplinary Applications, Oxford University Press.","DOI":"10.1093\/oso\/9780195159769.001.0001"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2433","DOI":"10.1023\/A:1026627327731","article-title":"Two-state paramagnetism induced by Tsallis and Renyi statistics","volume":"37","author":"Wolf","year":"1998","journal-title":"Int. J. .Theor. Phys."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"158","DOI":"10.1109\/34.982897","article-title":"Information theoretic clustering","volume":"24","author":"Gokcay","year":"2002","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1016\/0375-9601(95)00035-2","article-title":"Thermodynamic stability conditions for the Tsallis and Renyi entropies","volume":"198","author":"Ramshaw","year":"1995","journal-title":"Phys. Lett. A"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1111\/j.1751-5823.2002.tb00178.x","article-title":"On choosing and bounding probability metrics","volume":"70","author":"Gibbs","year":"2002","journal-title":"Int. Stat. Rev."},{"key":"ref_25","unstructured":"Silverman, B.W. (1986). Density Estimation for Statistics and Data Analysis, Chapman and Hall."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Scott, D.W. (1992). Multivariate Density Estimation: Theory, Practice, and Visualization, Wiley.","DOI":"10.1002\/9780470316849"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Devroye, L. (1986). Non-Uniform Random Variate Generation, Springer-Verlag.","DOI":"10.1007\/978-1-4613-8643-8"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"335","DOI":"10.1007\/BF02481032","article-title":"The student distribution and the principle of maximum-entropy","volume":"34","author":"Preda","year":"1982","journal-title":"Ann. Inst. Stat. Math."},{"key":"ref_29","unstructured":"Kapur, J.N. (1989). Maximum-Entropy Models in Science and Engineering, Wiley."},{"key":"ref_30","unstructured":"The Probable Error of a Mean. Available online: http:\/\/www.jstor.org\/discover\/10.2307\/2331554? uid=2&uid=4&sid=21102107492741\/."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"2153","DOI":"10.1214\/07-AOS539","article-title":"A class of Renyi information estimators for multidimensional densities","volume":"36","author":"Leonenko","year":"2008","journal-title":"Ann. Stat."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"650","DOI":"10.3390\/e13030650","article-title":"k-nearest neighbor based consistent entropy estimation for hyperspherical distributions","volume":"13","author":"Li","year":"2011","journal-title":"Entropy"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Wells, M.T., and SenGupta, A. (2011). Advances in Directional and Linear Statistics, Physica-Verlag.","DOI":"10.1007\/978-3-7908-2628-9"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1125","DOI":"10.3390\/e12051125","article-title":"Nearest neighbor estimates of entropy for multivariate circular distributions","volume":"12","author":"Misra","year":"2010","journal-title":"Entropy"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"261","DOI":"10.3103\/S106653070803006X","article-title":"k-Nearest neighbor estimators of entropy","volume":"17","author":"Mnatsakanov","year":"2008","journal-title":"Math. Method. Stat."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"2392","DOI":"10.1109\/TIT.2009.2016060","article-title":"Divergence estimation for multidimensional densities via k-nearest-neighbor distances","volume":"55","author":"Wang","year":"2009","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"2135","DOI":"10.1214\/07-AOS537","article-title":"Choice of neighbor order in nearest-neighbor classification","volume":"36","author":"Hall","year":"2008","journal-title":"Ann. Stat."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"2412","DOI":"10.1021\/ci060149f","article-title":"Melting point prediction employing k-nearest neighbor algorithms and genetic parameter optimization","volume":"46","author":"Nigsch","year":"2006","journal-title":"J. Chem. Inf. Model."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Beyer, K., Goldstein, J., Ramakrishnan, R., and Shaft, U. (1999, January 12). When Is \u201cNearest Neighbor\u201d Meaningful?. Proceedings of 7the International Conference on Database Theory, Jerusalem, Israel.","DOI":"10.1007\/3-540-49257-7_15"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"475","DOI":"10.1109\/TMI.2010.2086464","article-title":"Total bregman divergence and its applications to DTI analysis","volume":"30","author":"Vemuri","year":"2011","journal-title":"IEEE Trans. Med. Imag."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"2407","DOI":"10.1109\/TPAMI.2012.44","article-title":"Shape retrieval using hierarchical total bregman soft clustering","volume":"34","author":"Liu","year":"2012","journal-title":"IEEE T. Pattern Anal."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/15\/5\/1609\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T21:46:33Z","timestamp":1760219193000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/15\/5\/1609"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2013,5,6]]},"references-count":41,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2013,5]]}},"alternative-id":["e15051609"],"URL":"https:\/\/doi.org\/10.3390\/e15051609","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2013,5,6]]}}}