{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,14]],"date-time":"2026-02-14T06:46:35Z","timestamp":1771051595121,"version":"3.50.1"},"reference-count":23,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2022,5,12]],"date-time":"2022-05-12T00:00:00Z","timestamp":1652313600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Shannon\u2019s entropy is one of the building blocks of information theory and an essential aspect of Machine Learning (ML) methods (e.g., Random Forests). Yet, it is only finitely defined for distributions with fast decaying tails on a countable alphabet. The unboundedness of Shannon\u2019s entropy over the general class of all distributions on an alphabet prevents its potential utility from being fully realized. To fill the void in the foundation of information theory, Zhang (2020) proposed generalized Shannon\u2019s entropy, which is finitely defined everywhere. The plug-in estimator, adopted in almost all entropy-based ML method packages, is one of the most popular approaches to estimating Shannon\u2019s entropy. The asymptotic distribution for Shannon\u2019s entropy\u2019s plug-in estimator was well studied in the existing literature. This paper studies the asymptotic properties for the plug-in estimator of generalized Shannon\u2019s entropy on countable alphabets. The developed asymptotic properties require no assumptions on the original distribution. The proposed asymptotic properties allow for interval estimation and statistical tests with generalized Shannon\u2019s entropy.<\/jats:p>","DOI":"10.3390\/e24050683","type":"journal-article","created":{"date-parts":[[2022,5,12]],"date-time":"2022-05-12T21:46:53Z","timestamp":1652392013000},"page":"683","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Asymptotic Normality for Plug-In Estimators of Generalized Shannon\u2019s Entropy"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7527-758X","authenticated-orcid":false,"given":"Jialin","family":"Zhang","sequence":"first","affiliation":[{"name":"Department of Mathematics and Statistics, Mississippi State University, Mississippi State, MS 39762, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jingyi","family":"Shi","sequence":"additional","affiliation":[{"name":"Department of Mathematics and Statistics, Mississippi State University, Mississippi State, MS 39762, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,5,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J."},{"key":"ref_2","first-page":"1","article-title":"Feature selection: A data perspective","volume":"50","author":"Li","year":"2017","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"e004879","DOI":"10.1161\/CIRCOUTCOMES.118.004879","article-title":"Tree-based analysis: A practical approach to create clinical decision-making tools","volume":"12","author":"Banerjee","year":"2019","journal-title":"Circ. Cardiovasc. Qual. Outcomes"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"698","DOI":"10.1016\/j.promfg.2019.06.011","article-title":"Prediction performance of improved decision tree-based algorithms: A review","volume":"35","author":"Mienye","year":"2019","journal-title":"Procedia Manuf."},{"key":"ref_5","first-page":"13","article-title":"A comparative study of decision tree ID3 and C4. 5","volume":"4","author":"Hssina","year":"2014","journal-title":"Int. J. Adv. Comput. Sci. Appl."},{"key":"ref_6","unstructured":"Miller, G.A., and Madow, W.G. (1954). On the Maximum Likelihood Estimate of the Shannon-Weiner Measure of Information, Operational Applications Laboratory, Air Force Cambridge Research Center."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Harris, B. (1975). The Statistical Estimation of Entropy in the Non-Parametric Case, Wisconsin Univ-Madison Mathematics Research Center. Technical Report.","DOI":"10.21236\/ADA020217"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"905","DOI":"10.1214\/aos\/1176346256","article-title":"A normal limit law for a nonparametric estimator of the coverage of a random sample","volume":"11","author":"Esty","year":"1983","journal-title":"Ann. Stat."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1191","DOI":"10.1162\/089976603321780272","article-title":"Estimation of entropy and mutual information","volume":"15","author":"Paninski","year":"2003","journal-title":"Neural Comput."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1368","DOI":"10.1162\/NECO_a_00266","article-title":"Entropy estimation in Turing\u2019s perspective","volume":"24","author":"Zhang","year":"2012","journal-title":"Neural Comput."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2745","DOI":"10.1109\/TIT.2011.2179702","article-title":"A normal law for the plug-in estimator of entropy","volume":"58","author":"Zhang","year":"2012","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"504","DOI":"10.1109\/TIT.2012.2217393","article-title":"Asymptotic normality of an entropy estimator with exponentially decaying bias","volume":"59","author":"Zhang","year":"2013","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"P04010","DOI":"10.1088\/1742-5468\/2013\/04\/P04010","article-title":"Infinite shannon entropy","volume":"2013","author":"Baccetti","year":"2013","journal-title":"J. Stat. Mech. Theory Exp."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"158","DOI":"10.3390\/stats3020013","article-title":"Generalized Mutual Information","volume":"3","author":"Zhang","year":"2020","journal-title":"Stats"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Amig\u00f3, J.M., Balogh, S.G., and Hern\u00e1ndez, S. (2018). A brief review of generalized entropies. Entropy, 20.","DOI":"10.3390\/e20110813"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"261","DOI":"10.3390\/e10030261","article-title":"Axiomatic characterizations of information measures","volume":"10","year":"2008","journal-title":"Entropy"},{"key":"ref_17","unstructured":"Khinchin, A.Y. (2013). Mathematical Foundations of Information Theory, Courier Corporation."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2847","DOI":"10.1155\/IJMMS.2005.2847","article-title":"Shannon entropy: Axiomatic characterization and application","volume":"2005","author":"Chakrabarti","year":"2005","journal-title":"Int. J. Math. Math. Sci."},{"key":"ref_19","first-page":"547","article-title":"On measures of entropy and information","volume":"Volume 4","year":"1961","journal-title":"Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"479","DOI":"10.1007\/BF01016429","article-title":"Possible generalization of Boltzmann-Gibbs statistics","volume":"52","author":"Tsallis","year":"1988","journal-title":"J. Stat. Phys."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1214\/aoms\/1177729694","article-title":"On information and sufficiency","volume":"22","author":"Kullback","year":"1951","journal-title":"Ann. Math. Stat."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1080\/03610926.2018.1536786","article-title":"Estimation of population size in entropic perspective","volume":"49","author":"Zhang","year":"2020","journal-title":"Commun.-Stat.-Theory Methods"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"774","DOI":"10.1080\/10485252.2018.1482294","article-title":"Asymptotic normality for plug-in estimators of diversity indices on countable alphabets","volume":"30","author":"Grabchak","year":"2018","journal-title":"J. Nonparametr. Stat."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/5\/683\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:09:49Z","timestamp":1760137789000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/5\/683"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,12]]},"references-count":23,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2022,5]]}},"alternative-id":["e24050683"],"URL":"https:\/\/doi.org\/10.3390\/e24050683","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,5,12]]}}}