{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,6]],"date-time":"2026-04-06T10:24:17Z","timestamp":1775471057135,"version":"3.50.1"},"reference-count":38,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2023,7,13]],"date-time":"2023-07-13T00:00:00Z","timestamp":1689206400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Inspired by the development in modern data science, a shift is increasingly visible in the foundation of statistical inference, away from a real space, where random variables reside, toward a nonmetrized and nonordinal alphabet, where more general random elements reside. While statistical inferences based on random variables are theoretically well supported in the rich literature of probability and statistics, inferences on alphabets, mostly by way of various entropies and their estimation, are less systematically supported in theory. Without the familiar notions of neighborhood, real or complex moments, tails, et cetera, associated with random variables, probability and statistics based on random elements on alphabets need more attention to foster a sound framework for rigorous development of entropy-based statistical exercises. In this article, several basic elements of entropic statistics are introduced and discussed, including notions of general entropies, entropic sample spaces, entropic distributions, entropic statistics, entropic multinomial distributions, entropic moments, and entropic basis, among other entropic objects. In particular, an entropic-moment-generating function is defined and it is shown to uniquely characterize the underlying distribution in entropic perspective, and, hence, all entropies. An entropic version of the Glivenko\u2013Cantelli convergence theorem is also established.<\/jats:p>","DOI":"10.3390\/e25071060","type":"journal-article","created":{"date-parts":[[2023,7,14]],"date-time":"2023-07-14T00:28:06Z","timestamp":1689294486000},"page":"1060","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Several Basic Elements of Entropic Statistics"],"prefix":"10.3390","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1175-1259","authenticated-orcid":false,"given":"Zhiyi","family":"Zhang","sequence":"first","affiliation":[{"name":"Department of Mathematics and Statistics, UNC Charlotte, Charlotte, NC 28223, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,7,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J."},{"key":"ref_2","unstructured":"R\u00e9nyi, A. (1961, January 20\u201330). On measures of information and entropy. Proceedings of the Fourth Berkeley Symposium on Mathematics, Statistics and Probability, Berkeley, CA, USA."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"479","DOI":"10.1007\/BF01016429","article-title":"Possible generalization of Boltzmann-Gibbs statistics","volume":"52","author":"Tsallis","year":"1988","journal-title":"J. Stat. Phys."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"688","DOI":"10.1038\/163688a0","article-title":"Measurement of diversity","volume":"163","author":"Simpson","year":"1949","journal-title":"Nature"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1731","DOI":"10.1016\/j.jspi.2009.12.023","article-title":"Re-parameterization of multinomial distribution and diversity indices","volume":"140","author":"Zhang","year":"2010","journal-title":"J. Stat. Plan. Inference"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"427","DOI":"10.2307\/1934352","article-title":"Diversity and evenness: A unifying notation and its consequences","volume":"54","author":"Hill","year":"1973","journal-title":"Ecology"},{"key":"ref_7","unstructured":"Emlen, J.M. (1973). Ecology: An Evolutionary Approach, Addison-Wesley."},{"key":"ref_8","unstructured":"Miller, G.A., and Madow, W.G. (1954). On the Maximum Likelihood Estimate of the Shannon-Weaver Measure of Information, Operational Applications Laboratory, Air Force, Cambridge Research Center, Air Research and Development Command. Air Force Cambridge Research Center Technical Report AFCRC-TR-54-75."},{"key":"ref_9","first-page":"95","article-title":"Note on the bias of information estimates","volume":"11-B","author":"Miller","year":"1955","journal-title":"Inf. Theory Psychol. Probl. Methods"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Harris, B. (1975). The Statistical Estimation of Entropy in the Non-Parametric Case, Wisconsin University\u2014Madison Mathematics Research Center.","DOI":"10.21236\/ADA020217"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1002\/rsa.10019","article-title":"Convergence properties of functional estimates for discrete distributions","volume":"19","author":"Antos","year":"2001","journal-title":"Random Struct. Algorithms"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1191","DOI":"10.1162\/089976603321780272","article-title":"Estimation of entropy and mutual information","volume":"15","author":"Paninski","year":"2003","journal-title":"Neural Comput."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Silva, J.F. (2018). Shannon entropy estimation in \u221e-alphabets from convergence results: Studying plug-in estimators. Entropy, 20.","DOI":"10.3390\/e20060397"},{"key":"ref_14","unstructured":"Zhang, Z. (2017). Statistical Implications of Turing\u2019s Formula, John Wiley & Sons, Inc."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1093\/biomet\/40.3-4.237","article-title":"The population frequencies of species and estimation of population parameters","volume":"40","author":"Good","year":"1953","journal-title":"Biometrika"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Grabchak, M., Marcon, G., Lang, G., and Zhang, Z. (2017). The generalized Simpson\u2019s entropy is a measure of biodiversity. PLoS ONE, 12.","DOI":"10.1371\/journal.pone.0173305"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"623","DOI":"10.1007\/s11071-022-07665-3","article-title":"Mutual information matrix based on R\u00e9nyi entropy and application","volume":"110","year":"2022","journal-title":"Nonlinear Dyn."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Cover, T.M., and Thomas, J.A. (2006). Elements of Information Theory, Wiley & Son, Inc.","DOI":"10.1002\/047174882X"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"158","DOI":"10.3390\/stats3020013","article-title":"Generalized Mutual Information","volume":"3","author":"Zhang","year":"2020","journal-title":"Stats"},{"key":"ref_20","unstructured":"Khinchin, A.I. (1957). Mathematical Foundations of Information Theory, Dover Publications."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Amig\u00f3, J.M., Balogh, S.G., and Hern\u00e1ndez, S. (2018). A Brief Review of Generalized Entropies. Entropy, 20.","DOI":"10.3390\/e20110813"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"50005","DOI":"10.1209\/0295-5075\/133\/50005","article-title":"An overview of generalized entropic forms","volume":"133","author":"Korbel","year":"2021","journal-title":"Europhys. Lett."},{"key":"ref_23","first-page":"74","article-title":"Das Gesetz der Bev\u00f6lkerungskonzentration","volume":"59","author":"Auerbach","year":"1913","journal-title":"Petermann\u2019s Geogr. Mitteilungen"},{"key":"ref_24","unstructured":"Zipf, G.K. (1932). Selected Studies of the Principle of Relative Frequency in Language, Harvard University Press."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"873","DOI":"10.3150\/15-BEJ786","article-title":"Domains of attraction on countable alphabets","volume":"24","author":"Zhang","year":"2018","journal-title":"Bernoulli"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"60","DOI":"10.3103\/S1066530718010040","article-title":"Entropic Moments and Domains of Attraction on Countable Alphabets","volume":"27","author":"Molchanov","year":"2018","journal-title":"Math. Meth. Stat."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1109\/TIT.1981.1056331","article-title":"The Performance of Universal Encoding","volume":"27","author":"Krichevsky","year":"1981","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"2551","DOI":"10.1088\/0305-4470\/31\/11\/007","article-title":"Bayes\u2019 estimators of generalized entropies","volume":"31","author":"Holste","year":"1998","journal-title":"J. Phys. A Math. Gen."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"414","DOI":"10.1063\/1.166191","article-title":"Entropy estimation of symbol sequences","volume":"6","author":"Schurmann","year":"1996","journal-title":"Chaos"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Dietterich, T.G., Becker, S., and Ghahramani, Z. (2002). Advances in Neural Information Processing Systems, MIT Press.","DOI":"10.7551\/mitpress\/1120.001.0001"},{"key":"ref_31","first-page":"1469","article-title":"Entropy inference and the James-Stein estimator, with application to nonlinear gene association networks","volume":"10","author":"Hausser","year":"2009","journal-title":"J. Mach. Learn. Res."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"429","DOI":"10.1023\/A:1026096204727","article-title":"Non-parametric estimation of Shannon\u2019s Index of diversity when there are unseen species in sample","volume":"10","author":"Chao","year":"2003","journal-title":"Environ. Ecol. Stat."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"4039","DOI":"10.1002\/sim.2942","article-title":"Coverage-adjusted entropy estimation","volume":"26","author":"Vu","year":"2007","journal-title":"Stat. Med."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1368","DOI":"10.1162\/NECO_a_00266","article-title":"Entropy estimation in Turing\u2019s perspective","volume":"24","author":"Zhang","year":"2012","journal-title":"Neural Comput."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"2745","DOI":"10.1109\/TIT.2011.2179702","article-title":"A normal law for the plug-in estimator of entropy","volume":"58","author":"Zhang","year":"2012","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"504","DOI":"10.1109\/TIT.2012.2217393","article-title":"Asymptotic normality of an entropy estimator with exponentially decaying bias","volume":"59","author":"Zhang","year":"2013","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Chen, C., Grabchak, M., Stewart, A., Zhang, J., and Zhang, Z. (2018). Normal Laws for Two Entropy Estimators on Infinite Alphabets. Entropy, 20.","DOI":"10.3390\/e20050371"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"774","DOI":"10.1080\/10485252.2018.1482294","article-title":"Asymptotic Normality for Plug-in Estimators of Diversity Indices on Countable Alphabet","volume":"30","author":"Grabchak","year":"2018","journal-title":"J. Nonparametric Stat."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/7\/1060\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:11:37Z","timestamp":1760127097000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/7\/1060"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,13]]},"references-count":38,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2023,7]]}},"alternative-id":["e25071060"],"URL":"https:\/\/doi.org\/10.3390\/e25071060","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,13]]}}}