{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,13]],"date-time":"2026-03-13T01:16:41Z","timestamp":1773364601571,"version":"3.50.1"},"reference-count":24,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2022,5,11]],"date-time":"2022-05-11T00:00:00Z","timestamp":1652227200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>We present a new class of estimators of Shannon entropy for severely undersampled discrete distributions. It is based on a generalization of an estimator proposed by T. Sch\u00fcrmann, which itself is a generalization of an estimator proposed by myself.For a special set of parameters, they are completely free of bias and have a finite variance, something which is widely believed to be impossible. We present also detailed numerical tests, where we compare them with other recent estimators and with exact results, and point out a clash with Bayesian estimators for mutual information.<\/jats:p>","DOI":"10.3390\/e24050680","type":"journal-article","created":{"date-parts":[[2022,5,12]],"date-time":"2022-05-12T03:28:43Z","timestamp":1652326123000},"page":"680","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["On Generalized Sch\u00fcrmann Entropy Estimators"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4966-9998","authenticated-orcid":false,"given":"Peter","family":"Grassberger","sequence":"first","affiliation":[{"name":"J\u00fclich Supercomputing Center, J\u00fclich Research Center, D-52425 J\u00fclich, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,5,11]]},"reference":[{"key":"ref_1","unstructured":"Quastler, H. (1955). Note on the bias of information estimates. Information Theory in Psychology II-B, Free Press."},{"key":"ref_2","unstructured":"Harris, B. (1973). Colloquia Mathematica Societatis J\u00e1nos Bolyai. Infinite Finite Sets, 175."},{"key":"ref_3","first-page":"435","article-title":"Complexity of symbol sequences","volume":"5","author":"Herzel","year":"1988","journal-title":"Syst. Anal. Mod. Sim."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1016\/0375-9601(88)90193-4","article-title":"Entropy Estimates from Insufficient Samplings","volume":"128","author":"Grassberger","year":"1988","journal-title":"Phys. Lett."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1209\/0295-5075\/23\/5\/001","article-title":"A new method to calculate higher-order entropies from finite samples","volume":"23","author":"Schmitt","year":"1993","journal-title":"Europhys. Lett."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"6841","DOI":"10.1103\/PhysRevE.52.6841","article-title":"Estimating functions of probability distributions from a finite set of samples","volume":"52","author":"Wolpert","year":"1995","journal-title":"Phys. Rev. E"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1443","DOI":"10.1007\/BF02179880","article-title":"Guessing probability distributions from small samples","volume":"80","author":"Poschel","year":"1995","journal-title":"J. Stat. Phys."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1080\/0954898X.1996.11978656","article-title":"Analytical estimates of limited sampling biases in different information measures. Network: Computation in neural systems","volume":"7","author":"Panzeri","year":"1996","journal-title":"Netw. Comput. Neural Syst."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"414","DOI":"10.1063\/1.166191","article-title":"Entropy estimation of symbol sequences","volume":"6","author":"Grassberger","year":"1996","journal-title":"Chaos"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1103\/PhysRevLett.80.197","article-title":"Entropy and information in neural spike trains","volume":"80","author":"Strong","year":"1998","journal-title":"Phys. Rev. Lett."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2551","DOI":"10.1088\/0305-4470\/31\/11\/007","article-title":"Bayes\u2019 estimators of generalized entropies","volume":"31","author":"Holste","year":"1998","journal-title":"J. Phys. A"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Dietterich, T., Becker, S., and Ghahramani, Z. (2002). Entropy and inference, revisited. Advances in Neural Information Processing 14, MIT Press.","DOI":"10.7551\/mitpress\/1120.001.0001"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1191","DOI":"10.1162\/089976603321780272","article-title":"Estimation of entropy and mutual information","volume":"15","author":"Paninski","year":"2003","journal-title":"Neural Comput."},{"key":"ref_14","unstructured":"Grassberger, P. (2003). Entropy estimates from insufficient samplings. arXiv."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"L295","DOI":"10.1088\/0305-4470\/37\/27\/L02","article-title":"Bias analysis in entropy estimation","volume":"37","year":"2004","journal-title":"J. Phys. A Math. Gen."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"4039","DOI":"10.1002\/sim.2942","article-title":"Coverage-adjusted entropy estimation","volume":"26","author":"Vu","year":"2007","journal-title":"Stat. Med."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"202001","DOI":"10.1088\/1751-8113\/41\/20\/202001","article-title":"Entropy estimates of small data sets","volume":"41","author":"Bonachela","year":"2008","journal-title":"J. Phys. A Math. Gen."},{"key":"ref_18","first-page":"1469","article-title":"Entropy inference and the James-Stein estimator, with application to nonlinear gene association networks","volume":"10","author":"Hausser","year":"2009","journal-title":"J. Mach. Learn. Res."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Wolpert, D.H., and Deo, S.D. (2013). Estimating functions of distributions defined over spaces of unknown size. Entropy, 15.","DOI":"10.3390\/e15114668"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1091","DOI":"10.1111\/2041-210X.12108","article-title":"Entropy and the species accumulation curve: A novel entropy estimator via discovery rates of new species","volume":"2013","author":"Chao","year":"2013","journal-title":"Methods Ecol. Evol."},{"key":"ref_21","first-page":"2833","article-title":"Bayesian entropy estimation for countable discrete distributions","volume":"2014","author":"Archer","year":"2013","journal-title":"J. Mach. Learn. Res."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Hern\u00e1ndez, D.G., and Samengo, I. (2019). Estimating the Mutual Information between Two Discrete, Asymmetric Variables with Limited Samples. Entropy, 21.","DOI":"10.3390\/e21060623"},{"key":"ref_23","unstructured":"Abramowitz, M., and Stegun, I. (1965). Handbook of Mathematical Functions, Dover."},{"key":"ref_24","unstructured":"Schwartz-Ziv, R., and Tishby, N. (2017). Opening the black box of Deep Neural Networks via Information. arXiv."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/5\/680\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:09:34Z","timestamp":1760137774000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/5\/680"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,11]]},"references-count":24,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2022,5]]}},"alternative-id":["e24050680"],"URL":"https:\/\/doi.org\/10.3390\/e24050680","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,5,11]]}}}