{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,21]],"date-time":"2026-03-21T21:08:58Z","timestamp":1774127338697,"version":"3.50.1"},"reference-count":73,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2024,1,17]],"date-time":"2024-01-17T00:00:00Z","timestamp":1705449600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Agencia Estatal de Investigaci\u00f3n","award":["MCIN\/AEI\/10.13039\/501100011033"],"award-info":[{"award-number":["MCIN\/AEI\/10.13039\/501100011033"]}]},{"name":"Agencia Estatal de Investigaci\u00f3n","award":["PID2021-122256NB-C21"],"award-info":[{"award-number":["PID2021-122256NB-C21"]}]},{"name":"Agencia Estatal de Investigaci\u00f3n","award":["CEX2021-001164-M"],"award-info":[{"award-number":["CEX2021-001164-M"]}]},{"DOI":"10.13039\/501100011033","name":"Fondo Europeo de Desarrollo Regional","doi-asserted-by":"publisher","award":["MCIN\/AEI\/10.13039\/501100011033"],"award-info":[{"award-number":["MCIN\/AEI\/10.13039\/501100011033"]}],"id":[{"id":"10.13039\/501100011033","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100011033","name":"Fondo Europeo de Desarrollo Regional","doi-asserted-by":"publisher","award":["PID2021-122256NB-C21"],"award-info":[{"award-number":["PID2021-122256NB-C21"]}],"id":[{"id":"10.13039\/501100011033","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100011033","name":"Fondo Europeo de Desarrollo Regional","doi-asserted-by":"publisher","award":["CEX2021-001164-M"],"award-info":[{"award-number":["CEX2021-001164-M"]}],"id":[{"id":"10.13039\/501100011033","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100011033","name":"Mar\u00eda de Maeztu Program for units of Excellence in R&amp;D","doi-asserted-by":"publisher","award":["MCIN\/AEI\/10.13039\/501100011033"],"award-info":[{"award-number":["MCIN\/AEI\/10.13039\/501100011033"]}],"id":[{"id":"10.13039\/501100011033","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100011033","name":"Mar\u00eda de Maeztu Program for units of Excellence in R&amp;D","doi-asserted-by":"publisher","award":["PID2021-122256NB-C21"],"award-info":[{"award-number":["PID2021-122256NB-C21"]}],"id":[{"id":"10.13039\/501100011033","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100011033","name":"Mar\u00eda de Maeztu Program for units of Excellence in R&amp;D","doi-asserted-by":"publisher","award":["CEX2021-001164-M"],"award-info":[{"award-number":["CEX2021-001164-M"]}],"id":[{"id":"10.13039\/501100011033","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Entropy estimation is a fundamental problem in information theory that has applications in various fields, including physics, biology, and computer science. Estimating the entropy of discrete sequences can be challenging due to limited data and the lack of unbiased estimators. Most existing entropy estimators are designed for sequences of independent events and their performances vary depending on the system being studied and the available data size. In this work, we compare different entropy estimators and their performance when applied to Markovian sequences. Specifically, we analyze both binary Markovian sequences and Markovian systems in the undersampled regime. We calculate the bias, standard deviation, and mean squared error for some of the most widely employed estimators. We discuss the limitations of entropy estimation as a function of the transition probabilities of the Markov processes and the sample size. Overall, this paper provides a comprehensive comparison of entropy estimators and their performance in estimating entropy for systems with memory, which can be useful for researchers and practitioners in various fields.<\/jats:p>","DOI":"10.3390\/e26010079","type":"journal-article","created":{"date-parts":[[2024,1,17]],"date-time":"2024-01-17T08:57:32Z","timestamp":1705481852000},"page":"79","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Entropy Estimators for Markovian Sequences: A Comparative Analysis"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8814-4708","authenticated-orcid":false,"given":"Juan","family":"De Gregorio","sequence":"first","affiliation":[{"name":"Institute for Cross-Disciplinary Physics and Complex Systems IFISC (UIB-CSIC), Campus Universitat de les Illes Balears, E-07122 Palma de Mallorca, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2549-7071","authenticated-orcid":false,"given":"David","family":"S\u00e1nchez","sequence":"additional","affiliation":[{"name":"Institute for Cross-Disciplinary Physics and Complex Systems IFISC (UIB-CSIC), Campus Universitat de les Illes Balears, E-07122 Palma de Mallorca, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2046-7620","authenticated-orcid":false,"given":"Ra\u00fal","family":"Toral","sequence":"additional","affiliation":[{"name":"Institute for Cross-Disciplinary Physics and Complex Systems IFISC (UIB-CSIC), Campus Universitat de les Illes Balears, E-07122 Palma de Mallorca, Spain"}]}],"member":"1968","published-online":{"date-parts":[[2024,1,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Dobzhansky, T., Hecht, M.K., and Steere, W.C. (1972). Evolutionary Biology: Volume 6, Springer.","DOI":"10.1007\/978-1-4757-0256-9"},{"key":"ref_2","unstructured":"Stinson, D.R. (1995). Cryptography: Theory and Practice, CRC Press Inc.. [1st ed.]."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1103\/PhysRevLett.80.197","article-title":"Entropy and Information in Neural Spike Trains","volume":"80","author":"Strong","year":"1998","journal-title":"Phys. Rev. Lett."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"377","DOI":"10.1089\/1066527041410418","article-title":"Maximum Entropy Modeling of Short Sequence Motifs with Applications to RNA Splicing Signals","volume":"11","author":"Yeo","year":"2004","journal-title":"J. Comput. Biol. J. Comput. Mol. Cell Biol."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Cover, T., and Thomas, J. (2006). Elements of Information Theory, John Wiley and Sons.","DOI":"10.1002\/047174882X"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"254102","DOI":"10.1103\/PhysRevLett.96.254102","article-title":"Estimating the Shannon Entropy: Recurrence Plots versus Symbolic Dynamics","volume":"96","author":"Letellier","year":"2006","journal-title":"Phys. Rev. Lett."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"302","DOI":"10.1162\/biot.2006.1.3.302","article-title":"Approaches to Information-Theoretic Analysis of Neural Activity","volume":"1","author":"Victor","year":"2006","journal-title":"Biol. Theory"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.physrep.2006.12.004","article-title":"Causality detection based on information-theoretic approaches in time series analysis","volume":"441","author":"Vejmelka","year":"2007","journal-title":"Phys. Rep."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"154102","DOI":"10.1103\/PhysRevLett.99.154102","article-title":"Distinguishing Noise from Chaos","volume":"99","author":"Rosso","year":"2007","journal-title":"Phys. Rev. Lett."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1765","DOI":"10.3390\/e12071765","article-title":"Entropy and Information Approaches to Genetic Diversity and its Expression: Genomic Geography","volume":"12","author":"Sherwin","year":"2010","journal-title":"Entropy"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1553","DOI":"10.3390\/e14081553","article-title":"Permutation Entropy and Its Main Biomedical and Econophysics Applications: A Review","volume":"14","author":"Zanin","year":"2012","journal-title":"Entropy"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Bentz, C., Alikaniotis, D., Cysouw, M., and Ferrer-i Cancho, R. (2017). The Entropy of Words\u2014Learnability and Expressivity across More than 1000 Languages. Entropy, 19.","DOI":"10.20944\/preprints201704.0180.v1"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Cassetti, J., Delgadino, D., Rey, A., and Frery, A.C. (2022). Entropy Estimators in SAR Image Classification. Entropy, 24.","DOI":"10.3390\/e24040509"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A Mathematical Theory of Communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1191","DOI":"10.1162\/089976603321780272","article-title":"Estimation of Entropy and Mutual Information","volume":"15","author":"Paninski","year":"2003","journal-title":"Neural Comput."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Contreras Rodr\u00edguez, L., Madarro-Cap\u00f3, E.J., Leg\u00f3n-P\u00e9rez, C.M., Rojas, O., and Sosa-G\u00f3mez, G. (2021). Selecting an Effective Entropy Estimator for Short Sequences of Bits and Bytes with Maximum Entropy. Entropy, 23.","DOI":"10.3390\/e23050561"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"770","DOI":"10.1038\/s42254-022-00532-5","article-title":"Tackling the subsampling problem to infer collective properties from limited data","volume":"4","author":"Levina","year":"2022","journal-title":"Nat. Rev. Phys."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"429","DOI":"10.1023\/A:1026096204727","article-title":"Nonparametric estimation of Shannon\u2019s diversity index when there are unseen species in sample","volume":"10","author":"Chao","year":"2003","journal-title":"Environ. Ecol. Stat."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"4039","DOI":"10.1002\/sim.2942","article-title":"Coverage-adjusted entropy estimation","volume":"Volume 26","author":"Vu","year":"2007","journal-title":"Statistics in Medicine"},{"key":"ref_20","first-page":"1469","article-title":"Entropy Inference and the James-Stein Estimator, with Application to Nonlinear Gene Association Networks","volume":"10","author":"Hausser","year":"2009","journal-title":"J. Mach. Learn. Res."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Arora, A., Meister, C., and Cotterell, R. (2022, January 22\u201327). Estimating the Entropy of Linguistic Distributions. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Dublin, Ireland.","DOI":"10.18653\/v1\/2022.acl-short.20"},{"key":"ref_22","unstructured":"Gardiner, C.W. (1965). Handbook of Stochastic Methods for Physics, Chemistry and the Natural Sciences, Springer."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1016\/S0092-8240(89)80049-7","article-title":"Stochastic models for heterogeneous DNA sequences","volume":"51","author":"Churchill","year":"1989","journal-title":"Bull. Math. Biol."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1177\/030913339902300302","article-title":"The weather generation game: A review of stochastic weather models","volume":"23","author":"Wilks","year":"1999","journal-title":"Prog. Phys. Geogr. Earth Environ."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"4559","DOI":"10.1103\/PhysRevLett.74.4559","article-title":"Markov Processes: Linguistics and Zipf\u2019s Law","volume":"74","author":"Kanter","year":"1995","journal-title":"Phys. Rev. Lett."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1063\/1.1530990","article-title":"Regularities unseen, randomness observed: Levels of entropy convergence","volume":"13","author":"Crutchfield","year":"2003","journal-title":"Chaos Interdiscip. J. Nonlinear Sci."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"112797","DOI":"10.1016\/j.chaos.2022.112797","article-title":"An improved estimator of Shannon entropy with applications to systems with memory","volume":"165","author":"Toral","year":"2022","journal-title":"Chaos Solitons Fractals"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"655","DOI":"10.1016\/j.physa.2006.01.077","article-title":"Regular and stochastic behavior of Parkinsonian pathological tremor signals","volume":"369","author":"Yulmetyev","year":"2006","journal-title":"Phys. Stat. Mech. Appl."},{"key":"ref_29","unstructured":"Ho, D.T., and Cao, T.H. (2012). Pacific Rim Knowledge Acquisition Workshop, Springer."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Seifert, M., Gohr, A., Strickert, M., and Grosse, I. (2012). Parsimonious higher-order hidden Markov models for improved array-CGH analysis with applications to Arabidopsis Thaliana. PLoS Comput. Biol., 8.","DOI":"10.1371\/journal.pcbi.1002286"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Singer, P., Helic, D., Taraghi, B., and Strohmaier, M. (2014). Detecting memory and structure in human navigation patterns using Markov chain models of varying order. PLoS ONE, 9.","DOI":"10.1371\/journal.pone.0102070"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"070601","DOI":"10.1103\/PhysRevLett.127.070601","article-title":"Optimal Non-Markovian Search Strategies with n-Step Memory","volume":"127","author":"Meyer","year":"2021","journal-title":"Phys. Rev. Lett."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"6223","DOI":"10.1002\/joc.7175","article-title":"Selecting Markov chain orders for generating daily precipitation series across different K\u00f6ppen climate regimes","volume":"41","author":"Osborn","year":"2021","journal-title":"Int. J. Climatol."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Wei\u00df, C.H. (2019). Measures of Dispersion and Serial Dependence in Categorical Time Series. Econometrics, 7.","DOI":"10.3390\/econometrics7020017"},{"key":"ref_35","first-page":"40","article-title":"On a Markov multinomial distribution","volume":"20","author":"Wang","year":"1995","journal-title":"Math. Sci."},{"key":"ref_36","unstructured":"Grassberger, P. (2008). Entropy Estimates from Insufficient Samplings. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"202001","DOI":"10.1088\/1751-8113\/41\/20\/202001","article-title":"Entropy estimates of small data sets","volume":"41","author":"Bonachela","year":"2008","journal-title":"J. Phys. Math. Theor."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"677","DOI":"10.2307\/1427041","article-title":"Number of successes in Markov trials","volume":"20","author":"Bhat","year":"1988","journal-title":"Adv. Appl. Probab."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"625","DOI":"10.1093\/biomet\/65.3.625","article-title":"Estimation of the size of a closed population when capture probabilities vary among animals","volume":"65","author":"Burnham","year":"1978","journal-title":"Biometrika"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"6841","DOI":"10.1103\/PhysRevE.52.6841","article-title":"Estimating functions of probability distributions from a finite set of samples","volume":"52","author":"Wolpert","year":"1995","journal-title":"Phys. Rev. E"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"051139","DOI":"10.1103\/PhysRevE.85.051139","article-title":"Estimation of the entropy based on its polynomial representation","volume":"85","author":"Vinck","year":"2012","journal-title":"Phys. Rev. E"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1368","DOI":"10.1162\/NECO_a_00266","article-title":"Entropy Estimation in Turing\u2019s Perspective","volume":"24","author":"Zhang","year":"2012","journal-title":"Neural Comput."},{"key":"ref_43","unstructured":"Burges, C., Bottou, L., Welling, M., Ghahramani, Z., and Weinberger, K. (2013). Advances in Neural Information Processing Systems, Curran Associates, Inc."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"4668","DOI":"10.3390\/e15114668","article-title":"Estimating Functions of Distributions Defined over Spaces of Unknown Size","volume":"15","author":"Wolpert","year":"2013","journal-title":"Entropy"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1145\/3125643","article-title":"Estimating the Unseen: Improved Estimators for Entropy and Other Properties","volume":"64","author":"Valiant","year":"2017","journal-title":"Assoc. Comput. Mach."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Grassberger, P. (2022). On Generalized Sch\u00fcrmann Entropy Estimators. Entropy, 24.","DOI":"10.3390\/e24050680"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Piga, A., Font-Pomarol, L., Sales-Pardo, M., and Guimer\u00e0, R. (2023). Bayesian estimation of information-theoretic metrics for sparsely sampled distributions. arXiv.","DOI":"10.2139\/ssrn.4621333"},{"key":"ref_48","first-page":"108","article-title":"Note on the bias of information estimates","volume":"71","author":"Miller","year":"1955","journal-title":"Inf. Theory Psychol. Probl. Methods"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"L295","DOI":"10.1088\/0305-4470\/37\/27\/L02","article-title":"Bias analysis in entropy estimation","volume":"37","year":"2004","journal-title":"J. Phys. Math. Gen."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1214\/aoms\/1177706722","article-title":"Some Problems of Simultaneous Minimax Estimation","volume":"29","author":"Trybula","year":"1958","journal-title":"Ann. Math. Stat."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1109\/TIT.1981.1056331","article-title":"The performance of universal encoding","volume":"27","author":"Krichevsky","year":"1981","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"414","DOI":"10.1063\/1.166191","article-title":"Entropy estimation of symbol sequences","volume":"6","author":"Grassberger","year":"1996","journal-title":"Chaos Interdiscip. J. Nonlinear Sci."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"2551","DOI":"10.1088\/0305-4470\/31\/11\/007","article-title":"Bayes\u2019 estimators of generalized entropies","volume":"31","author":"Holste","year":"1998","journal-title":"J. Phys. Math. Gen."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Dietterich, T., Becker, S., and Ghahramani, Z. (2001). Advances in Neural Information Processing Systems, MIT Press.","DOI":"10.7551\/mitpress\/1120.001.0001"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"056111","DOI":"10.1103\/PhysRevE.69.056111","article-title":"Entropy and information in neural spike trains: Progress on the sampling problem","volume":"69","author":"Nemenman","year":"2004","journal-title":"Phys. Rev. E"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"2013","DOI":"10.3390\/e13122013","article-title":"Coincidences and Estimation of Entropies of Random Variables with Large Cardinalities","volume":"13","author":"Nemenman","year":"2011","journal-title":"Entropy"},{"key":"ref_57","unstructured":"(2023, October 30). Simomarsili. ndd\u2014Bayesian Entropy Estimation from Discrete Data. Available online: https:\/\/github.com\/simomarsili\/ndd."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"663","DOI":"10.1080\/01621459.1952.10483446","article-title":"A Generalization of Sampling without Replacement from a Finite Universe","volume":"47","author":"Horvitz","year":"1952","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_59","first-page":"547","article-title":"On measures of entropy and information","volume":"Volume 1","year":"1961","journal-title":"Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics Probability"},{"key":"ref_60","unstructured":"Gruber, M.H.J. (1998). Improving Efficiency by Shrinkage: The James-Stein and Ridge Regression Estimators, Routledge."},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Sch\u00e4fer, J., and Strimmer, K. (2005). A Shrinkage Approach to Large-Scale Covariance Matrix Estimation and Implications for Functional Genomics. Stat. Appl. Genet. Mol. Biol., 4.","DOI":"10.2202\/1544-6115.1175"},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"1091","DOI":"10.1111\/2041-210X.12108","article-title":"Entropy and the species accumulation curve: A novel entropy estimator via discovery rates of new species","volume":"4","author":"Chao","year":"2013","journal-title":"Methods Ecol. Evol."},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"528","DOI":"10.1111\/j.2517-6161.1985.tb01383.x","article-title":"A model for high-order Markov chains","volume":"47","author":"Raftery","year":"1985","journal-title":"J. R. Stat. Soc. Ser. Stat. Methodol."},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"011106","DOI":"10.1103\/PhysRevE.76.011106","article-title":"Inferring Markov chains: Bayesian estimation, model comparison, entropy rate, and out-of-class modeling","volume":"76","author":"Strelioff","year":"2007","journal-title":"Phys. Rev. E"},{"key":"ref_65","doi-asserted-by":"crossref","first-page":"1687","DOI":"10.1109\/78.845926","article-title":"Estimating the entropy of a signal with applications","volume":"48","author":"Bercher","year":"2000","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_66","doi-asserted-by":"crossref","unstructured":"Feutrill, A., and Roughan, M. (2021). A review of Shannon and differential entropy rate estimation. Entropy, 23.","DOI":"10.3390\/e23081046"},{"key":"ref_67","doi-asserted-by":"crossref","first-page":"495","DOI":"10.1080\/00107510902823517","article-title":"Generalised information and entropy measures in physics","volume":"50","author":"Beck","year":"2009","journal-title":"Contemp. Phys."},{"key":"ref_68","doi-asserted-by":"crossref","first-page":"066138","DOI":"10.1103\/PhysRevE.69.066138","article-title":"Estimating mutual information","volume":"69","author":"Kraskov","year":"2004","journal-title":"Phys. Rev. E"},{"key":"ref_69","unstructured":"Walters-Williams, J., and Li, Y. (2009, January 14\u201316). Estimation of mutual information: A survey. Proceedings of the Rough Sets and Knowledge Technology: 4th International Conference, RSKT 2009, Gold Coast, Australia. Proceedings 4."},{"key":"ref_70","unstructured":"Minculete, N., and Savin, D. (2023). Some properties of a type of the entropy of an ideal and the divergence of two ideals. arXiv."},{"key":"ref_71","unstructured":"Camaglia, F., Nemenman, I., Mora, T., and Walczak, A.M. (2023). Bayesian estimation of the Kullback-Leibler divergence for categorical sytems using mixtures of Dirichlet priors. arXiv."},{"key":"ref_72","unstructured":"Montgomery-Smith, S., and Sch\u00fcrmann, T. (2014). Unbiased Estimators for Entropy and Class Number. arXiv."},{"key":"ref_73","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1093\/biomet\/40.3-4.237","article-title":"The population frequencies of species and the estimation of population parameters","volume":"40","author":"Good","year":"1953","journal-title":"Biometrika"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/1\/79\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T13:49:00Z","timestamp":1760104140000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/1\/79"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,17]]},"references-count":73,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,1]]}},"alternative-id":["e26010079"],"URL":"https:\/\/doi.org\/10.3390\/e26010079","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,1,17]]}}}