{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,18]],"date-time":"2026-07-18T01:27:49Z","timestamp":1784338069845,"version":"3.55.0"},"reference-count":84,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2017,6,14]],"date-time":"2017-06-14T00:00:00Z","timestamp":1497398400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>The choice associated with words is a fundamental property of natural languages. It lies at the heart of quantitative linguistics, computational linguistics and language sciences more generally. Information theory gives us tools at hand to measure precisely the average amount of choice associated with words: the word entropy. Here, we use three parallel corpora, encompassing ca. 450 million words in 1916 texts and 1259 languages, to tackle some of the major conceptual and practical problems of word entropy estimation: dependence on text size, register, style and estimation method, as well as non-independence of words in co-text. We present two main findings: Firstly, word entropies display relatively narrow, unimodal distributions. There is no language in our sample with a unigram entropy of less than six bits\/word. We argue that this is in line with information-theoretic models of communication. Languages are held in a narrow range by two fundamental pressures: word learnability and word expressivity, with a potential bias towards expressivity. Secondly, there is a strong linear relationship between unigram entropies and entropy rates. The entropy difference between words with and without co-textual information is narrowly distributed around ca. three bits\/word. In other words, knowing the preceding text reduces the uncertainty of words by roughly the same amount across languages of the world.<\/jats:p>","DOI":"10.3390\/e19060275","type":"journal-article","created":{"date-parts":[[2017,6,14]],"date-time":"2017-06-14T10:49:53Z","timestamp":1497437393000},"page":"275","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":83,"title":["The Entropy of Words\u2014Learnability and Expressivity across More than 1000 Languages"],"prefix":"10.3390","volume":"19","author":[{"given":"Christian","family":"Bentz","sequence":"first","affiliation":[{"name":"DFG Center for Advanced Studies, University of T\u00fcbingen, R\u00fcmelinstra\u00dfe 23, D-72070 T\u00fcbingen, Germany"},{"name":"Department of General Linguistics, University of T\u00fcbingen, Wilhemstra\u00dfe 19-23, D-72074 T\u00fcbingen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dimitrios","family":"Alikaniotis","sequence":"additional","affiliation":[{"name":"Department of Theoretical and Applied Linguistics, University of Cambridge, Cambridge CB3 9DP, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3168-4946","authenticated-orcid":false,"given":"Michael","family":"Cysouw","sequence":"additional","affiliation":[{"name":"Forschungszentrum Deutscher Sprachatlas, Philipps-Universit\u00e4t Marburg, Pilgrimstein 16, D-35032 Marburg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7820-923X","authenticated-orcid":false,"given":"Ramon","family":"Ferrer-i-Cancho","sequence":"additional","affiliation":[{"name":"Complexity and Quantitative Linguistics Lab, LARCA Research Group, Departament de Ci\u00e8ncies de la Computaci\u00f3, Universitat Polit\u00e8cnica de Catalunya, 08034 Barcelona, Catalonia, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2017,6,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1002\/j.1538-7305.1951.tb01366.x","article-title":"Prediction and entropy of printed English","volume":"30","author":"Shannon","year":"1951","journal-title":"Bell Syst. Tech. J."},{"key":"ref_3","first-page":"31","article-title":"An estimate of an upper bound for the entropy of English","volume":"18","author":"Brown","year":"1992","journal-title":"Comput. Linguist."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1319","DOI":"10.1109\/18.669425","article-title":"Nonparametric entropy estimation for stationary processes and random fields, with applications to English text","volume":"44","author":"Kontoyiannis","year":"1998","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"71","DOI":"10.3390\/entropy-e10020071","article-title":"Estimating the entropy of binary time series: Methodology, some theory and a simulation study","volume":"10","author":"Gao","year":"2008","journal-title":"Entropy"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"414","DOI":"10.1063\/1.166191","article-title":"Entropy estimation of symbol sequences","volume":"6","author":"Grassberger","year":"1996","journal-title":"Chaos Interdiscip. J. Nonlinear Sci."},{"key":"ref_7","unstructured":"Behr, F.H., Fossum, V., Mitzenmacher, M., and Xiao, D. (2003, January 25\u201327). Estimating and comparing entropies across written natural languages using PPM compression. Proceedings of the DCC 2003 Data Compression Conference, Snowbird, UT, USA."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Montemurro, M.A., and Zanette, D.H. (2011). Universal entropy of word ordering across linguistic families. PLoS ONE, 6.","DOI":"10.1371\/journal.pone.0019875"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Montemurro, M.A., and Zanette, D.H. (2016). Complexity and universality in the long-range order of words. Creativity and Universality in Language, Springer.","DOI":"10.1007\/978-3-319-24403-7_3"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Koplenig, A., Meyer, P., Wolfer, S., and M\u00fcller-Spitzer, C. (2017). The statistical trade-off between word order and word structure\u2013Large-scale evidence for the principle of least effort. PLoS ONE, 12.","DOI":"10.1371\/journal.pone.0173614"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Takahira, R., Tanaka-Ishii, K., and D\u0119bowski, \u0141. (2016). Entropy Rate Estimates for Natural Language\u2014A New Extrapolation of Compressed Large-Scale Corpora. Entropy, 18.","DOI":"10.3390\/e18100364"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Bentz, C., Verkerk, A., Kiela, D., Hill, F., and Buttery, P. (2015). Adaptive communication: Languages with more non-native speakers tend to have fewer word forms. PLoS ONE, 10.","DOI":"10.1371\/journal.pone.0128254"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Baechler, R., and Seiler, G. (2016). An information-theoretic approach to assess linguistic complexity. Complexity and Isolation, de Gruyter.","DOI":"10.1515\/9783110348965"},{"key":"ref_14","unstructured":"Bentz, C. (2016, January 21). The Low-Complexity-Belt: Evidence for large-scale language contact in human prehistory?. Proceedings of the 11th International Conference (EVOLANG11) on The Evolution of Language, New Orleans, LA, USA."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"206","DOI":"10.1080\/09296179808590128","article-title":"Measuring linguistic complexity: The morphological tier","volume":"5","author":"Juola","year":"1998","journal-title":"J. Quant. Linguist."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Miestamo, M., Sinnem\u00e4ki, K., and Karlsson, F. (2008). Assessing linguistic complexity. Language Complexity: Typology, Contact, Change, John Benjamins.","DOI":"10.1075\/slcs.94"},{"key":"ref_17","first-page":"021009","article-title":"Similarity of symbol frequency distributions with heavy tails","volume":"6","author":"Gerlach","year":"2016","journal-title":"Phys. Rev. X"},{"key":"ref_18","unstructured":"Bentz, C., and Berdicevskis, A. (2016, January 11). Learning pressures reduce morphological complexity: Linking corpus, computational and experimental evidence. Proceedings of the Workshop on Computational Linguistics for Linguistic Complexity (CL4LC), 26th International Conference on Computational Linguistics, Osaka, Japan."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"20140841","DOI":"10.1098\/rsif.2014.0841","article-title":"Universals versus historical contingencies in lexical evolution","volume":"11","author":"Bochkarev","year":"2014","journal-title":"J. R. Soc. Interface"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1165","DOI":"10.1126\/science.1170391","article-title":"Entropic evidence for linguistic structure in the Indus script","volume":"324","author":"Rao","year":"2009","journal-title":"Science"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1109\/MC.2010.112","article-title":"Probabilistic analysis of an acient undeciphered script","volume":"43","author":"Rao","year":"2010","journal-title":"Computer"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"457","DOI":"10.1353\/lan.2014.0031","article-title":"A statistical comparison of written language and nonlinguistic symbol systems","volume":"90","author":"Sproat","year":"2014","journal-title":"Language"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"795","DOI":"10.1162\/coli_c_00030","article-title":"Entropy, the Indus script, and language: A reply to R. Sproat","volume":"36","author":"Rao","year":"2010","journal-title":"Comput. Linguist."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"3526","DOI":"10.1073\/pnas.1012551108","article-title":"Word lengths are optimized for efficient communication","volume":"108","author":"Piantadosi","year":"2010","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1016\/j.cognition.2012.09.010","article-title":"Info\/information theory: Speakers choose shorter words in predictive contexts","volume":"126","author":"Mahowald","year":"2013","journal-title":"Cognition"},{"key":"ref_26","unstructured":"Ferrer-i Cancho, R., Bentz, C., and Seguin, C. (arXiv, 2015). Compression and the origins of Zipf\u2019s law of abbreviation, arXiv."},{"key":"ref_27","unstructured":"Bentz, C., J\u00e4ger, G., and Yanovich, I. (2015, January 26\u201330). Zipf\u2019s law of abbreviation as a language universal. Proceedings of the Leiden Workshop on Capturing Phylogenetic Algorithms for Linguistics, Leiden, The Netherlands."},{"key":"ref_28","unstructured":"Futrell, R., Mahowald, K., and Gibson, E. (2015, January 24\u201326). Quantifying word order freedom in dependency corpora. Proceedings of the Third International Conference on Dependency Linguistics (Depling 2015), Uppsala, Sweden."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"429","DOI":"10.1353\/lan.2013.0054","article-title":"Morphological organization: The low conditional entropy conjecture","volume":"89","author":"Ackerman","year":"2013","journal-title":"Language"},{"key":"ref_30","unstructured":"Blevins, J.P., and Blevins, J. (2009). Paradigms bit by bit: An information theoretic approach to the processing of paradigmatic structure in inflection and derivation. Analogy in Grammar: Form and Acquisition, Oxford University Press."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1126","DOI":"10.1016\/j.cognition.2007.05.006","article-title":"Expectation-based syntactic comprehension","volume":"106","author":"Levy","year":"2008","journal-title":"Cognition"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1016\/j.cogpsych.2010.02.002","article-title":"Redundancy and reduction: Speakers manage syntactic information density","volume":"61","author":"Jaeger","year":"2010","journal-title":"Cognit. Psychol."},{"key":"ref_33","first-page":"400","article-title":"Konstanz im Kurzzeitged\u00e4chtnis-Konstanz im sprachlichen Informationsflu\u00df","volume":"27","author":"Fenk","year":"1980","journal-title":"Z. Exp. Angew. Pshychol."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Bybee, J.L., and Hopper, P.J. (2001). Familiarity, information flow, and linguistic form. Frequency and the Emergence of Linguistic Structure, John Benjamins.","DOI":"10.1075\/tsl.45"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Ferrer-i Cancho, R., and del Prado Mart\u00edn, F.M. (2011). Information content versus word length in random typing. J. Stat. Mech. Theory Exp., L12002.","DOI":"10.1088\/1742-5468\/2011\/12\/L12002"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Ferrer-i Cancho, R., D\u0119bowski, \u0141., and del Prado Mart\u00edn, F.M. (2013). Constant conditional entropy and related hypotheses. J. Stat. Mech. Theory Exp., L07001.","DOI":"10.1088\/1742-5468\/2013\/07\/L07001"},{"key":"ref_37","first-page":"P06009","article-title":"The global minima of the communicative energy of natural communication systems","volume":"2007","year":"2007","journal-title":"J. Stat. Mech. Theory Exp."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"788","DOI":"10.1073\/pnas.0335980100","article-title":"Least effort and the origins of scaling in human language","volume":"100","year":"2003","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"449","DOI":"10.1140\/epjb\/e2005-00340-y","article-title":"Zipf\u2019s law from a communicative phase transition","volume":"47","year":"2005","journal-title":"Eur. Phys. J. B"},{"key":"ref_40","first-page":"1","article-title":"The optimality of attaching unlinked labels to unlinked meanings","volume":"36","year":"2016","journal-title":"Glottometrics"},{"key":"ref_41","first-page":"39","article-title":"A maximum entropy approach to natural language processing","volume":"22","author":"Berger","year":"1996","journal-title":"Comput. Linguist."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Och, F.J., and Ney, H. (2002, January 7\u201312). Discriminative training and maximum entropy models for statistical machine translation. Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, Philadelphia, PA, USA.","DOI":"10.3115\/1073083.1073133"},{"key":"ref_43","unstructured":"Herbelot, A., and Ganesalingam, M. (2013, January 4\u20139). Measuring semantic content in distributional vectors. Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, Short Papers (ACL Short Papers 2013), Sofia, Bulgaria."},{"key":"ref_44","unstructured":"Pad\u00f3, S., Palmer, A., Kisselew, M., and \u0160najder, J. (2015, January 14). Measuring Semantic Content To Assess Asymmetry in Derivation. Proceedings of the Workshop on Advances in Distributional Semantics, London, UK."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Santus, E., Lenci, A., Lu, Q., and Im Walde, S.S. (2014, January 26\u201330). Chasing Hypernyms in Vector Spaces with Entropy. Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, Gothenburg, Sweden.","DOI":"10.3115\/v1\/E14-4008"},{"key":"ref_46","unstructured":"Resnik, P. (1995, January 20\u201325). Using information content to evaluate semantic similarity in a taxonomy. Proceedings of the IJCAI\u201995 Proceedings of the 14th International Joint Conference on Artificial Intelligence, Montreal, QC, Canada."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Boon, M. (2014, January 26). Information density, Heaps\u2019 Law, and perception of factiness in news. Proceedings of the ACL Workshop on Language Technology and Computational Social Science (ACL LACSS 2014), Baltimore, MD, USA.","DOI":"10.3115\/v1\/W14-2511"},{"key":"ref_48","first-page":"742","article-title":"The sublanguage of cross-coverage","volume":"2002","author":"Stetson","year":"2002","journal-title":"Proc. AMIA Symp."},{"key":"ref_49","first-page":"426","article-title":"Perplexity analysis of obesity news coverage","volume":"2009","author":"McFarlane","year":"2009","journal-title":"AMIA Annu. Symp. Proc."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Kordoni, V., Villavicencio, A., and Idiart, M. (2006, January 23). Automated multi-word expression prediction for grammar engineering. Proceedings of the Workshop on Multiword Expressions: Identifying and Exploiting Underlying Properties, Association for Computational Linguistics, Sydney, Australia.","DOI":"10.3115\/1613692.1613700"},{"key":"ref_51","unstructured":"Ramisch, C., Schreiner, P., Idiart, M., and Villavicencio, A. (2008, January 1). An evaluation of methods for the extraction of multi-word expressions. Proceedings of the LREC Workshop-Towards a Shared Task for Multiword Expressions (MWE 2008), Marrakech, Morocco."},{"key":"ref_52","first-page":"471","article-title":"Entropy and inference, revisited","volume":"1","author":"Nemenman","year":"2002","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"1250223","DOI":"10.1142\/S0218127412502239","article-title":"Entropy analysis of word-length series of natural language texts: Effects of text language and genre","volume":"22","author":"Kalimeri","year":"2012","journal-title":"Int. J. Bifurc. Chaos"},{"key":"ref_54","unstructured":"Koehn, P. (2005, January 12\u201316). Europarl: A parallel corpus for statistical machine translation. Proceedings of the tenth Machine Translation Summit, Phuket, Thailand."},{"key":"ref_55","unstructured":"Calzolari, N., Choukri, K., Declerck, T., Loftsson, H., Maegaard, B., Mariani, J., Moreno, A., Odijk, J., and Piperidis, S. (2014, January 26\u201331). Creating a massively parallel Bible corpus. Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC-2014), Reykjavik, Iceland."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1515\/flin.2011.002","article-title":"The indeterminacy of word segmentation and the nature of morphology and syntax","volume":"45","author":"Haspelmath","year":"2011","journal-title":"Folia Linguist."},{"key":"ref_57","unstructured":"Taylor, J. (2014). Why are we so sure we know what a word is? In The Oxford Handbook of the Word, Oxford University Press. Chapter 42."},{"key":"ref_58","first-page":"25","article-title":"Informativeness of linguistic unit boundaries","volume":"28","author":"Geertzen","year":"2016","journal-title":"Ital. J. Linguist."},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"D\u0119bowski, \u0141. (2016, January 10\u201315). Consistency of the plug-in estimator of the entropy rate for ergodic processes. Proceedings of the 2016 IEEE International Symposium on Information Theory (ISIT), Barcelona, Spain.","DOI":"10.1109\/ISIT.2016.7541579"},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"2835","DOI":"10.1109\/TIT.2015.2412945","article-title":"Minimax estimation of functionals of discrete distributions","volume":"61","author":"Jiao","year":"2015","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"046208","DOI":"10.1103\/PhysRevE.79.046208","article-title":"Entropy estimation of very short symbolic sequences","volume":"79","author":"Lesne","year":"2009","journal-title":"Phys. Rev. E"},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"333","DOI":"10.1137\/1104033","article-title":"On a statistical estimate for the entropy of a sequence of independent random variables","volume":"4","author":"Basharin","year":"1959","journal-title":"Theory Probab. Appl."},{"key":"ref_63","first-page":"1469","article-title":"Entropy inference and the James\u2013Stein estimator, with application to nonlinear gene association networks","volume":"10","author":"Hausser","year":"2009","journal-title":"J. Mach. Learn. Res."},{"key":"ref_64","unstructured":"Shannon, C.E., and Weaver, W. (1949). The Mathematical Theory of Communication, The University of Illinois Press."},{"key":"ref_65","unstructured":"Csiszar, I. (1975). The statistical estimation of entropy in the non-parametric case. Topics in Information Theory, North-Holland."},{"key":"ref_66","doi-asserted-by":"crossref","unstructured":"Chomsky, N. (1965). Aspects of the Theory of Syntax, MIT Press.","DOI":"10.21236\/AD0616323"},{"key":"ref_67","doi-asserted-by":"crossref","unstructured":"L\u00fcdeling, A., and Kyt\u00f6, M. (2009). Distributions in text. Corpus Linguistics. An international handbook, De Gruyter.","DOI":"10.1515\/9783110213881.2"},{"key":"ref_68","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1076\/jqul.8.3.165.4101","article-title":"Two Regimes in the Frequency of Words and the Origins of Complex Lexicons: Zipf \u2019s Law Revisited","volume":"8","author":"Ferrer","year":"2001","journal-title":"J. Quant. Linguist."},{"key":"ref_69","first-page":"021006","article-title":"Stochastic model for the vocabulary growth in natural languages","volume":"3","author":"Gerlach","year":"2013","journal-title":"Phys. Rev. X"},{"key":"ref_70","doi-asserted-by":"crossref","first-page":"943","DOI":"10.1038\/srep00943","article-title":"Languages cool as they expand: Allometric scaling and the decreasing need for new words","volume":"2","author":"Petersen","year":"2012","journal-title":"Sci. Rep."},{"key":"ref_71","unstructured":"Manning, C.D., and Sch\u00fctze, H. (1999). Foundations of Statistical Natural Language Processing, MIT Press. Chapter 5."},{"key":"ref_72","doi-asserted-by":"crossref","first-page":"451","DOI":"10.1142\/S0218348X02001257","article-title":"Long-range fractal correlations in literary corpora","volume":"10","author":"Montemurro","year":"2002","journal-title":"Fractals"},{"key":"ref_73","unstructured":"Cover, T.M., and Thomas, J.A. (2006). Elements of Information Theory, John Wiley and Sons."},{"key":"ref_74","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1109\/TIT.1977.1055714","article-title":"A universal algorithm for sequential data compression","volume":"23","author":"Ziv","year":"1977","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_75","doi-asserted-by":"crossref","first-page":"530","DOI":"10.1109\/TIT.1978.1055934","article-title":"Compression of individual sequences via variable-rate coding","volume":"24","author":"Ziv","year":"1978","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_76","unstructured":"Hausser, J., and Strimmer, K. (2017, June 13). Available online: https:\/\/CRAN.R-project.org\/package=entropy."},{"key":"ref_77","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v067.i01","article-title":"Fitting Linear Mixed-Effects Models Using lme4","volume":"67","author":"Bates","year":"2015","journal-title":"J. Stat. Softw."},{"key":"ref_78","unstructured":"Hammarstr\u00f6m, H., Forkel, R., Haspelmath, M., and Bank, S. (2016). Glottolog 2.7, Max Planck Institute for the Science of Human History."},{"key":"ref_79","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1016\/j.cognition.2015.03.016","article-title":"Compression and communication in the cultural evolution of linguistic structure","volume":"141","author":"Kirby","year":"2015","journal-title":"Cognition"},{"key":"ref_80","unstructured":"Ferrer-i-Cancho, R. (arXiv, 2014). Optimization models of natural communication, arXiv."},{"key":"ref_81","unstructured":"Bentz, C., Ruzsics, T., Koplenig, A., and Samar\u017ei\u0107, T. (2016, January 11). A comparison between morphological complexity measures: typological data vs. language corpora. Proceedings of the Workshop on Computational Linguistics for Linguistic Complexity (CL4LC), 26th International Conference on Computational Linguistics, Osaka, Japan."},{"key":"ref_82","doi-asserted-by":"crossref","unstructured":"Baayen, H.R. (2001). Word Frequency Distributions, Kluwer.","DOI":"10.1007\/978-94-010-0844-0"},{"key":"ref_83","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1007\/s10260-005-0121-y","article-title":"Bayesian inference for categorical data analysis","volume":"14","author":"Agresti","year":"2005","journal-title":"Stat. Methods Appl."},{"key":"ref_84","doi-asserted-by":"crossref","first-page":"429","DOI":"10.1023\/A:1026096204727","article-title":"Nonparametric estimation of Shannon\u2019s index of diversity when there are unseen species in sample","volume":"10","author":"Chao","year":"2003","journal-title":"Environ. Ecol. Stat."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/19\/6\/275\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:39:04Z","timestamp":1760207944000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/19\/6\/275"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,6,14]]},"references-count":84,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2017,6]]}},"alternative-id":["e19060275"],"URL":"https:\/\/doi.org\/10.3390\/e19060275","relation":{"has-preprint":[{"id-type":"doi","id":"10.20944\/preprints201704.0180.v1","asserted-by":"object"}]},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,6,14]]}}}