{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,4]],"date-time":"2025-12-04T18:44:25Z","timestamp":1764873865350,"version":"build-2065373602"},"reference-count":36,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2021,10,2]],"date-time":"2021-10-02T00:00:00Z","timestamp":1633132800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000913","name":"James S. McDonnell Foundation","doi-asserted-by":"publisher","award":["Scholar Award"],"award-info":[{"award-number":["Scholar Award"]}],"id":[{"id":"10.13039\/100000913","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Optimal coding theories of language predict that speakers will keep the amount of information in their utterances relatively uniform under the constraints imposed by their language, but how much do these constraints influence information structure, and how does this influence vary across languages? We present a novel method for characterizing the information structure of sentences across a diverse set of languages. While the structure of English is broadly consistent with the shape predicted by optimal coding, many languages are not consistent with this prediction. We proceed to show that the characteristic information curves of languages are partly related to a variety of typological features from phonology to word order. These results present an important step in the direction of exploring upper bounds for the extent to which linguistic codes can be optimal for communication.<\/jats:p>","DOI":"10.3390\/e23101300","type":"journal-article","created":{"date-parts":[[2021,10,4]],"date-time":"2021-10-04T09:24:15Z","timestamp":1633339455000},"page":"1300","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Characterizing the Typical Information Curves of Diverse Languages"],"prefix":"10.3390","volume":"23","author":[{"given":"Josef","family":"Klafka","sequence":"first","affiliation":[{"name":"Department of Psychology, Carnegie Mellon University, 5000 Forbes Ave, Pittsburgh, PA 15213, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8515-0029","authenticated-orcid":false,"given":"Daniel","family":"Yurovsky","sequence":"additional","affiliation":[{"name":"Department of Psychology, Carnegie Mellon University, 5000 Forbes Ave, Pittsburgh, PA 15213, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,10,2]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Austin, J.L. (1975). How to Do Things with Words, Oxford University Press.","DOI":"10.1093\/acprof:oso\/9780198245537.001.0001"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"703","DOI":"10.1037\/0033-295X.96.4.703","article-title":"Human memory: An adaptive perspective","volume":"96","author":"Anderson","year":"1989","journal-title":"Psychol. Rev."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"621","DOI":"10.1146\/annurev.psych.093008.131123","article-title":"Thirty years and counting: Finding meaning in the N400 component of the event-related brain potential (ERP)","volume":"62","author":"Kutas","year":"2011","journal-title":"Annu. Rev. Psychol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1632","DOI":"10.1126\/science.7777863","article-title":"Integration of visual and linguistic information in spoken language comprehension","volume":"268","author":"Tanenhaus","year":"1995","journal-title":"Science"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1017\/S0140525X12001495","article-title":"An integrated theory of language production and comprehension","volume":"36","author":"Pickering","year":"2013","journal-title":"Behav. Brain Sci."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1177\/00238309040470010201","article-title":"The smooth signal redundancy hypothesis: A functional explanation for relationships between redundancy, prosodic prominence, and duration in spontaneous speech","volume":"47","author":"Aylett","year":"2004","journal-title":"Lang. Speech"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Genzel, D., and Charniak, E. (2002, January 6\u201312). Entropy rate constancy in text. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, PA, USA.","DOI":"10.3115\/1073083.1073117"},{"key":"ref_9","first-page":"849","article-title":"Speakers optimize information density through syntactic reduction","volume":"19","author":"Levy","year":"2007","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1016\/j.cognition.2012.09.010","article-title":"Info\/information theory: Speakers choose shorter words in predictive contexts","volume":"126","author":"Mahowald","year":"2013","journal-title":"Cognition"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1016\/j.cogpsych.2010.02.002","article-title":"Redundancy and reduction: Speakers manage syntactic information density","volume":"61","author":"Jaeger","year":"2010","journal-title":"Cogn. Psychol."},{"key":"ref_12","unstructured":"Frank, A.F., and Jaeger, T.F. (2008, January 23\u201326). Speaking rationally: Uniform information density as an optimal strategy for language production. Proceedings of the Annual Meeting of the Cognitive Science Society, Washington, DC, USA."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"3526","DOI":"10.1073\/pnas.1012551108","article-title":"Word lengths are optimized for efficient communication","volume":"108","author":"Piantadosi","year":"2011","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1049","DOI":"10.1126\/science.1218811","article-title":"Kinship categories across languages reflect general communicative principles","volume":"336","author":"Kemp","year":"2012","journal-title":"Science"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1016\/j.cognition.2015.03.016","article-title":"Compression and communication in the cultural evolution of linguistic structure","volume":"141","author":"Kirby","year":"2015","journal-title":"Cognition"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"775","DOI":"10.1111\/cogs.12102","article-title":"Language evolution can be shaped by the structure of the world","volume":"38","author":"Perfors","year":"2014","journal-title":"Cogn. Sci."},{"key":"ref_17","unstructured":"Yu, S., Cong, J., Liang, J., and Liu, H. (2016). The distribution of information content in English sentences. arXiv."},{"key":"ref_18","unstructured":"Leech, G.N. (1992). 100 Million Words of English: The British National Corpus (BNC), SNU Korean Language Education Center."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1006\/csla.1999.0128","article-title":"An empirical study of smoothing techniques for language modeling","volume":"13","author":"Chen","year":"1999","journal-title":"Comput. Speech Lang."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"302","DOI":"10.1016\/j.cognition.2013.02.013","article-title":"The effect of word predictability on reading time is logarithmic","volume":"128","author":"Smith","year":"2013","journal-title":"Cognition"},{"key":"ref_21","unstructured":"Heafield, K., Pouzyrevsky, I., Clark, J.H., and Koehn, P. (2013, January 4\u20139). Scalable modified Kneser-Ney language model estimation. Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, Sofia, Bulgaria."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"678","DOI":"10.1016\/j.patcog.2010.09.013","article-title":"A global averaging method for dynamic time warping, with applications to clustering","volume":"44","author":"Petitjean","year":"2011","journal-title":"Pattern Recognit."},{"key":"ref_23","unstructured":"Tavenard, R., Faouzi, J., Vandewiele, G., Divo, F., Androz, G., Holtz, C., Payne, M., Yurchak, R., Ru\u00dfwurm, M., and Kolar, K. (2021, September 26). Tslearn: A Machine Learning Toolkit Dedicated to Time-Series Data. Available online: https:\/\/github.com\/rtavenar\/tslearn."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1109\/79.543975","article-title":"The expectation-maximization algorithm","volume":"13","author":"Moon","year":"1996","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Christiansen, M.H., and Chater, N. (2016). The now-or-never bottleneck: A fundamental constraint on language. Behav. Brain Sci., 39.","DOI":"10.1017\/S0140525X1500031X"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Snow, C.E. (1972). Mothers\u2019 speech to children learning language. Child Development, Wiley-VCH.","DOI":"10.2307\/1127555"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Clark, E.V. (2009). First Language Acquisition, Cambridge University Press.","DOI":"10.1017\/CBO9780511806698"},{"key":"ref_28","unstructured":"Du Bois, J.W., Chafe, W.L., Meyer, C., Thompson, S.A., and Martey, N. (2000). Santa Barbara Corpus of Spoken American English, Linguistic Data Consortium."},{"key":"ref_29","unstructured":"MacWhinney, B. (2000). The CHILDES Project: The Database, Psychology Press."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1928","DOI":"10.3758\/s13428-018-1176-7","article-title":"Childes-db: A flexible and reproducible interface to the Child Language Data Exchange System","volume":"51","author":"Sanchez","year":"2019","journal-title":"Behav. Res. Methods"},{"key":"ref_31","unstructured":"Attardi, G., Dei Rossi, S., and Simi, M. (2010, January 17\u201318). The tanl pipeline. Proceedings of the Workshop on Web Services and Processing Pipelines in HLT, Valletta, Malta."},{"key":"ref_32","unstructured":"Dryer, M.S., and Haspelmath, M. (2013). WALS Online, Max Planck Institute for Evolutionary Anthropology."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"501","DOI":"10.1007\/s11222-016-9635-4","article-title":"MIMCA: Multiple imputation for categorical variables with multiple correspondence analysis","volume":"27","author":"Audigier","year":"2017","journal-title":"Stat. Comput."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1838","DOI":"10.1121\/1.4933848","article-title":"Human spoken language diversity and the acoustic adaptation hypothesis","volume":"138","author":"Maddieson","year":"2015","journal-title":"J. Acoust. Soc. Am."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Lupyan, G., and Dale, R. (2010). Language structure is partly determined by social structure. PLoS ONE, 5.","DOI":"10.1371\/journal.pone.0008559"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1016\/j.specom.2005.06.005","article-title":"Duration and spectral balance of intervocalic consonants: A case for efficient communication","volume":"47","year":"2005","journal-title":"Speech Commun."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/10\/1300\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:08:43Z","timestamp":1760166523000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/10\/1300"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,2]]},"references-count":36,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2021,10]]}},"alternative-id":["e23101300"],"URL":"https:\/\/doi.org\/10.3390\/e23101300","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2021,10,2]]}}}