{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T02:42:29Z","timestamp":1760236949330,"version":"build-2065373602"},"reference-count":63,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2020,1,31]],"date-time":"2020-01-31T00:00:00Z","timestamp":1580428800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>What are relevant levels of description when investigating human language? How are these levels connected to each other? Does one description yield smoothly into the next one such that different models lie naturally along a hierarchy containing each other? Or, instead, are there sharp transitions between one description and the next, such that to gain a little bit accuracy it is necessary to change our framework radically? Do different levels describe the same linguistic aspects with increasing (or decreasing) accuracy? Historically, answers to these questions were guided by intuition and resulted in subfields of study, from phonetics to syntax and semantics. Need for research at each level is acknowledged, but seldom are these different aspects brought together (with notable exceptions). Here, we propose a methodology to inspect empirical corpora systematically, and to extract from them, blindly, relevant phenomenological scales and interactions between them. Our methodology is rigorously grounded in information theory, multi-objective optimization, and statistical physics. Salient levels of linguistic description are readily interpretable in terms of energies, entropies, phase transitions, or criticality. Our results suggest a critical point in the description of human language, indicating that several complementary models are simultaneously necessary (and unavoidable) to describe it.<\/jats:p>","DOI":"10.3390\/e22020165","type":"journal-article","created":{"date-parts":[[2020,1,31]],"date-time":"2020-01-31T11:55:56Z","timestamp":1580471756000},"page":"165","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Criticality in Pareto Optimal Grammars?"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0045-8145","authenticated-orcid":false,"given":"Lu\u00eds F","family":"Seoane","sequence":"first","affiliation":[{"name":"Instituto de F\u00edsica Interdisciplinar y Sistemas Complejos IFISC (CSIC-UIB), Campus UIB, 07122 Palma de Mallorca, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6974-1008","authenticated-orcid":false,"given":"Ricard","family":"Sol\u00e9","sequence":"additional","affiliation":[{"name":"ICREA-Complex Systems Lab, Universitat Pompeu Fabra (GRIB), Dr Aiguader 80, 08003 Barcelona, Spain"},{"name":"Institut de Biologia Evolutiva, CSIC-UPF, Pg Maritim de la Barceloneta 37, 08003 Barcelona, Spain"},{"name":"Santa Fe Institute, 1399 Hyde Park Road, Santa Fe, NM 87501, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,1,31]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"561","DOI":"10.1098\/rspb.2004.2957","article-title":"The consequences of Zipf\u2019s law for syntax and symbolic reference","volume":"272","author":"Ferrer","year":"2005","journal-title":"Proc. R. Soc. B"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1038\/434289a","article-title":"Language: Syntax for free?","volume":"434","year":"2005","journal-title":"Nature"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"371","DOI":"10.1142\/S0219525909002192","article-title":"The ontogeny of scale-free syntax networks: Phase transitions in early language acquisition","volume":"12","author":"Valverde","year":"2009","journal-title":"Adv. Complex Syst."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"679","DOI":"10.1142\/S021812741002596X","article-title":"The structure of phonological networks across multiple languages","volume":"20","author":"Arbesman","year":"2010","journal-title":"Int. J. Bifurcat. Chaos"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1002\/cplx.20305","article-title":"Language networks: Their structure, function, and evolution","volume":"15","author":"Valverde","year":"2010","journal-title":"Complexity"},{"key":"ref_6","first-page":"5","article-title":"Ambiguity in language networks","volume":"32","author":"Seoane","year":"2015","journal-title":"Linguist. Rev."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41598-018-28820-0","article-title":"The morphospace of language networks","volume":"8","author":"Seoane","year":"2018","journal-title":"Sci. Rep."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1016\/j.physa.2016.03.082","article-title":"Multilayer network of language: A unified framework for structural analysis of linguistic subsystems","volume":"457","author":"Margan","year":"2016","journal-title":"Phys. A"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1017\/S0140525X08004998","article-title":"Language as shaped by the brain","volume":"31","author":"Christiansen","year":"2008","journal-title":"Behav. Brain Sci."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Christiansen, M.H., and Chater, N. (2016). Creating Language: Integrating Evolution, Acquisition, and Processing, MIT Press.","DOI":"10.7551\/mitpress\/10406.001.0001"},{"key":"ref_11","unstructured":"Tishby, N., Pereira, F.C., and Bialek, W. (2000). The information bottleneck method. arXiv."},{"key":"ref_12","unstructured":"Still, S., Bialek, W., and Bottou, L. (2003). Geometric clustering using the information bottleneck method. Advances in Neural Information Processing Systems, MIT Press."},{"key":"ref_13","unstructured":"Still, S., and Crutchfield, J.P. (2007). Structure or Noise?. arXiv."},{"key":"ref_14","unstructured":"Still, S., Crutchfield, J.P., and Ellison, C.J. (2007). Optimal Causal Inference, Santa Fe Institute. Santa Fe Institute Working Paper #2007-08-024."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"968","DOI":"10.3390\/e16020968","article-title":"Information bottleneck approach to predictive inference","volume":"16","author":"Still","year":"2014","journal-title":"Entropy"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"2001","journal-title":"Bell Syst. Tech. J."},{"key":"ref_17","unstructured":"Shannon, C.E., and Weaver, W. (1949). The Mathematical Theory of Communication, Univ of Illinois Press."},{"key":"ref_18","unstructured":"Shalizi, C.R., and Moore, C. (2003). What is a macrostate? Subjective observations and objective dynamics. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"026203","DOI":"10.1103\/PhysRevE.73.026203","article-title":"Coarse-graining of cellular automata, emergence, and the predictability of complex systems","volume":"73","author":"Israeli","year":"2006","journal-title":"Phys. Rev. E"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1142\/S0219525910002542","article-title":"A method for finding aggregated representations of linear dynamical systems","volume":"13","author":"Jacobi","year":"2010","journal-title":"Adv. Complex Syst."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1450007","DOI":"10.1142\/S0219525914500076","article-title":"Comparison between different methods of level identification","volume":"17","author":"Pfante","year":"2014","journal-title":"Adv. Complex Syst."},{"key":"ref_22","unstructured":"Wolpert, D.H., Grochow, J.A., Libby, E., and DeDeo, S. (2014). Optimal High-Level Descriptions of Dynamical Systems, Santa Fe Institute. Santa Fe Institute working paper #2015-06-017."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1109\/MCI.2006.1597059","article-title":"Twenty years of evolutionary multi-objective optimization: A historical view of the field","volume":"1","author":"Coello","year":"2006","journal-title":"IEEE Comput. Intell. Mag."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1002\/cplx.21426","article-title":"Optimization of multiple criteria: Pareto efficiency and fast heuristics should be more popular than they are","volume":"18","author":"Schuster","year":"2012","journal-title":"Complexity"},{"key":"ref_25","unstructured":"Seoane, L.F. (2016). Multiobjetive Optimization in Models of Synthetic and Natural Living Systems. [PhD Thesis, Universitat Pompeu Fabra]."},{"key":"ref_26","unstructured":"Seoane, L.F., and Sol\u00e9, R. (2013). A multiobjective optimization approach to statistical mechanics. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"032807","DOI":"10.1103\/PhysRevE.92.032807","article-title":"Phase transitions in Pareto optimal complex networks","volume":"92","author":"Seoane","year":"2015","journal-title":"Phys. Rev. E"},{"key":"ref_28","unstructured":"Seoane, L.F., and Sol\u00e9, R. (2015). Systems poised to criticality through Pareto selective forces. arXiv."},{"key":"ref_29","unstructured":"Seoane, L.F., and Sol\u00e9, R. (2014). Multiobjective optimization and phase transitions. Proceedings of ECCS, Springer."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/0167-2789(84)90245-8","article-title":"Universality and complexity in cellular automata","volume":"10","author":"Wolfram","year":"1984","journal-title":"Phys. D"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1016\/0167-2789(90)90064-V","article-title":"Computation at the edge of chaos: Phase transitions and emergent computation","volume":"42","author":"Langton","year":"1990","journal-title":"Phys. D"},{"key":"ref_32","unstructured":"Mitchell, M., Hraber, P., and Crutchfield, J.P. (1993). Revisiting the edge of chaos: Evolving cellular automata to perform computations. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Bak, P. (1996). How Nature Works: The Science of Self-Organized Criticality, Springer Science & Business Media.","DOI":"10.1007\/978-1-4757-5426-1"},{"key":"ref_34","unstructured":"Kauffman, S. (1996). At Home in the Universe: The Search for the Laws of Self-Organization and Complexity, Oxford University Press."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Legenstein, R., and Maass, W. (2007). What makes a dynamical system computationally powerful. New Directions in Statistical Signal Processing: From Systems to Brain, MIT Press.","DOI":"10.7551\/mitpress\/4977.003.0008"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Sol\u00e9, R. (2011). Phase Transitions, Princeton U. Press.","DOI":"10.1515\/9781400838929"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"268","DOI":"10.1007\/s10955-011-0229-4","article-title":"Are biological systems poised at criticality?","volume":"144","author":"Mora","year":"2011","journal-title":"J. Stat. Phys."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"031001","DOI":"10.1103\/RevModPhys.90.031001","article-title":"Colloquium: Criticality and dynamical scaling in living systems","volume":"90","year":"2018","journal-title":"Rev. Mod. Phys."},{"key":"ref_39","unstructured":"(2020, January 28). Corpus of Contemporary American English. Available online: http:\/\/corpus.byu.edu\/coca\/."},{"key":"ref_40","unstructured":"(2020, January 28). NLTK 3.4.5 documentation. Available online: http:\/\/www.nltk.org\/."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"620","DOI":"10.1103\/PhysRev.106.620","article-title":"Information theory and statistical mechanics","volume":"106","author":"Jaynes","year":"1957","journal-title":"Phys. Rev."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1103\/PhysRev.108.171","article-title":"Information theory and statistical mechanics. II","volume":"108","author":"Jaynes","year":"1957","journal-title":"Phys. Rev."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"5405","DOI":"10.1073\/pnas.1001705107","article-title":"Maximum entropy models for antibody diversity","volume":"107","author":"Mora","year":"2010","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"066119","DOI":"10.1103\/PhysRevE.81.066119","article-title":"Statistical mechanics of letters in words","volume":"81","author":"Stephens","year":"2010","journal-title":"Phys. Rev. E"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Harte, J. (2011). Maximum Entropy and Ecology: A Theory of Abundance, Distribution, and Energetics, Oxford University Press.","DOI":"10.1093\/acprof:oso\/9780199593415.001.0001"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"P03011","DOI":"10.1088\/1742-5468\/2013\/03\/P03011","article-title":"The simplest maximum entropy model for collective behavior in a neural network","volume":"2013","author":"Marre","year":"2013","journal-title":"J. Stat. Mech."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"018701","DOI":"10.1103\/PhysRevLett.110.018701","article-title":"Statistical thermodynamics of natural images","volume":"110","author":"Stephens","year":"2013","journal-title":"Phys. Rev. Lett."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"11508","DOI":"10.1073\/pnas.1514188112","article-title":"Thermodynamics and signatures of criticality in a network of neurons","volume":"112","author":"Mora","year":"2015","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"275","DOI":"10.1007\/s10955-015-1253-6","article-title":"Statistical mechanics of the US Supreme Court","volume":"160","author":"Lee","year":"2015","journal-title":"J. Stat. Phys."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"220601","DOI":"10.1103\/PhysRevLett.107.220601","article-title":"New method for parameter estimation in probabilistic models: minimum probability flow","volume":"107","author":"Battaglino","year":"2011","journal-title":"Phys. Rev. Lett."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Chomsky, N., and Chomsky, N. (2002). An interview on minimalism. On Nature and Language, Cambridge University Press.","DOI":"10.1017\/CBO9780511613876"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"1569","DOI":"10.1126\/science.298.5598.1569","article-title":"The faculty of language: what is it, who has it, and how did it evolve?","volume":"298","author":"Hauser","year":"2002","journal-title":"Science"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Berwick, R.C., and Chomsky, N. (2016). Why only Us: Language and Evolution, MIT Press.","DOI":"10.7551\/mitpress\/9780262034241.001.0001"},{"key":"ref_54","unstructured":"Zipf, G.K. (2020, January 28). Human behavior and the principle of least effort. Available online: https:\/\/psycnet.apa.org\/record\/1950-00412-000."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Altmann, E.G., and Gerlach, M. (2016). Statistical laws in linguistics. Creativity and Universality in Language, Springer.","DOI":"10.1007\/978-3-319-24403-7_2"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"788","DOI":"10.1073\/pnas.0335980100","article-title":"Least effort and the origins of scaling in human language","volume":"100","author":"Ferrer","year":"2003","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"036115","DOI":"10.1103\/PhysRevE.83.036115","article-title":"Emergence of Zipf\u2019s law in the evolution of communication","volume":"83","author":"Fortuny","year":"2011","journal-title":"Phys. Rev. E"},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"20180395","DOI":"10.1098\/rsif.2018.0395","article-title":"Zipf\u2019s law, unbounded complexity and open-ended evolution","volume":"15","author":"Seoane","year":"2018","journal-title":"J. R. Soc. Interface"},{"key":"ref_59","unstructured":"Bickerton, D. (1992). Language and Species, University of Chicago Press."},{"key":"ref_60","unstructured":"Deacon, T.W. (1998). The Symbolic Species: The Co-Evolution of Language and the Brain, WW Norton & Company."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1103\/PhysRevLett.63.105","article-title":"Inferring statistical complexity","volume":"63","author":"Crutchfield","year":"1989","journal-title":"Phys. Rev. Let."},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/0167-2789(94)90273-9","article-title":"The calculi of emergence: computation, dynamics and induction","volume":"75","author":"Crutchfield","year":"1994","journal-title":"Physica D"},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"275","DOI":"10.1103\/PhysRevE.59.275","article-title":"Thermodynamic depth of causal states: Objective complexity via minimal representations","volume":"59","author":"Crutchfield","year":"1999","journal-title":"Phys. Rev. E"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/2\/165\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T08:53:28Z","timestamp":1760172808000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/2\/165"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,1,31]]},"references-count":63,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2020,2]]}},"alternative-id":["e22020165"],"URL":"https:\/\/doi.org\/10.3390\/e22020165","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2020,1,31]]}}}