{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T17:06:58Z","timestamp":1790615218334,"version":"4.1.0"},"reference-count":62,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2020,4,16]],"date-time":"2020-04-16T00:00:00Z","timestamp":1586995200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003135","name":"Fondation Fyssen","doi-asserted-by":"publisher","award":["ANR-17-EURE-0017"],"award-info":[{"award-number":["ANR-17-EURE-0017"]}],"id":[{"id":"10.13039\/501100003135","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Sentence comprehension requires inferring, from a sequence of words, the structure of syntactic relationships that bind these words into a semantic representation. Our limited ability to build some specific syntactic structures, such as nested center-embedded clauses (e.g., \u201cThe dog that the cat that the mouse bit chased ran away\u201d), suggests a striking capacity limitation of sentence processing, and thus offers a window to understand how the human brain processes sentences. Here, we review the main hypotheses proposed in psycholinguistics to explain such capacity limitation. We then introduce an alternative approach, derived from our recent work on artificial neural networks optimized for language modeling, and predict that capacity limitation derives from the emergence of sparse and feature-specific syntactic units. Unlike psycholinguistic theories, our neural network-based framework provides precise capacity-limit predictions without making any a priori assumptions about the form of the grammar or parser. Finally, we discuss how our framework may clarify the mechanistic underpinning of language processing and its limitations in the human brain.<\/jats:p>","DOI":"10.3390\/e22040446","type":"journal-article","created":{"date-parts":[[2020,4,16]],"date-time":"2020-04-16T13:01:39Z","timestamp":1587042099000},"page":"446","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":22,"title":["What Limits Our Capacity to Process Nested Long-Range Dependencies in Sentence Comprehension?"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8774-6427","authenticated-orcid":false,"given":"Yair","family":"Lakretz","sequence":"first","affiliation":[{"name":"Cognitive Neuroimaging Unit, Commissariat \u00e0 l\u2019Energie Atomique (CEA), Institut National de la Sant\u00e9 et de la Recherche M\u00e9dicale (INSERM) U992, NeuroSpin Center, 91191 Gif-sur-Yvette, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stanislas","family":"Dehaene","sequence":"additional","affiliation":[{"name":"Cognitive Neuroimaging Unit, Commissariat \u00e0 l\u2019Energie Atomique (CEA), Institut National de la Sant\u00e9 et de la Recherche M\u00e9dicale (INSERM) U992, NeuroSpin Center, 91191 Gif-sur-Yvette, France"},{"name":"Coll\u00e8ge de France, 11 Place Marcelin Berthelot, 75005 Paris, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jean-R\u00e9mi","family":"King","sequence":"additional","affiliation":[{"name":"Laboratoire des Syst\u00e8mes Perceptifs, D\u00e9partement D\u2019\u00e9tudes Cognitives, \u00c9cole Normale Sup\u00e9rieure, PSL University, CNRS, 75005 Paris, France"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,4,16]]},"reference":[{"key":"ref_1","unstructured":"Duncan Luce, R., Bush, R.R., and Galanter, E. (1963). Introduction to the Formal Analysis of Natural Languages, John Wiley & Sons. Handbook of Mathematical Psychology."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"365","DOI":"10.1017\/S0022226707004616","article-title":"Constraints on multiple center-embedding of clauses","volume":"43","author":"Karlsson","year":"2007","journal-title":"J. Linguist."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1207\/s15516709cog0000_7","article-title":"Consequences of the serial nature of linguistic input for sentenial complexity","volume":"29","author":"Grodner","year":"2005","journal-title":"Cogn. Sci."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1016\/0010-0285(91)90003-7","article-title":"Broken agreement","volume":"23","author":"Bock","year":"1991","journal-title":"Cogn. Psychol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"371","DOI":"10.1080\/01690960143000254","article-title":"Subject-verb agreement errors in French and English: The role of syntactic hierarchy","volume":"17","author":"Franck","year":"2002","journal-title":"Lang. Cogn. Process."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1016\/j.cognition.2005.10.003","article-title":"Agreement and movement: A syntactic analysis of attraction","volume":"101","author":"Franck","year":"2006","journal-title":"Cognition"},{"key":"ref_7","unstructured":"Franck, J., Frauenfelder, U.H., and Rizzi, L. (2007). A syntactic analysis of interference in subject\u2013verb agreement. Mit Working Papers in Linguistics, University of Geneva."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1016\/j.lingua.2008.09.002","article-title":"Relativized relatives: Types of intervention in the acquisition of A-bar dependencies","volume":"119","author":"Friedmann","year":"2009","journal-title":"Lingua"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1016\/j.visres.2004.07.037","article-title":"Parafoveal-on-foveal effects in normal reading","volume":"45","author":"Kennedy","year":"2005","journal-title":"Vis. Res."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1037\/0096-3445.135.1.12","article-title":"Tracking the mind during reading: The influence of past, present, and future words on fixation durations","volume":"135","author":"Kliegl","year":"2006","journal-title":"J. Exp. Psychol. Gen."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Hagoort, P. (2019). Cognitive Models of Syntax and Sentence Processing. Human Language: From Genes and Brains to Behavior, The MIT Press.","DOI":"10.7551\/mitpress\/10841.001.0001"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/S0010-0277(98)00034-1","article-title":"Linguistic complexity: Locality of syntactic dependencies","volume":"68","author":"Gibson","year":"1998","journal-title":"Cognition"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"574","DOI":"10.1037\/0278-7393.27.2.574","article-title":"Recency in verb phrase attachment","volume":"27","author":"Pearlmutter","year":"2001","journal-title":"J. Exp. Psychol. Learn. Mem. Cogn."},{"key":"ref_14","first-page":"95","article-title":"The dependency locality theory: A distance-based theory of linguistic complexity","volume":"2000","author":"Gibson","year":"2000","journal-title":"Image Lang. Brain"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Anderson, J.R. (2013). The Architecture of Cognition, Psychology Press.","DOI":"10.4324\/9781315799438"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"375","DOI":"10.1207\/s15516709cog0000_25","article-title":"An activation-based model of sentence processing as skilled memory retrieval","volume":"29","author":"Lewis","year":"2005","journal-title":"Cogn. Sci."},{"key":"ref_17","unstructured":"Chomsky, N. (1986). Barriers, MIT Press."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Hale, J. (2001). A probabilistic Earley parser as a psycholinguistic model. Proceedings of the Second Meeting of the North American Chapter of the Association for Computational Linguistics on Language Technologies, Association for Computational Linguistics.","DOI":"10.3115\/1073336.1073357"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1207\/s15516709cog2002_1","article-title":"A probabilistic model of lexical and syntactic access and disambiguation","volume":"20","author":"Jurafsky","year":"1996","journal-title":"Cogn. Sci."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1126","DOI":"10.1016\/j.cognition.2007.05.006","article-title":"Expectation-based syntactic comprehension","volume":"106","author":"Levy","year":"2008","journal-title":"Cognition"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1016\/0004-3702(90)90007-M","article-title":"Tensor product variable binding and the representation of symbolic structures in connectionist networks","volume":"46","author":"Smolensky","year":"1990","journal-title":"Artif. Intell."},{"key":"ref_22","unstructured":"Gayler, R.W. (2004). Vector symbolic architectures answer Jackendoff\u2019s challenges for cognitive neuroscience. arXiv."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Kanerva, P. (1996). Binary spatter-coding of ordered K-tuples. International Conference on Artificial Neural Networks, Springer.","DOI":"10.1007\/3-540-61510-5_146"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"623","DOI":"10.1109\/72.377968","article-title":"Holographic reduced representations","volume":"6","author":"Plate","year":"1995","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1207\/s15516709cog2302_2","article-title":"Toward a connectionist model of recursion in human linguistic performance","volume":"23","author":"Christiansen","year":"1999","journal-title":"Cogn. Sci."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Lakretz, Y., Kruszewski, G., Desbordes, T., Hupkes, D., Dehaene, S., and Baroni, M. (2019). The emergence of number and syntax units in LSTM language models. arXiv.","DOI":"10.18653\/v1\/N19-1002"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"E3669","DOI":"10.1073\/pnas.1701590114","article-title":"Neurophysiological dynamics of phrase-structure building during sentence processing","volume":"114","author":"Nelson","year":"2017","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1016\/j.tics.2014.01.002","article-title":"Characterizing the dynamics of mental representations: The temporal generalization method","volume":"18","author":"King","year":"2014","journal-title":"Trends Cogn. Sci."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1207\/s15516709cog1402_1","article-title":"Finding structure in time","volume":"14","author":"Elman","year":"1990","journal-title":"Cogn. Sci."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Gulordava, K., Bojanowski, P., Grave, E., Linzen, T., and Baroni, M. (2018, January 1). Colorless green recurrent networks dream hierarchically. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, LA, USA.","DOI":"10.18653\/v1\/N18-1108"},{"key":"ref_31","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019, January 1). Bert: Pre-Training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"521","DOI":"10.1162\/tacl_a_00115","article-title":"Assessing the ability of LSTMs to learn syntax-sensitive dependencies","volume":"4","author":"Linzen","year":"2016","journal-title":"Trans. Assoc. Comput. Linguist."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Marvin, R., and Linzen, T. (2018). Targeted syntactic evaluation of language models. arXiv.","DOI":"10.18653\/v1\/D18-1151"},{"key":"ref_34","unstructured":"Futrell, R., Wilcox, E., Morita, T., and Levy, R. (2018). RNNs as psycholinguistic subjects: Syntactic state and grammatical dependency. arXiv."},{"key":"ref_35","unstructured":"Linzen, T., and Leonard, B. (2018, January 25\u201328). Distinct patterns of syntactic agreement errors in recurrent networks and humans. Proceedings of the CogSci 2018\u201440th Annual Meeting of the Cognitive Science Society, Madison, WI, USA."},{"key":"ref_36","unstructured":"Goldberg, Y. (2019). Assessing BERT\u2019s Syntactic Abilities. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1162\/tacl_a_00254","article-title":"Analysis methods in neural language processing: A survey","volume":"7","author":"Belinkov","year":"2019","journal-title":"Trans. Assoc. Comput. Linguist."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"907","DOI":"10.1613\/jair.1.11196","article-title":"Visualisation and \u2018diagnostic classifiers\u2019 reveal how recurrent and recursive neural networks process hierarchical structure","volume":"61","author":"Hupkes","year":"2018","journal-title":"J. Artif. Intell. Res."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Giulianelli, M., Harding, J., Mohnert, F., Hupkes, D., and Zuidema, W. (2018, January 1). Under the hood: Using diagnostic classifiers to investigate and improve how language models track agreement information. Proceedings of the EMNLP BlackboxNLP Workshop 2018, Brussels, Belgium.","DOI":"10.18653\/v1\/W18-5426"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1006\/jmla.2001.2836","article-title":"Processing subject and object relative clauses: Evidence from eye movements","volume":"47","author":"Traxler","year":"2002","journal-title":"J. Mem. Lang."},{"key":"ref_41","unstructured":"Lakretz, Y., Hupkes, D., Vergallito, A., Marelli, M., Baroni, M., and Dehaene, S. (2020). Processing of nested dependencies by humans and neural language models. in preparation."},{"key":"ref_42","unstructured":"Lakretz, Y., Desbordes, T., King, J.R., Crabb\u00e9, B., Oquab, M., and Dehaene, S. (2020). Can RNNs learn Recursive Nested Subject-Verb Agreements?. ACL, under review."},{"key":"ref_43","unstructured":"Joulin, A., and Mikolov, T. (2015, January 7\u201312). Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets. Proceedings of the 28th International Conference on Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"591","DOI":"10.1145\/355598.362773","article-title":"Transition network grammars for natural language analysis","volume":"13","author":"Woods","year":"1970","journal-title":"Commun. ACM"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"225","DOI":"10.1080\/016909699386293","article-title":"Memory limitations and structural forgetting: The perception of complex ungrammatical sentences as grammatical","volume":"14","author":"Gibson","year":"1999","journal-title":"Lang. Cogn. Process."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1016\/j.neuron.2015.09.019","article-title":"The neural representation of sequences: From transition probabilities to algebraic patterns and linguistic trees","volume":"88","author":"Dehaene","year":"2015","journal-title":"Neuron"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"1102","DOI":"10.1038\/nature03687","article-title":"Invariant visual representation by single neurons in the human brain","volume":"435","author":"Quiroga","year":"2005","journal-title":"Nature"},{"key":"ref_48","unstructured":"Chomsky, N. (2006). On cognitive structures and their development: A reply to Piaget. Philosophy of Mind: Classical Problems\/Contemporary Issues, The MIT Press."},{"key":"ref_49","unstructured":"Lake, B., and Baroni, M. (2018, January 10\u201315). Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks. Proceedings of the ICML 2018\u2014International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Chomsky, N. (1957). Syntactic Structures, Mouton.","DOI":"10.1515\/9783112316009"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1111\/j.1755-2567.1970.tb00434.x","article-title":"Universal Grammar","volume":"36","author":"Montague","year":"1970","journal-title":"Theoria"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/0010-0277(88)90031-5","article-title":"Connectionism and cognitive architecture: A critical analysis","volume":"28","author":"Fodor","year":"1988","journal-title":"Cognition"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Fodor, J., and Lepore, E. (2002). The Compositionality Papers, Oxford University Press.","DOI":"10.1093\/oso\/9780199252152.001.0001"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Wehbe, L., Murphy, B., Talukdar, P., Fyshe, A., Ramdas, A., and Mitchell, T. (2014). Simultaneously uncovering the patterns of brain regions involved in different story reading subprocesses. PLoS ONE, 9.","DOI":"10.1371\/journal.pone.0112575"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"453","DOI":"10.1038\/nature17637","article-title":"Natural speech reveals the semantic maps that tile human cerebral cortex","volume":"532","author":"Huth","year":"2016","journal-title":"Nature"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Jain, S., and Huth, A. (2018, January 3\u20138). Incorporating context into language encoding models for fmri. Proceedings of the Thirty-Second Annual Conference on Neural Information Processing Systems, Montreal, QC, Canada.","DOI":"10.1101\/327601"},{"key":"ref_57","unstructured":"Toneva, M., and Wehbe, L. (2019, January 8\u201314). Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain). Proceedings of the Thirty-Third Annual Conference on Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_58","unstructured":"Hebb, D.O. (1949). The Organization of Behavior: A Neuropsychological Theory, J. Wiley."},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1146\/annurev.neuro.24.1.139","article-title":"Synaptic modification by correlated activity: Hebb\u2019s postulate revisited","volume":"24","author":"Bi","year":"2001","journal-title":"Ann. Rev. Neurosci."},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"533","DOI":"10.1038\/323533a0","article-title":"Learning representations by back-propagating errors","volume":"323","author":"Rumelhart","year":"1986","journal-title":"Nature"},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"276","DOI":"10.1080\/02643290442000329","article-title":"Frontal latching networks: A possible neural basis for infinite recursion","volume":"22","author":"Treves","year":"2005","journal-title":"Cogn. Neuropsychol."},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Russo, E., Pirmoradian, S., and Treves, A. (2011). Associative latching dynamics vs. syntax. Advances in Cognitive Neurodynamics (II), Springer.","DOI":"10.1007\/978-90-481-9695-1_17"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/4\/446\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T13:21:12Z","timestamp":1760361672000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/4\/446"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,4,16]]},"references-count":62,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2020,4]]}},"alternative-id":["e22040446"],"URL":"https:\/\/doi.org\/10.3390\/e22040446","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,4,16]]}}}