{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T18:06:36Z","timestamp":1775671596645,"version":"3.50.1"},"reference-count":64,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2011,11,23]],"date-time":"2011-11-23T00:00:00Z","timestamp":1322006400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/3.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>It has been argued that the mental representation resulting from sentence comprehension is not (just) an abstract symbolic structure but a \u201cmental simulation\u201d of the state-of-affairs described by the sentence. We present a particular formalization of this theory and show how it gives rise to quantifications of the amount of syntactic and semantic information conveyed by each word in a sentence. These information measures predict simulated word-processing times in a dynamic connectionist model of sentence comprehension as mental simulation. A quantitatively similar relation between information content and reading time is known to be present in human reading-time data.<\/jats:p>","DOI":"10.3390\/info2040672","type":"journal-article","created":{"date-parts":[[2011,11,23]],"date-time":"2011-11-23T11:12:28Z","timestamp":1322046748000},"page":"672-696","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Sentence Comprehension as Mental Simulation: An Information-Theoretic Perspective"],"prefix":"10.3390","volume":"2","author":[{"given":"Stefan L.","family":"Frank","sequence":"first","affiliation":[{"name":"Department of Cognitive, Perceptual and Brain Sciences, University College London, 26 Bedford Way, London WC1H 0AP, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gabriella","family":"Vigliocco","sequence":"additional","affiliation":[{"name":"Department of Cognitive, Perceptual and Brain Sciences, University College London, 26 Bedford Way, London WC1H 0AP, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2011,11,23]]},"reference":[{"key":"ref_1","first-page":"1","article-title":"Interpretation-based processing: A unified theory of semantic sentence comprehension","volume":"28","author":"Budiu","year":"2004","journal-title":"Cogn. Sci."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1037\/0033-295X.95.2.163","article-title":"The role of knowledge in discourse comprehension: A construction-integration model","volume":"95","author":"Kintsch","year":"1988","journal-title":"Psychol. Rev."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"794","DOI":"10.1111\/j.1551-6709.2009.01033.x","article-title":"A probabilistic model of semantic plausibility in sentence processing","volume":"33","author":"Crocker","year":"2009","journal-title":"Cogn. Sci."},{"key":"ref_4","unstructured":"Innis, R.E. (1903\/1985). Semiotics: An Introductory Anthology, Indiana University Press."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/S1389-0417(01)00040-7","article-title":"Bootstrapping in miniature language acquisition","volume":"3","author":"Desai","year":"2002","journal-title":"Cogn. Syst. Res."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2273","DOI":"10.1016\/j.neucom.2006.02.027","article-title":"A model of frame and verb compliance in language acquisition","volume":"70","author":"Desai","year":"2007","journal-title":"Neurocomputing"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"449","DOI":"10.1111\/j.1551-6709.2009.01019.x","article-title":"Learning to attend: A connectionist model of situated language comprehension","volume":"33","author":"Mayberry","year":"2009","journal-title":"Cogn. Sci."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Wermter, S., and Sun, R. (2000). Hybrid Neural Systems, Springer.","DOI":"10.1007\/10719871"},{"key":"ref_9","unstructured":"Fodor, J.A. (1975). The Language of Thought, Harvard University Press."},{"key":"ref_10","unstructured":"Johnson-Laird, P.N. (1983). Mental Models, Cambridge University Press."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"577","DOI":"10.1017\/S0140525X99002149","article-title":"Perceptual symbol systems","volume":"22","author":"Barsalou","year":"1999","journal-title":"Behav. Brain Sci."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1006\/jmla.2000.2714","article-title":"Symbol grounding and meaning: A comparison of high-dimensional and embodied theories of meaning","volume":"43","author":"Glenberg","year":"2000","journal-title":"J. Mem. Lang."},{"key":"ref_13","unstructured":"Vega, M.D., Glenberg, A.M., and Graesser, A.C. (2008). Symbols, Embodiment, and Meaning: Debates on Meaning and Cognition, Oxford University Press."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1111\/1467-9280.00326","article-title":"The effect of implied orientation derived from verbal context on picture recognition","volume":"12","author":"Stanfield","year":"2001","journal-title":"Psychol. Sci."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1080\/09540099408915708","article-title":"Perceptually grounded language learning: Part 2-DETE: A neural\/procedural model","volume":"6","author":"Nenov","year":"1994","journal-title":"Connect. Sci."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"\u00d6stman, J., and Fried, M. (2005). Construction Grammars: Cognitive Grounding and Theoretical Extensions, John Benjamins.","DOI":"10.1075\/cal.3"},{"key":"ref_17","first-page":"875","article-title":"Modeling knowledge-based inferences in story comprehension","volume":"27","author":"Frank","year":"2003","journal-title":"Cogn. Sci."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1307","DOI":"10.3758\/BF03193603","article-title":"Coherence-driven resolution of referential ambiguity: A computational model","volume":"35","author":"Frank","year":"2007","journal-title":"Mem. Cogn."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"358","DOI":"10.1016\/j.cognition.2008.11.013","article-title":"Connectionist semantic systematicity","volume":"110","author":"Frank","year":"2009","journal-title":"Cognition"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Hale, J. (2001, January 2\u20137). A Probabilistic Early Parser as a Psycholinguistic Model. Pittsburgh, PA, USA.","DOI":"10.3115\/1073336.1073357"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1023\/A:1022492123056","article-title":"The information conveyed by words","volume":"32","author":"Hale","year":"2003","journal-title":"J. Psycholinguist. Res."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"399","DOI":"10.1111\/j.1551-6709.2010.01145.x","article-title":"What a rational parser would do","volume":"35","author":"Hale","year":"2011","journal-title":"Cogn. Sci."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1126","DOI":"10.1016\/j.cognition.2007.05.006","article-title":"Expectation-based syntactic comprehension","volume":"106","author":"Levy","year":"2008","journal-title":"Cognition"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"438","DOI":"10.1126\/science.1095455","article-title":"Integration of word meaning and world knowledge in language comprehension","volume":"304","author":"Hagoort","year":"2004","journal-title":"Science"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"443","DOI":"10.1037\/0278-7393.31.3.443","article-title":"Anticipating upcoming words in discourse: Evidence from ERPs and reading times","volume":"31","author":"Brown","year":"2005","journal-title":"J. Exp. Psychol. Learni. Mem. Cogn."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1016\/j.brainres.2007.09.014","article-title":"Immediate integration of novel meanings: N400 support for an embodied view of language comprehension","volume":"1183","author":"Chwilla","year":"2007","journal-title":"Brain Res."},{"key":"ref_27","unstructured":"From hereon, we leave implicit that by \u201cworld\u201d we mean the non-linguistic part of the world."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1016\/j.jml.2010.08.004","article-title":"Effects of event knowledge in processing verbal arguments","volume":"63","author":"Bicknell","year":"2010","journal-title":"J. Mem. Lang."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1016\/j.jml.2006.07.005","article-title":"The interplay of discourse congruence and lexical association during sentence processing: Evidence from ERPs and eye tracking","volume":"56","author":"Camblin","year":"2007","journal-title":"J. Mem. Lang."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"913","DOI":"10.1037\/a0022964","article-title":"Event-based plausibility immediately influences on-line language comprehension","volume":"37","author":"Matsuki","year":"2011","journal-title":"J. Exp. Psychol. Learn. Mem. Cogn."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"581","DOI":"10.1023\/A:1026416225168","article-title":"Priming in sentence processing: Intralexical spreading activation, schemas, and situation models","volume":"29","author":"Traxler","year":"2000","journal-title":"J. Psycholinguist. Res."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Oaksford, M., and Chater, N. (2007). Bayesian Rationality: The Probabilistic Approach to Human Reasoning, Oxford University Press.","DOI":"10.1093\/acprof:oso\/9780198524496.001.0001"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1207\/s15516709cog1402_1","article-title":"Finding structure in time","volume":"14","author":"Elman","year":"1990","journal-title":"Cogn. Sci."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1207\/s15516709cog2302_2","article-title":"Toward a connectionist model of recursion in human linguistic performance","volume":"23","author":"Christiansen","year":"1999","journal-title":"Cogn. Sci."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1111\/j.1467-9922.2009.00538.x","article-title":"A usage-based approach to recursion in sentence processing","volume":"59","author":"Christiansen","year":"2009","journal-title":"Lang. Learn."},{"key":"ref_36","unstructured":"Taatgen, N.A., and van Rijn, H. (August, January 29). Surprisal-Based Comparison Between a Symbolic and a Connectionist Model of Sentence Processings. Amsterdam, The Netherlands."},{"key":"ref_37","unstructured":"Rohde, D.L.T. (2002). Connectionist Model of Sentence Comprehension and Production. [PhD thesis, Carnegie Mellon University]."},{"key":"ref_38","unstructured":"Held, C., Knauff, M., and Vosgerau, G. (2006). Mental Models and the Mind, Elsevier."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"643","DOI":"10.1207\/s15516709cog0000_64","article-title":"Uncertainty about the rest of the sentence","volume":"30","author":"Hale","year":"2006","journal-title":"Cogn. Sci."},{"key":"ref_40","unstructured":"If the system has perfect knowledge of the language, a word that it considers impossible will (by definition) never occur, so surprisal cannot reach infinity. In practice, when there is no perfect knowledge, infinite surprisal is prevented by smoothing the word probabilities so that they are always greater than zero."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1","DOI":"10.16910\/jemr.2.1.1","article-title":"Parsing costs as predictors of reading difficulty: An evaluation using the Potsdam Sentence Corpus","volume":"2","author":"Boston","year":"2008","journal-title":"J. Eye Mov. Res."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1080\/01690965.2010.492228","article-title":"Parallel processing and sentence comprehension difficulty","volume":"26","author":"Boston","year":"2011","journal-title":"Lang. Cogn. Process."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1016\/j.cognition.2008.07.008","article-title":"Data from eye-tracking corpora as evidence for theories of syntactic processing complexity","volume":"109","author":"Demberg","year":"2008","journal-title":"Cognition"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"829","DOI":"10.1177\/0956797611409589","article-title":"Insensitivity of the human sentence-processing system to hierarchical structure","volume":"22","author":"Frank","year":"2011","journal-title":"Psychol. Sci."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Roark, B., Bachrach, A., Cardenas, C., and Pallier, C. (2009, January 6\u20137). Deriving Lexical and Syntactic Expectation-Based Measures For Psycholinguistic Modeling via Incremental Top-Down Parsing. Singapore.","DOI":"10.3115\/1699510.1699553"},{"key":"ref_46","unstructured":"Love, B.C., McRae, K., and Sloutsky, V.M. (, January July). Optimal Processing Times in Reading: A Formal Model and Empirical Investigation. Washington, DC, USA."},{"key":"ref_47","unstructured":"Wu, S., Bachrach, A., Cardenas, C., and Schuler, W. (2010, January 11\u201316). Complexity Metrics in an Incremental Right-Corner Parser. Uppsala, Sweden."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1093\/bjps\/IV.14.147","article-title":"Semantic information","volume":"4","author":"Carnap","year":"1953","journal-title":"Br. J. Philos. Sci."},{"key":"ref_49","unstructured":"Frank, S.L. (, January July). Uncertainty Reduction as a Measure of Cognitive Processing Effort. Uppsala, Sweden."},{"key":"ref_50","unstructured":"Frank, S.L. Uncertainty reduction as a measure of cognitive effort in sentence comprehension. Manuscript in preparation."},{"key":"ref_51","unstructured":"This is probably an artifact of the artificial nature of the language or of the fact that the true sentence probabilities are known rather than estimated by a model. On a corpus of newspaper texts, the correlation between the two syntactic information measures was found to be only around .25 [49] and it was even lower on a random selection of sentences from novels [50]."},{"key":"ref_52","unstructured":"The correlations coefficients of Table 3 are based on this reduced data set."},{"key":"ref_53","unstructured":"Traxler, M., and Gernsbacher, M. (2006). Handbook of Psycholinguistics, Academic Press. [2nd ed.]."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1016\/S0749-596X(03)00023-8","article-title":"The time-course of prediction in incremental sentence processing: Evidence from anticipatory eye movements","volume":"49","author":"Kamide","year":"2003","journal-title":"J. Mem. Lang."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"481","DOI":"10.1207\/s15516709cog0000_65","article-title":"The coordinated interplay of scene, utterance, and world knowledge: Evidence from eye-tracking","volume":"30","author":"Knoeferle","year":"2006","journal-title":"Cogn. Sci."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1126\/science.283.5398.77","article-title":"Rule learning by seven-month-old infants","volume":"283","author":"Marcus","year":"1999","journal-title":"Science"},{"key":"ref_57","unstructured":"Pinker, S. (1984). Language Learnability and Language Development, Harvard University Press."},{"key":"ref_58","unstructured":"MacWhinney, B. (1987). Mechanisms of Language Acquisition, Erlbaum."},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1207\/s15327817la0101_2","article-title":"The structural sources of verb meanings","volume":"1","author":"Gleitman","year":"1990","journal-title":"Lang. Acquis."},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1207\/s15473341lld0101_4","article-title":"Hard words","volume":"1","author":"Gleitman","year":"2005","journal-title":"Lang. Learn. Dev."},{"key":"ref_61","unstructured":"Hoff, E., and Shatz, M. (2008). Blackwell Handbook of Language Development, Blackwell Publishing Ltd."},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1002\/wcs.17","article-title":"Syntactic bootstrapping","volume":"1","author":"Fisher","year":"2010","journal-title":"WIREs Cogn. Sci."},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"463","DOI":"10.1037\/a0016261","article-title":"Integrating experiential and distributional data to learn semantic representations","volume":"116","author":"Andrews","year":"2009","journal-title":"Psychol. Rev."},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"234","DOI":"10.1016\/j.actpsy.2009.10.010","article-title":"Combining feature norms and text data with topic models","volume":"3","author":"Steyvers","year":"2010","journal-title":"Acta Psychol."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/2\/4\/672\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T21:58:06Z","timestamp":1760219886000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/2\/4\/672"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2011,11,23]]},"references-count":64,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2011,12]]}},"alternative-id":["info2040672"],"URL":"https:\/\/doi.org\/10.3390\/info2040672","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2011,11,23]]}}}