{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,3]],"date-time":"2026-08-03T02:45:16Z","timestamp":1785725116161,"version":"3.56.0"},"reference-count":35,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2026,8,1]],"date-time":"2026-08-01T00:00:00Z","timestamp":1785542400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Research and Planning Fund for Humanities and Social Sciences of the Ministry of Education of China","award":["24YJA740034"],"award-info":[{"award-number":["24YJA740034"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Intell."],"abstract":"<jats:p>Surprisal, the negative log probability of a word given its context, is the dominant computational metric for quantifying reading difficulty and a common item difficulty estimator in reading research. Yet how the language model family, surprisal granularity, and corpus type jointly shape the surprisal\u2013reading time link remains unclear. We conducted a secondary analysis of two public English reading corpora: the Natural Stories Corpus (self-paced reading, 181 readers) and the Provo Corpus (eye tracking with cloze norms). Surprisal was computed from GPT 2 (Small, XL), Llama 2 (7B, 13B), and Llama 3 (8B), together with a 5-gram baseline and human cloze norms. Linear mixed-effects models tested the baseline contribution of neural surprisal, inverse scaling within model families, word-level versus sub-word aggregation, and the linking function shape. Neural surprisal contributed reliable variance above a strong baseline in both corpora. A clear inverse scaling pattern emerged: GPT 2 Small produced the largest fit improvements, exceeding GPT 2 XL, Llama 2 13B, and Llama 3 8B. Word-level aggregation outperformed sub-word aggregation, especially for measures of early lexical access. Non-parametric analyses supported an approximately linear linking function, and cloze norms carried information not fully captured by neural surprisal. These findings show that larger models are not automatically better cognitive models of reading and that surprisal granularity is not a neutral analytic choice.<\/jats:p>","DOI":"10.3390\/jintelligence14080171","type":"journal-article","created":{"date-parts":[[2026,8,3]],"date-time":"2026-08-03T01:43:27Z","timestamp":1785721407000},"page":"171","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Cognitive Processes of Probabilistic Prediction in Reading: Language Model Surprisal Across Model Sizes, Token Granularity, and Reading Paradigms"],"prefix":"10.3390","volume":"14","author":[{"given":"Shuting","family":"Liu","sequence":"first","affiliation":[{"name":"School of Foreign Languages, Hubei University, No. 368 Youyi Avenue, Wuchang District, Wuhan 430062, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yong","family":"Mei","sequence":"additional","affiliation":[{"name":"School of Foreign Languages, Hubei University, No. 368 Youyi Avenue, Wuchang District, Wuhan 430062, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,8,1]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1016\/j.jml.2012.11.001","article-title":"Random effects structure for confirmatory hypothesis testing: Keep it maximal","volume":"68","author":"Barr","year":"2013","journal-title":"Journal of Memory and Language"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"266","DOI":"10.3758\/s13428-025-02774-4","article-title":"A systematic evaluation of Dutch large language models\u2019 surprisal estimates in sentence, paragraph and book reading","volume":"57","author":"Boeve","year":"2025","journal-title":"Behavior Research Methods"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.5070\/G6011190","article-title":"A-maze of Natural Stories: Comprehension and surprisal in the Maze task","volume":"2","author":"Boyce","year":"2023","journal-title":"Glossa Psycholinguistics"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"104174","DOI":"10.1016\/j.jml.2020.104174","article-title":"Word predictability effects are linear, not logarithmic: Implications for probabilistic models of sentence comprehension","volume":"116","author":"Brothers","year":"2021","journal-title":"Journal of Memory and Language"},{"key":"ref_5","first-page":"5190","article-title":"Cloze probability, predictability ratings, and computational estimates for 205 English sentences, aligned with existing EEG and reading time data","volume":"56","author":"Marelli","year":"2024","journal-title":"Behavior Research Methods"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1007\/s10579-020-09503-7","article-title":"The Natural Stories corpus: A reading time corpus of English texts containing rare syntactic constructions","volume":"55","author":"Futrell","year":"2021","journal-title":"Language Resources and Evaluation"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Goodkind, A., and Bicknell, K. (2018). Predictive power of word surprisal for reading times is a linear function of language model quality. Proceedings of the 8th workshop on cognitive modeling and computational linguistics (CMCL 2018), Association for Computational Linguistics.","DOI":"10.18653\/v1\/W18-0102"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Hale, J. (2001). A probabilistic earley parser as a psycholinguistic model. Proceedings of the second meeting of the North American chapter of the Association for Computational Linguistics, Association for Computational Linguistics.","DOI":"10.3115\/1073336.1073357"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Hao, Y., Mendelsohn, S., Sterneck, R., Martinez, R., and Frank, R. (2020). Probabilistic predictions of people perusing: Evaluating metrics of language model performance for psycholinguistic modeling. Proceedings of the workshop on cognitive modeling and computational linguistics, Association for Computational Linguistics.","DOI":"10.18653\/v1\/2020.cmcl-1.10"},{"key":"ref_10","first-page":"350","article-title":"The plausibility of sampling as an algorithmic theory of sentence processing","volume":"7","author":"Hoover","year":"2023","journal-title":"Open Mind"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"104510","DOI":"10.1016\/j.jml.2024.104510","article-title":"Large scale benchmark yields no evidence that language model surprisal explains syntactic disambiguation difficulty","volume":"137","author":"Huang","year":"2024","journal-title":"Journal of Memory and Language"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1202","DOI":"10.1177\/1747021821999007","article-title":"Online inference making and comprehension monitoring in children during reading: Evidence from eye movements","volume":"74","author":"Joseph","year":"2021","journal-title":"Quarterly Journal of Experimental Psychology"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1126","DOI":"10.1016\/j.cognition.2007.05.006","article-title":"Expectation-based syntactic comprehension","volume":"106","author":"Levy","year":"2008","journal-title":"Cognition"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Liu, T., \u0160krjanec, I., and Demberg, V. (2024). Temperature scaling surprisal estimates improve fit to human reading times: But does it do so for the right reasons?. Proceedings of the 62nd annual meeting of the Association for Computational Linguistics (Volume 1: Long papers), Association for Computational Linguistics.","DOI":"10.18653\/v1\/2024.acl-long.519"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"826","DOI":"10.3758\/s13428-017-0908-4","article-title":"The Provo Corpus: A large eye-tracking corpus with predictability norms","volume":"50","author":"Luke","year":"2018","journal-title":"Behavior Research Methods"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1162\/nol_a_00105","article-title":"Strong prediction: Language model surprisal explains multiple N400 effects","volume":"5","author":"Michaelov","year":"2024","journal-title":"Neurobiology of Language"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Nair, S., and Resnik, P. (2023). Words, subwords, and morphemes: What really matters in the surprisal\u2013reading time relationship?. Findings of the Association for Computational Linguistics: EMNLP 2023, Association for Computational Linguistics.","DOI":"10.18653\/v1\/2023.findings-emnlp.752"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1111\/j.2041-210x.2012.00261.x","article-title":"A general and simple method for obtaining R2 from generalized linear mixed-effects models","volume":"4","author":"Nakagawa","year":"2013","journal-title":"Methods in Ecology and Evolution"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Oh, B.-D., Clark, C., and Schuler, W. (2022). Comparison of structural parsers and neural language models as surprisal estimators. Frontiers in Artificial Intelligence, 5.","DOI":"10.3389\/frai.2022.777963"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"336","DOI":"10.1162\/tacl_a_00548","article-title":"Why does surprisal from larger Transformer-based language models provide a poorer fit to human reading times?","volume":"11","author":"Oh","year":"2023","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Oh, B.-D., and Schuler, W. (2025). The impact of token granularity on the predictive power of language model surprisal. Proceedings of the 63rd annual meeting of the Association for Computational Linguistics (Volume 1: Long papers), Association for Computational Linguistics.","DOI":"10.18653\/v1\/2025.acl-long.209"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Oh, B.-D., Zhu, H., and Schuler, W. (2025). The inverse scaling effect of pre-trained language model surprisal is not due to data leakage. Findings of the Association for Computational Linguistics: ACL 2025, Association for Computational Linguistics.","DOI":"10.18653\/v1\/2025.findings-acl.91"},{"key":"ref_23","first-page":"1","article-title":"Integrating cognitive factors and eye movement data in reading predictive models for children with dyslexia and ADHD-I","volume":"16","author":"Pereira","year":"2024","journal-title":"Journal of Eye Movement Research"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1624","DOI":"10.1162\/tacl_a_00603","article-title":"On the effect of anticipation on reading times","volume":"11","author":"Pimentel","year":"2023","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"ref_25","unstructured":"Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I. (Language models are unsupervised multitask learners (OpenAI Technical Report), 2019). Language models are unsupervised multitask learners (OpenAI Technical Report)."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"68","DOI":"10.3758\/s13428-024-02572-4","article-title":"A beginner\u2019s guide to eye tracking for psycholinguistic studies of reading","volume":"57","author":"Schotter","year":"2025","journal-title":"Behavior Research Methods"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"RP108176","DOI":"10.7554\/eLife.108176","article-title":"Executive resources shape the impact of language predictability across the adult lifespan","volume":"14","author":"Schuckart","year":"2026","journal-title":"eLife"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1162\/opmi_a_00119","article-title":"Word frequency and predictability dissociate in naturalistic reading","volume":"8","author":"Shain","year":"2024","journal-title":"Open Mind"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"e2307876121","DOI":"10.1073\/pnas.2307876121","article-title":"Large scale evidence for logarithmic effects of word predictability on reading time","volume":"121","author":"Shain","year":"2024","journal-title":"Proceedings of the National Academy of Sciences of the United States of America"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Shubi, O., Meiri, Y., Hadar, C. A., and Berzak, Y. (2024). Fine grained prediction of reading comprehension from eye movements. Proceedings of the 2024 conference on empirical methods in natural language processing, Association for Computational Linguistics.","DOI":"10.18653\/v1\/2024.emnlp-main.198"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"302","DOI":"10.1016\/j.cognition.2013.02.013","article-title":"The effect of word predictability on reading time is logarithmic","volume":"128","author":"Smith","year":"2013","journal-title":"Cognition"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Pollatsek, A., and Treiman, R. (2015). Reading sentences: Syntactic parsing and semantic interpretation. The Oxford handbook of reading, Oxford University Press.","DOI":"10.1093\/oxfordhb\/9780199324576.013.15"},{"key":"ref_33","unstructured":"Wilcox, E. G., Gauthier, J., Hu, J., Qian, P., and Levy, R. (2020). On the predictive power of neural language models for human real time comprehension behavior. Proceedings of the 42nd annual meeting of the cognitive science society, Cognitive Science Society."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1451","DOI":"10.1162\/tacl_a_00612","article-title":"Testing the predictions of surprisal theory in 11 languages","volume":"11","author":"Wilcox","year":"2023","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"973","DOI":"10.3758\/s13423-024-02588-z","article-title":"Prediction in reading: A review of predictability effects, their theoretical implications, and beyond","volume":"32","author":"Wong","year":"2025","journal-title":"Psychonomic Bulletin & Review"}],"container-title":["Journal of Intelligence"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2079-3200\/14\/8\/171\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,3]],"date-time":"2026-08-03T02:07:02Z","timestamp":1785722822000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2079-3200\/14\/8\/171"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,1]]},"references-count":35,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2026,8]]}},"alternative-id":["jintelligence14080171"],"URL":"https:\/\/doi.org\/10.3390\/jintelligence14080171","relation":{},"ISSN":["2079-3200"],"issn-type":[{"value":"2079-3200","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,1]]}}}