{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,24]],"date-time":"2026-09-24T04:39:26Z","timestamp":1790224766299,"version":"4.1.0"},"reference-count":151,"publisher":"The Royal Society","issue":"1932","license":[{"start":{"date-parts":[[2025,8,14]],"date-time":"2025-08-14T00:00:00Z","timestamp":1755129600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/royalsociety.org\/-\/media\/journals\/author\/Licence-to-Publish-20062019-final.pdf"},{"start":{"date-parts":[[2025,8,14]],"date-time":"2025-08-14T00:00:00Z","timestamp":1755129600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/royalsociety.org\/journals\/ethics-policies\/data-sharing-mining\/"}],"content-domain":{"domain":["royalsocietypublishing.org"],"crossmark-restriction":true},"short-container-title":["Phil. Trans. R. Soc. B"],"published-print":{"date-parts":[[2025,8,14]]},"abstract":"<jats:p>The question of whether large language models (LLMs) possess Theory of Mind (ToM)\u2014often defined as the ability to reason about others\u2019 mental states\u2014has sparked significant scientific and public interest. However, the evidence as to whether LLMs possess ToM is mixed, and the recent growth in evaluations has not resulted in a convergence. Here, we take inspiration from cognitive science to re-evaluate the state of ToM evaluation in LLMs. We argue that a major reason for the disagreement on whether LLMs have ToM is a lack of clarity on whether models should be expected to match human behaviours, or the computations underlying those behaviours. We also highlight ways in which current evaluations may be deviating from \u2018pure\u2019 measurements of ToM abilities, which also contributes to the confusion. We conclude by discussing several directions for future research, including the relationship between ToM and pragmatic communication, which could advance our understanding of artificial systems as well as human cognition.<\/jats:p>\n                  <jats:p>This article is part of the theme issue \u2018At the heart of human communication: new views on the complex relationship between pragmatics and Theory of Mind\u2019.<\/jats:p>","DOI":"10.1098\/rstb.2023.0499","type":"journal-article","created":{"date-parts":[[2025,8,14]],"date-time":"2025-08-14T07:06:43Z","timestamp":1755155203000},"update-policy":"https:\/\/doi.org\/10.1098\/crossmark-policy","source":"Crossref","is-referenced-by-count":14,"title":["Re-evaluating Theory of Mind evaluation in large language models"],"prefix":"10.1098","volume":"380","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4075-6876","authenticated-orcid":true,"given":"Jennifer","family":"Hu","sequence":"first","affiliation":[{"name":"Department of Cognitive Science, Johns Hopkins University","place":["Baltimore, USA"]},{"name":"Data Science and AI Institute, Johns Hopkins University","place":["Baltimore, USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Felix","family":"Sosa","sequence":"additional","affiliation":[{"name":"Department of Psychology, Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University","place":["Cambridge, USA"]},{"name":"Center for Brain Minds and Machines","place":["Cambridge, USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tomer","family":"Ullman","sequence":"additional","affiliation":[{"name":"Department of Psychology, Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University","place":["Cambridge, USA"]},{"name":"Center for Brain Minds and Machines","place":["Cambridge, USA"]},{"name":"Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University","place":["Cambridge, USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"175","published-online":{"date-parts":[[2025,8,14]]},"reference":[{"key":"e_1_3_8_2_2","doi-asserted-by":"publisher","DOI":"10.1017\/S0140525X00076512"},{"key":"e_1_3_8_3_2","volume-title":"Mindreaders: the cognitive basis of \u2018theory of mind.\u2019","author":"Apperly I","year":"2011","unstructured":"Apperly I. 2011 Mindreaders: the cognitive basis of \u2018theory of mind.\u2019 New York, NY: Psychology Press."},{"key":"e_1_3_8_4_2","first-page":"10","volume-title":"Elements of discourse understanding","author":"Clark HH","year":"1981","unstructured":"Clark HH, Marshall CR. 1981 Definite reference and mutual knowledge. In Elements of discourse understanding (eds A Joshi, B Webber, I Sag), pp. 10\u201363. Cambridge, UK: Cambridge University Press."},{"key":"e_1_3_8_5_2","doi-asserted-by":"publisher","DOI":"10.1016\/0010-0277(86)90010-7"},{"key":"e_1_3_8_6_2","first-page":"301","volume-title":"Psychology of learning and motivation","author":"Brennan SE","year":"2010","unstructured":"Brennan SE, Galati A, Kuhlen AK. 2010 Two minds, one dialog: coordinating speaking and understanding. In Psychology of learning and motivation (ed. BH Ross), pp. 301\u2013344, vol. 53. Elsevier. (doi:10.1016\/S0079-7421(10)53008-1)"},{"key":"e_1_3_8_7_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.1218633"},{"key":"e_1_3_8_8_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.tics.2016.08.005"},{"key":"e_1_3_8_9_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2012.06.046"},{"key":"e_1_3_8_10_2","doi-asserted-by":"publisher","DOI":"10.1037\/a0031128"},{"key":"e_1_3_8_11_2","article-title":"Understanding jokes draws most heavily on the Theory of Mind brain network","author":"Kline Struhl M","year":"2018","unstructured":"Kline Struhl M, Gall\u00e9eJ J, Balewski Z, Fedorenko E. 2018 Understanding jokes draws most heavily on the Theory of Mind brain network. PsyArXiv (doi:10.31234\/osf.io\/h2nyx)","journal-title":"PsyArXiv"},{"key":"e_1_3_8_12_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12144-019-00295-w"},{"key":"e_1_3_8_13_2","doi-asserted-by":"publisher","DOI":"10.1080\/23273798.2018.1525494"},{"key":"e_1_3_8_14_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.joep.2005.06.015"},{"key":"e_1_3_8_15_2","doi-asserted-by":"publisher","DOI":"10.1075\/is.8.2.05kry"},{"key":"e_1_3_8_16_2","doi-asserted-by":"publisher","DOI":"10.1002\/wics.1631"},{"key":"e_1_3_8_17_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-9280.2006.01722.x"},{"key":"e_1_3_8_18_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0701408104"},{"key":"e_1_3_8_19_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1011734108"},{"key":"e_1_3_8_20_2","doi-asserted-by":"publisher","DOI":"10.1037\/a0037077"},{"key":"e_1_3_8_21_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cognition.2021.104890"},{"key":"e_1_3_8_22_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cognition.2009.07.005"},{"key":"e_1_3_8_23_2","doi-asserted-by":"publisher","DOI":"10.1016\/B978-0-12-398532-3.00007-5"},{"key":"e_1_3_8_24_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41562-017-0064"},{"key":"e_1_3_8_25_2","doi-asserted-by":"publisher","DOI":"10.1037\/xge0001035"},{"key":"e_1_3_8_26_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.tics.2022.08.003"},{"key":"e_1_3_8_27_2","doi-asserted-by":"publisher","DOI":"10.1017\/S0140525X00031368"},{"key":"e_1_3_8_28_2","doi-asserted-by":"publisher","DOI":"10.2307\/1131463"},{"key":"e_1_3_8_29_2","doi-asserted-by":"publisher","DOI":"10.1016\/B978-012273965-1\/50016-9"},{"key":"e_1_3_8_30_2","doi-asserted-by":"publisher","DOI":"10.1002\/9781444325485.ch2"},{"key":"e_1_3_8_31_2","doi-asserted-by":"publisher","DOI":"10.1093\/acrefore\/9780199384655.013.321"},{"key":"e_1_3_8_32_2","first-page":"154","article-title":"The cognitive prerequisites for language: insights from iterated learning","volume":"21","author":"Smith K","year":"2018","unstructured":"Smith K. 2018 The cognitive prerequisites for language: insights from iterated learning. J. Lang. Evol. 21, 154\u2013160. (doi:10.1016\/j.cobeha.2018.05.003)","journal-title":"J. Lang. Evol."},{"key":"e_1_3_8_33_2","doi-asserted-by":"publisher","DOI":"10.2307\/1127956"},{"key":"e_1_3_8_34_2","doi-asserted-by":"publisher","DOI":"10.1037\/0012-1649.17.1.99"},{"key":"e_1_3_8_35_2","doi-asserted-by":"publisher","DOI":"10.1111\/desc.12017"},{"key":"e_1_3_8_36_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1804761115"},{"key":"e_1_3_8_37_2","doi-asserted-by":"publisher","DOI":"10.1006\/anbe.2000.1518"},{"key":"e_1_3_8_38_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cognition.2008.08.010"},{"key":"e_1_3_8_39_2","doi-asserted-by":"publisher","DOI":"10.30965\/9783957438843_008"},{"key":"e_1_3_8_40_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.aaf8110"},{"key":"e_1_3_8_41_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1910095116"},{"key":"e_1_3_8_42_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cobeha.2022.101137"},{"key":"e_1_3_8_43_2","doi-asserted-by":"publisher","DOI":"10.1017\/S0140525X98000703"},{"key":"e_1_3_8_44_2","volume-title":"Can a machine know that we know what it knows?","author":"Whang O","year":"2023","unstructured":"Whang O. 2023 Can a machine know that we know what it knows?. New York, NY: The New York Times."},{"key":"e_1_3_8_45_2","volume-title":"Theory of mind as an emergent property of generative ai could be a linchpin for ai-powered mental health advisement apps","author":"Eliot L","year":"2023","unstructured":"Eliot L. 2023 Theory of mind as an emergent property of generative ai could be a linchpin for ai-powered mental health advisement apps. Jersey City, NJ: Forbes. See https:\/\/www.forbes.com\/sites\/lanceeliot\/2023\/12\/20\/theory-of-mind-as-an-emergent-property-of-generative-ai-could-be-a-linchpin-for-ai-powered-mental-health-advisement-apps\/."},{"key":"e_1_3_8_46_2","volume-title":"Understanding human intentions will be the next big breakthrough in ai","author":"Gent E","year":"2023","unstructured":"Gent E. 2023 Understanding human intentions will be the next big breakthrough in ai. Published by New Scientist on February 22, 2023. See https:\/\/www.newscientist.com\/article\/mg25734260-500-understanding-human-intentions-will-be-the-next-big-breakthrough-in-ai."},{"key":"e_1_3_8_47_2","article-title":"LLM theory of mind and alignment: opportunities and risks","author":"Street W","year":"2024","unstructured":"Street W. 2024 LLM theory of mind and alignment: opportunities and risks. arXiv 2405.08154. (doi:10.48550\/arXiv.2405.08154)","journal-title":"arXiv"},{"key":"e_1_3_8_48_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2405460121"},{"key":"e_1_3_8_49_2","article-title":"Sparks of artificial general intelligence: early experiments with GPT-4","author":"Bubeck S","year":"2023","unstructured":"Bubeck S et al. 2023 Sparks of artificial general intelligence: early experiments with GPT-4. arXiv 2303.12712. (doi:10.48550\/arXiv.2303.12712)","journal-title":"arXiv"},{"key":"e_1_3_8_50_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41562-024-01882-z"},{"key":"e_1_3_8_51_2","article-title":"LLMs achieve adult human performance on higher-order theory of mind tasks","author":"Street W","year":"2024","unstructured":"Street W et al. 2024 LLMs achieve adult human performance on higher-order theory of mind tasks. arXiv 2405.18870. (doi:10.48550\/arXiv.2405.18870)","journal-title":"arXiv"},{"key":"e_1_3_8_52_2","article-title":"Large language models fail on trivial alterations to theory-of-mind tasks","author":"Ullman T","year":"2023","unstructured":"Ullman T. 2023 Large language models fail on trivial alterations to theory-of-mind tasks. arXiv 2302.08399. (doi:10.48550\/2302.08399)","journal-title":"arXiv"},{"key":"e_1_3_8_53_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.eacl-long.138"},{"key":"e_1_3_8_54_2","doi-asserted-by":"publisher","DOI":"10.1037\/h0040957"},{"key":"e_1_3_8_55_2","doi-asserted-by":"publisher","DOI":"10.1177\/1745691619896607"},{"key":"e_1_3_8_56_2","volume-title":"Proc. First Conf. on Lang. Modell. (COLM 2024), Philadelphia, PA 7\u20139 October 2024","author":"Hu J","year":"2024","unstructured":"Hu J, Frank MC. 2024 Auxiliary task demands mask the capabilities of smaller language models. In Proc. First Conf. on Lang. Modell. (COLM 2024), Philadelphia, PA 7\u20139 October 2024. arXiv. (doi:10.48550\/arXiv.2404.02418)"},{"key":"e_1_3_8_57_2","doi-asserted-by":"publisher","DOI":"10.1016\/0010-0277(95)00661-H"},{"key":"e_1_3_8_58_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-9280.2005.01649.x"},{"key":"e_1_3_8_59_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8624.2011.01635.x"},{"key":"e_1_3_8_60_2","doi-asserted-by":"publisher","DOI":"10.1037\/a0026790"},{"key":"e_1_3_8_61_2","volume-title":"The intentional stance","author":"Dennett DC","year":"1989","unstructured":"Dennett DC. 1989 The intentional stance. Cambridge, MA: MIT press."},{"key":"e_1_3_8_62_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cobeha.2019.04.010"},{"key":"e_1_3_8_63_2","first-page":"257","volume-title":"Handbook of imagination and mental simulation","author":"Saxe R","year":"2009","unstructured":"Saxe R. 2009 The happiness of the fish: evidence for a common theory of one\u2019s own and others\u2019 actions. In Handbook of imagination and mental simulation (eds KD Markman, W Klein, J Suhr), pp. 257\u2013266. New York, NY: Routledge. (doi:10.4324\/9780203809846.ch17)"},{"key":"e_1_3_8_64_2","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/9780262019279.003.0009"},{"key":"e_1_3_8_65_2","volume-title":"Connectionism and the mind: an introduction to parallel processing in networks","author":"Bechtel W","year":"1991","unstructured":"Bechtel W, Abrahamsen A. 1991 Connectionism and the mind: an introduction to parallel processing in networks. Cambridge, MA: Basil Blackwell."},{"key":"e_1_3_8_66_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.tics.2023.08.006"},{"key":"e_1_3_8_67_2","doi-asserted-by":"publisher","DOI":"10.1016\/0010-0277(85)90022-8"},{"key":"e_1_3_8_68_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.890"},{"key":"e_1_3_8_69_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.167.3914.86"},{"key":"e_1_3_8_70_2","doi-asserted-by":"publisher","DOI":"10.1002\/dev.420050403"},{"key":"e_1_3_8_71_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.findings-emnlp.72"},{"key":"e_1_3_8_72_2","article-title":"Boosting theory-of-mind performance in large language models via prompting","author":"Moghaddam SR","year":"2023","unstructured":"Moghaddam SR, Honey CJ. 2023 Boosting theory-of-mind performance in large language models via prompting. arXiv 2304.11490. (doi:10.48550\/arXiv.2304.11490)","journal-title":"arXiv"},{"key":"e_1_3_8_73_2","article-title":"Understanding social reasoning in language models with language models","author":"Gandhi K","year":"2023","unstructured":"Gandhi K, Fr\u00e4nken JP, Gerstenberg T, Goodman ND. 2023 Understanding social reasoning in language models with language models. arXiv 2306.15448. (doi:10.48550\/arXiv.2306.15448)","journal-title":"arXiv"},{"key":"e_1_3_8_74_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.conll-1.25"},{"key":"e_1_3_8_75_2","article-title":"Unveiling theory of mind in large language models: a parallel to single neurons in the human brain","author":"Jamali M","year":"2023","unstructured":"Jamali M, Williams ZM, Cai J. 2023 Unveiling theory of mind in large language models: a parallel to single neurons in the human brain. arXiv 2309.01660. (doi:10.48550\/arXiv.2309.01660)","journal-title":"arXiv"},{"key":"e_1_3_8_76_2","article-title":"How far are large language models from agents with theory-of-mind?","author":"Zhou P","year":"2023","unstructured":"Zhou P et al. 2023 How far are large language models from agents with theory-of-mind? arXiv 2310.03051. (doi:10.48550\/arXiv.2310.03051)","journal-title":"arXiv"},{"key":"e_1_3_8_77_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-main.248"},{"key":"e_1_3_8_78_2","doi-asserted-by":"publisher","DOI":"10.1111\/cogs.13309"},{"key":"e_1_3_8_79_2","article-title":"Hi-tom: a benchmark for evaluating higher-order theory of mind reasoning in large language models","author":"He Y","year":"2023","unstructured":"He Y, Wu Y, Jia Y, Mihalcea R, Chen Y, Deng N. 2023 Hi-tom: a benchmark for evaluating higher-order theory of mind reasoning in large language models. arXiv 2310.16755. (doi:10.48550\/arXiv.2310.16755)","journal-title":"arXiv"},{"key":"e_1_3_8_80_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.acl-long.451"},{"key":"e_1_3_8_81_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.acl-long.780"},{"key":"e_1_3_8_82_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-71170-1_20"},{"key":"e_1_3_8_83_2","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2019.02905"},{"key":"e_1_3_8_84_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00674"},{"key":"e_1_3_8_85_2","doi-asserted-by":"publisher","DOI":"10.1037\/a0029137"},{"key":"e_1_3_8_86_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00391"},{"key":"e_1_3_8_87_2","first-page":"9614","volume-title":"Proc. of the 38th Int. Conf. on Machine Learning","volume":"139","author":"Shu T","year":"2021","unstructured":"Shu T, Bhandwaldar A, Gan C, Smith KA, Liu S, Gutfreund D, Spelke E, Tenenbaum JB, Ullman TD. 2021 AGENT: a benchmark for core psychological reasoning. In Proc. of the 38th Int. Conf. on Machine Learning, vol. 139, pp. 9614\u20139625, PMLR. https:\/\/proceedings.mlr.press\/v139\/shu21a.html."},{"key":"e_1_3_8_88_2","article-title":"Elements of world knowledge (EWOK): a cognition-inspired framework for evaluating basic world knowledge in language models","author":"Ivanova AA","year":"2024","unstructured":"Ivanova AA et al. 2024 Elements of world knowledge (EWOK): a cognition-inspired framework for evaluating basic world knowledge in language models. arXiv 2405.09605. (doi:10.48550\/arXiv.2405.09605)","journal-title":"arXiv"},{"key":"e_1_3_8_89_2","article-title":"top uploading test data in plain text: practical strategies for mitigating data contamination by evaluation benchmarks","author":"Jacovi A","year":"2023","unstructured":"Jacovi A, Caciularu A, Goldman O, Goldberg Y. 2023 top uploading test data in plain text: practical strategies for mitigating data contamination by evaluation benchmarks. arXiv 2305.10160. (doi:10.48550\/arXiv.2305.10160)","journal-title":"arXiv"},{"key":"e_1_3_8_90_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.acl-short.18"},{"key":"e_1_3_8_91_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.98"},{"key":"e_1_3_8_92_2","first-page":"2633","volume-title":"30th USENIX Security Symposium (USENIX Security 21), 11\u201313 August 2021","author":"Carlini N","year":"2021","unstructured":"Carlini N. 2021 Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21), 11\u201313 August 2021, pp. 2633\u20132650. USENIX Association. https:\/\/www.usenix.org\/conference\/usenixsecurity21\/presentation\/carlini-extracting."},{"key":"e_1_3_8_93_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41562-023-01732-4"},{"key":"e_1_3_8_94_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1334"},{"key":"e_1_3_8_95_2","article-title":"Leaving the barn door open for Clever Hans: simple features predict LLM benchmark answers","author":"Pacchiardi L","year":"2024","unstructured":"Pacchiardi L, Tesic M, Cheke LG, Hern\u00e1ndez-Orallo J. 2024 Leaving the barn door open for Clever Hans: simple features predict LLM benchmark answers. arXiv 2410.11672. (doi:10.48550\/arXiv.2410.11672)","journal-title":"arXiv"},{"key":"e_1_3_8_96_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41562-024-02096-z"},{"key":"e_1_3_8_97_2","doi-asserted-by":"publisher","DOI":"10.1093\/pnasnexus\/pgae233"},{"key":"e_1_3_8_98_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2322420121"},{"key":"e_1_3_8_99_2","article-title":"Does ChatGPT have theory of mind?","author":"Holterman B","year":"2023","unstructured":"Holterman B, Deemter K. 2023 Does ChatGPT have theory of mind? arXiv 2305.14020. (doi:10.48550\/arXiv.2305.14020)","journal-title":"arXiv"},{"key":"e_1_3_8_100_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-023-10401-x"},{"key":"e_1_3_8_101_2","doi-asserted-by":"publisher","DOI":"10.2307\/1130760"},{"key":"e_1_3_8_102_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8624.1998.00672.x"},{"key":"e_1_3_8_103_2","doi-asserted-by":"publisher","DOI":"10.1348\/026151099165203"},{"key":"e_1_3_8_104_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1609203113"},{"key":"e_1_3_8_105_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.dr.2022.101061"},{"key":"e_1_3_8_106_2","doi-asserted-by":"publisher","DOI":"10.1162\/coli_a_00525"},{"key":"e_1_3_8_107_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1905334117"},{"key":"e_1_3_8_108_2","doi-asserted-by":"publisher","DOI":"10.1145\/2509558.2509563"},{"key":"e_1_3_8_109_2","doi-asserted-by":"publisher","DOI":"10.1163\/9789004368811_003"},{"key":"e_1_3_8_110_2","volume-title":"Relevance: communication and cognition","author":"Sperber D","year":"1986","unstructured":"Sperber D, Wilson D. 1986 Relevance: communication and cognition. Oxford, UK: Wiley-Blackwell."},{"key":"e_1_3_8_111_2","first-page":"27730","volume-title":"Advances in neural information processing systems","author":"Ouyang L","year":"2022","unstructured":"Ouyang L et al. 2022 Training language models to follow instructions with human feedback. In Advances in neural information processing systems (eds S Koyejo, S Mohamed, A Agarwal, D Belgrave, K Cho, A Oh), pp. 27730\u201327744, vol. 35. New Orleans, LA: Curran Associates, Inc."},{"key":"e_1_3_8_112_2","article-title":"Training a helpful and harmless assistant with reinforcement learning from human feedback","author":"Bai Y","year":"2022","unstructured":"Bai Y et al. 2022 Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv 2204.05862. (doi:10.48550\/arXiv.2204.05862)","journal-title":"arXiv"},{"key":"e_1_3_8_113_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.acl-long.851"},{"key":"e_1_3_8_114_2","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2018.01453"},{"key":"e_1_3_8_115_2","doi-asserted-by":"publisher","DOI":"10.1075\/pc.19010.enr"},{"key":"e_1_3_8_116_2","doi-asserted-by":"publisher","DOI":"10.1093\/oxfordhb\/9780198791768.013.23"},{"key":"e_1_3_8_117_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8624.2007.01018.x"},{"key":"e_1_3_8_118_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcomdis.2024.106466"},{"key":"e_1_3_8_119_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.findings-acl.719"},{"key":"e_1_3_8_120_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.findings-acl.182"},{"key":"e_1_3_8_121_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.acl-long.230"},{"key":"e_1_3_8_122_2","volume-title":"Thirty-seventh Conf. on Neural Information Processing Systems","author":"Ruis LE","year":"2023","unstructured":"Ruis LE, Khan A, Biderman S, Hooker S, Rockt\u00e4schelT T, Grefenstette E. 2023 The Goldilocks of pragmatic understanding: fine-tuning strategy matters for implicature resolution by LLMs. In Thirty-seventh Conf. on Neural Information Processing Systems. Curran Associates, Inc. https:\/\/openreview.net\/forum?id=5bWW9Eop7l."},{"key":"e_1_3_8_123_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.768"},{"key":"e_1_3_8_124_2","article-title":"A tripartite structure of pragmatic language abilities: comprehension of social conventions, intonation processing and causal reasoning","author":"Floyd S","year":"2023","unstructured":"Floyd S, Jouravlev O, Mineroff Z, Poliak M, Gibson E, Fedorenko E. 2023 A tripartite structure of pragmatic language abilities: comprehension of social conventions, intonation processing and causal reasoning. PsyArXiv. (doi:10.31234\/osf.io\/e2xta)","journal-title":"PsyArXiv"},{"key":"e_1_3_8_125_2","first-page":"379","article-title":"The pragmatic profile of ChatGPT: assessing the communicative skills of a conversational agent","volume":"35","author":"Barattieri di San Pietro C","year":"2023","unstructured":"Barattieri di San Pietro C, Frau F, Mangiaterra V, Bambini V. 2023 The pragmatic profile of ChatGPT: assessing the communicative skills of a conversational agent. Sistemi Intelligenti 35, 379\u2013400. (doi:10.1422\/108136)","journal-title":"Sistemi Intelligenti"},{"key":"e_1_3_8_126_2","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-linguistics-031220-010811"},{"key":"e_1_3_8_127_2","volume-title":"NeurIPS 2023 workshop: information-theoretic principles in cognitive systems","author":"Carenini G","year":"2023","unstructured":"Carenini G, BodotL, Bischetti L, Schaeken W, Bambini V. 2023 Large language models behave (Almost) as rational speech actors: insights from metaphor understanding. In NeurIPS 2023 workshop: information-theoretic principles in cognitive systems. https:\/\/openreview.net\/forum?id=SosbRhZLBV."},{"key":"e_1_3_8_128_2","volume-title":"NeurIPS 2024 Workshop on Behavioral Machine Learning","author":"Jian M","year":"2024","unstructured":"Jian M, Siddharth N. 2024 Are LLMs good pragmatic speakers?. In NeurIPS 2024 Workshop on Behavioral Machine Learning. https:\/\/openreview.net\/forum?id=htFtNdgWdf."},{"key":"e_1_3_8_129_2","volume-title":"Proc. of the Annual Meeting of the Cognitive Science Society","author":"McCoy RT","year":"2018","unstructured":"McCoy RT, Frank R, Linzen T. 2018 Revisiting the poverty of the stimulus: hierarchical generalization without a hierarchical bias in recurrent neural networks. In Proc. of the Annual Meeting of the Cognitive Science Society. https:\/\/escholarship.org\/uc\/item\/12w6n3xn."},{"key":"e_1_3_8_130_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.acl-long.521"},{"key":"e_1_3_8_131_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.emnlp-main.53"},{"key":"e_1_3_8_132_2","volume-title":"First workshop on theory of mind in communicating agents","author":"Nguyen KX","year":"2023","unstructured":"Nguyen KX. 2023 Language Models are Bounded Pragmatic Speakers. In First workshop on theory of mind in communicating agents. https:\/\/openreview.net\/forum?id=kEdFOtiZ5C."},{"key":"e_1_3_8_133_2","doi-asserted-by":"publisher","DOI":"10.1097\/TLD.0000000000000037"},{"key":"e_1_3_8_134_2","doi-asserted-by":"publisher","DOI":"10.2307\/1131864"},{"key":"e_1_3_8_135_2","doi-asserted-by":"publisher","DOI":"10.1111\/1467-8624.00435"},{"key":"e_1_3_8_136_2","doi-asserted-by":"publisher","DOI":"10.1111\/1467-7687.00289"},{"key":"e_1_3_8_137_2","doi-asserted-by":"publisher","DOI":"10.1093\/acprof:oso\/9780195159912.001.0001"},{"key":"e_1_3_8_138_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-9280.2009.02377.x"},{"key":"e_1_3_8_139_2","doi-asserted-by":"publisher","DOI":"10.1093\/acprof:oso\/9780199592722.003.0001"},{"key":"e_1_3_8_140_2","doi-asserted-by":"publisher","DOI":"10.1038\/s44159-023-00211-x"},{"key":"e_1_3_8_141_2","volume-title":"Why would anyone believe in God?","author":"Barrett J","year":"2004","unstructured":"Barrett J. 2004 Finding Agents Everywhere. In Why would anyone believe in God?. Walnut Creek, CA: AltaMira Press."},{"key":"e_1_3_8_142_2","first-page":"243","article-title":"An experimental study of apparent behavior","volume":"57","author":"Heider F","year":"1944","unstructured":"Heider F, Simmel M. 1944 An experimental study of apparent behavior. Am. J. 57, 243\u2013259. (doi:10.2307\/1416950)","journal-title":"Am. J."},{"key":"e_1_3_8_143_2","doi-asserted-by":"publisher","DOI":"10.1111\/tops.12308"},{"key":"e_1_3_8_144_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cognition.2019.104011"},{"key":"e_1_3_8_145_2","volume-title":"First Conf. on language modeling","author":"Guo J","year":"2024","unstructured":"Guo J, Yang B, Yoo P, Lin BY, Iwasawa Y, Matsuo Y. 2024 Suspicion agent: playing imperfect information games with theory of mind aware GPT-4. In First Conf. on language modeling. https:\/\/openreview.net\/forum?id=F2yGbwXJAi."},{"key":"e_1_3_8_146_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-60405-8_5"},{"key":"e_1_3_8_147_2","article-title":"Interpretability in the wild: a circuit for indirect object Identification in GPT-2 Small","author":"WangK K","year":"2023","unstructured":"WangK K, Variengien A, Conmy A, Shlegeris B, Steinhardt J. 2023 Interpretability in the wild: a circuit for indirect object Identification in GPT-2 Small. arXiv 2211.00593v1. (doi:10.48550\/arXiv.2211.00593.v1)","journal-title":"arXiv"},{"key":"e_1_3_8_148_2","unstructured":"Merullo J Eickhoff C Pavlick E. 2024 Circuit component reuse across tasks in transformer language models 2310.08744. (doi:10.48550\/arXiv.2310.08744)"},{"key":"e_1_3_8_149_2","unstructured":"Singh C Inala JP Galley M Caruana R Gao J. 2024 Rethinking interpretability in the era of large language models 2402.01761. (doi:10.48550\/arXiv.2402.01761)"},{"key":"e_1_3_8_150_2","article-title":"Interpreting Gpt: the logit lens","author":"nostalgebraist","year":"2020","unstructured":"nostalgebraist. 2020 Interpreting Gpt: the logit lens. Blog Post https:\/\/www.lesswrong.com\/posts\/AcKRB8wDpdaN6v6ru\/interpreting-gpt-the-logit-lens","journal-title":"Blog Post"},{"key":"e_1_3_8_151_2","doi-asserted-by":"publisher","DOI":"10.23915\/distill.00007"},{"key":"e_1_3_8_152_2","first-page":"2668","volume-title":"Proc. 35th Int.Conf. Machine Learning. arXiv","author":"Kim B","year":"2018","unstructured":"Kim B, Wattenberg M, Gilmer J, Cai C, Wexler J, Viegas F, Sayres R. 2018 Interpretability beyond feature attribution: quantitative testing with concept activation vectors (tcav). In Proc. 35th Int.Conf. Machine Learning. arXiv, pp. 2668\u20132677. 1711.11279. (doi:10.48550\/arXiv.1711.11279)"}],"container-title":["Philosophical Transactions of the Royal Society B: Biological Sciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/royalsocietypublishing.org\/doi\/pdf\/10.1098\/rstb.2023.0499","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/royalsocietypublishing.org\/doi\/full-xml\/10.1098\/rstb.2023.0499","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/royalsocietypublishing.org\/doi\/pdf\/10.1098\/rstb.2023.0499","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,12]],"date-time":"2026-05-12T21:25:38Z","timestamp":1778621138000},"score":1,"resource":{"primary":{"URL":"http:\/\/royalsocietypublishing.org\/rstb\/article\/235070"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,14]]},"references-count":151,"journal-issue":{"issue":"1932","published-print":{"date-parts":[[2025,8,14]]}},"alternative-id":["10.1098\/rstb.2023.0499"],"URL":"https:\/\/doi.org\/10.1098\/rstb.2023.0499","relation":{},"ISSN":["0962-8436","1471-2970"],"issn-type":[{"value":"0962-8436","type":"print"},{"value":"1471-2970","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,14]]},"assertion":[{"value":"2024-06-28","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-11-22","order":1,"name":"revised","label":"Revised","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-03-23","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-08-14","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"20230499"}}