{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,30]],"date-time":"2026-04-30T16:08:34Z","timestamp":1777565314977,"version":"3.51.4"},"reference-count":163,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,4,1]],"date-time":"2025-04-01T00:00:00Z","timestamp":1743465600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100021856","name":"Ministero Dell'Universit\u00e0 e della Ricerca","doi-asserted-by":"publisher","award":["CUP J53D23007150006 \u2013 IC PRIN_2022MM8LKM_003"],"award-info":[{"award-number":["CUP J53D23007150006 \u2013 IC PRIN_2022MM8LKM_003"]}],"id":[{"id":"10.13039\/501100021856","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100021856","name":"Ministero Dell'Universit\u00e0 e della Ricerca","doi-asserted-by":"publisher","award":["CUP J53D23019490006 \u2013 IC PRIN_20223E8Y4X_002"],"award-info":[{"award-number":["CUP J53D23019490006 \u2013 IC PRIN_20223E8Y4X_002"]}],"id":[{"id":"10.13039\/501100021856","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:p>This article provides an epistemological analysis of current attempts of explaining how the relatively simple algorithmic components of neural language models (NLMs) provide them with genuine linguistic competence. After introducing the Transformer architecture, at the basis of most of current NLMs, the paper firstly emphasizes how the central question in the philosophy of AI has been shifted from \u201ccan machines think?\u201d, as originally put by Alan Turing, to \u201chow <jats:italic>can<\/jats:italic> machines think?\u201d, pointing to an explanatory gap for NLMs. Subsequently, existing explanatory strategies for the functioning of NLMs are analyzed to argue that they, however debated, do not differ from the explanatory strategies used in cognitive science to explain intelligent behaviors of humans. In particular, available experimental studies turned to test the <jats:italic>theory of mind, discourse entity tracking<\/jats:italic>, and <jats:italic>property induction<\/jats:italic> in NLMs are examined under the light of the <jats:italic>functional analysis<\/jats:italic> in the philosophy of cognitive science; the so-called <jats:italic>copying algorithm<\/jats:italic> and the <jats:italic>induction head<\/jats:italic> phenomenon of a Transformer are shown to provide a <jats:italic>mechanist<\/jats:italic> explanation of <jats:italic>in-context learning<\/jats:italic>; finally, current pioneering attempts to use NLMs to predict brain activation patterns when processing language are here shown to involve what we call a <jats:italic>co-simulation<\/jats:italic>, in which a NLM and the brain are used to simulate and understand each other.<\/jats:p>","DOI":"10.3389\/frai.2025.1509338","type":"journal-article","created":{"date-parts":[[2025,4,1]],"date-time":"2025-04-01T05:22:00Z","timestamp":1743484920000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Making sense of transformer success"],"prefix":"10.3389","volume":"8","author":[{"given":"Nicola","family":"Angius","sequence":"first","affiliation":[]},{"given":"Pietro","family":"Perconti","sequence":"additional","affiliation":[]},{"given":"Alessio","family":"Plebe","sequence":"additional","affiliation":[]},{"given":"Alessandro","family":"Acciai","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2025,4,1]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2305.17040","article-title":"A mechanism for sample-efficient in-context learning for sparse retrieval tasks","author":"Abernethy","year":"2023","journal-title":"arXiv"},{"key":"B2","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1162\/daed_a_01909","article-title":"Do large language models understand us?","volume":"151","author":"Ag\u00fcera y Arcas","year":"2022","journal-title":"Daedalus"},{"key":"B3","doi-asserted-by":"publisher","first-page":"543","DOI":"10.1017\/S135132491900024X","article-title":"Analyzing and interpreting neural networks for NLP: A report on the first BlackboxNLP workshop","volume":"25","author":"Alishahi","year":"2019","journal-title":"Nat. Lang. Eng"},{"key":"B4","volume-title":"Human Associative Memory","author":"Anderson","year":"1974"},{"key":"B5","first-page":"4596","article-title":"\u201cCo-simulations of brain language processing using neural language models,\u201d","volume-title":"Proceedings of the Annual Meeting of the Cognitive Science Society, Vol. 46","author":"Angius","year":"2024"},{"key":"B6","first-page":"47","article-title":"From coding to curing. functions, implementations, and correctness in deep learning. Philos","volume":"36","author":"Angius","year":"2023","journal-title":"Technol"},{"key":"B7","first-page":"21895","article-title":"\u201cScaling laws for language encoding models in fMRI,\u201d","volume-title":"Advances in Neural Information Processing Systems","author":"Antonello","year":"2023"},{"key":"B8","doi-asserted-by":"crossref","DOI":"10.4324\/9780203833926","volume-title":"Mindreaders: The Cognitive Basis of Theory of Mind","author":"Apperly","year":"2010"},{"key":"B9","doi-asserted-by":"publisher","first-page":"337","DOI":"10.1017\/pan.2023.2","article-title":"Out of one, many: using language models to simulate human samples","volume":"31","author":"Argyle","year":"2023","journal-title":"Polit. Analy"},{"key":"B10","article-title":"\u201cNeural machine translation by jointly learning to align and translate,\u201d","volume-title":"3rd International Conference on Learning Representations","author":"Bahdanau","year":"2016"},{"key":"B11","article-title":"\u201cFolk psychology as mental simulation,\u201d","author":"Barlassina","year":"2017","journal-title":"The Stanford Encyclopedia of Philosophy"},{"key":"B12","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1016\/0010-0277(85)90022-8","article-title":"Does the autistic child have a \u201ctheory of mind\u201d?","volume":"21","author":"Baron-Cohen","year":"1985","journal-title":"Cognition"},{"key":"B13","doi-asserted-by":"publisher","first-page":"20190307","DOI":"10.1098\/rstb.2019.0307","article-title":"Linguistic generalization and compositionality in modern artificial neural networks","volume":"375","author":"Baroni","year":"2019","journal-title":"Philos. Trans. R. Soc. B, Biol. Sci"},{"key":"B14","volume-title":"Mental Mechanisms-Philosophical Perspectives on Cognitive Neuroscience","author":"Bechtel","year":"2008"},{"key":"B15","doi-asserted-by":"crossref","first-page":"5185","DOI":"10.18653\/v1\/2020.acl-main.463","article-title":"\u201cClimbing towards NLU: on meaning, form, and understanding in the age of data,\u201d","volume-title":"58th Annual Meeting of the Association for Computational Linguistics","author":"Bender","year":"2020"},{"key":"B16","first-page":"1137","article-title":"A neural probabilistic language model","volume":"3","author":"Bengio","year":"2003","journal-title":"J. Mach. Learn. Res"},{"key":"B17","doi-asserted-by":"publisher","first-page":"1553","DOI":"10.1109\/TNNLS.2013.2293637","article-title":"On the complexity of neural network classifiers: a comparison between shallow and deep architectures","volume":"25","author":"Bianchini","year":"","journal-title":"IEEE Trans. Neural Netw. Learn. Syst"},{"key":"B18","first-page":"371","article-title":"\u201cOn the complexity of shallow and deep neural network classifiers,\u201d","author":"Bianchini","year":"","journal-title":"ESANN 2014 proceedings"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2306.00802","article-title":"Birth of a transformer: a memory viewpoint","author":"Bietti","year":"2023","journal-title":"arXiv"},{"key":"B20","unstructured":"Bills\n              S.\n            \n            \n              Cammarata\n              N.\n            \n            \n              Mossing\n              D.\n            \n            \n              Tillman\n              H.\n            \n            \n              Gao\n              L.\n            \n            \n              Goh\n              G.\n            \n          \n          Language Models Can Explain Neurons in Language Models\n          \n          2023"},{"key":"B21","doi-asserted-by":"publisher","first-page":"213","DOI":"10.1146\/annurev-linguistics-011619-030303","article-title":"Distributional semantics and linguistic theory","volume":"6","author":"Boleda","year":"2020","journal-title":"Ann. Rev. Linguist"},{"key":"B22","first-page":"217","article-title":"\u201cLarge scale online learning,\u201d","volume-title":"Advances in Neural Information Processing Systems 16: Proceedings of the 2003 Conference, Vol. 16","author":"Bottou","year":"2004"},{"key":"B23","article-title":"\u201cNonmonotonic Reasoning: an Overview,\u201d","author":"Brewka","year":"1997"},{"key":"B24","unstructured":"\u201cTowards monosemanticity: Decomposing language models with dictionary learning,\u201d\n          \n          \n            \n              Bricken\n              T.\n            \n            \n              Templeton\n              A.\n            \n            \n              Batson\n              J.\n            \n            \n              Chen\n              B.\n            \n            \n              Jermyn\n              A.\n            \n            \n              Conerly\n              T.\n            \n          \n          Transformer Circuits Thread\n          \n          2023"},{"key":"B25","first-page":"1877","article-title":"\u201cLanguage models are few-shot learners,\u201d","volume-title":"Advances in Neural Information Processing Systems","author":"Brown","year":"2020"},{"key":"B26","volume-title":"Can a Conversational Agent Pass Theory-of-Mind Tasks? A Case Study of ChatGPT with the Hinting, False Beliefs, and Strange Stories Paradigms","author":"Brunet-Gouet","year":"2023"},{"key":"B27","doi-asserted-by":"publisher","first-page":"16327","DOI":"10.1038\/s41598-022-20460-9","article-title":"Deep language algorithms predict semantic comprehension from brain activity","volume":"12","author":"Caucheteux","year":"2022","journal-title":"Sci. Rep"},{"key":"B28","doi-asserted-by":"publisher","first-page":"430","DOI":"10.1038\/s41562-022-01516-2","article-title":"Evidence of a predictive coding hierarchy in the human brain listening to speech","volume":"7","author":"Caucheteux","year":"2023","journal-title":"Nat. Human Behav"},{"key":"B29","doi-asserted-by":"publisher","first-page":"134","DOI":"10.1038\/s42003-022-03036-1","article-title":"Brains and algorithms partially converge in natural language processing","volume":"5","author":"Caucheteux","year":"2022","journal-title":"Commun. Biol"},{"key":"B30","volume-title":"The Conscious Mind: In Search of a Fundamental Theory","author":"Chalmers","year":"1996"},{"key":"B31","first-page":"7","article-title":"The singularity: a philosophical analysis","volume":"17","author":"Chalmers","year":"2010","journal-title":"J. Consciousn. Stud"},{"key":"B32","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2303.07103","article-title":"Could a large language model be conscious?","author":"Chalmers","year":"2023"},{"key":"B33","first-page":"276","article-title":"\u201cWhat does BERT look at? An analysis of BERT's attention,\u201d","volume-title":"BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP","author":"Clark","year":"2019"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2304.14997","article-title":"Towards automated circuit discovery for mechanistic interpretability","author":"Conmy","year":"2023","journal-title":"arXiv"},{"key":"B35","first-page":"1","article-title":"\u201cLevels,\u201d","author":"Craver","year":"2016","journal-title":"Open MIND: Philosophy and the Mind Sciences in the 21st Century"},{"key":"B36","doi-asserted-by":"publisher","first-page":"741","DOI":"10.2307\/2024640","article-title":"Functional analysis","volume":"72","author":"Cummins","year":"1975","journal-title":"J. Philosophy"},{"key":"B37","volume-title":"The Nature of Psychological Explanation","author":"Cummins","year":"1983"},{"key":"B38","first-page":"817","article-title":"\u201cBiorobotics,\u201d","author":"Datteri","year":"2017","journal-title":"Agent-Based Modelling in Population Studies: Concepts, Methods, and Applications"},{"key":"B39","doi-asserted-by":"publisher","first-page":"136","DOI":"10.1109\/72.182704","article-title":"Backpropagation neural nets with one and two hidden layers","volume":"4","author":"de Villers","year":"1992","journal-title":"IEEE Trans. Neural Netw"},{"key":"B40","doi-asserted-by":"publisher","first-page":"688","DOI":"10.1038\/s44159-023-00241-5","article-title":"Using large language models in psychology","volume":"2","author":"Demszky","year":"2023","journal-title":"Nat. Rev. Psychol"},{"key":"B41","article-title":"\u201cCan machines think?,\u201d","author":"Dennett","year":"1985","journal-title":"How We Know: Nobel Conference XX"},{"key":"B42","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/1663.001.0001","volume-title":"Brainchildren-Essays on Designing Minds","author":"Dennett","year":"1998"},{"key":"B43","first-page":"4171","article-title":"\u201cBERT: pre-training of deep bidirectional transformers for language understanding,\u201d","volume-title":"Proceedings North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Devlin","year":"2019"},{"key":"B44","doi-asserted-by":"publisher","first-page":"23","DOI":"10.2307\/3131645","article-title":"Machines and the mental","volume":"59","author":"Dretske","year":"1985","journal-title":"Proc. Address. Am. Philosoph. Assoc"},{"key":"B45","volume-title":"What Computers Can't Do: A Critique of Artificial Reason","author":"Dreyfus","year":"1972"},{"key":"B46","volume-title":"What Computers Still Can't Do: A Critique of Artificial Reason","author":"Dreyfus","year":"1992"},{"key":"B47","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-90882-3","volume-title":"Computer simulations in science and engineering: Concepts-Practices-Perspectives","author":"Dur\u00e1n","year":"2018"},{"key":"B48","doi-asserted-by":"publisher","first-page":"184","DOI":"10.1016\/j.neuroimage.2016.10.001","article-title":"Seeing it all: Convolutional network layers map the function of the human visual system","volume":"152","author":"Eickenberg","year":"2017","journal-title":"Neuroimage"},{"key":"B49","unstructured":"\u201cA mathematical framework for Transformer circuits,\u201d\n          \n          \n            \n              Elhage\n              N.\n            \n            \n              Nanda\n              N.\n            \n            \n              Olsson\n              C.\n            \n            \n              Henighan\n              T.\n            \n            \n              Joseph\n              N.\n            \n            \n              Mann\n              B.\n            \n          \n          36921485\n          Transformer Circuits Thread\n          \n          2021"},{"key":"B50","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1016\/0364-0213(90)90002-E","article-title":"Finding structure in time","volume":"14","author":"Elman","year":"1990","journal-title":"Cogn. Sci"},{"key":"B51","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1007\/s13347-023-00621-y","article-title":"AI as agency without intelligence: on ChatGPT, large language models and other generative models","volume":"36","author":"Floridi","year":"2023","journal-title":"Philos. Technol"},{"key":"B52","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1016\/S1364-6613(00)01453-4","article-title":"The turing test: the first 50 years","volume":"4","author":"French","year":"2000","journal-title":"Trends Cogn. Sci"},{"key":"B53","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2306.01128","article-title":"Learning transformer programs","author":"Friedman","year":"2023","journal-title":"arXiv"},{"key":"B54","doi-asserted-by":"publisher","first-page":"452","DOI":"10.1016\/j.concog.2015.04.002","article-title":"The new hybrids: Continuing debates on social perception","volume":"36","author":"Gallagher","year":"2015","journal-title":"Conscious. Cogn"},{"key":"B55","first-page":"30583","article-title":"\u201cWhat can transformers learn in-context? a case study of simple function classes\u201d","volume-title":"Advances in Neural Information Processing Systems","author":"Garg","year":"2022"},{"key":"B56","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-vision-111815-114621","article-title":"Visual object recognition: do we (finally) know more now than we did?","author":"Gauthier","year":"2016","journal-title":"Ann. Rev. Vision Sci"},{"key":"B57","first-page":"30","article-title":"\u201cTransformer feed-forward layers build predictions by promoting concepts in the vocabulary space,\u201d","volume-title":"Conference on Empirical Methods in Natural Language Processing","author":"Geva","year":"2022"},{"key":"B58","first-page":"5484","article-title":"\u201cTransformer feed-forward layers are key-value memories,\u201d","volume-title":"Conference on Empirical Methods in Natural Language Processing","author":"Geva","year":"2021"},{"key":"B59","doi-asserted-by":"crossref","DOI":"10.1093\/0195138929.001.0001","volume-title":"Simulating Minds: The Philosophy, Psychology, and Neuroscience of Mindreading","author":"Goldman","year":"2006"},{"key":"B60","volume-title":"Deep Learning","author":"Goodfellow","year":"2016"},{"key":"B61","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2407.21783","article-title":"The Llama 3 herd of models","author":"Grattafiori","year":"2024","journal-title":"arXiv"},{"key":"B62","doi-asserted-by":"publisher","first-page":"323","DOI":"10.1007\/s11023-022-09589-8","article-title":"What might machines mean?","volume":"32","author":"Green","year":"2022","journal-title":"Minds Mach"},{"key":"B63","doi-asserted-by":"publisher","first-page":"20180013","DOI":"10.1098\/rsfs.2018.0013","article-title":"The functional neuroanatomy of face perception: from brain measurements to deep neural networks","volume":"8","author":"Grill-Spector","year":"2018","journal-title":"Interface Focus"},{"key":"B64","doi-asserted-by":"publisher","first-page":"10005","DOI":"10.1523\/JNEUROSCI.5023-14.2015","article-title":"Deep neural networks reveal a gradient in the complexity of neural representations across the ventral stream","volume":"35","author":"G\u00fc\u00e7l\u00fc","year":"2015","journal-title":"J. Neurosci"},{"key":"B65","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2310.10616","article-title":"How do transformers learn in-context beyond simple functions? A case study on learning with representations","author":"Guo","year":"2023","journal-title":"arXiv"},{"key":"B66","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2305.01610","article-title":"Finding neurons in a haystack: case studies with sparse probing","author":"Gurnee","year":"2023","journal-title":"arXiv"},{"key":"B67","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2303.13988","article-title":"Machine psychology: Investigating emergent capabilities and behavior in large language models using psychological methods","author":"Hagendorff","year":"2023","journal-title":"arXiv"},{"key":"B68","doi-asserted-by":"publisher","first-page":"101155","DOI":"10.1016\/j.cogsys.2023.101155","article-title":"Inductive reasoning in humans and large language models","volume":"83","author":"Han","year":"2024","journal-title":"Cogn. Syst. Res"},{"key":"B69","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1080\/09528138908953691","article-title":"Minds, machines and Searle","volume":"1","author":"Harnad","year":"1989","journal-title":"J. Exp. Theoret. Artif. Intellig"},{"key":"B70","doi-asserted-by":"publisher","first-page":"335","DOI":"10.1016\/0167-2789(90)90087-6","article-title":"Symbol grounding problem","volume":"42","author":"Harnad","year":"1990","journal-title":"Physica D"},{"key":"B71","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1007\/s12559-023-10179-8","article-title":"Interpreting black box models: A review on explainable artificial intelligence","volume":"16","author":"Hassija","year":"2024","journal-title":"Cognit. Comput"},{"key":"B72","volume-title":"Artificial Intelligence: The Very Idea","author":"Haugeland","year":"1985"},{"key":"B73","volume-title":"Mind Design II","author":"Haugeland","year":"1991"},{"key":"B74","first-page":"4129","article-title":"\u201cA structural probe for finding syntax in word representations,\u201d","volume-title":"Proceedings North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Hewitt","year":"2019"},{"key":"B75","doi-asserted-by":"publisher","first-page":"1243091","DOI":"10.1126\/science.1243091","article-title":"The cultural evolution of mind reading","volume":"344","author":"Heyes","year":"2010","journal-title":"Science"},{"key":"B76","first-page":"6","article-title":"\u201cAutoencoders, minimum description length and Helmholtz free energy,\u201d","volume-title":"Advances in Neural Information Processing Systems","author":"Hinton","year":"1994"},{"key":"B77","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2305.14020","article-title":"Does ChatGPT have theory of mind?","author":"Holterman","year":"2023"},{"key":"B78","doi-asserted-by":"crossref","first-page":"317","DOI":"10.18653\/v1\/2023.blackboxnlp-1.24","article-title":"\u201cRigorously assessing natural language explanations of neurons,\u201d","volume-title":"BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP","author":"Huang","year":"2023"},{"key":"B79","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2309.08600","article-title":"Sparse autoencoders find highly interpretable features in language model","author":"Huben","year":"2023"},{"key":"B80","volume-title":"Mirroring People: The Science of Empathy and How We Connect with Others","author":"Iacoboni","year":"2014"},{"key":"B81","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2305.20010","article-title":"Human or not? A gamified approach to the Turing Test","author":"Jannai","year":"2023","journal-title":"arXiv"},{"key":"B82","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2305.13417","article-title":"Interpreting transformer's attention dynamic memory and visualizing the semantic information flow of GPT","author":"Katz","year":"2023","journal-title":"arXiv"},{"key":"B83","doi-asserted-by":"publisher","first-page":"e1003915","DOI":"10.1371\/journal.pcbi.1003915","article-title":"Deep supervised, but not unsupervised, models may explain it cortical representation","volume":"10","author":"Khaligh-Razavi","year":"2014","journal-title":"PLoS Comput. Biol"},{"key":"B84","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.acl-long.213","article-title":"Entity tracking in language models","author":"Kim","year":"2023","journal-title":"arXiv"},{"key":"B85","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1412.6980","article-title":"Adam: A method for stochastic optimization","author":"Kingma","year":"2014","journal-title":"arXiv [preprint]"},{"key":"B86","article-title":"Theory of mind may have spontaneously emerged in large language models","author":"Kosinski","year":"2023","journal-title":"arXiv"},{"key":"B87","first-page":"25","article-title":"\u201cImageNet classification with deep convolutional neural networks,\u201d","volume-title":"Advances in Neural Information Processing Systems","author":"Krizhevsky","year":"2012"},{"key":"B88","doi-asserted-by":"publisher","DOI":"10.1101\/2022.06.08.495348","article-title":"Reconstructing the cascade of language processing in the brain using the internal computations of a transformer-based language model","author":"Kumar","year":"2023","journal-title":"bioRxiv"},{"key":"B89","doi-asserted-by":"publisher","first-page":"103473","DOI":"10.1016\/j.artint.2021.103473","article-title":"What do we want from explainable artificial intelligence (XAI)?-a stakeholder perspective on XAI and a conceptual model guiding interdisciplinary XAI research","volume":"296","author":"Langer","year":"2021","journal-title":"Artif. Intell"},{"key":"B90","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"B91","article-title":"Violation of expectation via metacognitive prompting reduces theory of mind prediction error in large language models","author":"Leer","year":"2023","journal-title":"arXiv"},{"key":"B92","doi-asserted-by":"crossref","DOI":"10.53288\/0398.1.00","volume-title":"Exoanthropology-Dialogues with AI","author":"Leib","year":"2023"},{"key":"B93","article-title":"Tracr: compiled transformers as a laboratory for interpretability","author":"Lindner","year":"2023","journal-title":"arXiv"},{"key":"B94","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1606.03490","article-title":"The Mythos of Model Interpretability","author":"Lipton","year":"2016","journal-title":"arXiv [preprint]"},{"key":"B95","first-page":"230","article-title":"\u201cIs machine psychology here? on requirements for using human psychological tests on large language models,\u201d","volume-title":"International Natural Language Generation Conference","author":"L\u00f6hn","year":"2024"},{"key":"B96","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.findings-emnlp.72","article-title":"Towards a holistic landscape of situated theory of mind in large language models","author":"Ma","year":"2023","journal-title":"arXiv"},{"key":"B97","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1086\/392759","article-title":"Thinking about mechanisms","volume":"67","author":"Machamer","year":"2000","journal-title":"Philos. Sci"},{"key":"B98","doi-asserted-by":"publisher","first-page":"1189525","DOI":"10.3389\/frobt.2023.1189525","article-title":"Developing chatGPT's theory of mind","volume":"10","author":"Marchetti","year":"2023","journal-title":"Front. Robot. AI"},{"key":"B99","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.naacl-long.281","article-title":"Language models implement simple word2vec-style vector arithmetic","author":"Merullo","year":"2023","journal-title":"arXiv"},{"key":"B100","doi-asserted-by":"publisher","first-page":"981","DOI":"10.1162\/tacl_a_00501","article-title":"How to dissect a muppet: the structure of transformer embedding spaces","volume":"10","author":"Mickus","year":"2022","journal-title":"Trans. Assoc. Comput. Linguist"},{"key":"B101","first-page":"3111","article-title":"\u201cDistributed representations of words and phrases and their compositionality,\u201d","volume-title":"Advances in Neural Information Processing Systems","author":"Mikolov","year":"2013"},{"key":"B102","doi-asserted-by":"publisher","first-page":"622","DOI":"10.1111\/j.1467-8624.2007.01018.x","article-title":"Language and theory of mind: Meta-analysis of the relation between language ability and false-belief understanding","volume":"78","author":"Milligan","year":"2007","journal-title":"Child Dev"},{"key":"B103","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3605943","article-title":"Recent advances in natural language processing via large pre-trained language models: a survey","volume":"56","author":"Min","year":"2023","journal-title":"ACM Comp. Surveys"},{"key":"B104","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2110.09419","article-title":"Compositional attention: Disentangling search and retrieval","author":"Mittal","year":"2021","journal-title":"arXiv [preprint]"},{"key":"B105","doi-asserted-by":"publisher","first-page":"98","DOI":"10.1109\/MRA.2012.2192811","article-title":"\u201cThe uncanny valley","volume":"19","author":"Mori","year":"1970","journal-title":"IEEE Robot. Automat"},{"key":"B106","doi-asserted-by":"crossref","DOI":"10.1119\/1.1937571","volume-title":"The Structure of Science","author":"Nagel","year":"1961"},{"key":"B107","doi-asserted-by":"publisher","first-page":"250","DOI":"10.1038\/s41597-021-01033-3","article-title":"The \u201cnarratives\u201d fMRI dataset for evaluating models of naturalistic language comprehension","volume":"8","author":"Nastase","year":"2021","journal-title":"Scientific Data"},{"key":"B108","volume-title":"Human Problem Solving","author":"Newell","year":"1972"},{"key":"B109","doi-asserted-by":"publisher","first-page":"481","DOI":"10.1016\/j.conb.2004.07.007","article-title":"Sparse coding of sensory inputs","volume":"14","author":"Olshausen","year":"2004","journal-title":"Curr. Opin. Neurobiol"},{"key":"B110","article-title":"\u201cIn-context learning and induction heads,\u201d","author":"Olsson","year":"2022","journal-title":"Transformer Circuits Thread"},{"key":"B111","first-page":"27730","article-title":"\u201cTraining language models to follow instructions with human feedback,\u201d","volume-title":"Advances in Neural Information Processing Systems","author":"Ouyang","year":"2022"},{"key":"B112","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3526113.3545616","article-title":"\u201cSocial simulacra: Creating populated prototypes for social computing systems,\u201d","volume-title":"ACM Symposium on User Interface Software and Technology","author":"Park","year":"2022"},{"key":"B113","doi-asserted-by":"publisher","first-page":"20220041","DOI":"10.1098\/rsta.2022.0041","article-title":"Symbols and grounding in large language models","volume":"381","author":"Pavlick","year":"2023","journal-title":"Philosoph. Trans. Royal Soc. A"},{"key":"B114","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1142\/S2705078522500205","article-title":"Do machines really understand meaning?","volume":"10","author":"Perconti","year":"2023","journal-title":"J. Artif. Intellig. Conscious"},{"key":"B115","volume-title":"Turing's Vision - How AI is Shaping the World","author":"Perconti","year":"2025"},{"key":"B116","doi-asserted-by":"publisher","first-page":"283","DOI":"10.1007\/s11229-011-9898-4","article-title":"Integrating psychology and neuroscience: Functional analyses as mechanism sketches","volume":"183","author":"Piccinini","year":"2011","journal-title":"Synthese"},{"key":"B117","article-title":"\u201cThe slowdown hypothesis,\u201d","volume-title":"Singularity Hypotheses","author":"Plebe","year":"2013"},{"key":"B118","doi-asserted-by":"crossref","DOI":"10.1201\/9781003120865","volume-title":"The Future of the Artificial Mind.","author":"Plebe","year":"2022"},{"key":"B119","doi-asserted-by":"publisher","first-page":"515","DOI":"10.1017\/S0140525X00076512","article-title":"Does the chimpanzee have a theory of mind?","volume":"4","author":"Premack","year":"1978","journal-title":"Behav. Brain Sci"},{"key":"B120","doi-asserted-by":"crossref","DOI":"10.1093\/oso\/9780198250579.001.0001","volume-title":"Views into the Chinese Room: New Essays on Searle and Artificial Intelligence","author":"Preston","year":"2002"},{"key":"B121","volume-title":"On the Foundations of Computing","author":"Primiero","year":"2020"},{"key":"B122","volume-title":"The Robots Dilemma-The Frame Problem in Artificial Intelligence","author":"Pylyshyn","year":"1987"},{"key":"B123","doi-asserted-by":"publisher","first-page":"168","DOI":"10.1162\/daed_a_01908","article-title":"Non-human words: On GPT-3 as a philosophical laboratory","volume":"151","author":"Rees","year":"2022","journal-title":"Daedalus"},{"key":"B124","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1093\/acprof:oso\/9780195326598.003.0007","article-title":"\u201cPrimate social cognition: thirty years after premack and woodruff,\u201d","author":"Rosati","year":"2010","journal-title":"Primate Neuroethology Theory"},{"key":"B125","first-page":"4064","article-title":"\u201cTheory theory (simulation theory, theory of mind),\u201d","author":"R\u00f6ska-Hardy","year":"2008","journal-title":"Encyclopedia of Neuroscience"},{"key":"B126","volume-title":"Transformers for Natural Language Processing","author":"Rothman","year":"2022"},{"key":"B127","doi-asserted-by":"publisher","first-page":"533","DOI":"10.1038\/323533a0","article-title":"Learning representations by back-propagating errors","volume":"323","author":"Rumelhart","year":"1986","journal-title":"Nature"},{"key":"B128","first-page":"216","article-title":"\u201cOn learning the past tenses of English verbs,\u201d","author":"Rumelhart","year":"1986","journal-title":"Parallel Distributed Processing: Explorations in the Microstructure of Cognition"},{"key":"B129","volume-title":"Artificial Intelligence","author":"Russell","year":"1995"},{"key":"B130","volume-title":"Introduction to Modern Information Retrieval","author":"Salton","year":"1983"},{"key":"B131","doi-asserted-by":"publisher","author":"Schrimpf","year":"","DOI":"10.1101\/407007"},{"key":"B132","doi-asserted-by":"publisher","first-page":"413","DOI":"10.1016\/j.neuron.2020.07.040","article-title":"Integrative benchmarking to advance neurally mechanistic models of human intelligence","volume":"108","author":"Schrimpf","year":"","journal-title":"Neuron"},{"key":"B133","doi-asserted-by":"publisher","first-page":"417","DOI":"10.1017\/S0140525X00005756","article-title":"Mind, brain and programs","volume":"3","author":"Searle","year":"1980","journal-title":"Behav. Brain Sci"},{"key":"B134","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1409.1556","article-title":"Very deep convolutional networks for large-scale image recognition","author":"Simonyan","year":"2015","journal-title":"arXiv [preprint]"},{"key":"B135","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2103.07601","article-title":"Approximating how single head attention learns","author":"Snell","year":"2021","journal-title":"arXiv"},{"key":"B136","doi-asserted-by":"publisher","first-page":"443","DOI":"10.1007\/s11229-022-03931-4","article-title":"Understanding models understanding language","volume":"200","author":"S\u00f8gaard","year":"2022","journal-title":"Synthese"},{"key":"B137","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1007\/s11023-023-09622-4","article-title":"Grounding the vector space of an octopus: Word meaning from raw text","volume":"33","author":"S\u00f8gaard","year":"2023","journal-title":"Minds Mach"},{"key":"B138","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2206.04615","article-title":"Beyond the imitation game: Quantifying and extrapolating the capabilities of language models","author":"Srivastava","year":"2022","journal-title":"arXiv [preprint]"},{"key":"B139","first-page":"1","article-title":"\u201cGoing deeper with convolutions,\u201d","volume-title":"Proc. of IEEE International Conference on Computer Vision and Pattern Recognition","author":"Szegedy","year":"2015"},{"key":"B140","unstructured":"\u201cScaling monosemanticity: Extracting interpretable features from claude 3 sonnet,\u201d\n          \n          \n            \n              Templeton\n              A.\n            \n            \n              Conerly\n              T.\n            \n            \n              Marcus\n              J.\n            \n            \n              Lindsey\n              J.\n            \n            \n              Bricken\n              T.\n            \n            \n              Chen\n              B.\n            \n          \n          Transformer Circuits Thread\n          \n          2024"},{"key":"B141","doi-asserted-by":"crossref","first-page":"4593","DOI":"10.18653\/v1\/P19-1452","article-title":"\u201cBeRT rediscovers the classical NLP pipeline,\u201d","volume-title":"57th Annual Meeting of the Association for Computational Linguistics","author":"Tenney","year":"2019"},{"key":"B142","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2305.16380","article-title":"Scan and snap: understanding training dynamics and token composition in 1-layer Transformer","author":"Tian","year":"2023","journal-title":"arXiv"},{"key":"B143","volume-title":"Exploring GPT-3","author":"Tingiris","year":"2022"},{"key":"B144","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2302.13971","article-title":"LLaMA: Open and efficient foundation language models","author":"Touvron","year":"2023","journal-title":"arXiv"},{"key":"B145","doi-asserted-by":"publisher","first-page":"e13309","DOI":"10.1111\/cogs.13309","article-title":"Do large language models know what humans know?","volume":"47","author":"Trott","year":"2023","journal-title":"Cogn. Sci"},{"key":"B146","doi-asserted-by":"publisher","first-page":"433","DOI":"10.1093\/mind\/LIX.236.433","article-title":"Computing machinery and intelligence","volume":"59","author":"Turing","year":"1950","journal-title":"Mind"},{"key":"B147","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-662-55565-1","volume-title":"Computational Artefacts: Towards a Philosophy of Computer Science","author":"Turner","year":"2018"},{"key":"B148","doi-asserted-by":"publisher","first-page":"142","DOI":"10.3389\/fpsyg.2017.00142","article-title":"Perception science in the age of deep neural networks","volume":"8","author":"VanRullen","year":"2017","journal-title":"Front. Psychol"},{"key":"B149","first-page":"30","article-title":"\u201cAttention is all you need,\u201d","volume-title":"Advances in Neural Information Processing Systems","author":"Vaswani","year":"2017"},{"key":"B150","first-page":"5483","article-title":"\u201cPluralism in social cognition and predictive processing,\u201d","volume-title":"Proceedings of the Annual Meeting of the Cognitive Science Society, Vol, 46","author":"Venter","year":"2024"},{"key":"B151","first-page":"35151","article-title":"\u201cTransformers learn in-context by gradient descent,\u201d","volume-title":"International Conference on Machine Learning","author":"Von Oswald","year":"2023"},{"key":"B152","first-page":"11080","article-title":"\u201cThinking like transformers,\u201d","volume-title":"International Conference on Machine Learning","author":"Weiss","year":"2021"},{"key":"B153","article-title":"\u201cOn the role of unstructured training data in transformers' in-context learning capabilities,\u201d","volume-title":"NeurIPS 2023 Workshop on Mathematics of Modern Machine Learning","author":"Wibisono","year":"2023"},{"key":"B154","doi-asserted-by":"crossref","DOI":"10.7208\/chicago\/9780226902050.001.0001","volume-title":"Science in the Age of Computer Simulation","author":"Winsberg","year":"2010"},{"key":"B155","article-title":"\u201cComputer simulations in science,\u201d","author":"Winsberg","year":"2022","journal-title":"The Stanford Encyclopedia of Philosophy"},{"key":"B156","author":"Wolfram","year":"2023","journal-title":"What Is ChatGPT Doing"},{"key":"B157","volume-title":"Making Things Happen: A Theory of Causal Explanation","author":"Woodward","year":"2003"},{"key":"B158","first-page":"70","article-title":"\u201cExplanation in neurobiology: An interventionist perspective,\u201d","author":"Woodward","year":"2018","journal-title":"Explanation and Integration in Mind and Brain Science"},{"key":"B159","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2023.3327163","article-title":"AttentionViz: A global view of Transformer attention","author":"Yeh","year":"2023","journal-title":"arXiv"},{"key":"B160","doi-asserted-by":"publisher","first-page":"265","DOI":"10.1007\/s13347-019-00382-7","article-title":"Solving the black box problem: A normative framework for explainable artificial intelligence","volume":"40","author":"Zednik","year":"2021","journal-title":"Philosophy and Technology"},{"key":"B161","doi-asserted-by":"publisher","first-page":"1","DOI":"10.48550\/arXiv.2303.10158","article-title":"Data-centric artificial intelligence: a survey","volume":"57","author":"Zha","year":"2025","journal-title":"ACM Comp."},{"key":"B162","doi-asserted-by":"publisher","first-page":"560","DOI":"10.1111\/j.1551-6709.2012.01238.x","article-title":"Perspective-taking and depth of theory-of-mind reasoning in sequential-move games","volume":"36","author":"Zhang","year":"2012","journal-title":"Cogn. Sci"},{"key":"B163","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2305.19420","article-title":"What and how does in-context learning learn? bayesian model averaging, parameterization, and generalization","author":"Zhang","year":"2023","journal-title":"arXiv"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2025.1509338\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,4,1]],"date-time":"2025-04-01T05:29:20Z","timestamp":1743485360000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2025.1509338\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,1]]},"references-count":163,"alternative-id":["10.3389\/frai.2025.1509338"],"URL":"https:\/\/doi.org\/10.3389\/frai.2025.1509338","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,4,1]]},"article-number":"1509338"}}