{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,26]],"date-time":"2026-03-26T19:24:19Z","timestamp":1774553059590,"version":"3.50.1"},"reference-count":61,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2023,8,4]],"date-time":"2023-08-04T00:00:00Z","timestamp":1691107200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000780","name":"European Union\u2019s Horizon 2020 research and innovation programme","doi-asserted-by":"publisher","award":["820742"],"award-info":[{"award-number":["820742"]}],"id":[{"id":"10.13039\/501100000780","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000780","name":"European Union\u2019s Horizon 2020 research and innovation programme","doi-asserted-by":"publisher","award":["101071178"],"award-info":[{"award-number":["101071178"]}],"id":[{"id":"10.13039\/501100000780","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000780","name":"European Union\u2019s Horizon EIC Grants 2021","doi-asserted-by":"publisher","award":["820742"],"award-info":[{"award-number":["820742"]}],"id":[{"id":"10.13039\/501100000780","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000780","name":"European Union\u2019s Horizon EIC Grants 2021","doi-asserted-by":"publisher","award":["101071178"],"award-info":[{"award-number":["101071178"]}],"id":[{"id":"10.13039\/501100000780","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>A major challenge in cognitive science and AI has been to understand how intelligent autonomous agents might acquire and predict the behavioral and mental states of other agents in the course of complex social interactions. How does such an agent model the goals, beliefs, and actions of other agents it interacts with? What are the computational principles to model a Theory of Mind (ToM)? Deep learning approaches to address these questions fall short of a better understanding of the problem. In part, this is due to the black-box nature of deep networks, wherein computational mechanisms of ToM are not readily revealed. Here, we consider alternative hypotheses seeking to model how the brain might realize a ToM. In particular, we propose embodied and situated agent models based on distributed adaptive control theory to predict the actions of other agents in five different game-theoretic tasks (Harmony Game, Hawk-Dove, Stag Hunt, Prisoner\u2019s Dilemma, and Battle of the Exes). Our multi-layer control models implement top-down predictions from adaptive to reactive layers of control and bottom-up error feedback from reactive to adaptive layers. We test cooperative and competitive strategies among seven different agent models (cooperative, greedy, tit-for-tat, reinforcement-based, rational, predictive, and internal agents). We show that, compared to pure reinforcement-based strategies, probabilistic learning agents modeled on rational, predictive, and internal phenotypes perform better in game-theoretic metrics across tasks. The outlined autonomous multi-agent models might capture systems-level processes underlying a ToM and suggest architectural principles of ToM from a control-theoretic perspective.<\/jats:p>","DOI":"10.3390\/info14080441","type":"journal-article","created":{"date-parts":[[2023,8,4]],"date-time":"2023-08-04T09:28:34Z","timestamp":1691141314000},"page":"441","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Modeling Theory of Mind in Dyadic Games Using Adaptive Feedback Control"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5740-1513","authenticated-orcid":false,"given":"Ismael T.","family":"Freire","sequence":"first","affiliation":[{"name":"Donders Institute for Brain, Cognition and Behaviour, Radboud University, 6525 AJ Nijmegen, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1485-1853","authenticated-orcid":false,"given":"Xerxes D.","family":"Arsiwalla","sequence":"additional","affiliation":[{"name":"Department of Information and Communication Technologies, Universitat Pompeu Fabra, 08018 Barcelona, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jordi-Ysard","family":"Puigb\u00f2","sequence":"additional","affiliation":[{"name":"Department of Information and Communication Technologies, Universitat Pompeu Fabra, 08018 Barcelona, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Paul","family":"Verschure","sequence":"additional","affiliation":[{"name":"Donders Institute for Brain, Cognition and Behaviour, Radboud University, 6525 AJ Nijmegen, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,8,4]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"515","DOI":"10.1017\/S0140525X00076512","article-title":"Does the chimpanzee have a theory of mind?","volume":"1","author":"Premack","year":"1978","journal-title":"Behav. Brain Sci."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1016\/0010-0277(85)90022-8","article-title":"Does the autistic child have a \u201ctheory of mind\u201d?","volume":"21","author":"Leslie","year":"1985","journal-title":"Cognition"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/0010-0277(90)90051-K","article-title":"The infant\u2019s theory of self-propelled objects","volume":"36","author":"Premack","year":"1990","journal-title":"Cognition"},{"key":"ref_4","unstructured":"Lanctot, M., Zambaldi, V., Gruslys, A., Lazaridou, A., Tuyls, K., P\u00e9rolat, J., Silver, D., and Graepel, T. (2017, January 4\u20139). A unified game-theoretic approach to multiagent reinforcement learning. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_5","unstructured":"Lerer, A., and Peysakhovich, A. (2018). Learning social conventions in markov games. arXiv."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Zhao, Z., Zhao, F., Zhao, Y., Zeng, Y., and Sun, Y. (2023). A brain-inspired theory of mind spiking neural network improves multi-agent cooperation and competition. Patterns, 100775.","DOI":"10.1016\/j.patter.2023.100775"},{"key":"ref_7","unstructured":"Rabinowitz, N.C., Perbet, F., Song, H.F., Zhang, C., Eslami, S., and Botvinick, M. (2018). Machine Theory of Mind. arXiv."},{"key":"ref_8","unstructured":"Sclar, M., Neubig, G., and Bisk, Y. (2022, January 17\u201323). Symmetric machine theory of mind. Proceedings of the International Conference on Machine Learning, PMLR, Baltimore, MD, USA."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/j.neunet.2014.09.003","article-title":"Deep learning in neural networks: An overview","volume":"61","author":"Schmidhuber","year":"2015","journal-title":"Neural Netw."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Yoshida, W., Dolan, R.J., and Friston, K.J. (2008). Game theory of mind. PLoS Comput. Biol., 4.","DOI":"10.1371\/journal.pcbi.1000254"},{"key":"ref_11","unstructured":"Baker, C., Saxe, R., and Tenenbaum, J. (2011, January 20\u201323). Bayesian Theory of Mind: Modeling Joint Belief-Desire Attribution. Proceedings of the Annual Meeting of the Cognitive Science Society, Boston, MA, USA."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"0064","DOI":"10.1038\/s41562-017-0064","article-title":"Rational quantitative attribution of beliefs, desires and percepts in human mentalizing","volume":"1","author":"Baker","year":"2017","journal-title":"Nat. Hum. Behav."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"e253","DOI":"10.1017\/S0140525X16001837","article-title":"Building machines that learn and think like people","volume":"40","author":"Lake","year":"2017","journal-title":"Behav. Brain Sci."},{"key":"ref_14","unstructured":"Berke, M., and Jara-Ettinger, J. (2022, January 27\u201330). Integrating Experience into Bayesian Theory of Mind. Proceedings of the Annual Meeting of the Cognitive Science Society, Toronto, ON, Canada."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Abbeel, P., and Ng, A.Y. (2004, January 4\u20138). Apprenticeship learning via inverse reinforcement learning. Proceedings of the twenty-First International Conference on Machine Learning, Banff, AB, Canada.","DOI":"10.1145\/1015330.1015430"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1016\/j.cobeha.2019.04.010","article-title":"Theory of mind as inverse reinforcement learning","volume":"29","year":"2019","journal-title":"Curr. Opin. Behav. Sci."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Wu, H., Sequeira, P., and Pynadath, D.V. (2023). Multiagent Inverse Reinforcement Learning via Theory of Mind Reasoning. arXiv.","DOI":"10.1109\/ICCCR56747.2023.10194048"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Ruiz-Serra, J., and Harr\u00e9, M.S. (2023). Inverse Reinforcement Learning as the Algorithmic Basis for Theory of Mind: Current Methods and Open Problems. Algorithms, 16.","DOI":"10.3390\/a16020068"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Kahneman, D., Slovic, P., and Tversky, A. (1982). Judgment under Uncertainty: Heuristics and Biases, Cambridge University Press.","DOI":"10.1017\/CBO9780511809477"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1057","DOI":"10.1017\/S0033291720000835","article-title":"Knowing me, knowing you: Theory of mind in AI","volume":"50","author":"Cuzzolin","year":"2020","journal-title":"Psychol. Med."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1016\/j.artint.2018.01.002","article-title":"Autonomous agents modelling other agents: A comprehensive survey and open problems","volume":"258","author":"Albrecht","year":"2018","journal-title":"Artif. Intell."},{"key":"ref_22","unstructured":"Wang, Y., Zhong, F., Xu, J., and Wang, Y. (2021). Tom2c: Target-oriented multi-agent communication and cooperation with theory of mind. arXiv."},{"key":"ref_23","unstructured":"Yuan, L., Fu, Z., Zhou, L., Yang, K., and Zhu, S.C. (2021). Emergence of theory of mind collaboration in multiagent systems. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Freire, I.T., Moulin-Frier, C., Sanchez-Fibla, M., Arsiwalla, X.D., and Verschure, P.F. (2020). Modeling the formation of social conventions from embodied real-time interactions. PLoS ONE, 15.","DOI":"10.1371\/journal.pone.0234434"},{"key":"ref_25","first-page":"297","article-title":"Limits of Multi-Agent Predictive Models in the Formation of Social Conventions","volume":"308","author":"Freire","year":"2018","journal-title":"Artif. Intell. Res. Dev. Curr. Chall. New Trends Appl."},{"key":"ref_26","unstructured":"K\u00f6ster, R., McKee, K.R., Everett, R., Weidinger, L., Isaac, W.S., Hughes, E., Du\u00e9\u00f1ez-Guzm\u00e1n, E.A., Graepel, T., Botvinick, M., and Leibo, J.Z. (2020). Model-free conventions in multi-agent reinforcement learning with heterogeneous preferences. arXiv."},{"key":"ref_27","unstructured":"Kleiman-Weiner, M., Ho, M.K., Austerweil, J.L., Littman, M.L., and Tenenbaum, J.B. (2016, January 10\u201313). Coordinate to cooperate or compete: Abstract goals and joint intentions in social interaction. Proceedings of the CogSci, Philadelphia, PA, USA."},{"key":"ref_28","unstructured":"Perolat, J., Leibo, J.Z., Zambaldi, V., Beattie, C., Tuyls, K., and Graepel, T. (2017, January 4\u20139). A multi-agent reinforcement learning model of common-pool resource appropriation. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_29","unstructured":"Peysakhovich, A., and Lerer, A. (2018, January 10\u201315). Prosocial learning agents solve generalized stag hunts better than selfish ones. Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems, Stockholm, Sweden."},{"key":"ref_30","unstructured":"Freire, I.T., Puigb\u00f2, J.Y., Arsiwalla, X.D., and Verschure, P.F. (2018). Proceedings of the Conference on Biomimetic and Biohybrid Systems, Springer."},{"key":"ref_31","first-page":"307","article-title":"Loss Aversion Fosters Coordination in Independent Reinforcement Learners","volume":"308","author":"Gaparrini","year":"2018","journal-title":"Artif. Intell. Res. Dev. Curr. Challenges New Trends Appl."},{"key":"ref_32","unstructured":"Leibo, J.Z., Zambaldi, V., Lanctot, M., Marecki, J., and Graepel, T. (2017, January 8\u201312). Multi-agent reinforcement learning in sequential social dilemmas. Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems, Sao Paulo, Brazil."},{"key":"ref_33","unstructured":"Peysakhovich, A., and Lerer, A. (2017). Consequentialist conditional cooperation in social dilemmas with imperfect information. arXiv."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1073\/pnas.36.1.48","article-title":"Equilibrium points in n-person games","volume":"36","author":"Nash","year":"1950","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Hawkins, R.X., and Goldstone, R.L. (2016). The formation of social conventions in real-time environments. PLoS ONE, 11.","DOI":"10.1371\/journal.pone.0151670"},{"key":"ref_36","unstructured":"Hawkins, R.X., Goodman, N.D., and Goldstone, R.L. (2018). The emergence of social norms and conventions. Trends Cogn. Sci."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"e1600451","DOI":"10.1126\/sciadv.1600451","article-title":"Humans display a reduced set of consistent behavioral phenotypes in dyadic games","volume":"2","author":"Vicens","year":"2016","journal-title":"Sci. Adv."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"598","DOI":"10.1126\/science.1142996","article-title":"Social decision-making: Insights from game theory and neuroscience","volume":"318","author":"Sanfey","year":"2007","journal-title":"Science"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"620","DOI":"10.1038\/nature02024","article-title":"Environmentally mediated synergy between perception and behaviour in mobile robots","volume":"425","author":"Verschure","year":"2003","journal-title":"Nature"},{"key":"ref_40","unstructured":"Moulin-Frier, C., Arsiwalla, X.D., Puigb\u00f2, J.Y., Sanchez-Fibla, M., Duff, A., and Verschure, P.F. (2016, January 9). Top-Down and Bottom-Up Interactions between Low-Level Reactive Control and Symbolic Rule Learning in Embodied Agents. Proceedings of the CoCo@ NIPS, Barcelona, Spain."},{"key":"ref_41","unstructured":"Braitenberg, V. (1986). Vehicles: Experiments in Synthetic Psychology, MIT Press."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1038\/nrn755","article-title":"Control of goal-directed and stimulus-driven attention in the brain","volume":"3","author":"Corbetta","year":"2002","journal-title":"Nat. Rev. Neurosci."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1181","DOI":"10.1126\/science.1088545","article-title":"The architecture of cognitive control in the human prefrontal cortex","volume":"302","author":"Koechlin","year":"2003","journal-title":"Science"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"453","DOI":"10.1016\/j.tics.2011.07.011","article-title":"A unified framework for inhibitory control","volume":"15","author":"Munakata","year":"2011","journal-title":"Trends Cogn. Sci."},{"key":"ref_45","first-page":"548","article-title":"How prediction errors shape perception, attention, and motivation","volume":"3","author":"Kok","year":"2012","journal-title":"Front. Psychol."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"20754","DOI":"10.1073\/pnas.1117807108","article-title":"Evidence for a hierarchy of predictions and prediction errors in human cortex","volume":"108","author":"Wacongne","year":"2011","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1007\/BF00115009","article-title":"Learning to predict by the methods of temporal differences","volume":"3","author":"Sutton","year":"1988","journal-title":"Mach. Learn."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"1390","DOI":"10.1126\/science.7466396","article-title":"The evolution of cooperation","volume":"211","author":"Axelrod","year":"1981","journal-title":"Science"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1177\/002200278002400101","article-title":"Effective choice in the prisoner\u2019s dilemma","volume":"24","author":"Axelrod","year":"1980","journal-title":"J. Confl. Resolut."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J."},{"key":"ref_51","unstructured":"Lengyel, M., and Dayan, P. (2007). Hippocampal contributions to control: The third way. Adv. Neural Inf. Process. Syst., 20."},{"key":"ref_52","unstructured":"Freire, I.T., Amil, A.F., and Verschure, P.F. (2021). Sequential Episodic Control. arXiv."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"1052998","DOI":"10.3389\/frobt.2022.1052998","article-title":"Drive competition underlies effective allostatic orchestration","volume":"9","author":"Rosado","year":"2022","journal-title":"Front. Robot. AI"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"178","DOI":"10.1126\/science.aar8644","article-title":"Sensitivity to \u201csunk costs\u201d in mice, rats, and humans","volume":"361","author":"Sweis","year":"2018","journal-title":"Science"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Tuti\u0107, A., and Voss, T. (2020). Trust and game theory. The Routledge Handbook of Trust and Philosophy, Routledge.","DOI":"10.4324\/9781315542294-14"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Moulin-Frier, C., Puigbo, J.Y., Arsiwalla, X.D., Sanchez-Fibla, M., and Verschure, P. (2017). Embodied artificial intelligence through distributed adaptive control: An integrated framework. arXiv.","DOI":"10.1109\/DEVLRN.2017.8329825"},{"key":"ref_57","unstructured":"Freire, I.T., Urikh, D., Arsiwalla, X.D., and Verschure, P.F. (2020). Proceedings of the Conference on Biomimetic and Biohybrid Systems, Springer."},{"key":"ref_58","unstructured":"Arsiwalla, X.D., Herreros, I., Moulin-Frier, C., S\u00e1nchez-Fibla, M., and Verschure, P.F. (2016, January 19\u201321). Is Consciousness a Control Process?. Proceedings of the CCIA, Catalonia, Spain."},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Arsiwalla, X.D., Sole, R., Moulin-Frier, C., Herreros, I., Sanchez-Fibla, M., and Verschure, P. (2017). The Morphospace of Consciousness. arXiv.","DOI":"10.31234\/osf.io\/emhnp"},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"521","DOI":"10.1017\/S0140525X00031368","article-title":"Imitation, cultural learning and the origins of \u201ctheory of mind\u201d","volume":"16","author":"Gopnik","year":"1993","journal-title":"Behav. Brain Sci."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"e44","DOI":"10.1017\/ehs.2021.40","article-title":"Coevolution of actions, personal norms and beliefs about others in social dilemmas","volume":"3","author":"Gavrilets","year":"2021","journal-title":"Evol. Hum. Sci."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/14\/8\/441\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:25:46Z","timestamp":1760127946000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/14\/8\/441"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,4]]},"references-count":61,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2023,8]]}},"alternative-id":["info14080441"],"URL":"https:\/\/doi.org\/10.3390\/info14080441","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,4]]}}}