{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,14]],"date-time":"2026-03-14T12:49:41Z","timestamp":1773492581143,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":46,"publisher":"ACM","license":[{"start":{"date-parts":[[2024,9,16]],"date-time":"2024-09-16T00:00:00Z","timestamp":1726444800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"UK Research & Innovation","award":["EP\/S02266X\/1"],"award-info":[{"award-number":["EP\/S02266X\/1"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,9,16]]},"DOI":"10.1145\/3652988.3673960","type":"proceedings-article","created":{"date-parts":[[2024,12,26]],"date-time":"2024-12-26T12:22:57Z","timestamp":1735215777000},"page":"1-9","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Exploring Theory of Mind in Large Language Models through Multimodal Negotiation"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1332-0727","authenticated-orcid":false,"given":"Nutchanon","family":"Yongsatianchot","sequence":"first","affiliation":[{"name":"Department of Electrical and Computer Engineering, Faculty of Engineering, Thammasat School of Engineering, Thammasat University, Thailand"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3359-1527","authenticated-orcid":false,"given":"Tobias","family":"Thejll-Madsen","sequence":"additional","affiliation":[{"name":"University of Glasgow, United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5711-7934","authenticated-orcid":false,"given":"Stacy","family":"Marsella","sequence":"additional","affiliation":[{"name":"Northeastern University, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,12,26]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1111\/1469-7610.00715"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.3390\/s24010126"},{"key":"e_1_3_2_1_3_1","volume-title":"Language models are few-shot learners. Advances in neural information processing systems 33","author":"Brown Tom","year":"2020","unstructured":"Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared\u00a0D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877\u20131901."},{"key":"e_1_3_2_1_4_1","volume-title":"Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712","author":"Bubeck S\u00e9bastien","year":"2023","unstructured":"S\u00e9bastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin\u00a0Tat Lee, Yuanzhi Li, Scott Lundberg, 2023. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712 (2023)."},{"key":"e_1_3_2_1_5_1","volume-title":"Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi.","author":"Dai Wenliang","year":"2023","unstructured":"Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng\u00a0Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. 2023. InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning. arxiv:2305.06500\u00a0[cs.CV]"},{"key":"e_1_3_2_1_6_1","volume-title":"Evaluating language model agency through negotiations. arXiv preprint arXiv:2401.04536","author":"Davidson R","year":"2024","unstructured":"Tim\u00a0R Davidson, Veniamin Veselovsky, Martin Josifoski, Maxime Peyrard, Antoine Bosselut, Michal Kosinski, and Robert West. 2024. Evaluating language model agency through negotiations. arXiv preprint arXiv:2401.04536 (2024)."},{"key":"e_1_3_2_1_7_1","volume-title":"The 10th International Conference on Autonomous Agents and Multiagent Systems -","volume":"3","author":"de Melo M.","year":"2011","unstructured":"Celso\u00a0M. de Melo, Peter Carnevale, and Jonathan Gratch. [n. d.]. The effect of expression of anger and happiness in computer agents on negotiations with humans. In The 10th International Conference on Autonomous Agents and Multiagent Systems - Volume 3 (Richland, SC, 2011-05-02) (AAMAS \u201911). International Foundation for Autonomous Agents and Multiagent Systems, 937\u2013944."},{"key":"e_1_3_2_1_8_1","volume-title":"Reading people\u2019s minds from emotion expressions in interdependent decision making.Journal of personality and social psychology 106, 1","author":"De\u00a0Melo M","year":"2014","unstructured":"Celso\u00a0M De\u00a0Melo, Peter\u00a0J Carnevale, Stephen\u00a0J Read, and Jonathan Gratch. 2014. Reading people\u2019s minds from emotion expressions in interdependent decision making.Journal of personality and social psychology 106, 1 (2014), 73."},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"crossref","unstructured":"Morteza Dehghani Peter\u00a0J. Carnevale and Jonathan Gratch. [n. d.]. Interpersonal effects of expressed anger and sorrow in morally charged negotiation. 9 2 ([n. d.]) 104\u2013113.","DOI":"10.1017\/S1930297500005477"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","unstructured":"Shelley Derksen and H.\u00a0J. Keselman. [n. d.]. Backward forward and stepwise automated subset selection algorithms: Frequency of obtaining authentic and noise variables. 45 2 ([n. d.]) 265\u2013282. https:\/\/doi.org\/10.1111\/j.2044-8317.1992.tb00992.x","DOI":"10.1111\/j.2044-8317.1992.tb00992.x"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"crossref","unstructured":"P Ekman and H Oster. [n. d.]. Facial Expressions of Emotion. 30 ([n. d.]) 527\u201354.","DOI":"10.1146\/annurev.ps.30.020179.002523"},{"key":"e_1_3_2_1_12_1","volume-title":"Theory of mind. Current biology 15, 17","author":"Frith Chris","year":"2005","unstructured":"Chris Frith and Uta Frith. 2005. Theory of mind. Current biology 15, 17 (2005), R644\u2013R645."},{"key":"e_1_3_2_1_13_1","volume-title":"Improving language model negotiation with self-play and in-context learning from ai feedback. arXiv preprint arXiv:2305.10142","author":"Fu Yao","year":"2023","unstructured":"Yao Fu, Hao Peng, Tushar Khot, and Mirella Lapata. 2023. Improving language model negotiation with self-play and in-context learning from ai feedback. arXiv preprint arXiv:2305.10142 (2023)."},{"key":"e_1_3_2_1_14_1","volume-title":"Understanding social reasoning in language models with language models. Advances in Neural Information Processing Systems 36","author":"Gandhi Kanishk","year":"2024","unstructured":"Kanishk Gandhi, Jan-Philipp Fr\u00e4nken, Tobias Gerstenberg, and Noah Goodman. 2024. Understanding social reasoning in language models with language models. Advances in Neural Information Processing Systems 36 (2024)."},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-47665-0_25"},{"key":"e_1_3_2_1_16_1","volume-title":"Negotiation as a Challenge Problem for Virtual Humans","author":"Gratch Jonathan","unstructured":"Jonathan Gratch, David DeVault, Gale\u00a0M. Lucas, and Stacy Marsella. 2015. Negotiation as a Challenge Problem for Virtual Humans. In Intelligent Virtual Agents, Willem-Paul Brinkman, Joost Broekens, and Dirk Heylen (Eds.). Springer International Publishing, 201\u2013215."},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4419-9863-7_1213"},{"key":"e_1_3_2_1_18_1","volume-title":"Exploring the Function of Expressions in Negotiation: The DyNego-WOZ Corpus","author":"Hoegen Jessie","year":"2022","unstructured":"Jessie Hoegen, David DeVault, and Jonathan Gratch. 2022. Exploring the Function of Expressions in Negotiation: The DyNego-WOZ Corpus. IEEE Transactions on Affective Computing (2022)."},{"key":"e_1_3_2_1_19_1","volume-title":"Mmtom-qa: Multimodal theory of mind question answering. arXiv preprint arXiv:2401.08743","author":"Jin Chuanyang","year":"2024","unstructured":"Chuanyang Jin, Yutong Wu, Jing Cao, Jiannan Xiang, Yen-Ling Kuo, Zhiting Hu, Tomer Ullman, Antonio Torralba, Joshua\u00a0B Tenenbaum, and Tianmin Shu. 2024. Mmtom-qa: Multimodal theory of mind question answering. arXiv preprint arXiv:2401.08743 (2024)."},{"key":"e_1_3_2_1_20_1","first-page":"3","article-title":"BiLAT: A Game-Based Environment for Practicing Negotiation in a Cultural","volume":"19","author":"Kim M.","year":"2009","unstructured":"Julia\u00a0M. Kim, Randall\u00a0W. Hill, Paula\u00a0J. Durlach, H.\u00a0Chad Lane, Eric Forbell, Mark Core, Stacy Marsella, David Pynadath, and John Hart. 2009. BiLAT: A Game-Based Environment for Practicing Negotiation in a Cultural Context. Int. J. Artif. Intell. Ed. 19, 3 (aug 2009), 289\u2013308.","journal-title":"Context. Int. J. Artif. Intell. Ed."},{"key":"e_1_3_2_1_21_1","volume-title":"Theory of mind may have spontaneously emerged in large language models. arXiv preprint arXiv:2302.02083 4","author":"Kosinski Michal","year":"2023","unstructured":"Michal Kosinski. 2023. Theory of mind may have spontaneously emerged in large language models. arXiv preprint arXiv:2302.02083 4 (2023), 169."},{"key":"e_1_3_2_1_22_1","volume-title":"Are LLMs Effective Negotiators? Systematic Evaluation of the Multifaceted Capabilities of LLMs in Negotiation Dialogues. arXiv preprint arXiv:2402.13550","author":"Kwon Deuksin","year":"2024","unstructured":"Deuksin Kwon, Emily Weiss, Tara Kulshrestha, Kushal Chawla, Gale\u00a0M Lucas, and Jonathan Gratch. 2024. Are LLMs Effective Negotiators? Systematic Evaluation of the Multifaceted Capabilities of LLMs in Negotiation Dialogues. arXiv preprint arXiv:2402.13550 (2024)."},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1598"},{"key":"e_1_3_2_1_24_1","unstructured":"Raz Lin Yinon Oshrat and Sarit Kraus. [n. d.]. Investigating the Bene\ufb01ts of Automated Negotiations in Enhancing People\u2019s Negotiation Skills. ([n. d.])."},{"key":"e_1_3_2_1_25_1","volume-title":"Visual instruction tuning. Advances in neural information processing systems 36","author":"Liu Haotian","year":"2024","unstructured":"Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong\u00a0Jae Lee. 2024. Visual instruction tuning. Advances in neural information processing systems 36 (2024)."},{"key":"e_1_3_2_1_26_1","volume-title":"Computational models of emotion inference in theory of mind: A review and roadmap. Topics in cognitive science 11, 2","author":"Ong C","year":"2019","unstructured":"Desmond\u00a0C Ong, Jamil Zaki, and Noah\u00a0D Goodman. 2019. Computational models of emotion inference in theory of mind: A review and roadmap. Topics in cognitive science 11, 2 (2019), 338\u2013357."},{"key":"e_1_3_2_1_27_1","unstructured":"OpenAI. 2023. GPT-4 Technical Report. arXiv preprint arXiv:2303.08774 (2023)."},{"key":"e_1_3_2_1_28_1","first-page":"27730","article-title":"Training language models to follow instructions with human feedback","volume":"35","author":"Ouyang Long","year":"2022","unstructured":"Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, 2022. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems 35 (2022), 27730\u201327744.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_1_29_1","volume-title":"Does the chimpanzee have a theory of mind?Behavioral and brain sciences 1, 4","author":"Premack David","year":"1978","unstructured":"David Premack and Guy Woodruff. 1978. Does the chimpanzee have a theory of mind?Behavioral and brain sciences 1, 4 (1978), 515\u2013526."},{"key":"e_1_3_2_1_30_1","volume-title":"R: A Language and Environment for Statistical Computing","author":"Team R Core","year":"2023","unstructured":"R Core Team. 2023. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. https:\/\/www.R-project.org\/"},{"key":"e_1_3_2_1_31_1","volume-title":"Teaching reverse appraisal to improve negotiation skills","author":"Sato Motoaki","year":"2023","unstructured":"Motoaki Sato, Kazunori Terada, and Jonathan Gratch. 2023. Teaching reverse appraisal to improve negotiation skills. IEEE Transactions on Affective Computing (2023)."},{"key":"e_1_3_2_1_32_1","volume-title":"Clever hans or neural theory of mind? stress testing social reasoning in large language models. arXiv preprint arXiv:2305.14763","author":"Shapira Natalie","year":"2023","unstructured":"Natalie Shapira, Mosh Levy, Seyed\u00a0Hossein Alavi, Xuhui Zhou, Yejin Choi, Yoav Goldberg, Maarten Sap, and Vered Shwartz. 2023. Clever hans or neural theory of mind? stress testing social reasoning in large language models. arXiv preprint arXiv:2305.14763 (2023)."},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.findings-acl.663"},{"key":"e_1_3_2_1_34_1","volume-title":"Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971","author":"Touvron Hugo","year":"2023","unstructured":"Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth\u00e9e Lacroix, Baptiste Rozi\u00e8re, Naman Goyal, Eric Hambro, Faisal Azhar, 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023)."},{"key":"e_1_3_2_1_35_1","volume-title":"J. & Clobert, M.(in press). Cultural influences on emotion: Empirical patterns and emerging trends. In S. Kitayama & D. Cohen (Eds). Handbook of cultural psychology","author":"Tsai L","unstructured":"Jeanne\u00a0L Tsai. [n. d.]. Tsai, J. & Clobert, M.(in press). Cultural influences on emotion: Empirical patterns and emerging trends. In S. Kitayama & D. Cohen (Eds). Handbook of cultural psychology. Oxford University Press. ([n. d.])."},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1177\/0022022197285006"},{"key":"e_1_3_2_1_37_1","volume-title":"Large language models fail on trivial alterations to theory-of-mind tasks. arXiv preprint arXiv:2302.08399","author":"Ullman Tomer","year":"2023","unstructured":"Tomer Ullman. 2023. Large language models fail on trivial alterations to theory-of-mind tasks. arXiv preprint arXiv:2302.08399 (2023)."},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1002\/ejsp.320"},{"key":"e_1_3_2_1_39_1","volume-title":"Attention is all you need. Advances in neural information processing systems 30","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan\u00a0N Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017)."},{"key":"e_1_3_2_1_40_1","volume-title":"Theory of Mind abilities of Large Language Models in Human-Robot Interaction: An Illusion?arXiv preprint arXiv:2401.05302","author":"Verma Mudit","year":"2024","unstructured":"Mudit Verma, Siddhant Bhambri, and Subbarao Kambhampati. 2024. Theory of Mind abilities of Large Language Models in Human-Robot Interaction: An Illusion?arXiv preprint arXiv:2401.05302 (2024)."},{"key":"e_1_3_2_1_41_1","volume-title":"Denny Zhou","author":"Wei Jason","year":"2022","unstructured":"Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc\u00a0V Le, Denny Zhou, 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35 (2022), 24824\u201324837."},{"key":"e_1_3_2_1_42_1","volume-title":"The child\u2019s theory of mind","author":"Wellman M","unstructured":"Henry\u00a0M Wellman, Susan Carey, Lila Gleitman, Elissa\u00a0L Newport, and Elizabeth\u00a0S Spelke. 1990. The child\u2019s theory of mind. The MIT Press."},{"key":"e_1_3_2_1_43_1","volume-title":"Natural theories of mind: Evolution, development and simulation of everyday mindreading","author":"Whiten Andrew","unstructured":"Andrew Whiten and RW Byrne. 1991. Natural theories of mind: Evolution, development and simulation of everyday mindreading. B. Blackwell Oxford, UK."},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.21105\/joss.01686"},{"key":"e_1_3_2_1_45_1","unstructured":"Yujian Yuan Jiabei Zeng and Shiguang Shan. 2023. Describe Your Facial Expressions by Linking Image Encoders and Large Language Models. (2023)."},{"key":"e_1_3_2_1_46_1","volume-title":"How FaR Are Large Language Models From Agents with Theory-of-Mind?arXiv preprint arXiv:2310.03051","author":"Zhou Pei","year":"2023","unstructured":"Pei Zhou, Aman Madaan, Srividya\u00a0Pranavi Potharaju, Aditya Gupta, Kevin\u00a0R McKee, Ari Holtzman, Jay Pujara, Xiang Ren, Swaroop Mishra, Aida Nematzadeh, 2023. How FaR Are Large Language Models From Agents with Theory-of-Mind?arXiv preprint arXiv:2310.03051 (2023)."}],"event":{"name":"IVA '24: ACM International Conference on Intelligent Virtual Agents","location":"GLASGOW United Kingdom","acronym":"IVA '24","sponsor":["SIGAI ACM Special Interest Group on Artificial Intelligence"]},"container-title":["Proceedings of the ACM International Conference on Intelligent Virtual Agents"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3652988.3673960","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3652988.3673960","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,22]],"date-time":"2025-08-22T17:11:12Z","timestamp":1755882672000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3652988.3673960"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,16]]},"references-count":46,"alternative-id":["10.1145\/3652988.3673960","10.1145\/3652988"],"URL":"https:\/\/doi.org\/10.1145\/3652988.3673960","relation":{},"subject":[],"published":{"date-parts":[[2024,9,16]]},"assertion":[{"value":"2024-12-26","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}