{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T07:43:06Z","timestamp":1791877386217,"version":"4.3.4"},"reference-count":70,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2025,5,2]],"date-time":"2025-05-02T00:00:00Z","timestamp":1746144000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Science Foundation","award":["NSF-2418582"],"award-info":[{"award-number":["NSF-2418582"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2025,5,2]]},"abstract":"<jats:p>\n                    Significant advancements have occurred in the application of Large Language Models (LLMs) for social simulations. Despite this, their abilities to perform teaming in task-oriented social events are underexplored. Such capabilities are crucial if LLMs are to effectively mimic human-like social behaviors and form efficient teams to solve tasks. To bridge this gap, we introduce\n                    <jats:sc>MetaAgents<\/jats:sc>\n                    , a social simulation framework populated with LLM-based agents.\n                    <jats:sc>MetaAgents<\/jats:sc>\n                    facilitates agent engagement in conversations and a series of decision making within social contexts, serving as an appropriate platform for investigating interactions and interpersonal decision-making of agents. In particular, we construct a job fair environment as a case study to scrutinize the team assembly and skill-matching behaviors of LLM-based agents. We take advantage of both quantitative metrics evaluation and qualitative text analysis to assess their teaming abilities at the job fair. Our evaluation demonstrates that LLM-based agents perform competently in making rational decisions to develop efficient teams. However, we also identify limitations that hinder their effectiveness in more complex team assembly tasks. Our work provides valuable insights into the role and evolution of LLMs in task-oriented social simulations.\n                  <\/jats:p>","DOI":"10.1145\/3711032","type":"journal-article","created":{"date-parts":[[2025,5,3]],"date-time":"2025-05-03T01:35:05Z","timestamp":1746236105000},"page":"1-27","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":22,"title":["<scp>MetaAgents:<\/scp>\n                    Large Language Model Based Agents for Decision-Making on Teaming"],"prefix":"10.1145","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-5678-3185","authenticated-orcid":false,"given":"Yuan","family":"Li","sequence":"first","affiliation":[{"name":"University of Cambridge, Cambridge, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1539-7939","authenticated-orcid":false,"given":"Lichao","family":"Sun","sequence":"additional","affiliation":[{"name":"Lehigh University, Bethlehem, PA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7412-4669","authenticated-orcid":false,"given":"Yixuan","family":"Zhang","sequence":"additional","affiliation":[{"name":"William &amp; Mary, Williamsburg, VA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,5,2]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2313790120"},{"key":"e_1_2_1_2_1","volume-title":"International Conference on Machine Learning. PMLR, 337--371","author":"Aher Gati V","year":"2023","unstructured":"Gati V Aher, Rosa I Arriaga, and Adam Tauman Kalai. 2023. Using large language models to simulate multiple humans and replicate human subject studies. In International Conference on Machine Learning. PMLR, 337--371."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1017\/pan.2023.2"},{"key":"e_1_2_1_4_1","unstructured":"Amanda Askell Yuntao Bai Anna Chen Dawn Drain Deep Ganguli Tom Henighan Andy Jones Nicholas Joseph Ben Mann Nova DasSarma et al. 2021. A general language assistant as a laboratory for alignment. arXiv preprint arXiv:2112.00861 (2021)."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858557"},{"key":"e_1_2_1_6_1","volume-title":"A simulation model for the waterfall software development life cycle. arXiv preprint arXiv:1205.6904","author":"Bassil Youssef","year":"2012","unstructured":"Youssef Bassil. 2012. A simulation model for the waterfall software development life cycle. arXiv preprint arXiv:1205.6904 (2012)."},{"key":"e_1_2_1_7_1","volume-title":"NegotiationArena Platform and Analysis. In Forty-first International Conference on Machine Learning. https:\/\/openreview.net\/forum?id=CmOmaxkt8p","author":"Bianchi Federico","year":"2024","unstructured":"Federico Bianchi, Patrick John Chia, Mert Yuksekgonul, Jacopo Tagliabue, Dan Jurafsky, and James Zou. 2024. How Well Can LLMs Negotiate? NegotiationArena Platform and Analysis. In Forty-first International Conference on Machine Learning. https:\/\/openreview.net\/forum?id=CmOmaxkt8p"},{"key":"e_1_2_1_8_1","unstructured":"Rishi Bommasani Drew A Hudson Ehsan Adeli Russ Altman Simran Arora Sydney von Arx Michael S Bernstein Jeannette Bohg Antoine Bosselut Emma Brunskill et al. 2021. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 (2021)."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.4493398"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1111\/1468--2389.00234"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3613904.3642377"},{"key":"e_1_2_1_12_1","unstructured":"Weize Chen Yusheng Su Jingwei Zuo Cheng Yang Chenfei Yuan Chi-Min Chan Heyang Yu Yaxi Lu Yi-Hsin Hung Chen Qian Yujia Qin Xin Cong Ruobing Xie Zhiyuan Liu Maosong Sun and Jie Zhou. 2024. AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors. (2024). https:\/\/openreview.net\/forum?id=EHg5GDnyq1"},{"key":"e_1_2_1_13_1","volume-title":"Towards ecologically valid research on language user interfaces. arXiv preprint arXiv:2007.14435","author":"Vries Harm De","year":"2020","unstructured":"Harm De Vries, Dzmitry Bahdanau, and Christopher Manning. 2020. Towards ecologically valid research on language user interfaces. arXiv preprint arXiv:2007.14435 (2020)."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.findings-emnlp.88"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/143457.143468"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3670865.3673513"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/289444.289502"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3415252"},{"key":"e_1_2_1_19_1","volume-title":"Skills and skilled work : an economic and social analysis","author":"Green Francis","unstructured":"Francis Green. 2013. Skills and skilled work : an economic and social analysis. Oxford University Press, Oxford."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.1106340"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.soc.29.010202.100030"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359250"},{"key":"e_1_2_1_23_1","volume-title":"MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework. In The Twelfth International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=VtmBAGCN7o","author":"Hong Sirui","year":"2024","unstructured":"Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, ChenglinWu, and J\u00fcrgen Schmidhuber. 2024. MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework. In The Twelfth International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=VtmBAGCN7o"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3613905.3650839"},{"key":"e_1_2_1_25_1","volume-title":"War and peace (waragent): Large language model-based multi-agent simulation of world wars. arXiv preprint arXiv:2311.17227","author":"Hua Wenyue","year":"2023","unstructured":"Wenyue Hua, Lizhou Fan, Lingyao Li, Kai Mei, Jianchao Ji, Yingqiang Ge, Libby Hemphill, and Yongfeng Zhang. 2023. War and peace (waragent): Large language model-based multi-agent simulation of world wars. arXiv preprint arXiv:2311.17227 (2023)."},{"key":"e_1_2_1_26_1","volume-title":"International Conference on Machine Learning. PMLR","author":"Huang Yue","year":"2024","unstructured":"Yue Huang, Lichao Sun, Haoran Wang, Siyuan Wu, Qihui Zhang, Yuan Li, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, et al. 2024. Position: TrustLLM: Trustworthiness in large language models. In International Conference on Machine Learning. PMLR, 20166--20270."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3586182.3615796"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3026011"},{"key":"e_1_2_1_29_1","volume-title":"LEXI: Large Language Models Experimentation Interface. arXiv preprint arXiv:2407.01488","author":"Laban Guy","year":"2024","unstructured":"Guy Laban, Tomer Laban, and Hatice Gunes. 2024. LEXI: Large Language Models Experimentation Interface. arXiv preprint arXiv:2407.01488 (2024)."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-orgpsych-041015-062445"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.acl-long.829"},{"key":"e_1_2_1_32_1","volume-title":"AgentSims: An Open-Source Sandbox for Large Language Model Evaluation. arXiv preprint arXiv:2308.04026","author":"Lin Jiaju","year":"2023","unstructured":"Jiaju Lin, Haoran Zhao, Aochi Zhang, Yiting Wu, Huqiuyue Ping, and Qin Chen. 2023. AgentSims: An Open-Source Sandbox for Large Language Model Evaluation. arXiv preprint arXiv:2308.04026 (2023)."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1207\/S15327957PSPR0401_8"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1111\/joes.12254"},{"key":"e_1_2_1_35_1","unstructured":"Robert Mills. 2016. How to Define a Workflow That Keeps Content Production on Track. https:\/\/contentmarketinginstitute.com\/articles\/define-workflow-content-production\/"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300356"},{"key":"e_1_2_1_37_1","unstructured":"OpenAI. 2023. ChatGPT. https:\/\/openai.com\/product\/chatgpt"},{"key":"e_1_2_1_38_1","unstructured":"OpenAI. 2023. GPT-4 Technical Report. ArXiv abs\/2303.08774 (2023)."},{"key":"e_1_2_1_39_1","volume-title":"What is skills mismatch and why should we care? www.ilo.org (Apr","author":"International Labour Organization","year":"2020","unstructured":"International Labour Organization. 2020. What is skills mismatch and why should we care? www.ilo.org (Apr 2020). https:\/\/www.ilo.org\/skills\/Whatsnew\/WCMS_740388\/lang--en\/index.htm"},{"key":"e_1_2_1_40_1","first-page":"27730","article-title":"Training language models to follow instructions with human feedback","volume":"35","author":"Ouyang Long","year":"2022","unstructured":"Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems 35 (2022), 27730--27744.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3586183.3606763"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3526113.3545616"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.acl-long.810"},{"key":"e_1_2_1_44_1","first-page":"4","article-title":"Teamwork","volume":"2","author":"Salas Eduardo","year":"2000","unstructured":"Eduardo Salas, C. Shawn Burke, and Janis A. Cannon-Bowers. 2000. Teamwork: Emerging Principles. International Journal of Management Reviews 2, 4 (Dec 2000), 339--356.","journal-title":"Emerging Principles. International Journal of Management Reviews"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274420"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-main.248"},{"key":"e_1_2_1_47_1","volume-title":"Talent management systems : best practices in technology solutions for recruitment, retention, and workforce planning","author":"Schweyer Allan","unstructured":"Allan Schweyer. 2004. Talent management systems : best practices in technology solutions for recruitment, retention, and workforce planning. Wiley, Toronto, Ont."},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3613904.3642159"},{"key":"e_1_2_1_49_1","unstructured":"Significant-Gravitas. 2023. AutoGPT. https:\/\/github.com\/Significant-Gravitas\/Auto-GPT."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544549.3573820"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1177\/1088868306294789"},{"key":"e_1_2_1_52_1","volume-title":"Standard operating procedures: A writing guide","author":"Stup Richard","year":"2001","unstructured":"Richard Stup. 2001. Standard operating procedures: A writing guide. State College: Penn State University (2001)."},{"key":"e_1_2_1_53_1","volume-title":"Team assembly. Strategies for team science success: Handbook of evidence-based principles for cross-disciplinary science and practical lessons learned from health researchers","author":"Twyman Marlon","year":"2019","unstructured":"Marlon Twyman and Noshir Contractor. 2019. Team assembly. Strategies for team science success: Handbook of evidence-based principles for cross-disciplinary science and practical lessons learned from health researchers (2019), 217--240."},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1080\/13594320143000573"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359313"},{"key":"e_1_2_1_56_1","doi-asserted-by":"crossref","unstructured":"Lei Wang Chen Ma Xueyang Feng Zeyu Zhang Hao Yang Jingsen Zhang Zhiyuan Chen Jiakai Tang Xu Chen Yankai Lin et al. 2023. A Survey on Large Language Model based Autonomous Agents. arXiv preprint arXiv:2308.11432 (2023).","DOI":"10.1007\/s11704-024-40231-1"},{"key":"e_1_2_1_57_1","volume-title":"Wayne Xin Zhao, and Ji-Rong Wen","author":"Wang Lei","year":"2023","unstructured":"Lei Wang, Jingsen Zhang, Xu Chen, Yankai Lin, Ruihua Song, Wayne Xin Zhao, and Ji-Rong Wen. 2023. RecAgent: A Novel Simulation Paradigm for Recommender Systems. arXiv preprint arXiv:2306.02552 (2023)."},{"key":"e_1_2_1_58_1","volume-title":"Avalon's Game of Thoughts: Battle Against Deception through Recursive Contemplation. arXiv preprint arXiv:2310.01320","author":"Wang Shenzhi","year":"2023","unstructured":"Shenzhi Wang, Chang Liu, Zilong Zheng, Siyuan Qi, Shuo Chen, Qisen Yang, Andrew Zhao, Chaofei Wang, Shiji Song, and Gao Huang. 2023. Avalon's Game of Thoughts: Battle Against Deception through Recursive Contemplation. arXiv preprint arXiv:2310.01320 (2023)."},{"key":"e_1_2_1_59_1","volume-title":"Persuasion for good: Towards a personalized persuasive dialogue system for social good. arXiv preprint arXiv:1906.06725","author":"Wang Xuewei","year":"2019","unstructured":"Xuewei Wang, Weiyan Shi, Richard Kim, Yoojung Oh, Sijia Yang, Jingwen Zhang, and Zhou Yu. 2019. Persuasion for good: Towards a personalized persuasive dialogue system for social good. arXiv preprint arXiv:1906.06725 (2019)."},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1037\/0021--9010.76.6.803"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1177\/0963721415599543"},{"key":"e_1_2_1_62_1","unstructured":"Zhiheng Xi Wenxiang Chen Xin Guo Wei He Yiwen Ding Boyang Hong Ming Zhang JunzheWang Senjie Jin Enyu Zhou et al. 2023. The rise and potential of large language model based agents: A survey. arXiv preprint arXiv:2309.07864 (2023)."},{"key":"e_1_2_1_63_1","volume-title":"Exploring large language models for communication games: An empirical study on werewolf. arXiv preprint arXiv:2309.04658","author":"Xu Yuzhuang","year":"2023","unstructured":"Yuzhuang Xu, Shuo Wang, Peng Li, Fuwen Luo, Xiaolong Wang, Weidong Liu, and Yang Liu. 2023. Exploring large language models for communication games: An empirical study on werewolf. arXiv preprint arXiv:2309.04658 (2023)."},{"key":"e_1_2_1_64_1","volume-title":"Language agents with reinforcement learning for strategic play in the werewolf game. arXiv preprint arXiv:2310.18940","author":"Xu Zelai","year":"2023","unstructured":"Zelai Xu, Chao Yu, Fei Fang, Yu Wang, and Yi Wu. 2023. Language agents with reinforcement learning for strategic play in the werewolf game. arXiv preprint arXiv:2310.18940 (2023)."},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/3392826"},{"key":"e_1_2_1_66_1","volume-title":"CompeteAI: Understanding the Competition Dynamics of Large Language Model-based Agents. In Forty-first International Conference on Machine Learning.","author":"Zhao Qinlin","unstructured":"Qinlin Zhao, Jindong Wang, Yixuan Zhang, Yiqiao Jin, Kaijie Zhu, Hao Chen, and Xing Xie. [n.d.]. CompeteAI: Understanding the Competition Dynamics of Large Language Model-based Agents. In Forty-first International Conference on Machine Learning."},{"key":"e_1_2_1_67_1","volume-title":"Aditya Gupta, Kevin R McKee, Ari Holtzman, Jay Pujara, Xiang Ren, Swaroop Mishra, Aida Nematzadeh, et al.","author":"Zhou Pei","year":"2023","unstructured":"Pei Zhou, Aman Madaan, Srividya Pranavi Potharaju, Aditya Gupta, Kevin R McKee, Ari Holtzman, Jay Pujara, Xiang Ren, Swaroop Mishra, Aida Nematzadeh, et al. 2023. How FaR Are Large Language Models From Agents with Theory-of-Mind? arXiv preprint arXiv:2310.03051 (2023)."},{"key":"e_1_2_1_68_1","volume-title":"SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents. In The Twelfth International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=mM7VurbA4r","author":"Zhou Xuhui","year":"2024","unstructured":"Xuhui Zhou, Hao Zhu, Leena Mathur, Ruohong Zhang, Haofei Yu, Zhengyang Qi, Louis-Philippe Morency, Yonatan Bisk, Daniel Fried, Graham Neubig, and Maarten Sap. 2024. SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents. In The Twelfth International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=mM7VurbA4r"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1162\/coli_a_00502"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1145\/3613904.3642450"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3711032","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3711032","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T00:52:34Z","timestamp":1755737554000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3711032"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,2]]},"references-count":70,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,5,2]]}},"alternative-id":["10.1145\/3711032"],"URL":"https:\/\/doi.org\/10.1145\/3711032","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,2]]},"assertion":[{"value":"2025-05-02","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}