{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T20:54:42Z","timestamp":1791492882169,"version":"4.3.4"},"reference-count":74,"publisher":"Association for Computing Machinery (ACM)","issue":"FSE","license":[{"start":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T00:00:00Z","timestamp":1782777600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62325201"],"award-info":[{"award-number":["62325201"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Softw. Eng."],"published-print":{"date-parts":[[2026,6,30]]},"abstract":"<jats:p>Large Language Models (LLMs) have become foundational in modern language-driven software applications, profoundly influencing daily life. A critical technique in leveraging their potential is role-playing, where LLMs simulate diverse roles to enhance their real-world utility. However, while research has highlighted the presence of social biases in LLM outputs, it remains unclear whether and to what extent these biases emerge during role-playing scenarios. In this paper, we conduct an empirical study on fairness testing of LLMs in role-playing scenarios. To enable this testing, we use LLMs to generate 550 social roles spanning a comprehensive set of 11 demographic attributes, producing 33,000 role-specific questions that target various forms of bias. These questions, covering Yes\/No, multiple-choice, and open-ended formats, are designed to prompt LLMs to adopt specific roles and respond accordingly. We employ a combination of rule-based and LLM-based strategies to identify biased responses, rigorously validated through human evaluation. Using the generated questions as the test cases, we conduct extensive evaluations of 10 advanced LLMs. The evaluation reveal 107,580 biased responses across the studied LLMs, with individual models yielding between 7,579 and 16,963 biased responses, underscoring the prevalence of bias in role-playing contexts. To support future research, we have publicly released the dataset, along with all scripts and experimental results.<\/jats:p>","DOI":"10.1145\/3808106","type":"journal-article","created":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T17:06:14Z","timestamp":1782839174000},"page":"2211-2234","source":"Crossref","is-referenced-by-count":1,"title":["Fairness Testing of Large Language Models in Role-Playing"],"prefix":"10.1145","volume":"3","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-4058-6733","authenticated-orcid":false,"given":"Xinyue","family":"Li","sequence":"first","affiliation":[{"name":"Peking University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4765-1893","authenticated-orcid":false,"given":"Zhenpeng","family":"Chen","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0481-7264","authenticated-orcid":false,"given":"Jie M.","family":"Zhang","sequence":"additional","affiliation":[{"name":"King's College London, London, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8624-5740","authenticated-orcid":false,"given":"Ying","family":"Xiao","sequence":"additional","affiliation":[{"name":"King's College London, London, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2207-1622","authenticated-orcid":false,"given":"Tianlin","family":"Li","sequence":"additional","affiliation":[{"name":"Nanyang Technological University, Singapore, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9236-8264","authenticated-orcid":false,"given":"Weisong","family":"Sun","sequence":"additional","affiliation":[{"name":"Nanyang Technological University, Singapore, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7300-9215","authenticated-orcid":false,"given":"Yang","family":"Liu","sequence":"additional","affiliation":[{"name":"Nanyang Technological University, Singapore, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4066-3365","authenticated-orcid":false,"given":"Yiling","family":"Lou","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Champaign, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7908-8484","authenticated-orcid":false,"given":"Xuanzhe","family":"Liu","sequence":"additional","affiliation":[{"name":"Peking University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,6,30]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"2026. Chatbot Arena LLM Leaderboard. https:\/\/lmarena.ai\/leaderboard\/text."},{"key":"e_1_2_1_2_1","unstructured":"2026. Datamuse. https:\/\/www.datamuse.com\/api\/."},{"key":"e_1_2_1_3_1","unstructured":"2026. DeepSeek-V2.5. https:\/\/huggingface.co\/deepseek-ai\/DeepSeek-V2.5."},{"key":"e_1_2_1_4_1","unstructured":"2026. Gemeni-2.5-Flash. https:\/\/deepmind.google\/models\/gemini\/flash\/."},{"key":"e_1_2_1_5_1","unstructured":"2026. Giving Claude a role. https:\/\/docs.anthropic.com\/en\/docs\/build-with-claude\/prompt-engineering\/systemprompts."},{"key":"e_1_2_1_6_1","unstructured":"2026. GPT-4o. https:\/\/platform.openai.com\/docs\/models\/gpt-4o."},{"key":"e_1_2_1_7_1","unstructured":"2026. GPT-4o mini. https:\/\/platform.openai.com\/docs\/models\/gpt-4o-mini."},{"key":"e_1_2_1_8_1","unstructured":"2026. GPT-5-mini. https:\/\/platform.openai.com\/docs\/models\/gpt-5-mini."},{"key":"e_1_2_1_9_1","unstructured":"2026. Meta-Llama-3-70B. https:\/\/huggingface.co\/meta-llama\/Meta-Llama-3-70B."},{"key":"e_1_2_1_10_1","unstructured":"2026. Meta-Llama-3-8B. https:\/\/huggingface.co\/meta-llama\/Meta-Llama-3-8B."},{"key":"e_1_2_1_11_1","unstructured":"2026. Mistral-7B-Instruct-v0.3. https:\/\/huggingface.co\/mistralai\/Mistral-7B-Instruct-v0.3."},{"key":"e_1_2_1_12_1","unstructured":"2026. Prompt engineering -OpenAI. https:\/\/developers.openai.com\/api\/docs\/guides\/prompt-guidance."},{"key":"e_1_2_1_13_1","unstructured":"2026. Prompting -Meta. https:\/\/www.llama.com\/docs\/how-to-guides\/prompting."},{"key":"e_1_2_1_14_1","unstructured":"2026. Prompting capabilities -Mistral AI. https:\/\/docs.mistral.ai\/guides\/prompting_capabilities\/."},{"key":"e_1_2_1_15_1","unstructured":"2026. Qwen1.5-110B-Chat. https:\/\/huggingface.co\/Qwen\/Qwen1.5-110B-Chat."},{"key":"e_1_2_1_16_1","unstructured":"2026. Qwen3-235B-Chat. https:\/\/huggingface.co\/Qwen\/Qwen3-235B-A22B-Instruct-2507."},{"key":"e_1_2_1_17_1","unstructured":"2026. Role-specific prompts & use cases. https:\/\/support.google.com\/a\/users\/answer\/14667148?visit_id= 638649091395709697-2537054327&hl=en&rd=1."},{"key":"e_1_2_1_18_1","unstructured":"2026. Types of Questions in English. https:\/\/preply.com\/en\/blog\/types-of-questions-in-english\/."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2021.3136169"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2014.2372785"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409704"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/COG60054.2024.10645667"},{"key":"e_1_2_1_23_1","unstructured":"G. Casella and R.L. Berger. 2007. Statistical Inference (2 ed.). Duxbury Press."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409759"},{"key":"e_1_2_1_25_1","volume-title":"Proceedings of the 47th IEEE\/ACM International Conference on Software Engineering, ICSE 2025. 743-755","author":"Chen Zhenpeng","year":"2025","unstructured":"Zhenpeng Chen, Xinyue Li, Jie M. Zhang, Federica Sarro, and Yang Liu. 2025. Diversity Drives Fairness: Ensemble of Higher Order Mutants for Intersectional Fairness of Machine Learning Software. In Proceedings of the 47th IEEE\/ACM International Conference on Software Engineering, ICSE 2025. 743-755."},{"key":"e_1_2_1_26_1","first-page":"1780","volume-title":"Proc. ACM Softw. Eng. 2, FSE","author":"Chen Zhenpeng","year":"2025","unstructured":"Zhenpeng Chen, Xinyue Li, Jie M. Zhang, Weisong Sun, Ying Xiao, Tianlin Li, Yiling Lou, and Yang Liu. 2025. Software Fairness Dilemma: Is Bias Mitigation a Zero-Sum Game? Proc. ACM Softw. Eng. 2, FSE (2025), 1780-1801."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3652155"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549093"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3583561"},{"key":"e_1_2_1_30_1","volume-title":"Proceedings of the Forty-first International Conference on Machine Learning, ICML","author":"Chiang Wei-Lin","year":"2024","unstructured":"Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang, Michael I. Jordan, Joseph E. Gonzalez, and Ion Stoica. 2024. Chatbot arena: An open platform for evaluating LLMs by human preference. In Proceedings of the Forty-first International Conference on Machine Learning, ICML 2024."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3682112.3682117"},{"key":"e_1_2_1_32_1","unstructured":"Jiawei Gu Xuhui Jiang Zhichao Shi Hexiang Tan Xuehao Zhai Chengjin Xu Wei Li Yinghan Shen Shengjie Ma Honghao Liu et al. 2024. A survey on llm-as-a-judge. arXiv preprint arXiv:2411.15594 (2024)."},{"key":"e_1_2_1_33_1","volume-title":"Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 1068-1080","author":"Guo Yaoqi","year":"2025","unstructured":"Yaoqi Guo, Zhenpeng Chen, Jie M Zhang, Yang Liu, and Yun Ma. 2025. Personality-guided code generation using large language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 1068-1080."},{"key":"e_1_2_1_34_1","volume-title":"What's in a name? Auditing large language models for race and gender bias. arXiv preprint arXiv:2402.14875","author":"Haim Amit","year":"2024","unstructured":"Amit Haim, Alejandro Salinas, and Julian Nyarko. 2024. What's in a name? Auditing large language models for race and gender bias. arXiv preprint arXiv:2402.14875 (2024)."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1007\/S10639-023-11834-1"},{"key":"e_1_2_1_36_1","volume-title":"The effect of personas on cultural norm interpretation in LLMs. CoRR abs\/2409.11636","author":"Kamruzzaman Mahammed","year":"2024","unstructured":"Mahammed Kamruzzaman, Hieu Nguyen, Nazmul Hassan, and Gene Louis Kim. 2024. \"A woman is more culturally knowledgeable than a man?\": The effect of personas on cultural norm interpretation in LLMs. CoRR abs\/2409.11636 (2024)."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3582269.3615599"},{"key":"e_1_2_1_38_1","volume-title":"Koch","author":"Richard Landis J","year":"1977","unstructured":"J Richard Landis and Gary G. Koch. 1977. The measurement of observer agreement for categorical data. Biometrics 33 1 (1977), 159-74."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/"},{"key":"e_1_2_1_40_1","volume-title":"LLM discussion: Enhancing the creativity of large language models via discussion framework and role-play. CoRR abs\/2405.06373","author":"Lu Li-Chun","year":"2024","unstructured":"Li-Chun Lu, Shou-Jen Chen, Tsung-Min Pai, Chan-Hung Yu, Hung-yi Lee, and Shao-Hua Sun. 2024. LLM discussion: Enhancing the creativity of large language models via discussion framework and role-play. CoRR abs\/2405.06373 (2024)."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3585006"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00136"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/2021.ACL-"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/2950290.2950354"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3697010"},{"key":"e_1_2_1_46_1","volume-title":"Exploring the role of ChatGPT in the law enforcement and banking sectors. Artificial Intelligence for Risk Mitigation in the Financial Industry","author":"Pandey Shubham","year":"2024","unstructured":"Shubham Pandey, Archana Patel, and Purvi Pokhariyal. 2024. Exploring the role of ChatGPT in the law enforcement and banking sectors. Artificial Intelligence for Risk Mitigation in the Financial Industry (2024), 327-347."},{"key":"e_1_2_1_47_1","volume-title":"Addressing the notion of trust around ChatGPT in the high-stakes use case of insurance. Technology in Society","author":"Ressel Juliane","year":"2024","unstructured":"Juliane Ressel, Michaele V\u00f6ller, Finbarr Murphy, and Martin Mullins. 2024. Addressing the notion of trust around ChatGPT in the high-stakes use case of insurance. Technology in Society (2024), 102644."},{"key":"e_1_2_1_48_1","volume-title":"In-context impersonation reveals large language models' strengths and biases. Advances in neural information processing systems 36","author":"Salewski Leonard","year":"2023","unstructured":"Leonard Salewski, Stephan Alaniz, Isabel Rio-Torto, Eric Schulz, and Zeynep Akata. 2023. In-context impersonation reveals large language models' strengths and biases. Advances in neural information processing systems 36 (2023), 72044-72057."},{"key":"e_1_2_1_49_1","volume-title":"Discovering bias in the internal knowledge of large language models. CoRR abs\/2310.08780","author":"Salinas Abel","year":"2023","unstructured":"Abel Salinas, Louis Penafiel, Robert McCormack, and Fred Morstatter. 2023. \"Im not racist but... \": Discovering bias in the internal knowledge of large language models. CoRR abs\/2310.08780 (2023)."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/2020.ACL-MAIN.486"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/2024.CLINICALNLP-1.19"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1038\/S41586-023-06647-8"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/2022.EMNLP-MAIN.625"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2022.3141758"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/3664608"},{"key":"e_1_2_1_56_1","volume-title":"Cultural bias and cultural alignment of large language models. PNAS nexus 3, 9","author":"Tao Yan","year":"2024","unstructured":"Yan Tao, Olga Viberg, Ryan S Baker, and Ren\u00e9 F Kizilcec. 2024. Cultural bias and cultural alignment of large language models. PNAS nexus 3, 9 (2024)."},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510202"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/2024.FINDINGS-EMNLP.969"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1007\/S10664-"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/2023.FINDINGS-EMNLP.243"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3616310"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSR59073.2023.00019"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2024.3368208"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/2024.ACL-"},{"key":"e_1_2_1_65_1","volume-title":"Social bias, discrimination and inequity in healthcare: Mechanisms, implications and recommendations. BJA education 22, 4","author":"Webster Craig S","year":"2022","unstructured":"Craig S Webster, Saana Taylor, Courtney Thomas, and Jennifer M Weller. 2022. Social bias, discrimination and inequity in healthcare: Mechanisms, implications and recommendations. BJA education 22, 4 (2022), 131-137."},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3468558"},{"key":"e_1_2_1_67_1","volume-title":"Richard Dobson, and Jie M Zhang.","author":"Xiao Ying","year":"2025","unstructured":"Ying Xiao, Zhenpeng Chen, Jen-tse Huang, Wenting Chen, Yepang Liu, Kezhi Li, Mohammad Reza Mousavi, Richard Dobson, and Jie M Zhang. 2025. Bias in Large AI Models for Medicine and Healthcare: Survey and Challenges. (2025)."},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2962027"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2019.00020"},{"key":"e_1_2_1_70_1","volume-title":"Revolutionizing finance with LLMs: An overview of applications and insights. CoRR abs\/2401.11641","author":"Zhao Huaqin","year":"2024","unstructured":"Huaqin Zhao, Zhengliang Liu, Zihao Wu, Yiwei Li, Tianze Yang, Peng Shu, Shaochen Xu, Haixing Dai, Lin Zhao, Gengchen Mai, Ninghao Liu, and Tianming Liu. 2024. Revolutionizing finance with LLMs: An overview of applications and insights. CoRR abs\/2401.11641 (2024)."},{"key":"e_1_2_1_71_1","volume-title":"Bias and Toxicity in Role-Play Reasoning. CoRR abs\/2409.13979","author":"Zhao Jinman","year":"2024","unstructured":"Jinman Zhao, Zifan Qian, Linbo Cao, Yining Wang, and Yitian Ding. 2024. Bias and Toxicity in Role-Play Reasoning. CoRR abs\/2409.13979 (2024)."},{"key":"e_1_2_1_72_1","volume-title":"A survey of large language models. CoRR abs\/2303.18223","author":"Zhao Wayne Xin","year":"2023","unstructured":"Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. 2023. A survey of large language models. CoRR abs\/2303.18223 (2023)."},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1145\/3510003"},{"key":"e_1_2_1_74_1","volume-title":"Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica.","author":"Zheng Lianmin","year":"2023","unstructured":"Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zhuohan Li Zi Lin, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging LLM-as-a-judge with MT-bench and chatbot arena. Advances in neural information processing systems 36 (2023), 46595-46623."}],"container-title":["Proceedings of the ACM on Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3808106","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T18:35:59Z","timestamp":1782844559000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3808106"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,30]]},"references-count":74,"journal-issue":{"issue":"FSE","published-print":{"date-parts":[[2026,6,30]]}},"alternative-id":["10.1145\/3808106"],"URL":"https:\/\/doi.org\/10.1145\/3808106","relation":{},"ISSN":["2994-970X"],"issn-type":[{"value":"2994-970X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,30]]}}}