{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T12:32:35Z","timestamp":1776083555545,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":45,"publisher":"ACM","funder":[{"name":"ZERTIFIZIERTE KI"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2025,6,23]]},"DOI":"10.1145\/3715275.3732181","type":"proceedings-article","created":{"date-parts":[[2025,6,23]],"date-time":"2025-06-23T17:03:13Z","timestamp":1750698193000},"page":"2796-2814","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Detecting Linguistic Indicators for Stereotype Assessment with Large Language Models"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-3793-1374","authenticated-orcid":false,"given":"Rebekka","family":"G\u00f6rge","sequence":"first","affiliation":[{"name":"Fraunhofer Institute for Intelligent Analysis and Information Systems, Sankt Augustin, Germany and Lamarr-Institute for Machine Learning and Artificial Intelligence, Bonn, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7352-9165","authenticated-orcid":false,"given":"Michael","family":"Mock","sequence":"additional","affiliation":[{"name":"Fraunhofer Institute for Intelligent Analysis and Information Systems, St. Augustin, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3047-8817","authenticated-orcid":false,"given":"H\u00e9ctor","family":"Allende-Cid","sequence":"additional","affiliation":[{"name":"Fraunhofer Institute for Intelligent Analysis and Information Systems, St. Augustin, Germany and Lamarr-Institute for Machine Learning and Artificial Intelligence, Bonn, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,6,23]]},"reference":[{"key":"e_1_3_3_2_2_2","unstructured":"Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad Ilge Akkaya Florencia\u00a0Leoni Aleman Diogo Almeida Janko Altenschmidt Sam Altman Shyamal Anadkat et\u00a0al. 2023. Gpt-4 technical report. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2303.08774 (2023)."},{"key":"e_1_3_3_2_3_2","doi-asserted-by":"publisher","unstructured":"X. Bai A. Wang I. Sucholutsky and T.\u00a0L. Griffiths. 2025. Explicitly unbiased large language models still form biased associations. Proceedings of the National Academy of Sciences of the United States of America 122 8 (2025) e2416228122. https:\/\/doi.org\/10.1073\/pnas.2416228122","DOI":"10.1073\/pnas.2416228122"},{"key":"e_1_3_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.acl-long.151"},{"key":"e_1_3_3_2_5_2","doi-asserted-by":"crossref","unstructured":"Camiel\u00a0J Beukeboom and Christian Burgers. 2019. How stereotypes are shared through language: a review and introduction of the aocial categories and stereotypes communication (SCSC) framework. Review of Communication Research 7 (2019) 1\u201337.","DOI":"10.12840\/issn.2255-4165.017"},{"key":"e_1_3_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.485"},{"key":"e_1_3_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.acl-long.81"},{"key":"e_1_3_3_2_8_2","unstructured":"Tolga Bolukbasi Kai-Wei Chang James\u00a0Y Zou Venkatesh Saligrama and Adam\u00a0T Kalai. [n. d.]. Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings. ([n. d.])."},{"key":"e_1_3_3_2_9_2","unstructured":"Tom Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared\u00a0D Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell et\u00a0al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020) 1877\u20131901."},{"key":"e_1_3_3_2_10_2","doi-asserted-by":"crossref","unstructured":"Aylin Caliskan Joanna\u00a0J Bryson and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science 356 6334 (2017) 183\u2013186.","DOI":"10.1126\/science.aal4230"},{"key":"e_1_3_3_2_11_2","doi-asserted-by":"crossref","unstructured":"Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and psychological measurement 20 1 (1960) 37\u201346.","DOI":"10.1177\/001316446002000104"},{"key":"e_1_3_3_2_12_2","unstructured":"Omkar Dige Jacob-Junqi Tian David Emerson and Faiza\u00a0Khan Khattak. 2023. Can Instruction Fine-Tuned Language Models Identify Social Bias through Prompting?http:\/\/arxiv.org\/abs\/2307.10472 arXiv:https:\/\/arXiv.org\/abs\/2307.10472 [cs]."},{"key":"e_1_3_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.emnlp-main.64"},{"key":"e_1_3_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.4135\/9781446200919"},{"key":"e_1_3_3_2_15_2","unstructured":"Aaron\u00a0Grattafiori et al.2024. The Llama 3 Herd of Models. arxiv:https:\/\/arXiv.org\/abs\/2407.21783\u00a0[cs.AI] https:\/\/arxiv.org\/abs\/2407.21783"},{"key":"e_1_3_3_2_16_2","doi-asserted-by":"crossref","unstructured":"Susan\u00a0T Fiske Amy\u00a0JC Cuddy and Peter Glick. 2007. Universal dimensions of social cognition: Warmth and competence. Trends in cognitive sciences 11 2 (2007) 77\u201383.","DOI":"10.1016\/j.tics.2006.11.005"},{"key":"e_1_3_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.acl-long.343"},{"key":"e_1_3_3_2_18_2","first-page":"17764","volume-title":"Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)","author":"Fort Karen","year":"2024","unstructured":"Karen Fort, Laura Alonso\u00a0Alemany, Luciana Benotti, Julien Bezan\u00e7on, Claudia Borg, Marthese Borg, Yongjian Chen, Fanny Ducel, Yoann Dupont, Guido Ivetta, Zhijian Li, Margot Mieskes, Marco Naguib, Yuyan Qian, Matteo Radaelli, Wolfgang\u00a0S. Schmeisser-Nieto, Emma Raimundo\u00a0Schulz, Thiziri Saci, Sarah Saidi, Javier Torroba\u00a0Marchante, Shilin Xie, Sergio\u00a0E. Zanotto, and Aur\u00e9lie N\u00e9v\u00e9ol. 2024. Your Stereotypical Mileage May Vary: Practical Challenges of Evaluating Biases in Multiple Languages and Cultural Contexts. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, and Nianwen Xue (Eds.). ELRA and ICCL, Torino, Italia, 17764\u201317769. https:\/\/aclanthology.org\/2024.lrec-main.1545\/"},{"key":"e_1_3_3_2_19_2","doi-asserted-by":"publisher","unstructured":"Kathleen\u00a0C. Fraser Svetlana Kiritchenko and Isar Nejadgholi. 2022. Computational Modeling of Stereotype Content in Text. Frontiers in Artificial Intelligence 5 (April 2022) 826207. https:\/\/doi.org\/10.3389\/frai.2022.826207","DOI":"10.3389\/frai.2022.826207"},{"key":"e_1_3_3_2_20_2","doi-asserted-by":"publisher","unstructured":"Isabel\u00a0O. Gallegos Ryan\u00a0A. Rossi Joe Barrow Md\u00a0Mehrab Tanjim Sungchul Kim Franck Dernoncourt Tong Yu Ruiyi Zhang and Nesreen\u00a0K. Ahmed. 2024. Bias and Fairness in Large Language Models: A Survey. Computational Linguistics 50 3 (Sept. 2024) 1097\u20131179. https:\/\/doi.org\/10.1162\/colia00524","DOI":"10.1162\/colia00524"},{"key":"e_1_3_3_2_21_2","unstructured":"Jiawei Gu Xuhui Jiang Zhichao Shi Hexiang Tan Xuehao Zhai Chengjin Xu Wei Li Yinghan Shen Shengjie Ma Honghao Liu Yuanzhuo Wang and Jian Guo. 2025. A Survey on LLM-as-a-Judge. arxiv:https:\/\/arXiv.org\/abs\/2411.15594\u00a0[cs.CL] https:\/\/arxiv.org\/abs\/2411.15594"},{"key":"e_1_3_3_2_22_2","doi-asserted-by":"publisher","unstructured":"Dirk Hovy and Shrimai Prabhumoye. 2021. Five sources of bias in natural language processing. Language and Linguistics Compass 15 8 (2021) e12432. https:\/\/doi.org\/10.1111\/lnc3.12432 arXiv:https:\/\/compass.onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/lnc3.12432","DOI":"10.1111\/lnc3.12432"},{"key":"e_1_3_3_2_23_2","unstructured":"Yue Huang Lichao Sun Haoran Wang Siyuan Wu Qihui Zhang Yuan Li Chujie Gao Yixin Huang Wenhan Lyu Yixuan Zhang et\u00a0al. 2024. Trustllm: Trustworthiness in large language models. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2401.05561 (2024)."},{"key":"e_1_3_3_2_24_2","unstructured":"ISO\/IEC. 2021. ISO\/IEC TR 24027:2021- Information technology \u2014 Artificial intelligence (AI) \u2014 Bias in AI systems and AI aided decision making."},{"key":"e_1_3_3_2_25_2","unstructured":"Albert\u00a0Q Jiang Alexandre Sablayrolles Antoine Roux Arthur Mensch Blanche Savary Chris Bamford Devendra\u00a0Singh Chaplot Diego de\u00a0las Casas Emma\u00a0Bou Hanna Florian Bressand et\u00a0al. 2024. Mixtral of experts. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2401.04088 (2024)."},{"key":"e_1_3_3_2_26_2","volume-title":"Neurips Safe Generative AI Workshop 2024","author":"King Theo","year":"2024","unstructured":"Theo King, Zekun Wu, Adriano Koshiyama, Emre Kazim, and Philip\u00a0Colin Treleaven. 2024. HEARTS: A Holistic Framework for Explainable, Sustainable and Robust Text Stereotype Detection. In Neurips Safe Generative AI Workshop 2024. https:\/\/openreview.net\/forum?id=arh91riKiQ"},{"key":"e_1_3_3_2_27_2","first-page":"1223","volume-title":"Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Liu Yang","year":"2024","unstructured":"Yang Liu. 2024. Quantifying Stereotypes in Language. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), Yvette Graham and Matthew Purver (Eds.). Association for Computational Linguistics, St. Julian\u2019s, Malta, 1223\u20131240. https:\/\/aclanthology.org\/2024.eacl-long.74"},{"key":"e_1_3_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9781107337855"},{"key":"e_1_3_3_2_29_2","volume-title":"The AI Index 2024 Annual Report","author":"Maslej Nestor","year":"2024","unstructured":"Nestor Maslej, Loredana Fattorini, Raymond Perrault, Vanessa Parli, Anka Reuel, Erik Brynjolfsson, John Etchemendy, Katrina Ligett, Terah Lyons, James Manyika, Juan\u00a0Carlos Niebles, Yoav Shoham, Russell Wald, and Jack Clark. 2024. The AI Index 2024 Annual Report. Technical Report. AI Index Steering Committee, Institute for Human-Centered AI, Stanford University, Stanford, CA."},{"key":"e_1_3_3_2_30_2","unstructured":"Lucas Maystre and Matthias Grossglauser. 2015. Fast and accurate inference of Plackett\u2013Luce models. Advances in neural information processing systems 28 (2015)."},{"key":"e_1_3_3_2_31_2","unstructured":"Michael Mock Sebastian Schmidt Felix M\u00fcller Rebekka G\u00f6rge Anna Schmitz Elena Haedecke Angelika Voss Dirk Hecker and Maximillian Poretschkin. 2024. Developing trustworthy AI applications with foundation models. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2405.04937 (2024)."},{"key":"e_1_3_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.acl-long.416"},{"key":"e_1_3_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.154"},{"key":"e_1_3_3_2_34_2","doi-asserted-by":"publisher","unstructured":"Roberto Navigli Simone Conia and Bj\u00f6rn Ross. 2023. Biases in Large Language Models: Origins Inventory and Discussion. Journal of Data and Information Quality 15 2 (June 2023) 1\u201321. https:\/\/doi.org\/10.1145\/3597307","DOI":"10.1145\/3597307"},{"key":"e_1_3_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.acl-long.583"},{"key":"e_1_3_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.acl-long.462"},{"key":"e_1_3_3_2_37_2","doi-asserted-by":"publisher","unstructured":"Ahmed\u00a0M. Salih Zahra Raisi\u2010Estabragh Ilaria\u00a0Boscolo Galazzo Petia Radeva Steffen\u00a0E. Petersen Karim Lekadir and Gloria Menegaz. 2024. A Perspective on Explainable Artificial Intelligence Methods: SHAP and LIME. Advanced Intelligent Systems 7 1 (June 2024). https:\/\/doi.org\/10.1002\/aisy.202400304","DOI":"10.1002\/aisy.202400304"},{"key":"e_1_3_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.486"},{"key":"e_1_3_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/3600211.3604673"},{"key":"e_1_3_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.acl-long.304"},{"key":"e_1_3_3_2_41_2","first-page":"410","volume-title":"Proceedings of the 7th International Conference on Natural Language and Speech Processing (ICNLSP 2024)","author":"Soundararajan Shweta","year":"2024","unstructured":"Shweta Soundararajan and Sarah\u00a0Jane Delany. 2024. Investigating Gender Bias in Large Language Models Through Text Generation. In Proceedings of the 7th International Conference on Natural Language and Speech Processing (ICNLSP 2024), Mourad Abbas and Abed\u00a0Alhakim Freihat (Eds.). Association for Computational Linguistics, Trento, 410\u2013424. https:\/\/aclanthology.org\/2024.icnlsp-1.42\/"},{"key":"e_1_3_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1016\/B978-0-08-088579-7.50010-7"},{"key":"e_1_3_3_2_43_2","volume-title":"Socially Responsible Language Modelling Research","author":"Tian Jacob-Junqi","year":"2023","unstructured":"Jacob-Junqi Tian, Omkar Dige, D. Emerson, and Faiza Khattak. 2023. Using Chain-of-Thought Prompting for Interpretable Recognition of Social Bias. In Socially Responsible Language Modelling Research. https:\/\/openreview.net\/forum?id=QyRganPqPz"},{"key":"e_1_3_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.acl-long.868"},{"key":"e_1_3_3_2_45_2","volume-title":"NeurIPS","author":"Wang Boxin","year":"2023","unstructured":"Boxin Wang, Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, et\u00a0al. 2023. DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models.. In NeurIPS."},{"key":"e_1_3_3_2_46_2","unstructured":"Zekun Wu Sahan Bulathwela Maria Perez-Ortiz and Adriano\u00a0Soares Koshiyama. 2024. Auditing Large Language Models for Enhanced Text-Based Stereotype Detection and Probing-Based Bias Evaluation. http:\/\/arxiv.org\/abs\/2404.01768 arXiv:https:\/\/arXiv.org\/abs\/2404.01768 [cs]."}],"event":{"name":"FAccT '25: The 2025 ACM Conference on Fairness, Accountability, and Transparency","location":"Athens Greece","acronym":"FAccT '25"},"container-title":["Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3715275.3732181","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,24]],"date-time":"2025-06-24T11:16:31Z","timestamp":1750763791000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3715275.3732181"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,23]]},"references-count":45,"alternative-id":["10.1145\/3715275.3732181","10.1145\/3715275"],"URL":"https:\/\/doi.org\/10.1145\/3715275.3732181","relation":{},"subject":[],"published":{"date-parts":[[2025,6,23]]},"assertion":[{"value":"2025-06-23","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}