{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,4]],"date-time":"2026-08-04T00:22:15Z","timestamp":1785802935574,"version":"3.56.0"},"publisher-location":"New York, NY, USA","reference-count":71,"publisher":"ACM","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2025,6,23]]},"DOI":"10.1145\/3715275.3732044","type":"proceedings-article","created":{"date-parts":[[2025,6,23]],"date-time":"2025-06-23T17:03:13Z","timestamp":1750698193000},"page":"690-709","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["Normative Evaluation of Large Language Models with Everyday Moral Dilemmas"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6809-2437","authenticated-orcid":false,"given":"Pratik","family":"Sachdeva","sequence":"first","affiliation":[{"name":"D-Lab, University of California, Berkeley, Berkeley, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8357-2316","authenticated-orcid":false,"given":"Tom","family":"van Nuenen","sequence":"additional","affiliation":[{"name":"D-Lab, University of California, Berkeley, Berkeley, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,6,23]]},"reference":[{"key":"e_1_3_3_3_2_2","unstructured":"[n. d.]. The Claude 3 Model Family: Opus Sonnet Haiku. https:\/\/api.semanticscholar.org\/CorpusID:268232499"},{"key":"e_1_3_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.emnlp-main.982"},{"key":"e_1_3_3_3_4_2","unstructured":"Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad Ilge Akkaya Florencia\u00a0Leoni Aleman Diogo Almeida Janko Altenschmidt Sam Altman Shyamal Anadkat et\u00a0al. 2023. Gpt-4 technical report. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2303.08774 (2023)."},{"key":"e_1_3_3_3_5_2","doi-asserted-by":"publisher","unstructured":"Muhammad\u00a0Farid Adilazuarda Sagnik Mukherjee Pradhyumna Lavania Siddhant Singh Alham\u00a0Fikri Aji Jacki O\u2019Neill Ashutosh Modi and Monojit Choudhury. 2024. Towards Measuring and Modeling \"Culture\" in LLMs: A Survey. https:\/\/doi.org\/10.48550\/arXiv.2403.15412 arXiv:https:\/\/arXiv.org\/abs\/2403.15412 [cs].","DOI":"10.48550\/arXiv.2403.15412"},{"key":"e_1_3_3_3_6_2","unstructured":"Rohan Anil Andrew\u00a0M. Dai Orhan Firat Melvin Johnson Dmitry Lepikhin Alexandre Passos Siamak Shakeri Emanuel Taropa Paige Bailey Z. Chen Eric Chu J. Clark Laurent\u00a0El Shafey Yanping Huang Kathleen\u00a0S. Meier-Hellstern Gaurav Mishra Erica Moreira Mark Omernick Kevin Robinson Sebastian Ruder Yi Tay Kefan Xiao Yuanzhong Xu Yujing Zhang Gustavo\u00a0Hern\u00e1ndez Abrego Junwhan Ahn Jacob Austin Paul Barham Jan\u00a0A. Botha James Bradbury Siddhartha Brahma Kevin\u00a0Michael Brooks Michele Catasta Yongzhou Cheng Colin Cherry Christopher\u00a0A. Choquette-Choo Aakanksha Chowdhery C Cr\u00e9py Shachi Dave Mostafa Dehghani Sunipa Dev Jacob Devlin M.\u00a0C. D\u2019iaz Nan Du Ethan Dyer Vladimir Feinberg Fan Feng Vlad Fienber Markus Freitag Xavier Garc\u00eda Sebastian Gehrmann Lucas Gonz\u00e1lez Guy Gur-Ari Steven Hand Hadi Hashemi Le Hou Joshua Howland An\u00a0Ren Hu Jeffrey Hui Jeremy Hurwitz Michael Isard Abe Ittycheriah Matthew Jagielski Wen\u00a0Hao Jia Kathleen Kenealy Maxim Krikun Sneha Kudugunta Chang Lan Katherine Lee Benjamin Lee Eric Li Mu-Li Li Wei Li Yaguang Li Jun\u00a0Yu Li Hyeontaek Lim Han Lin Zhong-Zhong Liu Frederick Liu Marcello Maggioni Aroma Mahendru Joshua Maynez Vedant Misra Maysam Moussalem Zachary Nado John Nham Eric Ni Andrew Nystrom Alicia Parrish Marie Pellat Martin Polacek Oleksandr Polozov Reiner Pope Siyuan Qiao Emily Reif Bryan Richter Parker Riley Alexandra Ros Aurko Roy Brennan Saeta Rajkumar Samuel Renee\u00a0Marie Shelby Ambrose Slone Daniel Smilkov David\u00a0R. So Daniela Sohn Simon Tokumine Dasha Valter Vijay Vasudevan Kiran Vodrahalli Xuezhi Wang Pidong Wang Zirui Wang Tao Wang John Wieting Yuhuai Wu Ke Xu Yunhan Xu Lin\u00a0Wu Xue Pengcheng Yin Jiahui Yu Qiaoling Zhang Steven Zheng Ce Zheng Wei Zhou Denny Zhou Slav Petrov and Yonghui Wu. 2023. PaLM 2 Technical Report. ArXiv abs\/2305.10403 (2023). https:\/\/api.semanticscholar.org\/CorpusID:258740735"},{"key":"e_1_3_3_3_7_2","unstructured":"Anthropic. [n. d.]. Introducing the next generation of Claude. https:\/\/www.anthropic.com\/news\/claude-3-family."},{"key":"e_1_3_3_3_8_2","doi-asserted-by":"publisher","unstructured":"Yuntao Bai Saurav Kadavath Sandipan Kundu Amanda Askell Jackson Kernion Andy Jones Anna Chen Anna Goldie Azalia Mirhoseini Cameron McKinnon Carol Chen Catherine Olsson Christopher Olah Danny Hernandez Dawn Drain Deep Ganguli Dustin Li Eli Tran-Johnson Ethan Perez Jamie Kerr Jared Mueller Jeffrey Ladish Joshua Landau Kamal Ndousse Kamile Lukosuite Liane Lovitt Michael Sellitto Nelson Elhage Nicholas Schiefer Noemi Mercado Nova DasSarma Robert Lasenby Robin Larson Sam Ringer Scott Johnston Shauna Kravec Sheer\u00a0El Showk Stanislav Fort Tamera Lanham Timothy Telleen-Lawton Tom Conerly Tom Henighan Tristan Hume Samuel\u00a0R. Bowman Zac Hatfield-Dodds Ben Mann Dario Amodei Nicholas Joseph Sam McCandlish Tom Brown and Jared Kaplan. 2022. Constitutional AI: Harmlessness from AI Feedback. https:\/\/doi.org\/10.48550\/arXiv.2212.08073 arXiv:https:\/\/arXiv.org\/abs\/2212.08073 [cs].","DOI":"10.48550\/arXiv.2212.08073"},{"key":"e_1_3_3_3_9_2","volume-title":"Workshop on Responsibly Building the Next Generation of Multimodal Foundational Models","author":"Barnhart Logan","unstructured":"Logan Barnhart, Reza\u00a0Akbarian Bafghi, Maziar Raissi, and Stephen Becker. [n. d.]. Aligning to What? Limits to RLHF Based Alignment. In Workshop on Responsibly Building the Next Generation of Multimodal Foundational Models."},{"key":"e_1_3_3_3_10_2","unstructured":"Noam Benkler Drisana Mosaphir Scott Friedman Andrew Smart and Sonja Schmer-Galunder. 2023. Assessing LLMs for Moral Value Pluralism. arxiv:https:\/\/arXiv.org\/abs\/2312.10075\u00a0[cs.CL]"},{"key":"e_1_3_3_3_11_2","volume-title":"praw-dev\/praw","author":"Boe Bryce","year":"2025","unstructured":"Bryce Boe, Joel Payne, PythonCoderAS, Joey Rees-Hill, Andreas\u00a0Damgaard Pedersen, Ethan Dalool, Tim, Levi Roth, 13steinj, nmtake, Pyprohly, Julian Berman, Watchful1, kungming2, Alexander Putilin, MaybeNetwork, Liudvikam, Nemec, CrackedP0t, Jamie Magee, C.A.M. Gerlach, A. Baisero, Chad Birch, vallard192, bakonydraco, D0cR3d, Steve C, Aur\u00e9lien Bombo, Zhifu Ge, and Michael Lazar. 2025. praw-dev\/praw. https:\/\/github.com\/praw-dev\/praw"},{"key":"e_1_3_3_3_12_2","unstructured":"Vamshi\u00a0Krishna Bonagiri Sreeram Vennam Priyanshul Govil Ponnurangam Kumaraguru and Manas Gaur. 2024. SaGE: Evaluating Moral Consistency in Large Language Models. arxiv:https:\/\/arXiv.org\/abs\/2402.13709\u00a0[cs.CL]"},{"key":"e_1_3_3_3_13_2","doi-asserted-by":"crossref","unstructured":"William\u00a0J Brady M\u00a0J Crockett and Jay\u00a0J Van\u00a0Bavel. 2020. The MAD model of moral contagion: The role of motivation attention and design in the spread of moralized content online. Perspect. Psychol. Sci. 15 4 (July 2020) 978\u20131010.","DOI":"10.1177\/1745691620917336"},{"key":"e_1_3_3_3_14_2","unstructured":"Maarten Buyl Alexander Rogiers Sander Noels Iris Dominguez-Catena Edith Heiter Raphael Romero Iman Johary Alexandru-Cristian Mara Jefrey Lijffijt and Tijl\u00a0De Bie. 2024. Large Language Models Reflect the Ideology of their Creators. arxiv:https:\/\/arXiv.org\/abs\/2410.18417\u00a0[cs.CL] https:\/\/arxiv.org\/abs\/2410.18417"},{"key":"e_1_3_3_3_15_2","doi-asserted-by":"publisher","unstructured":"Maarten Buyl Alexander Rogiers Sander Noels Iris Dominguez-Catena Edith Heiter Raphael Romero Iman Johary Alexandru-Cristian Mara Jefrey Lijffijt and Tijl\u00a0De Bie. 2024. Large Language Models Reflect the Ideology of their Creators. https:\/\/doi.org\/10.48550\/arXiv.2410.18417 arXiv:https:\/\/arXiv.org\/abs\/2410.18417.","DOI":"10.48550\/arXiv.2410.18417"},{"key":"e_1_3_3_3_16_2","doi-asserted-by":"crossref","unstructured":"Yong Cao Li Zhou Seolhwa Lee Laura Cabello Min Chen and Daniel Hershcovich. 2023. Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Study. arxiv:https:\/\/arXiv.org\/abs\/2303.17466\u00a0[cs.CL]","DOI":"10.18653\/v1\/2023.c3nlp-1.7"},{"key":"e_1_3_3_3_17_2","doi-asserted-by":"publisher","unstructured":"Yupeng Chang Xu Wang Jindong Wang Yuan Wu Linyi Yang Kaijie Zhu Hao Chen Xiaoyuan Yi Cunxiang Wang Yidong Wang Wei Ye Yue Zhang Yi Chang Philip\u00a0S. Yu Qiang Yang and Xing Xie. 2024. A Survey on Evaluation of Large Language Models. ACM Trans. Intell. Syst. Technol. 15 3 Article 39 (March 2024) 45\u00a0pages. https:\/\/doi.org\/10.1145\/3641289","DOI":"10.1145\/3641289"},{"key":"e_1_3_3_3_18_2","doi-asserted-by":"publisher","unstructured":"Yu\u00a0Ying Chiu Liwei Jiang and Yejin Choi. 2024. DailyDilemmas: Revealing Value Preferences of LLMs with Quandaries of Daily Life. https:\/\/doi.org\/10.48550\/arXiv.2410.02683 arXiv:https:\/\/arXiv.org\/abs\/2410.02683 [cs].","DOI":"10.48550\/arXiv.2410.02683"},{"key":"e_1_3_3_3_19_2","unstructured":"Paul\u00a0F Christiano Jan Leike Tom Brown Miljan Martic Shane Legg and Dario Amodei. 2017. Deep reinforcement learning from human preferences. Advances in neural information processing systems 30 (2017)."},{"key":"e_1_3_3_3_20_2","doi-asserted-by":"crossref","unstructured":"Scott Clifford Vijeth Iyengar Roberto Cabeza and Walter Sinnott-Armstrong. 2015. Moral foundations vignettes: A standardized stimulus database of scenarios based on moral foundations theory. Behavior research methods 47 4 (2015) 1178\u20131198.","DOI":"10.3758\/s13428-014-0551-2"},{"key":"e_1_3_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3501247.3531549"},{"key":"e_1_3_3_3_22_2","first-page":"11","volume-title":"International Workshop on Value Engineering in AI","author":"De\u00a0Giorgis Stefano","year":"2023","unstructured":"Stefano De\u00a0Giorgis and Aldo Gangemi. 2023. That\u2019s all Folks: A KG of values as commonsense social norms and behaviors. In International Workshop on Value Engineering in AI. Springer, 11\u201327."},{"key":"e_1_3_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1017\/9781009303538"},{"key":"e_1_3_3_3_24_2","doi-asserted-by":"crossref","unstructured":"Morteza Dehghani Kate Johnson Joe Hoover Eyal Sagi Justin Garten Niki\u00a0Jitendra Parmar Stephen Vaisey Rumen Iliev and Jesse Graham. 2016. Purity homophily in social networks. J. Exp. Psychol. Gen. 145 3 (March 2016) 366\u2013375.","DOI":"10.1037\/xge0000139"},{"key":"e_1_3_3_3_25_2","unstructured":"Abhimanyu Dubey Abhinav Jauhri Abhinav Pandey Abhishek Kadian Ahmad Al-Dahle Aiesha Letman Akhil Mathur Alan Schelten Amy Yang Angela Fan et\u00a0al. 2024. The llama 3 herd of models. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2407.21783 (2024)."},{"key":"e_1_3_3_3_26_2","doi-asserted-by":"publisher","unstructured":"Ronald Fischer Markus Luczak-Roesch and Johannes\u00a0A. Karl. 2023. What does ChatGPT return about human values? Exploring value bias in ChatGPT using a descriptive value theory. https:\/\/doi.org\/10.48550\/arXiv.2304.03612 arXiv:https:\/\/arXiv.org\/abs\/2304.03612 [cs].","DOI":"10.48550\/arXiv.2304.03612"},{"key":"e_1_3_3_3_27_2","doi-asserted-by":"publisher","unstructured":"Kathleen\u00a0C. Fraser Svetlana Kiritchenko and Esma Balkir. 2022. Does Moral Code Have a Moral Code? Probing Delphi\u2019s Moral Philosophy. https:\/\/doi.org\/10.48550\/arXiv.2205.12771 arXiv:https:\/\/arXiv.org\/abs\/2205.12771 [cs].","DOI":"10.48550\/arXiv.2205.12771"},{"key":"e_1_3_3_3_28_2","doi-asserted-by":"crossref","unstructured":"Sasuke Fujimoto and Kazuhiro Takemoto. 2023. Revisiting the political biases of ChatGPT. Frontiers in Artificial Intelligence 6 (2023) 1232003.","DOI":"10.3389\/frai.2023.1232003"},{"key":"e_1_3_3_3_29_2","doi-asserted-by":"crossref","unstructured":"Iason Gabriel. 2020. Artificial intelligence values and alignment. Minds and machines 30 3 (2020) 411\u2013437.","DOI":"10.1007\/s11023-020-09539-2"},{"key":"e_1_3_3_3_30_2","doi-asserted-by":"publisher","unstructured":"Basile Garcia Crystal Qian and Stefano Palminteri. 2024. The Moral Turing Test: Evaluating Human-LLM Alignment in Moral Decision-Making. https:\/\/doi.org\/10.48550\/arXiv.2410.07304 arXiv:https:\/\/arXiv.org\/abs\/2410.07304 [cs].","DOI":"10.48550\/arXiv.2410.07304"},{"key":"e_1_3_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.findings-emnlp.420"},{"key":"e_1_3_3_3_32_2","doi-asserted-by":"publisher","DOI":"10.1609\/icwsm.v17i1.22141"},{"key":"e_1_3_3_3_33_2","first-page":"55","volume-title":"Advances in experimental social psychology","author":"Graham Jesse","year":"2013","unstructured":"Jesse Graham, Jonathan Haidt, Sena Koleva, Matt Motyl, Ravi Iyer, Sean\u00a0P Wojcik, and Peter\u00a0H Ditto. 2013. Moral foundations theory: The pragmatic validity of moral pluralism. In Advances in experimental social psychology. Vol.\u00a047. Elsevier, 55\u2013130."},{"key":"e_1_3_3_3_34_2","doi-asserted-by":"publisher","unstructured":"Andrea\u00a0L Guzman and Seth\u00a0C Lewis. 2020. Artificial intelligence and communication: A Human\u2013Machine Communication research agenda. New Media & Society 22 1 (2020) 70\u201386. https:\/\/doi.org\/10.1177\/1461444819858691 arXiv:10.1177\/1461444819858691","DOI":"10.1177\/1461444819858691"},{"key":"e_1_3_3_3_35_2","doi-asserted-by":"publisher","unstructured":"Jochen Hartmann Jasper Schwenzow and Maximilian Witte. 2023. The political ideology of conversational AI: Converging evidence on ChatGPT\u2019s pro-environmental left-libertarian orientation. Social Science Research Network (2023). https:\/\/doi.org\/10.48550\/arxiv.2301.01768","DOI":"10.48550\/arxiv.2301.01768"},{"key":"e_1_3_3_3_36_2","doi-asserted-by":"publisher","unstructured":"Dan Hendrycks Collin Burns Steven Basart Andrew Critch Jerry Li Dawn Song and Jacob Steinhardt. 2023. Aligning AI With Shared Human Values. https:\/\/doi.org\/10.48550\/arXiv.2008.02275 arXiv:https:\/\/arXiv.org\/abs\/2008.02275 [cs].","DOI":"10.48550\/arXiv.2008.02275"},{"key":"e_1_3_3_3_37_2","doi-asserted-by":"crossref","unstructured":"Yining Hua Fenglin Liu Kailai Yang Zehan Li Hongbin Na Yi-han Sheu Peilin Zhou Lauren\u00a0V Moran Sophia Ananiadou Andrew Beam et\u00a0al. 2024. Large language models in mental health care: a scoping review. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2401.02984 (2024).","DOI":"10.2196\/preprints.64088"},{"key":"e_1_3_3_3_38_2","doi-asserted-by":"publisher","unstructured":"Jianchao Ji Yutong Chen Mingyu Jin Wujiang Xu Wenyue Hua and Yongfeng Zhang. 2024. MoralBench: Moral Evaluation of LLMs. arXiv.org (June 2024). https:\/\/doi.org\/10.48550\/arxiv.2406.04428","DOI":"10.48550\/arxiv.2406.04428"},{"key":"e_1_3_3_3_39_2","unstructured":"Albert\u00a0Q Jiang Alexandre Sablayrolles Arthur Mensch Chris Bamford Devendra\u00a0Singh Chaplot Diego de\u00a0las Casas Florian Bressand Gianna Lengyel Guillaume Lample Lucile Saulnier et\u00a0al. 2023. Mistral 7B. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2310.06825 (2023)."},{"key":"e_1_3_3_3_40_2","unstructured":"Rebecca\u00a0L Johnson Giada Pistilli Natalia Men\u00e9dez-Gonz\u00e1lez Leslye Denisse\u00a0Dias Duran Enrico Panai Julija Kalpokiene and Donald\u00a0Jay Bertulfo. 2022. The Ghost in the Machine has an American accent: value conflict in GPT-3. arxiv:https:\/\/arXiv.org\/abs\/2203.07785\u00a0[cs.CL]"},{"key":"e_1_3_3_3_41_2","unstructured":"Klaus Krippendorff. 2011. Computing Krippendorff\u2019s alpha-reliability."},{"key":"e_1_3_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.emnlp-main.764"},{"key":"e_1_3_3_3_43_2","doi-asserted-by":"crossref","unstructured":"Hannah\u00a0R Lawrence Renee\u00a0A Schneider Susan\u00a0B Rubin Maja\u00a0J Matari\u0107 Daniel\u00a0J McDuff and Megan\u00a0Jones Bell. 2024. The opportunities and risks of large language models in mental health. JMIR Mental Health 11 1 (2024) e59479.","DOI":"10.2196\/59479"},{"key":"e_1_3_3_3_44_2","unstructured":"Haitao Li Qian Dong Junjie Chen Huixue Su Yujia Zhou Qingyao Ai Ziyi Ye and Yiqun Liu. 2024. LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2412.05579 (2024). https:\/\/arxiv.org\/abs\/2412.05579 To appear in ACM Transactions."},{"key":"e_1_3_3_3_45_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.findings-emnlp.513"},{"key":"e_1_3_3_3_46_2","first-page":"1105","volume-title":"AMIA Annual Symposium Proceedings","volume":"2023","author":"Ma Zilin","year":"2024","unstructured":"Zilin Ma, Yiyang Mei, and Zhaoyuan Su. 2024. Understanding the benefits and challenges of using large language model-based conversational agents for mental well-being support. In AMIA Annual Symposium Proceedings, Vol.\u00a02023. 1105."},{"key":"e_1_3_3_3_47_2","doi-asserted-by":"crossref","unstructured":"Fabio Motoki Valdemar Pinho\u00a0Neto and Victor Rodrigues. 2024. More human than human: measuring ChatGPT political bias. Public Choice 198 1 (2024) 3\u201323.","DOI":"10.1007\/s11127-023-01097-2"},{"key":"e_1_3_3_3_48_2","doi-asserted-by":"publisher","DOI":"10.1609\/icwsm.v16i1.19327"},{"key":"e_1_3_3_3_49_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.acl-long.26"},{"key":"e_1_3_3_3_50_2","unstructured":"Rajesh Ranjan Shailja Gupta and Surya\u00a0Narayan Singh. 2024. A Comprehensive Survey of Bias in LLMs: Current Landscape and Future Directions. arxiv:https:\/\/arXiv.org\/abs\/2409.16430\u00a0[cs.CL] https:\/\/arxiv.org\/abs\/2409.16430"},{"key":"e_1_3_3_3_51_2","doi-asserted-by":"publisher","unstructured":"Yuanyi Ren Haoran Ye Hanjun Fang Xin Zhang and Guojie Song. 2024. ValueBench: Towards Comprehensively Evaluating Value Orientations and Understanding of Large Language Models. https:\/\/doi.org\/10.48550\/arXiv.2406.04214 arXiv:https:\/\/arXiv.org\/abs\/2406.04214.","DOI":"10.48550\/arXiv.2406.04214"},{"key":"e_1_3_3_3_52_2","doi-asserted-by":"publisher","unstructured":"David Rozado. 2024. The Political Preferences of LLMs. https:\/\/doi.org\/10.48550\/arXiv.2402.01789 arXiv:https:\/\/arXiv.org\/abs\/2402.01789.","DOI":"10.48550\/arXiv.2402.01789"},{"key":"e_1_3_3_3_53_2","doi-asserted-by":"publisher","unstructured":"J\u00e9r\u00f4me Rutinowski Sven Franke Jan Endendyk Ina Dormuth and Markus Pauly. 2023. The Self-Perception and Political Biases of ChatGPT. https:\/\/doi.org\/10.48550\/arXiv.2304.07333 arXiv:https:\/\/arXiv.org\/abs\/2304.07333 [cs].","DOI":"10.48550\/arXiv.2304.07333"},{"key":"e_1_3_3_3_54_2","unstructured":"Paul R\u00f6ttger Valentin Hofmann Valentina Pyatkin Musashi Hinck Hannah\u00a0Rose Kirk Hinrich Sch\u00fctze and Dirk Hovy. 2024. Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models. http:\/\/arxiv.org\/abs\/2402.16786 arXiv:https:\/\/arXiv.org\/abs\/2402.16786."},{"key":"e_1_3_3_3_55_2","doi-asserted-by":"crossref","unstructured":"Pratik Sachdeva and Tom van Nuenen. 2025. Normative Evaluation of Large Language Models with Everyday Moral Dilemmas. https:\/\/github.com\/dlab-projects\/normative_evaluation_llms_everyday_dilemmas. Code repository for the paper to appear in the 2025 ACM Conference on Fairness Accountability and Transparency (FAccT).","DOI":"10.1145\/3715275.3732044"},{"key":"e_1_3_3_3_56_2","doi-asserted-by":"crossref","unstructured":"Pratik\u00a0S. Sachdeva and Tom van Nuenen. 2025. Normative Evaluation of Large Language Models with Everyday Moral Dilemmas. https:\/\/huggingface.co\/datasets\/ucberkeley-dlab\/normative_evaluation_llms_everyday_dilemmas. Dataset accompanying the paper to appear in the 2025 ACM Conference on Fairness Accountability and Transparency (FAccT).","DOI":"10.1145\/3715275.3732044"},{"key":"e_1_3_3_3_57_2","doi-asserted-by":"publisher","unstructured":"Shibani Santurkar Esin Durmus Faisal Ladhak Cinoo Lee Percy Liang and Tatsunori Hashimoto. 2023. Whose Opinions Do Language Models Reflect? International Conference on Machine Learning (2023). https:\/\/doi.org\/10.48550\/arxiv.2303.17548","DOI":"10.48550\/arxiv.2303.17548"},{"key":"e_1_3_3_3_58_2","unstructured":"Nino Scherrer Claudia Shi Amir Feder and David\u00a0M. Blei. 2023. Evaluating the Moral Beliefs Encoded in LLMs. arxiv:https:\/\/arXiv.org\/abs\/2307.14324\u00a0[cs.CL]"},{"key":"e_1_3_3_3_59_2","first-page":"1","volume-title":"Advances in experimental social psychology","author":"Schwartz Shalom\u00a0H","year":"1992","unstructured":"Shalom\u00a0H Schwartz. 1992. Universals in the content and structure of values: Theoretical advances and empirical tests in 20 countries. In Advances in experimental social psychology. Vol.\u00a025. Elsevier, 1\u201365."},{"key":"e_1_3_3_3_60_2","unstructured":"Melanie Sclar Yejin Choi Yulia Tsvetkov and Alane Suhr. 2024. Quantifying Language Models\u2019 Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting. arXiv 2310.11324 [preprint] https:\/\/arxiv. org\/abs\/2310.11324. Published October 17 2023. Accessed January (2024)."},{"key":"e_1_3_3_3_61_2","doi-asserted-by":"crossref","unstructured":"Itamar Shatz. 2017. Fast free and targeted: Reddit as a source for recruiting participants online. Social Science Computer Review 35 4 (2017) 537\u2013549.","DOI":"10.1177\/0894439316650163"},{"key":"e_1_3_3_3_62_2","unstructured":"Gemma Team Thomas Mesnard Cassidy Hardin Robert Dadashi Surya Bhupatiraju Shreya Pathak Laurent Sifre Morgane Rivi\u00e8re Mihir\u00a0Sanjay Kale Juliette Love et\u00a0al. 2024. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2403.08295 (2024)."},{"key":"e_1_3_3_3_63_2","unstructured":"Hugo Touvron Louis Martin Kevin Stone Peter Albert Amjad Almahairi Yasmine Babaei Nikolay Bashlykov Soumya Batra Prajjwal Bhargava Shruti Bhosale et\u00a0al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2307.09288 (2023)."},{"key":"e_1_3_3_3_64_2","unstructured":"Jackson Trager Alireza\u00a0S. Ziabari Aida\u00a0Mostafazadeh Davani Preni Golazizian Farzan Karimi-Malekabadi Ali Omrani Zhihe Li Brendan Kennedy Nils\u00a0Karl Reimer Melissa Reyes Kelsey Cheng Mellow Wei Christina Merrifield Arta Khosravi Evans Alvarez and Morteza Dehghani. 2022. The Moral Foundations Reddit Corpus. arxiv:https:\/\/arXiv.org\/abs\/2208.05545\u00a0[cs.CL]"},{"key":"e_1_3_3_3_65_2","unstructured":"Jindong Wang Xixu Hu Wenxin Hou Hao Chen Runkai Zheng Yidong Wang Linyi Yang Haojun Huang Wei Ye Xiubo Geng et\u00a0al. 2023. On the robustness of chatgpt: An adversarial and out-of-distribution perspective. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2302.12095 (2023)."},{"key":"e_1_3_3_3_66_2","unstructured":"Zhichao Wang Bin Bi Shiva\u00a0Kumar Pentyala Kiran Ramnath Sougata Chaudhuri Shubham Mehrotra Xiang-Bo Mao Sitaram Asur et\u00a0al. 2024. A comprehensive survey of LLM alignment techniques: RLHF RLAIF PPO DPO and more. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2407.16216 (2024)."},{"key":"e_1_3_3_3_67_2","unstructured":"Ziwei Xu Sanjay Jain and Mohan Kankanhalli. 2024. Hallucination is inevitable: An innate limitation of large language models. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2401.11817 (2024)."},{"key":"e_1_3_3_3_68_2","unstructured":"Jiahao Ying Yixin Cao Kai Xiong Yidong He Long Cui and Yongbin Liu. 2023. Intuitive or Dependent? Investigating LLMs\u2019 Robustness to Conflicting Prompts. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2309.17415 (2023)."},{"key":"e_1_3_3_3_69_2","unstructured":"Ye Yuan Kexin Tang Jianhao Shen Ming Zhang and Chenguang Wang. 2024. Measuring Social Norms of Large Language Models. arxiv:https:\/\/arXiv.org\/abs\/2404.02491\u00a0[cs.CL]"},{"key":"e_1_3_3_3_70_2","doi-asserted-by":"publisher","unstructured":"Daniel\u00a0A Yudkin Geoffrey Goodwin Andrew\u00a0G Reece Kurt Gray and Sudeep Bhatia. 2023. A Large-Scale Investigation of Everyday Moral Dilemmas. https:\/\/doi.org\/10.31234\/osf.io\/5pcew","DOI":"10.31234\/osf.io\/5pcew"},{"key":"e_1_3_3_3_71_2","unstructured":"Eliezer Yudkowsky. 2016. The AI alignment problem: why it is hard and where to start. Symbolic Systems Distinguished Speaker 4 (2016)."},{"key":"e_1_3_3_3_72_2","doi-asserted-by":"publisher","unstructured":"Chujie Zheng Hao Zhou Fandong Meng Jie Zhou and Minlie Huang. 2024. Large Language Models Are Not Robust Multiple Choice Selectors. https:\/\/doi.org\/10.48550\/arXiv.2309.03882 arXiv:https:\/\/arXiv.org\/abs\/2309.03882 [cs].","DOI":"10.48550\/arXiv.2309.03882"}],"event":{"name":"FAccT '25: The 2025 ACM Conference on Fairness, Accountability, and Transparency","location":"Athens Greece","acronym":"FAccT '25"},"container-title":["Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3715275.3732044","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,24]],"date-time":"2025-06-24T11:10:30Z","timestamp":1750763430000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3715275.3732044"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,23]]},"references-count":71,"alternative-id":["10.1145\/3715275.3732044","10.1145\/3715275"],"URL":"https:\/\/doi.org\/10.1145\/3715275.3732044","relation":{},"subject":[],"published":{"date-parts":[[2025,6,23]]},"assertion":[{"value":"2025-06-23","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}