{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T06:47:50Z","timestamp":1782370070625,"version":"3.54.5"},"publisher-location":"New York, NY, USA","reference-count":110,"publisher":"ACM","license":[{"start":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T00:00:00Z","timestamp":1776038400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"funder":[{"name":"USDA National Institute of Food and Agriculture","award":["2021-67022-33447"],"award-info":[{"award-number":["2021-67022-33447"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2026,4,13]]},"DOI":"10.1145\/3772318.3790897","type":"proceedings-article","created":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T04:12:21Z","timestamp":1776053541000},"page":"1-20","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Designing Staged Evaluation Workflows for LLMs: Integrating Domain Experts, Lay Users, and Model-Generated Evaluation Criteria"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-5472-282X","authenticated-orcid":false,"given":"Annalisa","family":"Szymanski","sequence":"first","affiliation":[{"name":"Department of Computer Science and Engineering, University of Notre Dame, Notre Dame, Indiana, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1772-6065","authenticated-orcid":false,"given":"Simret","family":"Araya Gebreegziabher","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, University of Notre Dame, Notre Dame, Indiana, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5773-8239","authenticated-orcid":false,"given":"Oghenemaro","family":"Anuyah","sequence":"additional","affiliation":[{"name":"Microsoft Corporation, Redmond, Washington, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2206-1720","authenticated-orcid":false,"given":"Ronald","family":"A. Metoyer","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, University of Notre Dame, Notre Dame, Indiana, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7902-7625","authenticated-orcid":false,"given":"Toby","family":"Jia-Jun Li","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, University of Notre Dame, Notre Dame, Indiana, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,4,13]]},"reference":[{"key":"e_1_3_3_1_2_2","doi-asserted-by":"crossref","unstructured":"Muhammad\u00a0Aurangzeb Ahmad Ilker Yaramis and Taposh\u00a0Dutta Roy. 2023. Creating trustworthy llms: Dealing with hallucinations in healthcare ai. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2311.01463 (2023).","DOI":"10.20944\/preprints202310.1662.v1"},{"key":"e_1_3_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR66628.2025.00086"},{"key":"e_1_3_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300233"},{"key":"e_1_3_3_1_5_2","unstructured":"Konstantinos Andriopoulos and Johan Pouwelse. 2023. Augmenting LLMs with Knowledge: A survey on hallucination prevention. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2309.16459 (2023)."},{"key":"e_1_3_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3613904.3642016"},{"key":"e_1_3_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/3586182.3616660"},{"key":"e_1_3_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/3613905.3636319"},{"key":"e_1_3_3_1_9_2","doi-asserted-by":"crossref","unstructured":"John\u00a0W Ayers Adam Poliak Mark Dredze Eric\u00a0C Leas Zechariah Zhu Jessica\u00a0B Kelley Dennis\u00a0J Faix Aaron\u00a0M Goodman Christopher\u00a0A Longhurst Michael Hogarth et\u00a0al. 2023. Comparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media forum. JAMA internal medicine 183 6 (2023) 589\u2013596.","DOI":"10.1001\/jamainternmed.2023.1838"},{"key":"e_1_3_3_1_10_2","doi-asserted-by":"crossref","unstructured":"Angeline Chatelan Aur\u00e9lien Clerc and Pierre-Alexandre Fonta. 2023. ChatGPT and future artificial intelligence chatbots: what may be the influence on credentialed nutrition and dietetics practitioners?Journal of the Academy of Nutrition and Dietetics 123 11 (2023) 1525\u20131531.","DOI":"10.1016\/j.jand.2023.08.001"},{"key":"e_1_3_3_1_11_2","unstructured":"Yida Chen Aoyu Wu Trevor DePodesta Catherine Yeh Kenneth Li Nicholas\u00a0Castillo Marin Oam Patel Jan Riecke Shivam Raval Olivia Seow et\u00a0al. 2024. Designing a Dashboard for Transparency and Control of Conversational AI. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2406.07882 (2024)."},{"key":"e_1_3_3_1_12_2","unstructured":"Zixiang Chen Yihe Deng Huizhuo Yuan Kaixuan Ji and Quanquan Gu. 2024. Self-play fine-tuning converts weak language models to strong language models. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2401.01335 (2024)."},{"key":"e_1_3_3_1_13_2","doi-asserted-by":"crossref","unstructured":"Szu-Wei Cheng Chung-Wen Chang Wan-Jung Chang Hao-Wei Wang Chih-Sung Liang Taishiro Kishimoto Jane Pei-Chen Chang John\u00a0S Kuo and Kuan-Pin Su. 2023. The now and future of ChatGPT and GPT in psychiatry. Psychiatry and clinical neurosciences 77 11 (2023) 592\u2013596.","DOI":"10.1111\/pcn.13588"},{"key":"e_1_3_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/3630106.3659048"},{"key":"e_1_3_3_1_15_2","volume-title":"Forty-first International Conference on Machine Learning","author":"Chiang Wei-Lin","year":"2024","unstructured":"Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios\u00a0Nikolas Angelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang, Michael Jordan, Joseph\u00a0E Gonzalez, et\u00a0al. 2024. Chatbot arena: An open platform for evaluating llms by human preference. In Forty-first International Conference on Machine Learning."},{"key":"e_1_3_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3746059.3747677"},{"key":"e_1_3_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3706599.3719675"},{"key":"e_1_3_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v39i28.35351"},{"key":"e_1_3_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3640544.3645216"},{"key":"e_1_3_3_1_20_2","unstructured":"Yijiang\u00a0River Dong Tiancheng Hu and Nigel Collier. 2024. Can LLM be a Personalized Judge?arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2406.11657 (2024)."},{"key":"e_1_3_3_1_21_2","unstructured":"Eva Eigner and Thorsten H\u00e4ndler. 2024. Determinants of llm-assisted decision-making. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2402.17385 (2024)."},{"key":"e_1_3_3_1_22_2","doi-asserted-by":"crossref","unstructured":"Iason Gabriel and Vafa Ghazavi. 2022. The challenge of value alignment. The Oxford handbook of digital ethics (2022) 336\u2013355.","DOI":"10.1093\/oxfordhb\/9780198857815.013.18"},{"key":"e_1_3_3_1_23_2","unstructured":"Mingqi Gao Xinyu Hu Xunjian Yin Jie Ruan Xiao Pu and Xiaojun Wan. 2025. Llm-based nlg evaluation: Current status and challenges. Computational Linguistics (2025) 1\u201327."},{"key":"e_1_3_3_1_24_2","doi-asserted-by":"crossref","unstructured":"Manuel\u00a0B Garcia. 2023. ChatGPT as a virtual dietitian: Exploring its potential as a tool for improving nutrition knowledge. Applied System Innovation 6 5 (2023) 96.","DOI":"10.3390\/asi6050096"},{"key":"e_1_3_3_1_25_2","unstructured":"Simret\u00a0Araya Gebreegziabher Kuangshi Ai Zheng Zhang Elena\u00a0L Glassman and Toby Jia-Jun Li. 2024. Leveraging Variation Theory in Counterfactual Data Augmentation for Optimized Active Learning. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2408.03819 (2024)."},{"key":"e_1_3_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/3729176.3729199"},{"key":"e_1_3_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/3706598.3713708"},{"key":"e_1_3_3_1_28_2","doi-asserted-by":"crossref","unstructured":"Sebastian Gehrmann Elizabeth Clark and Thibault Sellam. 2023. Repairing the cracked foundation: A survey of obstacles in evaluation practices for generated text. Journal of Artificial Intelligence Research 77 (2023) 103\u2013166.","DOI":"10.1613\/jair.1.13715"},{"key":"e_1_3_3_1_29_2","doi-asserted-by":"crossref","unstructured":"Fabrizio Gilardi Meysam Alizadeh and Ma\u00ebl Kubli. 2023. ChatGPT outperforms crowd workers for text-annotation tasks. Proceedings of the National Academy of Sciences 120 30 (2023) e2305016120.","DOI":"10.1073\/pnas.2305016120"},{"key":"e_1_3_3_1_30_2","doi-asserted-by":"crossref","unstructured":"Suchin Gururangan Ana Marasovi\u0107 Swabha Swayamdipta Kyle Lo Iz Beltagy Doug Downey and Noah\u00a0A Smith. 2020. Don\u2019t stop pretraining: Adapt language models to domains and tasks. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2004.10964 (2020).","DOI":"10.18653\/v1\/2020.acl-main.740"},{"key":"e_1_3_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3580688"},{"key":"e_1_3_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.findings-acl.263"},{"key":"e_1_3_3_1_33_2","doi-asserted-by":"crossref","unstructured":"Robin\u00a0M Hogarth and Hillel\u00a0J Einhorn. 1992. Order effects in belief updating: The belief-adjustment model. Cognitive psychology 24 1 (1992) 1\u201355.","DOI":"10.1016\/0010-0285(92)90002-J"},{"key":"e_1_3_3_1_34_2","unstructured":"Lei Huang Weijiang Yu Weitao Ma Weihong Zhong Zhangyin Feng Haotian Wang Qianglong Chen Weihua Peng Xiaocheng Feng Bing Qin et\u00a0al. 2023. A survey on hallucination in large language models: Principles taxonomy challenges and open questions. ACM Transactions on Information Systems (2023)."},{"key":"e_1_3_3_1_35_2","unstructured":"Yue Huang Lichao Sun Haoran Wang Siyuan Wu Qihui Zhang Yuan Li Chujie Gao Yixin Huang Wenhan Lyu Yixuan Zhang et\u00a0al. 2024. Trustllm: Trustworthiness in large language models. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2401.05561 (2024)."},{"key":"e_1_3_3_1_36_2","doi-asserted-by":"crossref","unstructured":"Leo Huovinen. 2024. Assessing usability of large language models in education. (2024).","DOI":"10.31237\/osf.io\/p54gk"},{"key":"e_1_3_3_1_37_2","unstructured":"Cory Hymel and Hiroe Johnson. 2025. Analysis of LLMs vs Human Experts in Requirements Engineering. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2501.19297 (2025)."},{"key":"e_1_3_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-17322-6_26"},{"key":"e_1_3_3_1_39_2","doi-asserted-by":"crossref","unstructured":"Ziwei Ji Nayeon Lee Rita Frieske Tiezheng Yu Dan Su Yan Xu Etsuko Ishii Ye\u00a0Jin Bang Andrea Madotto and Pascale Fung. 2023. Survey of hallucination in natural language generation. Comput. Surveys 55 12 (2023) 1\u201338.","DOI":"10.1145\/3571730"},{"key":"e_1_3_3_1_40_2","unstructured":"Junfeng Jiao Saleh Afroogh Yiming Xu and Connor Phillips. 2024. Navigating llm ethics: Advancements challenges and future directions. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2406.18841 (2024)."},{"key":"e_1_3_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3613905.3650755"},{"key":"e_1_3_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3618260.3649777"},{"key":"e_1_3_3_1_43_2","unstructured":"Daniel Khashabi Gabriel Stanovsky Jonathan Bragg Nicholas Lourie Jungo Kasai Yejin Choi Noah\u00a0A Smith and Daniel\u00a0S Weld. 2021. GENIE: Toward reproducible and standardized human evaluation for text generation. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2101.06561 (2021)."},{"key":"e_1_3_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.eval4nlp-1.14"},{"key":"e_1_3_3_1_45_2","unstructured":"Tae\u00a0Soo Kim Heechan Lee Yoonjoo Lee Joseph Seering and Juho Kim. 2025. Evalet: Evaluating Large Language Models by Fragmenting Outputs into Functions. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2509.11206 (2025)."},{"key":"e_1_3_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3613904.3642216"},{"key":"e_1_3_3_1_47_2","doi-asserted-by":"crossref","unstructured":"Darlene\u00a0R King Guransh Nanda Joel Stoddard Allison Dempsey Sarah Hergert Jay\u00a0H Shore and John Torous. 2023. An introduction to generative artificial intelligence in mental health care: considerations and guidance. Current psychiatry reports 25 12 (2023) 839\u2013846.","DOI":"10.1007\/s11920-023-01477-x"},{"key":"e_1_3_3_1_48_2","doi-asserted-by":"crossref","unstructured":"Daniel Kirk Elise van Eijnatten and Guido Camps. 2023. Comparison of answers between ChatGPT and human dieticians to common nutrition questions. Journal of Nutrition and Metabolism 2023 1 (2023) 5548684.","DOI":"10.1155\/2023\/5548684"},{"key":"e_1_3_3_1_49_2","unstructured":"Hannah\u00a0Rose Kirk Bertie Vidgen Paul R\u00f6ttger and Scott\u00a0A Hale. 2024. The benefits risks and bounds of personalizing the alignment of large language models to individuals. Nature Machine Intelligence (2024) 1\u201310."},{"key":"e_1_3_3_1_50_2","volume-title":"Tech. Rep., Technical report, Working Paper","author":"Ko\u00e7ak Korhan","year":"2018","unstructured":"Korhan Ko\u00e7ak. 2018. Sequential updating: A behavioral model of belief change. In Tech. Rep., Technical report, Working Paper."},{"key":"e_1_3_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3715070.3749265"},{"key":"e_1_3_3_1_52_2","unstructured":"Q\u00a0Vera Liao and Jennifer\u00a0Wortman Vaughan. 2023. Ai transparency in the age of llms: A human-centered research roadmap. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2306.01941 (2023) 5368\u20135393."},{"key":"e_1_3_3_1_53_2","doi-asserted-by":"publisher","DOI":"10.1145\/3613905.3647957"},{"key":"e_1_3_3_1_54_2","first-page":"74","volume-title":"Text summarization branches out","author":"Lin Chin-Yew","year":"2004","unstructured":"Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out. 74\u201381."},{"key":"e_1_3_3_1_55_2","unstructured":"Fang Liu Yang Liu Lin Shi Houkun Huang Ruifeng Wang Zhen Yang and Li Zhang. 2024. Exploring and evaluating hallucinations in llm-powered code generation. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2404.00971 (2024)."},{"key":"e_1_3_3_1_56_2","unstructured":"June\u00a0M Liu Donghao Li He Cao Tianhe Ren Zeyi Liao and Jiamin Wu. 2023. Chatcounselor: A large language models for mental health support. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2309.15461 (2023)."},{"key":"e_1_3_3_1_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3501825"},{"key":"e_1_3_3_1_58_2","doi-asserted-by":"publisher","DOI":"10.63317\/3posqwchnb6g"},{"key":"e_1_3_3_1_59_2","unstructured":"Yang Liu Yuanshun Yao Jean-Francois Ton Xiaoying Zhang Ruocheng Guo\u00a0Hao Cheng Yegor Klochkov Muhammad\u00a0Faaiz Taufiq and Hang Li. 2023. Trustworthy LLMs: A survey and guideline for evaluating large language models\u2019 alignment. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2308.05374 (2023)."},{"key":"e_1_3_3_1_60_2","doi-asserted-by":"crossref","unstructured":"Adian Liusie Vatsal Raina Yassir Fathullah and Mark Gales. 2024. Efficient LLM Comparative Assessment: a Product of Experts Framework for Pairwise Comparisons. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2405.05894 (2024).","DOI":"10.18653\/v1\/2024.emnlp-main.389"},{"key":"e_1_3_3_1_61_2","unstructured":"Xiaotian Lu Jiyi Li Koh Takeuchi and Hisashi Kashima. 2024. AHP-Powered LLM Reasoning for Multi-Criteria Evaluation of Open-Ended Responses. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2410.01246 (2024)."},{"key":"e_1_3_3_1_62_2","doi-asserted-by":"publisher","DOI":"10.1109\/RE59067.2024.00046"},{"key":"e_1_3_3_1_63_2","doi-asserted-by":"publisher","DOI":"10.1145\/3706598.3713423"},{"key":"e_1_3_3_1_64_2","doi-asserted-by":"crossref","unstructured":"John\u00a0M Malouff Sarah\u00a0J Stein Lodewicka\u00a0N Bothma Kimberley Coulter and Ashley\u00a0J Emmerton. 2014. Preventing halo bias in grading the work of university students. Cogent Psychology 1 1 (2014) 988937.","DOI":"10.1080\/23311908.2014.988937"},{"key":"e_1_3_3_1_65_2","unstructured":"Seyedhadi Marjaei Fahimeh\u00a0Ahmadian Yazdi and M Chandrashekara. 2019. MAXQDA and its Application to LIS Research. Library Philosophy and Practice (2019) 1\u20139."},{"key":"e_1_3_3_1_66_2","doi-asserted-by":"crossref","unstructured":"Pepper Miller and Kristen DiCerbo. 2024. LLM Based Math Tutoring: Challenges and Dataset.","DOI":"10.35542\/osf.io\/5zwv3"},{"key":"e_1_3_3_1_67_2","volume-title":"Software testing and quality assurance: theory and practice","author":"Naik Kshirasagar","year":"2011","unstructured":"Kshirasagar Naik and Priyadarshi Tripathy. 2011. Software testing and quality assurance: theory and practice. John Wiley & Sons."},{"key":"e_1_3_3_1_68_2","unstructured":"Jingcheng Niu Xingdi Yuan Tong Wang Hamidreza Saghir and Amir\u00a0H Abdi. 2025. Llama See Llama Do: A Mechanistic Perspective on Contextual Entrainment and Distraction in LLMs. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2505.09338 (2025)."},{"key":"e_1_3_3_1_69_2","volume-title":"Things that make us smart","author":"Norman Don","unstructured":"Don Norman. [n. d.]. Things that make us smart."},{"key":"e_1_3_3_1_70_2","doi-asserted-by":"crossref","unstructured":"Qian Pan Zahra Ashktorab Michael Desmond Martin\u00a0Santillan Cooper James Johnson Rahul Nair Elizabeth Daly and Werner Geyer. 2024. Human-Centered Design Recommendations for LLM-as-a-Judge. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2407.03479 (2024).","DOI":"10.18653\/v1\/2024.hucllm-1.2"},{"key":"e_1_3_3_1_71_2","first-page":"311","volume-title":"Proceedings of the 40th annual meeting of the Association for Computational Linguistics","author":"Papineni Kishore","year":"2002","unstructured":"Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics. 311\u2013318."},{"key":"e_1_3_3_1_72_2","unstructured":"Ji-Lun Peng Sijia Cheng Egil Diau Yung-Yu Shih Po-Heng Chen Yen-Ting Lin and Yun-Nung Chen. 2024. A survey of useful llm evaluation. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2406.00936 (2024)."},{"key":"e_1_3_3_1_73_2","doi-asserted-by":"crossref","unstructured":"Valentina Ponzo Ilaria Goitre Enrica Favaro Fabio\u00a0Dario Merlo Maria\u00a0Vittoria Mancino Sergio Riso and Simona Bo. 2024. Is ChatGPT an Effective Tool for Providing Dietary Advice?Nutrients 16 4 (2024) 469.","DOI":"10.3390\/nu16040469"},{"key":"e_1_3_3_1_74_2","doi-asserted-by":"crossref","unstructured":"Sahana Ramnath Anurag Mudgil Brihi Joshi Skyler Hallinan and Xiang Ren. 2025. Amulet: Putting Complex Multi-Turn Conversations on the Stand with LLM Juries. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2505.20451 (2025).","DOI":"10.18653\/v1\/2025.emnlp-main.1300"},{"key":"e_1_3_3_1_75_2","unstructured":"Rajesh Ranjan Shailja Gupta and Surya\u00a0Narayan Singh. 2024. A comprehensive survey of bias in llms: Current landscape and future directions. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2409.16430 (2024)."},{"key":"e_1_3_3_1_76_2","doi-asserted-by":"crossref","unstructured":"Mary\u00a0G Roseman and Sandra\u00a0N Miller. 2021. Academy of nutrition and dietetics: revised 2021 standards of professional performance for registered dietitian nutritionists (competent proficient and expert) in management of food and nutrition systems. Journal of the Academy of Nutrition and Dietetics 121 6 (2021) 1157\u20131174.","DOI":"10.1016\/j.jand.2021.02.007"},{"key":"e_1_3_3_1_77_2","doi-asserted-by":"publisher","DOI":"10.1145\/3613904.3642810"},{"key":"e_1_3_3_1_78_2","first-page":"887","volume-title":"Healthcare","author":"Sallam Malik","year":"2023","unstructured":"Malik Sallam. 2023. ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. In Healthcare , Vol.\u00a011. MDPI, 887."},{"key":"e_1_3_3_1_79_2","unstructured":"Souvika Sarkar Mohammad\u00a0Fakhruddin Babar Monowar Hasan and Shubhra\u00a0Kanti Karmaker. 2024. LLMs as On-demand Customizable Service. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2401.16577 (2024)."},{"key":"e_1_3_3_1_80_2","doi-asserted-by":"crossref","unstructured":"Ulrike Sch\u00e4fer Lars Sipos and Claudia M\u00fcller-Birn. 2025. \u2019The AI is uncertain so am I. What now?\u2019: Navigating Shortcomings of Uncertainty Representations in Human-AI Collaboration with Capability-focused Guidance. Proceedings of the ACM on Human-Computer Interaction 9 7 (2025) 1\u201348.","DOI":"10.1145\/3757451"},{"key":"e_1_3_3_1_81_2","doi-asserted-by":"publisher","DOI":"10.1145\/3654777.3676450"},{"key":"e_1_3_3_1_82_2","doi-asserted-by":"publisher","unstructured":"Robert Soden Austin Toombs and Michaelanne Thomas. 2024. Evaluating Interpretive Research in HCI. Interactions 31 1 (Jan. 2024) 38\u201342. 10.1145\/3633200","DOI":"10.1145\/3633200"},{"key":"e_1_3_3_1_83_2","doi-asserted-by":"crossref","unstructured":"Binyang Song Qihao Zhu and Jianxi Luo. 2024. Human-AI collaboration by design. Proceedings of the Design Society 4 (2024) 2247\u20132256.","DOI":"10.1017\/pds.2024.227"},{"key":"e_1_3_3_1_84_2","doi-asserted-by":"crossref","unstructured":"Shashank Sonkar Kangqi Ni Sapana Chaudhary and Richard\u00a0G Baraniuk. 2024. Pedagogical alignment of large language models. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2402.05000 (2024).","DOI":"10.18653\/v1\/2024.findings-emnlp.797"},{"key":"e_1_3_3_1_85_2","doi-asserted-by":"crossref","unstructured":"Philipp Spitzer Joshua Holstein Patrick Hemmer Michael V\u00f6ssing Niklas K\u00fchl Dominik Martin and Gerhard Satzger. 2025. Human Delegation Behavior in Human-AI Collaboration: The Effect of Contextual Information. Proceedings of the ACM on Human-Computer Interaction 9 2 (2025) 1\u201328.","DOI":"10.1145\/3710999"},{"key":"e_1_3_3_1_86_2","unstructured":"Chongyan Sun Ken Lin Shiwei Wang Hulong Wu Chengfei Fu and Zhen Wang. 2024. LalaEval: A Holistic Human Evaluation Framework for Domain-Specific Large Language Models. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2408.13338 (2024)."},{"key":"e_1_3_3_1_87_2","doi-asserted-by":"publisher","DOI":"10.1145\/3613905.3650837"},{"key":"e_1_3_3_1_88_2","doi-asserted-by":"publisher","DOI":"10.1145\/3613904.3641924"},{"key":"e_1_3_3_1_89_2","doi-asserted-by":"publisher","DOI":"10.1145\/3708359.3712091"},{"key":"e_1_3_3_1_90_2","unstructured":"Kassym-Jomart Tokayev. 2023. Ethical Implications of Large Language Models A Multidimensional Exploration of Societal Economic and Technical Concerns. International Journal of Social Analytics 8 9 (Sep. 2023) 17\u201333. https:\/\/norislab.com\/index.php\/ijsa\/article\/view\/42"},{"key":"e_1_3_3_1_91_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397481.3450668"},{"key":"e_1_3_3_1_92_2","unstructured":"Angelina Wang Jamie Morgenstern and John\u00a0P Dickerson. 2024. Large language models cannot replace human participants because they cannot portray identity groups. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2402.01908 (2024)."},{"key":"e_1_3_3_1_93_2","doi-asserted-by":"publisher","DOI":"10.1145\/3334480.3381069"},{"key":"e_1_3_3_1_94_2","unstructured":"Jiayin Wang Weizhi Ma Peijie Sun Min Zhang and Jian-Yun Nie. 2024. Understanding User Experience in Large Language Model Interactions. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2401.08329 (2024)."},{"key":"e_1_3_3_1_95_2","doi-asserted-by":"crossref","unstructured":"Li Wang Xi Chen XiangWen Deng Hao Wen MingKe You WeiZhi Liu Qi Li and Jian Li. 2024. Prompt engineering in consistency and reliability with the evidence-based guideline for LLMs. npj Digital Medicine 7 1 (2024) 41.","DOI":"10.1038\/s41746-024-01029-4"},{"key":"e_1_3_3_1_96_2","unstructured":"Weixuan Wang Barry Haddow Alexandra Birch and Wei Peng. 2023. Assessing the reliability of large language model knowledge. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2310.09820 (2023)."},{"key":"e_1_3_3_1_97_2","doi-asserted-by":"publisher","DOI":"10.1145\/3706598.3714148"},{"key":"e_1_3_3_1_98_2","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445045"},{"key":"e_1_3_3_1_99_2","unstructured":"Ian Webster. 2023. promptfoo: Test your prompts. https:\/\/www.promptfoo.dev\/."},{"key":"e_1_3_3_1_100_2","unstructured":"Michael Williams and Tami Moser. 2019. The art of coding and thematic exploration in qualitative research. International management review 15 1 (2019) 45\u201355."},{"key":"e_1_3_3_1_101_2","unstructured":"Sara Wiltberger. 2025. Stop\u201cVibe testing\u201d your LLMS. it\u2019s time for real evals.https:\/\/developers.googleblog.com\/en\/streamline-llm-evaluation-with-stax\/"},{"key":"e_1_3_3_1_102_2","doi-asserted-by":"crossref","unstructured":"Jennifer Wood and Doris\u00a0E Gillis. 2015. Exploring dietitians\u2019 engagement with health literacy: concept and practice. Canadian Journal of Dietetic Practice and Research 76 2 (2015) 51\u201355.","DOI":"10.3148\/cjdpr-2015-003"},{"key":"e_1_3_3_1_103_2","unstructured":"Shujin Wu May Fung Cheng Qian Jeonghwan Kim Dilek Hakkani-Tur and Heng Ji. 2024. Aligning LLMs with Individual Preferences via Interaction. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2410.03642 (2024)."},{"key":"e_1_3_3_1_104_2","unstructured":"Yang Wu Yifan Zhang Yiwei Wang Yujun Cai Yurong Wu Yuran Wang Ning Xu and Jian Cheng. 2025. Answer-Centric or Reasoning-Driven? Uncovering the Latent Memory Anchor in LLMs. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2506.17630 (2025)."},{"key":"e_1_3_3_1_105_2","doi-asserted-by":"publisher","DOI":"10.1109\/QRS-C63300.2024.00023"},{"key":"e_1_3_3_1_106_2","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3581388"},{"key":"e_1_3_3_1_107_2","doi-asserted-by":"publisher","DOI":"10.1145\/3690624.3709187"},{"key":"e_1_3_3_1_108_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v38i17.29934"},{"key":"e_1_3_3_1_109_2","doi-asserted-by":"crossref","unstructured":"Lianmin Zheng Wei-Lin Chiang Ying Sheng Siyuan Zhuang Zhanghao Wu Yonghao Zhuang Zi Lin Zhuohan Li Dacheng Li Eric Xing et\u00a0al. 2024. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems 36 (2024).","DOI":"10.52202\/075280-2020"},{"key":"e_1_3_3_1_110_2","unstructured":"Fengbin Zhu Wenqiang Lei Chao Wang Jianming Zheng Soujanya Poria and Tat-Seng Chua. 2021. Retrieving and reading: A comprehensive survey on open-domain question answering. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2101.00774 (2021)."},{"key":"e_1_3_3_1_111_2","unstructured":"Daniel\u00a0M Ziegler Nisan Stiennon Jeffrey Wu Tom\u00a0B Brown Alec Radford Dario Amodei Paul Christiano and Geoffrey Irving. 2019. Fine-tuning language models from human preferences. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/1909.08593 (2019)."}],"event":{"name":"CHI 2026: CHI Conference on Human Factors in Computing Systems","location":"Barcelona Spain","acronym":"CHI '26","sponsor":["SIGCHI ACM Special Interest Group on Computer-Human Interaction"]},"container-title":["Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3772318.3790897","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3772318.3790897","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T06:05:56Z","timestamp":1782367556000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3772318.3790897"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,13]]},"references-count":110,"alternative-id":["10.1145\/3772318.3790897","10.1145\/3772318"],"URL":"https:\/\/doi.org\/10.1145\/3772318.3790897","relation":{},"subject":[],"published":{"date-parts":[[2026,4,13]]},"assertion":[{"value":"2026-04-13","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}