{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T23:17:15Z","timestamp":1776122235917,"version":"3.50.1"},"reference-count":68,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW2","license":[{"start":{"date-parts":[[2023,9,28]],"date-time":"2023-09-28T00:00:00Z","timestamp":1695859200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2023,9,28]]},"abstract":"<jats:p>AI systems are increasingly being used to support human decision making. It is important that AI advice is followed appropriately. However, according to existing literature, users typically under-rely or over-rely on AI systems, and this leads to sub-optimal team performance. In this context, we investigate the role of stated system accuracy by contrasting the lack of system information with the presence of system accuracy in a loan prediction task. We explore how the degree to which humans understand system accuracy influences their reliance on the AI system, by investigating numeracy levels and with the aid of analogies to explain system accuracy in a first of its kind between-subjects study (N=281). We found that explaining the stated accuracy of a system using analogies failed to help users rely on the AI systemappropriately (i.e., the tendency of users to rely on the system when the system is correct, or on themselves otherwise). To eliminate the impact of subjective attitudes towards analogy domains, we conducted a within-subjects study (N=248) where each participant worked on tasks with analogy-based explanations from different domains. Results from this second study confirmed that explaining stated accuracy of the system with analogies was not sufficient to facilitate appropriate reliance on the AI system in the context of loan prediction tasks, irrespective of individual user differences. Based on our findings from the two studies, we reason that the under-reliance on the AI system may be a result of users' overestimation of their own ability to solve the given task. Thus, although familiar analogies can be effective in improving the intelligibility of stated accuracy of the system, an improved understanding of system accuracy does not necessarily lead to improved system reliance and team performance.<\/jats:p>","DOI":"10.1145\/3610067","type":"journal-article","created":{"date-parts":[[2023,10,4]],"date-time":"2023-10-04T15:54:10Z","timestamp":1696434850000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":36,"title":["How Stated Accuracy of an AI System and Analogies to Explain Accuracy Affect Human Reliance on the System"],"prefix":"10.1145","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8152-4791","authenticated-orcid":false,"given":"Gaole","family":"He","sequence":"first","affiliation":[{"name":"Delft University of Technology, Delft, Netherlands"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0004-0681","authenticated-orcid":false,"given":"Stefan","family":"Buijsman","sequence":"additional","affiliation":[{"name":"Delft University of Technology, Delft, Netherlands"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6189-6539","authenticated-orcid":false,"given":"Ujwal","family":"Gadiraju","sequence":"additional","affiliation":[{"name":"Delft University of Technology, Delft, Netherlands"}]}],"member":"320","published-online":{"date-parts":[[2023,10,4]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445717"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1080\/13698575.2010.515667"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1509\/jppm.25.1.8"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3461702.3462571"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173951"},{"key":"e_1_2_1_6_1","volume-title":"Proceedings of the 25th International Conference on Intelligent User Interfaces. 454--464","author":"Zana Bucc","year":"2020","unstructured":"Zana Bucc inca, Phoebe Lin, Krzysztof Z Gajos, and Elena L Glassman. 2020. Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems. In Proceedings of the 25th International Conference on Intelligent User Interfaces. 454--464."},{"key":"e_1_2_1_7_1","volume-title":"Proceedings of the ACM on Human-Computer Interaction","volume":"5","author":"Zana Bucc","year":"2021","unstructured":"Zana Bucc inca, Maja Barbara Malaya, and Krzysztof Z Gajos. 2021. To trust or to think: cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making. Proceedings of the ACM on Human-Computer Interaction, Vol. 5, CSCW1 (2021), 1--21."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3447535.3462487"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3397481.3450644"},{"key":"e_1_2_1_10_1","unstructured":"Sam Desiere Kristine Langenbucher and Ludo Struyven. 2019. Statistical profiling in public employment services: An international comparison. (2019)."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1037\/xge0000033"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1287\/mnsc.2016.2643"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1518\/0018720024494856"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1609\/hcomp.v8i1.7462"},{"key":"e_1_2_1_15_1","volume-title":"Dermatologist-level classification of skin cancer with deep neural networks. nature","author":"Esteva Andre","year":"2017","unstructured":"Andre Esteva, Brett Kuprel, Roberto A Novoa, Justin Ko, Susan M Swetter, Helen M Blau, and Sebastian Thrun. 2017. Dermatologist-level classification of skin cancer with deep neural networks. nature, Vol. 542, 7639 (2017), 115--118."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1177\/0272989X07304449"},{"key":"e_1_2_1_17_1","volume-title":"Statistical power analyses using G* Power 3.1: Tests for correlation and regression analyses. Behavior research methods","author":"Faul Franz","year":"2009","unstructured":"Franz Faul, Edgar Erdfelder, Axel Buchner, and Albert-Georg Lang. 2009. Statistical power analyses using G* Power 3.1: Tests for correlation and regression analyses. Behavior research methods, Vol. 41, 4 (2009), 1149--1160."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1080\/10447318.2018.1456150"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3119930"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702443"},{"key":"e_1_2_1_21_1","volume-title":"Communicating consequences of risky behaviors: Life expectancy versus risk of disease. Patient education and counseling","author":"Galesic Mirta","year":"2011","unstructured":"Mirta Galesic and Rocio Garcia-Retamero. 2011. Communicating consequences of risky behaviors: Life expectancy versus risk of disease. Patient education and counseling, Vol. 82, 1 (2011), 30--35."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1002\/acp.2866"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1008699112516"},{"key":"e_1_2_1_24_1","volume-title":"Numeracy and risk literacy: What have we learned so far? The Spanish journal of psychology","author":"Garcia-Retamero Rocio","year":"2019","unstructured":"Rocio Garcia-Retamero, Agata Sobkow, Dafina Petrova, Dunia Garrido, and Jakub Traczyk. 2019. Numeracy and risk literacy: What have we learned so far? The Spanish journal of psychology, Vol. 22 (2019)."},{"key":"e_1_2_1_25_1","volume-title":"Structure-mapping: A theoretical framework for analogy. Cognitive science","author":"Gentner Dedre","year":"1983","unstructured":"Dedre Gentner. 1983. Structure-mapping: A theoretical framework for analogy. Cognitive science, Vol. 7, 2 (1983), 155--170."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287563"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359152"},{"key":"e_1_2_1_28_1","volume-title":"A survey of methods for explaining black box models. ACM computing surveys (CSUR)","author":"Guidotti Riccardo","year":"2018","unstructured":"Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. 2018. A survey of methods for explaining black box models. ACM computing surveys (CSUR), Vol. 51, 5 (2018), 1--42."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1609\/hcomp.v10i1.21990"},{"key":"e_1_2_1_30_1","volume-title":"Proceedings of the Workshop on Trust and Reliance on AI-Human Teams at the ACM Conference on Human Factors in Computing Systems (CHI'22)","author":"He Gaole","year":"2022","unstructured":"Gaole He and Ujwal Gadiraju. 2022. Walking on Eggshells: Using Analogies to Promote Appropriate Reliance in Human-AI Decision Making. In Proceedings of the Workshop on Trust and Reliance on AI-Human Teams at the ACM Conference on Human Factors in Computing Systems (CHI'22)."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3581025"},{"key":"e_1_2_1_32_1","volume-title":"Surfaces and essences: Analogy as the fuel and fire of thinking","author":"Hofstadter Douglas R","unstructured":"Douglas R Hofstadter and Emmanuel Sander. 2013. Surfaces and essences: Analogy as the fuel and fire of thinking. Basic Books."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-59050-9_20"},{"key":"e_1_2_1_34_1","unstructured":"Ece Kamar. 2016. Directions in Hybrid Intelligence: Complementing AI Systems with Human Intelligence.. In IJCAI. 4070--4073."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1539-6924.2009.01261.x"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.31234\/osf.io\/nfc45"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2014.02.137"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1037\/0022-3514.77.6.1121"},{"key":"e_1_2_1_39_1","volume-title":"Trust in automation: Designing for appropriate reliance. Human factors","author":"Lee John D","year":"2004","unstructured":"John D Lee and Katrina A See. 2004. Trust in automation: Designing for appropriate reliance. Human factors, Vol. 46, 1 (2004), 50--80."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.obhdp.2018.12.005"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445562"},{"key":"e_1_2_1_42_1","article-title":"?It's Like? You Know\": The Use of Analogies and Heuristics in Teaching Introductory Statistical Methods","volume":"11","author":"Martin Michael A","year":"2003","unstructured":"Michael A Martin. 2003. ?It's Like? You Know\": The Use of Analogies and Heuristics in Teaching Introductory Statistical Methods. Journal of Statistics Education, Vol. 11, 2 (2003).","journal-title":"Journal of Statistics Education"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41562-021-01101-z"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359174"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.3109\/01421599209018857"},{"key":"e_1_2_1_46_1","volume-title":"How do humans understand explanations from machine learning systems? an evaluation of the human-interpretability of explanation. arXiv preprint arXiv:1802.00682","author":"Narayanan Menaka","year":"2018","unstructured":"Menaka Narayanan, Emily Chen, Jeffrey He, Been Kim, Sam Gershman, and Finale Doshi-Velez. 2018. How do humans understand explanations from machine learning systems? an evaluation of the human-interpretability of explanation. arXiv preprint arXiv:1802.00682 (2018)."},{"key":"e_1_2_1_47_1","volume-title":"Likert scales, levels of measurement and the ?laws\" of statistics. Advances in health sciences education","author":"Norman Geoff","year":"2010","unstructured":"Geoff Norman. 2010. Likert scales, levels of measurement and the ?laws\" of statistics. Advances in health sciences education, Vol. 15, 5 (2010), 625--632."},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1609\/hcomp.v7i1.5284"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1609\/hcomp.v8i1.7469"},{"key":"e_1_2_1_50_1","volume-title":"Anchoring Bias Affects Mental Model Formation and User Reliance in Explainable AI Systems. In 26th International Conference on Intelligent User Interfaces. 340--350","author":"Nourani Mahsan","year":"2021","unstructured":"Mahsan Nourani, Chiradeep Roy, Jeremy E Block, Donald R Honeycutt, Tahrima Rahman, Eric Ragan, and Vibhav Gogate. 2021. Anchoring Bias Affects Mental Model Formation and User Reliance in Explainable AI Systems. In 26th International Conference on Intelligent User Interfaces. 340--350."},{"key":"e_1_2_1_51_1","volume-title":"IJCAI Workshop on Explainable Artificial Intelligence (XAI)","author":"Papenmeier Andrea","year":"2019","unstructured":"Andrea Papenmeier, Gwenn Englebienne, and Christin Seifert. 2019. How model accuracy and explanation fidelity influence user trust in AI. In IJCAI Workshop on Explainable Artificial Intelligence (XAI) 2019."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1177\/0272989X12464433"},{"key":"e_1_2_1_53_1","volume-title":"COVID vaccines protect against Delta, but their effectiveness wanes. Nature","author":"Sanderson Katharine","year":"2021","unstructured":"Katharine Sanderson. 2021. COVID vaccines protect against Delta, but their effectiveness wanes. Nature (2021)."},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3301275.3302308"},{"key":"e_1_2_1_55_1","volume-title":"Measuring Appropriate Reliance in Human-AI Decision-Making. In ACM Conference on Human Factors in Computing Systems (CHI'22)","author":"Schemmer Max","year":"2022","unstructured":"Max Schemmer, Patrick Hemmer, Niklas K\u00fchl, Carina Benz, and Gerhard Satzger. 2022. Should I Follow AI-based Advice? Measuring Appropriate Reliance in Human-AI Decision-Making. In ACM Conference on Human Factors in Computing Systems (CHI'22), Workshop on Trust and Reliance in AI-Human Teams (trAIt)."},{"key":"e_1_2_1_56_1","volume-title":"Exploring the Gap Between Perceived Trust in and Reliance on Algorithmic Advice. In International Conference on Information Systems (ICIS).","author":"Schmitt Anuschka","year":"2021","unstructured":"Anuschka Schmitt, Thiemo Wambsganss, Matthias S\u00f6llner, and Andreas Janson. 2021. Towards a Trust Reliance Paradox? Exploring the Gap Between Perceived Trust in and Reliance on Algorithmic Advice. In International Conference on Information Systems (ICIS)."},{"key":"e_1_2_1_57_1","volume-title":"The persuasive effects of metaphor: A meta-analysis. Human communication research","author":"Sopory Pradeep","year":"2002","unstructured":"Pradeep Sopory and James Price Dillard. 2002. The persuasive effects of metaphor: A meta-analysis. Human communication research, Vol. 28, 3 (2002), 382--419."},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450613.3456817"},{"key":"e_1_2_1_59_1","volume-title":"Interpretable to whom? A role-based model for analyzing interpretable machine learning systems. arXiv preprint arXiv:1806.07552","author":"Tomsett Richard","year":"2018","unstructured":"Richard Tomsett, Dave Braines, Dan Harborne, Alun Preece, and Supriyo Chakraborty. 2018. Interpretable to whom? A role-based model for analyzing interpretable machine learning systems. arXiv preprint arXiv:1806.07552 (2018)."},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445365"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1080\/10510974.2018.1457553"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300831"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1002\/bdm.2118"},{"key":"e_1_2_1_64_1","volume-title":"Proceedings of ICML2018 Workshop on Human Interpretability in Machine Learning (WHI","volume":"7","author":"Yin Ming","year":"2018","unstructured":"Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2018. Does stated accuracy affect trust in machine learning algorithms. In Proceedings of ICML2018 Workshop on Human Interpretability in Machine Learning (WHI 2018), Vol. 7."},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300509"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372852"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359158"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1177\/0272989X07303824"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3610067","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3610067","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T04:27:40Z","timestamp":1755750460000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3610067"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,28]]},"references-count":68,"journal-issue":{"issue":"CSCW2","published-print":{"date-parts":[[2023,9,28]]}},"alternative-id":["10.1145\/3610067"],"URL":"https:\/\/doi.org\/10.1145\/3610067","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,28]]},"assertion":[{"value":"2023-10-04","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}