{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T04:41:15Z","timestamp":1760157675288,"version":"build-2065373602"},"publisher-location":"Cham","reference-count":33,"publisher":"Springer Nature Switzerland","isbn-type":[{"type":"print","value":"9783032083166"},{"type":"electronic","value":"9783032083173"}],"license":[{"start":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T00:00:00Z","timestamp":1760227200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T00:00:00Z","timestamp":1760227200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>Advances in artificial intelligence (AI) have shown significant potential in supporting decision-making in high-stakes domains such as medical diagnostics, where accuracy and reliability are crucial. In this context, presenting AI-generated confidence levels has been proposed as a strategy to promote appropriate reliance on AI systems, assuming both perfect confidence calibration of the AI and optimally calibrated trust of human decision-makers. However, the impact of providing users with indications of AI confidence on decision-making remains underexplored when these ideal conditions are not met. This study examines how different ways of presenting AI support\u2014through recommendations alone, recommendations with calibrated confidence scores, and recommendations with explicit correctness feedback (as an ideally extreme baseline condition)\u2014influence diagnostic accuracy, reliance, and cognitive biases in medical students. A total of 222 participants completed an image-based diagnostic task with a misaligned mental model of AI behavior (a kind of \u2018theory of mind\u2019), reflecting a knowledge mismatch induced by instructing the participants to consider two diagnostic criteria while ignoring a third one that the AI system correctly applied. Results showed that, unsurprisingly, providing correctness feedback led to the most significant improvement in appropriate reliance, outperforming both the confidence and advice-only conditions, and proving to be an optimal strategy to reduce knowledge mismatches between humans and machines. More interestingly, we found that providing confidence levels can result in significantly worse reliance and more conservatism bias than not providing them when such a knowledge mismatch exists. Therefore, when human and AI knowledge cannot be assumed to be aligned, confidence levels should be presented with caution, even if their calibration is assured. This study provides insights into the design of hybrid intelligence systems that enhance diagnostic decision-making and supports the integration of AI into critical domains such as healthcare.<\/jats:p>","DOI":"10.1007\/978-3-032-08317-3_11","type":"book-chapter","created":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T03:36:29Z","timestamp":1760153789000},"page":"233-254","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Too Sure for\u00a0Trust. The Paradoxical Effect of\u00a0Calibrated Confidence in\u00a0Case of\u00a0Uncalibrated Trust in\u00a0Hybrid Decision Making"],"prefix":"10.1007","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4065-3415","authenticated-orcid":false,"given":"Federico","family":"Cabitza","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-7626-8131","authenticated-orcid":false,"given":"Caterina","family":"Fregosi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2769-5028","authenticated-orcid":false,"given":"Lucia","family":"Vicente","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2025,10,12]]},"reference":[{"key":"11_CR1","doi-asserted-by":"publisher","unstructured":"Antifakos, S., Kern, N., Schiele, B., Schwaninger, A.: Towards improving trust in context-aware systems by displaying system confidence. In: Proceedings of the 7th International Conference on Human Computer Iteraction with Mobile Devices & Services, pp. 9\u201314 (2005). https:\/\/doi.org\/10.1145\/1085777.1085780","DOI":"10.1145\/1085777.1085780"},{"issue":"12","key":"11_CR2","doi-asserted-by":"publisher","first-page":"2996","DOI":"10.1038\/s41591-023-02562-7","volume":"29","author":"CR Banerji","year":"2023","unstructured":"Banerji, C.R., Chakraborti, T., Harbron, C., MacArthur, B.D.: Clinical ai tools must convey predictive uncertainty for each individual patient. Nat. Med. 29(12), 2996\u20132998 (2023). https:\/\/doi.org\/10.1038\/s41591-023-02562-7","journal-title":"Nat. Med."},{"key":"11_CR3","doi-asserted-by":"publisher","unstructured":"Bansal, G., Nushi, B., Kamar, E., Lasecki, W.S., Weld, D.S., Horvitz, E.: Beyond accuracy: The role of mental models in human-ai team performance. In: Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, vol.\u00a07, pp. 2\u201311 (2019). https:\/\/doi.org\/10.1609\/hcomp.v7i1.5285","DOI":"10.1609\/hcomp.v7i1.5285"},{"key":"11_CR4","doi-asserted-by":"publisher","unstructured":"Cabitza, F., Campagner, A., Angius, R., Natali, C., Reverberi, C.: Ai shall have no dominion: on how to measure technology dominance in ai-supported human decision-making. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pp. 1\u201320 (2023). https:\/\/doi.org\/10.1145\/3544548.3581095","DOI":"10.1145\/3544548.3581095"},{"key":"11_CR5","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2022.118888","volume":"213","author":"F Cabitza","year":"2023","unstructured":"Cabitza, F., Campagner, A., Malgieri, G., Natali, C., Schneeberger, D., Stoeger, K., Holzinger, A.: Quod erat demonstrandum?-towards a typology of the concept of explanation for the design of explainable ai. Expert Syst. Appl. 213, 118888 (2023). https:\/\/doi.org\/10.1016\/j.eswa.2022.118888","journal-title":"Expert Syst. Appl."},{"issue":"1","key":"11_CR6","doi-asserted-by":"publisher","first-page":"269","DOI":"10.3390\/make5010017","volume":"5","author":"F Cabitza","year":"2023","unstructured":"Cabitza, F., Campagner, A., Natali, C., Parimbelli, E., Ronzio, L., Cameli, M.: Painting the black box white: Experimental findings from applying xai to an ecg reading setting. Mach. Learn. Knowl. Extraction 5(1), 269\u2013286 (2023). https:\/\/doi.org\/10.3390\/make5010017","journal-title":"Mach. Learn. Knowl. Extraction"},{"key":"11_CR7","doi-asserted-by":"publisher","DOI":"10.1016\/j.artmed.2023.102506","volume":"138","author":"F Cabitza","year":"2023","unstructured":"Cabitza, F., et al.: Rams, hounds and white boxes: investigating human-ai collaboration protocols in medical diagnosis. Artif. Intell. Med. 138, 102506 (2023). https:\/\/doi.org\/10.1016\/j.artmed.2023.102506","journal-title":"Artif. Intell. Med."},{"key":"11_CR8","doi-asserted-by":"publisher","unstructured":"Cao, S., Liu, A., Huang, C.M.: Designing for appropriate reliance: The roles of ai uncertainty presentation, initial user decision, and user demographics in ai-assisted decision-making. Proceedings of the ACM on Human-Computer Interaction 8(CSCW1), pp. 1\u201332 (2024). https:\/\/doi.org\/10.1145\/3637318","DOI":"10.1145\/3637318"},{"key":"11_CR9","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2021.107018","volume":"127","author":"L Chong","year":"2022","unstructured":"Chong, L., Zhang, G., Goucher-Lambert, K., Kotovsky, K., Cagan, J.: Human confidence in artificial intelligence and in themselves: the evolution and impact of confidence on adoption of ai advice. Comput. Hum. Behav. 127, 107018 (2022). https:\/\/doi.org\/10.1016\/j.chb.2021.107018","journal-title":"Comput. Hum. Behav."},{"key":"11_CR10","doi-asserted-by":"publisher","unstructured":"Famiglini, L., Campagner, A., Cabitza, F.: Towards a rigorous calibration assessment framework: advancements in metrics, methods, and use. In: ECAI 2023, pp. 645\u2013652. IOS Press (2023). https:\/\/doi.org\/10.3233\/FAIA230327","DOI":"10.3233\/FAIA230327"},{"key":"11_CR11","doi-asserted-by":"publisher","unstructured":"Green, B., Chen, Y.: The principles and limits of algorithm-in-the-loop decision making. Proc. ACM Hum.-Comput. Interact. 3(CSCW) (Nov 2019). https:\/\/doi.org\/10.1145\/3359152","DOI":"10.1145\/3359152"},{"key":"11_CR12","unstructured":"Guo, C., Pleiss, G., Sun, Y., Weinberger, K.Q.: On calibration of modern neural networks. In: Proceedings of the 34th International Conference on Machine Learning - Volume 70, pp. 1321\u20131330. ICML\u201917 (2017)"},{"key":"11_CR13","doi-asserted-by":"publisher","unstructured":"Helldin, T., Falkman, G., Riveiro, M., Davidsson, S.: Presenting system uncertainty in automotive uis for supporting trust calibration in autonomous driving. In: Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, pp. 210\u2013217. AutomotiveUI \u201913. Association for Computing Machinery, New York (2013). https:\/\/doi.org\/10.1145\/2516540.2516554","DOI":"10.1145\/2516540.2516554"},{"issue":"1683","key":"11_CR14","doi-asserted-by":"publisher","first-page":"20150013","DOI":"10.1098\/rstb.2015.0013","volume":"370","author":"J Henrich","year":"2015","unstructured":"Henrich, J., Chudek, M., Boyd, R.: The big man mechanism: how prestige fosters cooperation and creates prosocial leaders. Philosophical Trans. Roy. Soc. B: Biological Sci. 370(1683), 20150013 (2015)","journal-title":"Philosophical Trans. Roy. Soc. B: Biological Sci."},{"issue":"3","key":"11_CR15","doi-asserted-by":"publisher","first-page":"165","DOI":"10.1016\/S1090-5138(00)00071-4","volume":"22","author":"J Henrich","year":"2001","unstructured":"Henrich, J., Gil-White, F.J.: The evolution of prestige: Freely conferred deference as a mechanism for enhancing the benefits of cultural transmission. Evol. Hum. Behav. 22(3), 165\u2013196 (2001)","journal-title":"Evol. Hum. Behav."},{"key":"11_CR16","doi-asserted-by":"publisher","unstructured":"Ishizu, N., Yeoh, W.L., Okumura, H., Fukuda, O.: The effect of communicating ai confidence on human decision making when performing a binary decision task. Appl. Sci. 14(16) (2024). https:\/\/doi.org\/10.3390\/app14167192","DOI":"10.3390\/app14167192"},{"key":"11_CR17","doi-asserted-by":"publisher","unstructured":"Li, J., Yang, Y., Liao, Q.V., Zhang, J., Lee, Y.C.: As confidence aligns: Exploring the effect of ai confidence on human self-confidence in human-ai decision making. arXiv preprint (2025). https:\/\/doi.org\/10.48550\/arXiv.2501.12868","DOI":"10.48550\/arXiv.2501.12868"},{"key":"11_CR18","doi-asserted-by":"crossref","unstructured":"Logg, J.: Theory of machine: When do people rely on algorithms. Harvard Business School working paper series (17-086) (2017)","DOI":"10.2139\/ssrn.2941774"},{"key":"11_CR19","doi-asserted-by":"publisher","unstructured":"Ma, S., Wang, X., Lei, Y., Shi, C., Yin, M., Ma, X.: \u201care you really sure?\u201d understanding the effects of human self-confidence calibration in ai-assisted decision making. In: Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. CHI \u201924. Association for Computing Machinery, New York (2024). https:\/\/doi.org\/10.1145\/3613904.3642671","DOI":"10.1145\/3613904.3642671"},{"issue":"1","key":"11_CR20","doi-asserted-by":"publisher","first-page":"34","DOI":"10.1177\/0018720814561675","volume":"57","author":"SM Merritt","year":"2015","unstructured":"Merritt, S.M., Lee, D., Unnerstall, J.L., Huber, K.: Are well-calibrated users effective users? associations between calibration of trust and performance on an automation-aided task. Hum. Factors 57(1), 34\u201347 (2015). https:\/\/doi.org\/10.1177\/0018720814561675","journal-title":"Hum. Factors"},{"key":"11_CR21","doi-asserted-by":"publisher","first-page":"8","DOI":"10.1016\/j.jclinepi.2022.11.015","volume":"154","author":"CLA Navarro","year":"2023","unstructured":"Navarro, C.L.A., et al.: Systematic review identifies the design and methodological conduct of studies on machine learning-based prediction models. J. Clin. Epidemiol. 154, 8\u201322 (2023). https:\/\/doi.org\/10.1016\/j.jclinepi.2022.11.015","journal-title":"J. Clin. Epidemiol."},{"key":"11_CR22","doi-asserted-by":"publisher","unstructured":"Niculescu-Mizil, A., Caruana, R.: Predicting good probabilities with supervised learning. In: Proceedings of the 22nd International Conference on Machine Learning, pp. 625\u2013632. ICML \u201905. Association for Computing Machinery, New York (2005). https:\/\/doi.org\/10.1145\/1102351.1102430","DOI":"10.1145\/1102351.1102430"},{"key":"11_CR23","doi-asserted-by":"publisher","unstructured":"Okamura, K., Yamada, S.: Adaptive trust calibration for human-ai collaboration. PLOS ONE 15(2), 1\u201320 (2020). https:\/\/doi.org\/10.1371\/journal.pone.0229132","DOI":"10.1371\/journal.pone.0229132"},{"key":"11_CR24","doi-asserted-by":"crossref","unstructured":"Rezaeian, O., Bayrak, A.E., Asan, O.: Explainability and ai confidence in clinical decision support systems: Effects on trust, diagnostic performance, and cognitive load in breast cancer care (2025). https:\/\/arxiv.org\/abs\/2501.16693","DOI":"10.1080\/10447318.2025.2539458"},{"issue":"2","key":"11_CR25","doi-asserted-by":"publisher","first-page":"246","DOI":"10.1016\/j.obhdp.2013.02.001","volume":"121","author":"S Sah","year":"2013","unstructured":"Sah, S., Moore, D.A., MacCoun, R.J.: Cheap talk and credibility: the consequences of confidence and accuracy on advisor credibility and persuasiveness. Organ. Behav. Hum. Decis. Process. 121(2), 246\u2013255 (2013). https:\/\/doi.org\/10.1016\/j.obhdp.2013.02.001","journal-title":"Organ. Behav. Hum. Decis. Process."},{"key":"11_CR26","doi-asserted-by":"publisher","unstructured":"Schemmer, M., Kuehl, N., Benz, C., Bartos, A., Satzger, G.: Appropriate reliance on ai advice: Conceptualization and the effect of explanations. In: IUI: Proceedings of the 28th International Conference on Intelligent User Interfaces, pp. 410\u2013422 (2023). https:\/\/doi.org\/10.1145\/3581641.3584066","DOI":"10.1145\/3581641.3584066"},{"issue":"4","key":"11_CR27","doi-asserted-by":"publisher","first-page":"491","DOI":"10.1007\/s42113-022-00157-y","volume":"5","author":"H Tejeda","year":"2022","unstructured":"Tejeda, H., Kumar, A., Smyth, P., Steyvers, M.: Ai-assisted decision-making: a cognitive modeling approach to infer latent reliance strategies. Comput. Brain Behav. 5(4), 491\u2013508 (2022). https:\/\/doi.org\/10.1007\/s42113-022-00157-y","journal-title":"Comput. Brain Behav."},{"key":"11_CR28","doi-asserted-by":"publisher","unstructured":"Vaccaro, M., Almaatouq, A., Malone, T.: When combinations of humans and ai are useful: a systematic review and meta-analysis. Nature Human Behaviour, pp. 1\u201311 (2024). https:\/\/doi.org\/10.1038\/s41562-024-02024-1","DOI":"10.1038\/s41562-024-02024-1"},{"key":"11_CR29","doi-asserted-by":"publisher","unstructured":"Van\u00a0Calster, B., McLernon, D.J., Van\u00a0Smeden, M., Wynants, L., Steyerberg, E.W., diagnostic tests, T.G. prediction models\u2019 of\u00a0the STRATOS\u00a0initiative: Calibration: the achilles heel of predictive analytics. BMC Med. 17(1), 230 (2019). https:\/\/doi.org\/10.1186\/s12916-019-1466-7","DOI":"10.1186\/s12916-019-1466-7"},{"key":"11_CR30","unstructured":"Vodrahalli, K., Gerstenberg, T., Zou, J.: Uncalibrated Models Can Improve Human-AI Collaboration. Advances in Neural Information Processing Systems 35(NeurIPS) (2022)"},{"key":"11_CR31","doi-asserted-by":"crossref","unstructured":"Wellman, H.M.: Making minds: How theory of mind develops. Oxford University Press (2014)","DOI":"10.1093\/acprof:oso\/9780199334919.001.0001"},{"key":"11_CR32","doi-asserted-by":"publisher","unstructured":"Zhang, Q.s., Zhu, S.C.: Visual interpretability for deep learning: a survey. Front. Inf. Technol. Electron. Eng. 19(1), 27\u201339 (2018). https:\/\/doi.org\/10.1631\/FITEE.1700808","DOI":"10.1631\/FITEE.1700808"},{"key":"11_CR33","doi-asserted-by":"publisher","unstructured":"Zhang, Y., Liao, Q.V., Bellamy, R.K.: Effect of confidence and explanation on accuracy and trust calibration in ai-assisted decision making. In: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pp. 295\u2013305 (2020). https:\/\/doi.org\/10.1145\/3351095.3372852","DOI":"10.1145\/3351095.3372852"}],"container-title":["Communications in Computer and Information Science","Explainable Artificial Intelligence"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/978-3-032-08317-3_11","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T04:03:39Z","timestamp":1760155419000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/978-3-032-08317-3_11"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,12]]},"ISBN":["9783032083166","9783032083173"],"references-count":33,"URL":"https:\/\/doi.org\/10.1007\/978-3-032-08317-3_11","relation":{},"ISSN":["1865-0929","1865-0937"],"issn-type":[{"type":"print","value":"1865-0929"},{"type":"electronic","value":"1865-0937"}],"subject":[],"published":{"date-parts":[[2025,10,12]]},"assertion":[{"value":"12 October 2025","order":1,"name":"first_online","label":"First Online","group":{"name":"ChapterHistory","label":"Chapter History"}},{"value":"xAI","order":1,"name":"conference_acronym","label":"Conference Acronym","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"World Conference on Explainable Artificial Intelligence","order":2,"name":"conference_name","label":"Conference Name","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Istanbul","order":3,"name":"conference_city","label":"Conference City","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"T\u00fcrkiye","order":4,"name":"conference_country","label":"Conference Country","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"2025","order":5,"name":"conference_year","label":"Conference Year","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"9 July 2025","order":7,"name":"conference_start_date","label":"Conference Start Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"11 July 2025","order":8,"name":"conference_end_date","label":"Conference End Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"3","order":9,"name":"conference_number","label":"Conference Number","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"xai2025","order":10,"name":"conference_id","label":"Conference ID","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"https:\/\/xaiworldconference.com\/2025\/","order":11,"name":"conference_url","label":"Conference URL","group":{"name":"ConferenceInfo","label":"Conference Information"}}]}}