{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,10]],"date-time":"2026-04-10T16:18:57Z","timestamp":1775837937133,"version":"3.50.1"},"reference-count":84,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2025,10,28]],"date-time":"2025-10-28T00:00:00Z","timestamp":1761609600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Informatics"],"abstract":"<jats:p>While Artificial Intelligence (AI) promises significant enhancements for Clinical Decision Support Systems (CDSSs), the opacity of many AI models remains a major barrier to clinical adoption, primarily due to interpretability and trust challenges. Explainable AI (XAI) seeks to bridge this gap by making model reasoning understandable to clinicians, but technical XAI solutions have too often failed to address real-world clinician needs, workflow integration, and usability concerns. This study synthesizes persistent challenges in applying XAI to CDSS\u2014including mismatched explanation methods, suboptimal interface designs, and insufficient evaluation practices\u2014and proposes a structured, user-centered framework to guide more effective and trustworthy XAI-CDSS development. Drawing on a comprehensive literature review, we detail a three-phase framework encompassing user-centered XAI method selection, interface co-design, and iterative evaluation and refinement. We demonstrate its application through a retrospective case study analysis of a published XAI-CDSS for sepsis care. Our synthesis highlights the importance of aligning XAI with clinical workflows, supporting calibrated trust, and deploying robust evaluation methodologies that capture real-world clinician\u2013AI interaction patterns, such as negotiation. The case analysis shows how the framework can systematically identify and address user-centric gaps, leading to better workflow integration, tailored explanations, and more usable interfaces. We conclude that achieving trustworthy and clinically useful XAI-CDSS requires a fundamentally user-centered approach; our framework offers actionable guidance for creating explainable, usable, and trusted AI systems in healthcare.<\/jats:p>","DOI":"10.3390\/informatics12040119","type":"journal-article","created":{"date-parts":[[2025,10,28]],"date-time":"2025-10-28T14:29:34Z","timestamp":1761661774000},"page":"119","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Explainable AI for Clinical Decision Support Systems: Literature Review, Key Gaps, and Research Synthesis"],"prefix":"10.3390","volume":"12","author":[{"given":"Mozhgan","family":"Salimparsa","sequence":"first","affiliation":[{"name":"Insight Lab, Department of Computer Science, Western University, London, ON N6A 3K7, Canada"}]},{"given":"Kamran","family":"Sedig","sequence":"additional","affiliation":[{"name":"Insight Lab, Department of Computer Science, Western University, London, ON N6A 3K7, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9258-8619","authenticated-orcid":false,"given":"Daniel J.","family":"Lizotte","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Faculty of Science, Western University, London, ON N6A 3K7, Canada"},{"name":"Department of Epidemiology & Biostatistics, Western University, London, ON N6G 2M1, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2452-8494","authenticated-orcid":false,"given":"Sheikh S.","family":"Abdullah","sequence":"additional","affiliation":[{"name":"Insight Lab, Department of Computer Science, Western University, London, ON N6A 3K7, Canada"},{"name":"Department of Computer Science, MacEwan University, Edmonton, AB T5J 2P2, Canada"},{"name":"London Health Sciences Centre Research Institute, London, ON N6A 5W9, Canada"},{"name":"ICES Western, London, ON N6A 5W9, Canada"},{"name":"Lawson Health Research Institute, London Health Sciences Centre, London, ON N6A 4V2, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7228-4725","authenticated-orcid":false,"given":"Niaz","family":"Chalabianloo","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Faculty of Science, Western University, London, ON N6A 3K7, Canada"},{"name":"ICES Western, London, ON N6A 5W9, Canada"},{"name":"Department of Physiology and Pharmacology, Western University, London, ON N6A 5C1, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4682-6564","authenticated-orcid":false,"given":"Flory T.","family":"Muanda","sequence":"additional","affiliation":[{"name":"Department of Epidemiology & Biostatistics, Western University, London, ON N6G 2M1, Canada"},{"name":"ICES Western, London, ON N6A 5W9, Canada"},{"name":"Lawson Health Research Institute, London Health Sciences Centre, London, ON N6A 4V2, Canada"},{"name":"Department of Physiology and Pharmacology, Western University, London, ON N6A 5C1, Canada"}]}],"member":"1968","published-online":{"date-parts":[[2025,10,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Berner, E.S., and La Lande, T.J. (2016). Overview of clinical decision support systems. Clinical Decision Support Systems: Theory and Practice, Springer.","DOI":"10.1007\/978-3-319-31913-1"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Kubben, P., Dumontier, M., and Dekker, A. (2019). Fundamentals of Clinical Data Science, Springer.","DOI":"10.1007\/978-3-319-99713-1"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"116","DOI":"10.1109\/RBME.2020.3007816","article-title":"Machine learning for clinical outcome prediction","volume":"14","author":"Shamout","year":"2020","journal-title":"IEEE Rev. Biomed. Eng."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1055\/s-0039-1677911","article-title":"Artificial intelligence in clinical decision support: A focused literature survey","volume":"28","author":"Montani","year":"2019","journal-title":"Yearb. Med. Inform."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Gilpin, L.H., Bau, D., Yuan, B.Z., Bajwa, A., Specter, M., and Kagal, L. (2018, January 1\u20133). Explaining explanations: An overview of interpretability of machine learning. Proceedings of the 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA), Turin, Italy.","DOI":"10.1109\/DSAA.2018.00018"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"206","DOI":"10.1038\/s42256-019-0048-x","article-title":"Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead","volume":"1","author":"Rudin","year":"2019","journal-title":"Nat. Mach. Intell."},{"key":"ref_7","unstructured":"Tonekaboni, S., Joshi, S., McCradden, M.D., and Goldenberg, A. (2019, January 9\u201310). What clinicians want: Contextualizing explainable machine learning for clinical end use. Proceedings of the Machine Learning for Healthcare Conference, Ann Arbor, MI, USA."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/j.inffus.2019.12.012","article-title":"Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI","volume":"58","author":"Arrieta","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"4793","DOI":"10.1109\/TNNLS.2020.3027314","article-title":"A survey on explainable artificial intelligence (XAI): Toward medical XAI","volume":"32","author":"Tjoa","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_10","first-page":"143","article-title":"Working in contexts for which transparency is important: A recordkeeping view of explainable artificial intelligence (XAI)","volume":"30","author":"Bunn","year":"2020","journal-title":"Rec. Manag. J."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"52138","DOI":"10.1109\/ACCESS.2018.2870052","article-title":"Peeking inside the black-box: A survey on explainable artificial intelligence (XAI)","volume":"6","author":"Adadi","year":"2018","journal-title":"IEEE Access"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3236009","article-title":"A survey of methods for explaining black box models","volume":"51","author":"Guidotti","year":"2018","journal-title":"Acm Comput. Surv. (CSUR)"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Tan, S., Caruana, R., Hooker, G., and Lou, Y. (2018, January 2\u20133). Distill\u2013and\u2013compare: Auditing black-box models using transparent model distillation. Proceedings of the 2018 AAAI\/ACM Conference on AI, Ethics, and Society, New Orleans, LA, USA.","DOI":"10.1145\/3278721.3278725"},{"key":"ref_14","unstructured":"Xu, K., Park, D.H., Yi, C., and Sutton, C. (2018). Interpreting deep classifier by visual distillation of dark knowledge. arXiv."},{"key":"ref_15","unstructured":"Che, Z., Purushotham, S., Khemani, R., and Liu, Y. (2015). Distilling knowledge from deep networks with applications to healthcare domain. arXiv."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1057","DOI":"10.1109\/72.728352","article-title":"The truth will come to light: Directions and challenges in extracting the knowledge embedded within trained artificial neural networks","volume":"9","author":"Tickle","year":"1998","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"645","DOI":"10.1007\/s00500-011-0762-8","article-title":"Rule extraction algorithm from support vector machines and its application to credit screening","volume":"16","author":"Su","year":"2012","journal-title":"Soft Comput."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2664","DOI":"10.1109\/TNNLS.2015.2389037","article-title":"Active learning-based pedagogical rule extraction","volume":"26","author":"Martens","year":"2015","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Bologna, G., and Hayashi, Y. (2018). A rule extraction study from svm on sentiment analysis. Big Data Cogn. Comput., 2.","DOI":"10.3390\/bdcc2010006"},{"key":"ref_20","unstructured":"Hailesilassie, T. (2016). Rule extraction algorithm for deep neural networks: A review. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Ribeiro, M.T., Singh, S., and Guestrin, C. (2016, January 13\u201317). \u201cWhy should i trust you?\u201d Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939778"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Magesh, P.R., Myloth, R.D., and Tom, R.J. (2020). An explainable machine learning model for early detection of Parkinson\u2019s disease using LIME on DaTSCAN imagery. Comput. Biol. Med., 126.","DOI":"10.1016\/j.compbiomed.2020.104041"},{"key":"ref_23","unstructured":"Zhang, Y., and Wallace, B. (2015). A sensitivity analysis of (and practitioners\u2019 guide to) convolutional neural networks for sentence classification. arXiv."},{"key":"ref_24","unstructured":"Hooker, S., Erhan, D., Kindermans, P.J., and Kim, B. (2019). A benchmark for interpretability methods in deep neural networks. Adv. Neural Inf. Process. Syst., 32."},{"key":"ref_25","unstructured":"Hara, S., Ikeno, K., Soma, T., and Maehara, T. (2018). Maximally invariant data perturbation as explanation. arXiv."},{"key":"ref_26","unstructured":"Lundberg, S.M., and Lee, S.I. (2017). A unified approach to interpreting model predictions. Adv. Neural Inf. Process. Syst., 30."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"647","DOI":"10.1007\/s10115-013-0679-x","article-title":"Explaining prediction models and individual predictions with feature contributions","volume":"41","author":"Kononenko","year":"2014","journal-title":"Knowl. Inf. Syst."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Fong, R.C., and Vedaldi, A. (2017, January 22\u201329). Interpretable explanations of black boxes by meaningful perturbation. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.371"},{"key":"ref_29","unstructured":"Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhudinov, R., Zemel, R., and Bengio, Y. (2015, January 6\u201311). Show, attend and tell: Neural image caption generation with visual attention. Proceedings of the International Conference on Machine Learning, Lille, France."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.dsp.2017.10.011","article-title":"Methods for interpreting and understanding deep neural networks","volume":"73","author":"Montavon","year":"2018","journal-title":"Digit. Signal Process."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Bach, S., Binder, A., Montavon, G., Klauschen, F., M\u00fcller, K.R., and Samek, W. (2015). On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS ONE, 10.","DOI":"10.1371\/journal.pone.0130140"},{"key":"ref_32","unstructured":"Samek, W., Montavon, G., Binder, A., Lapuschkin, S., and M\u00fcller, K.R. (2016). Interpreting the predictions of complex ML models by layer-wise relevance propagation. arXiv."},{"key":"ref_33","unstructured":"Nguyen, A., Dosovitskiy, A., Yosinski, J., Brox, T., and Clune, J. (2016). Synthesizing the preferred inputs for neurons in neural networks via deep generator networks. Adv. Neural Inf. Process. Syst., 29."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Bien, J., and Tibshirani, R. (2011). Prototype selection for interpretable classification. arXiv.","DOI":"10.1214\/11-AOAS495"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Sharma, S., Henderson, J., and Ghosh, J. (2019). Certifai: Counterfactual explanations for robustness, transparency, interpretability, and fairness of artificial intelligence models. arXiv.","DOI":"10.1145\/3375627.3375812"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Mothilal, R.K., Sharma, A., and Tan, C. (2020, January 27\u201330). Explaining machine learning classifiers through diverse counterfactual explanations. Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, Barcelona, Spain.","DOI":"10.1145\/3351095.3372850"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"2805","DOI":"10.1109\/TNNLS.2018.2886017","article-title":"Adversarial examples: Attacks and defenses for deep learning","volume":"30","author":"Yuan","year":"2019","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1016\/j.artmed.2019.01.001","article-title":"Explainable artificial intelligence for breast cancer: A visual case-based reasoning approach","volume":"94","author":"Lamy","year":"2019","journal-title":"Artif. Intell. Med."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"84","DOI":"10.17705\/1thci.00055","article-title":"Interaction design for complex cognitive activities with visual representations: A pattern-based approach","volume":"5","author":"Sedig","year":"2013","journal-title":"Ais Trans. Hum. Comput. Interact."},{"key":"ref_40","first-page":"180","article-title":"Aligning information technologies with evidence-based health-care activities: A design and evaluation framework","volume":"13","author":"Sedig","year":"2017","journal-title":"Hum. Technol. Interdiscip. J. Humans Ict Environ."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Parsons, P., and Sedig, K. (2013). Distribution of information processing while performing complex cognitive activities with visualization tools. Handbook of Human Centric Visualization, Springer.","DOI":"10.1007\/978-1-4614-7485-2_28"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Lamy, J.B., Sedki, K., and Tsopra, R. (2020). Explainable decision support through the learning and visualization of preferences from a formal ontology of antibiotic treatments. J. Biomed. Inform., 104.","DOI":"10.1016\/j.jbi.2020.103407"},{"key":"ref_43","unstructured":"Choi, E., Bahadori, M.T., Sun, J., Kulas, J., Schuetz, A., and Stewart, W. (2016). Retain: An interpretable predictive model for healthcare using reverse time attention mechanism. Adv. Neural Inf. Process. Syst., 29."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"101812","DOI":"10.1016\/j.artmed.2020.101812","article-title":"An incremental explanation of inference in Bayesian networks for increasing model trustworthiness and supporting clinical decision making","volume":"103","author":"Kyrimi","year":"2020","journal-title":"Artif. Intell. Med."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"342","DOI":"10.1109\/TVCG.2018.2864812","article-title":"Rulematrix: Visualizing and understanding classifiers with rules","volume":"25","author":"Ming","year":"2018","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_46","unstructured":"Che, Z., Purushotham, S., Khemani, R., and Liu, Y. (2017, January 4\u20138). Interpretable deep models for ICU outcome prediction. Proceedings of the AMIA Annual Symposium Proceedings, Washington, DC, USA."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Yang, Y., Tresp, V., Wunderle, M., and Fasching, P.A. (2018, January 4\u20137). Explaining therapy predictions with layer-wise relevance propagation in neural networks. Proceedings of the 2018 IEEE International Conference on Healthcare Informatics (ICHI), New York, NY, USA.","DOI":"10.1109\/ICHI.2018.00025"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"B\u00f6hle, M., Eitel, F., Weygandt, M., and Ritter, K. (2019). Layer-wise relevance propagation for explaining deep neural network decisions in MRI-based Alzheimer\u2019s disease classification. Front. Aging Neurosci., 11.","DOI":"10.3389\/fnagi.2019.00194"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Slijepcevic, D., Horst, F., Lapuschkin, S., Raberger, A.M., Zeppelzauer, M., Samek, W., Breiteneder, C., Sch\u00f6llhorn, W.I., and Horsak, B. (2020). On the explanation of machine learning predictions in clinical gait analysis. arXiv.","DOI":"10.1145\/3474121"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"\u0160ajnovi\u0107, U., Vo\u0161ner, H.B., Zavr\u0161nik, J., \u017dlahti\u010d, B., and Kokol, P. (2024). Internet of things and big data analytics in preventive healthcare: A synthetic review. Electronics, 13.","DOI":"10.20944\/preprints202408.0445.v1"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"e55766","DOI":"10.2196\/55766","article-title":"Health Care Professionals\u2019 Experience of Using AI: Systematic Review With Narrative Synthesis","volume":"26","author":"Ayorinde","year":"2024","journal-title":"J. Med. Internet Res."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Amann, J., Vetter, D., Blomberg, S.N., Christensen, H.C., Coffee, M., Gerke, S., Gilbert, T.K., Hagendorff, T., Holm, S., and Livne, M. (2022). To explain or not to explain? Artificial intelligence explainability in clinical decision support systems. PLoS Digit. Health, 1.","DOI":"10.1371\/journal.pdig.0000016"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Pierce, R.L., Van Biesen, W., Van Cauwenberge, D., Decruyenaere, J., and Sterckx, S. (2022). Explainability in medicine in an era of AI-based clinical decision support systems. Front. Genet., 13.","DOI":"10.3389\/fgene.2022.903600"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Kim, S.Y., Kim, D.H., Kim, M.J., Ko, H.J., and Jeong, O.R. (2024). XAI-Based Clinical Decision Support Systems: A Systematic Review. Appl. Sci., 14.","DOI":"10.20944\/preprints202406.0721.v1"},{"key":"ref_55","unstructured":"Aziz, N.A., Manzoor, A., Mazhar Qureshi, M.D., Qureshi, M.A., and Rashwan, W. (2024). Explainable AI in Healthcare: Systematic Review of Clinical Decision Support Systems. medRxiv."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3587271","article-title":"Co-design of human-centered, explainable AI for clinical decision support","volume":"13","author":"Panigutti","year":"2023","journal-title":"ACM Trans. Interact. Intell. Syst."},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Turri, V., Morrison, K., Robinson, K.M., Abidi, C., Perer, A., Forlizzi, J., and Dzombak, R. (2024, January 3\u20136). Transparency in the Wild: Navigating Transparency in a Deployed AI System to Broaden Need-Finding Approaches. Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, Rio de Janeiro, Brazil.","DOI":"10.1145\/3630106.3658985"},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Micocci, M., Borsci, S., Thakerar, V., Walne, S., Manshadi, Y., Edridge, F., Mullarkey, D., Buckle, P., and Hanna, G.B. (2021). Attitudes towards trusting artificial intelligence insights and factors to prevent the passive adherence of GPs: A pilot study. J. Clin. Med., 10.","DOI":"10.3390\/jcm10143101"},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"e53207","DOI":"10.2196\/53207","article-title":"How Explainable Artificial Intelligence Can Increase or Decrease Clinicians\u2019 Trust in AI Applications in Health Care: Systematic Review","volume":"3","author":"Rosenbacke","year":"2024","journal-title":"JMIR AI"},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1038\/s43856-024-00568-x","article-title":"Explainable AI decision support improves accuracy during telehealth strep throat screening","volume":"4","author":"Gomez","year":"2024","journal-title":"Commun. Med."},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Laxar, D., Eitenberger, M., Maleczek, M., Kaider, A., Hammerle, F.P., and Kimberger, O. (2023). The influence of explainable vs non-explainable clinical decision support systems on rapid triage decisions: A mixed methods study. BMC Med., 21.","DOI":"10.1186\/s12916-023-03068-2"},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Sivaraman, V., Bukowski, L.A., Levin, J., Kahn, J.M., and Perer, A. (2023, January 23\u201328). Ignore, trust, or negotiate: Understanding clinician acceptance of AI-based treatment recommendations in health care. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, Hamburg, Germany.","DOI":"10.1145\/3544548.3581075"},{"key":"ref_63","unstructured":"Sivaraman, V., Morrison, K., Epperson, W., and Perer, A. (2025). Over-Relying on Reliance: Towards Realistic Evaluations of AI-Based Clinical Decision Support. arXiv."},{"key":"ref_64","first-page":"1","article-title":"Eye into AI: Evaluating the Interpretability of Explainable AI Techniques through a Game with a Purpose","volume":"7","author":"Morrison","year":"2023","journal-title":"Proc. ACM Hum. Comput. Interact."},{"key":"ref_65","first-page":"1","article-title":"Evaluating the impact of human explanation strategies on human-AI visual decision-making","volume":"7","author":"Morrison","year":"2023","journal-title":"Proc. ACM Hum. Comput. Interact."},{"key":"ref_66","unstructured":"Katuwal, G.J., and Chen, R. (2016). Machine learning model interpretability for precision medicine. arXiv."},{"key":"ref_67","doi-asserted-by":"crossref","unstructured":"Giordano, C., Brennan, M., Mohamed, B., Rashidi, P., Modave, F., and Tighe, P. (2021). Accessing artificial intelligence for clinical decision-making. Front. Digit. Health, 3.","DOI":"10.3389\/fdgth.2021.645232"},{"key":"ref_68","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1109\/TVCG.2018.2865027","article-title":"Retainvis: Visual analytics with interpretable and interactive recurrent neural networks on electronic medical records","volume":"25","author":"Kwon","year":"2018","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_69","doi-asserted-by":"crossref","unstructured":"Cabrera, \u00c1.A., Fu, E., Bertucci, D., Holstein, K., Talwalkar, A., Hong, J.I., and Perer, A. (2023, January 23\u201328). Zeno: An interactive framework for behavioral evaluation of machine learning. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, Hamburg, Germany.","DOI":"10.1145\/3544548.3581268"},{"key":"ref_70","doi-asserted-by":"crossref","unstructured":"Zihni, E., Madai, V.I., Livne, M., Galinovic, I., Khalil, A.A., Fiebach, J.B., and Frey, D. (2020). Opening the black box of artificial intelligence for clinical decision support: A study predicting stroke outcome. PLoS ONE, 15.","DOI":"10.1371\/journal.pone.0231166"},{"key":"ref_71","doi-asserted-by":"crossref","first-page":"e50295","DOI":"10.2196\/50295","article-title":"Resilient artificial intelligence in health: Synthesis and research agenda toward next-generation trustworthy clinical decision support","volume":"26","author":"Ferri","year":"2024","journal-title":"J. Med. Internet Res."},{"key":"ref_72","doi-asserted-by":"crossref","first-page":"102412","DOI":"10.1016\/j.inffus.2024.102412","article-title":"Designing interpretable ML system to enhance trust in healthcare: A systematic review to proposed responsible clinician-AI-collaboration framework","volume":"108","author":"Nasarian","year":"2024","journal-title":"Inf. Fusion"},{"key":"ref_73","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.artint.2018.07.007","article-title":"Explanation in artificial intelligence: Insights from the social sciences","volume":"267","author":"Miller","year":"2019","journal-title":"Artif. Intell."},{"key":"ref_74","unstructured":"Sedig, K., and Liang, H.N. (2007, January 25\u201329). On the design of interactive visual representations: Fitness of interaction. Proceedings of the EdMedia+ Innovate Learning, Vancouver, BC, Canada."},{"key":"ref_75","first-page":"275","article-title":"Designing online mathematical investigation","volume":"23","author":"Gadanidis","year":"2004","journal-title":"J. Comput. Math. Sci. Teach."},{"key":"ref_76","unstructured":"Xie, Y., Gao, G., and Chen, X. (2019). Outlining the design space of explainable intelligent systems for medical diagnosis. arXiv."},{"key":"ref_77","doi-asserted-by":"crossref","unstructured":"Tory, M. (2013). User studies in visualization: A reflection on methods. Handbook of Human Centric Visualization, Springer.","DOI":"10.1007\/978-1-4614-7485-2_16"},{"key":"ref_78","doi-asserted-by":"crossref","first-page":"455","DOI":"10.1002\/asi.23002","article-title":"Adjustable properties of visual representations: Improving the quality of human-information interaction","volume":"65","author":"Parsons","year":"2014","journal-title":"J. Assoc. Inf. Sci. Technol."},{"key":"ref_79","doi-asserted-by":"crossref","unstructured":"Sedig, K., Parsons, P., Dittmer, M., and Haworth, R. (2014). Human-centered interactivity of visualization tools: Micro-and macro-level considerations. Handbook of Human Centric Visualization, Springer.","DOI":"10.1007\/978-1-4614-7485-2_29"},{"key":"ref_80","doi-asserted-by":"crossref","unstructured":"Hundhausen, C.D. (2013). Evaluating visualization environments: Cognitive, social, and cultural perspectives. Handbook of Human Centric Visualization, Springer.","DOI":"10.1007\/978-1-4614-7485-2_5"},{"key":"ref_81","doi-asserted-by":"crossref","unstructured":"Freitas, C.M., Pimenta, M.S., and Scapin, D.L. (2014). User-centered evaluation of information visualization techniques: Making the HCI-InfoVis connection explicit. Handbook of Human Centric Visualization, Springer.","DOI":"10.1007\/978-1-4614-7485-2_12"},{"key":"ref_82","doi-asserted-by":"crossref","unstructured":"Abbas, Q., Jeong, W., and Lee, S.W. (2025). Explainable AI in Clinical Decision Support Systems: A Meta-Analysis of Methods, Applications, and Usability Challenges. Healthcare, 13.","DOI":"10.3390\/healthcare13172154"},{"key":"ref_83","doi-asserted-by":"crossref","first-page":"1161","DOI":"10.1109\/TVCG.2021.3114864","article-title":"Sibyl: Understanding and addressing the usability challenges of machine learning in high-stakes decision making","volume":"28","author":"Zytek","year":"2021","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_84","unstructured":"Gambetti, A., Han, Q., Shen, H., and Soares, C. (2025). A Survey on Human-Centered Evaluation of Explainable AI Methods in Clinical Decision Support Systems. arXiv."}],"container-title":["Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2227-9709\/12\/4\/119\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,28]],"date-time":"2025-10-28T15:09:28Z","timestamp":1761664168000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2227-9709\/12\/4\/119"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,28]]},"references-count":84,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["informatics12040119"],"URL":"https:\/\/doi.org\/10.3390\/informatics12040119","relation":{},"ISSN":["2227-9709"],"issn-type":[{"value":"2227-9709","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,28]]}}}