{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,20]],"date-time":"2026-02-20T05:03:22Z","timestamp":1771563802782,"version":"3.50.1"},"reference-count":30,"publisher":"SAGE Publications","issue":"4","license":[{"start":{"date-parts":[[2024,10,1]],"date-time":"2024-10-01T00:00:00Z","timestamp":1727740800000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"NA"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Health Informatics J"],"published-print":{"date-parts":[[2024,10]]},"abstract":"<jats:sec><jats:title>Objective<\/jats:title><jats:p> This study aimed to assess the practicality and trustworthiness of explainable artificial intelligence (XAI) methods used for explaining clinical predictive models. <\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p> Two popular XAIs used for explaining clinical predictive models were evaluated based on their ability to generate domain-appropriate representations, impact clinical workflow, and consistency. Explanations were benchmarked against true clinical deterioration triggers recorded in the data system and agreement was quantified. The evaluation was conducted using two Electronic Medical Records datasets from major hospitals in Australia. Results were examined and commented on by a senior clinician. <\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p> Findings demonstrate a violation of consistency criteria and moderate concordance (0.47-0.8) with true triggers, undermining reliability and actionability, criteria for clinicians\u2019 trust in XAI. <\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p> Explanations are not trustworthy to guide clinical interventions, though they may offer useful insights and help model troubleshooting. Clinician-informed XAI development and presentation, clear disclaimers on limitations, and critical clinical judgment can promote informed decisions and prevent over-reliance. <\/jats:p><\/jats:sec>","DOI":"10.1177\/14604582241304730","type":"journal-article","created":{"date-parts":[[2024,12,24]],"date-time":"2024-12-24T08:56:12Z","timestamp":1735030572000},"update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":8,"title":["Benchmarking the most popular XAI used for explaining clinical predictive models: Untrustworthy but could be useful"],"prefix":"10.1177","volume":"30","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7978-575X","authenticated-orcid":false,"given":"Aida","family":"Brankovic","sequence":"first","affiliation":[{"name":"CSIRO Australian e-Health Research Centre, Brisbane, QLD, Australia"}]},{"given":"David","family":"Cook","sequence":"additional","affiliation":[{"name":"Intensive Care Unit, Princess Alexandra Hospital, Brisbane, QLD, Australia"}]},{"given":"Jessica","family":"Rahman","sequence":"additional","affiliation":[{"name":"CSIRO Australian e-Health Research Centre, Brisbane, QLD, Australia"}]},{"given":"Sankalp","family":"Khanna","sequence":"additional","affiliation":[{"name":"CSIRO Australian e-Health Research Centre, Brisbane, QLD, Australia"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-0443-7902","authenticated-orcid":false,"given":"Wenjie","family":"Huang","sequence":"additional","affiliation":[{"name":"The University of Queensland, Brisbane, QLD, Australia"}]}],"member":"179","published-online":{"date-parts":[[2024,12,24]]},"reference":[{"key":"bibr1-14604582241304730","doi-asserted-by":"publisher","DOI":"10.1016\/B978-0-12-809523-2.00019-4"},{"key":"bibr2-14604582241304730","unstructured":"Tonekaboni S, Joshi S, McCradden MD, et al. What clinicians want: contextualizing explainable machine learning for clinical end use. In: Machine learning for healthcare conference. USA : PMLR, pp. 359\u2013380."},{"key":"bibr3-14604582241304730","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2020.3027314"},{"key":"bibr4-14604582241304730","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pmed.1002689"},{"key":"bibr5-14604582241304730","author":"Smilkov D","year":"2017","journal-title":"arXiv preprint arXiv:170603825"},{"key":"bibr6-14604582241304730","unstructured":"Sundararajan M, Taly A, Yan Q. Axiomatic attribution for deep networks. In: International conference on machine learning. USA, PMLR, pp. 3319\u20133328."},{"key":"bibr7-14604582241304730","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2016.11.008"},{"key":"bibr8-14604582241304730","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-28954-6_10"},{"key":"bibr9-14604582241304730","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939778"},{"key":"bibr10-14604582241304730","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-019-0138-9"},{"key":"bibr11-14604582241304730","author":"Krishna S","year":"2022","journal-title":"arXiv preprint arXiv:220201602"},{"key":"bibr12-14604582241304730","author":"Brankovic A","year":"2023","journal-title":"MedInfo2023"},{"key":"bibr13-14604582241304730","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-020-17431-x"},{"key":"bibr14-14604582241304730","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-022-15877-1"},{"key":"bibr15-14604582241304730","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-022-20907-z"},{"key":"bibr16-14604582241304730","volume-title":"Pattern recognition and machine learning","volume":"4","author":"Bishop CM","year":"2006"},{"key":"bibr17-14604582241304730","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939785"},{"key":"bibr18-14604582241304730","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2021.103502"},{"key":"bibr19-14604582241304730","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijmedinf.2022.104884"},{"key":"bibr20-14604582241304730","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocz229"},{"key":"bibr21-14604582241304730","first-page":"1","volume-title":"30\u2013May 5 2022","author":"Panigutti C"},{"key":"bibr22-14604582241304730","doi-asserted-by":"publisher","DOI":"10.1016\/S2589-7500(21)00208-9"},{"key":"bibr23-14604582241304730","doi-asserted-by":"publisher","DOI":"10.2196\/33960"},{"key":"bibr24-14604582241304730","first-page":"255","volume-title":"World conference on explainable artificial intelligence","author":"Cabitza F"},{"key":"bibr25-14604582241304730","doi-asserted-by":"publisher","DOI":"10.1016\/j.artmed.2023.102506"},{"key":"bibr26-14604582241304730","first-page":"4778","volume":"33","author":"Heskes T","year":"2020","journal-title":"Adv Neural Inf Process Syst"},{"key":"bibr27-14604582241304730","unstructured":"Salih AM. Explainable artificial intelligence and multicollinearity: a mini review of current approaches. arXiv preprint arXiv:240611524 2024."},{"key":"bibr28-14604582241304730","doi-asserted-by":"publisher","DOI":"10.1145\/3531146.3533168"},{"key":"bibr29-14604582241304730","doi-asserted-by":"publisher","DOI":"10.1145\/3580305.3599240"},{"key":"bibr30-14604582241304730","doi-asserted-by":"crossref","unstructured":"Cinquini M, Guidotti R. Causality-aware local interpretable model-agnostic explanations. In: World Conference on Explainable Artificial Intelligence, July 17\u201319, 2024, Valletta, Malta. Springer, pp. 108\u2013124.","DOI":"10.1007\/978-3-031-63800-8_6"}],"container-title":["Health Informatics Journal"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/14604582241304730","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/14604582241304730","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/14604582241304730","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,1]],"date-time":"2025-03-01T07:42:45Z","timestamp":1740814965000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/14604582241304730"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10]]},"references-count":30,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2024,10]]}},"alternative-id":["10.1177\/14604582241304730"],"URL":"https:\/\/doi.org\/10.1177\/14604582241304730","relation":{},"ISSN":["1460-4582","1741-2811"],"issn-type":[{"value":"1460-4582","type":"print"},{"value":"1741-2811","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10]]},"article-number":"14604582241304730"}}