{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,13]],"date-time":"2026-03-13T05:12:26Z","timestamp":1773378746118,"version":"3.50.1"},"reference-count":48,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2026,1,13]],"date-time":"2026-01-13T00:00:00Z","timestamp":1768262400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000002","name":"NIH","doi-asserted-by":"publisher","award":["T15LM007450"],"award-info":[{"award-number":["T15LM007450"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"NIH","doi-asserted-by":"publisher","award":["U54HG012510"],"award-info":[{"award-number":["U54HG012510"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"NIH","doi-asserted-by":"publisher","award":["T32HL170986"],"award-info":[{"award-number":["T32HL170986"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,3,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>Healthcare decisions are increasingly made with the assistance of machine learning (ML). ML has been known to have unfairness\u2014inconsistent outcomes across subpopulations. Clinicians interacting with these systems can perpetuate such unfairness by overreliance. Recent work exploring ML suppression\u2014silencing predictions based on auditing the ML\u2014shows promise in mitigating performance issues originating from overreliance. This study aims to evaluate the impact of suppression on collaboration fairness and evaluate ML uncertainty as desiderata to audit the ML.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Materials and Methods<\/jats:title>\n                    <jats:p>We used data from the Vanderbilt University Medical Center electronic health record (n\u2009=\u200958\u2009817) and the MIMIC-IV-ED dataset (n\u2009=\u2009363\u2009145) to predict likelihood of death or intensive care unit transfer and likelihood of 30-day readmission using gradient-boosted trees and an artificially high-performing oracle model. We derived clinician decisions directly from the dataset and simulated clinician acceptance of ML predictions based on previous empirical work on acceptance of clinical decision support alerts. We measured performance as area under the receiver operating characteristic curve and algorithmic fairness using absolute averaged odds difference.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>When the ML outperforms humans, suppression outperforms the human alone (P\u2009&amp;lt;\u20098.2\u2009\u00d7\u200910\u22126) and at least does not degrade fairness. When the human outperforms the ML, the human is either fairer than suppression (P\u2009&amp;lt;\u20098.2\u2009\u00d7\u200910\u22124) or there is no statistically significant difference in fairness. Incorporating uncertainty quantification into suppression approaches can improve performance.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Suppression of poor-quality ML predictions through an auditor model shows promise in improving collaborative human-AI performance and fairness.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/jamia\/ocaf235","type":"journal-article","created":{"date-parts":[[2026,1,1]],"date-time":"2026-01-01T12:46:16Z","timestamp":1767271576000},"page":"621-631","source":"Crossref","is-referenced-by-count":0,"title":["Auditor models to suppress poor artificial intelligence predictions can improve human-artificial intelligence collaborative performance"],"prefix":"10.1093","volume":"33","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4443-8541","authenticated-orcid":false,"given":"Katherine E","family":"Brown","sequence":"first","affiliation":[{"name":"Vanderbilt University Medical Center Department of Biomedical Informatics, , Nashville, TN 37203,","place":["United States"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9869-9328","authenticated-orcid":false,"given":"Jesse O","family":"Wrenn","sequence":"additional","affiliation":[{"name":"Vanderbilt University Medical Center Department of Biomedical Informatics, , Nashville, TN 37203,","place":["United States"]},{"name":"Vanderbilt University Medical Center Department of Emergency Medicine, , Nashville, TN 37232,","place":["United States"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6763-599X","authenticated-orcid":false,"given":"Nicholas J","family":"Jackson","sequence":"additional","affiliation":[{"name":"Vanderbilt University Medical Center Department of Biomedical Informatics, , Nashville, TN 37203,","place":["United States"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3835-2725","authenticated-orcid":false,"given":"Michael R","family":"Cauley","sequence":"additional","affiliation":[{"name":"Vanderbilt University Medical Center Department of Biomedical Informatics, , Nashville, TN 37203,","place":["United States"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6884-3819","authenticated-orcid":false,"given":"Benjamin X","family":"Collins","sequence":"additional","affiliation":[{"name":"Vanderbilt University Medical Center Department of Biomedical Informatics, , Nashville, TN 37203,","place":["United States"]},{"name":"Vanderbilt University Medical Center Department of Medicine, , Nashville, TN 37232,","place":["USA"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0415-4301","authenticated-orcid":false,"given":"Laurie L","family":"Novak","sequence":"additional","affiliation":[{"name":"Vanderbilt University Medical Center Department of Biomedical Informatics, , Nashville, TN 37203,","place":["United States"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3040-5175","authenticated-orcid":false,"given":"Bradley A","family":"Malin","sequence":"additional","affiliation":[{"name":"Vanderbilt University Medical Center Department of Biomedical Informatics, , Nashville, TN 37203,","place":["United States"]},{"name":"Vanderbilt University Medical Center Department of Biostatistics, , Nashville, TN 37203,","place":["United States"]},{"name":"Vanderbilt University Department of Computer Science, , Nashville, TN 37212,","place":["United States"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3859-9130","authenticated-orcid":false,"given":"Jessica S","family":"Ancker","sequence":"additional","affiliation":[{"name":"Vanderbilt University Medical Center Department of Biomedical Informatics, , Nashville, TN 37203,","place":["United States"]}]}],"member":"286","published-online":{"date-parts":[[2026,1,13]]},"reference":[{"key":"2026031216465856700_ocaf235-B1","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1055\/s-0039-1677903","article-title":"Artificial intelligence in clinical decision support: challenges for evaluating AI and practical implications","volume":"28","author":"Magrabi","year":"2019","journal-title":"Yearb Med Inform"},{"key":"2026031216465856700_ocaf235-B2","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1055\/s-0039-1677911","article-title":"Artificial intelligence in clinical decision support: a focused literature survey","volume":"28","author":"Montani","year":"2019","journal-title":"Yearb Med Inform"},{"key":"2026031216465856700_ocaf235-B3","doi-asserted-by":"crossref","first-page":"334","DOI":"10.1038\/s41390-022-02226-1","article-title":"Artificial intelligence-based clinical decision support in pediatrics","volume":"93","author":"Ramgopal","year":"2023","journal-title":"Pediatr Res"},{"key":"2026031216465856700_ocaf235-B4","doi-asserted-by":"crossref","first-page":"2199","DOI":"10.1001\/jama.2018.17163","article-title":"Clinical decision support in the era of artificial intelligence","volume":"320","author":"Shortliffe","year":"2018","journal-title":"JAMA"},{"key":"2026031216465856700_ocaf235-B5","author":"Yang","year":"2019"},{"key":"2026031216465856700_ocaf235-B6","doi-asserted-by":"crossref","first-page":"6878","DOI":"10.1109\/ACCESS.2022.3142100","article-title":"AI-driven clinical decision support: enhancing disease diagnosis exploiting patients similarity","volume":"10","author":"Comito","year":"2022","journal-title":"IEEE Access"},{"key":"2026031216465856700_ocaf235-B7","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1038\/s41746-021-00438-z","article-title":"Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis","volume":"4","author":"Aggarwal","year":"2021","journal-title":"NPJ Digit Med"},{"key":"2026031216465856700_ocaf235-B8","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1038\/s41746-024-01031-w","article-title":"Human-AI interaction in skin cancer diagnosis: a systematic review and meta-analysis","volume":"7","author":"Krakowski","year":"2024","journal-title":"NPJ Digit Med"},{"issue":"6","key":"2026031216465856700_ocaf235-B9","doi-asserted-by":"publisher","first-page":"e271","DOI":"10.1016\/S2589-7500(19)30123-2","article-title":"A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis","volume":"1","author":"Liu","year":"2019","journal-title":"Lancet Digit Health"},{"key":"2026031216465856700_ocaf235-B10","first-page":"3529","author":"Ahmad","year":"2020"},{"key":"2026031216465856700_ocaf235-B11","doi-asserted-by":"crossref","first-page":"3","DOI":"10.3390\/sci6010003","article-title":"Fairness and bias in artificial intelligence: a brief survey of sources, impacts, and mitigation strategies","volume":"6","author":"Ferrara","year":"2023","journal-title":"Sci"},{"key":"2026031216465856700_ocaf235-B12","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/s11604-023-01474-3","article-title":"Fairness of artificial intelligence in healthcare: review and recommendations","volume":"42","author":"Ueda","year":"2024","journal-title":"Jpn J Radiol"},{"key":"2026031216465856700_ocaf235-B13","doi-asserted-by":"crossref","first-page":"2275","DOI":"10.1001\/jama.2023.22295","article-title":"Measuring the impact of AI in the diagnosis of hospitalized patients: a randomized clinical vignette survey study","volume":"330","author":"Jabbour","year":"2023","journal-title":"JAMA"},{"key":"2026031216465856700_ocaf235-B14","doi-asserted-by":"crossref","first-page":"2255","DOI":"10.1001\/jama.2023.22557","article-title":"Automation bias and assistive AI: risk of harm from AI-driven clinical decision support","volume":"330","author":"Khera","year":"2023","journal-title":"JAMA"},{"key":"2026031216465856700_ocaf235-B15","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1136\/amiajnl-2011-000089","article-title":"Automation bias: a systematic review of frequency, effect mediators, and mitigators","volume":"19","author":"Goddard","year":"2012","journal-title":"J Am Med Inform Assoc"},{"key":"2026031216465856700_ocaf235-B16","first-page":"23","author":"Zhou","year":"2017"},{"key":"2026031216465856700_ocaf235-B17","doi-asserted-by":"crossref","first-page":"1684","DOI":"10.1093\/jamia\/ocad118","article-title":"Artificial intelligence suppression as a strategy to mitigate artificial intelligence automation bias","volume":"30","author":"Wang","year":"2023","journal-title":"J Am Med Inform Assoc"},{"key":"2026031216465856700_ocaf235-B18","doi-asserted-by":"crossref","first-page":"1080","DOI":"10.1093\/jamia\/ocy052","article-title":"Good intentions are not enough: how informatics interventions can worsen inequality","volume":"25","author":"Veinot","year":"2018","journal-title":"J Am Med Inform Assoc"},{"key":"2026031216465856700_ocaf235-B19","doi-asserted-by":"crossref","first-page":"e214622","DOI":"10.1001\/jamanetworkopen.2021.4622","article-title":"Algorithmovigilance\u2014advancing methods to analyze and monitor artificial intelligence\u2013driven health care for effectiveness and equity","volume":"4","author":"Embi","year":"2021","journal-title":"JAMA Netw Open"},{"issue":"1","key":"2026031216465856700_ocaf235-B20","doi-asserted-by":"publisher","first-page":"132","DOI":"10.1016\/j.acra.2019.09.009","article-title":"The Algorithmic Audit: Working with Vendors to Validate Radiology-AI Algorithms-How We Do It","volume":"27","author":"Mahajan","year":"2020","journal-title":"Acad Radiol"},{"key":"2026031216465856700_ocaf235-B21","first-page":"1050","author":"Gal","year":"2016"},{"key":"2026031216465856700_ocaf235-B22","author":"Malinin"},{"key":"2026031216465856700_ocaf235-B23","volume-title":"AMIA Annual Symposium Proceedings","author":"Brown","year":"2023"},{"key":"2026031216465856700_ocaf235-B24","author":"Chan"},{"key":"2026031216465856700_ocaf235-B25","doi-asserted-by":"crossref","first-page":"e0000606","DOI":"10.1371\/journal.pdig.0000606","article-title":"Aspiring to clinical significance: insights from developing and evaluating a machine learning model to predict emergency department return visit admissions","volume":"3","author":"Zhang","year":"2024","journal-title":"PLOS Digit Health"},{"key":"2026031216465856700_ocaf235-B26","first-page":"e80","article-title":"LACE+ index: extension of a validated index to predict early death or urgent readmission after hospital discharge using administrative data","volume":"6","author":"van Walraven","year":"2012","journal-title":"Open Med"},{"key":"2026031216465856700_ocaf235-B27","doi-asserted-by":"crossref","first-page":"658","DOI":"10.1038\/s41597-022-01782-9","article-title":"Benchmarking emergency department prediction models with machine learning and public electronic health records","volume":"9","author":"Xie","year":"2022","journal-title":"Sci Data"},{"key":"2026031216465856700_ocaf235-B28","doi-asserted-by":"publisher","author":"Johnson","year":"2021","DOI":"10.13026\/5ntk-km72"},{"key":"2026031216465856700_ocaf235-B29","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1038\/s41597-023-02136-9","article-title":"MIMIC-IV, a freely accessible electronic health record dataset","volume":"10","author":"Johnson","year":"2023","journal-title":"Sci Data"},{"key":"2026031216465856700_ocaf235-B30","author":"Dorogush"},{"key":"2026031216465856700_ocaf235-B31","author":"grid_search | [Internet]","year":"2024"},{"key":"2026031216465856700_ocaf235-B32","author":"Dwork C, Ilvento C, Rothblum GN, Sur P. Abstracting Fairness: Oracles, Metrics, and Interpretability. In1st Symposium on Foundations of Responsible Computing (FORC 2020) 2020 (pp. 8-1). Schloss Dagstuhl\u2013Leibniz-Zentrum f\u00fcr Informatik."},{"key":"2026031216465856700_ocaf235-B33","author":"Tai","year":"2022"},{"key":"2026031216465856700_ocaf235-B34","author":"Brown","year":"2025"},{"key":"2026031216465856700_ocaf235-B35","doi-asserted-by":"crossref","first-page":"953","DOI":"10.1016\/j.dss.2012.09.018","article-title":"Factors influencing decision support system acceptance","volume":"54","author":"Shibl","year":"2013","journal-title":"Decis Support Syst"},{"key":"2026031216465856700_ocaf235-B36","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1186\/1472-6947-6-6","article-title":"A survey of factors affecting clinician acceptance of clinical decision support","volume":"6","author":"Sittig","year":"2006","journal-title":"BMC Med Inform Decis Mak"},{"key":"2026031216465856700_ocaf235-B37","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1038\/s41586-020-2649-2","article-title":"Array programming with NumPy","volume":"585","author":"Harris","year":"2020","journal-title":"Nature"},{"key":"2026031216465856700_ocaf235-B38","doi-asserted-by":"crossref","first-page":"746","DOI":"10.4338\/ACI-2014-02-RA-0018","article-title":"Reducing risk with clinical decision support","volume":"5","author":"Zuccotti","year":"2014","journal-title":"Appl Clin Inform"},{"key":"2026031216465856700_ocaf235-B39","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1055\/s-0040-1701986","article-title":"Clinical decision support and implications for the clinician burnout crisis","volume":"29","author":"Jankovic","year":"2020","journal-title":"Yearb Med Inform"},{"key":"2026031216465856700_ocaf235-B40","doi-asserted-by":"crossref","first-page":"737","DOI":"10.1016\/j.jchf.2015.05.007","article-title":"Identification of emergency department patients with acute heart failure at low risk for 30-day adverse events","volume":"3","author":"Collins","year":"2015","journal-title":"JACC Heart Fail"},{"key":"2026031216465856700_ocaf235-B41","doi-asserted-by":"crossref","first-page":"671","DOI":"10.1111\/acem.14641","article-title":"Emergency physician risk tolerance in acute heart failure is higher than previously thought and compatible with modern disposition decision instruments","volume":"30","author":"Harrison","year":"2023","journal-title":"Acad Emerg Med"},{"key":"2026031216465856700_ocaf235-B42","doi-asserted-by":"crossref","first-page":"2463","DOI":"10.3390\/diagnostics12102463","article-title":"Barriers and opportunities regarding implementation of a machine learning-based acute heart failure risk stratification tool in the emergency department","volume":"12","author":"Sax","year":"2022","journal-title":"Diagnostics (Basel)"},{"key":"2026031216465856700_ocaf235-B43","volume-title":"Advances in Neural Information Processing Systems [Internet]","author":"Hardt","year":"2016"},{"key":"2026031216465856700_ocaf235-B44","first-page":"212","article-title":"Derivation and experimental performance of standard and novel uncertainty calibration techniques","volume":"202","author":"Brown","year":"2025","journal-title":"AMIA Annu Symp Proc"},{"key":"2026031216465856700_ocaf235-B45","first-page":"5574","article-title":"What uncertainties do we need in Bayesian deep learning for computer vision?","volume":"30","author":"Kendall","year":"2017","journal-title":"Adv Neural Inf Process Syst"},{"key":"2026031216465856700_ocaf235-B46","author":"Lu","year":"2021"},{"key":"2026031216465856700_ocaf235-B47","first-page":"26","article-title":"Some unintended consequences of clinical decision support systems","volume":"2007","author":"Ash","year":"2007","journal-title":"AMIA Annu Symp Proc"},{"key":"2026031216465856700_ocaf235-B48","doi-asserted-by":"crossref","first-page":"29","DOI":"10.7326\/0003-4819-157-1-201207030-00450","article-title":"Effect of clinical decision-support systems: a systematic review","volume":"157","author":"Bright","year":"2012","journal-title":"Ann Intern Med"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/33\/3\/621\/66395649\/ocaf235.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/33\/3\/621\/66395649\/ocaf235.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,12]],"date-time":"2026-03-12T20:47:08Z","timestamp":1773348428000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/33\/3\/621\/8424253"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1,13]]},"references-count":48,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2026,1,13]]},"published-print":{"date-parts":[[2026,3,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocaf235","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,3]]},"published":{"date-parts":[[2026,1,13]]}}}