{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T19:49:03Z","timestamp":1776109743329,"version":"3.50.1"},"reference-count":58,"publisher":"Oxford University Press (OUP)","issue":"4","license":[{"start":{"date-parts":[[2020,2,27]],"date-time":"2020-02-27T00:00:00Z","timestamp":1582761600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"name":"Precision Driven Health"},{"name":"Auckland Regional Health Boards"},{"DOI":"10.13039\/501100001537","name":"University of Auckland","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100001537","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Orion Health"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2020,4,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>Implementation of machine learning (ML) may be limited by patients\u2019 right to \u201cmeaningful information about the logic involved\u201d when ML influences healthcare decisions. Given the complexity of healthcare decisions, it is likely that ML outputs will need to be understood and trusted by physicians, and then explained to patients. We therefore investigated the association between physician understanding of ML outputs, their ability to explain these to patients, and their willingness to trust the ML outputs, using various ML explainability methods.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>We designed a survey for physicians with a diagnostic dilemma that could be resolved by an ML risk calculator. Physicians were asked to rate their understanding, explainability, and trust in response to 3 different ML outputs. One ML output had no explanation of its logic (the control) and 2 ML outputs used different model-agnostic explainability methods. The relationships among understanding, explainability, and trust were assessed using Cochran-Mantel-Haenszel tests of association.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>The survey was sent to 1315 physicians, and 170 (13%) provided completed surveys. There were significant associations between physician understanding and explainability (P\u2009&amp;lt;\u2009.001), between physician understanding and trust (P\u2009&amp;lt;\u2009.001), and between explainability and trust (P\u2009&amp;lt;\u2009.001). ML outputs that used model-agnostic explainability methods were preferred by 88% of physicians when compared with the control condition; however, no particular ML explainability method had a greater influence on intended physician behavior.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusions<\/jats:title>\n                  <jats:p>Physician understanding, explainability, and trust in ML risk calculators are related. Physicians preferred ML outputs accompanied by model-agnostic explanations but the explainability method did not alter intended physician behavior.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocz229","type":"journal-article","created":{"date-parts":[[2019,12,31]],"date-time":"2019-12-31T12:09:13Z","timestamp":1577794153000},"page":"592-600","source":"Crossref","is-referenced-by-count":171,"title":["Physician understanding, explainability, and trust in a hypothetical machine learning risk calculator"],"prefix":"10.1093","volume":"27","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3808-4428","authenticated-orcid":false,"given":"William K","family":"Diprose","sequence":"first","affiliation":[{"name":"Department of Medicine, University of Auckland, Auckland, New Zealand"}]},{"given":"Nicholas","family":"Buist","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, Whangarei Hospital, Whangarei, New Zealand"}]},{"given":"Ning","family":"Hua","sequence":"additional","affiliation":[{"name":"Orion Health, Auckland, New Zealand"}]},{"given":"Quentin","family":"Thurier","sequence":"additional","affiliation":[{"name":"Orion Health, Auckland, New Zealand"}]},{"given":"George","family":"Shand","sequence":"additional","affiliation":[{"name":"Clinical Education and Training Unit, Waitemat\u0101 District Health Board, Auckland, New Zealand"}]},{"given":"Reece","family":"Robinson","sequence":"additional","affiliation":[{"name":"Orion Health, Auckland, New Zealand"}]}],"member":"286","published-online":{"date-parts":[[2020,2,27]]},"reference":[{"issue":"2","key":"2020110613091405600_ocz229-B1","doi-asserted-by":"crossref","first-page":"98","DOI":"10.7326\/0003-4819-135-2-200107170-00010","article-title":"Excluding pulmonary embolism at the bedside without diagnostic imaging: management of patients with suspected pulmonary embolism presenting to the emergency department by using a simple clinical model and d-dimer","volume":"135","author":"Wells","year":"2001","journal-title":"Ann Intern Med"},{"issue":"8","key":"2020110613091405600_ocz229-B2","doi-asserted-by":"crossref","first-page":"1247","DOI":"10.1111\/j.1538-7836.2004.00790.x","article-title":"Clinical criteria to prevent unnecessary diagnostic testing in emergency department patients with suspected pulmonary embolism","volume":"2","author":"Kline","year":"2004","journal-title":"J Thromb Haemost"},{"issue":"1","key":"2020110613091405600_ocz229-B3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1136\/bmjquality.u208698.w4222","article-title":"Improving the diagnosis of pulmonary embolism in the emergency department","volume":"4","author":"Cooper","year":"2015","journal-title":"BMJ Qual Improv Rep"},{"issue":"8","key":"2020110613091405600_ocz229-B4","doi-asserted-by":"crossref","first-page":"e198719","DOI":"10.1001\/jamanetworkopen.2019.8719","article-title":"Development and performance of the pulmonary embolism result forecast model (PERFORM) for computed tomography clinical decision support","volume":"2","author":"Banerjee","year":"2019","journal-title":"JAMA Netw Open"},{"issue":"2","key":"2020110613091405600_ocz229-B5","doi-asserted-by":"crossref","first-page":"E160","DOI":"10.1001\/amajethics.2019.160","article-title":"Are current tort liability doctrines adequate for addressing injury caused by AI?","volume":"21","author":"Sullivan","year":"2019","journal-title":"AMA J Ethics"},{"issue":"2","key":"2020110613091405600_ocz229-B6","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1001\/amajethics.2019.125","article-title":"How should AI be developed, validated, and implemented in patient care?","volume":"21","author":"Anderson","year":"2019","journal-title":"AMA J Ethics"},{"key":"2020110613091405600_ocz229-B7","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1017\/9781108147972.027","volume-title":"Big Data, Health Law, and Bioethics","author":"Price","year":"2018"},{"issue":"4","key":"2020110613091405600_ocz229-B8","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1093\/idpl\/ipx022","article-title":"Meaningful information and the right to explanation","volume":"7","author":"Selbst","year":"2017","journal-title":"Int Data Priv Law"},{"key":"2020110613091405600_ocz229-B9","first-page":"325","author":"Thurier","year":"2019"},{"issue":"10","key":"2020110613091405600_ocz229-B10","doi-asserted-by":"crossref","first-page":"749","DOI":"10.1038\/s41551-018-0304-0","article-title":"Explainable machine-learning predictions for the prevention of hypoxaemia during surgery","volume":"2","author":"Lundberg","year":"2018","journal-title":"Nat Biomed Eng"},{"issue":"10","key":"2020110613091405600_ocz229-B11","doi-asserted-by":"crossref","first-page":"1392","DOI":"10.1093\/jamia\/ocy106","article-title":"Mechanistic machine learning: how data assimilation leverages physiologic knowledge using bayesian inference to forecast the future, infer the present, and phenotype","volume":"25","author":"Albers","year":"2018","journal-title":"J Am Med Inform Assoc"},{"issue":"4","key":"2020110613091405600_ocz229-B12","doi-asserted-by":"crossref","first-page":"e1005232","DOI":"10.1371\/journal.pcbi.1005232","article-title":"Personalized glucose forecasting for type 2 diabetes using data assimilation","volume":"13","author":"Albers","year":"2017","journal-title":"PLoS Comput Biol"},{"issue":"5","key":"2020110613091405600_ocz229-B13","doi-asserted-by":"crossref","first-page":"1079","DOI":"10.1016\/j.cell.2015.11.001","article-title":"Personalized nutrition by prediction of glycemic responses","volume":"163","author":"Zeevi","year":"2015","journal-title":"Cell"},{"key":"2020110613091405600_ocz229-B14","author":"Ribeiro","year":"2016"},{"key":"2020110613091405600_ocz229-B15","first-page":"1803","article-title":"How to explain individual classification decisions","volume":"11","author":"Baehrens","year":"2010","journal-title":"J Mach Learn Res"},{"key":"2020110613091405600_ocz229-B16","first-page":"5686","author":"Krause","year":"2016"},{"key":"2020110613091405600_ocz229-B17","first-page":"1135","author":"Ribeiro","year":"2016"},{"issue":"1","key":"2020110613091405600_ocz229-B18","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach Learn"},{"issue":"1","key":"2020110613091405600_ocz229-B19","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1080\/10618600.2014.907095","article-title":"Peeking inside the black box: visualizing statistical learning with plots of individual conditional expectation","volume":"24","author":"Goldstein","year":"2015","journal-title":"J Comput Graph Stat"},{"key":"2020110613091405600_ocz229-B20","first-page":"4765","volume-title":"Advances in Neural Information Processing Systems","author":"Lundberg","year":"2017"},{"key":"2020110613091405600_ocz229-B21","first-page":"1","author":"Wang","year":"2019"},{"issue":"7313","key":"2020110613091405600_ocz229-B22","doi-asserted-by":"crossref","first-page":"625","DOI":"10.1136\/bmj.323.7313.625","article-title":"The challenge of complexity in health care","volume":"323","author":"Plsek","year":"2001","journal-title":"BMJ"},{"issue":"1","key":"2020110613091405600_ocz229-B23","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1002\/hast.973","article-title":"Artificial intelligence and black-box medical decisions: accuracy versus explainability","volume":"49","author":"London","year":"2019","journal-title":"Hastings Cent Rep"},{"issue":"1","key":"2020110613091405600_ocz229-B24","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1001\/jama.2017.19198","article-title":"What this computer needs is a physician: humanism and artificial intelligence","volume":"319","author":"Verghese","year":"2018","journal-title":"JAMA"},{"issue":"1434","key":"2020110613091405600_ocz229-B25","first-page":"73","article-title":"Artificial intelligence in medicine: humans need not apply?","volume":"129","author":"Diprose","year":"2016","journal-title":"N Z Med J"},{"issue":"4","key":"2020110613091405600_ocz229-B26","doi-asserted-by":"crossref","first-page":"547","DOI":"10.1097\/CCM.0000000000002936","article-title":"An interpretable machine learning model for accurate prediction of sepsis in the ICU","volume":"46","author":"Nemati","year":"2018","journal-title":"Crit Care Med"},{"issue":"1","key":"2020110613091405600_ocz229-B27","doi-asserted-by":"crossref","first-page":"617","DOI":"10.1186\/s13104-015-1554-5","article-title":"Neural hypernetwork approach for pulmonary embolism diagnosis","volume":"8","author":"Rucco","year":"2015","journal-title":"BMC Res Notes"},{"issue":"7639","key":"2020110613091405600_ocz229-B28","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1038\/nature21056","article-title":"Dermatologist-level classification of skin cancer with deep neural networks","volume":"542","author":"Esteva","year":"2017","journal-title":"Nature"},{"issue":"1","key":"2020110613091405600_ocz229-B29","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1148\/radiol.2017170236","article-title":"Performance of a deep-learning neural network model in assessing skeletal maturity on pediatric hand radiographs","volume":"287","author":"Larson","year":"2018","journal-title":"Radiology"},{"issue":"11","key":"2020110613091405600_ocz229-B30","doi-asserted-by":"crossref","first-page":"e0188532","DOI":"10.1371\/journal.pone.0188532","article-title":"A machine learning approach to triaging patients with chronic obstructive pulmonary disease","volume":"12","author":"Swaminathan","year":"2017","journal-title":"PLoS One"},{"key":"2020110613091405600_ocz229-B31","first-page":"2019","year":"2019"},{"key":"2020110613091405600_ocz229-B32","first-page":"2019","year":"2019"},{"issue":"19","key":"2020110613091405600_ocz229-B33","doi-asserted-by":"crossref","first-page":"1240","DOI":"10.1056\/NEJM199205073261902","article-title":"The clinical course of pulmonary embolism","volume":"326","author":"Carson","year":"1992","journal-title":"N Engl J Med"},{"key":"2020110613091405600_ocz229-B34","author":"Hall","year":"2019"},{"key":"2020110613091405600_ocz229-B35","first-page":"2019","year":"2019"},{"key":"2020110613091405600_ocz229-B36","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-642-04898-2","volume-title":"International Encyclopedia of Statistical Science","author":"Lovric","year":"2011"},{"key":"2020110613091405600_ocz229-B37","doi-asserted-by":"crossref","DOI":"10.1007\/s10479-018-2818-y","article-title":"Investigating the impacting factors for the healthcare professionals to adopt artificial intelligence-based medical diagnosis support system (AIMDSS)","author":"Fan","year":"2018","journal-title":"Ann Oper Res"},{"issue":"1","key":"2020110613091405600_ocz229-B38","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1186\/s12911-015-0205-z","article-title":"Implementation of multiple-domain covering computerized decision support systems in primary care: a focus group study on perceived barriers","volume":"15","author":"Lugtenberg","year":"2015","journal-title":"BMC Med Inform Decis Mak"},{"issue":"3","key":"2020110613091405600_ocz229-B39","doi-asserted-by":"crossref","first-page":"333","DOI":"10.14236\/jhi.v22i3.153","article-title":"Primary care physicians\u2019 perspectives on computer-based health risk assessment tools for chronic diseases: a mixed methods study","volume":"22","author":"Voruganti","year":"2015","journal-title":"J Innov Health Inform"},{"key":"2020110613091405600_ocz229-B40","author":"Xie","year":"2019"},{"issue":"6","key":"2020110613091405600_ocz229-B41","doi-asserted-by":"crossref","first-page":"517","DOI":"10.1001\/jama.2017.7797","article-title":"Unintended consequences of machine learning in medicine","volume":"318","author":"Cabitza","year":"2017","journal-title":"JAMA"},{"issue":"3","key":"2020110613091405600_ocz229-B42","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1136\/bmjqs-2018-008370","article-title":"Artificial intelligence, bias and clinical safety","volume":"28","author":"Challen","year":"2019","journal-title":"BMJ Qual Saf"},{"issue":"21","key":"2020110613091405600_ocz229-B43","doi-asserted-by":"crossref","first-page":"1599","DOI":"10.1161\/CIRCULATIONAHA.116.025204","article-title":"The preventive-pill paradox: how shared decision making could increase cardiovascular morbidity and mortality","volume":"134","author":"Diprose","year":"2016","journal-title":"Circulation"},{"key":"2020110613091405600_ocz229-B44","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-57959-7","volume-title":"The EU General Data Protection Regulation (GDPR). A Practical Guide","author":"Voigt","year":"2017","edition":"1st ed"},{"issue":"6","key":"2020110613091405600_ocz229-B45","doi-asserted-by":"crossref","first-page":"608","DOI":"10.1197\/jamia.M2115","article-title":"Design features of graphs in health risk communication: a systematic review","volume":"13","author":"Ancker","year":"2006","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613091405600_ocz229-B46","volume-title":"Problem-Based Learning: An Approach to Medical Education","author":"Barrows","year":"1980"},{"issue":"7339","key":"2020110613091405600_ocz229-B47","doi-asserted-by":"crossref","first-page":"729","DOI":"10.1136\/bmj.324.7339.729","article-title":"Clinical problem solving and diagnostic decision making: selective review of the cognitive literature","volume":"324","author":"Elstein","year":"2002","journal-title":"BMJ"},{"issue":"1","key":"2020110613091405600_ocz229-B48","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1136\/amiajnl-2012-001145","article-title":"Next-generation phenotyping of electronic health records","volume":"20","author":"Hripcsak","year":"2013","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613091405600_ocz229-B49","doi-asserted-by":"crossref","first-page":"k1479","DOI":"10.1136\/bmj.k1479","article-title":"Biases in electronic health record data due to processes within the healthcare system: retrospective observational study","volume":"361","author":"Agniel","year":"2018","journal-title":"BMJ"},{"issue":"e2","key":"2020110613091405600_ocz229-B50","doi-asserted-by":"crossref","first-page":"e311","DOI":"10.1136\/amiajnl-2013-001922","article-title":"Correlating electronic health record concepts with healthcare process events","volume":"20","author":"Hripcsak","year":"2013","journal-title":"J Am Med Inform Assoc"},{"issue":"4","key":"2020110613091405600_ocz229-B51","doi-asserted-by":"crossref","first-page":"306","DOI":"10.4037\/ajcc2013426","article-title":"Relationship between nursing documentation and patients\u2019 mortality","volume":"22","author":"Collins","year":"2013","journal-title":"Am J Crit Care"},{"key":"2020110613091405600_ocz229-B52","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511819322","volume-title":"The Psychology of Survey Response","author":"Tourangeau","year":"2000"},{"issue":"8","key":"2020110613091405600_ocz229-B53","doi-asserted-by":"crossref","first-page":"548","DOI":"10.1016\/j.ijmedinf.2015.03.007","article-title":"Adoption of clinical decision support systems in a developing country: antecedents and outcomes of physician\u2019s threat to perceived professional autonomy","volume":"84","author":"Esmaeilzadeh","year":"2015","journal-title":"Int J Med Inform"},{"issue":"2","key":"2020110613091405600_ocz229-B54","doi-asserted-by":"crossref","first-page":"296","DOI":"10.1016\/j.eswa.2006.05.001","article-title":"Physicians\u2019 acceptance of pharmacokinetics-based clinical decision support systems","volume":"33","author":"Chang","year":"2007","journal-title":"Expert Syst Appl"},{"issue":"2","key":"2020110613091405600_ocz229-B55","doi-asserted-by":"crossref","first-page":"e14","DOI":"10.2196\/humanfactors.4537","article-title":"Usability testing of a complex clinical decision support tool in the emergency department: lessons learned","volume":"2","author":"Press","year":"2015","journal-title":"JMIR Human Factors"},{"issue":"920","key":"2020110613091405600_ocz229-B56","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1136\/pmj.78.920.319","article-title":"Clinical decision-making: coping with uncertainty","volume":"78","author":"West","year":"2002","journal-title":"Postgrad Med J"},{"issue":"4","key":"2020110613091405600_ocz229-B57","doi-asserted-by":"crossref","first-page":"e0193187","DOI":"10.1371\/journal.pone.0193187","article-title":"Acceptance and barriers pertaining to a general practice decision support system for multiple clinical conditions: a mixed methods evaluation","volume":"13","author":"Arts","year":"2018","journal-title":"PLoS One"},{"key":"2020110613091405600_ocz229-B58","first-page":"1045","article-title":"Assessing the motivation of MDs to use computer-based support at the point-of-care in the emergency department","volume":"2011","author":"O'Sullivan","year":"2011","journal-title":"AMIA Annu Symp Proc"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/27\/4\/592\/34153285\/ocz229.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/27\/4\/592\/34153285\/ocz229.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,11,6]],"date-time":"2020-11-06T19:26:03Z","timestamp":1604690763000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/27\/4\/592\/5762808"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,2,27]]},"references-count":58,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2020,2,27]]},"published-print":{"date-parts":[[2020,4,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocz229","relation":{},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2020,4]]},"published":{"date-parts":[[2020,2,27]]}}}