{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,3]],"date-time":"2026-11-03T15:14:47Z","timestamp":1793718887645,"version":"4.6.1"},"reference-count":64,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,1,5]],"date-time":"2021-01-05T00:00:00Z","timestamp":1609804800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,1,5]],"date-time":"2021-01-05T00:00:00Z","timestamp":1609804800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>There is great excitement that medical artificial intelligence (AI) based on machine learning (ML) can be used to improve decision making at the patient level in a variety of healthcare settings. However, the quantification and communication of uncertainty for individual predictions is often neglected even though uncertainty estimates could lead to more principled decision-making and enable machine learning models to automatically or semi-automatically abstain on samples for which there is high uncertainty. In this article, we provide an overview of different approaches to uncertainty quantification and abstention for machine learning and highlight how these techniques could improve the safety and reliability of current ML systems being used in healthcare settings. Effective quantification and communication of uncertainty could help to engender trust with healthcare workers, while providing safeguards against known failure modes of current machine learning approaches. As machine learning becomes further integrated into healthcare environments, the ability to say \u201cI\u2019m not sure\u201d or \u201cI don\u2019t know\u201d when uncertain is a necessary capability to enable safe clinical deployment.<\/jats:p>","DOI":"10.1038\/s41746-020-00367-3","type":"journal-article","created":{"date-parts":[[2021,1,5]],"date-time":"2021-01-05T06:07:08Z","timestamp":1609826828000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":504,"title":["Second opinion needed: communicating uncertainty in medical machine learning"],"prefix":"10.1038","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2961-0270","authenticated-orcid":false,"given":"Benjamin","family":"Kompa","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jasper","family":"Snoek","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6657-2787","authenticated-orcid":false,"given":"Andrew L.","family":"Beam","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,1,5]]},"reference":[{"key":"367_CR1","doi-asserted-by":"publisher","first-page":"1317","DOI":"10.1001\/jama.2017.18391","volume":"319","author":"AL Beam","year":"2018","unstructured":"Beam, A. L. & Kohane, I. S. Big data and machine learning in health care. JAMA 319, 1317\u20131318 (2018).","journal-title":"JAMA"},{"key":"367_CR2","doi-asserted-by":"publisher","first-page":"719","DOI":"10.1038\/s41551-018-0305-z","volume":"2","author":"K-H Yu","year":"2018","unstructured":"Yu, K.-H., Beam, A. L. & Kohane, I. S. Artificial intelligence in healthcare. Nat. Biomed. Eng. 2, 719\u2013731 (2018).","journal-title":"Nat. Biomed. Eng."},{"key":"367_CR3","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1038\/s41591-018-0300-7","volume":"25","author":"EJ Topol","year":"2019","unstructured":"Topol, E. J. High-performance medicine: the convergence of human and artificial intelligence. Nat. Med. 25, 44\u201356 (2019).","journal-title":"Nat. Med."},{"key":"367_CR4","doi-asserted-by":"publisher","unstructured":"Dusenberry, M. W. et al. Analyzing the role of model uncertainty for electronic health records. In Proceedings of the ACM Conference on Health, Inference, and Learning 204\u2013213 (Association for Computing Machinery, 2020). https:\/\/doi.org\/10.1145\/3368555.3384457.","DOI":"10.1145\/3368555.3384457"},{"key":"367_CR5","doi-asserted-by":"publisher","unstructured":"Beam, A. L., Manrai, A. K. & Ghassemi, M. Challenges to the Reproducibility of Machine Learning Models in Health Care. JAMA. https:\/\/doi.org\/10.1001\/jama.2019.20866 (2020).","DOI":"10.1001\/jama.2019.20866"},{"key":"367_CR6","doi-asserted-by":"publisher","unstructured":"Beede, E. et al. A Human-Centered Evaluation of a Deep Learning System Deployed in Clinics for the Detection of Diabetic Retinopathy. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems 1\u201312 (Association for Computing Machinery, 2020). https:\/\/doi.org\/10.1145\/3313831.3376718.","DOI":"10.1145\/3313831.3376718"},{"key":"367_CR7","doi-asserted-by":"publisher","first-page":"2402","DOI":"10.1001\/jama.2016.17216","volume":"316","author":"V Gulshan","year":"2016","unstructured":"Gulshan, V. et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316, 2402\u20132410 (2016).","journal-title":"JAMA"},{"key":"367_CR8","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1038\/nature21056","volume":"542","author":"A Esteva","year":"2017","unstructured":"Esteva, A. et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115\u2013118 (2017).","journal-title":"Nature"},{"key":"367_CR9","unstructured":"Rajpurkar, P. et al. CheXNet: radiologist-level pneumonia detection on chest X-rays with deep learning. Preprint at https:\/\/arxiv.org\/abs\/1711.05225 (2017)."},{"key":"367_CR10","doi-asserted-by":"publisher","first-page":"574","DOI":"10.1148\/radiol.2017162326","volume":"284","author":"P Lakhani","year":"2017","unstructured":"Lakhani, P. & Sundaram, B. Deep learning at chest radiography: automated classification of pulmonary tuberculosis by using convolutional neural networks. Radiology 284, 574\u2013582 (2017).","journal-title":"Radiology"},{"key":"367_CR11","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1016\/j.jclinepi.2019.02.004","volume":"110","author":"E Christodoulou","year":"2019","unstructured":"Christodoulou, E. et al. A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models. J. Clin. Epidemiol. 110, 12\u201322 (2019).","journal-title":"J. Clin. Epidemiol."},{"key":"367_CR12","doi-asserted-by":"publisher","first-page":"895","DOI":"10.1007\/s00381-020-04577-8","volume":"36","author":"H-J Mijderwijk","year":"2020","unstructured":"Mijderwijk, H.-J., Beez, T., H\u00e4nggi, D. & Nieboer, D. Clinical prediction models. Childs Nerv. Syst. 36, 895\u2013897 (2020).","journal-title":"Childs Nerv. Syst."},{"key":"367_CR13","doi-asserted-by":"publisher","first-page":"199","DOI":"10.1214\/ss\/1009213726","volume":"16","author":"L Breiman","year":"2001","unstructured":"Breiman, L. Statistical modeling: the two cultures. Stat. Sci. 16, 199\u2013231 (2001).","journal-title":"Stat. Sci."},{"key":"367_CR14","doi-asserted-by":"publisher","first-page":"167","DOI":"10.1016\/j.jclinepi.2015.12.005","volume":"74","author":"B Van Calster","year":"2016","unstructured":"Van Calster, B. et al. A calibration hierarchy for risk models was defined: from utopia to empirical data. J. Clin. Epidemiol. 74, 167\u2013176 (2016).","journal-title":"J. Clin. Epidemiol."},{"key":"367_CR15","doi-asserted-by":"publisher","first-page":"601","DOI":"10.1002\/bimj.201300297","volume":"56","author":"EW Steyerberg","year":"2014","unstructured":"Steyerberg, E. W., van der Ploeg, T. & Van Calster, B. Risk prediction with machine learning and regression methods. Biom. J. 56, 601\u2013606 (2014).","journal-title":"Biom. J."},{"key":"367_CR16","doi-asserted-by":"publisher","first-page":"162","DOI":"10.1177\/0272989X14547233","volume":"35","author":"B Van Calster","year":"2015","unstructured":"Van Calster, B. & Vickers, A. J. Calibration of risk prediction models: impact on decision-analytic performance. Med. Decis. Mak. 35, 162\u2013169 (2015).","journal-title":"Med. Decis. Mak."},{"key":"367_CR17","doi-asserted-by":"publisher","first-page":"1577","DOI":"10.1016\/S0140-6736(19)30037-6","volume":"393","author":"GS Collins","year":"2019","unstructured":"Collins, G. S. & Moons, K. G. M. Reporting of artificial intelligence prediction models. Lancet 393, 1577\u20131579 (2019).","journal-title":"Lancet"},{"key":"367_CR18","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.e3186","volume":"344","author":"GS Collins","year":"2012","unstructured":"Collins, G. S. & Moons, K. G. M. Comparing risk prediction models. BMJ 344, e3186 (2012).","journal-title":"BMJ"},{"key":"367_CR19","unstructured":"Janosi, A., Steinbrunn, W., Pfisterer, M. & Detrano, R. UCI machine learning repository-heart disease data set. School Inf. Comput. Sci., Univ. California, Irvine, CA, USA (1988)."},{"key":"367_CR20","unstructured":"Gal, Y. Uncertainty in Deep Learning (University of Cambridge, 2016)."},{"key":"367_CR21","unstructured":"Kendall, A. & Gal, Y. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? In Advances in Neural Information Processing Systems 30 (eds. Guyon, I. et al.) 5574\u20135584 (Curran Associates, Inc., 2017)."},{"key":"367_CR22","unstructured":"Kull, M. & Flach, P. Patterns of dataset shift. In First International Workshop on Learning over Multiple Contexts (LMCE) at ECML-PKDD (2014)."},{"key":"367_CR23","doi-asserted-by":"crossref","unstructured":"Quionero-Candela, J., Sugiyama, M., Schwaighofer, A. & Lawrence, N. D. Dataset Shift in Machine Learning (The MIT Press, 2009).","DOI":"10.7551\/mitpress\/9780262170055.001.0001"},{"key":"367_CR24","unstructured":"Ovadia, Y. et al. Can you trust your model\u2019s uncertainty? Evaluating predictive uncertainty under dataset shift. In Advances in Neural Information Processing Systems 13991\u201314002 (2019)."},{"key":"367_CR25","unstructured":"Geisser, S. Predictive Inference (CRC Press, 1993)."},{"key":"367_CR26","first-page":"371","volume":"9","author":"G Shafer","year":"2008","unstructured":"Shafer, G. & Vovk, V. A tutorial on conformal prediction. J. Mach. Learn. Res. 9, 371\u2013421 (2008).","journal-title":"J. Mach. Learn. Res."},{"key":"367_CR27","unstructured":"Barber, R. F., Cand\u00e8s, E. J., Ramdas, A. & Tibshirani, R. J. The limits of distribution-free conditional predictive inference. arXiv [math.ST] (2019)."},{"key":"367_CR28","unstructured":"Barber, R. F., Candes, E. J., Ramdas, A. & Tibshirani, R. J. Conformal Prediction Under Covariate Shift. In 33rd Conference on Neural Information Processing Systems (2019)."},{"key":"367_CR29","unstructured":"Gal, Y. & Ghahramani, Z. Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. In International Conference on Machine Learning 1050\u20131059 (2016)."},{"key":"367_CR30","unstructured":"Lakshminarayanan, B., Pritzel, A. & Blundell, C. Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles. In 31st Conference on Neural Information Processing Systems (2017)."},{"key":"367_CR31","unstructured":"Wen, Y., Tran, D. & Ba, J. BatchEnsemble: Efficient Ensemble of Deep Neural Networks via Rank-1 Perturbation. In Eighth International Conference on Learning Representations (2020)."},{"key":"367_CR32","unstructured":"Ashukha, A., Lyzhov, A., Molchanov, D. & Vetrov, D. Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning. In Eighth International Conference on Learning Representations (2020)."},{"key":"367_CR33","doi-asserted-by":"publisher","unstructured":"Dietterich, T. G. Ensemble Methods in Machine Learning. In Multiple Classifier Systems 1\u201315 (Springer Berlin Heidelberg, 2000). https:\/\/doi.org\/10.1007\/3-540-45014-9_1.","DOI":"10.1007\/3-540-45014-9_1"},{"key":"367_CR34","unstructured":"Neal, R. M. Bayesian Learning for Neural Networksributed to the conception, writing, and editing (Springer Science & Business Media, 2012)."},{"key":"367_CR35","unstructured":"Wilson, A. G. The case for Bayesian deep learning. Preprint at https:\/\/arxiv.org\/abs\/2001.10995 (2020)."},{"key":"367_CR36","unstructured":"Graves, A. Practical Variational Inference for Neural Networks. In Advances in Neural Information Processing Systems 24 (eds. Shawe-Taylor, J., Zemel, R. S., Bartlett, P. L., Pereira, F. & Weinberger, K. Q.) 2348\u20132356 (Curran Associates, Inc., 2011)."},{"key":"367_CR37","unstructured":"Blundell, C., Cornebise, J., Kavukcuoglu, K. & Wierstra, D. Weight Uncertainty in Neural Networks. In 32nd International Conference on Machine Learning (2015)."},{"key":"367_CR38","unstructured":"Louizos, C. & Welling, M. Multiplicative normalizing flows for variational bayesian neural networks. In Proceedings of the 34th International Conference on Machine Learning (2017)."},{"key":"367_CR39","unstructured":"Louizos, C. & Welling, M. Structured and Efficient Variational Deep Learning with Matrix Gaussian Posteriors. In International Conference on Machine Learning 1708\u20131716 (2016)."},{"key":"367_CR40","unstructured":"Hern\u00e1ndez-Lobato, J. M. et al. Black-box \u03b1-divergence Minimization. In Proceedings of the 33rd International Conference on Machine Learning (2016)."},{"key":"367_CR41","first-page":"1861","volume":"37","author":"JM Hernandez-Lobato","year":"2015","unstructured":"Hernandez-Lobato, J. M. & Adams, R. Probabilistic backpropagation for scalable learning of Bayesian neural networks. Proc. Mach. Learn. Res. 37, 1861\u20131869 (2015).","journal-title":"Proc. Mach. Learn. Res."},{"key":"367_CR42","unstructured":"Pawlowski, N., Brock, A., Lee, M. C. H., Rajchl, M. & Glocker, B. Implicit weight uncertainty in neural networks. Preprint at https:\/\/arxiv.org\/abs\/1711.01297 (2017)."},{"key":"367_CR43","doi-asserted-by":"publisher","first-page":"859","DOI":"10.1080\/01621459.2017.1285773","volume":"112","author":"DM Blei","year":"2017","unstructured":"Blei, D. M., Kucukelbir, A. & McAuliffe, J. D. Variational inference: a review for statisticians. J. Am. Stat. Assoc. 112, 859\u2013877 (2017).","journal-title":"J. Am. Stat. Assoc."},{"key":"367_CR44","unstructured":"Yao, J., Pan, W., Ghosh, S. & Doshi-Velez, F. Quality of uncertainty quantification for Bayesian neural network inference. Preprint at https:\/\/arxiv.org\/abs\/1906.09686 (2019)."},{"key":"367_CR45","unstructured":"Wenzel, F. et al. How Good is the Bayes Posterior in Deep Neural Networks Really? In Proceedings of the 37th International Conference on Machine Learning (2020)."},{"key":"367_CR46","doi-asserted-by":"publisher","first-page":"359","DOI":"10.1198\/016214506000001437","volume":"102","author":"T Gneiting","year":"2007","unstructured":"Gneiting, T. & Raftery, A. E. Strictly proper scoring rules, prediction, and estimation. J. Am. Stat. Assoc. 102, 359\u2013378 (2007).","journal-title":"J. Am. Stat. Assoc."},{"key":"367_CR47","unstructured":"Williams, C. K. I. & Rasmussen, C. E. Gaussian processes for machine learning. vol. 2 (MIT Press Cambridge, MA, 2006)."},{"key":"367_CR48","unstructured":"Lee, J. et al. Deep Neural Networks as Gaussian Processes. In International Conference on Learning Representations (2018)."},{"key":"367_CR49","unstructured":"Novak, R. et al. Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes. In International Conference on Learning Representations (2018)."},{"key":"367_CR50","unstructured":"de G. Matthews, A. G., Hron, J., Rowland, M., Turner, R. E. & Ghahramani, Z. Gaussian Process Behaviour in Wide Deep Neural Networks. In International Conference on Learning Representations (2018)."},{"key":"367_CR51","doi-asserted-by":"crossref","unstructured":"Sugiyama, M. & Storkey, A. J. Mixture Regression for Covariate Shift. In Advances in Neural Information Processing Systems 19 (eds. Sch\u00f6lkopf, B., Platt, J. C. & Hoffman, T.) 1337\u20131344 (MIT Press, 2007).","DOI":"10.7551\/mitpress\/7503.003.0172"},{"key":"367_CR52","unstructured":"Geifman, Y. & El-Yaniv, R. SelectiveNet: A Deep Neural Network with an Integrated Reject Option. In Proceedings of the 36th International Conference on Machine Learning (2019)."},{"key":"367_CR53","unstructured":"Shrikumar, A., Alexandari, A. & Kundaje, A. A flexible and adaptive framework for abstention under class imbalance. Preprint at https:\/\/arxiv.org\/abs\/1802.07024 (2018)."},{"key":"367_CR54","doi-asserted-by":"publisher","first-page":"530","DOI":"10.1214\/17-EJS1388","volume":"12","author":"HG Ramaswamy","year":"2018","unstructured":"Ramaswamy, H. G., Tewari, A. & Agarwal, S. Consistent algorithms for multiclass classification with an abstain option. Electron. J. Stat. 12, 530\u2013554 (2018).","journal-title":"Electron. J. Stat."},{"key":"367_CR55","doi-asserted-by":"publisher","first-page":"247","DOI":"10.1109\/TEC.1957.5222035","volume":"EC-6","author":"CK Chow","year":"1957","unstructured":"Chow, C. K. An optimum character recognition system using decision functions. IRE Trans. Electron. Computers EC-6, 247\u2013254 (1957).","journal-title":"IRE Trans. Electron. Computers"},{"key":"367_CR56","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1109\/TIT.1970.1054406","volume":"16","author":"C Chow","year":"1970","unstructured":"Chow, C. On optimum recognition error and reject tradeoff. IEEE Trans. Inf. Theory 16, 41\u201346 (1970).","journal-title":"IEEE Trans. Inf. Theory"},{"key":"367_CR57","doi-asserted-by":"publisher","first-page":"159","DOI":"10.1023\/A:1009643503022","volume":"4","author":"LK Hansen","year":"1997","unstructured":"Hansen, L. K., Liisberg, C. & Salamon, P. The error-reject tradeoff. Open Syst. Inf. Dyn. 4, 159\u2013184 (1997).","journal-title":"Open Syst. Inf. Dyn."},{"key":"367_CR58","doi-asserted-by":"publisher","unstructured":"Tortorella, F. An Optimal Reject Rule for Binary Classifiers. In Advances in Pattern Recognition 611\u2013620 (Springer Berlin Heidelberg, 2000). https:\/\/doi.org\/10.1007\/3-540-44522-6_63.","DOI":"10.1007\/3-540-44522-6_63"},{"key":"367_CR59","unstructured":"Mozannar, H. & Sontag, D. Consistent Estimators for Learning to Defer to an Expert. In Proceedings of the 37th International Conference on Machine Learning (2020)."},{"key":"367_CR60","unstructured":"Geifman, Y. & El-Yaniv, R. Selective classification for deep neural networks. Preprint at https:\/\/arxiv.org\/abs\/1705.08500 (2017)."},{"key":"367_CR61","unstructured":"Ziyin, L. et al. Deep Gamblers: Learning to Abstain with Portfolio Theory. In 33rd Conference on Neural Information Processing Systems (2019)."},{"key":"367_CR62","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pmed.1002683","volume":"15","author":"JR Zech","year":"2018","unstructured":"Zech, J. R. et al. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study. PLoS Med. 15, e1002683 (2018).","journal-title":"PLoS Med."},{"key":"367_CR63","doi-asserted-by":"publisher","first-page":"1287","DOI":"10.1126\/science.aaw4399","volume":"363","author":"SG Finlayson","year":"2019","unstructured":"Finlayson, S. G. et al. Adversarial attacks on medical machine learning. Science 363, 1287\u20131289 (2019).","journal-title":"Science"},{"key":"367_CR64","unstructured":"Tagasovska, N. & Lopez-Paz, D. Frequentist uncertainty estimates for deep learning. Preprint at https:\/\/arxiv.org\/pdf\/2006.13707.pdf (2018)."}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-00367-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-00367-3","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-00367-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,16]],"date-time":"2023-10-16T18:28:37Z","timestamp":1697480917000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-00367-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,1,5]]},"references-count":64,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2021,12]]}},"alternative-id":["367"],"URL":"https:\/\/doi.org\/10.1038\/s41746-020-00367-3","relation":{},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,1,5]]},"assertion":[{"value":"23 April 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 November 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 January 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"4"}}