{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,4]],"date-time":"2026-11-04T19:32:13Z","timestamp":1793820733494,"version":"4.6.1"},"reference-count":50,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2025,1,31]],"date-time":"2025-01-31T00:00:00Z","timestamp":1738281600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100002347","name":"Federal Ministry of Education and Research of Germany","doi-asserted-by":"publisher","award":["ScaDS.AI"],"award-info":[{"award-number":["ScaDS.AI"]}],"id":[{"id":"10.13039\/501100002347","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JPM"],"abstract":"<jats:p>Introduction: Widespread adoption of AI for medical decision-making is still hindered due to ethical and safety-related concerns. For AI-based decision support systems in healthcare settings, it is paramount to be reliable and trustworthy. Common deep learning approaches, however, have the tendency towards overconfidence when faced with unfamiliar or changing conditions. Inappropriate extrapolation beyond well-supported scenarios may have dire consequences highlighting the importance of the reliable estimation of local knowledge uncertainty and its communication to the end user. Materials and Methods: While neural network ensembles (ENNs) have been heralded as a potential solution to these issues for many years, deep learning methods, specifically modeling the amount of knowledge, promise more principled and reliable behavior. This study compares their reliability in clinical applications. We centered our analysis on experiments with low-dimensional toy datasets and the exemplary case study of mortality prediction for intensive care unit hospitalizations using Electronic Health Records (EHRs) from the MIMIC3 study. For predictions on the EHR time series, Encoder-Only Transformer models were employed. Knowledge uncertainty estimation is achieved with both ensemble and Spectral Normalized Neural Gaussian Process (SNGP) variants of the common Transformer model. We designed two datasets to test their reliability in detecting token level and more subtle discrepancies both for toy datasets and an EHR dataset. Results: While both SNGP and ENN model variants achieve similar prediction performance (AUROC: \u22480.85, AUPRC: \u22480.52 for in-hospital mortality prediction from a selected MIMIC3 benchmark), the former demonstrates improved capabilities to quantify knowledge uncertainty for individual samples\/patients. Discussion\/Conclusions: Methods including a knowledge model, such as SNGP, offer superior uncertainty estimation compared to traditional stochastic deep learning, leading to more trustworthy and safe clinical decision support.<\/jats:p>","DOI":"10.3390\/jpm15020058","type":"journal-article","created":{"date-parts":[[2025,1,31]],"date-time":"2025-01-31T05:08:26Z","timestamp":1738300106000},"page":"58","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":20,"title":["Towards Trustworthy AI in Healthcare: Epistemic Uncertainty Estimation for Clinical Decision Support"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8977-5790","authenticated-orcid":false,"given":"Adrian","family":"Lindenmeyer","sequence":"first","affiliation":[{"name":"Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI) Dresden\/Leipzig, Leipzig University, Humboldtstra\u00dfe 25, 04105 Leipzig, Germany"},{"name":"Innovation Center Computer Assisted Surgery (ICCAS), Leipzig University, Semmelweisstrasse 14, 04103 Leipzig, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2282-0723","authenticated-orcid":false,"given":"Malte","family":"Blattmann","sequence":"additional","affiliation":[{"name":"Innovation Center Computer Assisted Surgery (ICCAS), Leipzig University, Semmelweisstrasse 14, 04103 Leipzig, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stefan","family":"Franke","sequence":"additional","affiliation":[{"name":"Innovation Center Computer Assisted Surgery (ICCAS), Leipzig University, Semmelweisstrasse 14, 04103 Leipzig, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6999-5024","authenticated-orcid":false,"given":"Thomas","family":"Neumuth","sequence":"additional","affiliation":[{"name":"Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI) Dresden\/Leipzig, Leipzig University, Humboldtstra\u00dfe 25, 04105 Leipzig, Germany"},{"name":"Innovation Center Computer Assisted Surgery (ICCAS), Leipzig University, Semmelweisstrasse 14, 04103 Leipzig, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3273-9283","authenticated-orcid":false,"given":"Daniel","family":"Schneider","sequence":"additional","affiliation":[{"name":"Innovation Center Computer Assisted Surgery (ICCAS), Leipzig University, Semmelweisstrasse 14, 04103 Leipzig, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,1,31]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"47","DOI":"10.2147\/RMHP.S12985","article-title":"Benefits and Drawbacks of Electronic Health Record Systems","volume":"4","author":"Menachemi","year":"2011","journal-title":"Risk Manag. Healthc. Policy"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1186\/2047-2501-2-3","article-title":"Big Data Analytics in Healthcare: Promise and Potential","volume":"2","author":"Raghupathi","year":"2014","journal-title":"Health Inf. Sci. Syst."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Ristevski, B., and Chen, M. (2018). Big Data Analytics in Medicine and Healthcare. J. Integr. Bioinform., 15.","DOI":"10.1515\/jib-2017-0030"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1109\/TBME.2016.2573285","article-title":"\u2013Omic and Electronic Health Record Big Data Analytics for Precision Medicine","volume":"64","author":"Wu","year":"2017","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"416","DOI":"10.1139\/gen-2020-0131","article-title":"Machine Learning for Precision Medicine","volume":"64","author":"MacEachern","year":"2021","journal-title":"Genome"},{"key":"ref_6","first-page":"9516","article-title":"Building Trust in Deep Learning System towards Automated Disease Detection","volume":"33","author":"Lim","year":"2019","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Dusenberry, M.W., Tran, D., Choi, E., Kemp, J., Nixon, J., Jerfel, G., Heller, K., and Dai, A.M. (2020, January 2\u20134). Analyzing the Role of Model Uncertainty for Electronic Health Records. Proceedings of the ACM Conference on Health, Inference, and Learning, Toronto, ON, Canada.","DOI":"10.1145\/3368555.3384457"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1038\/s41746-020-00367-3","article-title":"Second Opinion Needed: Communicating Uncertainty in Medical Machine Learning","volume":"4","author":"Kompa","year":"2021","journal-title":"Npj Digit. Med."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1589","DOI":"10.1109\/JBHI.2017.2767063","article-title":"Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis","volume":"22","author":"Shickel","year":"2018","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1038\/s42256-018-0004-1","article-title":"The Need for Uncertainty Quantification in Machine-Assisted Medical Decision Making","volume":"1","author":"Begoli","year":"2019","journal-title":"Nat. Mach. Intell."},{"key":"ref_11","unstructured":"Gal, Y. (2016). Uncertainty in Deep Learning. [Ph.D. Thesis, University of Cambridge]."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1136\/bmjqs-2018-008370","article-title":"Artificial intelligence, bias and clinical safety","volume":"28","author":"Challen","year":"2019","journal-title":"Bmj Qual. Saf."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1016\/j.inffus.2021.05.008","article-title":"A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges","volume":"76","author":"Abdar","year":"2021","journal-title":"Inf. Fusion"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"100347","DOI":"10.1016\/j.patter.2021.100347","article-title":"Addressing bias in big data and AI for health care: A call for open science","volume":"2","author":"Norori","year":"2021","journal-title":"Patterns"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1513","DOI":"10.1007\/s10462-023-10562-9","article-title":"A survey of uncertainty in deep neural networks","volume":"56","author":"Gawlikowski","year":"2023","journal-title":"Artif. Intell. Rev."},{"key":"ref_16","first-page":"e45255","article-title":"Clinicians\u2019 views on using artificial intelligence in healthcare: Opportunities, challenges, and beyond","volume":"15","author":"Alanazi","year":"2023","journal-title":"Cureus"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1126\/science.130.3366.9","article-title":"Reasoning Foundations of Medical Diagnosis: Symbolic Logic, Probability, and Value Theory aid our Understanding of how Physicians reason","volume":"130","author":"Ledley","year":"1959","journal-title":"Science"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1038\/s41746-021-00455-y","article-title":"Med-BERT: Pretrained Contextualized Embeddings on Large-Scale Structured Electronic Health Records for Disease Prediction","volume":"4","author":"Rasmy","year":"2021","journal-title":"Npj Digit. Med."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Si, Y., Du, J., Li, Z., Jiang, X., Miller, T., Wang, F., Jim Zheng, W., and Roberts, K. (2021). Deep Representation Learning of Patient Data from Electronic Health Records (EHR): A Systematic Review. J. Biomed. Inform., 115.","DOI":"10.1016\/j.jbi.2020.103671"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"e24813","DOI":"10.2196\/24813","article-title":"Adoption of Electronic Health Records (EHRs) in China during the past 10 Years: Consecutive Survey Data Analysis and Comparison of Sino-American Challenges and Experiences","volume":"23","author":"Liang","year":"2021","journal-title":"J. Med. Internet Res."},{"key":"ref_21","first-page":"2008","article-title":"Adoption of Electronic Health Record Systems among US Non-Federal Acute Care Hospitals: 2008\u20132015","volume":"35","author":"Henry","year":"2016","journal-title":"ONC Data Brief"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Song, H., Rajan, D., Thiagarajan, J.J., and Spanias, A. (2017). Attend and Diagnose: Clinical Time Series Analysis using Attention Models. arXiv.","DOI":"10.1609\/aaai.v32i1.11635"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Shang, J., Ma, T., Xiao, C., and Sun, J. (2019, January 10\u201316). Pre-Training of Graph Augmented Transformers for Medication Recommendation. Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, Macao, China.","DOI":"10.24963\/ijcai.2019\/825"},{"key":"ref_24","unstructured":"Tonekaboni, S., Joshi, S., McCradden, M.D., and Goldenberg, A. (2019, January 9\u201310). What Clinicians want: Contextualizing Explainable Machine Learning for Clinical End Use. Proceedings of the 4th Machine Learning for Healthcare Conference, Ann Arbor, MI, USA."},{"key":"ref_25","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017, January 4\u20139). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_26","unstructured":"Devlin, J., Chang, M., Lee, K., and Toutanova, K. (August, January 28). BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Florence, Italy."},{"key":"ref_27","unstructured":"Choi, E., Bahadori, M.T., Schuetz, A., Stewart, W.F., and Sun, J. (2016, January 19\u201320). Doctor AI: Predicting Clinical Events via Recurrent Neural Networks. Proceedings of the 1st Machine Learning for Healthcare Conference, Los Angeles, CA, USA."},{"key":"ref_28","unstructured":"Choi, E., Bahadori, M.T., Kulas, J.A., Schuetz, A., Stewart, W.F., and Sun, J. (2016, January 5\u201310). RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism. Proceedings of the 30th International Conference on Neural Information Processing Systems, Barcelona, Spain."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1038\/s41597-019-0103-9","article-title":"Multitask Learning and Benchmarking with Clinical Time Series Data","volume":"6","author":"Harutyunyan","year":"2019","journal-title":"Sci. Data"},{"key":"ref_30","unstructured":"Johnson, A.E.W., Pollard, T.J., and Mark, R.G. (2016). MIMIC-III Clinical Database (Version 1.4), PhysioNet."},{"key":"ref_31","unstructured":"Qiu, R. (2020). Modeling Uncertainty in Deep Learning Models of Electronic Health Records. [Ph.D. Thesis, University of North Carolina at Charlotte]."},{"key":"ref_32","unstructured":"Qiu, R., Jia, Y., Hadzikadic, M., Dulin, M., Niu, X., and Wang, X. (2019). Modeling the Uncertainty in Electronic Health Records: A Bayesian Deep Learning Approach. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Peng, X., Long, G., Shen, T., Wang, S., Jiang, J., and Blumenstein, M. (2019, January 8\u201311). Temporal Self-Attention Network for Medical Concept Embedding. Proceedings of the 2019 IEEE International Conference on Data Mining, Beijing, China.","DOI":"10.1109\/ICDM.2019.00060"},{"key":"ref_34","first-page":"239","article-title":"CEHR-BERT: Incorporating Temporal Information from Structured EHR Data to improve Prediction Tasks","volume":"158","author":"Pang","year":"2021","journal-title":"Proc. Mach. Learn. Health"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Li, Y., Rao, S., Solares, J.R.A., Hassaine, A., Ramakrishnan, R., Canoy, D., Zhu, Y., Rahimi, K., and Salimi-Khorshidi, G. (2020). BEHRT: Transformer for Electronic Health Records. Sci. Rep., 10.","DOI":"10.1038\/s41598-020-62922-y"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Blinov, P., Avetisian, M., Kokh, V., Umerenkov, D., and Tuzhilin, A. (2020, January 25\u201328). Predicting Clinical Diagnosis from Patients Electronic Health Records using BERT-Based Neural Networks. Proceedings of the Artificial Intelligence in Medicine: 18th International Conference on Artificial Intelligence in Medicine, Minneapolis, MN, USA.","DOI":"10.1007\/978-3-030-59137-3_11"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"3268","DOI":"10.1109\/JBHI.2020.2984931","article-title":"TAPER: Time-Aware Patient EHR Representation","volume":"24","author":"Darabi","year":"2020","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Wang, Y., Xu, X., Jin, T., Li, X., Xie, G., and Wang, J. (2019, January 18\u201321). Inpatient2Vec: Medical Representation Learning for Inpatients. Proceedings of the 2019 IEEE International Conference on Bioinformatics and Biomedicine, San Diego, CA, USA.","DOI":"10.1109\/BIBM47256.2019.8983281"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Popat, R., and Ive, J. (2023). Embracing the uncertainty in human\u2013machine collaboration to support clinical decision-making for mental health conditions. Front. Digit. Health, 5.","DOI":"10.3389\/fdgth.2023.1188338"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Wu, H., Shi, W., Choudhary, A., and Wang, M.D. (2024). Clinical decision making under uncertainty: A bootstrapped counterfactual inference approach. BMC Med. Inform. Decis. Mak., 24.","DOI":"10.1186\/s12911-024-02606-z"},{"key":"ref_41","unstructured":"Liu, J., Lin, Z., Padhy, S., Tran, D., Bedrax Weiss, T., and Lakshminarayanan, B. (2020, January 6\u201312). Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness. Proceedings of the Advances in Neural Information Processing Systems, Online."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Sullivan, T.J. (2015). Measures of Information and Uncertainty. Introduction to Uncertainty Quantification, Springer International Publishing.","DOI":"10.1007\/978-3-319-23395-6_5"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Smith, R.C. (2013). Uncertainty Quantification: Theory, Implementation, and Applications, Society for Industrial and Applied Mathematics.","DOI":"10.1137\/1.9781611973228"},{"key":"ref_44","unstructured":"Wenzel, F., Snoek, J., Tran, D., and Jenatton, R. (2020, January 6\u201312). Hyperparameter Ensembles for Robustness and Uncertainty Quantification. Proceedings of the Advances in Neural Information Processing Systems, Online."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"160035","DOI":"10.1038\/sdata.2016.35","article-title":"MIMIC-III, a freely accessible Critical Care Database","volume":"3","author":"Johnson","year":"2016","journal-title":"Sci. Data"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"e215","DOI":"10.1161\/01.CIR.101.23.e215","article-title":"PhysioBank, PhysioToolkit, and PhysioNet: Components of a new Research Resource for Complex Physiologic Signals","volume":"101","author":"Goldberger","year":"2000","journal-title":"Circulation"},{"key":"ref_47","unstructured":"Kim, H., Papamakarios, G., and Mnih, A. (2021). The Lipschitz Constant of Self-Attention. arXiv."},{"key":"ref_48","unstructured":"Huang, Y., Zhang, H., Shi, Y., Kolter, J.Z., and Anandkumar, A. (2021). Training Certifiably Robust Neural Networks with Efficient Local Lipschitz Bounds. arXiv."},{"key":"ref_49","unstructured":"Ye, W., Ma, Y., Cao, X., and Tang, K. (August, January 31). Mitigating Transformer Overconfidence via Lipschitz Regularization. Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence, PMLR, Pittsburgh, PA, USA. Proceedings of Machine Learning Research."},{"key":"ref_50","unstructured":"Qi, X., Wang, J., Chen, Y., Shi, Y., and Zhang, L. (2023). LipsFormer: Introducing Lipschitz Continuity to Vision Transformers. arXiv."}],"container-title":["Journal of Personalized Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4426\/15\/2\/58\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T16:24:48Z","timestamp":1760027088000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4426\/15\/2\/58"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,31]]},"references-count":50,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,2]]}},"alternative-id":["jpm15020058"],"URL":"https:\/\/doi.org\/10.3390\/jpm15020058","relation":{},"ISSN":["2075-4426"],"issn-type":[{"value":"2075-4426","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,31]]}}}