{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,7]],"date-time":"2026-05-07T19:26:12Z","timestamp":1778181972255,"version":"3.51.4"},"reference-count":105,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,5,31]],"date-time":"2022-05-31T00:00:00Z","timestamp":1653955200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,5,31]],"date-time":"2022-05-31T00:00:00Z","timestamp":1653955200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100005831","name":"Foundation for Anesthesia Education and Research","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100005831","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000070","name":"U.S. Department of Health & Human Services | NIH | National Institute of Biomedical Imaging and Bioengineering","doi-asserted-by":"publisher","award":["R01EB017205"],"award-info":[{"award-number":["R01EB017205"]}],"id":[{"id":"10.13039\/100000070","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Machine learning (ML) and artificial intelligence (AI) algorithms have the potential to derive insights from clinical data and improve patient outcomes. However, these highly complex systems are sensitive to changes in the environment and liable to performance decay. Even after their successful integration into clinical practice, ML\/AI algorithms should be continuously monitored and updated to ensure their long-term safety and effectiveness. To bring AI into maturity in clinical care, we advocate for the creation of hospital units responsible for quality assurance and improvement of these algorithms, which we refer to as \u201cAI-QI\u201d units. We discuss how tools that have long been used in hospital quality assurance and quality improvement can be adapted to monitor static ML algorithms. On the other hand, procedures for continual model updating are still nascent. We highlight key considerations when choosing between existing methods and opportunities for methodological innovation.<\/jats:p>","DOI":"10.1038\/s41746-022-00611-y","type":"journal-article","created":{"date-parts":[[2022,5,31]],"date-time":"2022-05-31T10:02:33Z","timestamp":1653991353000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":321,"title":["Clinical artificial intelligence quality improvement: towards continual monitoring and updating of AI algorithms in healthcare"],"prefix":"10.1038","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2041-3104","authenticated-orcid":false,"given":"Jean","family":"Feng","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8474-591X","authenticated-orcid":false,"given":"Rachael V.","family":"Phillips","sequence":"additional","affiliation":[]},{"given":"Ivana","family":"Malenica","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5427-482X","authenticated-orcid":false,"given":"Andrew","family":"Bishara","sequence":"additional","affiliation":[]},{"given":"Alan E.","family":"Hubbard","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6712-6626","authenticated-orcid":false,"given":"Leo A.","family":"Celi","sequence":"additional","affiliation":[]},{"given":"Romain","family":"Pirracchio","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,5,31]]},"reference":[{"key":"611_CR1","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1038\/s41591-018-0268-3","volume":"25","author":"AY Hannun","year":"2019","unstructured":"Hannun, A. Y. et al. Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network. Nat. Med. 25, 65\u201369 (2019).","journal-title":"Nat. Med."},{"key":"611_CR2","doi-asserted-by":"publisher","first-page":"24","DOI":"10.1038\/s41591-018-0316-z","volume":"25","author":"A Esteva","year":"2019","unstructured":"Esteva, A. et al. A guide to deep learning in healthcare. Nat. Med. 25, 24\u201329 (2019).","journal-title":"Nat. Med."},{"key":"611_CR3","doi-asserted-by":"publisher","first-page":"377","DOI":"10.1016\/j.accpm.2018.09.008","volume":"38","author":"R Pirracchio","year":"2019","unstructured":"Pirracchio, R. et al. Big data and targeted machine learning in action to assist medical decision in the ICU. Anaesth. Crit Care Pain Med. 38, 377\u2013384 (2019).","journal-title":"Anaesth. Crit Care Pain Med."},{"key":"611_CR4","doi-asserted-by":"publisher","first-page":"e18477","DOI":"10.2196\/18477","volume":"22","author":"S Liu","year":"2020","unstructured":"Liu, S. et al. Reinforcement learning for clinical decision support in critical care: comprehensive review. J. Med. Internet Res. 22, e18477 (2020).","journal-title":"J. Med. Internet Res."},{"key":"611_CR5","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1542\/hpeds.2021-006094","volume":"12","author":"CO Adegboro","year":"2022","unstructured":"Adegboro, C. O., Choudhury, A., Asan, O. & Kelly, M. M. Artificial intelligence to improve health outcomes in the NICU and PICU: a systematic review. Hosp Pediatr 12, 93\u2013110 (2022).","journal-title":"Hosp Pediatr"},{"key":"611_CR6","doi-asserted-by":"publisher","first-page":"e18599","DOI":"10.2196\/18599","volume":"8","author":"A Choudhury","year":"2020","unstructured":"Choudhury, A. & Asan, O. Role of artificial intelligence in patient safety outcomes: systematic literature review. JMIR Med Inform. 8, e18599 (2020).","journal-title":"JMIR Med Inform."},{"key":"611_CR7","doi-asserted-by":"publisher","first-page":"118","DOI":"10.1038\/s41746-020-00324-0","volume":"3","author":"S Benjamens","year":"2020","unstructured":"Benjamens, S., Dhunnoo, P. & Mesk\u00f3, B. The state of artificial intelligence-based (fda-approved) medical devices and algorithms: an online database. NPJ Digit Med 3, 118 (2020).","journal-title":"NPJ Digit Med"},{"key":"611_CR8","unstructured":"Sculley, D. et al. Machine Learning: The High Interest Credit Card of Technical Debt. In Advances In Neural Information Processing Systems, vol. 28 (eds. Cortes, C., Lawrence, N., Lee, D., Sugiyama, M. & Garnett, R.) (Curran Associates, Inc., 2015)."},{"key":"611_CR9","doi-asserted-by":"publisher","first-page":"1052","DOI":"10.1093\/jamia\/ocx030","volume":"24","author":"SE Davis","year":"2017","unstructured":"Davis, S. E., Lasko, T. A., Chen, G., Siew, E. D. & Matheny, M. E. Calibration drift in regression and machine learning models for acute kidney injury. J. Am. Med. Inform. Assoc. 24, 1052\u20131061 (2017).","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"611_CR10","doi-asserted-by":"publisher","first-page":"71","DOI":"10.1016\/j.ijmedinf.2017.03.006","volume":"102","author":"JH Chen","year":"2017","unstructured":"Chen, J. H., Alagappan, M., Goldstein, M. K., Asch, S. M. & Altman, R. B. Decaying relevance of clinical data towards future decisions in data-driven inpatient clinical order sets. Int. J. Med. Inform. 102, 71\u201379 (2017).","journal-title":"Int. J. Med. Inform."},{"key":"611_CR11","first-page":"381","volume":"106","author":"B Nestor","year":"2019","unstructured":"Nestor, B. et al. Feature robustness in non-stationary health records: caveats to deployable model performance in common clinical machine learning tasks. Machine Learning for Healthcare 106, 381\u2013405 (2019).","journal-title":"Machine Learning for Healthcare"},{"key":"611_CR12","doi-asserted-by":"publisher","first-page":"163","DOI":"10.1055\/s-0038-1632397","volume":"9","author":"E Yoshida","year":"2018","unstructured":"Yoshida, E., Fei, S., Bavuso, K., Lagor, C. & Maviglia, S. The value of monitoring clinical decision support interventions. Appl. Clin. Inform. 9, 163\u2013173 (2018).","journal-title":"Appl. Clin. Inform."},{"key":"611_CR13","doi-asserted-by":"publisher","first-page":"e279","DOI":"10.1016\/S2589-7500(20)30102-3","volume":"2","author":"CS Lee","year":"2020","unstructured":"Lee, C. S. & Lee, A. Y. Clinical applications of continual learning machine learning. Lancet Digital Health 2, e279\u2013e281 (2020).","journal-title":"Lancet Digital Health"},{"key":"611_CR14","doi-asserted-by":"publisher","first-page":"e337","DOI":"10.1016\/S2589-7500(21)00076-5","volume":"3","author":"KN Vokinger","year":"2021","unstructured":"Vokinger, K. N., Feuerriegel, S. & Kesselheim, A. S. Continual learning in medical devices: FDA\u2019s action plan and beyond. Lancet Digital Health 3, e337\u2013e338 (2021).","journal-title":"Lancet Digital Health"},{"key":"611_CR15","unstructured":"U.S. Food and Drug Administration. Proposed regulatory framework for modifications to artificial intelligence\/machine learning (AI\/ML)-based software as a medical device (SaMD): discussion paper and request for feedback. Tech. Rep. (2019)."},{"key":"611_CR16","doi-asserted-by":"publisher","first-page":"1806","DOI":"10.1001\/jama.2019.16489","volume":"322","author":"Y Liu","year":"2019","unstructured":"Liu, Y., Chen, P.-H. C., Krause, J. & Peng, L. How to read articles that use machine learning: Users\u2019 guides to the medical literature. JAMA 322, 1806\u20131816 (2019).","journal-title":"JAMA"},{"key":"611_CR17","doi-asserted-by":"publisher","first-page":"283","DOI":"10.1056\/NEJMc2104626","volume":"385","author":"SG Finlayson","year":"2021","unstructured":"Finlayson, S. G. et al. The clinician and dataset shift in artificial intelligence. N. Engl. J. Med. 385, 283\u2013286 (2021).","journal-title":"N. Engl. J. Med."},{"key":"611_CR18","doi-asserted-by":"crossref","unstructured":"Breck, E., Cai, S., Nielsen, E., Salib, M. & Sculley, D. The ML test score: A rubric for ML production readiness and technical debt reduction. In: 2017 IEEE International Conference on Big Data (Big Data), 1123\u20131132 (ieeexplore.ieee.org, 2017).","DOI":"10.1109\/BigData.2017.8258038"},{"key":"611_CR19","doi-asserted-by":"crossref","unstructured":"Amershi, S. et al. Software engineering for machine learning: a case study. In: 2019 IEEE\/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), 291\u2013300 (2019).","DOI":"10.1109\/ICSE-SEIP.2019.00042"},{"key":"611_CR20","doi-asserted-by":"publisher","first-page":"458","DOI":"10.1136\/qhc.12.6.458","volume":"12","author":"JC Benneyan","year":"2003","unstructured":"Benneyan, J. C., Lloyd, R. C. & Plsek, P. E. Statistical process control as a tool for research and healthcare improvement. Qual. Saf. Health Care 12, 458\u2013464 (2003).","journal-title":"Qual. Saf. Health Care"},{"key":"611_CR21","doi-asserted-by":"publisher","first-page":"387","DOI":"10.1136\/qshc.2006.022194","volume":"16","author":"J Thor","year":"2007","unstructured":"Thor, J. et al. Application of statistical process control in healthcare improvement: systematic review. Qual. Saf. Health Care 16, 387\u2013399 (2007).","journal-title":"Qual. Saf. Health Care"},{"key":"611_CR22","doi-asserted-by":"publisher","first-page":"m865","DOI":"10.1136\/bmj.m865","volume":"368","author":"A Backhouse","year":"2020","unstructured":"Backhouse, A. & Ogunlayi, F. Quality improvement into practice. BMJ 368, m865 (2020).","journal-title":"BMJ"},{"key":"611_CR23","doi-asserted-by":"publisher","first-page":"663","DOI":"10.1097\/ALN.0000000000002300","volume":"129","author":"F Hatib","year":"2018","unstructured":"Hatib, F. et al. Machine-learning algorithm to predict hypotension based on high-fidelity arterial pressure waveform analysis. Anesthesiology 129, 663\u2013674 (2018).","journal-title":"Anesthesiology"},{"key":"611_CR24","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-021-02481-y","volume":"11","author":"C Duckworth","year":"2021","unstructured":"Duckworth, C. et al. Using explainable machine learning to characterise data drift and detect emergent health risks for emergency department admissions during COVID-19. Sci. Rep. 11, 23017 (2021).","journal-title":"Sci. Rep."},{"key":"611_CR25","doi-asserted-by":"publisher","first-page":"1309","DOI":"10.1016\/j.jacr.2019.05.036","volume":"16","author":"DL Rubin","year":"2019","unstructured":"Rubin, D. L. Artificial intelligence in imaging: The radiologist\u2019s role. J. Am. Coll. Radiol. 16, 1309\u20131317 (2019).","journal-title":"J. Am. Coll. Radiol."},{"key":"611_CR26","doi-asserted-by":"crossref","unstructured":"Gossmann, A., Cha, K. H. & Sun, X. Performance deterioration of deep neural networks for lesion classification in mammography due to distribution shift: an analysis based on artificially created distribution shift. In: Medical Imaging 2020: Computer-Aided Diagnosis, Vol. 11314, (eds. Hahn, H. K. & Mazurowski, M. A.)1131404 (International Society for Optics and Photonics, 2020).","DOI":"10.1117\/12.2551346"},{"key":"611_CR27","doi-asserted-by":"publisher","first-page":"106288","DOI":"10.1016\/j.cmpb.2021.106288","volume":"208","author":"F Cabitza","year":"2021","unstructured":"Cabitza, F. et al. The importance of being external. methodological insights for the external validation of machine learning models in medicine. Comput. Methods Programs Biomed. 208, 106288 (2021).","journal-title":"Comput. Methods Programs Biomed."},{"key":"611_CR28","unstructured":"Subbaswamy, A., Schulam, P. & Saria, S. Preventing failures due to dataset shift: Learning predictive models that transport. In: Proc. Machine Learning Research Vol. 89 (eds. Chaudhuri, K. & Sugiyama, M.) 3118\u20133127 (PMLR, 2019)."},{"key":"611_CR29","unstructured":"Sch\u00f6lkopf, B. et al. On causal and anticausal learning. In: Proc. 29th International Coference on International Conference on Machine Learning, ICML\u201912 459\u2013466 (Omnipress, 2012)."},{"key":"611_CR30","doi-asserted-by":"crossref","unstructured":"Quionero-Candela, J., Sugiyama, M., Schwaighofer, A. & Lawrence, N. D. Dataset Shift in Machine Learning (The MIT Press, 2009).","DOI":"10.7551\/mitpress\/9780262170055.001.0001"},{"key":"611_CR31","unstructured":"Montgomery, D. Introduction to Statistical Quality Control (Wiley, 2020)."},{"key":"611_CR32","doi-asserted-by":"crossref","unstructured":"Aggarwal, C. C. An introduction to outlier analysis. In: Outlier analysis 1\u201334 (Springer, 2017).","DOI":"10.1007\/978-3-319-54765-7_1"},{"key":"611_CR33","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1097\/00001648-199901000-00008","volume":"10","author":"S Greenland","year":"1999","unstructured":"Greenland, S., Pearl, J. & Robins, J. M. Causal diagrams for epidemiologic research. Epidemiology 10, 37\u201348 (1999).","journal-title":"Epidemiology"},{"key":"611_CR34","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-020-17478-w","volume":"11","author":"DC Castro","year":"2020","unstructured":"Castro, D. C., Walker, I. & Glocker, B. Causality matters in medical imaging. Nat. Commun. 11, 3673 (2020).","journal-title":"Nat. Commun."},{"key":"611_CR35","doi-asserted-by":"publisher","first-page":"100","DOI":"10.1093\/biomet\/41.1-2.100","volume":"41","author":"ES Page","year":"1954","unstructured":"Page, E. S. Continuous inspection schemes. Biometrika 41, 100\u2013115 (1954).","journal-title":"Biometrika"},{"key":"611_CR36","doi-asserted-by":"publisher","first-page":"517","DOI":"10.1002\/qre.829","volume":"23","author":"S Bersimis","year":"2007","unstructured":"Bersimis, S., Psarakis, S. & Panaretos, J. Multivariate statistical process control charts: an overview. Qual. Reliab. Eng. Int. 23, 517\u2013543 (2007).","journal-title":"Qual. Reliab. Eng. Int."},{"key":"611_CR37","doi-asserted-by":"publisher","first-page":"1586","DOI":"10.1198\/jasa.2009.tm08128","volume":"104","author":"C Zou","year":"2009","unstructured":"Zou, C. & Qiu, P. Multivariate statistical process control using LASSO. J. Am. Stat. Assoc. 104, 1586\u20131596 (2009).","journal-title":"J. Am. Stat. Assoc."},{"key":"611_CR38","doi-asserted-by":"crossref","unstructured":"Qahtan, A. A., Alharbi, B., Wang, S. & Zhang, X. A PCA-Based change detection framework for multidimensional data streams: change detection in multidimensional data streams. In: Proc. 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 935\u2013944 (Association for Computing Machinery, 2015).","DOI":"10.1145\/2783258.2783359"},{"key":"611_CR39","unstructured":"Boracchi, G., Carrera, D., Cervellera, C. & Macci\u00f2, D. QuantTree: Histograms for change detection in multivariate data streams. In: Proc. 35th International Conference on Machine Learning Vol. 80 (eds. Dy, J. & Krause, A.) 639\u2013648 (PMLR, 2018)."},{"key":"611_CR40","unstructured":"Rabanser, S., G\u00fcnnemann, S. & Lipton, Z. Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift. In: Advances in Neural Information Processing Systems Vol. 32 (eds. Wallach, H., Larochelle, H., Beygelzimer, A., d\u2019Alch\u00e9-Buc, F., Fox, E. & Garnett, R.) 1396\u20131408 https:\/\/proceedings.neurips.cc\/paper\/2019\/file\/846c260d715e5b854ffad5f70a516c88-Paper.pdf (Curran Associates, Inc., 2019)."},{"key":"611_CR41","doi-asserted-by":"publisher","first-page":"329","DOI":"10.1080\/00031305.2019.1700163","volume":"74","author":"P Qiu","year":"2020","unstructured":"Qiu, P. Big data? statistical process control can help! Am. Stat. 74, 329\u2013344 (2020).","journal-title":"Am. Stat."},{"key":"611_CR42","doi-asserted-by":"crossref","unstructured":"Ditzler, G. & Polikar, R. Hellinger distance based drift detection for nonstationary environments. In: 2011 IEEE Symposium on Computational Intelligence in Dynamic and Uncertain Environments (CIDUE) 41-48 (2011).","DOI":"10.1109\/CIDUE.2011.5948491"},{"key":"611_CR43","doi-asserted-by":"crossref","unstructured":"Gretton, A., Borgwardt, K., Rasch, M., Sch\u00f6lkopf, B. & Smola, A. A kernel method for the Two-Sample-Problem. In: Advances in Neural Information Processing Systems Vol. 19 (eds. Sch\u00f6lkopf, B., Platt, J. & Hoffman, T.) (MIT Press, 2007).","DOI":"10.7551\/mitpress\/7503.003.0069"},{"key":"611_CR44","unstructured":"Harchaoui, Z., Moulines, E. & Bach, F. Kernel change-point analysis. In Advances in Neural Information Processing Systems Vol. 21 (eds. Koller, D., Schuurmans, D., Bengio, Y. & Bottou, L.) (Curran Associates, Inc., 2009)."},{"key":"611_CR45","unstructured":"Williamson, B. D. & Feng, J. Efficient nonparametric statistical inference on population feature importance using shapley values. In: Proc. of the 37th International Conference on Machine Learning Vol. 119 (eds. Daum\u00e9. H. III & Singh, A.) 10282\u201310291 (PMLR, 2020)."},{"key":"611_CR46","doi-asserted-by":"publisher","unstructured":"Nishida, K. & Yamauchi, K. Detecting Concept Drift Using Statistical Testing. In: Discovery Science 264\u2013269 https:\/\/doi.org\/10.1007\/978-3-540-75488-6_27 (Springer Berlin Heidelberg, 2007).","DOI":"10.1007\/978-3-540-75488-6_27"},{"key":"611_CR47","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1137\/1108002","volume":"8","author":"AN Shiryaev","year":"1963","unstructured":"Shiryaev, A. N. On optimum methods in quickest detection problems. Theory Probab. Appl. 8, 22\u201346 (1963).","journal-title":"Theory Probab. Appl."},{"key":"611_CR48","doi-asserted-by":"publisher","first-page":"411","DOI":"10.1080\/00401706.1966.10490374","volume":"8","author":"SW Roberts","year":"1966","unstructured":"Roberts, S. W. A comparison of some control chart procedures. Technometrics 8, 411\u2013430 (1966).","journal-title":"Technometrics"},{"key":"611_CR49","doi-asserted-by":"publisher","first-page":"255","DOI":"10.1214\/aos\/1176324466","volume":"23","author":"D Siegmund","year":"1995","unstructured":"Siegmund, D. & Venkatraman, E. S. Using the generalized likelihood ratio statistic for sequential detection of a Change-Point. Ann. Statistics 23, 255\u2013271 (1995).","journal-title":"Ann. Statistics"},{"key":"611_CR50","doi-asserted-by":"publisher","first-page":"162","DOI":"10.1080\/07474941003741078","volume":"29","author":"TL Lai","year":"2010","unstructured":"Lai, T. L. & Xing, H. Sequential change-point detection when the pre- and post-change parameters are unknown. Seq. Anal. 29, 162\u2013175 (2010).","journal-title":"Seq. Anal."},{"key":"611_CR51","doi-asserted-by":"publisher","first-page":"488","DOI":"10.1111\/j.1467-9574.2007.00371.x","volume":"61","author":"A Zeileis","year":"2007","unstructured":"Zeileis, A. & Hornik, K. Generalized m-fluctuation tests for parameter instability. Stat. Neerl. 61, 488\u2013508 (2007).","journal-title":"Stat. Neerl."},{"key":"611_CR52","doi-asserted-by":"publisher","first-page":"103611","DOI":"10.1016\/j.jbi.2020.103611","volume":"112","author":"SE Davis","year":"2020","unstructured":"Davis, S. E., Greevy, R. A. Jr., Lasko, T. A., Walsh, C. G. & Matheny, M. E. Detection of calibration drift in clinical prediction models to inform model updating. J. Biomed. Inform. 112, 103611 (2020).","journal-title":"J. Biomed. Inform."},{"key":"611_CR53","doi-asserted-by":"publisher","first-page":"174","DOI":"10.1080\/00224065.2010.11917815","volume":"42","author":"C Zou","year":"2010","unstructured":"Zou, C. & Tsung, F. Likelihood ratio-based distribution-free EWMA control charts. J. Commod. Sci. Technol. Qual. 42, 174\u2013196 (2010).","journal-title":"J. Commod. Sci. Technol. Qual."},{"key":"611_CR54","doi-asserted-by":"publisher","first-page":"691","DOI":"10.1109\/JSAIT.2021.3081105","volume":"2","author":"J Shin","year":"2021","unstructured":"Shin, J., Ramdas, A. & Rinaldo, A. Nonparametric Iterated-Logarithm extensions of the sequential generalized likelihood ratio test. IEEE J. Sel. Areas in Inform. Theory 2, 691\u2013704 (2021).","journal-title":"IEEE J. Sel. Areas in Inform. Theory"},{"key":"611_CR55","doi-asserted-by":"publisher","unstructured":"Leonardi, F. & B\u00fchlmann, P. Computationally efficient change point detection for high-dimensional regression Preprint at https:\/\/doi.org\/10.48550\/ARXIV.1601.03704 (arXiv, 2016).","DOI":"10.48550\/ARXIV.1601.03704"},{"key":"611_CR56","doi-asserted-by":"publisher","first-page":"2051","DOI":"10.1214\/18-AOS1740","volume":"47","author":"F Enikeeva","year":"2019","unstructured":"Enikeeva, F. & Harchaoui, Z. High-dimensional change-point detection under sparse alternatives. Ann. Stat. 47, 2051\u20132079 (2019).","journal-title":"Ann. Stat."},{"key":"611_CR57","doi-asserted-by":"crossref","unstructured":"Liu, L., Salmon, J. & Harchaoui, Z. Score-Based change detection for Gradient-Based learning machines. In: ICASSP 2021\u20132021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 4990\u20134994 (2021).","DOI":"10.1109\/ICASSP39728.2021.9414085"},{"key":"611_CR58","doi-asserted-by":"publisher","first-page":"89","DOI":"10.1080\/00224065.2006.11918593","volume":"38","author":"WH Woodall","year":"2006","unstructured":"Woodall, W. H. The use of control charts in health-care and public-health surveillance. J. Qual. Technol. 38, 89\u2013104 (2006).","journal-title":"J. Qual. Technol."},{"key":"611_CR59","doi-asserted-by":"publisher","first-page":"1442","DOI":"10.1111\/j.1541-0420.2011.01603.x","volume":"67","author":"Y Huang","year":"2011","unstructured":"Huang, Y. & Gilbert, P. B. Comparing biomarkers as principal surrogate endpoints. Biometrics 67, 1442\u20131451 (2011).","journal-title":"Biometrics"},{"key":"611_CR60","doi-asserted-by":"crossref","unstructured":"Price, B. L., Gilbert, P. B. & van der Laan, M. J. Estimation of the optimal surrogate based on a randomized trial. Biometrics 74, 1271\u20131281 (2018).","DOI":"10.1111\/biom.12879"},{"key":"611_CR61","doi-asserted-by":"publisher","first-page":"e28236","DOI":"10.2196\/28236","volume":"8","author":"O Asan","year":"2021","unstructured":"Asan, O. & Choudhury, A. Research trends in artificial intelligence applications in human factors health care: mapping review. JMIR Hum. Factors 8, e28236 (2021).","journal-title":"JMIR Hum. Factors"},{"key":"611_CR62","first-page":"1109","volume":"2013","author":"C Paxton","year":"2013","unstructured":"Paxton, C., Niculescu-Mizil, A. & Saria, S. Developing predictive models using electronic medical records: challenges and pitfalls. AMIA Annu. Symp. Proc. 2013, 1109\u20131115 (2013).","journal-title":"AMIA Annu. Symp. Proc."},{"key":"611_CR63","doi-asserted-by":"publisher","first-page":"323","DOI":"10.1007\/s10994-015-5527-7","volume":"102","author":"K Dyagilev","year":"2016","unstructured":"Dyagilev, K. & Saria, S. Learning (predictive) risk scores in the presence of censoring due to interventions. Mach. Learn. 102, 323\u2013348 (2016).","journal-title":"Mach. Learn."},{"key":"611_CR64","doi-asserted-by":"publisher","first-page":"1645","DOI":"10.1093\/jamia\/ocz145","volume":"26","author":"MC Lenert","year":"2019","unstructured":"Lenert, M. C., Matheny, M. E. & Walsh, C. G. Prognostic models will be victims of their own success, unless. J. Am. Med. Inform. Assoc. 26, 1645\u20131650 (2019).","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"611_CR65","unstructured":"Perdomo, J., Zrnic, T., Mendler-D\u00fcnner, C. & Hardt, M. Performative prediction. In Proc. of the 37th International Conference on Machine Learning Vol. 119 (eds. Daum\u00e9. H. III & Singh, A.) 7599\u20137609 http:\/\/proceedings.mlr.press\/v119\/perdomo20a\/perdomo20a.pdf (PMLR, 2020)."},{"key":"611_CR66","first-page":"3916","volume":"130","author":"J Liley","year":"2021","unstructured":"Liley, J. et al. Model updating after interventions paradoxically introduces bias. Int. Conf. Artif. Intell. Statistics 130, 3916\u20133924 (2021).","journal-title":"Int. Conf. Artif. Intell. Statistics"},{"key":"611_CR67","doi-asserted-by":"crossref","unstructured":"Imbens, G. W. & Rubin, D. B. Causal Inference in Statistics, Social, and Biomedical Sciences (Cambridge University Press, 2015).","DOI":"10.1017\/CBO9781139025751"},{"key":"611_CR68","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18637\/jss.v007.i02","volume":"7","author":"A Zeileis","year":"2002","unstructured":"Zeileis, A., Leisch, F., Hornik, K. & Kleiber, C. strucchange: an r package for testing for structural change in linear regression models. J. Statistical Softw. 7, 1\u201338 (2002).","journal-title":"J. Statistical Softw."},{"key":"611_CR69","doi-asserted-by":"publisher","first-page":"1378","DOI":"10.1097\/01.CCM.0000216702.94014.75","volume":"34","author":"DA Harrison","year":"2006","unstructured":"Harrison, D. A., Brady, A. R., Parry, G. J., Carpenter, J. R. & Rowan, K. Recalibration of risk prediction models in a large multicenter cohort of admissions to adult, general critical care units in the united kingdom. Crit. Care Med. 34, 1378\u20131388 (2006).","journal-title":"Crit. Care Med."},{"key":"611_CR70","doi-asserted-by":"publisher","first-page":"361","DOI":"10.1007\/s00134-013-3202-7","volume":"40","author":"M van den Boogaard","year":"2014","unstructured":"van den Boogaard, M. et al. Recalibration of the delirium prediction model for ICU patients (PRE-DELIRIC): a multinational observational study. Intensive Care Med. 40, 361\u2013369 (2014).","journal-title":"Intensive Care Med."},{"key":"611_CR71","doi-asserted-by":"publisher","first-page":"1202","DOI":"10.1126\/science.aay9547","volume":"366","author":"B Babic","year":"2019","unstructured":"Babic, B., Gerke, S., Evgeniou, T. & Cohen, I. G. Algorithms on regulatory lockdown in medicine. Science 366, 1202\u20131204 (2019).","journal-title":"Science"},{"key":"611_CR72","unstructured":"European Medicines Agency. Regulation (EU) 2017\/745 of the european parliament and of the council. Tech. Rep. (2020)."},{"key":"611_CR73","unstructured":"Ensign, D., Friedler, S. A., Neville, S., Scheidegger, C. & Venkatasubramanian, S. Runaway feedback loops in predictive policing. In: Accountability and Transparency Vol. 81 (eds. Friedler, S. A. & Wilson, C.) 160\u2013171 (PMLR, 2018)."},{"key":"611_CR74","unstructured":"Hashimoto, T., Srivastava, M., Namkoong, H. & Liang, P. Fairness without demographics in repeated loss minimization. In Proc. 35th International Conference on Machine Learning Vol. 80 (eds. Dy, J. & Krause, A.) 1929\u20131938 (PMLR, 2018)."},{"key":"611_CR75","doi-asserted-by":"crossref","unstructured":"Liu, L. T., Dean, S., Rolf, E., Simchowitz, M. & Hardt, M. Delayed Impact of Fair Machine Learning Vol. 80, 3150-3158 (PMLR, 2018).","DOI":"10.24963\/ijcai.2019\/862"},{"key":"611_CR76","doi-asserted-by":"publisher","unstructured":"Chouldechova, A. & Roth, A. The frontiers of fairness in machine learning Preprint at https:\/\/doi.org\/10.48550\/ARXIV.1810.08810 (arXiv, 2018).","DOI":"10.48550\/ARXIV.1810.08810"},{"key":"611_CR77","doi-asserted-by":"crossref","unstructured":"Hastie, T., Tibshirani, R. & Friedman, J. The Elements of Statistical Learning (Springer, 2009) .","DOI":"10.1007\/978-0-387-84858-7"},{"key":"611_CR78","doi-asserted-by":"crossref","unstructured":"James, G., Witten, D., Hastie, T. & Tibshirani, R. An Introduction to Statistical Learning (Springer, 2021).","DOI":"10.1007\/978-1-0716-1418-1"},{"key":"611_CR79","first-page":"61","volume":"10","author":"J Platt","year":"1999","unstructured":"Platt, J. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Adv. Large Margin Classifiers 10, 61\u201374 (1999).","journal-title":"Adv. Large Margin Classifiers"},{"key":"611_CR80","doi-asserted-by":"crossref","unstructured":"Niculescu-Mizil, A. & Caruana, R. Predicting good probabilities with supervised learning. In: Proc. 22nd international conference on Machine learning, ICML\u201905 625\u2013632 (Association for Computing Machinery, 2005).","DOI":"10.1145\/1102351.1102430"},{"key":"611_CR81","first-page":"1321","volume":"70","author":"C Guo","year":"2017","unstructured":"Guo, C., Pleiss, G., Sun, Y. & Weinberger, K. Q. On calibration of modern neural networks. Int. Conf. Mach. Learning 70, 1321\u20131330 (2017).","journal-title":"Int. Conf. Mach. Learning"},{"key":"611_CR82","doi-asserted-by":"publisher","first-page":"1394","DOI":"10.1177\/0962280216661371","volume":"27","author":"W Chen","year":"2018","unstructured":"Chen, W., Sahiner, B., Samuelson, F., Pezeshk, A. & Petrick, N. Calibration of medical diagnostic classifier scores to the probability of disease. Stat. Methods Med. Res. 27, 1394\u20131409 (2018).","journal-title":"Stat. Methods Med. Res."},{"key":"611_CR83","unstructured":"Steyerberg, E. W. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating (Springer, 2009). ."},{"key":"611_CR84","doi-asserted-by":"publisher","first-page":"2567","DOI":"10.1002\/sim.1844","volume":"23","author":"EW Steyerberg","year":"2004","unstructured":"Steyerberg, E. W., Borsboom, G. J. J. M., van Houwelingen, H. C., Eijkemans, M. J. C. & Habbema, J. D. F. Validation and updating of predictive logistic regression models: a study on sample size and shrinkage. Stat. Med. 23, 2567\u20132586 (2004).","journal-title":"Stat. Med."},{"key":"611_CR85","doi-asserted-by":"publisher","first-page":"249","DOI":"10.1002\/sim.7320","volume":"37","author":"D Benkeser","year":"2018","unstructured":"Benkeser, D., Ju, C., Lendle, S. & van der Laan, M. Online cross-validation-based ensemble learning. Statistics Med. 37, 249\u2013260 (2018).","journal-title":"Statistics Med."},{"key":"611_CR86","doi-asserted-by":"crossref","unstructured":"McCormick, T. H. Dynamic logistic regression and dynamic model averaging for binary classification. Biometrics 68, 23\u201330 (2012).","DOI":"10.1111\/j.1541-0420.2011.01645.x"},{"key":"611_CR87","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1016\/j.jbi.2015.05.001","volume":"56","author":"AN Strobl","year":"2015","unstructured":"Strobl, A. N. et al. Improving patient prostate cancer risk assessment: Moving from static, globally-applied to dynamic, practice-specific risk calculators. J. Biomed. Inform. 56, 87\u201393 (2015).","journal-title":"J. Biomed. Inform."},{"key":"611_CR88","doi-asserted-by":"publisher","first-page":"e489","DOI":"10.1016\/S2589-7500(20)30186-2","volume":"2","author":"J Futoma","year":"2020","unstructured":"Futoma, J., Simons, M., Panch, T., Doshi-Velez, F. & Celi, L. A. The myth of generalisability in clinical research and machine learning in health care. Lancet Digit Health 2, e489\u2013e492 (2020).","journal-title":"Lancet Digit Health"},{"key":"611_CR89","doi-asserted-by":"publisher","first-page":"e337","DOI":"10.1016\/S2589-7500(21)00076-5","volume":"3","author":"KN Vokinger","year":"2021","unstructured":"Vokinger, K. N., Feuerriegel, S. & Kesselheim, A. S. Continual learning in medical devices: FDA\u2019s action plan and beyond. Lancet Digit Health 3, e337\u2013e338 (2021).","journal-title":"Lancet Digit Health"},{"key":"611_CR90","doi-asserted-by":"publisher","unstructured":"Viering, T. J., Mey, A. & Loog, M. Making learners (more) monotone. In: Advances in Intelligent Data Analysis XVIII (eds. Berthold, M. R., Feelders, Ad & Krempl, G.) 535\u2013547 https:\/\/doi.org\/10.1007\/978-3-030-44584-3_42 (Springer International Publishing, 2020).","DOI":"10.1007\/978-3-030-44584-3_42"},{"key":"611_CR91","doi-asserted-by":"crossref","unstructured":"Feng, J., Emerson, S. & Simon, N. Approval policies for modifications to machine learning-based software as a medical device: a study of bio-creep. Biometrics (2020).","DOI":"10.1111\/biom.13379"},{"key":"611_CR92","doi-asserted-by":"crossref","unstructured":"Feng, J., Gossmann, A., Sahiner, B. & Pirracchio, R. Bayesian logistic regression for online recalibration and revision of risk prediction models with performance guarantees. J. Am. Med. Inform. Assoc. (2022).","DOI":"10.1093\/jamia\/ocab280"},{"key":"611_CR93","doi-asserted-by":"crossref","unstructured":"Feng, J. Learning to safely approve updates to machine learning algorithms. In: Proc. Conference on Health, Inference, and Learning, CHIL\u201921 164\u2013173 (Association for Computing Machinery, 2021).","DOI":"10.1145\/3450439.3451864"},{"key":"611_CR94","doi-asserted-by":"publisher","first-page":"e22219","DOI":"10.2196\/22219","volume":"23","author":"IS Kohane","year":"2021","unstructured":"Kohane, I. S. et al. What every reader should know about studies using electronic health record data but may be afraid to ask. J. Med. Internet Res. 23, e22219 (2021).","journal-title":"J. Med. Internet Res."},{"key":"611_CR95","doi-asserted-by":"publisher","first-page":"773","DOI":"10.1002\/sim.8445","volume":"39","author":"LJ Beesley","year":"2020","unstructured":"Beesley, L. J. et al. The emerging landscape of health research based on biobanks linked to electronic health records: existing resources, statistical challenges, and potential opportunities. Stat. Med. 39, 773\u2013800 (2020).","journal-title":"Stat. Med."},{"key":"611_CR96","doi-asserted-by":"publisher","first-page":"e100183","DOI":"10.1136\/bmjhci-2020-100183","volume":"27","author":"CV Cosgriff","year":"2020","unstructured":"Cosgriff, C. V., Stone, D. J., Weissman, G., Pirracchio, R. & Celi, L. A. The clinical artificial intelligence department: a prerequisite for success. BMJ Health Care Inform. 27, e100183 (2020).","journal-title":"BMJ Health Care Inform."},{"key":"611_CR97","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-020-69250-1","volume":"10","author":"MJ Sheller","year":"2020","unstructured":"Sheller, M. J. et al. Federated learning in medicine: facilitating multi-institutional collaborations without sharing patient data. Sci. Rep. 10, 12598 (2020).","journal-title":"Sci. Rep."},{"key":"611_CR98","doi-asserted-by":"publisher","first-page":"265","DOI":"10.1038\/s41586-021-03583-3","volume":"594","author":"S Warnat-Herresthal","year":"2021","unstructured":"Warnat-Herresthal, S. et al. Swarm Learning for decentralized and confidential clinical machine learning. Nature 594, 265\u2013270 (2021).","journal-title":"Nature"},{"key":"611_CR99","unstructured":"U.S. Food and Drug Administration. Sentinel system: 5-year strategy 2019-2023. Tech. Rep. (2019)."},{"key":"611_CR100","unstructured":"Harvey, H. & Cabitza, F. Algorithms are the new drugs? Reflections for a culture of impact assessment and vigilance. In: IADIS International Conference ICT, Society and Human Beings 2018 (eds. Macedo, M. & Kommers, P.) (part of MCCSIS 2018) (2018)."},{"key":"611_CR101","doi-asserted-by":"publisher","first-page":"161","DOI":"10.21037\/atm.2019.04.07","volume":"7","author":"F Cabitza","year":"2019","unstructured":"Cabitza, F. & Zeitoun, J.-D. The proof of the pudding: in praise of a culture of real-world validation for medical artificial intelligence. Ann Transl Med 7, 161 (2019).","journal-title":"Ann Transl Med"},{"key":"611_CR102","doi-asserted-by":"publisher","DOI":"10.1038\/sdata.2016.35","volume":"3","author":"AE Johnson","year":"2016","unstructured":"Johnson, A. E. et al. MIMIC-III, a freely accessible critical care database. Sci Data 3, 160035 (2016).","journal-title":"Sci Data"},{"key":"611_CR103","first-page":"1","volume":"7","author":"A Zeileis","year":"2002","unstructured":"Zeileis, A., Leisch, F., Hornik, K. & Kleiber, C. strucchange: an r package for testing for structural change in linear regression models. J. Statistical Softw. Articles 7, 1\u201338 (2002).","journal-title":"J. Statistical Softw. Articles"},{"key":"611_CR104","first-page":"1601","volume":"11","author":"A Bifet","year":"2010","unstructured":"Bifet, A., Holmes, G., Kirkby, R. & Pfahringer, B. MOA: massive online analysis. J. Mach. Learn. Res. 11, 1601\u20131604 (2010).","journal-title":"J. Mach. Learn. Res."},{"key":"611_CR105","first-page":"1","volume":"19","author":"J Montiel","year":"2018","unstructured":"Montiel, J., Read, J., Bifet, A. & Abdessalem, T. Scikit-multiflow: a multi-output streaming framework. J. Mach. Learn. Res. 19, 1\u20135 (2018).","journal-title":"J. Mach. Learn. Res."}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-022-00611-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-022-00611-y","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-022-00611-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,22]],"date-time":"2023-11-22T12:27:47Z","timestamp":1700656067000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-022-00611-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,31]]},"references-count":105,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["611"],"URL":"https:\/\/doi.org\/10.1038\/s41746-022-00611-y","relation":{},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,5,31]]},"assertion":[{"value":"16 November 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 April 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"31 May 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Dr. Bishara is a co-founder of Bezel Health, a company building software to measure and improve healthcare quality interventions. Other authors declare that there are no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"66"}}