{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T08:59:59Z","timestamp":1782377999170,"version":"3.54.5"},"reference-count":90,"publisher":"Oxford University Press (OUP)","issue":"9","license":[{"start":{"date-parts":[[2023,6,27]],"date-time":"2023-06-27T00:00:00Z","timestamp":1687824000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000002","name":"NIH","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"name":"National Institute on Drug Abuse Clinical Trials Network","award":["UG1DA015815\u2013CTN-0136"],"award-info":[{"award-number":["UG1DA015815\u2013CTN-0136"]}]},{"name":"Stanford Artificial Intelligence in Medicine and Imaging\u2013 Human-Centered Artificial Intelligence"},{"name":"Doris Duke Charitable Foundation\u2014Covid-19 Fund to Retain Clinical Scientists","award":["20211260"],"award-info":[{"award-number":["20211260"]}]},{"name":"American Heart Association\u2014Strategically Focused Research Network\u2014Diversity in Clinical Trials"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,8,18]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>Heatlhcare institutions are establishing frameworks to govern and promote the implementation of accurate, actionable, and reliable machine learning models that integrate with clinical workflow. Such governance frameworks require an accompanying technical framework to deploy models in a resource efficient, safe and high-quality manner. Here we present DEPLOYR, a technical framework for enabling real-time deployment and monitoring of researcher-created models into a widely used electronic medical record system.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>We discuss core functionality and design decisions, including mechanisms to trigger inference based on actions within electronic medical record software, modules that collect real-time data to make inferences, mechanisms that close-the-loop by displaying inferences back to end-users within their workflow, monitoring modules that track performance of deployed models over time, silent deployment capabilities, and mechanisms to prospectively evaluate a deployed model\u2019s impact.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>We demonstrate the use of DEPLOYR by silently deploying and prospectively evaluating 12 machine learning models trained using electronic medical record data that predict laboratory diagnostic results, triggered by clinician button-clicks in Stanford Health Care\u2019s electronic medical record.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Discussion<\/jats:title>\n                  <jats:p>Our study highlights the need and feasibility for such silent deployment, because prospectively measured performance varies from retrospective estimates. When possible, we recommend using prospectively estimated performance measures during silent trials to make final go decisions for model deployment.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>Machine learning applications in healthcare are extensively researched, but successful translations to the bedside are rare. By describing DEPLOYR, we aim to inform machine learning deployment best practices and help bridge the model implementation gap.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocad114","type":"journal-article","created":{"date-parts":[[2023,6,27]],"date-time":"2023-06-27T20:35:36Z","timestamp":1687898136000},"page":"1532-1542","source":"Crossref","is-referenced-by-count":43,"title":["DEPLOYR: a technical framework for deploying custom real-time machine learning models into the electronic medical record"],"prefix":"10.1093","volume":"30","author":[{"given":"Conor K","family":"Corbin","sequence":"first","affiliation":[{"name":"Department of Biomedical Data Science , Stanford, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rob","family":"Maclay","sequence":"additional","affiliation":[{"name":"Stanford Children\u2019s Health , Palo Alto, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aakash","family":"Acharya","sequence":"additional","affiliation":[{"name":"Stanford Health Care , Palo Alto, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sreedevi","family":"Mony","sequence":"additional","affiliation":[{"name":"Stanford Health Care , Palo Alto, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Soumya","family":"Punnathanam","sequence":"additional","affiliation":[{"name":"Stanford Health Care , Palo Alto, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rahul","family":"Thapa","sequence":"additional","affiliation":[{"name":"Stanford Health Care , Palo Alto, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nikesh","family":"Kotecha","sequence":"additional","affiliation":[{"name":"Stanford Health Care , Palo Alto, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9385-7158","authenticated-orcid":false,"given":"Nigam H","family":"Shah","sequence":"additional","affiliation":[{"name":"Center for Biomedical Informatics Research, Division of Hospital Medicine, Department of Medicine, Stanford University, School of Medicine , Stanford, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jonathan H","family":"Chen","sequence":"additional","affiliation":[{"name":"Center for Biomedical Informatics Research, Division of Hospital Medicine, Department of Medicine, Stanford University, School of Medicine , Stanford, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2023,6,27]]},"reference":[{"issue":"24","key":"2023081808481383100_ocad114-B1","doi-asserted-by":"crossref","first-page":"2405","DOI":"10.1001\/jama.2019.5284","article-title":"The proliferation of reports on clinical scoring systems: issues about uptake and clinical utility","volume":"321","author":"Challener","year":"2019","journal-title":"JAMA"},{"issue":"4","key":"2023081808481383100_ocad114-B2","doi-asserted-by":"crossref","first-page":"808","DOI":"10.1055\/s-0041-1735184","article-title":"Systematic review of approaches to preserve machine learning performance in the presence of temporal dataset shift in clinical medicine","volume":"12","author":"Guo","year":"2021","journal-title":"Appl Clin Inform"},{"issue":"26","key":"2023081808481383100_ocad114-B3","doi-asserted-by":"crossref","first-page":"2507","DOI":"10.1056\/NEJMp1702071","article-title":"Machine learning and prediction in medicine\u2014beyond the peak of inflated expectations","volume":"376","author":"Chen","year":"2017","journal-title":"N Engl J Med"},{"key":"2023081808481383100_ocad114-B4","doi-asserted-by":"crossref","DOI":"10.17226\/27111","volume-title":"Artificial Intelligence in Health Care: The Hope, the Hype, the Promise, the Peril","author":"Matheny","year":"2019"},{"issue":"2","key":"2023081808481383100_ocad114-B5","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1136\/bmjinnov-2019-000359","article-title":"Bridging the implementation gap of machine learning in healthcare","volume":"6","author":"Seneviratne","year":"2020","journal-title":"BMJ Innov"},{"key":"2023081808481383100_ocad114-B6","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1016\/B978-0-12-809523-2.00019-4","volume-title":"Key Advances in Clinical Informatics","author":"Callahan","year":"2017"},{"issue":"14","key":"2023081808481383100_ocad114-B7","doi-asserted-by":"crossref","first-page":"1351","DOI":"10.1001\/jama.2019.10306","article-title":"Making machine learning models clinically useful","volume":"322","author":"Shah","year":"2019","journal-title":"JAMA"},{"key":"2023081808481383100_ocad114-B8","doi-asserted-by":"crossref","first-page":"103826","DOI":"10.1016\/j.jbi.2021.103826","article-title":"Improving hospital readmission prediction using individualized utility analysis","volume":"119","author":"Ko","year":"2021","journal-title":"J Biomed Inform"},{"issue":"6","key":"2023081808481383100_ocad114-B9","doi-asserted-by":"crossref","first-page":"1149","DOI":"10.1093\/jamia\/ocaa318","article-title":"A framework for making predictive models useful in practice","volume":"28","author":"Jung","year":"2021","journal-title":"J Am Med Inform Assoc"},{"issue":"12","key":"2023081808481383100_ocad114-B10","doi-asserted-by":"crossref","first-page":"2011","DOI":"10.1093\/jamia\/ocaa088","article-title":"MINIMAR (MINimum Information for Medical AI Reporting): developing reporting standards for artificial intelligence in health care","volume":"27","author":"Hernandez-Boussard","year":"2020","journal-title":"J Am Med Inform Assoc"},{"key":"2023081808481383100_ocad114-B11","doi-asserted-by":"crossref","first-page":"43768","DOI":"10.3389\/fdgth.2022.943768","article-title":"Considerations in the reliability and fairness audits of predictive models for advance care planning","volume":"4","author":"Lu","year":"2022","journal-title":"Front Digit Health"},{"issue":"11","key":"2023081808481383100_ocad114-B12","doi-asserted-by":"crossref","first-page":"981","DOI":"10.1056\/NEJMp1714229","article-title":"Implementing machine learning in health care\u2014addressing ethical challenges","volume":"378","author":"Char","year":"2018","journal-title":"N Engl J Med"},{"key":"2023081808481383100_ocad114-B13","article-title":"Researchers create guide for fair and equitable AI in health care","author":"Armitage","year":"2022","journal-title":"Logo Left ContentLogo Right Content 10,000+ Posts Scope Stanford University School of Medicine Blog"},{"issue":"9","key":"2023081808481383100_ocad114-B14","doi-asserted-by":"crossref","first-page":"1631","DOI":"10.1093\/jamia\/ocac078","article-title":"A framework for the oversight and local deployment of safe and high-quality prediction models","volume":"29","author":"Bedoya","year":"2022","journal-title":"J Am Med Inform Assoc"},{"issue":"3","key":"2023081808481383100_ocad114-B15","doi-asserted-by":"crossref","first-page":"491","DOI":"10.1093\/jamia\/ocz192","article-title":"A governance model for the application of AI in health care","volume":"27","author":"Reddy","year":"2020","journal-title":"J Am Med Inform Assoc"},{"issue":"9","key":"2023081808481383100_ocad114-B16","doi-asserted-by":"crossref","first-page":"1337","DOI":"10.1038\/s41591-019-0548-6","article-title":"Do no harm: a roadmap for responsible machine learning for health care","volume":"25","author":"Wiens","year":"2019","journal-title":"Nat Med"},{"key":"2023081808481383100_ocad114-B17","first-page":"1396","author":"Kim","year":"2023"},{"issue":"3","key":"2023081808481383100_ocad114-B18","doi-asserted-by":"crossref","first-page":"826","DOI":"10.4338\/ACI-2017-03-CR-0046","article-title":"Barriers to achieving economies of scale in analysis of EHR data","volume":"8","author":"Sendak","year":"2017","journal-title":"Appl Clin Inform"},{"key":"2023081808481383100_ocad114-B19","article-title":"Hidden technical debt in machine learning systems","volume":"28","author":"Sculley","year":"2015","journal-title":"Adv Neural Inform Process Syst"},{"issue":"1","key":"2023081808481383100_ocad114-B20","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1038\/s41591-019-0651-8","article-title":"Estimate the hidden deployment cost of predictive models to improve patient care","volume":"26","author":"Morse","year":"2020","journal-title":"Nat Med"},{"issue":"11","key":"2023081808481383100_ocad114-B21","doi-asserted-by":"crossref","first-page":"2445","DOI":"10.1093\/jamia\/ocab154","article-title":"A survey of extant organizational and computational setups for deploying predictive models in health systems","volume":"28","author":"Kashyap","year":"2021","journal-title":"J Am Med Inform Assoc"},{"key":"2023081808481383100_ocad114-B22","author":"Siwicki","year":"2021"},{"issue":"3","key":"2023081808481383100_ocad114-B23","doi-asserted-by":"crossref","first-page":"632","DOI":"10.1055\/s-0042-1750416","article-title":"Evaluating the effect of a COVID-19 predictive model to facilitate discharge: a randomized controlled trial","volume":"13","author":"Major","year":"2022","journal-title":"Appl Clin Inform"},{"issue":"3","key":"2023081808481383100_ocad114-B24","article-title":"Supporting acute advance care planning with precise, timely mortality risk predictions","volume":"2","author":"Wang","year":"2021","journal-title":"NEJM Catal Innov Care Deliv"},{"key":"2023081808481383100_ocad114-B25","doi-asserted-by":"crossref","first-page":"e44977","DOI":"10.2196\/44977","article-title":"Deployment of real-time natural language processing and deep learning clinical decision support in the electronic health record: pipeline implementation for an opioid misuse screener in hospitalized adults","volume":"11","author":"Afshar","year":"2023","journal-title":"JMIR Med Inform"},{"issue":"4","key":"2023081808481383100_ocad114-B26","first-page":"CAT\u201321","article-title":"Using AI to empower collaborative team workflows: two implementations for advance care planning and care escalation","volume":"3","author":"Li","year":"2022","journal-title":"NEJM Catal Innov Care Deliv"},{"key":"2023081808481383100_ocad114-B27"},{"key":"2023081808481383100_ocad114-B28","author":"Streamlit","year":"2023"},{"issue":"1","key":"2023081808481383100_ocad114-B29","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s40537-019-0217-0","article-title":"Big data in healthcare: management, analysis and future prospects","volume":"6","author":"Dash","year":"2019","journal-title":"J Big Data"},{"issue":"1","key":"2023081808481383100_ocad114-B30","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s43856-022-00094-8","article-title":"Personalized antibiograms for machine learning driven antibiotic selection","volume":"2","author":"Corbin","year":"2022","journal-title":"Commun Med"},{"issue":"9","key":"2023081808481383100_ocad114-B31","doi-asserted-by":"crossref","first-page":"e1910967","DOI":"10.1001\/jamanetworkopen.2019.10967","article-title":"Prevalence and predictability of low-yield inpatient laboratory diagnostic tests","volume":"2","author":"Xu","year":"2019","journal-title":"JAMA Netw Open"},{"key":"2023081808481383100_ocad114-B32","author":"Datta","year":"2020"},{"key":"2023081808481383100_ocad114-B33","first-page":"708","author":"Krall","year":"1995"},{"key":"2023081808481383100_ocad114-B34","first-page":"326","author":"Bender","year":"2013"},{"issue":"11","key":"2023081808481383100_ocad114-B35","doi-asserted-by":"crossref","first-page":"2379","DOI":"10.1093\/jamia\/ocab171","article-title":"The ecosystem of apps and software integrated with certified health information technology","volume":"28","author":"Barker","year":"2021","journal-title":"J Am Med Inform Assoc"},{"issue":"2 Pt 1","key":"2023081808481383100_ocad114-B36","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1097\/AOG.0b013e318242032a","article-title":"Effect of a best-practice alert on the rate of influenza vaccination of pregnant women","volume":"119","author":"Klatt","year":"2012","journal-title":"Obstet Gynecol"},{"issue":"4","key":"2023081808481383100_ocad114-B37","doi-asserted-by":"crossref","first-page":"e75","DOI":"10.1016\/j.jvs.2022.07.158","article-title":"Interacting with best practice advisory (BPA) notifications in epic significantly improves screening rates for abdominal aortic aneurysms","volume":"76","author":"Ahmed","year":"2022","journal-title":"J Vasc Surg"},{"issue":"7","key":"2023081808481383100_ocad114-B38","doi-asserted-by":"crossref","first-page":"e13719","DOI":"10.2196\/13719","article-title":"A real-time early warning system for monitoring inpatient mortality risk: prospective study using electronic medical record data","volume":"21","author":"Ye","year":"2019","journal-title":"J Med Internet Res"},{"issue":"4","key":"2023081808481383100_ocad114-B39","doi-asserted-by":"crossref","first-page":"547","DOI":"10.1097\/CCM.0000000000002936","article-title":"An interpretable machine learning model for accurate prediction of sepsis in the ICU","volume":"46","author":"Nemati","year":"2018","journal-title":"Crit Care Med"},{"key":"2023081808481383100_ocad114-B40","first-page":"4038","author":"Saqib","year":"2018"},{"issue":"7767","key":"2023081808481383100_ocad114-B41","doi-asserted-by":"crossref","first-page":"116","DOI":"10.1038\/s41586-019-1390-1","article-title":"A clinically applicable approach to continuous prediction of future acute kidney injury","volume":"572","author":"Toma\u0161ev","year":"2019","journal-title":"Nature"},{"issue":"65es","key":"2023081808481383100_ocad114-B42","first-page":"15","article-title":"Take command: cron: job scheduler","volume":"1999","author":"Keller","year":"1999","journal-title":"Linux J"},{"key":"2023081808481383100_ocad114-B43","author":"Lundberg","year":"2017"},{"issue":"3","key":"2023081808481383100_ocad114-B44","doi-asserted-by":"crossref","first-page":"659","DOI":"10.1016\/j.jaad.2022.04.048","article-title":"Tracking procedure outcomes using Epic SmartText and SmartData Elements following minor dermatologic procedures in the ambulatory setting","volume":"88","author":"Flynn","year":"2023","journal-title":"J Am Acad Dermatol"},{"key":"2023081808481383100_ocad114-B45","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1016\/j.ijmedinf.2015.12.002","article-title":"Use of a remote clinical decision support service for a multicenter trial to implement prediction rules for children with minor blunt head trauma","volume":"87","author":"Goldberg","year":"2016","journal-title":"Int J Med Inform"},{"key":"2023081808481383100_ocad114-B46","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.ijmedinf.2016.12.005","article-title":"Opening the Duke electronic health record to apps: implementing SMART on FHIR","volume":"99","author":"Bloomfield","year":"2017","journal-title":"Int J Med Inform"},{"issue":"1","key":"2023081808481383100_ocad114-B47","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41746-022-00611-y","article-title":"Clinical artificial intelligence quality improvement: towards continual monitoring and updating of AI algorithms in healthcare","volume":"5","author":"Feng","year":"2022","journal-title":"NPJ Digit Med"},{"issue":"3","key":"2023081808481383100_ocad114-B48","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1016\/j.dsm.2022.07.004","article-title":"Monitoring machine learning models: a categorization of challenges and methods","volume":"5","author":"Schr\u00f6der","year":"2022","journal-title":"Data Sci Manage"},{"key":"2023081808481383100_ocad114-B49","author":"Klaise","year":"2020"},{"key":"2023081808481383100_ocad114-B50","doi-asserted-by":"crossref","first-page":"168","DOI":"10.1016\/j.jbi.2015.10.006","article-title":"Implications of non-stationarity on predictive modeling using EHRs","volume":"58","author":"Jung","year":"2015","journal-title":"J Biomed Inform"},{"key":"2023081808481383100_ocad114-B51","doi-asserted-by":"crossref","first-page":"i6","DOI":"10.1136\/bmj.i6","article-title":"Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests","volume":"352","author":"Vickers","year":"2016","journal-title":"BMJ"},{"key":"2023081808481383100_ocad114-B52","first-page":"169","author":"Tonekaboni","year":"2022"},{"key":"2023081808481383100_ocad114-B53","year":"2023"},{"key":"2023081808481383100_ocad114-B54","first-page":"506","author":"Otles","year":"2021"},{"key":"2023081808481383100_ocad114-B55","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1155\/2017\/7847531","article-title":"Correcting classifiers for sample selection bias in two-phase case-control studies","volume":"2017","author":"Krautenbacher","year":"2017","journal-title":"Comput Math Methods Med"},{"issue":"1","key":"2023081808481383100_ocad114-B56","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s40537-021-00501-2","article-title":"Design matters in patient-level prediction: evaluation of a cohort vs. case-control design when developing predictive models in observational healthcare datasets","volume":"8","author":"Reps","year":"2021","journal-title":"J Big Data"},{"key":"2023081808481383100_ocad114-B57","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1016\/j.jhealeco.2019.02.002","article-title":"The effect of predictive analytics-driven interventions on healthcare utilization","volume":"64","author":"David","year":"2019","journal-title":"J Health Econ"},{"issue":"7","key":"2023081808481383100_ocad114-B58","doi-asserted-by":"crossref","first-page":"1455","DOI":"10.1038\/s41591-022-01894-0","article-title":"Prospective, multi-site study of patient outcomes after implementation of the TREWS machine learning-based early warning system for sepsis","volume":"28","author":"Adams","year":"2022","journal-title":"Nat Med"},{"key":"2023081808481383100_ocad114-B59","author":"Nemati","year":"2022"},{"issue":"20","key":"2023081808481383100_ocad114-B60","doi-asserted-by":"crossref","first-page":"1951","DOI":"10.1056\/NEJMsa2001090","article-title":"Automated identification of adults at risk for in-hospital clinical deterioration","volume":"383","author":"Escobar","year":"2020","journal-title":"N Engl J Med"},{"key":"2023081808481383100_ocad114-B61","first-page":"515","article-title":"A machine learning approach to predicting the stability of inpatient lab test results","volume":"2019","author":"Aikens","year":"2019","journal-title":"AMIA Summits Transl Sci Proc"},{"key":"2023081808481383100_ocad114-B62","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1016\/j.clinbiochem.2023.01.002","article-title":"Targeting repetitive laboratory testing with electronic health records-embedded predictive decision support: a pre-implementation study","volume":"113","author":"Rabbani","year":"2023","journal-title":"Clin Biochem"},{"key":"2023081808481383100_ocad114-B63","first-page":"641","author":"Kim","year":"2021"},{"key":"2023081808481383100_ocad114-B64","doi-asserted-by":"crossref","first-page":"103637","DOI":"10.1016\/j.jbi.2020.103637","article-title":"Language models are an effective representation learning technique for electronic health record data","volume":"113","author":"Steinberg","year":"2021","journal-title":"J Biomed Inform"},{"key":"2023081808481383100_ocad114-B65","first-page":"108","article-title":"Personalized antibiograms: machine learning for precision selection of empiric antibiotics","volume":"2020","author":"Corbin","year":"2020","journal-title":"AMIA Summits Transl Sci Proc"},{"issue":"1","key":"2023081808481383100_ocad114-B66","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1038\/s41746-018-0029-1","article-title":"Scalable and accurate deep learning with electronic health records","volume":"1","author":"Rajkomar","year":"2018","journal-title":"NPJ Digit Med"},{"key":"2023081808481383100_ocad114-B67","author":"Grinsztajn","year":"2022"},{"key":"2023081808481383100_ocad114-B68","doi-asserted-by":"crossref","DOI":"10.1007\/978-0-387-84858-7","volume-title":"The Elements of Statistical Learning: data Mining, Inference, and Prediction","author":"Hastie","year":"2009"},{"key":"2023081808481383100_ocad114-B69","author":"Garg","year":"2020"},{"key":"2023081808481383100_ocad114-B70","article-title":"Failing loudly: an empirical study of methods for detecting dataset shift","volume":"32","author":"Rabanser","year":"2019","journal-title":"Adv Neural Inf Process Syst"},{"key":"2023081808481383100_ocad114-B71","doi-asserted-by":"crossref","first-page":"103611","DOI":"10.1016\/j.jbi.2020.103611","article-title":"Detection of calibration drift in clinical prediction models to inform model updating","volume":"112","author":"Davis","year":"2020","journal-title":"J Biomed Inform"},{"issue":"12","key":"2023081808481383100_ocad114-B72","doi-asserted-by":"crossref","first-page":"1448","DOI":"10.1093\/jamia\/ocz127","article-title":"A nonparametric updating method to correct clinical prediction model drift","volume":"26","author":"Davis","year":"2019","journal-title":"J Am Med Inform Assoc"},{"issue":"12","key":"2023081808481383100_ocad114-B73","doi-asserted-by":"crossref","first-page":"1645","DOI":"10.1093\/jamia\/ocz145","article-title":"Prognostic models will be victims of their own success, unless","volume":"26","author":"Lenert","year":"2019","journal-title":"J Am Med Inform Assoc"},{"key":"2023081808481383100_ocad114-B74","first-page":"7599","author":"Perdomo","year":"2020"},{"key":"2023081808481383100_ocad114-B75","first-page":"710","author":"Adam","year":"2020"},{"key":"2023081808481383100_ocad114-B76","first-page":"5","author":"Adam","year":"2022"},{"key":"2023081808481383100_ocad114-B77","author":"Corbin","year":"2022"},{"issue":"11","key":"2023081808481383100_ocad114-B78","doi-asserted-by":"crossref","first-page":"2423","DOI":"10.1093\/jamia\/ocab118","article-title":"Developing machine learning models to personalize care levels among emergency room patients for hospital admission","volume":"28","author":"Nguyen","year":"2021","journal-title":"J Am Med Inform Assoc"},{"issue":"2","key":"2023081808481383100_ocad114-B79","doi-asserted-by":"crossref","first-page":"339","DOI":"10.1093\/jamia\/ocv091","article-title":"OrderRex: clinical order decision support and outcome predictions by data-mining electronic medical records","volume":"23","author":"Chen","year":"2016","journal-title":"J Am Med Inform Assoc"},{"issue":"10","key":"2023081808481383100_ocad114-B80","doi-asserted-by":"crossref","first-page":"2212","DOI":"10.1093\/jamia\/ocab099","article-title":"Machine learning for initial insulin estimation in hospitalized patients","volume":"28","author":"Nguyen","year":"2021","journal-title":"J Am Med Inform Assoc"},{"issue":"12","key":"2023081808481383100_ocad114-B81","doi-asserted-by":"crossref","first-page":"e24824","DOI":"10.2196\/24824","article-title":"The 21st century cures act: a competitive apps market and the risk of innovation blocking","volume":"22","author":"Gordon","year":"2020","journal-title":"J Med Internet Res"},{"key":"2023081808481383100_ocad114-B82","author":"Centers for Medicare & Medicaid Services (CMS)"},{"key":"2023081808481383100_ocad114-B83","author":"FHIR (Fast Healthcare Interoperability Resources)","year":"2023"},{"issue":"1","key":"2023081808481383100_ocad114-B84","first-page":"33","article-title":"Electronic alerting and decision support for early sepsis detection and management: impact on clinical outcomes","volume":"19","author":"Rico","year":"2017","journal-title":"Eur J Clin Pharm: Farm"},{"key":"2023081808481383100_ocad114-B85","author":"Cerner","year":"2023"},{"key":"2023081808481383100_ocad114-B86","author":"Pricing\u2014functions: Microsoft Azure"},{"key":"2023081808481383100_ocad114-B87","article-title":"Azure cosmos DB autoscale provisioned throughput;","author":"Microsoft","year":"2023"},{"issue":"7","key":"2023081808481383100_ocad114-B88","doi-asserted-by":"crossref","first-page":"409","DOI":"10.1136\/jclinpath-2020-206908","article-title":"Current and future applications of artificial intelligence in pathology: a clinical perspective","volume":"74","author":"Rakha","year":"2021","journal-title":"J Clin Pathol"},{"issue":"4","key":"2023081808481383100_ocad114-B89","doi-asserted-by":"crossref","first-page":"1805","DOI":"10.1007\/s00330-020-07230-9","article-title":"Applications of artificial intelligence (AI) in diagnostic radiology: a technography study","volume":"31","author":"Rezazade Mehrizi","year":"2021","journal-title":"Eur Radiol"},{"issue":"6","key":"2023081808481383100_ocad114-B90","doi-asserted-by":"crossref","first-page":"881","DOI":"10.4103\/idoj.IDOJ_388_20","article-title":"Artificial intelligence in dermatology: a practical introduction to a paradigm shift","volume":"11","author":"Eapen","year":"2020","journal-title":"Indian Dermatol Online J"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/30\/9\/1532\/51141458\/ocad114.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/30\/9\/1532\/51141458\/ocad114.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,18]],"date-time":"2023-08-18T10:25:03Z","timestamp":1692354303000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/30\/9\/1532\/7208854"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,27]]},"references-count":90,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2023,6,27]]},"published-print":{"date-parts":[[2023,8,18]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocad114","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,9,1]]},"published":{"date-parts":[[2023,6,27]]}}}