{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T10:44:12Z","timestamp":1776077052927,"version":"3.50.1"},"reference-count":75,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2020,7,25]],"date-time":"2020-07-25T00:00:00Z","timestamp":1595635200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000092","name":"U.S. National Library of Medicine","doi-asserted-by":"publisher","award":["T15LM011271"],"award-info":[{"award-number":["T15LM011271"]}],"id":[{"id":"10.13039\/100000092","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100005242","name":"Heed Ophthalmic Foundation","doi-asserted-by":"publisher","award":["NA"],"award-info":[{"award-number":["NA"]}],"id":[{"id":"10.13039\/100005242","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Informatics"],"abstract":"<jats:p>Predictive analytics using electronic health record (EHR) data have rapidly advanced over the last decade. While model performance metrics have improved considerably, best practices for implementing predictive models into clinical settings for point-of-care risk stratification are still evolving. Here, we conducted a systematic review of articles describing predictive models integrated into EHR systems and implemented in clinical practice. We conducted an exhaustive database search and extracted data encompassing multiple facets of implementation. We assessed study quality and level of evidence. We obtained an initial 3393 articles for screening, from which a final set of 44 articles was included for data extraction and analysis. The most common clinical domains of implemented predictive models were related to thrombotic disorders\/anticoagulation (25%) and sepsis (16%). The majority of studies were conducted in inpatient academic settings. Implementation challenges included alert fatigue, lack of training, and increased work burden on the care team. Of 32 studies that reported effects on clinical outcomes, 22 (69%) demonstrated improvement after model implementation. Overall, EHR-based predictive models offer promising results for improving clinical outcomes, although several gaps in the literature remain, and most study designs were observational. Future studies using randomized controlled trials may help improve the generalizability of findings.<\/jats:p>","DOI":"10.3390\/informatics7030025","type":"journal-article","created":{"date-parts":[[2020,7,27]],"date-time":"2020-07-27T04:39:50Z","timestamp":1595824790000},"page":"25","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":104,"title":["Clinical Implementation of Predictive Models Embedded within Electronic Health Record Systems: A Systematic Review"],"prefix":"10.3390","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8560-250X","authenticated-orcid":false,"given":"Terrence C.","family":"Lee","sequence":"first","affiliation":[{"name":"Viterbi Family Department of Ophthalmology and Shiley Eye Institute, University of California San Diego, La Jolla, CA 92093, USA"},{"name":"Division of Biomedical Informatics, Department of Medicine, University of California San Diego, La Jolla, CA 92093, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7622-093X","authenticated-orcid":false,"given":"Neil U.","family":"Shah","sequence":"additional","affiliation":[{"name":"Viterbi Family Department of Ophthalmology and Shiley Eye Institute, University of California San Diego, La Jolla, CA 92093, USA"},{"name":"Division of Biomedical Informatics, Department of Medicine, University of California San Diego, La Jolla, CA 92093, USA"}]},{"given":"Alyssa","family":"Haack","sequence":"additional","affiliation":[{"name":"Viterbi Family Department of Ophthalmology and Shiley Eye Institute, University of California San Diego, La Jolla, CA 92093, USA"},{"name":"Division of Biomedical Informatics, Department of Medicine, University of California San Diego, La Jolla, CA 92093, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5271-7690","authenticated-orcid":false,"given":"Sally L.","family":"Baxter","sequence":"additional","affiliation":[{"name":"Viterbi Family Department of Ophthalmology and Shiley Eye Institute, University of California San Diego, La Jolla, CA 92093, USA"},{"name":"Division of Biomedical Informatics, Department of Medicine, University of California San Diego, La Jolla, CA 92093, USA"}]}],"member":"1968","published-online":{"date-parts":[[2020,7,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"651","DOI":"10.1001\/jama.2015.19417","article-title":"Integrating Predictive Analytics into High-Value Care: The Dawn of Precision Delivery","volume":"315","author":"Parikh","year":"2016","journal-title":"JAMA"},{"key":"ref_2","unstructured":"Henry, J., Pylypchuk, Y., Searcy, T., and Patel, V. (2019, September 18). Adoption of Electronic Health Record Systems among U.S. Non-Federal Acute Care Hospitals: 2008\u20132015. Available online: \/evaluations\/data-briefs\/non-federal-acute-care-hospital-ehr-adoption-2008-2015.php."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1016\/j.ijmedinf.2016.06.017","article-title":"Impact of the HITECH financial incentives on EHR adoption in small, physician-owned practices","volume":"94","author":"Cohen","year":"2016","journal-title":"Int. J. Med. Inform."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"375","DOI":"10.1093\/jamia\/ocv103","article-title":"Impact of the HITECH Act on physicians\u2019 adoption of electronic health records","volume":"23","author":"Mennemeyer","year":"2016","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"652","DOI":"10.1097\/SLA.0000000000002706","article-title":"MySurgeryRisk: Development and Validation of a Machine-learning Risk Algorithm for Major Complications and Death After Surgery","volume":"269","author":"Bihorac","year":"2018","journal-title":"Ann. Surg."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"3271","DOI":"10.1177\/0962280217696115","article-title":"Leveraging electronic health records for predictive modeling of post-surgical complications","volume":"27","author":"Weller","year":"2017","journal-title":"Stat. Methods Med. Res."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Corey, K.M., Kashyap, S., Lorenzi, E., Lagoo-Deenadayalan, S.A., Heller, K., Whalen, K., Balu, S., Heflin, M.T., McDonald, S.R., and Swaminathan, M. (2018). Development and validation of machine learning models to identify high-risk surgical patients using automatically curated electronic health record data (Pythia): A retrospective, single-site study. PLoS Med., 15.","DOI":"10.1371\/journal.pmed.1002701"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"547","DOI":"10.1097\/CCM.0000000000002936","article-title":"An Interpretable Machine Learning Model for Accurate Prediction of Sepsis in the ICU","volume":"46","author":"Nemati","year":"2018","journal-title":"Crit. Care Med."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1038\/s41746-018-0029-1","article-title":"Scalable and accurate deep learning with electronic health records","volume":"1","author":"Rajkomar","year":"2018","journal-title":"NPJ Digit. Med."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1016\/j.jbi.2018.09.009","article-title":"Predicting the risk of acute care readmissions among rehabilitation inpatients: A machine learning approach","volume":"86","author":"Xue","year":"2018","journal-title":"J. Biomed. Inform."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1016\/j.cmpb.2018.06.006","article-title":"Predictive models for hospital readmission risk: A systematic review of methods","volume":"164","author":"Artetxe","year":"2018","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"983","DOI":"10.1016\/j.ijmedinf.2012.12.005","article-title":"Automatic identification of heart failure diagnostic criteria, using text analysis of clinical notes from electronic health records","volume":"83","author":"Byrd","year":"2014","journal-title":"Int. J. Med. Inform."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"214","DOI":"10.1089\/big.2018.0002","article-title":"Electronic Health Record-Based Screening for Substance Abuse","volume":"6","author":"Alemi","year":"2018","journal-title":"Big Data"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1111\/acem.12876","article-title":"Prediction of In-hospital Mortality in Emergency Department Patients with Sepsis: A Local Big Data-Driven, Machine Learning Approach","volume":"23","author":"Taylor","year":"2016","journal-title":"Acad. Emerg. Med."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Hoogendoorn, M., El Hassouni, A., Mok, K., Ghassemi, M., and Szolovits, P. (2016, January 16\u201320). Prediction using patient comparison vs. modeling: A case study for mortality prediction. Proceedings of the 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Orlando, FL, USA.","DOI":"10.1109\/EMBC.2016.7591229"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1589","DOI":"10.1109\/JBHI.2017.2767063","article-title":"Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis","volume":"22","author":"Shickel","year":"2018","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"198","DOI":"10.1093\/jamia\/ocw042","article-title":"Opportunities and challenges in developing risk prediction models with electronic health records data: A systematic review","volume":"24","author":"Goldstein","year":"2017","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"e38","DOI":"10.2196\/medinform.5359","article-title":"Challenges and Opportunities of Big Data in Health Care: A Systematic Review","volume":"4","author":"Kruse","year":"2016","journal-title":"JMIR Med. Inform."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"6556","DOI":"10.1200\/JCO.2019.37.15_suppl.6556","article-title":"Development of an artificial intelligence model to predict survival at specific time intervals for lung cancer patients","volume":"37","author":"Agrawal","year":"2019","journal-title":"J. Clin. Oncol."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"e13659","DOI":"10.2196\/13659","article-title":"Artificial Intelligence and the Implementation Challenge","volume":"21","author":"Shaw","year":"2019","journal-title":"J. Med. Internet Res."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1038\/s41591-018-0307-0","article-title":"The practical implementation of artificial intelligence technologies in medicine","volume":"25","author":"He","year":"2019","journal-title":"Nat. Med."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Liberati, A., Altman, D.G., Tetzlaff, J., Mulrow, C., G\u00f8tzsche, P.C., Ioannidis, J.P.A., Clarke, M., Devereaux, P.J., Kleijnen, J., and Moher, D. (2009). The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration. PLoS Med., 6.","DOI":"10.2427\/5768"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Moher, D., Liberati, A., Tetzlaff, J., and Altman, D.G. (2009). The PRISMA Group Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med., 6.","DOI":"10.1371\/journal.pmed.1000097"},{"key":"ref_24","unstructured":"Rights (OCR), O. for C (2020, June 29). HITECH Act Enforcement Interim Final Rule, Available online: https:\/\/www.hhs.gov\/hipaa\/for-professionals\/special-topics\/HITECH-act-enforcement-interim-final-rule\/index.html."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"377","DOI":"10.1136\/jech.52.6.377","article-title":"The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions","volume":"52","author":"Downs","year":"1998","journal-title":"J. Epidemiol. Community Health"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1002\/jhm.562","article-title":"Optimizing Prevention of Hospital-acquired Venous Thromboembolism (VTE): Prospective Validation of a VTE Risk Assessment Model","volume":"5","author":"Maynard","year":"2010","journal-title":"J. Hosp. Med."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"648","DOI":"10.1016\/j.jvs.2009.08.097","article-title":"Prevention of thromboembolic events in surgical patients through the creation and implementation of a computerized risk assessment program","volume":"51","author":"Novis","year":"2010","journal-title":"J. Vasc. Surg."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"607","DOI":"10.1016\/j.ijmedinf.2011.06.009","article-title":"Effects of a computerized decision support system on pressure ulcers and malnutrition in nursing homes for the elderly","volume":"80","author":"Fossum","year":"2011","journal-title":"Int. J. Med. Inform."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1097\/CCM.0b013e3181fa4184","article-title":"Limiting ventilator-induced lung injury through individual electronic medical record surveillance","volume":"39","author":"Herasevich","year":"2011","journal-title":"Crit. Care Med."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"500","DOI":"10.1016\/j.annemergmed.2010.12.008","article-title":"Prospective trial of real-time electronic surveillance to expedite early care of severe sepsis","volume":"57","author":"Nelson","year":"2011","journal-title":"Ann. Emerg. Med."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Umscheid, C.A., Hanish, A., Chittams, J., Weiner, M.G., and Hecht, T.E.H. (2012). Effectiveness of a novel and scalable clinical decision support intervention to improve venous thromboembolism prophylaxis: A quasi-experimental study. BMC Med. Inform. Decis. Mak., 12.","DOI":"10.1186\/1472-6947-12-92"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"689","DOI":"10.1002\/jhm.2106","article-title":"The readmission risk flag: Using the electronic health record to automatically identify patients at risk for 30-day readmission","volume":"8","author":"Baillie","year":"2013","journal-title":"J. Hosp. Med."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"998","DOI":"10.1136\/bmjqs-2013-001901","article-title":"Allocating scarce resources in real-time to reduce heart failure readmissions: A prospective, controlled study","volume":"22","author":"Amarasingham","year":"2013","journal-title":"BMJ Qual. Saf."},{"key":"ref_34","first-page":"S448","article-title":"Use of clinical decision support tools to improve primary care chronic kidney disease management","volume":"28","author":"Litvin","year":"2013","journal-title":"J. Gen. Intern. Med."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"281","DOI":"10.1111\/nicc.12048","article-title":"Automatic delirium prediction system in a Korean surgical intensive care unit","volume":"19","author":"Oh","year":"2014","journal-title":"Nurs. Crit. Care"},{"key":"ref_36","first-page":"1010","article-title":"Development, Implementation and Use of Electronic Surveillance for Ventilator-Associated Events (VAE) in Adults","volume":"2014","author":"Resetar","year":"2014","journal-title":"AMIA Annu. Symp. Proc."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1111\/jhq.12069","article-title":"Computerized Clinical Decision Support to Prevent Venous Thromboembolism Among Hospitalized Patients: Proximal Outcomes from a Multiyear Quality Improvement Project","volume":"37","author":"Amland","year":"2015","journal-title":"J. Healthc Qual."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Faerber, A.E., Horvath, R., Stillman, C., O\u2019Connell, M.L., Hamilton, A.L., Newhall, K.A., Likosky, D.S., and Goodney, P.P. (2015). Development and pilot feasibility study of a health information technology tool to calculate mortality risk for patients with asymptomatic carotid stenosis: The Carotid Risk Assessment Tool (CARAT). BMC Med. Inform. Decis. Mak., 15.","DOI":"10.1186\/s12911-015-0141-y"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Hao, S., Wang, Y., Jin, B., Shin, A.Y., Zhu, C., Huang, M., Zheng, L., Luo, J., Hu, Z., and Fu, C. (2015). Development, Validation and Deployment of a Real Time 30 Day Hospital Readmission Risk Assessment Tool in the Maine Healthcare Information Exchange. PLoS ONE, 10.","DOI":"10.1371\/journal.pone.0140271"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1142","DOI":"10.13063\/2327-9214.1142","article-title":"TeenBP: Development and Piloting of an EHR-Linked Clinical Decision Support System to Improve Recognition of Hypertension in Adolescents","volume":"3","author":"Kharbanda","year":"2015","journal-title":"EGEMS"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1099","DOI":"10.1016\/j.thromres.2015.08.002","article-title":"Implementation and validation of a risk stratification method at The Ottawa Hospital to guide thromboprophylaxis in ambulatory cancer patients at intermediate-high risk for venous thrombosis","volume":"136","author":"Lustig","year":"2015","journal-title":"Thromb. Res."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1002\/jhm.2259","article-title":"Development, Implementation, and Impact of an Automated Early Warning and Response System for Sepsis","volume":"10","author":"Umscheid","year":"2015","journal-title":"J. Hosp. Med."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"e20154031","DOI":"10.1542\/peds.2015-4031","article-title":"Risk Stratification to Decrease Unnecessary Diagnostic Imaging for Acute Appendicitis","volume":"138","author":"Depinet","year":"2016","journal-title":"Pediatrics"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/j.ajem.2015.10.005","article-title":"Effect of an electronic medical record alert for severe sepsis among ED patients","volume":"34","author":"Narayanan","year":"2016","journal-title":"Am. J. Emerg. Med."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"S54","DOI":"10.1016\/j.annemergmed.2016.08.148","article-title":"Impact of electronic clinical decision support on initial site of care for emergency department patients with acute pulmonary embolism","volume":"68","author":"Vinson","year":"2016","journal-title":"Ann. Emerg. Med."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.ijmedinf.2017.04.001","article-title":"Prospective validation of a near real-time EHR-integrated automated SOFA score calculator","volume":"103","author":"Aakre","year":"2017","journal-title":"Int. J. Med. Inform."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Arts, D.L., Abu-Hanna, A., Medlock, S.K., and van Weert, H.C.P.M. (2017). Effectiveness and usage of a decision support system to improve stroke prevention in general practice: A cluster randomized controlled trial. PLoS ONE, 12.","DOI":"10.1371\/journal.pone.0170974"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"839","DOI":"10.1111\/acem.13195","article-title":"Embedded Clinical Decision Support in Electronic Health Record Decreases Use of High-cost Imaging in the Emergency Department: EmbED study","volume":"24","author":"Bookman","year":"2017","journal-title":"Acad. Emerg. Med."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"462","DOI":"10.1097\/NNR.0000000000000245","article-title":"Automated Pressure Injury Risk Assessment System Incorporated Into an Electronic Health Record System","volume":"66","author":"Jin","year":"2017","journal-title":"Nurs. Res."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"1111","DOI":"10.1093\/jamia\/ocx065","article-title":"Implementation of a scalable, web-based, automated clinical decision support risk-prediction tool for chronic kidney disease using C-CDA and application programming interfaces","volume":"24","author":"Samal","year":"2017","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"e000234","DOI":"10.1136\/bmjresp-2017-000234","article-title":"Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: A randomised clinical trial","volume":"4","author":"Shimabukuro","year":"2017","journal-title":"BMJ Open Respir. Res."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"569","DOI":"10.1212\/WNL.90.15_supplement.P5.234","article-title":"Electronic decision support for improvement of contemporary therapy for stroke prevention (EDICTS)","volume":"90","author":"Chaturvedi","year":"2018","journal-title":"Neurology"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"1324","DOI":"10.1007\/s11606-018-4468-9","article-title":"Effect of Low Back Pain Risk-Stratification Strategy on Patient Outcomes and Care Processes: The MATCH Randomized Trial in Primary Care","volume":"33","author":"Cherkin","year":"2018","journal-title":"J. Gen. Intern. Med."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"e004553","DOI":"10.1161\/CIRCOUTCOMES.118.004553","article-title":"Value-Based ST-Segment-Elevation Myocardial Infarction Care Using Risk-Guided Triage and Early Discharge","volume":"11","author":"Ebinger","year":"2018","journal-title":"Circ. Cardiovasc. Qual. Outcomes"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"316","DOI":"10.1016\/j.ajic.2017.09.006","article-title":"Development and validation of an automated ventilator-associated event electronic surveillance system: A report of a successful implementation","volume":"46","author":"Hebert","year":"2018","journal-title":"Am. J. Infect. Control."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1016\/j.jss.2018.05.078","article-title":"Sooner is better: Use of a real-time automated bedside dashboard improves sepsis care","volume":"231","author":"Jung","year":"2018","journal-title":"J. Surg. Res."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1097\/NCQ.0000000000000266","article-title":"Automated Medication Error Risk Assessment System (Auto-MERAS)","volume":"33","author":"Kang","year":"2018","journal-title":"J. Nurs. Care Qual."},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Karlsson, L., Nilsson, S., B\u00e5ng, M., Nilsson, L., Charitakis, E., and Janzon, M. (2018). A clinical decision support tool for improving adherence to guidelines on anticoagulant therapy in patients with atrial fibrillation at risk of stroke: A cluster-randomized trial in a Swedish primary care setting (the CDS-AF study). PLoS Med., 15.","DOI":"10.1371\/journal.pmed.1002528"},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"46","DOI":"10.1016\/j.ijnurstu.2017.09.014","article-title":"Development and validation of an automated delirium risk assessment system (Auto-DelRAS) implemented in the electronic health record system","volume":"77","author":"Moon","year":"2018","journal-title":"Int. J. Nurs. Stud."},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"202","DOI":"10.1089\/apc.2018.0011","article-title":"Which Patients in the Emergency Department Should Receive Preexposure Prophylaxis? Implementation of a Predictive Analytics Approach","volume":"32","author":"Ridgway","year":"2018","journal-title":"AIDS Patient Care STDS"},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1016\/j.jss.2018.01.013","article-title":"Fewer thromboembolic events after implementation of a venous thromboembolism risk stratification tool","volume":"225","author":"Turrentine","year":"2018","journal-title":"J. Surg. Res."},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"375","DOI":"10.1093\/intqhc\/mzy056","article-title":"Decreasing triage time: Effects of implementing a step-wise ESI algorithm in an EHR","volume":"30","author":"Villa","year":"2018","journal-title":"Int. J. Qual. Health Care"},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"855","DOI":"10.7326\/M18-1206","article-title":"Increasing Safe Outpatient Management of Emergency Department Patients with Pulmonary Embolism A Controlled Pragmatic Trial","volume":"169","author":"Vinson","year":"2018","journal-title":"Ann. Intern. Med."},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1097\/CCM.0000000000003439","article-title":"Minimal Impact of Implemented Early Warning Score and Best Practice Alert for Patient Deterioration","volume":"47","author":"Bedoya","year":"2019","journal-title":"Crit. Care Med."},{"key":"ref_65","doi-asserted-by":"crossref","first-page":"1035","DOI":"10.1016\/j.surg.2019.01.002","article-title":"Comparing clinical judgment with the MySurgeryRisk algorithm for preoperative risk assessment: A pilot usability study","volume":"165","author":"Brennan","year":"2019","journal-title":"Surgery"},{"key":"ref_66","doi-asserted-by":"crossref","first-page":"15","DOI":"10.5334\/egems.282","article-title":"Development of a Clinical Decision Support System for Pediatric Abdominal Pain in Emergency Department Settings Across Two Health Systems Within the HCSRN","volume":"7","author":"Ekstrom","year":"2019","journal-title":"EGEMS"},{"key":"ref_67","doi-asserted-by":"crossref","first-page":"1485","DOI":"10.1097\/CCM.0000000000003891","article-title":"A Machine Learning Algorithm to Predict Severe Sepsis and Septic Shock: Development, Implementation, and Impact on Clinical Practice","volume":"47","author":"Giannini","year":"2019","journal-title":"Crit. Care Med."},{"key":"ref_68","doi-asserted-by":"crossref","first-page":"e14022","DOI":"10.2196\/14022","article-title":"A Pragmatic Cluster Randomized Trial of an Electronic Clinical Decision Support System to Improve Chronic Kidney Disease Management in Primary Care: Design, Rationale, and Implementation Experience","volume":"8","author":"Khoong","year":"2019","journal-title":"JMIR Res. Protoc."},{"key":"ref_69","doi-asserted-by":"crossref","first-page":"e000386","DOI":"10.1136\/bmjoq-2018-000386","article-title":"Putting veterans with heart failure FIRST improves follow-up and reduces readmissions","volume":"8","author":"Ogunwole","year":"2019","journal-title":"BMJ Open Qual."},{"key":"ref_70","doi-asserted-by":"crossref","first-page":"e145","DOI":"10.1136\/amiajnl-2011-000743","article-title":"Evaluating alert fatigue over time to EHR-based clinical trial alerts: Findings from a randomized controlled study","volume":"19","author":"Embi","year":"2012","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_71","doi-asserted-by":"crossref","first-page":"461","DOI":"10.1097\/SLA.0000000000002023","article-title":"Path from Predictive Analytics to Improved Patient Outcomes: A Framework to Guide Use, Implementation, and Evaluation of Accurate Surgical Predictive Models","volume":"265","author":"Harris","year":"2017","journal-title":"Ann. Surg."},{"key":"ref_72","doi-asserted-by":"crossref","first-page":"709","DOI":"10.1093\/jamia\/ocaa021","article-title":"\u201cHow did you get to this number?\u201d Stakeholder needs for implementing predictive analytics: A pre-implementation qualitative study","volume":"27","author":"Benda","year":"2020","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_73","unstructured":"Fountaine, T., McCarthy, B., and Saleh, T. (2019, July 01). Building the AI-Powered Organization. Available online: https:\/\/hbr.org\/2019\/07\/building-the-ai-powered-organization."},{"key":"ref_74","doi-asserted-by":"crossref","first-page":"e367","DOI":"10.2196\/jmir.8775","article-title":"Beyond Adoption: A New Framework for Theorizing and Evaluating Nonadoption, Abandonment, and Challenges to the Scale-Up, Spread, and Sustainability of Health and Care Technologies","volume":"19","author":"Greenhalgh","year":"2017","journal-title":"J. Med. Internet Res."},{"key":"ref_75","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1055\/s-0039-1688753","article-title":"Local Investment in Training Drives Electronic Health Record User Satisfaction","volume":"10","author":"Longhurst","year":"2019","journal-title":"Appl. Clin. Inform."}],"container-title":["Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2227-9709\/7\/3\/25\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:51:40Z","timestamp":1760176300000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2227-9709\/7\/3\/25"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,7,25]]},"references-count":75,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2020,9]]}},"alternative-id":["informatics7030025"],"URL":"https:\/\/doi.org\/10.3390\/informatics7030025","relation":{},"ISSN":["2227-9709"],"issn-type":[{"value":"2227-9709","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,7,25]]}}}