{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T19:52:35Z","timestamp":1792785155728,"version":"4.5.0"},"reference-count":35,"publisher":"Georg Thieme Verlag KG","issue":"04","funder":[{"name":"Patient-Centered Outcomes Research Institute","award":["P30HS02744"],"award-info":[{"award-number":["P30HS02744"]}],"id":[{"id":"https:\/\/ror.org\/014q65q44","id-type":"ROR","asserted-by":"crossref"}]},{"name":"Agency for Healthcare Quality and Research","award":["P30HS02744"],"award-info":[{"award-number":["P30HS02744"]}]},{"name":"National Center for Advancing Translational Sciences of the National Institutes of Health","award":["UM1TR004405"],"award-info":[{"award-number":["UM1TR004405"]}],"id":[{"id":"https:\/\/ror.org\/04pw6fb54","id-type":"ROR","asserted-by":"crossref"}]},{"name":"Minnesota Partnership for Biotechnology and Medical Genomics Grant","award":["00110451"],"award-info":[{"award-number":["00110451"]}],"id":[{"id":"https:\/\/ror.org\/0417t4z61","id-type":"ROR","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Appl Clin Inform"],"published-print":{"date-parts":[[2026,8]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Background Sepsis is a life-threatening condition in which delayed recognition and treatment are associated with increased mortality. While predictive models such as Epic\u2019s Early Detection of Sepsis Model (ESM) were developed to support early intervention, their real-world effects after integration into clinical workflows remain difficult to evaluate.<\/jats:p>\n                  <jats:p>Objectives This study aimed to evaluate the real-world effects of ESM integrated into clinical workflow on clinical outcomes, antibiotic use, and harm\u2013benefit trade-offs.<\/jats:p>\n                  <jats:p>Methods We conducted a quasi-experimental study in a single health care system using encounter-level data from inpatient settings. Inpatient mortality, prolonged hospitalization, antibiotic use, and sepsis prevalence (defined by electronic health record [EHR]) were compared between the preacquisition period (June 3, 2023\u2013June 24, 2024) and the online period (August 21, 2024\u2013December 26, 2024) when the model became visible to clinicians. We also applied a counterfactual framework using models trained on preacquisition data to estimate expected outcomes without ESM and to quantify harms related to overtreatment and delayed treatment.<\/jats:p>\n                  <jats:p>Results Among 89,469 encounters, 75,215 occurred during the preacquisition period and 14,254 during the online period. In unadjusted analyses, inpatient mortality, prolonged hospitalization, antibiotic use, and EHR-defined sepsis proportion all decreased during the online period (all p \u2264 0.006). In the counterfactual analyses, observed outcomes were lower than expected without ESM for mortality (1.21 vs. 1.94%; p &lt; 0.001), prolonged hospitalization (5.56 vs. 8.14%; p &lt; 0.001), and antibiotic use (43.52 vs. 47.74%; p &lt; 0.001). False positive harm (37.72 vs. 42.31%; p &lt; 0.001) was also lower than expected.<\/jats:p>\n                  <jats:p>Conclusion Integration of ESM into clinical workflow was associated with improved patient outcomes, reduced antibiotic use, and decreased harm from overtreatment, without evidence of increased harm from delayed treatment, supporting a positive net clinical benefit and the safety and effectiveness of ESM under Software as a Medical Device principles.<\/jats:p>","DOI":"10.1055\/a-2962-8091","type":"journal-article","created":{"date-parts":[[2026,9,25]],"date-time":"2026-09-25T23:16:01Z","timestamp":1790378161000},"page":"854-862","source":"Crossref","is-referenced-by-count":0,"title":["Real-World Effects of a Sepsis Early Detection Model Integrated into Clinical Workflow: A Quasi-Experimental Study"],"prefix":"10.1055","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-2917-0542","authenticated-orcid":false,"given":"Yifan","family":"Zhang","sequence":"additional","affiliation":[{"name":"Center for Learning Health System Sciences, University of Minnesota, Minnesota, United States","place":["Minneapolis"]},{"name":"Institute for Health Informatics, University of Minnesota, Minnesota, United States","place":["Minneapolis"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9538-0864","authenticated-orcid":false,"given":"Sarolta H.","family":"Trinh","sequence":"additional","affiliation":[{"name":"Center for Learning Health System Sciences, University of Minnesota, Minnesota, United States","place":["Minneapolis"]},{"name":"Institute for Health Informatics, University of Minnesota, Minnesota, United States","place":["Minneapolis"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tom","family":"Phelan","sequence":"additional","affiliation":[{"name":"Center for Learning Health System Sciences, University of Minnesota, Minnesota, United States","place":["Minneapolis"]},{"name":"Fairview Health Services, Minnesota, United States","place":["Minneapolis"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thomas F.","family":"Byrd","sequence":"additional","affiliation":[{"name":"Center for Learning Health System Sciences, University of Minnesota, Minnesota, United States","place":["Minneapolis"]},{"name":"University of Minnesota, Minnesota, United States","place":["Minneapolis"],"department":["Department of Medicine"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Roshan","family":"Tourani","sequence":"additional","affiliation":[{"name":"Center for Learning Health System Sciences, University of Minnesota, Minnesota, United States","place":["Minneapolis"]},{"name":"Institute for Health Informatics, University of Minnesota, Minnesota, United States","place":["Minneapolis"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Vipin","family":"Kumar","sequence":"additional","affiliation":[{"name":"University of Minnesota, Minnesota, United States","place":["Minneapolis"],"department":["Department of Computer Science and Engineering"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pedro J.","family":"Caraballo","sequence":"additional","affiliation":[{"name":"Mayo Clinic, Minnesota, United States","place":["Rochester"],"department":["Department of Medicine"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Genevieve B.","family":"Melton-Meaux","sequence":"additional","affiliation":[{"name":"Center for Learning Health System Sciences, University of Minnesota, Minnesota, United States","place":["Minneapolis"]},{"name":"Institute for Health Informatics, University of Minnesota, Minnesota, United States","place":["Minneapolis"]},{"name":"University of Minnesota, Minnesota, United States","place":["Minneapolis"],"department":["Department of Surgery"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gyorgy J.","family":"Simon","sequence":"additional","affiliation":[{"name":"Center for Learning Health System Sciences, University of Minnesota, Minnesota, United States","place":["Minneapolis"]},{"name":"Institute for Health Informatics, University of Minnesota, Minnesota, United States","place":["Minneapolis"]},{"name":"University of Minnesota, Minnesota, United States","place":["Minneapolis"],"department":["Department of Medicine"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"194","published-online":{"date-parts":[[2026,10,7]]},"reference":[{"issue":"12","key":"ref1","doi-asserted-by":"crossref","first-page":"e2013","DOI":"10.1016\/S2214-109X(25)00356-0","article-title":"Global, regional, and national sepsis incidence and mortality, 1990-2021: a systematic analysis","volume":"13","author":"GBD 2021 Global Sepsis Collaborators","year":"2025","journal-title":"Lancet Glob Health"},{"issue":"01","key":"ref3","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1186\/s13054-021-03883-0","article-title":"Korean sepsis Alliance (KSA) investigators. Time-to-antibiotics and clinical outcomes in patients with sepsis and septic shock: a prospective nationwide multicenter cohort study","volume":"26","author":"Y Im","year":"2022","journal-title":"Crit Care"},{"issue":"08","key":"ref4","doi-asserted-by":"crossref","first-page":"1749","DOI":"10.1097\/CCM.0000000000000330","article-title":"Empiric antibiotic treatment reduces mortality in severe sepsis and septic shock from the first hour: results from a guideline-based performance improvement program","volume":"42","author":"R Ferrer","year":"2014","journal-title":"Crit Care Med"},{"key":"ref5","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1016\/j.ejim.2024.06.015","article-title":"Door-to-antibiotic time and mortality in patients with sepsis: systematic review and meta-analysis","volume":"129","author":"L Y Leung","year":"2024","journal-title":"Eur J Intern Med"},{"issue":"11","key":"ref6","doi-asserted-by":"crossref","first-page":"1181","DOI":"10.1007\/s00134-021-06506-y","article-title":"Surviving sepsis campaign: international guidelines for management of sepsis and septic shock 2021","volume":"47","author":"L Evans","year":"2021","journal-title":"Intensive Care Med"},{"issue":"07","key":"ref7","doi-asserted-by":"crossref","first-page":"856","DOI":"10.1164\/rccm.201609-1848OC","article-title":"The timing of early antibiotics and hospital mortality in sepsis","volume":"196","author":"V X Liu","year":"2017","journal-title":"Am J Respir Crit Care Med"},{"issue":"07","key":"ref8","doi-asserted-by":"crossref","first-page":"1349","DOI":"10.1093\/jamia\/ocad075","article-title":"Deployment of machine learning algorithms to predict sepsis: systematic review and application of the SALIENT clinical AI implementation framework","volume":"30","author":"A H van der","year":"2023","journal-title":"J Am Med Inform Assoc"},{"issue":"03","key":"ref9","doi-asserted-by":"crossref","first-page":"466","DOI":"10.1055\/a-2885-7810","article-title":"A novel digital phenotype for burn sepsis: leveraging electronic health record data and natural language processing to improve case definition","volume":"17","author":"N D Soulakis","year":"2026","journal-title":"Appl Clin Inform"},{"issue":"04","key":"ref10","doi-asserted-by":"crossref","first-page":"1332","DOI":"10.1055\/a-2618-4470","article-title":"Refining a machine learning model for predicting infant sepsis: a multidisciplinary team supported by human-centered design methods","volume":"16","author":"D Karavite","year":"2025","journal-title":"Appl Clin Inform"},{"issue":"04","key":"ref11","doi-asserted-by":"crossref","first-page":"1192","DOI":"10.1055\/a-2605-1847","article-title":"Performance degradation between development and deployment of a predictive model for central line-associated bloodstream infections in hospitalized children","volume":"16","author":"J M Beus","year":"2025","journal-title":"Appl Clin Inform"},{"issue":"01","key":"ref12","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1055\/a-1990-3037","article-title":"SEPRES: intensive care unit clinical data integration system to predict sepsis","volume":"14","author":"Q Chen","year":"2023","journal-title":"Appl Clin Inform"},{"issue":"05","key":"ref13","doi-asserted-by":"crossref","first-page":"1588","DOI":"10.1055\/a-2720-5448","article-title":"A FHIR-powered python implementation of the SENECA algorithm for sepsis subtyping","volume":"16","author":"A J King","year":"2025","journal-title":"Appl Clin Inform"},{"key":"ref14","volume-title":"Cognitive Computing Model Brief: Early Detection of Sepsis (Version 2)","author":"Epic Systems Corporation","year":"2022"},{"issue":"08","key":"ref15","doi-asserted-by":"crossref","first-page":"e2329729","DOI":"10.1001\/jamanetworkopen.2023.29729","article-title":"Sepsis prediction model for determining sepsis vs SIRS, qSOFA, and SOFA","volume":"6","author":"A R Schertz","year":"2023","journal-title":"JAMA Netw Open"},{"issue":"02","key":"ref16","doi-asserted-by":"crossref","first-page":"e260181","DOI":"10.1001\/jamanetworkopen.2026.0181","article-title":"Multicenter prospective validation of an updated proprietary sepsis prediction model","volume":"9","author":"A Wong","year":"2026","journal-title":"JAMA Netw Open"},{"issue":"01","key":"ref17","doi-asserted-by":"crossref","first-page":"e100109","DOI":"10.1136\/bmjhci-2019-100109","article-title":"Effect of a sepsis prediction algorithm on patient mortality, length of stay and readmission: a prospective multicentre clinical outcomes evaluation of real-world patient data from US hospitals","volume":"27","author":"H Burdick","year":"2020","journal-title":"BMJ Health Care Inform"},{"issue":"11","key":"ref18","doi-asserted-by":"crossref","first-page":"1485","DOI":"10.1097\/CCM.0000000000003891","article-title":"A machine learning algorithm to predict severe sepsis and septic shock: development, implementation, and impact on clinical practice","volume":"47","author":"H M Giannini","year":"2019","journal-title":"Crit Care Med"},{"key":"ref19","doi-asserted-by":"crossref","first-page":"378","DOI":"10.1016\/j.ajem.2021.09.086","article-title":"Implementation and evaluation of sepsis surveillance and decision support in medical ICU and emergency department","volume":"51","author":"K Lipatov","year":"2022","journal-title":"Am J Emerg Med"},{"key":"ref20","doi-asserted-by":"crossref","first-page":"154061","DOI":"10.1016\/j.jcrc.2022.154061","article-title":"Evaluation of the effectiveness of an automated sepsis predictive tool on patient outcomes","volume":"71","author":"M Schootman","year":"2022","journal-title":"J Crit Care"},{"key":"ref21","doi-asserted-by":"crossref","first-page":"108083","DOI":"10.1016\/j.ijid.2025.108083","article-title":"Assessing the early sepsis warning system in the emergency department","volume":"161","author":"S C Hsu","year":"2025","journal-title":"Int J Infect Dis"},{"issue":"01","key":"ref22","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1038\/s41746-023-00986-6","article-title":"Impact of a deep learning sepsis prediction model on quality of care and survival","volume":"7","author":"A Boussina","year":"2024","journal-title":"NPJ Digit Med"},{"issue":"09","key":"ref23","first-page":"e0046","article-title":"Retrospective observational study of the clinical performance characteristics of a machine learning approach to early sepsis identification","volume":"1","author":"R Topiwala","year":"2019","journal-title":"Crit Care Explor"},{"issue":"07","key":"ref24","doi-asserted-by":"crossref","first-page":"e15182","DOI":"10.2196\/15182","article-title":"Real-world integration of a sepsis deep learning technology into routine clinical care: implementation study","volume":"8","author":"M P Sendak","year":"2020","journal-title":"JMIR Med Inform"},{"issue":"01","key":"ref25","doi-asserted-by":"crossref","first-page":"e000234","DOI":"10.1136\/bmjresp-2017-000234","article-title":"Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: a randomised clinical trial","volume":"4","author":"D W Shimabukuro","year":"2017","journal-title":"BMJ Open Respir Res"},{"issue":"13","key":"ref26","doi-asserted-by":"crossref","first-page":"1716","DOI":"10.1111\/1471-0528.15199","article-title":"Randomised controlled trials - the gold standard for effectiveness research: study design: randomised controlled trials","volume":"125","author":"E Hariton","year":"2018","journal-title":"BJOG"},{"issue":"08","key":"ref27","doi-asserted-by":"crossref","first-page":"801","DOI":"10.1001\/jama.2016.0287","article-title":"The third international consensus definitions for sepsis and septic shock (Sepsis-3)","volume":"315","author":"M Singer","year":"2016","journal-title":"JAMA"},{"key":"ref28","volume-title":"Adult Sepsis Empiric Antibiotic Guidelines","author":"Partners Sepsis Task Force","year":"2018"},{"key":"ref29","volume-title":"Systemic inflammatory response syndrome","author":"S Baddam","year":"2025"},{"issue":"05","key":"ref30","doi-asserted-by":"crossref","first-page":"799","DOI":"10.1097\/CCM.0000000000005429","article-title":"Patient heterogeneity and the J-curve relationship between time-to-antibiotics and the outcomes of patients admitted with bacterial infection","volume":"50","author":"M G Usher","year":"2022","journal-title":"Crit Care Med"},{"issue":"02","key":"ref31","doi-asserted-by":"crossref","first-page":"199","DOI":"10.11613\/BM.2014.022","article-title":"Observational and interventional study design types; an overview","volume":"24","author":"M S Thiese","year":"2014","journal-title":"Biochem Med"},{"issue":"12","key":"ref32","doi-asserted-by":"crossref","first-page":"1302","DOI":"10.1177\/08850666251353423","article-title":"Seasonal patterns of sepsis incidence and mortality in the United States: a nationwide analysis","volume":"40","author":"R Sato","year":"2025","journal-title":"J Intensive Care Med"},{"issue":"03","key":"ref33","doi-asserted-by":"crossref","first-page":"375","DOI":"10.1016\/j.jtho.2020.12.008","article-title":"Statistical considerations for subgroup analyses","volume":"16","author":"X Wang","year":"2021","journal-title":"J Thorac Oncol"},{"key":"ref34","volume-title":"Chapter 9.2. Evaluation","author":"G J Simon","year":"2024"},{"key":"ref35","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1186\/s41512-019-0064-7","article-title":"A simple, step-by-step guide to interpreting decision curve analysis","volume":"3","author":"A J Vickers","year":"2019","journal-title":"Diagn Progn Res"},{"key":"ref36","doi-asserted-by":"publisher","DOI":"10.64898\/2026.05.22.26353890","article-title":"Real-world impact of a sepsis early detection model integrated into clinical workflow: a quasi-experimental study","author":"Y Zhang","year":"2026","journal-title":"medRxiv"}],"container-title":["Applied Clinical Informatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.thieme-connect.de\/products\/ejournals\/pdf\/10.1055\/a-2962-8091.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T23:51:13Z","timestamp":1791417073000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.thieme-connect.de\/DOI\/DOI?10.1055\/a-2962-8091"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8]]},"references-count":35,"journal-issue":{"issue":"04","published-online":{"date-parts":[[2026,8,14]]},"published-print":{"date-parts":[[2026,8]]}},"URL":"https:\/\/doi.org\/10.1055\/a-2962-8091","archive":["Portico","CLOCKSS"],"relation":{},"ISSN":["1869-0327"],"issn-type":[{"value":"1869-0327","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,9,25]]},"published":{"date-parts":[[2026,8]]}}}