{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,5]],"date-time":"2026-11-05T08:41:12Z","timestamp":1793868072141,"version":"4.6.1"},"reference-count":31,"publisher":"American Medical Association (AMA)","issue":"2","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JAMA Netw Open"],"abstract":"\n Importance<\/jats:title>\n The Epic Sepsis Model version 2 (ESM v2) is a widely implemented proprietary sepsis prediction model, but no multicenter, external validation of its performance has been reported to guide adoption and use.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n To conduct a multicenter validation of the ESM v2 to compare performance against the original ESM v1, outline differences across heterogenous clinical sites, and compare model performance against clinician recognition of sepsis.<\/jats:p>\n <\/jats:sec>\n \n Design, Setting, and Participants<\/jats:title>\n This prognostic study included adult inpatient encounters at 4 large US health systems between August 31, 2023, and March 11, 2025. At each site, data were collected for a consecutive period immediately following new model implementation. Data were analyzed from July 23 to August 19, 2025.<\/jats:p>\n <\/jats:sec>\n \n Main Outcomes and Measures<\/jats:title>\n Sepsis was defined using Sepsis-3 clinical consensus criteria. Model discrimination was assessed using area under the receiver operating characteristic curve (AUROC) at the encounter level and prediction level with 4-hour, 12-hour, and hospitalization-wide time horizons. Performance against clinician recognition of sepsis was measured using antibiotics, lactate, and body culture orders.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n Of 227\u202f091 inpatient encounters, 7401 (3.3%; median [IQR] age, 65 [54-75] years; 3359 [45.4%] female; 2.7% Asian; 24.6% Black; 64.6% White; 7.1% Hispanic ethnicity) met sepsis criteria and 219\u202f690 (96.7%; median [IQR] age, 48 [33-65] years; 123\u202f563 [56.2%] female; 2.5% Asian; 38.8% Black; 49.6% White; 9.6% Hispanic ethnicity) did not. At the encounter level, the AUROC ranged from 0.82 (95% CI, 0.81-0.83) to 0.92 (95% CI, 0.92-0.93) across study sites. At the prediction level with a 12-hour time horizon, the AUROC ranged from 0.75 (95% CI, 0.74-0.75) to 0.85 (95% CI, 0.85-0.85). Comparison against clinician recognition of sepsis yielded a minor decrease in performance, with the resulting encounter-level AUROC ranging from 0.80 (95% CI, 0.79-0.81) to 0.90 (95% CI, 0.89-0.90) across sites. Positive predictive values remained low, from 0.13 (95% CI, 0.13-0.14) to 0.26 (95% CI, 0.25-0.27), with a high number needed to evaluate and high alert burden.<\/jats:p>\n <\/jats:sec>\n \n Conclusions and Relevance<\/jats:title>\n In this prognostic study of a new sepsis prediction model, a multicenter prospective validation performed across 4 major US health systems found improved discrimination for the early prediction of sepsis but noted high institutional variability, low positive predictive value, and high alert burden.<\/jats:p>\n <\/jats:sec>","DOI":"10.1001\/jamanetworkopen.2026.0181","type":"journal-article","created":{"date-parts":[[2026,2,27]],"date-time":"2026-02-27T16:30:27Z","timestamp":1772209827000},"page":"e260181","source":"Crossref","is-referenced-by-count":25,"title":["Multicenter Prospective Validation of an Updated Proprietary Sepsis Prediction Model"],"prefix":"10.1001","volume":"9","author":[{"given":"Andrew","family":"Wong","sequence":"first","affiliation":[{"name":"University of Michigan Medical School, Ann Arbor"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Danielle","family":"Currey","sequence":"additional","affiliation":[{"name":"Case Western Reserve University School of Medicine, Cleveland, Ohio"},{"name":"Center for Clinical Informatics Research and Education, MetroHealth, Cleveland, Ohio"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Megan","family":"Schwinne","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, Georgia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Brenna","family":"Park-Egan","sequence":"additional","affiliation":[{"name":"Division of Pulmonary, Allergy, and Critical Care Medicine, Oregon Health & Science University, Portland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sean","family":"Meyer","sequence":"additional","affiliation":[{"name":"University of Michigan Medical School, Ann Arbor"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrew","family":"Gutting","sequence":"additional","affiliation":[{"name":"University of Michigan Medical School, Ann Arbor"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jie","family":"Cao","sequence":"additional","affiliation":[{"name":"Joan and Irwin Jacobs Center for Health Innovation, UC San Diego Health, San Diego, California"},{"name":"Division of Biomedical Informatics, Department of Medicine, University of California, San Diego"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sharaf","family":"Khan","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, Emory University School of Medicine, Atlanta, Georgia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Raymund","family":"Dantes","sequence":"additional","affiliation":[{"name":"Department of Medicine, Emory University School of Medicine, Atlanta, Georgia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tony","family":"Pan","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, Georgia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Timothy","family":"Buchman","sequence":"additional","affiliation":[{"name":"Department of Surgery, Emory University School of Medicine, Atlanta, Georgia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Karandeep","family":"Singh","sequence":"additional","affiliation":[{"name":"Joan and Irwin Jacobs Center for Health Innovation, UC San Diego Health, San Diego, California"},{"name":"Division of Biomedical Informatics, Department of Medicine, University of California, San Diego"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sivasubramanium V.","family":"Bhavani","sequence":"additional","affiliation":[{"name":"Department of Medicine, Emory University School of Medicine, Atlanta, Georgia"},{"name":"Critical Care Center, Emory University School of Medicine, Atlanta, Georgia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Patrick G.","family":"Lyons","sequence":"additional","affiliation":[{"name":"Division of Pulmonary, Allergy, and Critical Care Medicine, Oregon Health & Science University, Portland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael W.","family":"Sjoding","sequence":"additional","affiliation":[{"name":"University of Michigan Medical School, Ann Arbor"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yasir","family":"Tarabichi","sequence":"additional","affiliation":[{"name":"Case Western Reserve University School of Medicine, Cleveland, Ohio"},{"name":"Center for Clinical Informatics Research and Education, MetroHealth, Cleveland, Ohio"},{"name":"Department of Pulmonary and Critical Care Medicine, MetroHealth, Cleveland, Ohio"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"10","published-online":{"date-parts":[[2026,2,27]]},"reference":[{"issue":"34","key":"zoi260015r1","doi-asserted-by":"publisher","first-page":"907","DOI":"10.15585\/mmwr.mm7234a2","article-title":"Sepsis program activities in acute care hospitals\u2014National Healthcare Safety Network, United States, 2022.","volume":"72","author":"Dantes","year":"2023","journal-title":"MMWR Morb Mortal Wkly Rep"},{"issue":"8","key":"zoi260015r2","doi-asserted-by":"publisher","first-page":"1552","DOI":"10.1007\/s00134-020-06151-x","article-title":"Incidence and mortality of hospital- and ICU-treated sepsis: results from an updated and expanded systematic review and meta-analysis.","volume":"46","author":"Fleischmann-Struzek","year":"2020","journal-title":"Intensive Care Med"},{"issue":"13","key":"zoi260015r3","doi-asserted-by":"publisher","first-page":"1241","DOI":"10.1001\/jama.2017.13836","article-title":"Incidence and trends of sepsis in US hospitals using clinical vs claims data, 2009-2014.","volume":"318","author":"Rhee","year":"2017","journal-title":"JAMA"},{"issue":"8","key":"zoi260015r4","doi-asserted-by":"publisher","first-page":"801","DOI":"10.1001\/jama.2016.0287","article-title":"The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3).","volume":"315","author":"Singer","year":"2016","journal-title":"JAMA"},{"issue":"23","key":"zoi260015r5","doi-asserted-by":"publisher","first-page":"2235","DOI":"10.1056\/NEJMoa1703058","article-title":"Time to treatment and mortality during mandated emergency care for sepsis.","volume":"376","author":"Seymour","year":"2017","journal-title":"N Engl J Med"},{"issue":"11","key":"zoi260015r6","doi-asserted-by":"publisher","first-page":"1181","DOI":"10.1007\/s00134-021-06506-y","article-title":"Surviving sepsis campaign: international guidelines for management of sepsis and septic shock 2021.","volume":"47","author":"Evans","year":"2021","journal-title":"Intensive Care Med"},{"issue":"10","key":"zoi260015r7","doi-asserted-by":"publisher","first-page":"1784","DOI":"10.1513\/AnnalsATS.201603-172BC","article-title":"The nature and variability of automated practice alerts derived from electronic health records in a U.S. nationwide critical care research network.","volume":"13","author":"Benthin","year":"2016","journal-title":"Ann Am Thorac Soc"},{"issue":"1","key":"zoi260015r8","doi-asserted-by":"publisher","first-page":"90","DOI":"10.1377\/hlthaff.2024.00842","article-title":"Current use and evaluation of artificial intelligence and predictive models in US hospitals.","volume":"44","author":"Nong","year":"2025","journal-title":"Health Aff (Millwood)"},{"issue":"14","key":"zoi260015r9","doi-asserted-by":"publisher","first-page":"1351","DOI":"10.1001\/jama.2019.10306","article-title":"Making machine learning models clinically useful.","volume":"322","author":"Shah","year":"2019","journal-title":"JAMA"},{"key":"zoi260015r10","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2021.106288","article-title":"The importance of being external. methodological insights for the external validation of machine learning models in medicine.","volume":"208","author":"Cabitza","year":"2021","journal-title":"Comput Methods Programs Biomed"},{"issue":"1","key":"zoi260015r11","doi-asserted-by":"publisher","first-page":"70","DOI":"10.1186\/s12916-023-02779-w","article-title":"There is no such thing as a validated prediction model.","volume":"21","author":"Van Calster","year":"2023","journal-title":"BMC Med"},{"issue":"8","key":"zoi260015r12","doi-asserted-by":"publisher","first-page":"1065","DOI":"10.1001\/jamainternmed.2021.2626","article-title":"External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients.","volume":"181","author":"Wong","year":"2021","journal-title":"JAMA Intern Med"},{"issue":"6","key":"zoi260015r13","doi-asserted-by":"publisher","first-page":"611","DOI":"10.1001\/jamainternmed.2022.7182","article-title":"Factors associated with variability in the performance of a proprietary sepsis prediction model across 9 networked hospitals in the US.","volume":"183","author":"Lyons","year":"2023","journal-title":"JAMA Intern Med"},{"issue":"4","key":"zoi260015r14","doi-asserted-by":"publisher","DOI":"10.1093\/jamiaopen\/ooae133","article-title":"External validation of the Epic sepsis predictive model in 2 county emergency departments.","volume":"7","author":"Ostermayer","year":"2024","journal-title":"JAMIA Open"},{"key":"zoi260015r15","doi-asserted-by":"publisher","DOI":"10.1056\/AIoa2300032","article-title":"Evaluation of sepsis prediction models before onset of treatment.","author":"Kamran","year":"2024","journal-title":"NEJM AI"},{"key":"zoi260015r16","doi-asserted-by":"publisher","DOI":"10.1136\/bmj-2023-078378","article-title":"TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods.","volume":"385","author":"Collins","year":"2024","journal-title":"BMJ"},{"key":"zoi260015r17","doi-asserted-by":"publisher","first-page":"147","DOI":"10.1016\/j.ajem.2025.07.056","article-title":"External validation of a widely available, localizable sepsis early detection model in the emergency department setting.","volume":"97","author":"Currey","year":"2025","journal-title":"Am J Emerg Med"},{"issue":"1","key":"zoi260015r18","doi-asserted-by":"publisher","first-page":"14","DOI":"10.1038\/s41746-023-00986-6","article-title":"Impact of a deep learning sepsis prediction model on quality of care and survival.","volume":"7","author":"Boussina","year":"2024","journal-title":"NPJ Digit Med"},{"issue":"7","key":"zoi260015r19","doi-asserted-by":"publisher","first-page":"1447","DOI":"10.1038\/s41591-022-01895-z","article-title":"Factors driving provider adoption of the TREWS machine learning-based early warning system and its effects on sepsis treatment timing.","volume":"28","author":"Henry","year":"2022","journal-title":"Nat Med"},{"issue":"1","key":"zoi260015r20","doi-asserted-by":"publisher","first-page":"350","DOI":"10.1038\/s41746-025-01664-5","article-title":"Evaluating sepsis watch generalizability through multisite external validation of a sepsis machine learning model.","volume":"8","author":"Valan","year":"2025","journal-title":"NPJ Digit Med"},{"issue":"1","key":"zoi260015r21","doi-asserted-by":"publisher","first-page":"144","DOI":"10.1186\/s13049-021-00958-3","article-title":"Performance of NEWS2, RETTS, clinical judgment and the Predict Sepsis screening tools with respect to identification of sepsis among ambulance patients with suspected infection: a prospective cohort study.","volume":"29","author":"Wallgren","year":"2021","journal-title":"Scand J Trauma Resusc Emerg Med"},{"issue":"8","key":"zoi260015r22","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2023.29729","article-title":"Sepsis prediction model for determining sepsis vs SIRS, qSOFA, and SOFA.","volume":"6","author":"Schertz","year":"2023","journal-title":"JAMA Netw Open"},{"issue":"9","key":"zoi260015r23","doi-asserted-by":"publisher","first-page":"1169","DOI":"10.1097\/CCM.0000000000003817","article-title":"Epidemiology of hospital-onset versus community-onset sepsis in U.S. hospitals and association with mortality: a retrospective analysis using electronic clinical data.","volume":"47","author":"Rhee","year":"2019","journal-title":"Crit Care Med"},{"issue":"9","key":"zoi260015r24","doi-asserted-by":"publisher","first-page":"1945","DOI":"10.1097\/CCM.0000000000001164","article-title":"Community-, healthcare-, and hospital-acquired severe sepsis hospitalizations in the University HealthSystem Consortium.","volume":"43","author":"Page","year":"2015","journal-title":"Crit Care Med"},{"issue":"1","key":"zoi260015r25","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1186\/s12967-022-03228-7","article-title":"Ground truth labels challenge the validity of sepsis consensus definitions in critical illness.","volume":"20","author":"Lindner","year":"2022","journal-title":"J Transl Med"},{"issue":"10","key":"zoi260015r26","doi-asserted-by":"publisher","DOI":"10.1097\/CCE.0000000000000053","article-title":"Comparison of automated sepsis identification methods and electronic health record-based sepsis phenotyping: improving case identification accuracy by accounting for confounding comorbid conditions.","volume":"1","author":"Henry","year":"2019","journal-title":"Crit Care Explor"},{"issue":"4","key":"zoi260015r27","doi-asserted-by":"publisher","first-page":"493","DOI":"10.1097\/CCM.0000000000003554","article-title":"Variation in identifying sepsis and organ dysfunction using administrative versus electronic clinical data and impact on hospital outcome comparisons.","volume":"47","author":"Rhee","year":"2019","journal-title":"Crit Care Med"},{"issue":"9","key":"zoi260015r28","doi-asserted-by":"publisher","first-page":"1309","DOI":"10.1513\/AnnalsATS.202302-105OC","article-title":"Comparison of administrative versus electronic health record-based methods for identifying sepsis hospitalizations.","volume":"20","author":"Karlic","year":"2023","journal-title":"Ann Am Thorac Soc"},{"issue":"5","key":"zoi260015r29","doi-asserted-by":"publisher","DOI":"10.1056\/AIp2401116","article-title":"Challenges in the postmarket surveillance of clinical prediction models.","volume":"2","author":"Ansari","year":"2025","journal-title":"NEJM AI"},{"issue":"8","key":"zoi260015r30","doi-asserted-by":"publisher","first-page":"1536","DOI":"10.1007\/s00134-020-06106-2","article-title":"Epidemiology and burden of sepsis acquired in hospitals and intensive care units: a systematic review and meta-analysis.","volume":"46","author":"Markwart","year":"2020","journal-title":"Intensive Care Med"},{"issue":"3","key":"zoi260015r31","doi-asserted-by":"publisher","first-page":"418","DOI":"10.1097\/CCM.0000000000005267","article-title":"Improving timeliness of antibiotic administration using a provider and pharmacist facing sepsis early warning system in the emergency department setting: a randomized controlled quality improvement initiative.","volume":"50","author":"Tarabichi","year":"2022","journal-title":"Crit Care Med"}],"container-title":["JAMA Network Open"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/jamanetwork.com\/journals\/jamanetworkopen\/articlepdf\/2845595\/wong_2026_oi_260015_1772554921.69484.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,3]],"date-time":"2026-03-03T17:04:38Z","timestamp":1772557478000},"score":1,"resource":{"primary":{"URL":"https:\/\/jamanetwork.com\/journals\/jamanetworkopen\/fullarticle\/2845595"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,27]]},"references-count":31,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,2,2]]}},"URL":"https:\/\/doi.org\/10.1001\/jamanetworkopen.2026.0181","relation":{},"ISSN":["2574-3805"],"issn-type":[{"value":"2574-3805","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,27]]}}}