{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,27]],"date-time":"2026-08-27T08:22:20Z","timestamp":1787818940140,"version":"build-2784847793"},"reference-count":36,"publisher":"American Medical Association (AMA)","issue":"11","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JAMA Netw Open"],"abstract":"Importance<\/jats:title>The results of prediction models that stratify patients with sepsis and risk of resistant gram-negative bacilli (GNB) infections inform treatment guidelines. However, these models do not extrapolate well across hospitals.<\/jats:p><\/jats:sec>Objective<\/jats:title>To assess whether patient case mix and local prevalence rates of resistance contributed to the variable performance of a general risk stratification GNB sepsis model for community-onset and hospital-onset sepsis across hospitals.<\/jats:p><\/jats:sec>Design, Setting, and Participants<\/jats:title>This was a retrospective cohort study conducted from January 2016 and October 2021. Adult patients with sepsis at 10 acute-care hospitals in rural and urban areas across Missouri and Illinois were included. Inclusion criteria were blood cultures indicating sepsis, having received 4 days of antibiotic treatment, and having organ dysfunction (vasopressor use, mechanical ventilation, increased creatinine or bilirubin levels, and thrombocytopenia). Analyses were completed in April 2024.<\/jats:p><\/jats:sec>Exposure<\/jats:title>The model included demographic characteristics, comorbidities, vital signs, laboratory values, procedures, and medications administered.<\/jats:p><\/jats:sec>Main Outcomes and Measures<\/jats:title>Culture results were stratified for ceftriaxone-susceptible GNB (SS), ceftriaxone-resistant but cefepime-susceptible GNB (RS), and ceftriaxone- and cefepime-resistant GNB (RR). Negative cultures and other pathogens were labeled SS. Deep learning models were developed separately for community-onset (patient presented with sepsis) and hospital-onset (sepsis developed \u226548 hours after admission) sepsis. The models were tested across hospitals and patient subgroups. Models were assessed using area under the receiver operating characteristic curve (AUROC) and area under precision recall curve (AUPRC).<\/jats:p><\/jats:sec>Results<\/jats:title>A total of 39\u202f893 patients with 85\u202f238 sepsis episodes (43\u202f207 [50.7%] community onset; 42\u202f031 [48.3%] hospital onset) were included. Median (IQR) age was 65 (54-74) years, 21\u202f241 patients (53.2%) were male, and 18\u202f830 (47.2%) had a previous episode of sepsis. RS contributed to 3.9% (1667 episodes) and 5.7% (2389 episodes) of community-onset and hospital-onset sepsis episodes, respectively, and RR contributed to 1.8% (796 episodes) and 3.9% (1626 episodes), respectively. Previous infections and exposure to antibiotics were associated with the risk of resistant GNB. For example, in community-onset sepsis, 375 RR episodes (47.1%), 420 RS episodes (25.2%) and 3483 of 40\u202f744 (8.5%) SS episodes were among patients with resistance to antimicrobial drugs (P\u2009&amp;lt;\u2009.001). The AUROC and AUPRC results varied across hospitals and patient subgroups for both community-onset and hospital-onset sepsis. AUPRC values correlated with the prevalence rates of resistant GNB (R<\/jats:italic>\u2009=\u20090.79; P<\/jats:italic>\u2009=\u2009.001).<\/jats:p><\/jats:sec>Conclusions and Relevance<\/jats:title>In this cohort study of 39\u202f893 patients with sepsis, variable model performance was associated with prevalence rates of antimicrobial resistance rather than patient case mix. This variability suggests caution is needed when using generalized models for predicting resistant GNB etiologies in sepsis.<\/jats:p><\/jats:sec>","DOI":"10.1001\/jamanetworkopen.2024.43658","type":"journal-article","created":{"date-parts":[[2024,11,7]],"date-time":"2024-11-07T16:32:00Z","timestamp":1730997120000},"page":"e2443658","source":"Crossref","is-referenced-by-count":5,"title":["Performance of Risk Models for Antimicrobial Resistance in Adult Patients With Sepsis"],"prefix":"10.1001","volume":"7","author":[{"given":"M. Cristina","family":"Vazquez Guillamet","sequence":"first","affiliation":[{"name":"Division of Infectious Diseases, Department of Medicine, Washington University School of Medicine, St Louis, Missouri"},{"name":"Division of Pulmonary and Critical Care Medicine, Department of Medicine, Washington University School of Medicine, St Louis, Missouri"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hanyang","family":"Liu","sequence":"additional","affiliation":[{"name":"McKelvey School of Engineering, Department of Computer Science & Engineering, Washington University, St Louis, Missouri"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrew","family":"Atkinson","sequence":"additional","affiliation":[{"name":"Division of Infectious Diseases, Department of Medicine, Washington University School of Medicine, St Louis, Missouri"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Victoria J.","family":"Fraser","sequence":"additional","affiliation":[{"name":"Department of Medicine, Washington University School of Medicine, St Louis, Missouri"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chenyang","family":"Lu","sequence":"additional","affiliation":[{"name":"McKelvey School of Engineering, Department of Computer Science & Engineering, Washington University, St Louis, Missouri"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marin H.","family":"Kollef","sequence":"additional","affiliation":[{"name":"Division of Pulmonary and Critical Care Medicine, Department of Medicine, Washington University School of Medicine, St Louis, Missouri"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"10","published-online":{"date-parts":[[2024,11,7]]},"reference":[{"issue":"11","key":"zoi241246r1","doi-asserted-by":"publisher","first-page":"e1063","DOI":"10.1097\/CCM.0000000000005337","article-title":"Surviving Sepsis Campaign: international guidelines for management of sepsis and septic shock 2021.","volume":"49","author":"Evans","year":"2021","journal-title":"Crit Care Med"},{"issue":"4","key":"zoi241246r2","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2020.2899","article-title":"Prevalence of antibiotic-resistant pathogens in culture-proven sepsis and outcomes associated with inadequate and broad-spectrum empiric antibiotic use.","volume":"3","author":"Rhee","year":"2020","journal-title":"JAMA Netw Open"},{"issue":"23","key":"zoi241246r3","doi-asserted-by":"publisher","first-page":"2235","DOI":"10.1056\/NEJMoa1703058","article-title":"Time to treatment and mortality during mandated emergency care for sepsis.","volume":"376","author":"Seymour","year":"2017","journal-title":"N Engl J Med"},{"issue":"2","key":"zoi241246r4","doi-asserted-by":"publisher","first-page":"241","DOI":"10.1016\/S1473-3099(20)30477-1","article-title":"Inappropriate empirical antibiotic therapy for bloodstream infections based on discordant in-vitro susceptibilities: a retrospective cohort analysis of prevalence, predictors, and mortality risk in US hospitals.","volume":"21","author":"Kadri","year":"2021","journal-title":"Lancet Infect Dis"},{"issue":"1","key":"zoi241246r5","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2022.49353","article-title":"Association of appropriate empirical antimicrobial therapy with in-hospital mortality in patients with bloodstream infections in the US.","volume":"6","author":"Ohnuma","year":"2023","journal-title":"JAMA Netw Open"},{"issue":"1","key":"zoi241246r6","doi-asserted-by":"publisher","first-page":"108.e1","DOI":"10.1016\/j.cmi.2018.03.029","article-title":"Decision-support models for empiric antibiotic selection in Gram-negative bloodstream infections.","volume":"25","author":"MacFadden","year":"2019","journal-title":"Clin Microbiol Infect"},{"issue":"8","key":"zoi241246r7","doi-asserted-by":"publisher","first-page":"573","DOI":"10.7326\/0003-4819-147-8-200710160-00010","article-title":"The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies.","volume":"147","author":"von Elm","year":"2007","journal-title":"Ann Intern Med"},{"issue":"10","key":"zoi241246r8","doi-asserted-by":"publisher","DOI":"10.1161\/CIRCOUTCOMES.120.006556","article-title":"Recommendations for reporting machine learning analyses in clinical research.","volume":"13","author":"Stevens","year":"2020","journal-title":"Circ Cardiovasc Qual Outcomes"},{"issue":"3","key":"zoi241246r10","doi-asserted-by":"publisher","first-page":"307","DOI":"10.1097\/CCM.0000000000003521","article-title":"Sepsis surveillance using Adult Sepsis Events Simplified eSOFA criteria versus Sepsis-3 Sequential Organ Failure Assessment criteria.","volume":"47","author":"Rhee","year":"2019","journal-title":"Crit Care Med"},{"issue":"3","key":"zoi241246r13","doi-asserted-by":"publisher","first-page":"647","DOI":"10.1007\/s10115-013-0679-x","article-title":"Explaining prediction models and individual predictions with feature contributions.","volume":"41","author":"\u0160trumbelj","year":"2014","journal-title":"Knowl Inf Syst"},{"key":"zoi241246r17","doi-asserted-by":"publisher","first-page":"321","DOI":"10.1613\/jair.953","article-title":"SMOTE: synthetic minority over-sampling technique.","volume":"16","author":"Chawla","year":"2002","journal-title":"J Artif Intell Res"},{"issue":"1","key":"zoi241246r19","doi-asserted-by":"publisher","first-page":"718","DOI":"10.1186\/s12879-019-4363-y","article-title":"Development of a bedside tool to predict the probability of drug-resistant pathogens among hospitalized adult patients with gram-negative infections.","volume":"19","author":"Lodise","year":"2019","journal-title":"BMC Infect Dis"},{"key":"zoi241246r20","doi-asserted-by":"publisher","first-page":"615","DOI":"10.1186\/s12879-014-0615-z","article-title":"A prediction tool for nosocomial multi-drug resistant Gram-negative Bacilli infections in critically ill patients\u2014prospective observational study.","volume":"14","author":"Vasudevan","year":"2014","journal-title":"BMC Infect Dis"},{"issue":"20","key":"zoi241246r21","doi-asserted-by":"publisher","first-page":"2205","DOI":"10.1001\/archinte.168.20.2205","article-title":"Prediction of infection due to antibiotic-resistant bacteria by select risk factors for health care-associated pneumonia.","volume":"168","author":"Shorr","year":"2008","journal-title":"Arch Intern Med"},{"issue":"2","key":"zoi241246r22","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1093\/cid\/cir813","article-title":"Validation of a clinical score for assessing the risk of resistant pathogens in patients with pneumonia presenting to the emergency department.","volume":"54","author":"Shorr","year":"2012","journal-title":"Clin Infect Dis"},{"issue":"11","key":"zoi241246r23","doi-asserted-by":"publisher","first-page":"1534","DOI":"10.1093\/cid\/ciad450","article-title":"Risk of misleading conclusions in observational studies of time-to-antibiotics and mortality in suspected sepsis.","volume":"77","author":"Pak","year":"2023","journal-title":"Clin Infect Dis"},{"issue":"10","key":"zoi241246r24","doi-asserted-by":"publisher","first-page":"1607","DOI":"10.1093\/cid\/cix612","article-title":"Predicting resistance to piperacillin-tazobactam, cefepime and meropenem in septic patients with bloodstream infection due to Gram-negative bacteria.","volume":"65","author":"Vazquez-Guillamet","year":"2017","journal-title":"Clin Infect Dis"},{"issue":"7","key":"zoi241246r25","doi-asserted-by":"publisher","first-page":"896","DOI":"10.1093\/cid\/ciw425","article-title":"A clinical decision tree to predict whether a bacteremic patient is infected with an extended-spectrum \u00df-lactamase-producing organism.","volume":"63","author":"Goodman","year":"2016","journal-title":"Clin Infect Dis"},{"issue":"1","key":"zoi241246r26","doi-asserted-by":"publisher","DOI":"10.1017\/ash.2022.363","article-title":"Analysis of infections among patients with historical culture positive for extended-spectrum beta-lactamase (ESBL)-producing Escherichia coli or Klebsiella pneumoniae: is ESBL-targeted therapy always needed?","volume":"3","author":"Stone","year":"2023","journal-title":"Antimicrob Steward Healthc Epidemiol"},{"issue":"8","key":"zoi241246r27","doi-asserted-by":"publisher","first-page":"1046","DOI":"10.1016\/j.cmi.2019.11.025","article-title":"Infections caused by extended-spectrum \u00df-lactamase-producing Enterobacterales after rectal colonization with ESBL-producing Escherichia coli or Klebsiella pneumoniae.","volume":"26","author":"Denkel","year":"2020","journal-title":"Clin Microbiol Infect"},{"issue":"11","key":"zoi241246r28","doi-asserted-by":"publisher","first-page":"1392","DOI":"10.1513\/AnnalsATS.201902-162OC","article-title":"Potential impact of hospital-acquired pneumonia guidelines on empiric antibiotics: an evaluation of 113 Veterans Affairs medical centers.","volume":"16","author":"Bostwick","year":"2019","journal-title":"Ann Am Thorac Soc"},{"issue":"1","key":"zoi241246r29","doi-asserted-by":"publisher","DOI":"10.1017\/ash.2023.143","article-title":"The CDC antimicrobial use measure is not ready for public reporting or value-based programs.","volume":"3","author":"Shively","year":"2023","journal-title":"Antimicrob Steward Healthc Epidemiol"},{"issue":"2","key":"zoi241246r30","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1093\/cid\/ciy075","article-title":"The Standardized Antimicrobial Administration Ratio: a new metric for measuring and comparing antibiotic use.","volume":"67","author":"van Santen","year":"2018","journal-title":"Clin Infect Dis"},{"issue":"11","key":"zoi241246r31","doi-asserted-by":"publisher","first-page":"e4484","DOI":"10.1093\/cid\/ciaa1127","article-title":"Electronically available patient claims data improve models for comparing antibiotic use across hospitals: results from 576 US facilities.","volume":"73","author":"Goodman","year":"2021","journal-title":"Clin Infect Dis"},{"issue":"11","key":"zoi241246r32","doi-asserted-by":"publisher","first-page":"1677","DOI":"10.1093\/cid\/ciy354","article-title":"Benchmarking inpatient antimicrobial use: a comparison of risk-adjusted observed-to-expected ratios.","volume":"67","author":"Yu","year":"2018","journal-title":"Clin Infect Dis"},{"issue":"8","key":"zoi241246r34","doi-asserted-by":"publisher","first-page":"1065","DOI":"10.1001\/jamainternmed.2021.2626","article-title":"External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients.","volume":"181","author":"Wong","year":"2021","journal-title":"JAMA Intern Med"},{"key":"zoi241246r14","author":"Chen","year":"2016"},{"key":"zoi241246r16","author":"Grover","year":"2016"},{"key":"zoi241246r9","unstructured":"Pineau? J, Vincent-Lamarre? P, Sinha? K, . Improving reproducibility in machine learning research (a report from the NeurIPS 2019 Reproducibility Program).? arXiv. Preprint posted online December 30, 2020. doi:10.48550\/arXiv.2003.12206"},{"key":"zoi241246r11","unstructured":"Clinical and Laboratory Standards Institute. Performance standards for antimicrobial susceptibility testing, 34th Edition. Accessed May 23, 2024. https:\/\/clsi.org\/standards\/products\/microbiology\/documents\/m100\/"},{"key":"zoi241246r12","unstructured":"National Bureau of Economic Research. ICD-9-CM to and from ICD-10-CM and ICD-10-PCS crosswalk or general equivalence mappings. Accessed May 23, 2024. https:\/\/www.nber.org\/research\/data\/icd-9-cm-and-icd-10-cm-and-icd-10-pcs-crosswalk-or-general-equivalence-mappings"},{"key":"zoi241246r15","unstructured":"Vaswani? A, Shazeer? N, Parmar? N, . Attention is all you need.? arXiv. Preprint posted online August 1, 2023. doi:10.48550\/arXiv.1706.03762"},{"key":"zoi241246r18","doi-asserted-by":"crossref","unstructured":"Lin? TY, Goyal? P, Girshick? R, He? K, Doll\u00e1r? P. Focal loss for dense object detection.? arXiv. Preprint posted online February 7, 2018. doi:10.48550\/arXiv.1708.02002","DOI":"10.1109\/ICCV.2017.324"},{"key":"zoi241246r33","unstructured":"Bennett? T, Russell? S, King? J, . Accuracy of the Epic sepsis prediction model in a regional health system.? arXiv. Preprint posted online February 19, 2019. doi:10.48550\/arXiv.1902.07276"},{"key":"zoi241246r35","unstructured":"Wu Z, Yao H, Liebovitz DM, Sun J. An iterative self-learning framework for medical domain generalization. SlidesLive. Accessed September 5, 2024. https:\/\/slideslive.com\/embed\/presentation\/39009312?js_embed_version=3&embed_init_token=eyJhbGciOiJIUzI1NiJ9.eyJpYXQiOjE3MjU1NzAwNDgsImV4cCI6MTcyNTY5OTY0OCwidSI6eyJ1dWlkIjoiMDU2NjM3MGYtYzg3Mi00ZmQyLTg1MGItNWRkYTBjOThjODI1IiwiaSI6bnVsbCwiZSI6bnVsbCwibSI6ZmFsc2V9LCJkIjoibmlwcy5jYyJ9.NlSb7EBcx-nMVS0GDt2oT3xEnFmg0p1RYXMoeAkNhuc&embed_parent_url=https%3A%2F%2Fnips.cc%2Fvirtual%2F2023%2Fposter%2F71775&embed_origin=https%3A%2F%2Fnips.cc&embed_container_id=presentation-embed-39009312&auto_load=true&auto_play=false&zoom_ratio=&disable_fullscreen=false&locale=en&vertical_enabled=true&vertical_enabled_on_mobile=false&allow_hidden_controls_when_paused=true&fit_to_viewport=true&custom_user_id=&user_uuid=0566370f-c872-4fd2-850b-5dda0c98c825"},{"key":"zoi241246r36","unstructured":"Kim? JS, Chen? J, Talwalkar? A. FACT: a diagnostic for group fairness trade-offs.? arXiv. Preprint posted online April 7, 2020. doi:10.48550\/arXiv.2004.03424"}],"container-title":["JAMA Network Open"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/jamanetwork.com\/journals\/jamanetworkopen\/articlepdf\/2825990\/vazquez_guillamet_2024_oi_241246_1730394547.30392.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,7]],"date-time":"2024-11-07T16:32:07Z","timestamp":1730997127000},"score":1,"resource":{"primary":{"URL":"https:\/\/jamanetwork.com\/journals\/jamanetworkopen\/fullarticle\/2825990"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,7]]},"references-count":36,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2024,11,4]]}},"URL":"https:\/\/doi.org\/10.1001\/jamanetworkopen.2024.43658","relation":{},"ISSN":["2574-3805"],"issn-type":[{"value":"2574-3805","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,7]]}}}