{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,8]],"date-time":"2026-07-08T00:06:04Z","timestamp":1783469164705,"version":"3.55.0"},"reference-count":46,"publisher":"Oxford University Press (OUP)","issue":"11","license":[{"start":{"date-parts":[[2021,8,30]],"date-time":"2021-08-30T00:00:00Z","timestamp":1630281600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/100000060","name":"National Institute of Allergy and Infectious Diseases","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000060","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["U19AI09871"],"award-info":[{"award-number":["U19AI09871"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R21AI120599"],"award-info":[{"award-number":["R21AI120599"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["U01AI124255"],"award-info":[{"award-number":["U01AI124255"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000133","name":"Agency for Healthcare Research and Quality","doi-asserted-by":"publisher","award":["R01HS027431"],"award-info":[{"award-number":["R01HS027431"]}],"id":[{"id":"10.13039\/100000133","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,6,10]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Background<\/jats:title>\n                  <jats:p>Many models have been developed to predict severe outcomes from Clostridioides difficile infection (CDI). These models are usually developed at a single institution and largely are not externally validated. Our aim in this study was to validate previously published risk scores in a multicenter cohort of patients with CDI.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Methods<\/jats:title>\n                  <jats:p>This was a retrospective study on 4 inpatient cohorts with CDI from 3 distinct sites: the universities of Michigan (2010\u20132012 and 2016), Chicago (2012), and Wisconsin (2012). The primary composite outcome was admission to an intensive care unit, colectomy, and\/or death attributed to CDI within 30 days of positive testing. Both within each cohort and combined across all cohorts, published CDI severity scores were assessed and compared to each other and the Infectious Diseases Society of America (IDSA) guideline definitions of severe and fulminant CDI.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>A total of 3646 patients were included for analysis. Including the 2 IDSA guideline definitions, 14 scores were assessed. Performance of scores varied within each cohort and in the combined set (mean area under the receiver operator characteristic curve [AuROC], 0.61; range, 0.53\u20130.66). Only half of the scores had performance at or better than IDSA severe and fulminant definitions (AuROCs of 0.64 and 0.63, respectively). Most of the scoring systems had more false than true positives in the combined set (mean, 81.5%; range, 0%\u201391.5%).<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusions<\/jats:title>\n                  <jats:p>No published CDI severity score showed stable, good predictive ability for adverse outcomes across multiple cohorts\/institutions or in a combined multicenter cohort.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/cid\/ciab737","type":"journal-article","created":{"date-parts":[[2021,8,30]],"date-time":"2021-08-30T12:31:06Z","timestamp":1630326666000},"page":"2028-2035","source":"Crossref","is-referenced-by-count":15,"title":["External Validation and Comparison of <i>Clostridioides difficile<\/i> Severity Scoring Systems"],"prefix":"10.1093","volume":"74","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1534-153X","authenticated-orcid":false,"given":"D Alexander","family":"Perry","sequence":"first","affiliation":[{"name":"Department of Internal Medicine, Division of Infectious Diseases, University of Michigan , Ann Arbor, Michigan , USA"},{"name":"University of Michigan Medical School , Ann Arbor, Michigan , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Shirley","sequence":"additional","affiliation":[{"name":"Division of Infectious Disease, University of Wisconsin School of Medicine and Public Health , Madison, Wisconsin , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dejan","family":"Micic","sequence":"additional","affiliation":[{"name":"Department of Internal Medicine, Section of Gastroenterology, Hepatology and Nutrition, University of Chicago Medicine , Chicago, Illinois, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pratish C","family":"Patel","sequence":"additional","affiliation":[{"name":"Division of Infectious Diseases, Vanderbilt University Medical Center , Nashville, Tennessee , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rosemary","family":"Putler","sequence":"additional","affiliation":[{"name":"Department of Internal Medicine, Division of Infectious Diseases, University of Michigan , Ann Arbor, Michigan , USA"},{"name":"University of Michigan Medical School , Ann Arbor, Michigan , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anitha","family":"Menon","sequence":"additional","affiliation":[{"name":"University of Michigan Medical School , Ann Arbor, Michigan , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Vincent B","family":"Young","sequence":"additional","affiliation":[{"name":"Department of Internal Medicine, Division of Infectious Diseases, University of Michigan , Ann Arbor, Michigan , USA"},{"name":"University of Michigan Medical School , Ann Arbor, Michigan , USA"},{"name":"Department of Microbiology and Immunology, University of Michigan , Ann Arbor, Michigan , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Krishna","family":"Rao","sequence":"additional","affiliation":[{"name":"Department of Internal Medicine, Division of Infectious Diseases, University of Michigan , Ann Arbor, Michigan , USA"},{"name":"University of Michigan Medical School , Ann Arbor, Michigan , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2021,8,30]]},"reference":[{"key":"2022061019014405800_CIT0001","doi-asserted-by":"crossref","first-page":"2369","DOI":"10.1056\/NEJMoa1408913","article-title":"Burden of Clostridium difficile infection in the United States","volume":"372","author":"Lessa","year":"2015","journal-title":"N Engl J Med"},{"key":"2022061019014405800_CIT0002","doi-asserted-by":"crossref","first-page":"1539","DOI":"10.1056\/NEJMra1403772","article-title":"Clostridium difficile infection","volume":"372","author":"Leffler","year":"2015","journal-title":"N Engl J Med"},{"key":"2022061019014405800_CIT0003","doi-asserted-by":"crossref","first-page":"570","DOI":"10.1097\/EDE.0000000000000119","article-title":"Hospital-acquired Clostridium difficile infections: estimating all-cause mortality and length of stay","volume":"25","author":"Lofgren","year":"2014","journal-title":"Epidemiology"},{"key":"2022061019014405800_CIT0004","doi-asserted-by":"crossref","first-page":"1913","DOI":"10.1053\/j.gastro.2009.02.073","article-title":"Clostridium difficile infection caused by the epidemic BI\/NAP1\/027 strain","volume":"136","author":"O\u2019Connor","year":"2009","journal-title":"Gastroenterology"},{"key":"2022061019014405800_CIT0005","doi-asserted-by":"crossref","first-page":"371","DOI":"10.2307\/30145589","article-title":"Ten years of prospective Clostridium difficile-associated disease surveillance and treatment at the Minneapolis VA Medical Center, 1982-1991","volume":"15","author":"Olson","year":"1994","journal-title":"Infect Control Hosp Epidemiol"},{"key":"2022061019014405800_CIT0006","doi-asserted-by":"crossref","first-page":"495","DOI":"10.1016\/j.jinf.2007.09.015","article-title":"Outcome of metronidazole therapy for Clostridium difficile disease and correlation with a scoring system","volume":"55","author":"Belmares","year":"2007","journal-title":"J Infect"},{"key":"2022061019014405800_CIT0007","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1111\/j.1463-1318.2009.01832.x","article-title":"Mortality and risk stratification in patients with Clostridium difficile-associated diarrhoea","volume":"12","author":"Bhangu","year":"2010","journal-title":"Colorectal Dis"},{"key":"2022061019014405800_CIT0008","doi-asserted-by":"crossref","first-page":"316","DOI":"10.1186\/1471-2334-13-316","article-title":"Derivation and validation of a simple, accurate and robust prediction rule for risk of mortality in patients with Clostridium difficile infection","volume":"13","author":"Butt","year":"2013","journal-title":"BMC Infect Dis"},{"key":"2022061019014405800_CIT0009","doi-asserted-by":"crossref","first-page":"700","DOI":"10.1097\/MEG.0b013e32835ed64d","article-title":"Mortality in Clostridium difficile infection: a prospective analysis of risk predictors","volume":"25","author":"Bloomfield","year":"2013","journal-title":"Eur J Gastroenterol Hepatol"},{"key":"2022061019014405800_CIT0010","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1016\/j.jhin.2008.09.020","article-title":"RUWA scoring system: a novel predictive tool for the identification of patients at high risk for complications from Clostridium difficile infection","volume":"71","author":"Drew","year":"2009","journal-title":"J Hosp Infect"},{"key":"2022061019014405800_CIT0011","doi-asserted-by":"crossref","first-page":"635","DOI":"10.1111\/j.1365-2036.2008.03914.x","article-title":"Predictors of serious complications due to Clostridium difficile infection","volume":"29","author":"Gujja","year":"2009","journal-title":"Aliment Pharmacol Ther"},{"key":"2022061019014405800_CIT0012","doi-asserted-by":"crossref","first-page":"0301","DOI":"10.1111\/1469-0691.12391","article-title":"Predicting a complicated course of Clostridium difficile infection at the bedside","volume":"20","author":"Hensgens","year":"2014","journal-title":"Clin Microbiol Infect"},{"key":"2022061019014405800_CIT0013","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1016\/j.jhin.2013.04.017","article-title":"Assessment of treatment patterns and patient outcomes before vs after implementation of a severity-based Clostridium difficile infection treatment policy","volume":"85","author":"Jardin","year":"2013","journal-title":"J Hosp Infect"},{"key":"2022061019014405800_CIT0014","doi-asserted-by":"crossref","first-page":"1127","DOI":"10.1001\/jamasurg.2018.3174","article-title":"Development and validation of a prediction model for mortality and adverse outcomes among patients with peripheral eosinopenia on admission for Clostridium difficile infection","volume":"153","author":"Kulaylat","year":"2018","journal-title":"JAMA Surg"},{"key":"2022061019014405800_CIT0015","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1016\/j.jhin.2011.04.017","article-title":"CSI: a severity index for Clostridium difficile infection at the time of admission","volume":"79","author":"Lungulescu","year":"2011","journal-title":"J Hosp Infect"},{"key":"2022061019014405800_CIT0016","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1186\/1471-2334-13-148","article-title":"Derivation and validation of a simple clinical bedside score (ATLAS) for Clostridium difficile infection which predicts response to therapy","volume":"13","author":"Miller","year":"2013","journal-title":"BMC Infect Dis"},{"key":"2022061019014405800_CIT0017","doi-asserted-by":"crossref","first-page":"302","DOI":"10.1086\/519265","article-title":"A comparison of vancomycin and metronidazole for the treatment of Clostridium difficile-associated diarrhea, stratified by disease severity","volume":"45","author":"Zar","year":"2007","journal-title":"Clin Infect Dis"},{"key":"2022061019014405800_CIT0018","doi-asserted-by":"crossref","first-page":"e0123405","DOI":"10.1371\/journal.pone.0123405","article-title":"A multi-center prospective derivation and validation of a clinical prediction tool for severe Clostridium difficile infection","volume":"10","author":"Na","year":"2015","journal-title":"PLoS One"},{"key":"2022061019014405800_CIT0019","doi-asserted-by":"crossref","first-page":"336","DOI":"10.1097\/IPC.0b013e31821895a8","article-title":"An innovative severity score index for Clostridium difficile infection: a prospective study","volume":"19","author":"Toro","year":"2011","journal-title":"Infect Dis Clin Pract"},{"key":"2022061019014405800_CIT0020","doi-asserted-by":"crossref","first-page":"376","DOI":"10.1097\/IPC.0b013e318182557f","article-title":"A severity score index for Clostridium difficile infection","volume":"16","author":"Velazquez-Gomez","year":"2008","journal-title":"Infect Dis Clin Pract"},{"key":"2022061019014405800_CIT0021","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1016\/j.jhin.2011.08.015","article-title":"Co-morbidities as predictors of mortality in Clostridium difficile infection and derivation of the ARC predictive score","volume":"79","author":"Welfare","year":"2011","journal-title":"J Hosp Infect"},{"key":"2022061019014405800_CIT0022","doi-asserted-by":"crossref","first-page":"2583","DOI":"10.1097\/CCM.0b013e3181ab8388","article-title":"Clostridium difficile-associated disease and mortality among the elderly critically ill","volume":"37","author":"Zilberberg","year":"2009","journal-title":"Crit Care Med"},{"key":"2022061019014405800_CIT0023","doi-asserted-by":"crossref","first-page":"725","DOI":"10.1111\/apt.13546","article-title":"Clostridium difficile associated risk of death score (CARDS): a novel severity score to predict mortality among hospitalised patients with C. difficile infection","volume":"43","author":"Kassam","year":"2016","journal-title":"Aliment Pharmacol Ther"},{"key":"2022061019014405800_CIT0024","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1086\/427113","article-title":"A hospital outbreak of Clostridium difficile disease associated with isolates carrying binary toxin genes","volume":"40","author":"McEllistrem","year":"2005","journal-title":"Clin Infect Dis"},{"key":"2022061019014405800_CIT0025","doi-asserted-by":"crossref","first-page":"e1","DOI":"10.1093\/cid\/cix1085","article-title":"Clinical practice guidelines for Clostridium difficile infection in adults and children: 2017 update by the Infectious Diseases Society of America (IDSA) and Society for Healthcare Epidemiology of America (SHEA)","volume":"66","author":"McDonald","year":"2018","journal-title":"Clin Infect Dis"},{"key":"2022061019014405800_CIT0026","doi-asserted-by":"crossref","first-page":"897","DOI":"10.1586\/14787210.6.6.897","article-title":"Measuring the severity of Clostridium difficile infection: implications for management and drug development","volume":"6","author":"Belmares","year":"2008","journal-title":"Expert Rev Anti Infect Ther"},{"key":"2022061019014405800_CIT0027","doi-asserted-by":"crossref","first-page":"897","DOI":"10.1017\/ice.2017.89","article-title":"External validation of three prediction tools for patients at risk of a complicated course of Clostridium difficile infection: disappointing in an outbreak setting","volume":"38","author":"van Beurden","year":"2017","journal-title":"Infect Control Hosp Epidemiol"},{"key":"2022061019014405800_CIT0028","doi-asserted-by":"crossref","first-page":"220","DOI":"10.1086\/658336","article-title":"Comparison of clinical severity score indices for Clostridium difficile infection","volume":"32","author":"Fujitani","year":"2011","journal-title":"Infect Control Hosp Epidemiol"},{"key":"2022061019014405800_CIT0029","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1016\/j.jhin.2011.04.027","article-title":"High Horn\u2019s index score predicts poor outcomes in patients with Clostridium difficile infection","volume":"79","author":"Arora","year":"2011","journal-title":"J Hosp Infect"},{"key":"2022061019014405800_CIT0030","volume-title":"dplyr: a grammar of data manipulation. R package version 0.8.3","author":"Wickham","year":"2019"},{"key":"2022061019014405800_CIT0031","volume-title":"PredictABEL: assessment of risk prediction models","author":"Kundu","year":"2014"},{"key":"2022061019014405800_CIT0032","doi-asserted-by":"crossref","first-page":"e92209","DOI":"10.1371\/journal.pone.0092209","article-title":"Area under precision-recall curves for weighted and unweighted data","volume":"9","author":"Keilwagen","year":"2014","journal-title":"PLoS One"},{"key":"2022061019014405800_CIT0033","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1186\/1471-2105-12-77","article-title":"pROC: an open-source package for R and S+ to analyze and compare ROC curves","volume":"12","author":"Robin","year":"2011","journal-title":"BMC Bioinformatics"},{"key":"2022061019014405800_CIT0034","volume-title":"rmda: risk model decision analysis","author":"Brown","year":"2018"},{"key":"2022061019014405800_CIT0035","volume-title":"stringr: simple, consistent wrappers for common string operations. R package version 1.4.0 ed","author":"Wickham","year":"2019"},{"key":"2022061019014405800_CIT0036","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1093\/bioinformatics\/btw570","article-title":"Precrec: fast and accurate precision-recall and ROC curve calculations in R","volume":"33","author":"Saito","year":"2017","journal-title":"Bioinformatics"},{"key":"2022061019014405800_CIT0037","doi-asserted-by":"crossref","DOI":"10.1002\/9781118548387","volume-title":"Applied logistic regression","author":"Hosmer","year":"2013"},{"key":"2022061019014405800_CIT0038","doi-asserted-by":"crossref","first-page":"e0226672","DOI":"10.1371\/journal.pone.0226672","article-title":"External validation of clinical prediction rules for complications and mortality following Clostridioides difficile infection","volume":"14","author":"Beauregard-Paultre","year":"2019","journal-title":"PLoS One"},{"key":"2022061019014405800_CIT0039","doi-asserted-by":"crossref","first-page":"855","DOI":"10.1016\/j.jclinepi.2015.02.010","article-title":"The precision\u2013recall curve overcame the optimism of the receiver operating characteristic curve in rare diseases","volume":"68","author":"Ozenne","year":"2015","journal-title":"J Clin Epidemiol"},{"key":"2022061019014405800_CIT0040","doi-asserted-by":"crossref","first-page":"ofz186","DOI":"10.1093\/ofid\/ofz186","article-title":"Using machine learning and the electronic health record to predict complicated Clostridium difficile infection","volume":"6","author":"Li","year":"2019","journal-title":"Open Forum Infect Dis"},{"key":"2022061019014405800_CIT0041","doi-asserted-by":"crossref","first-page":"e00037","DOI":"10.1128\/CVI.00037-17","article-title":"Cytokines are markers of the Clostridium difficile-induced inflammatory response and predict disease severity","volume":"24","author":"Yu","year":"2017","journal-title":"Clin Vaccine Immunol"},{"key":"2022061019014405800_CIT0042","first-page":"3","article-title":"Quantitative fecal lactoferrin as a biomarker for severe Clostridium difficile infection in hospitalized patients","volume":"2","author":"Archbald-Pannone","year":"2014","journal-title":"J Geriatr Palliat Care"},{"key":"2022061019014405800_CIT0043","doi-asserted-by":"crossref","first-page":"649","DOI":"10.1093\/cid\/ciy171","article-title":"Bezlotoxumab for prevention of recurrent Clostridium difficile infection in patients at increased risk for recurrence","volume":"67","author":"Gerding","year":"2018","journal-title":"Clin Infect Dis"},{"key":"2022061019014405800_CIT0044","doi-asserted-by":"crossref","first-page":"422","DOI":"10.1056\/NEJMoa0910812","article-title":"Fidaxomicin versus vancomycin for Clostridium difficile infection","volume":"364","author":"Louie","year":"2011","journal-title":"N Engl J Med"},{"key":"2022061019014405800_CIT0045","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1080\/19490976.2016.1273998","article-title":"Fecal microbiota transplant in severe and severe-complicated Clostridium difficile: a promising treatment approach","volume":"8","author":"Fischer","year":"2017","journal-title":"Gut Microbes"},{"key":"2022061019014405800_CIT0046","doi-asserted-by":"crossref","first-page":"398","DOI":"10.1097\/MCG.0000000000000374","article-title":"Long-term follow-up study of fecal microbiota transplantation for severe and\/or complicated Clostridium difficile infection: a multicenter experience","volume":"50","author":"Aroniadis","year":"2016","journal-title":"J Clin Gastroenterol"}],"container-title":["Clinical Infectious Diseases"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/cid\/advance-article-pdf\/doi\/10.1093\/cid\/ciab737\/40833905\/ciab737.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/cid\/article-pdf\/74\/11\/2028\/44018510\/ciab737.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/cid\/article-pdf\/74\/11\/2028\/44018510\/ciab737.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,6,10]],"date-time":"2022-06-10T19:04:36Z","timestamp":1654887876000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/cid\/article\/74\/11\/2028\/6359793"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,30]]},"references-count":46,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2021,8,30]]},"published-print":{"date-parts":[[2022,6,10]]}},"URL":"https:\/\/doi.org\/10.1093\/cid\/ciab737","relation":{},"ISSN":["1058-4838","1537-6591"],"issn-type":[{"value":"1058-4838","type":"print"},{"value":"1537-6591","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022,6,1]]},"published":{"date-parts":[[2021,8,30]]}}}