{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,10]],"date-time":"2026-05-10T10:26:06Z","timestamp":1778408766588,"version":"3.51.4"},"reference-count":62,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,1,9]],"date-time":"2023-01-09T00:00:00Z","timestamp":1673222400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100017413","name":"Innovation Fund","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100017413","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Med."],"abstract":"<jats:sec><jats:title>Background<\/jats:title><jats:p>Methods for assessing long-term outcome quality of acute care for sepsis are lacking. We investigated a method for measuring long-term outcome quality based on health claims data in Germany.<\/jats:p><\/jats:sec><jats:sec><jats:title>Materials and methods<\/jats:title><jats:p>Analyses were based on data of the largest German health insurer, covering 32% of the population. Cases (aged 15 years and older) with ICD-10-codes for severe sepsis or septic shock according to sepsis-1-definitions hospitalized in 2014 were included. Short-term outcome was assessed by 90-day mortality; long-term outcome was assessed by a composite endpoint defined by 1-year mortality or increased dependency on chronic care. Risk factors were identified by logistic regressions with backward selection. Hierarchical generalized linear models were used to correct for clustering of cases in hospitals. Predictive validity of the models was assessed by internal validation using bootstrap-sampling. Risk-standardized mortality rates (RSMR) were calculated with and without reliability adjustment and their univariate and bivariate distributions were described.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>Among 35,552 included patients, 53.2% died within 90 days after admission; 39.8% of 90-day survivors died within the first year or had an increased dependency on chronic care. Both risk-models showed a sufficient predictive validity regarding discrimination [<jats:italic>AUC<\/jats:italic> = 0.748 (95% CI: 0.742; 0.752) for 90-day mortality; <jats:italic>AUC<\/jats:italic> = 0.675 (95% CI: 0.665; 0.685) for the 1-year composite outcome, respectively], calibration (Brier Score of 0.203 and 0.220; calibration slope of 1.094 and 0.978), and explained variance (<jats:italic>R<\/jats:italic><jats:sup>2<\/jats:sup> = 0.242 and <jats:italic>R<\/jats:italic><jats:sup>2<\/jats:sup> = 0.111). Because of a small case-volume per hospital, applying reliability adjustment to the RSMR led to a great decrease in variability across hospitals [from median (1st quartile, 3rd quartile) 54.2% (44.3%, 65.5%) to 53.2% (50.7%, 55.9%) for 90-day mortality; from 39.2% (27.8%, 51.1%) to 39.9% (39.5%, 40.4%) for the 1-year composite endpoint]. There was no substantial correlation between the two endpoints at hospital level (observed rates: \u03c1 = 0, <jats:italic>p<\/jats:italic> = 0.99; RSMR: \u03c1 = 0.017, <jats:italic>p<\/jats:italic> = 0.56; reliability-adjusted RSMR: \u03c1 = 0.067; <jats:italic>p<\/jats:italic> = 0.026).<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p>Quality assurance and epidemiological surveillance of sepsis care should include indicators of long-term mortality and morbidity. Claims-based risk-adjustment models for quality indicators of acute sepsis care showed satisfactory predictive validity. To increase reliability of measurement, data sources should cover the full population and hospitals need to improve ICD-10-coding of sepsis.<\/jats:p><\/jats:sec>","DOI":"10.3389\/fmed.2022.1069042","type":"journal-article","created":{"date-parts":[[2023,1,9]],"date-time":"2023-01-09T09:55:23Z","timestamp":1673258123000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Development and validation of risk-adjusted quality indicators for the long-term outcome of acute sepsis care in German hospitals based on health claims data"],"prefix":"10.3389","volume":"9","author":[{"given":"Lisa","family":"Wedekind","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Carolin","family":"Fleischmann-Struzek","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Norman","family":"Rose","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Melissa","family":"Spoden","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Christian","family":"G\u00fcnster","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Peter","family":"Schlattmann","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Andr\u00e9","family":"Scherag","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Konrad","family":"Reinhart","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Daniel","family":"Schwarzkopf","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2023,1,9]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"801","DOI":"10.1001\/jama.2016.0287","article-title":"The third international consensus definitions for sepsis and septic shock (sepsis-3).","volume":"315","author":"Singer","year":"2016","journal-title":"J Am Med Assoc."},{"key":"B2","doi-asserted-by":"publisher","first-page":"200","DOI":"10.1016\/S0140-6736(19)32989-7","article-title":"Global, regional, and national sepsis incidence and mortality, 1990\u20132017: analysis for the global burden of disease study.","volume":"395","author":"Rudd","year":"2020","journal-title":"Lancet."},{"key":"B3","author":"Goodwin","year":"2015","journal-title":"Just Say Sepsis! A Review of the Process of Care Received by Patients With Sepsis."},{"key":"B4","doi-asserted-by":"publisher","first-page":"62","DOI":"10.1001\/jama.2017.17687","article-title":"Enhancing recovery from sepsis a review.","volume":"319","author":"Prescott","year":"2018","journal-title":"J Am Med Assoc."},{"key":"B5","year":"2017","journal-title":"EB140.R5 Improving the Prevention, Diagnosis and Management of Sepsis."},{"key":"B6","doi-asserted-by":"publisher","first-page":"966","DOI":"10.1056\/NEJM199609263351311","article-title":"Quality of health care .2. Measuring quality of care.","volume":"335","author":"Brook","year":"1996","journal-title":"N Engl J Med."},{"key":"B7","doi-asserted-by":"publisher","first-page":"666","DOI":"10.7326\/0003-4819-127-8_Part_2-199710151-00048","article-title":"Assessing quality using administrative data.","volume":"127","author":"Iezzoni","year":"1997","journal-title":"Ann Intern Med."},{"key":"B8","first-page":"251","article-title":"Risk adjustment.","author":"Iezzoni","year":"2009","journal-title":"Performance Measurement for Health System Improvement: Experiences, Challenges and Prospects."},{"key":"B9","doi-asserted-by":"publisher","first-page":"456","DOI":"10.1161\/CIRCULATIONAHA.105.170769","article-title":"Standards for statistical models used for public reporting of health outcomes \u2013 an American Heart association scientific statement from the quality of care and outcomes research interdisciplinary writing group - cosponsored by the council on epidemiology and prevention and the stroke council - endorsed by the American college of cardiology foundation.","volume":"113","author":"Krumholz","year":"2006","journal-title":"Circulation."},{"key":"B10","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1097\/CCM.0000000000001392","article-title":"Severe sepsis mortality prediction model and score for use with administrative data.","volume":"44","author":"Ford","year":"2016","journal-title":"Crit Care Med."},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1186\/s12913-019-4037-x","article-title":"An administrative model for benchmarking hospitals on their 30-day sepsis mortality.","volume":"19","author":"Darby","year":"2019","journal-title":"BMC Health Serv Res."},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0194371","article-title":"A risk-model for hospital mortality among patients with severe sepsis or septic shock based on German national administrative claims data.","volume":"13","author":"Schwarzkopf","year":"2018","journal-title":"PLoS One."},{"key":"B13","doi-asserted-by":"publisher","DOI":"10.1177\/20552076211072400","article-title":"Risk-standardized sepsis mortality map of the United States.","volume":"8","author":"Hu","year":"2022","journal-title":"Digit Health."},{"key":"B14","doi-asserted-by":"publisher","first-page":"240","DOI":"10.1001\/jama.2019.9021","article-title":"Association between state-mandated protocolized sepsis care and in-hospital mortality among adults with sepsis.","volume":"322","author":"Kahn","year":"2019","journal-title":"JAMA."},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.1186\/s13012-017-0706-5","article-title":"The German quality network sepsis: study protocol for the evaluation of a quality collaborative on decreasing sepsis-related mortality in a quasi-experimental difference-in-differences design.","volume":"13","author":"Schwarzkopf","year":"2018","journal-title":"Implement Sci."},{"key":"B16","doi-asserted-by":"publisher","first-page":"1295","DOI":"10.1001\/jama.2014.2639","article-title":"Declining case fatality rates for severe sepsis good data bring good news with ambiguous implications.","volume":"311","author":"Iwashyna","year":"2014","journal-title":"JAMA."},{"key":"B17","doi-asserted-by":"publisher","first-page":"785","DOI":"10.1097\/00005650-200008000-00003","article-title":"Identification of in-hospital complications from claims data: is it valid?","volume":"38","author":"Lawthers","year":"2000","journal-title":"Med Care."},{"key":"B18","doi-asserted-by":"publisher","first-page":"675","DOI":"10.1097\/00005650-200208000-00007","article-title":"Validity of information on comorbidity derived rom ICD-9-CCM administrative data.","volume":"40","author":"Quan","year":"2002","journal-title":"Med Care."},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2021.34290","article-title":"Epidemiology and costs of postsepsis morbidity, nursing care dependency, and mortality in Germany, 2013 to 2017.","volume":"4","author":"Fleischmann-Struzek","year":"2021","journal-title":"JAMA Netw Open."},{"key":"B20","year":"2021","journal-title":"QSR-Verfahren \u2013 Indikatorenhandbuch - Verfahrensjahr 2021."},{"key":"B21","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1016\/j.jclinepi.2016.12.001","article-title":"Criteria for use of composite end points for competing risks-a systematic survey of the literature with recommendations.","volume":"82","author":"Manja","year":"2017","journal-title":"J Clin Epidemiol."},{"key":"B22","doi-asserted-by":"publisher","first-page":"2425","DOI":"10.1097\/CCM.0b013e31822572e3","article-title":"Development and validation of a model that uses enhanced administrative data to predict mortality in patients with sepsis.","volume":"39","author":"Lagu","year":"2011","journal-title":"Crit Care Med."},{"key":"B23","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1016\/j.jcrc.2016.07.016","article-title":"Inter-hospital transfer is associated with increased mortality and costs in severe sepsis and septic shock: an instrumental variables approach.","volume":"36","author":"Mohr","year":"2016","journal-title":"J Crit Care."},{"key":"B24","doi-asserted-by":"publisher","first-page":"1969","DOI":"10.1097\/CCM.0000000000000416","article-title":"Sepsis severity score: an internationally derived scoring system from the surviving sepsis campaign database.","volume":"42","author":"Osborn","year":"2014","journal-title":"Crit Care Med."},{"key":"B25","doi-asserted-by":"publisher","first-page":"373","DOI":"10.1016\/0021-9681(87)90171-8","article-title":"A new method of classifying prognostic co-morbidity in longitudinal-studies - development and validation.","volume":"40","author":"Charlson","year":"1987","journal-title":"J Chronic Dis."},{"key":"B26","doi-asserted-by":"publisher","first-page":"8","DOI":"10.1097\/00005650-199801000-00004","article-title":"Comorbidity measures for use with administrative data.","volume":"36","author":"Elixhauser","year":"1998","journal-title":"Med Care."},{"key":"B27","doi-asserted-by":"publisher","first-page":"1130","DOI":"10.1097\/01.mlr.0000182534.19832.83","article-title":"Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data.","volume":"43","author":"Quan","year":"2005","journal-title":"Med Care."},{"key":"B28","author":"Hagn","year":"2014","journal-title":"Vergleich Verschiedener Komorbidit\u00e4ts-Scores in Routinedaten der Station\u00e4ren Versorgung."},{"key":"B29","doi-asserted-by":"publisher","first-page":"1303","DOI":"10.1097\/00003246-200107000-00002","article-title":"Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care.","volume":"29","author":"Angus","year":"2001","journal-title":"Crit Care Med."},{"key":"B30","doi-asserted-by":"publisher","first-page":"1928","DOI":"10.1097\/01.CCM.0000277043.85378.C1","article-title":"National estimates of severe sepsis in United States emergency departments.","volume":"35","author":"Wang","year":"2007","journal-title":"Crit Care Med."},{"key":"B31","doi-asserted-by":"publisher","first-page":"1442","DOI":"10.1097\/CCM.0b013e3181de4406","article-title":"Assessment of incidence of severe sepsis in Sweden using different ways of abstracting International classification of diseases codes: difficulties with methods and interpretation of results.","volume":"38","author":"Wilhelms","year":"2010","journal-title":"Crit Care Med."},{"key":"B32","doi-asserted-by":"publisher","first-page":"803","DOI":"10.1080\/01621459.1997.10474036","article-title":"Statistical methods for profiling providers of medical care: Issues and applications.","volume":"92","author":"Normand","year":"1997","journal-title":"J Am Stat Assoc."},{"key":"B33","doi-asserted-by":"publisher","first-page":"361","DOI":"10.1002\/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4","article-title":"Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors.","volume":"15","author":"Harrell","year":"1996","journal-title":"Stat Med."},{"key":"B34","doi-asserted-by":"publisher","first-page":"1987","DOI":"10.1002\/(SICI)1097-0258(19961015)15:19<1987::AID-SIM318>3.0.CO;2-9","article-title":"Explained variation for logistic regression.","volume":"15","author":"Mittlbock","year":"1996","journal-title":"Stat Med."},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0017401","article-title":"An administrative claims model for profiling hospital 30-day mortality rates for pneumonia patients.","volume":"6","author":"Bratzler","year":"2011","journal-title":"PLoS One."},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4614-6849-3","author":"Kuhn","year":"2013","journal-title":"Applied Predictive Modeling."},{"key":"B37","doi-asserted-by":"publisher","first-page":"2645","DOI":"10.1002\/(SICI)1097-0258(19971215)16:23<2645::AID-SIM696>3.0.CO;2-D","article-title":"Comparing risk-adjustment methods for provider profiling.","volume":"16","author":"DeLong","year":"1997","journal-title":"Stat Med."},{"key":"B38","doi-asserted-by":"publisher","first-page":"2106","DOI":"10.1016\/j.athoracsur.2005.06.078","article-title":"Massachusetts cardiac surgery repory card: Implications of statistical methodology.","volume":"80","author":"Shahian","year":"2005","journal-title":"Ann Thorac Surg."},{"key":"B39","doi-asserted-by":"publisher","first-page":"1614","DOI":"10.1111\/j.1475-6773.2010.01158.x","article-title":"Ranking hospitals on surgical mortality: the importance of reliability adjustment.","volume":"45","author":"Dimick","year":"2010","journal-title":"Health Serv Res."},{"key":"B40","doi-asserted-by":"publisher","first-page":"385","DOI":"10.2307\/2983325","article-title":"League tables and their limitations: statistical issues in comparisons of institutional performance.","volume":"159","author":"Goldstein","year":"1996","journal-title":"J R Stat Soc Ser Stat Soc."},{"key":"B41","doi-asserted-by":"publisher","first-page":"757","DOI":"10.1016\/j.athoracsur.2014.11.039","article-title":"A primer on using shrinkage to compare in-hospital mortality between centers.","volume":"99","author":"MacKenzie","year":"2015","journal-title":"Ann Thorac Surg."},{"key":"B42","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1007\/s10742-006-0018-8","article-title":"Assessing surrogacy of data sources for institutional comparisons.","volume":"7","author":"Normand","year":"2007","journal-title":"Health Servic Outcomes Res Methodol."},{"key":"B43","year":"2020","journal-title":"R: A Language and Environment for Statistical Computing."},{"key":"B44","doi-asserted-by":"publisher","first-page":"674","DOI":"10.1097\/CCM.0000000000002824","article-title":"The New York sepsis severity score: development of a risk-adjusted severity model for sepsis.","volume":"46","author":"Phillips","year":"2018","journal-title":"Crit Care Med."},{"key":"B45","doi-asserted-by":"publisher","first-page":"626","DOI":"10.1097\/MLR.0b013e31819432e5","article-title":"Modification of the elixhauser comorbidity measures into a point system for hospital death using administrative data.","volume":"47","author":"van Walraven","year":"2009","journal-title":"Med Care."},{"key":"B46","doi-asserted-by":"publisher","first-page":"374","DOI":"10.1097\/MLR.0000000000000326","article-title":"A new elixhauser-based comorbidity summary measure to predict in-hospital mortality.","volume":"53","author":"Thompson","year":"2015","journal-title":"Med Care."},{"key":"B47","doi-asserted-by":"crossref","first-page":"1181","DOI":"10.1007\/s00134-021-06506-y","article-title":"Surviving sepsis campaign: international guidelines for management of sepsis and septic shock 2021.","volume":"47","author":"Evans","year":"2021","journal-title":"Intensive Care Med."},{"key":"B48","doi-asserted-by":"publisher","first-page":"619","DOI":"10.1007\/s00134-019-05908-3","article-title":"Rate and risk factors for rehospitalisation in sepsis survivors: systematic review and meta-analysis.","volume":"46","author":"Shankar-Hari","year":"2020","journal-title":"Intensive Care Med."},{"key":"B49","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2020.13580","article-title":"Development, validation, and clinical utility assessment of a prognostic score for 1-year unplanned rehospitalization or death of adult sepsis survivors.","volume":"3","author":"Shankar-Hari","year":"2020","journal-title":"JAMA Netw Open."},{"key":"B50","article-title":"Measuring healthcare quality.","author":"Quentin","year":"2019","journal-title":"Improving Healthcare Quality in Europe: Characteristics, Effectiveness and Implementation of Different Strategies. Health Policy Series. 53."},{"key":"B51","doi-asserted-by":"publisher","first-page":"175","DOI":"10.1016\/j.ccc.2017.08.013","article-title":"Improving long-term outcomes after sepsis.","volume":"34","author":"Prescott","year":"2018","journal-title":"Crit Care Clin."},{"key":"B52","doi-asserted-by":"publisher","first-page":"972","DOI":"10.1164\/rccm.201812-2383CP","article-title":"Understanding and enhancing sepsis survivorship. Priorities for research and practice.","volume":"200","author":"Prescott","year":"2019","journal-title":"Am J Respir Crit Care Med."},{"key":"B53","year":"2021","journal-title":"Methodische Grundlagen: Entwurf f\u00fcr Version 2.0."},{"key":"B54","article-title":"G-IQI Version 5.0 \u2013 Messen, um zu verbessern.","author":"Nimptsch","year":"2017","journal-title":"Handbuch IQM: Kosequent transparent \u2013 Qualit\u00e4t mit Routinedaten!."},{"key":"B55","doi-asserted-by":"publisher","first-page":"487","DOI":"10.1016\/j.zefq.2014.09.023","article-title":"Hospital quality measurement \u2013 what matters?","volume":"108","author":"Mansky","year":"2014","journal-title":"Z Evid Fortbild Qual Gesundhwes."},{"key":"B56","doi-asserted-by":"publisher","first-page":"1826","DOI":"10.1007\/s00134-018-5377-4","article-title":"Challenges in assessing the burden of sepsis and understanding the inequalities of sepsis outcomes between national health systems: secular trends in sepsis and infection incidence and mortality in Germany.","volume":"44","author":"Fleischmann-Struzek","year":"2018","journal-title":"Intensive Care Med."},{"key":"B57","year":"2019","journal-title":"Qualit\u00e4tssicherungsverfahren Diagnostik, Therapie und Nachsorge der Sepsis. Konzeptstudie."},{"key":"B58","doi-asserted-by":"publisher","DOI":"10.1186\/s13054-015-0847-3","article-title":"Validity of administrative data in recording sepsis: a systematic review.","volume":"19","author":"Jolley","year":"2015","journal-title":"Crit Care."},{"key":"B59","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0198847","article-title":"Comparing the validity of different ICD coding abstraction strategies for sepsis case identification in German claims data.","volume":"13","author":"Fleischmann-Struzek","year":"2018","journal-title":"PLoS One."},{"key":"B60","doi-asserted-by":"publisher","first-page":"1247","DOI":"10.1097\/CCM.0000000000003184","article-title":"Mortality measures to profile hospital performance for patients with septic shock.","volume":"46","author":"Walkey","year":"2018","journal-title":"Crit Care Med."},{"key":"B61","doi-asserted-by":"publisher","first-page":"626","DOI":"10.1093\/bja\/aex234","article-title":"Epidemiology of sepsis and septic shock in critical care units: comparison between sepsis-2 and sepsis-3 populations using a national critical care database.","volume":"119","author":"Shankar-Hari","year":"2017","journal-title":"Br J Anaesth."},{"key":"B62","doi-asserted-by":"publisher","first-page":"160","DOI":"10.1016\/j.ajic.2015.09.005","article-title":"Extra length of stay and costs because of health care\u2013associated infections at a German university hospital.","volume":"44","author":"Arefian","year":"2016","journal-title":"Am J Infect Control."}],"container-title":["Frontiers in Medicine"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2022.1069042\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,9]],"date-time":"2023-01-09T09:55:33Z","timestamp":1673258133000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2022.1069042\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,9]]},"references-count":62,"alternative-id":["10.3389\/fmed.2022.1069042"],"URL":"https:\/\/doi.org\/10.3389\/fmed.2022.1069042","relation":{},"ISSN":["2296-858X"],"issn-type":[{"value":"2296-858X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,9]]},"article-number":"1069042"}}