{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T22:22:55Z","timestamp":1792794175233,"version":"4.5.0"},"reference-count":46,"publisher":"Oxford University Press (OUP)","issue":"4","license":[{"start":{"date-parts":[[2023,10,25]],"date-time":"2023-10-25T00:00:00Z","timestamp":1698192000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"ORCHESTRA"},{"name":"European Union\u2019s Horizon 2020 research and innovation program","award":["101016167"],"award-info":[{"award-number":["101016167"]}]},{"name":"German Federal Ministry of Education and Research","award":["01KX2121"],"award-info":[{"award-number":["01KX2121"]}],"id":[{"id":"https:\/\/ror.org\/04pz7b180","id-type":"ROR","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,4,10]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Numerous prognostic scores have been published to support risk stratification for patients with coronavirus disease 2019 (COVID-19).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We performed a systematic review to identify the scores for confirmed or clinically assumed COVID-19 cases. An in-depth assessment and risk of bias (ROB) analysis (Prediction model Risk Of Bias ASsessment Tool [PROBAST]) was conducted for scores fulfilling predefined criteria ([I] area under the curve [AUC)] \u2265 0.75; [II] a separate validation cohort present; [III] training data from a multicenter setting [\u22652 centers]; [IV] point-scale scoring system).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Out of 1522 studies extracted from MEDLINE\/Web of Science (20\/02\/2023), we identified 242 scores for COVID-19 outcome prognosis (mortality 109, severity 116, hospitalization 14, long-term sequelae 3). Most scores were developed using retrospective (75.2%) or single-center (57.1%) cohorts. Predictor analysis revealed the primary use of laboratory data and sociodemographic information in mortality and severity scores. Forty-nine scores were included in the in-depth analysis. The results indicated heterogeneous quality and predictor selection, with only five scores featuring low ROB. Among those, based on the number and heterogeneity of validation studies, only the 4C Mortality Score can be recommended for clinical application so far.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>The application and translation of most existing COVID scores appear unreliable. Guided development and predictor selection would have improved the generalizability of the scores and may enhance pandemic preparedness in the future.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/cid\/ciad618","type":"journal-article","created":{"date-parts":[[2023,10,23]],"date-time":"2023-10-23T16:59:22Z","timestamp":1698080362000},"page":"889-899","source":"Crossref","is-referenced-by-count":20,"title":["A Systematic Review of Predictor Composition, Outcomes, Risk of Bias, and Validation of COVID-19 Prognostic Scores"],"prefix":"10.1093","volume":"78","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3760-6992","authenticated-orcid":false,"given":"Katharina S","family":"Appel","sequence":"first","affiliation":[{"name":"Department II of Internal Medicine, Hematology\/Oncology, Goethe University Frankfurt , Frankfurt am Main ,","place":["Germany"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ramsia","family":"Geisler","sequence":"additional","affiliation":[{"name":"Department II of Internal Medicine, Hematology\/Oncology, Goethe University Frankfurt , Frankfurt am Main ,","place":["Germany"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Maier","sequence":"additional","affiliation":[{"name":"Department II of Internal Medicine, Hematology\/Oncology, Goethe University Frankfurt , Frankfurt am Main ,","place":["Germany"]},{"name":"German Cancer Consortium (DKTK), Partner Site Frankfurt\/Mainz and German Cancer Research Center (DKFZ) , Heidelberg ,","place":["Germany"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Olga","family":"Miljukov","sequence":"additional","affiliation":[{"name":"Institute of Clinical Epidemiology and Biometry, University of W\u00fcrzburg , W\u00fcrzburg ,","place":["Germany"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sina M","family":"P\u00fctz","sequence":"additional","affiliation":[{"name":"University of Cologne, Faculty of Medicine and University Hospital Cologne, Department I of Internal Medicine, Center for Integrated Oncology Aachen Bonn Cologne Duesseldorf, Cologne, Germany, University of Cologne"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"J Janne","family":"Vehreschild","sequence":"additional","affiliation":[{"name":"Department II of Internal Medicine, Hematology\/Oncology, Goethe University Frankfurt , Frankfurt am Main ,","place":["Germany"]},{"name":"University of Cologne, Faculty of Medicine and University Hospital Cologne, Department I of Internal Medicine , Cologne ,","place":["Germany"]},{"name":"German Centre for Infection Research (DZIF), partner site Bonn-Cologne , Cologne ,","place":["Germany"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2023,10,25]]},"reference":[{"key":"2026061100272343000_ciad618-B1","doi-asserted-by":"crossref","first-page":"1295","DOI":"10.3390\/jpm12081295","article-title":"Global challenges to public health care systems during the COVID-19 pandemic: a review of pandemic measures and problems","volume":"12","author":"Filip","year":"2022","journal-title":"J Pers Med"},{"key":"2026061100272343000_ciad618-B2","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1186\/s40246-022-00392-1","article-title":"COVID-19 2022 update: transition of the pandemic to the endemic phase","volume":"16","author":"Biancolella","year":"2022","journal-title":"Hum Genomics"},{"key":"2026061100272343000_ciad618-B3","doi-asserted-by":"crossref","first-page":"e2381","DOI":"10.1002\/rmv.2381","article-title":"Evolution of the SARS-CoV-2 Omicron variants BA.1 to BA.5: implications for immune escape and transmission","volume":"32","author":"Shrestha","year":"2022","journal-title":"Rev Med Virol"},{"key":"2026061100272343000_ciad618-B4","volume-title":"Therapeutics and COVID-19: living guideline, 13 January 2023","year":"2023"},{"key":"2026061100272343000_ciad618-B5","doi-asserted-by":"crossref","first-page":"46","DOI":"10.1016\/j.ajem.2022.01.028","article-title":"Clinical update on COVID-19 for the emergency clinician: presentation and evaluation","volume":"54","author":"Long","year":"2022","journal-title":"Am J Emerg Med"},{"key":"2026061100272343000_ciad618-B6","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1186\/s41512-019-0060-y","article-title":"Methodological standards for the development and evaluation of clinical prediction rules: a review of the literature","volume":"3","author":"Cowley","year":"2019","journal-title":"Diagn Progn Res"},{"key":"2026061100272343000_ciad618-B7","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pmed.1001744","article-title":"Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist","volume":"11","author":"Moons","year":"2014","journal-title":"PLoS Med"},{"key":"2026061100272343000_ciad618-B8","doi-asserted-by":"crossref","first-page":"m1328","DOI":"10.1136\/bmj.m1328","article-title":"Prediction models for diagnosis and prognosis of COVID-19: systematic review and critical appraisal","volume":"369","author":"Wynants","year":"2020","journal-title":"BMJ"},{"key":"2026061100272343000_ciad618-B9","doi-asserted-by":"crossref","first-page":"206","DOI":"10.1111\/acem.14447","article-title":"Prediction models for severe manifestations and mortality due to COVID-19: a rapid systematic review","volume":"29","author":"Miller","year":"2022","journal-title":"Acad Emerg Med"},{"key":"2026061100272343000_ciad618-B10","doi-asserted-by":"crossref","DOI":"10.1136\/bmjhci-2021-100389","article-title":"Evaluating risk stratification scoring systems to predict mortality in patients with COVID-19","volume":"28","author":"Chu","year":"2021","journal-title":"BMJ Health Care Inform"},{"key":"2026061100272343000_ciad618-B11","author":"Infectious Diseases Society of America","year":"2023"},{"key":"2026061100272343000_ciad618-B12","doi-asserted-by":"crossref","first-page":"m3339","DOI":"10.1136\/bmj.m3339","article-title":"Risk stratification of patients admitted to hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: development and validation of the 4C Mortality Score","volume":"370","author":"Knight","year":"2020","journal-title":"BMJ"},{"key":"2026061100272343000_ciad618-B13","author":"Clinical management of COVID-19: living guideline, 13 January","year":"2023"},{"key":"2026061100272343000_ciad618-B14","doi-asserted-by":"crossref","first-page":"n71","DOI":"10.1136\/bmj.n71","article-title":"The PRISMA 2020 statement: an updated guideline for reporting systematic reviews","volume":"372","author":"Page","year":"2021","journal-title":"BMJ"},{"key":"2026061100272343000_ciad618-B15","doi-asserted-by":"crossref","first-page":"W1","DOI":"10.7326\/M18-1377","article-title":"PROBAST: a tool to assess risk of bias and applicability of prediction model studies: explanation and elaboration","volume":"170","author":"Moons","year":"2019","journal-title":"Ann Intern Med"},{"key":"2026061100272343000_ciad618-B16","doi-asserted-by":"crossref","first-page":"g7594","DOI":"10.1136\/bmj.g7594","article-title":"Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement","volume":"350","author":"Collins","year":"2015","journal-title":"BMJ"},{"key":"2026061100272343000_ciad618-B17","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1186\/s12916-019-1466-7","article-title":"Calibration: the Achilles heel of predictive analytics","volume":"17","author":"van Calster","year":"2019","journal-title":"BMC Med"},{"key":"2026061100272343000_ciad618-B18","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1093\/ckj\/sfaa188","article-title":"External validation of prognostic models: what, why, how, when and where?","volume":"14","author":"Ramspek","year":"2021","journal-title":"Clin Kidney J"},{"key":"2026061100272343000_ciad618-B19","first-page":"1","article-title":"Derivation of a contextually-appropriate COVID-19 mortality scale for low-resource settings","volume":"87","author":"Pigoga","year":"2021","journal-title":"Ann Glob Health"},{"key":"2026061100272343000_ciad618-B20","doi-asserted-by":"crossref","first-page":"E90","DOI":"10.9778\/cmajo.20210243","article-title":"The CCEDRRN COVID-19 mortality score to predict death among nonpalliative patients with COVID-19 presenting to emergency departments: a derivation and validation study","volume":"10","author":"Hohl","year":"2022","journal-title":"CMAJ Open"},{"key":"2026061100272343000_ciad618-B21","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pone.0287091","article-title":"LMIC-PRIEST: derivation and validation of a clinical severity score for acutely ill adults with suspected COVID-19 in a middle-income setting","volume":"18","author":"Marincowitz","year":"2023","journal-title":"PLoS One"},{"key":"2026061100272343000_ciad618-B22","doi-asserted-by":"crossref","first-page":"917","DOI":"10.1080\/03007995.2021.1904862","article-title":"A risk score based on baseline risk factors for predicting mortality in COVID-19 patients","volume":"37","author":"Chen","year":"2021","journal-title":"Curr Med Res Opin"},{"key":"2026061100272343000_ciad618-B23","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pone.0245840","article-title":"Derivation and validation of a clinical severity score for acutely ill adults with suspected COVID-19: the PRIEST observational cohort study","volume":"16","author":"Goodacre","year":"2021","journal-title":"PLoS One"},{"key":"2026061100272343000_ciad618-B24","doi-asserted-by":"crossref","first-page":"4945","DOI":"10.1038\/s41598-021-84603-0","article-title":"SARS2 Simplified scores to estimate risk of hospitalization and death among patients with COVID-19","volume":"11","author":"Dashti","year":"2021","journal-title":"Sci Rep"},{"key":"2026061100272343000_ciad618-B25","doi-asserted-by":"crossref","first-page":"920","DOI":"10.1136\/thoraxjnl-2020-216001","article-title":"Development and validation of a prediction model for 30-day mortality in hospitalised patients with COVID-19: the COVID-19 SEIMC score","volume":"76","author":"Berenguer","year":"2021","journal-title":"Thorax"},{"key":"2026061100272343000_ciad618-B26","doi-asserted-by":"crossref","first-page":"183","DOI":"10.3390\/life13010183","article-title":"Is the 4C score still a valid item to predict in-hospital mortality in people with SARS-CoV-2 infections in the Omicron variant era?","volume":"13","author":"de Vito","year":"2023","journal-title":"Life"},{"key":"2026061100272343000_ciad618-B27","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1186\/s12874-016-0223-2","article-title":"Predictor characteristics necessary for building a clinically useful risk prediction model: a simulation study","volume":"16","author":"Schummers","year":"2016","journal-title":"BMC Med Res Methodol"},{"key":"2026061100272343000_ciad618-B28","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1002\/rmv.2146","article-title":"Predictors of COVID-19 severity: a literature review","volume":"31","author":"Marin","year":"2021","journal-title":"Rev Med Virol"},{"key":"2026061100272343000_ciad618-B29","doi-asserted-by":"crossref","first-page":"801","DOI":"10.1001\/jama.2016.0287","article-title":"The third international consensus definitions for sepsis and septic shock (sepsis-3)","volume":"315","author":"Singer","year":"2016","journal-title":"JAMA"},{"key":"2026061100272343000_ciad618-B30","first-page":"665","article-title":"Management of adult community-acquired pneumonia and prevention (AWMF-register-nr. 020-020)","volume":"75","author":"Ewig","year":"2021","journal-title":"Pneumologie"},{"key":"2026061100272343000_ciad618-B31","doi-asserted-by":"crossref","first-page":"691","DOI":"10.1055\/a-1498-2512","article-title":"S3-Leitlinie der Deutschen gesellschaft f\u00fcr gastroenterologie, verdauungs- und stoffwechselkrankheiten (DGVS) zur prophylaxe, diagnostik und therapie der hepatitis-B-virusinfektion, AWMF-Register-Nr. 021-11","volume":"59","author":"Cornberg","year":"2021","journal-title":"Z Gastroenterol"},{"key":"2026061100272343000_ciad618-B32","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1055\/a-1735-3864","article-title":"S3-Leitlinie pankreatitis\u2014leitlinie der deutschen gesellschaft f\u00fcr gastroenterologie, verdauungs- und stoffwechselkrankheiten (DGVS), September 2021\u2014 \u2013 AWMF Registernummer 021-003","volume":"60","author":"Beyer","year":"2022","journal-title":"Z Gastroenterol"},{"key":"2026061100272343000_ciad618-B33","doi-asserted-by":"crossref","first-page":"20709","DOI":"10.11613\/BM.2021.020709","article-title":"Performance of D-dimer for predicting sepsis mortality in the intensive care unit","volume":"31","author":"Han","year":"2021","journal-title":"Biochem Med"},{"key":"2026061100272343000_ciad618-B34","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1186\/s12893-016-0186-6","article-title":"Albumin and surgical site infection risk in orthopaedics: a meta-analysis","volume":"17","author":"Yuwen","year":"2017","journal-title":"BMC Surg"},{"key":"2026061100272343000_ciad618-B35","doi-asserted-by":"crossref","first-page":"127","DOI":"10.3122\/jabfm.2021.S1.200464","article-title":"Development and validation of the COVID-NoLab and COVID-SimpleLab risk scores for prognosis in 6 US health systems","volume":"34","author":"Ebell","year":"2021","journal-title":"J Am Board Fam Med"},{"key":"2026061100272343000_ciad618-B36","doi-asserted-by":"crossref","DOI":"10.3389\/fmicb.2022.1031231","article-title":"A practical scoring model to predict the occurrence of critical illness in hospitalized patients with SARS-CoV-2 omicron infection","volume":"13","author":"Zhang","year":"2022","journal-title":"Front Microbiol"},{"key":"2026061100272343000_ciad618-B37","doi-asserted-by":"crossref","first-page":"1774","DOI":"10.1056\/NEJMoa2113017","article-title":"Efficacy of the mRNA-1273 SARS-CoV-2 vaccine at completion of blinded phase","volume":"385","author":"El Sahly","year":"2021","journal-title":"N Engl J Med"},{"key":"2026061100272343000_ciad618-B38","doi-asserted-by":"crossref","first-page":"2603","DOI":"10.1056\/NEJMoa2034577","article-title":"Safety and efficacy of the BNT162b2 mRNA COVID-19 vaccine","volume":"383","author":"Polack","year":"2020","journal-title":"N Engl J Med"},{"key":"2026061100272343000_ciad618-B39","doi-asserted-by":"crossref","DOI":"10.1183\/13993003.02964-2020","article-title":"Prognostic factors for adverse outcomes in patients with COVID-19: a field-wide systematic review and meta-analysis","volume":"59","author":"Bellou","year":"2022","journal-title":"Eur Respir J"},{"key":"2026061100272343000_ciad618-B40","doi-asserted-by":"crossref","first-page":"362","DOI":"10.7326\/M20-6306","article-title":"Racial and ethnic disparities in COVID-19\u2013related infections, hospitalizations, and deaths: a systematic review","volume":"174","author":"Mackey","year":"2021","journal-title":"Ann Intern Med"},{"key":"2026061100272343000_ciad618-B41","doi-asserted-by":"crossref","first-page":"113","DOI":"10.3122\/jabfm.2021.S1.200429","article-title":"Primary care relevant risk factors for adverse outcomes in patients with COVID-19 infection: a systematic review","volume":"34","author":"Bentivegna","year":"2021","journal-title":"JABFM"},{"key":"2026061100272343000_ciad618-B42","doi-asserted-by":"crossref","first-page":"1377","DOI":"10.1001\/jama.2017.12126","article-title":"Discrimination and calibration of clinical prediction models: users' guides to the medical literature","volume":"318","author":"Alba","year":"2017","journal-title":"JAMA"},{"key":"2026061100272343000_ciad618-B43","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1038\/s41432-020-0115-5","article-title":"Can medical practitioners rely on prediction models for COVID-19? A systematic review","volume":"21","author":"Shamsoddin","year":"2020","journal-title":"Evid Based Dent"},{"key":"2026061100272343000_ciad618-B44","doi-asserted-by":"crossref","first-page":"355","DOI":"10.1007\/s10654-023-00973-x","article-title":"Prognostic models in COVID-19 infection that predict severity: a systematic review","volume":"38","author":"Buttia","year":"2023","journal-title":"Eur J Epidemiol"},{"key":"2026061100272343000_ciad618-B45","doi-asserted-by":"crossref","first-page":"1426","DOI":"10.1007\/s00134-021-06524-w","article-title":"External validation of prognostic scores for COVID-19: a multicenter cohort study of patients hospitalized in Greater Paris University Hospitals","volume":"47","author":"Lombardi","year":"2021","journal-title":"Intensive Care Med"},{"key":"2026061100272343000_ciad618-B46","doi-asserted-by":"crossref","DOI":"10.1183\/13993003.03498-2020","article-title":"Systematic evaluation and external validation of 22 prognostic models among hospitalised adults with COVID-19: an observational cohort study","volume":"56","author":"Gupta","year":"2020","journal-title":"Eur Respir J"}],"container-title":["Clinical Infectious Diseases"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/cid\/advance-article-pdf\/doi\/10.1093\/cid\/ciad618\/53919023\/ciad618.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/cid\/article-pdf\/78\/4\/889\/57208951\/ciad618.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/cid\/article-pdf\/78\/4\/889\/57208951\/ciad618.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,11]],"date-time":"2026-06-11T04:27:35Z","timestamp":1781152055000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/cid\/article\/78\/4\/889\/7329954"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,25]]},"references-count":46,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2023,10,25]]},"published-print":{"date-parts":[[2024,4,10]]}},"URL":"https:\/\/doi.org\/10.1093\/cid\/ciad618","relation":{},"ISSN":["1058-4838","1537-6591"],"issn-type":[{"value":"1058-4838","type":"print"},{"value":"1537-6591","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,4,15]]},"published":{"date-parts":[[2023,10,25]]}}}