{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T03:10:28Z","timestamp":1792725028758,"version":"4.5.0"},"reference-count":43,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,1,21]],"date-time":"2021-01-21T00:00:00Z","timestamp":1611187200000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2021,1,21]],"date-time":"2021-01-21T00:00:00Z","timestamp":1611187200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med"],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>The National Early Warning Score (NEWS2) is currently recommended in the UK for the risk stratification of COVID-19 patients, but little is known about its ability to detect severe cases. We aimed to evaluate NEWS2 for the prediction of severe COVID-19 outcome and identify and validate a set of blood and physiological parameters routinely collected at hospital admission to improve upon the use of NEWS2 alone for medium-term risk stratification.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>Training cohorts comprised 1276 patients admitted to King\u2019s College Hospital National Health Service (NHS) Foundation Trust with COVID-19 disease from 1 March to 30 April 2020. External validation cohorts included 6237 patients from five UK NHS Trusts (Guy\u2019s and St Thomas\u2019 Hospitals, University Hospitals Southampton, University Hospitals Bristol and Weston NHS Foundation Trust, University College London Hospitals, University Hospitals Birmingham), one hospital in Norway (Oslo University Hospital), and two hospitals in Wuhan, China (Wuhan Sixth Hospital and Taikang Tongji Hospital). The outcome was severe COVID-19 disease (transfer to intensive care unit (ICU) or death) at 14\u2009days after hospital admission. Age, physiological measures, blood biomarkers, sex, ethnicity, and comorbidities (hypertension, diabetes, cardiovascular, respiratory and kidney diseases) measured at hospital admission were considered in the models.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>A baseline model of \u2018NEWS2 + age\u2019 had poor-to-moderate discrimination for severe COVID-19 infection at 14\u2009days (area under receiver operating characteristic curve (AUC) in training cohort\u2009=\u20090.700, 95% confidence interval (CI) 0.680, 0.722; Brier score\u2009=\u20090.192, 95% CI 0.186, 0.197). A supplemented model adding eight routinely collected blood and physiological parameters (supplemental oxygen flow rate, urea, age, oxygen saturation, C-reactive protein, estimated glomerular filtration rate, neutrophil count, neutrophil\/lymphocyte ratio) improved discrimination (AUC\u2009=\u20090.735; 95% CI 0.715, 0.757), and these improvements were replicated across seven UK and non-UK sites. However, there was evidence of miscalibration with the model tending to underestimate risks in most sites.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>NEWS2 score had poor-to-moderate discrimination for medium-term COVID-19 outcome which raises questions about its use as a screening tool at hospital admission. Risk stratification was improved by including readily available blood and physiological parameters measured at hospital admission, but there was evidence of miscalibration in external sites. This highlights the need for a better understanding of the use of early warning scores for COVID.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1186\/s12916-020-01893-3","type":"journal-article","created":{"date-parts":[[2021,1,20]],"date-time":"2021-01-20T21:02:57Z","timestamp":1611176577000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":97,"title":["Evaluation and improvement of the National Early Warning Score (NEWS2) for COVID-19: a multi-hospital study"],"prefix":"10.1186","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1146-4922","authenticated-orcid":false,"given":"Ewan","family":"Carr","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rebecca","family":"Bendayan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Bean","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Matt","family":"Stammers","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wenjuan","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Huayu","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thomas","family":"Searle","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zeljko","family":"Kraljevic","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anthony","family":"Shek","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hang T. T.","family":"Phan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Walter","family":"Muruet","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rishi K.","family":"Gupta","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anthony J.","family":"Shinton","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mike","family":"Wyatt","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ting","family":"Shi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xin","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrew","family":"Pickles","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Stahl","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rosita","family":"Zakeri","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mahdad","family":"Noursadeghi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kevin","family":"O\u2019Gallagher","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Matt","family":"Rogers","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Amos","family":"Folarin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andreas","family":"Karwath","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kristin E.","family":"Wickstr\u00f8m","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alvaro","family":"K\u00f6hn-Luque","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Luke","family":"Slater","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Victor Roth","family":"Cardoso","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christopher","family":"Bourdeaux","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aleksander Rygh","family":"Holten","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Simon","family":"Ball","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chris","family":"McWilliams","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lukasz","family":"Roguski","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Florina","family":"Borca","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"James","family":"Batchelor","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Erik Koldberg","family":"Amundsen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaodong","family":"Wu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Georgios V.","family":"Gkoutos","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiaxing","family":"Sun","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ashwin","family":"Pinto","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bruce","family":"Guthrie","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cormac","family":"Breen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Abdel","family":"Douiri","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Honghan","family":"Wu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Vasa","family":"Curcin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"James T.","family":"Teo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ajay M.","family":"Shah","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Richard J. B.","family":"Dobson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,1,21]]},"reference":[{"key":"1893_CR1","volume-title":"WHO COVID-19 dashboard","author":"WHO","year":"2020","unstructured":"WHO. WHO COVID-19 dashboard. 2020. https:\/\/who.sprinklr.com\/."},{"key":"1893_CR2","doi-asserted-by":"publisher","first-page":"1054","DOI":"10.1016\/S0140-6736(20)30566-3","volume":"395","author":"F Zhou","year":"2020","unstructured":"Zhou F, Yu T, Du R, Fan G, Liu Y, Liu Z, et al. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study. Lancet. 2020;395:1054\u201362.","journal-title":"Lancet"},{"key":"1893_CR3","doi-asserted-by":"publisher","unstructured":"Scott LJ, Redmond NM, Tavar\u00e9 A, Little H, Srivastava S, Pullyblank A. Association between National Early Warning Scores in primary care and clinical outcomes: an observational study in UK primary and secondary care. Br J Gen Pract. 2020. https:\/\/doi.org\/10.3399\/bjgp20X709337.","DOI":"10.3399\/bjgp20X709337"},{"key":"1893_CR4","doi-asserted-by":"publisher","first-page":"374","DOI":"10.1186\/s13054-019-2663-7","volume":"23","author":"S Lambden","year":"2019","unstructured":"Lambden S, Laterre PF, Levy MM, Francois B. The SOFA score\u2014development, utility and challenges of accurate assessment in clinical trials. Crit Care. 2019;23:374.","journal-title":"Crit Care"},{"key":"1893_CR5","doi-asserted-by":"publisher","unstructured":"Liu S, Yao N, Qiu Y, He C. Predictive performance of SOFA and qSOFA for in-hospital mortality in severe novel coronavirus disease. Am J Emerg Med. 2020. https:\/\/doi.org\/10.1016\/j.ajem.2020.07.019.","DOI":"10.1016\/j.ajem.2020.07.019"},{"key":"1893_CR6","doi-asserted-by":"publisher","first-page":"377","DOI":"10.1136\/thorax.58.5.377","volume":"58","author":"WS Lim","year":"2003","unstructured":"Lim WS, van der Eerden MM, Laing R, Boersma WG, Karalus N, Town GI, et al. Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study. Thorax. 2003;58:377\u201382.","journal-title":"Thorax."},{"key":"1893_CR7","volume-title":"National Early Warning Score (NEWS) 2: standardising the assessment of acute-illness severity in the NHS. Updated report of a working party","author":"Royal College of Physicians","year":"2017","unstructured":"Royal College of Physicians. National Early Warning Score (NEWS) 2: standardising the assessment of acute-illness severity in the NHS. Updated report of a working party. London: RCP; 2017."},{"key":"1893_CR8","doi-asserted-by":"publisher","first-page":"465","DOI":"10.1016\/j.resuscitation.2012.12.016","volume":"84","author":"GB Smith","year":"2013","unstructured":"Smith GB, Prytherch DR, Meredith P, Schmidt PE, Featherstone PI. The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death. Resuscitation. 2013;84:465\u201370.","journal-title":"Resuscitation."},{"key":"1893_CR9","unstructured":"Greenhalgh T, Treadwell J, Burrow R. NEWS (or NEWS2) score when assessing possible COVID-19 patients in primary care. Cent Evid-Based Med Nuffield Dep Prim Care Health Sci Univ Oxf. 2020;20. https:\/\/www.cebm.net\/covid-19\/should-we-use-the-news-or-news2-score-when-assessing-patients-with-possible-covid-19-inprimary-care\/."},{"key":"1893_CR10","doi-asserted-by":"publisher","unstructured":"Ji D, Zhang D, Xu J, Chen Z, Yang T, Zhao P, et al. Prediction for progression risk in patients with COVID-19 pneumonia: the CALL score. Clin Infect Dis. https:\/\/doi.org\/10.1093\/cid\/ciaa414.","DOI":"10.1093\/cid\/ciaa414"},{"key":"1893_CR11","doi-asserted-by":"publisher","first-page":"108","DOI":"10.1186\/s13054-020-2833-7","volume":"24","author":"Y Shi","year":"2020","unstructured":"Shi Y, Yu X, Zhao H, Wang H, Zhao R, Sheng J. Host susceptibility to severe COVID-19 and establishment of a host risk score: findings of 487 cases outside Wuhan. Crit Care. 2020;24:108.","journal-title":"Crit Care"},{"key":"1893_CR12","unstructured":"COVIDAnalytics. https:\/\/www.covidanalytics.io\/calculator. Access date\u00a021 April 2020."},{"key":"1893_CR13","doi-asserted-by":"publisher","unstructured":"Wynants L, Calster BV, Bonten MMJ, Collins GS, Debray TPA, Vos MD, et al. Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal. BMJ. 2020;369. https:\/\/doi.org\/10.1136\/bmj.m1328.","DOI":"10.1136\/bmj.m1328"},{"key":"1893_CR14","doi-asserted-by":"publisher","first-page":"357","DOI":"10.1007\/s00134-020-05954-2","volume":"46","author":"X Liao","year":"2020","unstructured":"Liao X, Wang B, Kang Y. Novel coronavirus infection during the 2019\u20132020 epidemic: preparing intensive care units\u2014the experience in Sichuan Province, China. Intensive Care Med. 2020;46:357\u201360.","journal-title":"Intensive Care Med"},{"key":"1893_CR15","unstructured":"NHS England \u00bb National Early Warning Score (NEWS). https:\/\/www.england.nhs.uk\/ourwork\/clinical-policy\/sepsis\/nationalearlywarningscore\/. Access date\u00a023 April 2020."},{"key":"1893_CR16","doi-asserted-by":"publisher","first-page":"497","DOI":"10.1016\/S0140-6736(20)30183-5","volume":"395","author":"C Huang","year":"2020","unstructured":"Huang C, Wang Y, Li X, Ren L, Zhao J, Hu Y, et al. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. Lancet. 2020;395:497\u2013506.","journal-title":"Lancet"},{"key":"1893_CR17","doi-asserted-by":"publisher","unstructured":"Li K, Wu J, Wu F, Guo D, Chen L, Fang Z, et al. The clinical and chest CT features associated with severe and critical COVID-19 pneumonia. Invest Radiol. 2020;Publish Ahead of Print. doi:https:\/\/doi.org\/10.1097\/RLI.0000000000000672.","DOI":"10.1097\/RLI.0000000000000672"},{"key":"1893_CR18","doi-asserted-by":"publisher","first-page":"e205619","DOI":"10.1001\/jamanetworkopen.2020.5619","volume":"3","author":"J Xie","year":"2020","unstructured":"Xie J, Tong Z, Guan X, Du B, Qiu H. Clinical characteristics of patients who died of coronavirus disease 2019 in China. JAMA Netw Open. 2020;3:e205619.","journal-title":"JAMA Netw Open"},{"key":"1893_CR19","doi-asserted-by":"crossref","unstructured":"Zhang J-J, Dong X, Cao Y-Y, Yuan Y-D, Yang Y-B, Yan Y-Q, et al. Clinical characteristics of 140 patients infected with SARS-CoV-2 in Wuhan, China. Allergy. 2020;75:1730-41.\u00a0","DOI":"10.1111\/all.14238"},{"key":"1893_CR20","doi-asserted-by":"crossref","unstructured":"Ruan Q, Yang K, Wang W, Jiang L, Song J. Clinical predictors of mortality due to COVID-19 based on an analysis of data of 150 patients from Wuhan, China. Intensive Care Med. 2020;46:846\u20138.\u00a0","DOI":"10.1007\/s00134-020-05991-x"},{"key":"1893_CR21","doi-asserted-by":"publisher","unstructured":"Guan W, Ni Z, Hu Y, Liang W, Ou C, He J, et al. Clinical characteristics of coronavirus disease 2019 in China. N Engl J Med. 2020. https:\/\/doi.org\/10.1056\/NEJMoa2002032.","DOI":"10.1056\/NEJMoa2002032"},{"key":"1893_CR22","doi-asserted-by":"crossref","unstructured":"Lagunas-Rangel FA. Neutrophil-to-lymphocyte ratio and lymphocyte-to-C-reactive protein ratio in patients with severe coronavirus disease 2019 (COVID-19): a meta-analysis. J Med Virol. 2020;92:1733-4.\u00a0","DOI":"10.1002\/jmv.25819"},{"key":"1893_CR23","doi-asserted-by":"publisher","unstructured":"Guo T, Fan Y, Chen M, Wu X, Zhang L, He T, et al. Cardiovascular implications of fatal outcomes of patients with coronavirus disease 2019 (COVID-19). JAMA Cardiol. 2020. https:\/\/doi.org\/10.1001\/jamacardio.2020.1017.","DOI":"10.1001\/jamacardio.2020.1017"},{"key":"1893_CR24","doi-asserted-by":"crossref","unstructured":"Carr E, Bendayan R, Bean D, Stammers M, Wang W, Zhang H, et al. Evaluation and improvement of the National Early Warning Score (NEWS2) for COVID-19: a multi-hospital study. medRxiv. 2020. https:\/\/www.medrxiv.org\/content\/10.1101\/2020.04.24.20078006v4.article-info.","DOI":"10.1101\/2020.04.24.20078006"},{"key":"1893_CR25","doi-asserted-by":"publisher","first-page":"e192","DOI":"10.1016\/S1473-3099(20)30483-7","volume":"20","author":"JC Marshall","year":"2020","unstructured":"Marshall JC, Murthy S, Diaz J, Adhikari NK, Angus DC, Arabi YM, et al. A minimal common outcome measure set for COVID-19 clinical research. Lancet Infect Dis. 2020;20:e192\u20137.","journal-title":"Lancet Infect Dis"},{"key":"1893_CR26","doi-asserted-by":"publisher","unstructured":"Jackson R, Kartoglu I, Stringer C, Gorrell G, Roberts A, Song X, et al. CogStack - experiences of deploying integrated information retrieval and extraction services in a large National Health Service Foundation Trust hospital. BMC Med Inform Decis Mak. 2018;18. https:\/\/doi.org\/10.1186\/s12911-018-0623-9.","DOI":"10.1186\/s12911-018-0623-9"},{"key":"1893_CR27","unstructured":"Kraljevic Z, Bean D, Mascio A, Roguski L, Folarin A, Roberts A, et al. MedCAT -- Medical Concept Annotation Tool. ArXiv191210166 Cs Stat. 2019.\u00a0https:\/\/arxiv.org\/abs\/1912.10166."},{"key":"1893_CR28","doi-asserted-by":"crossref","unstructured":"Searle T, Kraljevic Z, Bendayan R, Bean D, Dobson R. MedCATTrainer: a biomedical free text annotation interface with active learning and research use case specific customisation. ArXiv190707322 Cs. 2019.\u00a0https:\/\/arxiv.org\/abs\/1907.07322.","DOI":"10.18653\/v1\/D19-3024"},{"key":"1893_CR29","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/sdata.2016.35","volume":"3","author":"AEW Johnson","year":"2016","unstructured":"Johnson AEW, Pollard TJ, Shen L, Lehman LH, Feng M, Ghassemi M, et al. MIMIC-III, a freely accessible critical care database. Sci Data. 2016;3:1\u20139.","journal-title":"Sci Data"},{"key":"1893_CR30","doi-asserted-by":"crossref","unstructured":"Bean D, Kraljevic Z, Searle T, Bendayan R, Pickles A, Folarin A, et al. Treatment with ACE-inhibitors is associated with less severe disease with SARS-Covid-19 infection in a multi-site UK acute Hospital Trust. medRxiv. 2020;:2020.04.07.20056788.\u00a0https:\/\/www.medrxiv.org\/content\/10.1101\/2020.04.07.20056788v1.full-text.","DOI":"10.1101\/2020.04.07.20056788"},{"key":"1893_CR31","volume-title":"Python 3 reference manual","author":"G Van Rossum","year":"2009","unstructured":"Van Rossum G, Drake FL. Python 3 reference manual. Scotts Valley: CreateSpace; 2009."},{"key":"1893_CR32","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-16399-0","volume-title":"Clinical prediction models","author":"E Steyerberg","year":"2019","unstructured":"Steyerberg E. Clinical prediction models. 2nd ed. Cham: Springer; 2019.","edition":"2"},{"key":"1893_CR33","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1093\/biomet\/80.1.27","volume":"80","author":"D Firth","year":"1993","unstructured":"Firth D. Bias reduction of maximum likelihood estimates. Biometrika. 1993;80:27\u201338.","journal-title":"Biometrika."},{"key":"1893_CR34","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1111\/j.2517-6161.1995.tb02031.x","volume":"57","author":"Y Benjamini","year":"1995","unstructured":"Benjamini Y, Hochberg Y. Controlling the false discovery rate - a practical and powerful approach. J R Stat Soc Ser B-Methodol. 1995;57:289\u2013300.","journal-title":"J R Stat Soc Ser B-Methodol"},{"key":"1893_CR35","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","volume":"58","author":"R Tibshirani","year":"1996","unstructured":"Tibshirani R. Regression shrinkage and selection via the Lasso. J R Stat Soc Ser B Methodol. 1996;58:267\u201388.","journal-title":"J R Stat Soc Ser B Methodol"},{"key":"1893_CR36","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4614-6849-3","volume-title":"Applied predictive modeling","author":"M Kuhn","year":"2013","unstructured":"Kuhn M, Johnson K. Applied predictive modeling. New York: Springer; 2013."},{"key":"1893_CR37","doi-asserted-by":"crossref","unstructured":"Seabold S, Perktold J. statsmodels: econometric and statistical modeling with Python. In: 9th Python in Science conference. Austin:\u00a0SciPy; 2010.\u00a0http:\/\/conference.scipy.org\/proceedings\/scipy2010\/pdfs\/seabold.pdf.","DOI":"10.25080\/Majora-92bf1922-011"},{"key":"1893_CR38","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, et al. Scikit-learn: machine learning in Python. J Mach Learn Res. 2011;12:2825\u201330.","journal-title":"J Mach Learn Res"},{"key":"1893_CR39","doi-asserted-by":"publisher","first-page":"285","DOI":"10.1186\/s13054-015-0999-1","volume":"19","author":"S Romero-Brufau","year":"2015","unstructured":"Romero-Brufau S, Huddleston JM, Escobar GJ, Liebow M. Why the C-statistic is not informative to evaluate early warning scores and what metrics to use. Crit Care Lond Engl. 2015;19:285.","journal-title":"Crit Care Lond Engl"},{"key":"1893_CR40","doi-asserted-by":"publisher","first-page":"230","DOI":"10.1186\/s12916-019-1466-7","volume":"17","author":"B Van Calster","year":"2019","unstructured":"Van Calster B, McLernon DJ, van Smeden M, Wynants L, Steyerberg EW, Bossuyt P, et al. Calibration: the Achilles heel of predictive analytics. BMC Med. 2019;17:230.","journal-title":"BMC Med"},{"key":"1893_CR41","doi-asserted-by":"publisher","first-page":"565","DOI":"10.1177\/0272989X06295361","volume":"26","author":"AJ Vickers","year":"2006","unstructured":"Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Mak. 2006;26:565\u201374.","journal-title":"Med Decis Mak"},{"key":"1893_CR42","unstructured":"NEWS2 and deterioration in COVID-19. RCP London. 2020. https:\/\/www.rcplondon.ac.uk\/news\/news2-and-deterioration-covid-19. Access date 24 April 2020."},{"key":"1893_CR43","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1186\/1471-2288-14-3","volume":"14","author":"I Ahmed","year":"2014","unstructured":"Ahmed I, Debray TP, Moons KG, Riley RD. Developing and validating risk prediction models in an individual participant data meta-analysis. BMC Med Res Methodol. 2014;14:3.","journal-title":"BMC Med Res Methodol"}],"container-title":["BMC Medicine"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s12916-020-01893-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1186\/s12916-020-01893-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s12916-020-01893-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,22]],"date-time":"2024-08-22T14:55:19Z","timestamp":1724338519000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedicine.biomedcentral.com\/articles\/10.1186\/s12916-020-01893-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,1,21]]},"references-count":43,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["1893"],"URL":"https:\/\/doi.org\/10.1186\/s12916-020-01893-3","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2020.04.24.20078006","asserted-by":"object"}]},"ISSN":["1741-7015"],"issn-type":[{"value":"1741-7015","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,1,21]]},"assertion":[{"value":"29 September 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 December 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 January 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The KCH component of the project operated under London South East Research Ethics Committee (reference 18\/LO\/2048) approval granted to the King\u2019s Electronic Records Research Interface (KERRI); specific work on COVID-19 research was reviewed with expert patient input on a virtual committee with Caldicott Guardian oversight. The UHS validation was performed as part of an urgent service evaluation agreed with approval from trust research leads and the Caldicott Guardian. For UCH, ethical approval was given by East Midlands - Nottingham 2 Research Ethics Committee (REF: 20\/EM\/0114; IRAS: 282900). The UHB component was operated under the PIONEER Health Data Research Hub in Acute Care ethical approval provided by the East Midlands Derby REC (reference: 20\/EM\/0158). For UHBW, the project was considered as service evaluation by the organisational review board. Informed consent was deemed unnecessary due to the retrospective observational nature of the data. Ethical approval for GSTT was granted by the London Bromley Research Ethics Committee (reference 20\/HRA\/1871) to the King\u2019s Health Partners Data Analytics and Modelling COVID-19 Group to collect clinically relevant data points from patient\u2019s electronic health records. The Wuhan validation was approved by the Research Ethics Committee of Shanghai Dongfang Hospital and Taikang Tongji Hospital. For the OUH validation, a project protocol was approved by the Regional Ethical Committee of South-East Norway (Reference number 137045) and the OUH data protection officer (Reference number 20\/08822). Informed consent in the OUH cohort was waived because of the strictly observational nature of the project.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"JTHT received research support and funding from InnovateUK, Bristol-Myers-Squibb, iRhythm Technologies, and holds shares <\u2009\u00a35000 in Glaxo Smithkline and Biogen.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"23"}}