{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,19]],"date-time":"2026-01-19T08:47:42Z","timestamp":1768812462016,"version":"3.49.0"},"reference-count":39,"publisher":"Oxford University Press (OUP)","issue":"7","license":[{"start":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T00:00:00Z","timestamp":1615507200000},"content-version":"vor","delay-in-days":1,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["F31LM012894"],"award-info":[{"award-number":["F31LM012894"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000092","name":"National Library of Medicine","doi-asserted-by":"publisher","award":["5T15LM007079"],"award-info":[{"award-number":["5T15LM007079"]}],"id":[{"id":"10.13039\/100000092","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"NIH","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000092","name":"NLM","doi-asserted-by":"publisher","award":["R01HL148248"],"award-info":[{"award-number":["R01HL148248"]}],"id":[{"id":"10.13039\/100000092","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"NIH","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000050","name":"National Heart, Lung, and Blood Institute","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000050","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,7,14]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Objective<\/jats:title><jats:p>Coronavirus disease 2019 (COVID-19) patients are at risk for resource-intensive outcomes including mechanical ventilation (MV), renal replacement therapy (RRT), and readmission. Accurate outcome prognostication could facilitate hospital resource allocation. We develop and validate predictive models for each outcome using retrospective electronic health record data for COVID-19 patients treated between March 2 and May 6, 2020.<\/jats:p><\/jats:sec><jats:sec><jats:title>Materials and Methods<\/jats:title><jats:p>For each outcome, we trained 3 classes of prediction models using clinical data for a cohort of SARS-CoV-2 (severe acute respiratory syndrome coronavirus 2)\u2013positive patients (n\u2009=\u20092256). Cross-validation was used to select the best-performing models per the areas under the receiver-operating characteristic and precision-recall curves. Models were validated using a held-out cohort (n\u2009=\u2009855). We measured each model\u2019s calibration and evaluated feature importances to interpret model output.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>The predictive performance for our selected models on the held-out cohort was as follows: area under the receiver-operating characteristic curve\u2014MV 0.743 (95% CI, 0.682-0.812), RRT 0.847 (95% CI, 0.772-0.936), readmission 0.871 (95% CI, 0.830-0.917); area under the precision-recall curve\u2014MV 0.137 (95% CI, 0.047-0.175), RRT 0.325 (95% CI, 0.117-0.497), readmission 0.504 (95% CI, 0.388-0.604). Predictions were well calibrated, and the most important features within each model were consistent with clinical intuition.<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussion<\/jats:title><jats:p>Our models produce performant, well-calibrated, and interpretable predictions for COVID-19 patients at risk for the target outcomes. They demonstrate the potential to accurately estimate outcome prognosis in resource-constrained care sites managing COVID-19 patients.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>We develop and validate prognostic models targeting MV, RRT, and readmission for hospitalized COVID-19 patients which produce accurate, interpretable predictions. Additional external validation studies are needed to further verify the generalizability of our results.<\/jats:p><\/jats:sec>","DOI":"10.1093\/jamia\/ocab029","type":"journal-article","created":{"date-parts":[[2021,2,5]],"date-time":"2021-02-05T20:29:35Z","timestamp":1612556975000},"page":"1480-1488","source":"Crossref","is-referenced-by-count":20,"title":["Development and validation of prediction models for mechanical ventilation, renal replacement therapy, and readmission in COVID-19 patients"],"prefix":"10.1093","volume":"28","author":[{"given":"Victor Alfonso","family":"Rodriguez","sequence":"first","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University, New York, New York, USA"}]},{"given":"Shreyas","family":"Bhave","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University, New York, New York, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5281-4143","authenticated-orcid":false,"given":"Ruijun","family":"Chen","sequence":"additional","affiliation":[]},{"given":"Chao","family":"Pang","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University, New York, New York, USA"},{"name":"Department of Translational Data Science and Informatics, Geisinger, Danville, Pennsylvania, USA"}]},{"given":"George","family":"Hripcsak","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University, New York, New York, USA"}]},{"given":"Soumitra","family":"Sengupta","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University, New York, New York, USA"}]},{"given":"Noemie","family":"Elhadad","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University, New York, New York, USA"}]},{"given":"Robert","family":"Green","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, Columbia University Irving Medical Center, New York, New York, USA"}]},{"given":"Jason","family":"Adelman","sequence":"additional","affiliation":[{"name":"Division of General Medicine, Department of Medicine, Columbia University Irving Medical Center, New York, New York, USA"}]},{"given":"Katherine Schlosser","family":"Metitiri","sequence":"additional","affiliation":[{"name":"Department of Pediatrics, Columbia University Irving Medical Center, New York, New York, USA"}]},{"given":"Pierre","family":"Elias","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University, New York, New York, USA"}]},{"given":"Holden","family":"Groves","sequence":"additional","affiliation":[{"name":"Department of Anesthesiology, Columbia University Irving Medical Center, New York, New York, USA"}]},{"given":"Sumit","family":"Mohan","sequence":"additional","affiliation":[{"name":"Division of Nephrology, Department of Medicine, Columbia University Irving Medical Center, New York, New York, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9066-9431","authenticated-orcid":false,"given":"Karthik","family":"Natarajan","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University, New York, New York, USA"}]},{"given":"Adler","family":"Perotte","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University, New York, New York, USA"}]}],"member":"286","published-online":{"date-parts":[[2021,3,11]]},"reference":[{"key":"2021071421221411900_ocab029-B1"},{"key":"2021071421221411900_ocab029-B2"},{"issue":"6","key":"2021071421221411900_ocab029-B3","doi-asserted-by":"crossref","first-page":"880","DOI":"10.2215\/CJN.05180420","article-title":"Impending shortages of kidney replacement therapy for covid-19 patients","volume":"15","author":"Goldfarb","year":"2020","journal-title":"Clin J Am Soc Nephrol"},{"issue":"18","key":"2021071421221411900_ocab029-B4","doi-asserted-by":"crossref","first-page":"e41","DOI":"10.1056\/NEJMp2006141","article-title":"Critical supply shortages - the need for ventilators and personal protective equipment during the Covid-19 pandemic","volume":"382","author":"Ranney","year":"2020","journal-title":"N Engl J Med"},{"key":"2021071421221411900_ocab029-B5","author":"Sanger-Katz"},{"key":"2021071421221411900_ocab029-B6","first-page":"369: m1996","article-title":"Characterization and clinical course of 1000 patients with coronavirus disease 2019 in New York: retrospective case series","author":"Argenziano","year":"2020","journal-title":"BMJ"},{"issue":"24","key":"2021071421221411900_ocab029-B7","doi-asserted-by":"crossref","first-page":"2372","DOI":"10.1056\/NEJMc2010419","article-title":"Clinical characteristics of COVID-19 in New York City","volume":"382","author":"Goyal","year":"2020","journal-title":"N Engl J Med"},{"issue":"20","key":"2021071421221411900_ocab029-B8","doi-asserted-by":"crossref","first-page":"2052","DOI":"10.1001\/jama.2020.6775","article-title":"Presenting characteristics, comorbidities, and outcomes among 5700 patients hospitalized with COVID-19 in the New York City Area","volume":"323","author":"Richardson","year":"2020","journal-title":"JAMA"},{"key":"2021071421221411900_ocab029-B9","first-page":"369: m1966","article-title":"Factors associated with hospital admission and critical illness among 5279 people with coronavirus disease 2019 in New York City: Prospective cohort study","author":"Petrilli","year":"2020","journal-title":"BMJ"},{"issue":"8","key":"2021071421221411900_ocab029-B10","doi-asserted-by":"crossref","first-page":"1149","DOI":"10.1016\/j.ekir.2020.06.013","article-title":"Outcomes for patients with COVID-19 and acute kidney injury: a systematic review and meta-analysis","volume":"5","author":"Robbins-Juarez","year":"2020","journal-title":"Kidney Int Rep"},{"key":"2021071421221411900_ocab029-B11","doi-asserted-by":"crossref","first-page":"1359","DOI":"10.2147\/IJGM.S275775","article-title":"Hospital readmissions of discharged patients with COVID-19","volume":"; 13","author":"Parra","year":"2020","journal-title":"Int J Gen Med"},{"issue":"13","key":"2021071421221411900_ocab029-B12","doi-asserted-by":"crossref","first-page":"1216","DOI":"10.1056\/NEJMp1606181","article-title":"Predicting the future-big data, machine learning, and clinical medicine","volume":"375","author":"Obermeyer","year":"2016","journal-title":"N Engl J Med"},{"issue":"1","key":"2021071421221411900_ocab029-B13","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1038\/s41591-019-0727-5","article-title":"Axes of a revolution: challenges and promises of big data in healthcare","volume":"26","author":"Shilo","year":"2020","journal-title":"Nat Med"},{"issue":"10239","key":"2021071421221411900_ocab029-B14","doi-asserted-by":"crossref","first-page":"1763","DOI":"10.1016\/S0140-6736(20)31189-2","article-title":"Epidemiology, clinical course, and outcomes of critically ill adults with COVID-19 in New York City: a prospective cohort study","volume":"395","author":"Cummings","year":"2020","journal-title":"Lancet"},{"issue":"1","key":"2021071421221411900_ocab029-B15","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1136\/amiajnl-2011-000376","article-title":"Validation of a common data model for active safety surveillance research","volume":"19","author":"Overhage","year":"2012","journal-title":"J Am Med Inform Assoc"},{"key":"2021071421221411900_ocab029-B16","first-page":"2825","article-title":"Scikit-learn: machine learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J Mach Learn Res"},{"issue":"3","key":"2021071421221411900_ocab029-B17","first-page":"1","article-title":"mice: Multivariate imputation by chained equations in R","volume":"45","author":"van Buuren","year":"2011","journal-title":"J Stat Softw"},{"key":"2021071421221411900_ocab029-B18","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1016\/j.jclinepi.2019.02.004","article-title":"A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models","volume":"110","author":"Christodoulou","year":"2019","journal-title":"J Clin Epidemiol"},{"issue":"7","key":"2021071421221411900_ocab029-B19","doi-asserted-by":"crossref","first-page":"1159","DOI":"10.1002\/sim.6782","article-title":"Review and evaluation of penalised regression methods for risk prediction in low-dimensional data with few events","volume":"35","author":"Pavlou","year":"2016","journal-title":"Stat Med"},{"issue":"6","key":"2021071421221411900_ocab029-B20","doi-asserted-by":"crossref","first-page":"1397","DOI":"10.1002\/jbmr.1577","article-title":"Assessing fracture risk using gradient boosting machine (GBM) models","volume":"27","author":"Atkinson","year":"2012","journal-title":"J Bone Miner Res"},{"issue":"7","key":"2021071421221411900_ocab029-B21","doi-asserted-by":"crossref","first-page":"e0132485","DOI":"10.1371\/journal.pone.0132485","article-title":"Prediction of outcome in acute lower gastrointestinal bleeding using gradient boosting","volume":"10","author":"Ayaru","year":"2015","journal-title":"PLoS One"},{"key":"2021071421221411900_ocab029-B22","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1016\/j.csda.2016.07.016","article-title":"Gradient boosting for high-dimensional prediction of rare events","volume":"113","author":"Blagus","year":"2017","journal-title":"Comput Stat Data Anal"},{"issue":"7","key":"2021071421221411900_ocab029-B23","doi-asserted-by":"crossref","first-page":"996","DOI":"10.1164\/rccm.202002-0347OC","article-title":"Machine learning classifier models can identify acute respiratory distress syndrome phenotypes using readily available clinical data","volume":"202","author":"Sinha","year":"2020","journal-title":"Am J Respir Crit Care Med"},{"issue":"1","key":"2021071421221411900_ocab029-B24","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","article-title":"Regression shrinkage and selection via the Lasso","volume":"58","author":"Tibshirani","year":"1996","journal-title":"J R Stat Soc Series B Stat Methodol"},{"issue":"2","key":"2021071421221411900_ocab029-B25","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1111\/j.1467-9868.2005.00503.x","article-title":"Regularization and variable selection via the elastic net","volume":"67","author":"Zou","year":"2005","journal-title":"J R Stat Soc Series B Stat Methodol"},{"key":"2021071421221411900_ocab029-B26","doi-asserted-by":"crossref","DOI":"10.1201\/b18401","volume-title":"Statistical Learning with Sparsity: The Lasso and Generalizations","author":"Hastie","year":"2015"},{"issue":"5","key":"2021071421221411900_ocab029-B27","doi-asserted-by":"crossref","first-page":"1189","DOI":"10.1214\/aos\/1013203451","article-title":"Greedy function approximation: a gradient boosting machine","volume":"29","author":"Friedman","year":"2001","journal-title":"Ann Stat"},{"issue":"1","key":"2021071421221411900_ocab029-B28","doi-asserted-by":"crossref","first-page":"1583","DOI":"10.1214\/15-EJS1035","article-title":"Computationally efficient confidence intervals for cross-validated area under the ROC curve estimates","volume":"9","author":"Ledell","year":"2015","journal-title":"Electron J Stat"},{"key":"2021071421221411900_ocab029-B29","first-page":"4765","article-title":"A unified approach to interpreting model predictions","author":"Lundberg","year":"2017","journal-title":"Adv Neural Inform Process Syst"},{"key":"2021071421221411900_ocab029-B30","article-title":"Development and prospective validation of a transparent deep learning algorithm for predicting need for mechanical ventilation","author":"Shashikumar","year":"2020","journal-title":"Chest"},{"issue":"1","key":"2021071421221411900_ocab029-B31","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1681\/ASN.2020050615","article-title":"AKI in hospitalized patients with COVID-19","volume":"32","author":"Chan","year":"2021","journal-title":"J Am Soc Nephrol"},{"issue":"10","key":"2021071421221411900_ocab029-B32","doi-asserted-by":"crossref","first-page":"2838","DOI":"10.1007\/s11606-020-06120-6","article-title":"Characterization of patients who return to hospital following discharge from hospitalization for COVID-19","volume":"35","author":"Somani","year":"2020","journal-title":"J Gen Intern Med"},{"issue":"9","key":"2021071421221411900_ocab029-B33","doi-asserted-by":"crossref","first-page":"657","DOI":"10.1093\/qjmed\/hcaa178","article-title":"The clinical features and outcomes of discharged coronavirus disease 2019 patients a prospective cohort study","volume":"113","author":"Wang","year":"2020","journal-title":"QJM"},{"issue":"2","key":"2021071421221411900_ocab029-B34","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1287\/moor.14.2.303","article-title":"Isotonic median regression: a linear programming approach","volume":"14","author":"Chakravarti","year":"1989","journal-title":"Math Oper Res"},{"issue":"5","key":"2021071421221411900_ocab029-B35","doi-asserted-by":"crossref","first-page":"375","DOI":"10.1111\/acem.13994","article-title":"Early self-proning in awake, non-intubated patients in the emergency department: a single ED\u2019s experience during the COVID-19 pandemic","volume":"27","author":"Caputo","year":"2020","journal-title":"Acad Emerg Med"},{"issue":"10","key":"2021071421221411900_ocab029-B36","doi-asserted-by":"crossref","first-page":"3202","DOI":"10.1093\/ndt\/gfr359","article-title":"Early nephrology consultation can have an impact on outcome of acute kidney injury patients","volume":"26","author":"Ponce","year":"2011","journal-title":"Nephrol Dial Transplant"},{"issue":"1\u20133","key":"2021071421221411900_ocab029-B37","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1159\/000452316","article-title":"Delayed nephrology consultation and high mortality on acute kidney injury: a meta-analysis","volume":"43","author":"Soares","year":"2017","journal-title":"Blood Purif"},{"issue":"3","key":"2021071421221411900_ocab029-B38","doi-asserted-by":"crossref","first-page":"581","DOI":"10.1093\/biomet\/63.3.581","article-title":"Inference and missing data","volume":"63","author":"Rubin","year":"1976","journal-title":"Biometrika"},{"key":"2021071421221411900_ocab029-B39","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1016\/j.jclinepi.2020.03.028","article-title":"Missing data should be handled differently for prediction than for description or causal explanation","volume":"125","author":"Sperrin","year":"2020","journal-title":"J Clin Epidemiol"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/7\/1480\/38983257\/ocab029.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/7\/1480\/38983257\/ocab029.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,23]],"date-time":"2024-08-23T17:58:39Z","timestamp":1724435919000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/28\/7\/1480\/6168488"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,11]]},"references-count":39,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2021,3,11]]},"published-print":{"date-parts":[[2021,7,14]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocab029","relation":{},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,7,1]]},"published":{"date-parts":[[2021,3,11]]}}}