{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,5]],"date-time":"2026-11-05T14:16:56Z","timestamp":1793888216573,"version":"4.8.0"},"reference-count":37,"publisher":"Ovid Technologies (Wolters Kluwer Health)","issue":"3","license":[{"start":{"date-parts":[[2024,3,1]],"date-time":"2024-03-01T00:00:00Z","timestamp":1709251200000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"content-domain":{"domain":["lww.com","ovid.com"],"crossmark-restriction":true},"short-container-title":[],"abstract":"<jats:sec>\n            <jats:title>OBJECTIVES:<\/jats:title>\n            <jats:p>To develop and validate machine learning (ML) models to predict high-flow nasal cannula (HFNC) failure in COVID-19, compare their performance to the respiratory rate-oxygenation (ROX) index, and evaluate model accuracy by self-reported race.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>DESIGN:<\/jats:title>\n            <jats:p>Retrospective cohort study.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>SETTING:<\/jats:title>\n            <jats:p>Four Emory University Hospitals in Atlanta, GA.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>PATIENTS:<\/jats:title>\n            <jats:p>Adult patients hospitalized with COVID-19 between March 2020 and April 2022 who received HFNC therapy within 24 hours of ICU admission were included.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>INTERVENTIONS:<\/jats:title>\n            <jats:p>None.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>MEASUREMENTS AND MAIN RESULTS:<\/jats:title>\n            <jats:p>Four types of supervised ML models were developed for predicting HFNC failure (defined as intubation or death within 7 d of HFNC initiation), using routine clinical variables from the first 24 hours of ICU admission. Models were trained on the first 60% (<jats:italic toggle=\"yes\">n<\/jats:italic> = 594) of admissions and validated on the latter 40% (<jats:italic toggle=\"yes\">n<\/jats:italic> = 390) of admissions to simulate prospective implementation. Among 984 patients included, 317 patients (32.2%) developed HFNC failure. eXtreme Gradient Boosting (XGB) model had the highest area under the receiver-operator characteristic curve (AUROC) for predicting HFNC failure (0.707), and was the only model with significantly better performance than the ROX index (AUROC 0.616). XGB model had significantly worse performance in Black patients compared with White patients (AUROC 0.663 vs. 0.808, <jats:italic toggle=\"yes\">p<\/jats:italic> = 0.02). Racial differences in the XGB model were reduced and no longer statistically significant when restricted to patients with nonmissing arterial blood gas data, and when XGB model was developed to predict mortality (rather than the composite outcome of failure, which could be influenced by biased clinical decisions for intubation).<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>CONCLUSIONS:<\/jats:title>\n            <jats:p>Our XGB model had better discrimination for predicting HFNC failure in COVID-19 than the ROX index, but had racial differences in accuracy of predictions. Further studies are needed to understand and mitigate potential sources of biases in clinical ML models and to improve their equitability.<\/jats:p>\n          <\/jats:sec>","DOI":"10.1097\/cce.0000000000001059","type":"journal-article","created":{"date-parts":[[2024,3,12]],"date-time":"2024-03-12T20:01:30Z","timestamp":1710273690000},"page":"e1059","update-policy":"https:\/\/doi.org\/10.1097\/lww.0000000000001000","source":"Crossref","is-referenced-by-count":4,"title":["Racial Differences in Accuracy of Predictive Models for High-Flow Nasal Cannula Failure in COVID-19"],"prefix":"10.1097","volume":"6","author":[{"given":"Philip","family":"Yang","sequence":"first","affiliation":[{"name":"Division of Pulmonary, Allergy, Critical Care, and Sleep Medicine, Emory University, Atlanta, GA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ismail A.","family":"Gregory","sequence":"additional","affiliation":[{"name":"Division of Pulmonary, Allergy, Critical Care, and Sleep Medicine, Emory University, Atlanta, GA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chad","family":"Robichaux","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, GA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andre L.","family":"Holder","sequence":"additional","affiliation":[{"name":"Division of Pulmonary, Allergy, Critical Care, and Sleep Medicine, Emory University, Atlanta, GA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Greg S.","family":"Martin","sequence":"additional","affiliation":[{"name":"Division of Pulmonary, Allergy, Critical Care, and Sleep Medicine, Emory University, Atlanta, GA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Annette M.","family":"Esper","sequence":"additional","affiliation":[{"name":"Division of Pulmonary, Allergy, Critical Care, and Sleep Medicine, Emory University, Atlanta, GA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rishikesan","family":"Kamaleswaran","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, GA."},{"name":"Department of Surgery, Duke University School of Medicine, Durham, NC."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Judy W.","family":"Gichoya","sequence":"additional","affiliation":[{"name":"Department of Radiology and Imaging Sciences, Emory University School of Medicine, Atlanta, GA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sivasubramanium V.","family":"Bhavani","sequence":"additional","affiliation":[{"name":"Division of Pulmonary, Allergy, Critical Care, and Sleep Medicine, Emory University, Atlanta, GA."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"276","published-online":{"date-parts":[[2024,3,12]]},"reference":[{"key":"R1-20240806","doi-asserted-by":"crossref","first-page":"227","DOI":"10.4187\/respcare.09056","article-title":"High-flow nasal cannula and COVID-19: A clinical review.","volume":"67","author":"Crimi","year":"2022","journal-title":"Respir Care"},{"key":"R2-20240806","doi-asserted-by":"crossref","first-page":"2161","DOI":"10.1001\/jama.2021.20714","article-title":"Effect of high-flow oxygen therapy vs conventional oxygen therapy on invasive mechanical ventilation and clinical recovery in patients with severe COVID-19: A randomized clinical trial.","volume":"326","author":"Ospina-Tascon","year":"2021","journal-title":"JAMA"},{"key":"R3-20240806","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1186\/s13054-021-03469-w","article-title":"High-flow nasal oxygen in patients with COVID-19-associated acute respiratory failure.","volume":"25","author":"Mellado-Artigas","year":"2021","journal-title":"Crit Care"},{"key":"R4-20240806","doi-asserted-by":"crossref","first-page":"1240","DOI":"10.1164\/rccm.202111-2509LE","article-title":"High-flow nasal cannula failure odds is largely independent of duration of use in COVID-19.","volume":"205","author":"Gershengorn","year":"2022","journal-title":"Am J Respir Crit Care Med"},{"key":"R5-20240806","doi-asserted-by":"crossref","first-page":"106474","DOI":"10.1016\/j.rmed.2021.106474","article-title":"Predictors of failure of high flow nasal cannula failure in acute hypoxemic respiratory failure due to COVID-19.","volume":"185","author":"Garner","year":"2021","journal-title":"Respir Med"},{"key":"R6-20240806","doi-asserted-by":"crossref","first-page":"909","DOI":"10.4187\/respcare.08631","article-title":"High-flow nasal cannula therapy in COVID-19: Using the ROX index to predict success.","volume":"66","author":"Chandel","year":"2021","journal-title":"Respir Care"},{"key":"R7-20240806","doi-asserted-by":"crossref","first-page":"607821","DOI":"10.3389\/fmed.2020.607821","article-title":"A novel risk-stratification models of the high-flow nasal cannula therapy in COVID-19 patients with hypoxemic respiratory failure.","volume":"7","author":"Xu","year":"2020","journal-title":"Front Med (Lausanne)"},{"key":"R8-20240806","doi-asserted-by":"crossref","first-page":"102","DOI":"10.1016\/j.jcrc.2021.08.012","article-title":"ROX index as a good predictor of high flow nasal cannula failure in COVID-19 patients with acute hypoxemic respiratory failure: A systematic review and meta-analysis.","volume":"66","author":"Prakash","year":"2021","journal-title":"J Crit Care"},{"key":"R9-20240806","doi-asserted-by":"crossref","first-page":"e22400","DOI":"10.2196\/22400","article-title":"A racially unbiased, machine learning approach to prediction of mortality: Algorithm development study.","volume":"6","author":"Allen","year":"2020","journal-title":"JMIR Public Health Surveill"},{"key":"R10-20240806","doi-asserted-by":"crossref","first-page":"273","DOI":"10.5847\/wjem.j.issn.1920-8642.2013.04.005","article-title":"Poor performance of the modified early warning score for predicting mortality in critically ill patients presenting to an emergency department.","volume":"4","author":"Ho","year":"2013","journal-title":"World J Emerg Med"},{"key":"R11-20240806","doi-asserted-by":"crossref","first-page":"178","DOI":"10.1164\/rccm.202012-4383OC","article-title":"Equitably allocating resources during crises: Racial differences in mortality prediction models.","volume":"204","author":"Ashana","year":"2021","journal-title":"Am J Respir Crit Care Med"},{"key":"R12-20240806","doi-asserted-by":"crossref","first-page":"1253","DOI":"10.1093\/jamia\/ocac062","article-title":"Development and validation of predictive models for COVID-19 outcomes in a safety-net hospital population.","volume":"29","author":"Hao","year":"2022","journal-title":"J Am Med Inform Assoc"},{"key":"R13-20240806","doi-asserted-by":"crossref","first-page":"306","DOI":"10.1001\/jama.2022.24683","article-title":"Predictive accuracy of stroke risk prediction models across black and white race, sex, and age groups.","volume":"329","author":"Hong","year":"2023","journal-title":"JAMA"},{"key":"R14-20240806","doi-asserted-by":"crossref","first-page":"e36388","DOI":"10.2196\/36388","article-title":"Evaluation and mitigation of racial bias in clinical machine learning models: Scoping review.","volume":"10","author":"Huang","year":"2022","journal-title":"JMIR Med Inform"},{"key":"R15-20240806","doi-asserted-by":"crossref","first-page":"447","DOI":"10.1126\/science.aax2342","article-title":"Dissecting racial bias in an algorithm used to manage the health of populations.","volume":"366","author":"Obermeyer","year":"2019","journal-title":"Science"},{"key":"R16-20240806","doi-asserted-by":"crossref","first-page":"1420","DOI":"10.1378\/chest.97.6.1420","article-title":"Reliability of pulse oximetry in titrating supplemental oxygen therapy in ventilator-dependent patients.","volume":"97","author":"Jubran","year":"1990","journal-title":"Chest"},{"key":"R17-20240806","doi-asserted-by":"crossref","first-page":"2477","DOI":"10.1056\/NEJMc2029240","article-title":"Racial bias in pulse oximetry measurement.","volume":"383","author":"Sjoding","year":"2020","journal-title":"N Engl J Med"},{"key":"R18-20240806","doi-asserted-by":"crossref","first-page":"e0758","DOI":"10.1097\/CCE.0000000000000758","article-title":"Pulse oximetry and race in critically ill adults.","volume":"4","author":"Seitz","year":"2022","journal-title":"Crit Care Explor"},{"key":"R19-20240806","doi-asserted-by":"crossref","first-page":"271","DOI":"10.1164\/rccm.202206-1092OC","article-title":"Do thresholds for invasive ventilation in hypoxemic respiratory failure exist? A cohort study.","volume":"207","author":"Yarnell","year":"2023","journal-title":"Am J Respir Crit Care Med"},{"key":"R20-20240806","doi-asserted-by":"crossref","first-page":"200","DOI":"10.1016\/j.jcrc.2016.05.022","article-title":"Predicting success of high-flow nasal cannula in pneumonia patients with hypoxemic respiratory failure: The utility of the ROX index.","volume":"35","author":"Roca","year":"2016","journal-title":"J Crit Care"},{"key":"R21-20240806","doi-asserted-by":"crossref","first-page":"1368","DOI":"10.1164\/rccm.201803-0589OC","article-title":"An index combining respiratory rate and oxygenation to predict outcome of nasal high-flow therapy.","volume":"199","author":"Roca","year":"2019","journal-title":"Am J Respir Crit Care Med"},{"key":"R22-20240806","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1186\/s12890-022-01914-2","article-title":"The ROX index as a predictor of high-flow nasal cannula outcome in pneumonia patients with acute hypoxemic respiratory failure: A systematic review and meta-analysis.","volume":"22","author":"Zhou","year":"2022","journal-title":"BMC Pulm Med"},{"key":"R23-20240806","doi-asserted-by":"crossref","first-page":"2094","DOI":"10.1007\/s00134-020-06199-9","article-title":"Monitoring of high-flow nasal cannula for SARS-CoV-2 severe pneumonia: Less is more, better look at respiratory rate.","volume":"46","author":"Blez","year":"2020","journal-title":"Intensive Care Med"},{"key":"R24-20240806","doi-asserted-by":"crossref","first-page":"100570","DOI":"10.1016\/j.eclinm.2020.100570","article-title":"The utility of high-flow nasal oxygen for severe COVID-19 pneumonia in a resource-constrained setting: A multi-centre prospective observational study.","volume":"28","author":"Calligaro","year":"2020","journal-title":"EClinicalMedicine"},{"key":"R25-20240806","doi-asserted-by":"crossref","first-page":"100828","DOI":"10.1016\/j.eclinm.2021.100828","article-title":"The ROX index has greater predictive validity than NEWS2 for deterioration in COVID-19.","volume":"35","author":"Prower","year":"2021","journal-title":"EClinicalMedicine"},{"key":"R26-20240806","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1186\/s13054-023-04370-4","article-title":"Update on the management of acute respiratory failure using non-invasive ventilation and pulse oximetry.","volume":"27","author":"Abe","year":"2023","journal-title":"Crit Care"},{"key":"R27-20240806","doi-asserted-by":"crossref","first-page":"874","DOI":"10.1056\/NEJMms2004740","article-title":"Eisenstein LG and Jones DS: Hidden in plain sight\u2014reconsidering the use of race correction in clinical algorithms.","volume":"383","author":"Vyas","year":"2020","journal-title":"N Engl J Med"},{"key":"R28-20240806","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1310\/sci2402-110","article-title":"Racial differences in data quality and completeness: Spinal cord injury model systems\u2019 experiences.","volume":"24","author":"Chen","year":"2018","journal-title":"Top Spinal Cord Inj Rehabil"},{"key":"R29-20240806","doi-asserted-by":"crossref","first-page":"707","DOI":"10.1177\/0898264306293264","article-title":"Comparison of data quality for reports and ratings of ambulatory care by African American and White Medicare managed care enrollees.","volume":"18","author":"Fongwa","year":"2006","journal-title":"J Aging Health"},{"key":"R30-20240806","doi-asserted-by":"crossref","first-page":"W1","DOI":"10.7326\/M14-0698","article-title":"Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): Explanation and elaboration.","volume":"162","author":"Moons","year":"2015","journal-title":"Ann Intern Med"},{"key":"R31-20240806","doi-asserted-by":"crossref","first-page":"623","DOI":"10.1097\/CCM.0000000000004246","article-title":"Development and reporting of prediction models: Guidance for authors from editors of respiratory, sleep, and critical care journals.","volume":"48","author":"Leisman","year":"2020","journal-title":"Crit Care Med"},{"key":"R32-20240806","doi-asserted-by":"crossref","first-page":"1003","DOI":"10.1177\/1536867X1801700414","article-title":"Assessing the calibration of dichotomous outcome models with the calibration belt.","volume":"17","author":"Nattino","year":"2017","journal-title":"Stata J"},{"key":"R33-20240806","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1186\/s12931-022-01951-9","article-title":"The value of ROX index in predicting the outcome of high flow nasal cannula: A systematic review and meta-analysis.","volume":"23","author":"Junhai","year":"2022","journal-title":"Respir Res"},{"key":"R34-20240806","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1186\/s40560-020-00458-z","article-title":"Early prediction of high flow nasal cannula therapy outcomes using a modified ROX index incorporating heart rate.","volume":"8","author":"Goh","year":"2020","journal-title":"J Intensive Care"},{"key":"R35-20240806","doi-asserted-by":"crossref","first-page":"2201426","DOI":"10.1183\/13993003.01426-2022","article-title":"Effects of intubation timing in patients with COVID-19 throughout the four waves of the pandemic: A matched analysis.","volume":"61","author":"Riera","year":"2023","journal-title":"Eur Respir J"},{"key":"R36-20240806","doi-asserted-by":"crossref","first-page":"513","DOI":"10.1007\/s10287-022-00425-z","article-title":"Accuracy and fairness trade-offs in machine learning: A stochastic multi-objective approach.","volume":"19","author":"Liu","year":"2022","journal-title":"Comput Manage Sci"},{"key":"R38-20240806","first-page":"e35037","article-title":"Racial equity in healthcare machine learning: Illustrating bias in models with minimal bias mitigation.","volume":"15","author":"Barton","year":"2023","journal-title":"Cureus"}],"container-title":["Critical Care Explorations"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.lww.com\/10.1097\/CCE.0000000000001059","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,6]],"date-time":"2024-08-06T23:03:25Z","timestamp":1722985405000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.lww.com\/10.1097\/CCE.0000000000001059"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3]]},"references-count":37,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2024]]}},"URL":"https:\/\/doi.org\/10.1097\/cce.0000000000001059","relation":{},"ISSN":["2639-8028"],"issn-type":[{"value":"2639-8028","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3]]}}}