{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,11]],"date-time":"2026-03-11T12:27:07Z","timestamp":1773232027072,"version":"3.50.1"},"reference-count":27,"publisher":"Georg Thieme Verlag KG","issue":"03","license":[{"start":{"date-parts":[[2022,7,27]],"date-time":"2022-07-27T00:00:00Z","timestamp":1658880000000},"content-version":"vor","delay-in-days":87,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"funder":[{"name":"NIH","award":["3UL1TR001445-05"],"award-info":[{"award-number":["3UL1TR001445-05"]}]},{"name":"National Science Foundation","award":["1928614"],"award-info":[{"award-number":["1928614"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Appl Clin Inform"],"published-print":{"date-parts":[[2022,5]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>\n          Background\u2003We previously developed and validated a predictive model to help clinicians identify hospitalized adults with coronavirus disease 2019 (COVID-19) who may be ready for discharge given their low risk of adverse events. Whether this algorithm can prompt more timely discharge for stable patients in practice is unknown.<\/jats:p><jats:p>\n          Objectives\u2003The aim of the study is to estimate the effect of displaying risk scores on length of stay (LOS).<\/jats:p><jats:p>\n          Methods\u2003We integrated model output into the electronic health record (EHR) at four hospitals in one health system by displaying a green\/orange\/red score indicating low\/moderate\/high-risk in a patient list column and a larger COVID-19 summary report visible for each patient. Display of the score was pseudo-randomized 1:1 into intervention and control arms using a patient identifier passed to the model execution code. Intervention effect was assessed by comparing LOS between intervention and control groups. Adverse safety outcomes of death, hospice, and re-presentation were tested separately and as a composite indicator. We tracked adoption and sustained use through daily counts of score displays.<\/jats:p><jats:p>\n          Results\u2003Enrolling 1,010 patients from May 15, 2020 to December 7, 2020, the trial found no detectable difference in LOS. The intervention had no impact on safety indicators of death, hospice or re-presentation after discharge. The scores were displayed consistently throughout the study period but the study lacks a causally linked process measure of provider actions based on the score. Secondary analysis revealed complex dynamics in LOS temporally, by primary symptom, and hospital location.<\/jats:p><jats:p>\n          Conclusion\u2003An AI-based COVID-19 risk score displayed passively to clinicians during routine care of hospitalized adults with COVID-19 was safe but had no detectable impact on LOS. Health technology challenges such as insufficient adoption, nonuniform use, and provider trust compounded with temporal factors of the COVID-19 pandemic may have contributed to the null result.<\/jats:p><jats:p>\n          Trial registration\u2003ClinicalTrials.gov identifier: NCT04570488.<\/jats:p>","DOI":"10.1055\/s-0042-1750416","type":"journal-article","created":{"date-parts":[[2022,7,28]],"date-time":"2022-07-28T02:40:24Z","timestamp":1658976024000},"page":"632-640","source":"Crossref","is-referenced-by-count":6,"title":["Evaluating the Effect of a COVID-19 Predictive Model to Facilitate Discharge: A Randomized Controlled Trial"],"prefix":"10.1055","volume":"13","author":[{"given":"Vincent J.","family":"Major","sequence":"additional","affiliation":[{"name":"Center for Healthcare Innovation & Delivery Science, Department of Population Health, NYU Grossman School of Medicine, New York, New York, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Simon A.","family":"Jones","sequence":"additional","affiliation":[{"name":"Center for Healthcare Innovation & Delivery Science, Department of Population Health, NYU Grossman School of Medicine, New York, New York, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Narges","family":"Razavian","sequence":"additional","affiliation":[{"name":"Center for Healthcare Innovation & Delivery Science, Department of Population Health, NYU Grossman School of Medicine, New York, New York, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ashley","family":"Bagheri","sequence":"additional","affiliation":[{"name":"Center for Healthcare Innovation & Delivery Science, Department of Population Health, NYU Grossman School of Medicine, New York, New York, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Felicia","family":"Mendoza","sequence":"additional","affiliation":[{"name":"Center for Healthcare Innovation & Delivery Science, Department of Population Health, NYU Grossman School of Medicine, New York, New York, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jay","family":"Stadelman","sequence":"additional","affiliation":[{"name":"Center for Healthcare Innovation & Delivery Science, Department of Population Health, NYU Grossman School of Medicine, New York, New York, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Leora I.","family":"Horwitz","sequence":"additional","affiliation":[{"name":"Center for Healthcare Innovation & Delivery Science, Department of Population Health, NYU Grossman School of Medicine, New York, New York, United States"},{"name":"Department of Medicine, NYU Grossman School of Medicine, New York, New York, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jonathan","family":"Austrian","sequence":"additional","affiliation":[{"name":"Department of Medicine, NYU Grossman School of Medicine, New York, New York, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yindalon","family":"Aphinyanaphongs","sequence":"additional","affiliation":[{"name":"Center for Healthcare Innovation & Delivery Science, Department of Population Health, NYU Grossman School of Medicine, New York, New York, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"194","published-online":{"date-parts":[[2022,7,27]]},"reference":[{"key":"ref1","doi-asserted-by":"crossref","first-page":"102763","DOI":"10.1016\/j.ebiom.2020.102763","article-title":"Longitudinal characteristics of lymphocyte responses and cytokine profiles in the peripheral blood of SARS-CoV-2 infected patients","volume":"55","author":"J Liu","year":"2020","journal-title":"EBioMedicine"},{"issue":"18","key":"ref2","doi-asserted-by":"crossref","first-page":"1708","DOI":"10.1056\/NEJMoa2002032","article-title":"Clinical characteristics of coronavirus disease 2019 in China","volume":"382","author":"W J Guan","year":"2020","journal-title":"N Engl J Med"},{"issue":"01","key":"ref3","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1016\/j.chest.2020.04.010","article-title":"Risk factors of fatal outcome in hospitalized subjects with coronavirus disease 2019 from a nationwide analysis in China","volume":"158","author":"R Chen","year":"2020","journal-title":"Chest"},{"key":"ref4","doi-asserted-by":"crossref","first-page":"497","DOI":"10.1016\/S0140-6736(20)30183-5","article-title":"Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China","volume":"395","author":"C Huang","year":"2020","journal-title":"Lancet"},{"issue":"02","key":"ref5","doi-asserted-by":"crossref","first-page":"282","DOI":"10.1016\/j.jinf.2020.05.064","article-title":"A clinical risk score to identify patients with COVID-19 at high risk of critical care admission or death: an observational cohort study","volume":"81","author":"J B Galloway","year":"2020","journal-title":"J Infect"},{"issue":"01","key":"ref6","doi-asserted-by":"crossref","first-page":"2001234","DOI":"10.1183\/13993003.01234-2020","article-title":"A predictive model for disease progression in non-severely ill patients with coronavirus disease 2019","volume":"56","author":"M Ji","year":"2020","journal-title":"Eur Respir J"},{"issue":"04","key":"ref7","doi-asserted-by":"crossref","first-page":"577","DOI":"10.1007\/s15010-020-01446-z","article-title":"A simple algorithm helps early identification of SARS-CoV-2 infection patients with severe progression tendency","volume":"48","author":"Q Li","year":"2020","journal-title":"Infection"},{"issue":"08","key":"ref8","doi-asserted-by":"crossref","first-page":"1081","DOI":"10.1001\/jamainternmed.2020.2033","article-title":"Development and validation of a clinical risk score to predict the occurrence of critical illness in hospitalized patients with COVID-19","volume":"180","author":"W Liang","year":"2020","journal-title":"JAMA Intern Med"},{"issue":"05","key":"ref9","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1038\/s42256-020-0180-7","article-title":"An interpretable mortality prediction model for COVID-19 patients","volume":"2","author":"L Yan","year":"2020","journal-title":"Nat Mach Intell"},{"issue":"07","key":"ref10","doi-asserted-by":"crossref","first-page":"1129","DOI":"10.1513\/AnnalsATS.202006-698OC","article-title":"Evaluating a widely implemented proprietary deterioration index model among hospitalized patients with COVID-19","volume":"18","author":"K Singh","year":"2021","journal-title":"Ann Am Thorac Soc"},{"key":"ref11","doi-asserted-by":"crossref","first-page":"m1328","DOI":"10.1136\/bmj.m1328","article-title":"Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal","volume":"369","author":"L Wynants","year":"2020","journal-title":"BMJ"},{"issue":"01","key":"ref12","doi-asserted-by":"crossref","first-page":"e100267","DOI":"10.1136\/bmjhci-2020-100267","article-title":"Validation of parsimonious prognostic models for patients infected with COVID-19","volume":"28","author":"K Harish","year":"2021","journal-title":"BMJ Health Care Inform"},{"key":"ref13","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1038\/s41746-020-00343-x","article-title":"A validated, real-time prediction model for favorable outcomes in hospitalized COVID-19 patients","volume":"3","author":"N Razavian","year":"2020","journal-title":"NPJ Digit Med"},{"issue":"09","key":"ref14","doi-asserted-by":"crossref","first-page":"1337","DOI":"10.1038\/s41591-019-0548-6","article-title":"Do no harm: a roadmap for responsible machine learning for health care","volume":"25","author":"J Wiens","year":"2019","journal-title":"Nat Med"},{"issue":"08","key":"ref16","doi-asserted-by":"crossref","first-page":"637","DOI":"10.1001\/jama.1996.03540080059030","article-title":"Improving the quality of reporting of randomized controlled trials. The CONSORT statement","volume":"276","author":"C Begg","year":"1996","journal-title":"JAMA"},{"issue":"06","key":"ref17","doi-asserted-by":"crossref","first-page":"466","DOI":"10.4037\/ajcc2015455","article-title":"Squire 2.0 (Standards for Quality Improvement Reporting Excellence): revised publication guidelines from a detailed consensus process","volume":"24","author":"G Ogrinc","year":"2015","journal-title":"Am J Crit Care"},{"key":"ref18","doi-asserted-by":"crossref","first-page":"i6795","DOI":"10.1136\/bmj.i6795","article-title":"Standards for Reporting Implementation Studies (StaRI) Statement","volume":"356","author":"H Pinnock","year":"2017","journal-title":"BMJ"},{"issue":"S 11","key":"ref19","doi-asserted-by":"crossref","first-page":"S73","DOI":"10.7326\/M19-0874","article-title":"Context and approach in reporting evaluations of electronic health record-based implementation projects","volume":"172","author":"R B Haynes","year":"2020","journal-title":"Ann Intern Med"},{"issue":"01","key":"ref21","doi-asserted-by":"crossref","first-page":"114","DOI":"10.1037\/xge0000033","article-title":"Algorithm aversion: people erroneously avoid algorithms after seeing them err","volume":"144","author":"B J Dietvorst","year":"2015","journal-title":"J Exp Psychol Gen"},{"issue":"05","key":"ref22","doi-asserted-by":"crossref","first-page":"575","DOI":"10.1197\/jamia.M2518","article-title":"Methodologic issues in health informatics trials: the complexities of complex interventions","volume":"15","author":"I Shcherbatykh","year":"2008","journal-title":"J Am Med Inform Assoc"},{"issue":"12","key":"ref23","doi-asserted-by":"crossref","first-page":"1175","DOI":"10.1056\/NEJMsb1900856","article-title":"Creating a learning health system through rapid-cycle, randomized testing","volume":"381","author":"L I Horwitz","year":"2019","journal-title":"N Engl J Med"},{"issue":"02","key":"ref24","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1016\/0021-9681(57)90048-6","article-title":"Reasons for prolonged hospital stay; a study of need for hospital care","volume":"6","author":"L S Rosenfeld","year":"1957","journal-title":"J Chronic Dis"},{"issue":"05","key":"ref25","doi-asserted-by":"crossref","first-page":"461","DOI":"10.1001\/archsurg.142.5.461","article-title":"Length of stay: an appropriate quality measure?","volume":"142","author":"K J Brasel","year":"2007","journal-title":"Arch Surg"},{"issue":"01","key":"ref26","doi-asserted-by":"crossref","first-page":"33","DOI":"10.7326\/M20-3905","article-title":"Patient trajectories among persons hospitalized for COVID-19: a cohort study","volume":"174","author":"B T Garibaldi","year":"2021","journal-title":"Ann Intern Med"},{"issue":"02","key":"ref27","doi-asserted-by":"crossref","first-page":"446","DOI":"10.1016\/j.mayocp.2020.11.024","article-title":"COVID-19: understanding inter-individual variability and implications for precision medicine","volume":"96","author":"N L Pereira","year":"2021","journal-title":"Mayo Clin Proc"},{"issue":"05","key":"ref28","doi-asserted-by":"crossref","first-page":"2000547","DOI":"10.1183\/13993003.00547-2020","article-title":"Comorbidity and its impact on 1590 patients with COVID-19 in China: a nationwide analysis","volume":"55","author":"W J Guan","year":"2020","journal-title":"Eur Respir J"},{"issue":"08","key":"ref29","doi-asserted-by":"crossref","first-page":"1069","DOI":"10.1007\/s42399-020-00363-4","article-title":"Comorbidity and its impact on patients with COVID-19","volume":"2","author":"A Sanyaolu","year":"2020","journal-title":"SN Compr Clin Med"}],"container-title":["Applied Clinical Informatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.thieme-connect.de\/products\/ejournals\/pdf\/10.1055\/s-0042-1750416.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,7,28]],"date-time":"2022-07-28T02:44:04Z","timestamp":1658976244000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.thieme-connect.de\/DOI\/DOI?10.1055\/s-0042-1750416"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5]]},"references-count":27,"journal-issue":{"issue":"03","published-online":{"date-parts":[[2022,5,25]]},"published-print":{"date-parts":[[2022,5]]}},"URL":"https:\/\/doi.org\/10.1055\/s-0042-1750416","archive":["Portico","CLOCKSS"],"relation":{},"ISSN":["1869-0327"],"issn-type":[{"value":"1869-0327","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,5]]}}}