{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,27]],"date-time":"2026-02-27T20:45:24Z","timestamp":1772225124098,"version":"3.50.1"},"reference-count":32,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,12,1]],"date-time":"2021-12-01T00:00:00Z","timestamp":1638316800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,12,17]],"date-time":"2021-12-17T00:00:00Z","timestamp":1639699200000},"content-version":"vor","delay-in-days":16,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Background<\/jats:title>\n                <jats:p>Patients with severe Coronavirus disease 19 (COVID-19) typically require supplemental oxygen as an essential treatment. We developed a machine learning algorithm, based on deep Reinforcement Learning (RL), for continuous management of oxygen flow rate for critically ill patients under intensive care, which can identify the optimal personalized oxygen flow rate with strong potentials to reduce mortality rate relative to the current clinical practice.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>We modeled the oxygen flow trajectory of COVID-19 patients and their health outcomes as a Markov decision process. Based on individual patient characteristics and health status, an optimal oxygen control policy is learned by using deep deterministic policy gradient (DDPG) and real-time recommends the oxygen flow rate to reduce the mortality rate. We assessed the performance of proposed methods through cross validation by using a retrospective cohort of 1372 critically ill patients with COVID-19 from New York University Langone Health ambulatory care with electronic health records from April 2020 to January 2021.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>The mean mortality rate under the RL algorithm is lower than the standard of care by 2.57% (95% CI: 2.08\u20133.06) reduction (<jats:italic>P<\/jats:italic>\u2009&lt;\u20090.001) from 7.94% under the standard of care to 5.37% under our proposed algorithm. The averaged recommended oxygen flow rate is 1.28\u00a0L\/min (95% CI: 1.14\u20131.42) lower than the rate delivered to patients. Thus, the RL algorithm could potentially lead to better intensive care treatment that can reduce the mortality rate, while saving the oxygen scarce resources. It can reduce the oxygen shortage issue and improve public health during the COVID-19 pandemic.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusions<\/jats:title>\n                <jats:p>A personalized reinforcement learning oxygen flow control algorithm for COVID-19 patients under intensive care showed a substantial reduction in 7-day mortality rate as compared to the standard of care. In the overall cross validation cohort independent of the training data, mortality was lowest in patients for whom intensivists\u2019 actual flow rate matched the RL decisions.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1186\/s12911-021-01712-6","type":"journal-article","created":{"date-parts":[[2021,12,17]],"date-time":"2021-12-17T11:14:19Z","timestamp":1639739659000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":21,"title":["Reinforcement learning assisted oxygen therapy for COVID-19 patients under intensive care"],"prefix":"10.1186","volume":"21","author":[{"given":"Hua","family":"Zheng","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jiahao","family":"Zhu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wei","family":"Xie","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Judy","family":"Zhong","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,12,17]]},"reference":[{"issue":"1","key":"1712_CR1","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s13054-020-03240-7","volume":"24","author":"SJ Tzotzos","year":"2020","unstructured":"Tzotzos SJ, Fischer B, Fischer H, Zeitlinger M. Incidence of ARDS and outcomes in hospitalized patients with COVID-19: a global literature survey. Crit Care. 2020;24(1):1\u20134.","journal-title":"Crit Care"},{"issue":"2","key":"1712_CR2","doi-asserted-by":"publisher","first-page":"95","DOI":"10.1002\/emp2.12071","volume":"1","author":"JS Whittle","year":"2020","unstructured":"Whittle JS, Pavlov I, Sacchetti AD, Atwood C, Rosenberg MS. Respiratory support for adult patients with COVID-19. J Am Coll Emerg Physicians Open. 2020;1(2):95\u2013101.","journal-title":"J Am Coll Emerg Physicians Open"},{"key":"1712_CR3","doi-asserted-by":"publisher","first-page":"109581","DOI":"10.1109\/ACCESS.2020.3001973","volume":"8","author":"MB Jamshidi","year":"2020","unstructured":"Jamshidi MB, Lalbakhsh A, Talla J, Peroutka Z, Hadjilooei F, Lalbakhsh P, Jamshidi M, Spada L, Mirmozafari M, Dehghani M, et al. Artificial intelligence and COVID-19: deep learning approaches for diagnosis and treatment. IEEE Access. 2020;8:109581\u201395.","journal-title":"IEEE Access"},{"key":"1712_CR4","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1007\/978-3-030-67716-9_2","volume":"348","author":"MB Jamshidi","year":"2021","unstructured":"Jamshidi MB, Lalbakhsh A, Talla J, Peroutka Z, Roshani S, Matousek V, Roshani S, Mirmozafari M, Malek Z, La-Spada L. Deep Learning Techniques and COVID-19 Drug Discovery: Fundamentals, State-of-the-Art and Future Directions. Emerg Technol Dur Era COVID-19 Pandemic. 2021;348:9\u201331.","journal-title":"Emerg Technol Dur Era COVID-19 Pandemic"},{"issue":"22","key":"1712_CR5","doi-asserted-by":"publisher","first-page":"2329","DOI":"10.1001\/jama.2020.6825","volume":"323","author":"JJ Marini","year":"2020","unstructured":"Marini JJ, Gattinoni L. Management of COVID-19 respiratory distress. JAMA. 2020;323(22):2329\u201330.","journal-title":"JAMA"},{"key":"1712_CR6","doi-asserted-by":"publisher","first-page":"n436","DOI":"10.1136\/bmj.n436","volume":"372","author":"AH Attaway","year":"2021","unstructured":"Attaway AH, Scheraga RG, Bhimraj A, Biehl M, Hatipo\u011flu U. Severe covid-19 pneumonia: pathogenesis and clinical management. BMJ. 2021;372:n436.","journal-title":"BMJ"},{"issue":"7","key":"1712_CR7","doi-asserted-by":"publisher","first-page":"e0235458","DOI":"10.1371\/journal.pone.0235458","volume":"15","author":"B Zhang","year":"2020","unstructured":"Zhang B, Zhou X, Qiu Y, Song Y, Feng F, Feng J, Song Q, Jia Q, Wang J. Clinical characteristics of 82 cases of death from COVID-19. PLoS ONE. 2020;15(7):e0235458.","journal-title":"PLoS ONE"},{"key":"1712_CR8","unstructured":"Lillicrap TP, Hunt JJ, Pritzel A, Heess N, Erez T, Tassa Y, Silver D, Wierstra D: Continuous control with deep reinforcement learning. arXiv:1509.02971; 2015."},{"key":"1712_CR9","doi-asserted-by":"crossref","unstructured":"Zheng H, Ryzhov IO, Xie W, Zhong J: Personalized multimorbidity management for patients with type 2 diabetes using reinforcement learning of electronic health records. Drugs 2021; 1\u201312.","DOI":"10.1007\/s40265-020-01435-4"},{"key":"1712_CR10","doi-asserted-by":"crossref","unstructured":"Ernst D, Stan G, Goncalves J, Wehenkel L: Clinical data based optimal STI strategies for HIV: a reinforcement learning approach. In: Proceedings of the 45th IEEE Conference on Decision and Control: 13\u201315 Dec. 2006; 2006. P. 667\u201372.","DOI":"10.1109\/CDC.2006.377527"},{"issue":"4","key":"1712_CR11","doi-asserted-by":"publisher","first-page":"1422","DOI":"10.1111\/j.1541-0420.2011.01572.x","volume":"67","author":"Y Zhao","year":"2011","unstructured":"Zhao Y, Zeng D, Socinski MA, Kosorok MR. Reinforcement learning strategies for clinical trials in nonsmall cell lung cancer. Biometrics. 2011;67(4):1422\u201333.","journal-title":"Biometrics"},{"issue":"1","key":"1712_CR12","doi-asserted-by":"publisher","first-page":"47","DOI":"10.1016\/j.artmed.2014.07.004","volume":"62","author":"P Escandell-Montero","year":"2014","unstructured":"Escandell-Montero P, Chermisi M, Mart\u00ednez-Mart\u00ednez JM, G\u00f3mez-Sanchis J, Barbieri C, Soria-Olivas E, Mari F, Vila-Franc\u00e9s J, Stopper A, Gatti E, et al. Optimization of anemia treatment in hemodialysis patients via reinforcement learning. Artif Intell Med. 2014;62(1):47\u201360.","journal-title":"Artif Intell Med"},{"issue":"7","key":"1712_CR13","doi-asserted-by":"publisher","first-page":"e18477","DOI":"10.2196\/18477","volume":"22","author":"S Liu","year":"2020","unstructured":"Liu S, See KC, Ngiam KY, Celi LA, Sun X, Feng M. Reinforcement learning for clinical decision support in critical care: comprehensive review. J Med Internet Res. 2020;22(7):e18477.","journal-title":"J Med Internet Res"},{"issue":"11","key":"1712_CR14","doi-asserted-by":"publisher","first-page":"1716","DOI":"10.1038\/s41591-018-0213-5","volume":"24","author":"M Komorowski","year":"2018","unstructured":"Komorowski M, Celi LA, Badawi O, Gordon AC, Faisal AA. The artificial intelligence clinician learns optimal treatment strategies for sepsis in intensive care. Nat Med. 2018;24(11):1716\u201320.","journal-title":"Nat Med"},{"key":"1712_CR15","unstructured":"Prasad N, Cheng L-F, Chivers C, Draugelis M, Engelhardt BE: A reinforcement learning approach to weaning of mechanical ventilation in intensive care units. 2017."},{"key":"1712_CR16","volume-title":"Reinforcement Learning: An Introduction","author":"RS Sutton","year":"2018","unstructured":"Sutton RS, Barto AG. Reinforcement Learning: An Introduction. 2nd ed. The MIT Press; 2018.","edition":"2"},{"key":"1712_CR17","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1016\/j.cca.2020.03.022","volume":"506","author":"G Lippi","year":"2020","unstructured":"Lippi G, Plebani M, Henry BM. Thrombocytopenia is associated with severe coronavirus disease 2019 (COVID-19) infections: a meta-analysis. Clin Chim Acta. 2020;506:145\u20138.","journal-title":"Clin Chim Acta"},{"key":"1712_CR18","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1016\/j.ajem.2020.12.003","volume":"40","author":"EV Moradi","year":"2021","unstructured":"Moradi EV, Teimouri A, Rezaee R, Morovatdar N, Foroughian M, Layegh P, Kakhki BR, Koupaei SRA, Ghorani V. Increased age, neutrophil-to-lymphocyte ratio (NLR) and white blood cells count are associated with higher COVID-19 mortality. Am J Emerg Med. 2021;40:11\u20134.","journal-title":"Am J Emerg Med"},{"issue":"12","key":"1712_CR19","doi-asserted-by":"publisher","first-page":"BSR20202690","DOI":"10.1042\/BSR20202690","volume":"40","author":"X Zhou","year":"2020","unstructured":"Zhou X, Chen D, Wang L, Zhao Y, Wei L, Chen Z, Yang B. Low serum calcium: a new, important indicator of COVID-19 patients from mild\/moderate to severe\/critical. Biosci Rep. 2020;40(12):BSR20202690.","journal-title":"Biosci Rep"},{"issue":"9","key":"1712_CR20","doi-asserted-by":"publisher","first-page":"593","DOI":"10.21037\/atm-20-3391","volume":"8","author":"C Wang","year":"2020","unstructured":"Wang C, Deng R, Gou L, Fu Z, Zhang X, Shao F, Wang G, Fu W, Xiao J, Ding X. Preliminary study to identify severe from moderate cases of COVID-19 using combined hematology parameters. Ann Transl Med. 2020;8(9):593.","journal-title":"Ann Transl Med"},{"key":"1712_CR21","doi-asserted-by":"crossref","unstructured":"Cheng Y, Luo R, Wang K, Zhang M, Wang Z, Dong L, Li J, Yao Y, Ge S, Xu G. Kidney impairment is associated with in-hospital death of COVID-19 patients. MedRxiv. 2020.","DOI":"10.1101\/2020.02.18.20023242"},{"issue":"6","key":"1712_CR22","doi-asserted-by":"publisher","first-page":"1068","DOI":"10.1016\/j.cmet.2020.04.021","volume":"31","author":"L Zhu","year":"2020","unstructured":"Zhu L, She Z-G, Cheng X, Qin J-J, Zhang X-J, Cai J, Lei F, Wang H, Xie J, Wang W. Association of blood glucose control and outcomes in patients with COVID-19 and pre-existing type 2 diabetes. Cell Metab. 2020;31(6):1068-1077.e1063.","journal-title":"Cell Metab"},{"issue":"6","key":"1712_CR23","doi-asserted-by":"publisher","first-page":"e2011122","DOI":"10.1001\/jamanetworkopen.2020.11122","volume":"3","author":"D Chen","year":"2020","unstructured":"Chen D, Li X, Song Q, Hu C, Su F, Dai J, Ye Y, Huang J, Zhang X. Assessment of hypokalemia and clinical characteristics in patients with coronavirus disease 2019 in Wenzhou, China. JAMA Netw Open. 2020;3(6):e2011122\u2013e2011122.","journal-title":"JAMA Netw Open"},{"issue":"2","key":"1712_CR24","doi-asserted-by":"publisher","first-page":"410","DOI":"10.1378\/chest.07-0617","volume":"132","author":"TW Rice","year":"2007","unstructured":"Rice TW, Wheeler AP, Bernard GR, Hayden DL, Schoenfeld DA, Ware LB, Network A. Health NIo: comparison of the SpO2\/FIO2 ratio and the PaO2\/FIO2 ratio in patients with acute lung injury or ARDS. Chest. 2007;132(2):410\u20137.","journal-title":"Chest"},{"issue":"6","key":"1712_CR25","doi-asserted-by":"publisher","first-page":"1477","DOI":"10.1378\/chest.15-0169","volume":"148","author":"W Chen","year":"2015","unstructured":"Chen W, Janz DR, Shaver CM, Bernard GR, Bastarache JA, Ware LB. Clinical characteristics and outcomes are similar in ARDS diagnosed by oxygen saturation\/Fio2 ratio compared with Pao2\/Fio2 ratio. Chest. 2015;148(6):1477\u201383.","journal-title":"Chest"},{"issue":"10239","key":"1712_CR26","doi-asserted-by":"publisher","first-page":"1763","DOI":"10.1016\/S0140-6736(20)31189-2","volume":"395","author":"MJ Cummings","year":"2020","unstructured":"Cummings MJ, Baldwin MR, Abrams D, Jacobson SD, Meyer BJ, Balough EM, Aaron JG, Claassen J, Rabbani LE, Hastie J. Epidemiology, clinical course, and outcomes of critically ill adults with COVID-19 in New York City: a prospective cohort study. The Lancet. 2020;395(10239):1763\u201370.","journal-title":"The Lancet"},{"issue":"3","key":"1712_CR27","doi-asserted-by":"publisher","first-page":"431","DOI":"10.1038\/sj.bjc.6601119","volume":"89","author":"MJ Bradburn","year":"2003","unstructured":"Bradburn MJ, Clark TG, Love SB, Altman DG. Survival analysis part II: multivariate data analysis\u2013an introduction to concepts and methods. Br J Cancer. 2003;89(3):431\u20136.","journal-title":"Br J Cancer"},{"issue":"1","key":"1712_CR28","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41598-019-56847-4","volume":"10","author":"C-H Ho","year":"2020","unstructured":"Ho C-H, Chen C-L, Yu C-C, Yang Y-H, Chen C-Y. High-flow nasal cannula ventilation therapy for obstructive sleep apnea in ischemic stroke patients requiring nasogastric tube feeding: a preliminary study. Sci Rep. 2020;10(1):1\u20138.","journal-title":"Sci Rep"},{"issue":"13","key":"1712_CR29","doi-asserted-by":"publisher","first-page":"1239","DOI":"10.1001\/jama.2020.2648","volume":"323","author":"Z Wu","year":"2020","unstructured":"Wu Z, McGoogan JM. Characteristics of and important lessons from the coronavirus disease 2019 (COVID-19) outbreak in China: summary of a report of 72 314 cases from the chinese center for disease control and prevention. JAMA. 2020;323(13):1239\u201342.","journal-title":"JAMA"},{"key":"1712_CR30","doi-asserted-by":"publisher","first-page":"m2446","DOI":"10.1136\/bmj.m2446","volume":"369","author":"TW Nicholson","year":"2020","unstructured":"Nicholson TW, Talbot NP, Nickol A, Chadwick AJ, Lawton O. Respiratory failure and non-invasive respiratory support during the covid-19 pandemic: an update for re-deployed hospital doctors and primary care physicians. BMJ. 2020;369:m2446.","journal-title":"BMJ"},{"issue":"7","key":"1712_CR31","doi-asserted-by":"publisher","first-page":"2219","DOI":"10.1161\/STROKEAHA.120.030365","volume":"51","author":"C Qin","year":"2020","unstructured":"Qin C, Zhou L, Hu Z, Yang S, Zhang S, Chen M, Yu H, Tian DS, Wang W. Clinical characteristics and outcomes of COVID-19 patients with a history of stroke in Wuhan, China. Stroke. 2020;51(7):2219\u201323.","journal-title":"Stroke"},{"key":"1712_CR32","doi-asserted-by":"publisher","first-page":"32","DOI":"10.11604\/pamj.2020.37.32.25192","volume":"37","author":"S Elamari","year":"2020","unstructured":"Elamari S, Motaib I, Zbiri S, Elaidaoui K, Chadli A, Elkettani C. Characteristics and outcomes of diabetic patients infected by the SARS-CoV-2. Pan Afr Med J. 2020;37:32.","journal-title":"Pan Afr Med J"}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-021-01712-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12911-021-01712-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-021-01712-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,12,17]],"date-time":"2021-12-17T11:14:57Z","timestamp":1639739697000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-021-01712-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12]]},"references-count":32,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["1712"],"URL":"https:\/\/doi.org\/10.1186\/s12911-021-01712-6","relation":{},"ISSN":["1472-6947"],"issn-type":[{"value":"1472-6947","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,12]]},"assertion":[{"value":"7 June 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 November 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 December 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The protocol was approved by the Institutional Review Board at NYU Langone Health (NYULH IRB).","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"350"}}