{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T14:25:29Z","timestamp":1740147929993,"version":"3.37.3"},"reference-count":29,"publisher":"Wiley","license":[{"start":{"date-parts":[[2021,2,22]],"date-time":"2021-02-22T00:00:00Z","timestamp":1613952000000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003399","name":"Science and Technology Commission of Shanghai Municipality","doi-asserted-by":"publisher","award":["17YF1426700","82073640"],"award-info":[{"award-number":["17YF1426700","82073640"]}],"id":[{"id":"10.13039\/501100003399","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["17YF1426700","82073640"],"award-info":[{"award-number":["17YF1426700","82073640"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computational and Mathematical Methods in Medicine"],"published-print":{"date-parts":[[2021,2,22]]},"abstract":"<jats:p>Dynamic decision-making was essential in the clinical care of surgical patients. Reinforcement learning (RL) algorithm is a computational method to find sequential optimal decisions among multiple suboptimal options. This review is aimed at introducing RL\u2019s basic concepts, including three basic components: the state, the action, and the reward. Most medical studies using reinforcement learning methods were trained on a fixed observational dataset. This paper also reviews the literature of existing practical applications using reinforcement learning methods, which can be further categorized as a statistical RL study and a computational RL study. The review proposes several potential aspects where reinforcement learning can be applied in neurocritical and neurosurgical care. These include sequential treatment strategies of intracranial tumors and traumatic brain injury and intraoperative endoscope motion control. Several limitations of reinforcement learning are representations of basic components, the positivity violation, and validation methods.<\/jats:p>","DOI":"10.1155\/2021\/6657119","type":"journal-article","created":{"date-parts":[[2021,2,23]],"date-time":"2021-02-23T22:20:41Z","timestamp":1614118841000},"page":"1-6","source":"Crossref","is-referenced-by-count":3,"title":["Reinforcement Learning in Neurocritical and Neurosurgical Care: Principles and Possible Applications"],"prefix":"10.1155","volume":"2021","author":[{"given":"Ying","family":"Liu","sequence":"first","affiliation":[{"name":"Lhorong People\u2019s Hospital, Tibet, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5478-3555","authenticated-orcid":true,"given":"Nidan","family":"Qiao","sequence":"additional","affiliation":[{"name":"Department of Neurosurgery, Huashan Hospital, Shanghai Medical School, Fudan University, Shanghai, China"},{"name":"Shanghai Clinical Medical Center of Neurosurgery, Shanghai, China"},{"name":"Neurosurgical Institute of Fudan University, Shanghai, China"},{"name":"Medical Science in Clinical Investigation, Harvard Medical School, Boston, USA"}]},{"given":"Yuksel","family":"Altinel","sequence":"additional","affiliation":[{"name":"Medical Science in Clinical Investigation, Harvard Medical School, Boston, USA"}]}],"member":"311","reference":[{"key":"1","doi-asserted-by":"publisher","DOI":"10.21037\/atm.2019.06.75"},{"volume-title":"Reinforcement Learning: An Introduction","year":"2018","author":"R. S. Sutton","key":"2"},{"key":"3","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.med.59.062606.122232"},{"key":"4","doi-asserted-by":"publisher","DOI":"10.1093\/oxfordjournals.schbul.a006986"},{"key":"5","doi-asserted-by":"publisher","DOI":"10.1176\/ps.2009.60.11.1439"},{"key":"6","doi-asserted-by":"publisher","DOI":"10.1177\/1740774509344633"},{"key":"7","doi-asserted-by":"publisher","DOI":"10.1158\/1078-0432.CCR-17-1355"},{"key":"8","doi-asserted-by":"publisher","DOI":"10.1093\/aje\/kwv083"},{"key":"9","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4614-7428-9"},{"key":"10","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.2016.1148611"},{"key":"11","doi-asserted-by":"publisher","DOI":"10.1093\/biostatistics\/kxaa025"},{"key":"12","doi-asserted-by":"publisher","DOI":"10.1111\/biom.12921"},{"key":"13","doi-asserted-by":"publisher","DOI":"10.1213\/ANE.0b013e31820334a7"},{"key":"14","doi-asserted-by":"publisher","DOI":"10.1038\/nature14236"},{"key":"15","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2009.02.041"},{"key":"16","doi-asserted-by":"publisher","DOI":"10.1016\/j.artmed.2014.07.004"},{"key":"17","first-page":"239","article-title":"Combining kernel and model based learning for HIV therapy selection","volume":"2017","author":"S. Parbhoo","year":"2017","journal-title":"AMIA Summits on Translational Science Proceedings"},{"key":"18","doi-asserted-by":"publisher","DOI":"10.1109\/ichi.2017.45"},{"key":"19","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-018-0213-5"},{"key":"20","doi-asserted-by":"publisher","DOI":"10.1109\/embc.2016.7591355"},{"key":"21","doi-asserted-by":"publisher","DOI":"10.1002\/cpt.1777"},{"key":"22","doi-asserted-by":"publisher","DOI":"10.1109\/JBHI.2020.3014556"},{"key":"23","doi-asserted-by":"publisher","DOI":"10.1016\/j.surg.2020.04.049"},{"key":"24","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-020-1120-5"},{"key":"25","first-page":"181","article-title":"Identifying distinct, effective treatments for acute hypotension with SODA-RL: safely optimized diverse accurate reinforcement learning","author":"J. Futoma","year":"2020","journal-title":"AMIA Summits on Translational Science Proceedings"},{"key":"26","doi-asserted-by":"publisher","DOI":"10.1007\/s11548-010-0481-0"},{"key":"27","doi-asserted-by":"publisher","DOI":"10.1097\/GCO.0000000000000186"},{"key":"28","doi-asserted-by":"publisher","DOI":"10.1007\/s00266-019-01592-2"},{"key":"29","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-018-0310-5"}],"container-title":["Computational and Mathematical Methods in Medicine"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/downloads.hindawi.com\/journals\/cmmm\/2021\/6657119.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.hindawi.com\/journals\/cmmm\/2021\/6657119.xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.hindawi.com\/journals\/cmmm\/2021\/6657119.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,2,23]],"date-time":"2021-02-23T22:20:47Z","timestamp":1614118847000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.hindawi.com\/journals\/cmmm\/2021\/6657119\/"}},"subtitle":[],"editor":[{"given":"Waqas Haider","family":"Bangyal","sequence":"additional","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2021,2,22]]},"references-count":29,"alternative-id":["6657119","6657119"],"URL":"https:\/\/doi.org\/10.1155\/2021\/6657119","relation":{},"ISSN":["1748-6718","1748-670X"],"issn-type":[{"type":"electronic","value":"1748-6718"},{"type":"print","value":"1748-670X"}],"subject":[],"published":{"date-parts":[[2021,2,22]]}}}