{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T16:18:29Z","timestamp":1782404309460,"version":"3.54.5"},"reference-count":56,"publisher":"World Scientific Pub Co Pte Ltd","issue":"01","funder":[{"DOI":"10.13039\/100000183","name":"Army Research Office","doi-asserted-by":"publisher","award":["W911NF-20-1-0132"],"award-info":[{"award-number":["W911NF-20-1-0132"]}],"id":[{"id":"10.13039\/100000183","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000006","name":"Office of Naval Research","doi-asserted-by":"publisher","award":["N00014-18-1-2221"],"award-info":[{"award-number":["N00014-18-1-2221"]}],"id":[{"id":"10.13039\/100000006","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Un. Sys."],"published-print":{"date-parts":[[2023,1]]},"abstract":"<jats:p> This paper briefly reviews the dynamics and the control architectures of unmanned vehicles; reinforcement learning (RL) in optimal control theory; and RL-based applications in unmanned vehicles. Nonlinearities and uncertainties in the dynamics of unmanned vehicles (e.g. aerial, underwater, and tailsitter vehicles) pose critical challenges to their control systems. Solving Hamilton\u2013Jacobi\u2013Bellman (HJB) equations to find optimal controllers becomes difficult in the presence of nonlinearities, uncertainties, and actuator faults. Therefore, RL-based approaches are widely used in unmanned vehicle systems to solve the HJB equations. To this end, they learn the optimal solutions by using online data measured along the system trajectories. This approach is very practical in partially or completely model-free optimal control design and optimal fault-tolerant control design for unmanned vehicle systems. <\/jats:p>","DOI":"10.1142\/s2301385023310027","type":"journal-article","created":{"date-parts":[[2022,6,27]],"date-time":"2022-06-27T06:25:56Z","timestamp":1656311156000},"page":"17-26","source":"Crossref","is-referenced-by-count":24,"title":["Reinforcement Learning Applications in Unmanned Vehicle Control: A Comprehensive Overview"],"prefix":"10.1142","volume":"11","author":[{"given":"Hao","family":"Liu","sequence":"first","affiliation":[{"name":"Institute of Artificial Intelligence, Beihang University, Beijing 100191, P.\u00a0R.\u00a0China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bahare","family":"Kiumarsi","sequence":"additional","affiliation":[{"name":"Michigan State University, East Lansing, MI 48823, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0295-8584","authenticated-orcid":false,"given":"Yusuf","family":"Kartal","sequence":"additional","affiliation":[{"name":"University of Texas at Arlington Research Institute, Fort Worth, TX 76118, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ahmet","family":"Taha Koru","sequence":"additional","affiliation":[{"name":"University of Texas at Arlington Research Institute, Fort Worth, TX 76118, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hamidreza","family":"Modares","sequence":"additional","affiliation":[{"name":"Michigan State University, East Lansing, MI 48823, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Frank L.","family":"Lewis","sequence":"additional","affiliation":[{"name":"University of Texas at Arlington Research Institute, Fort Worth, TX 76118, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"219","published-online":{"date-parts":[[2022,8,4]]},"reference":[{"key":"S2301385023310027BIB001","volume-title":"Neuro-Dynamic Programming","author":"Bertsekas D. P.","year":"1996"},{"key":"S2301385023310027BIB002","volume-title":"Reinforcement Learning: An Introduction","author":"Sutton R. S.","year":"1998"},{"key":"S2301385023310027BIB003","doi-asserted-by":"publisher","DOI":"10.1002\/9781118122631"},{"issue":"1","key":"S2301385023310027BIB004","first-page":"388","volume":"2","author":"Deisenroth M. P.","year":"2013","journal-title":"Found. Trends Robot."},{"key":"S2301385023310027BIB005","doi-asserted-by":"publisher","DOI":"10.1177\/0278364913495721"},{"key":"S2301385023310027BIB006","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2017.2773458"},{"key":"S2301385023310027BIB007","doi-asserted-by":"publisher","DOI":"10.1142\/S2301385021410041"},{"key":"S2301385023310027BIB008","doi-asserted-by":"publisher","DOI":"10.1109\/CDC.1989.70114"},{"key":"S2301385023310027BIB009","volume-title":"A Menu of Designs for Reinforcement Learning Over Time","author":"Werbos P. J.","year":"1990"},{"key":"S2301385023310027BIB010","first-page":"493","volume-title":"Handbook of Intelligent Control: Neural, Fuzzy and Adaptive Approaches","author":"Werbos P.","year":"1992"},{"key":"S2301385023310027BIB011","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2020.3011912"},{"key":"S2301385023310027BIB012","doi-asserted-by":"publisher","DOI":"10.1109\/TVT.2020.2982508"},{"key":"S2301385023310027BIB013","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2021.3068762"},{"key":"S2301385023310027BIB014","doi-asserted-by":"publisher","DOI":"10.1109\/TVT.2018.2890773"},{"key":"S2301385023310027BIB015","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2020.2974648"},{"key":"S2301385023310027BIB016","doi-asserted-by":"publisher","DOI":"10.23919\/JSEE.2021.000121"},{"key":"S2301385023310027BIB017","doi-asserted-by":"publisher","DOI":"10.1109\/TVT.2021.3074304"},{"key":"S2301385023310027BIB018","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2019.2893683"},{"key":"S2301385023310027BIB019","doi-asserted-by":"publisher","DOI":"10.1109\/TVT.2019.2922849"},{"key":"S2301385023310027BIB020","doi-asserted-by":"publisher","DOI":"10.1109\/TVT.2020.3023733"},{"key":"S2301385023310027BIB021","doi-asserted-by":"publisher","DOI":"10.1109\/TVT.2021.3129504"},{"key":"S2301385023310027BIB022","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2018.2848264"},{"key":"S2301385023310027BIB023","doi-asserted-by":"publisher","DOI":"10.1142\/S2301385021500096"},{"key":"S2301385023310027BIB024","doi-asserted-by":"publisher","DOI":"10.1142\/S2301385015500090"},{"key":"S2301385023310027BIB025","doi-asserted-by":"publisher","DOI":"10.1142\/S2301385021500126"},{"key":"S2301385023310027BIB026","doi-asserted-by":"publisher","DOI":"10.1142\/S2301385016400069"},{"key":"S2301385023310027BIB027","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2021.3053194"},{"key":"S2301385023310027BIB028","doi-asserted-by":"publisher","DOI":"10.1109\/CDC40024.2019.9030076"},{"key":"S2301385023310027BIB029","doi-asserted-by":"publisher","DOI":"10.1007\/s11768-020-9145-y"},{"key":"S2301385023310027BIB030","doi-asserted-by":"publisher","DOI":"10.1007\/s00773-018-0582-y"},{"key":"S2301385023310027BIB031","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2016.7487466"},{"key":"S2301385023310027BIB032","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2020.3023711"},{"key":"S2301385023310027BIB033","doi-asserted-by":"publisher","DOI":"10.1109\/TII.2017.2682900"},{"key":"S2301385023310027BIB034","doi-asserted-by":"publisher","DOI":"10.1109\/TCYB.2018.2875559"},{"key":"S2301385023310027BIB035","doi-asserted-by":"publisher","DOI":"10.1109\/TCST.2019.2922159"},{"key":"S2301385023310027BIB036","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2009.10.018"},{"key":"S2301385023310027BIB037","doi-asserted-by":"publisher","DOI":"10.1002\/asjc.1758"},{"key":"S2301385023310027BIB038","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2008.08.017"},{"key":"S2301385023310027BIB039","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2009.03.008"},{"key":"S2301385023310027BIB040","doi-asserted-by":"publisher","DOI":"10.1109\/JAS.2014.7004681"},{"key":"S2301385023310027BIB041","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2015.10.039"},{"key":"S2301385023310027BIB042","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2013.09.043"},{"key":"S2301385023310027BIB043","doi-asserted-by":"publisher","DOI":"10.1002\/acs.2297"},{"key":"S2301385023310027BIB044","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2013.2294968"},{"key":"S2301385023310027BIB045","doi-asserted-by":"publisher","DOI":"10.1109\/TCYB.2014.2319577"},{"key":"S2301385023310027BIB046","doi-asserted-by":"publisher","DOI":"10.1002\/rnc.5719"},{"key":"S2301385023310027BIB047","doi-asserted-by":"publisher","DOI":"10.1049\/iet-cta.2019.1151"},{"key":"S2301385023310027BIB048","doi-asserted-by":"publisher","DOI":"10.2514\/6.2022-1381"},{"key":"S2301385023310027BIB049","doi-asserted-by":"publisher","DOI":"10.1002\/9781119174882"},{"key":"S2301385023310027BIB050","doi-asserted-by":"publisher","DOI":"10.1109\/TCST.2014.2299555"},{"key":"S2301385023310027BIB051","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2012.06.096"},{"key":"S2301385023310027BIB052","doi-asserted-by":"publisher","DOI":"10.1109\/TCST.2017.2670522"},{"key":"S2301385023310027BIB053","author":"Zhao W.","year":"2021","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"S2301385023310027BIB054","doi-asserted-by":"publisher","DOI":"10.1016\/j.sysconle.2021.105063"},{"key":"S2301385023310027BIB055","doi-asserted-by":"publisher","DOI":"10.1049\/iet-cta.2011.0599"},{"key":"S2301385023310027BIB056","doi-asserted-by":"publisher","DOI":"10.1109\/TCYB.2018.2886735"}],"container-title":["Unmanned Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.worldscientific.com\/doi\/pdf\/10.1142\/S2301385023310027","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,17]],"date-time":"2023-01-17T04:02:07Z","timestamp":1673928127000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.worldscientific.com\/doi\/10.1142\/S2301385023310027"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,4]]},"references-count":56,"journal-issue":{"issue":"01","published-print":{"date-parts":[[2023,1]]}},"alternative-id":["10.1142\/S2301385023310027"],"URL":"https:\/\/doi.org\/10.1142\/s2301385023310027","relation":{},"ISSN":["2301-3850","2301-3869"],"issn-type":[{"value":"2301-3850","type":"print"},{"value":"2301-3869","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,8,4]]}}}