{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,21]],"date-time":"2026-01-21T23:33:50Z","timestamp":1769038430466,"version":"3.49.0"},"reference-count":38,"publisher":"Wiley","issue":"3","license":[{"start":{"date-parts":[[2025,2,2]],"date-time":"2025-02-02T00:00:00Z","timestamp":1738454400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["advanced.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Advanced Intelligent Systems"],"published-print":{"date-parts":[[2025,3]]},"abstract":"<jats:p>This article introduces a novel robust reinforcement learning (RL) control scheme for a quadrotor unmanned aerial vehicle (QUAV) under external disturbances and model uncertainties. First, the translational and rotational motions of the QUAV are decoupled and trained separately to mitigate the computational complexity of the controller design and training process. Then, the proximal policy optimization algorithm with a dual\u2010critic structure is proposed to address the overestimation issue and accelerate the convergence speed of RL controllers. Furthermore, a novel reward function and a robust compensator employing a switch value function are proposed to address model uncertainties and external disturbances. At last, simulation results and comparisons demonstrate the effectiveness and robustness of the proposed RL control framework.<\/jats:p>","DOI":"10.1002\/aisy.202400427","type":"journal-article","created":{"date-parts":[[2025,2,2]],"date-time":"2025-02-02T19:30:11Z","timestamp":1738524611000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Robust Reinforcement Learning Control Framework for a Quadrotor Unmanned Aerial Vehicle Using Critic Neural Network"],"prefix":"10.1002","volume":"7","author":[{"given":"Yu","family":"Cai","sequence":"first","affiliation":[{"name":"Center for Control Theory and Guidance Technology Harbin Institute of Technology  Harbin 150001 China"}]},{"given":"Yefeng","family":"Yang","sequence":"additional","affiliation":[{"name":"Center for Control Theory and Guidance Technology Harbin Institute of Technology  Harbin 150001 China"},{"name":"Department of Aeronautical and Aviation Engineering The Hong Kong Polytechnic University  Hong Kong China"}]},{"given":"Tao","family":"Huang","sequence":"additional","affiliation":[{"name":"Center for Control Theory and Guidance Technology Harbin Institute of Technology  Harbin 150001 China"},{"name":"Department of Aeronautical and Aviation Engineering The Hong Kong Polytechnic University  Hong Kong China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6900-0901","authenticated-orcid":false,"given":"Boyang","family":"Li","sequence":"additional","affiliation":[{"name":"School of Engineering The University of Newcastle  Callaghan NSW 2308 Australia"}]}],"member":"311","published-online":{"date-parts":[[2025,2,2]]},"reference":[{"key":"e_1_2_9_2_1","doi-asserted-by":"publisher","DOI":"10.1002\/rob.20414"},{"key":"e_1_2_9_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVT.2023.3306813"},{"key":"e_1_2_9_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compag.2018.02.026"},{"key":"e_1_2_9_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2016.2633562"},{"key":"e_1_2_9_6_1","first-page":"3","volume":"19","author":"Mahony R.","year":"2012","journal-title":"IEEE Rob. Autom. Mag."},{"key":"e_1_2_9_7_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ast.2022.107749"},{"key":"e_1_2_9_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2019.2900510"},{"key":"e_1_2_9_9_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.arcontrol.2023.100900"},{"key":"e_1_2_9_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMECH.2018.2855161"},{"key":"e_1_2_9_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2021.3072709"},{"key":"e_1_2_9_12_1","first-page":"2","volume":"7","author":"Nan F.","year":"2022","journal-title":"IEEE Rob. Autom. Lett."},{"key":"e_1_2_9_13_1","first-page":"6","volume":"38","author":"Sun S.","year":"2022","journal-title":"IEEE Trans. Rob."},{"key":"e_1_2_9_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2017.2783587"},{"key":"e_1_2_9_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jfranklin.2020.04.056"},{"key":"e_1_2_9_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ast.2022.107818"},{"key":"e_1_2_9_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMECH.2022.3230755"},{"key":"e_1_2_9_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/TAES.2022.3177580"},{"key":"e_1_2_9_19_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ast.2020.105716"},{"key":"e_1_2_9_20_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature16961"},{"key":"e_1_2_9_21_1","first-page":"3","volume":"66","author":"Zhu J.","year":"2023","journal-title":"Sci. China Inf. Sci."},{"key":"e_1_2_9_22_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compag.2019.104985"},{"key":"e_1_2_9_23_1","unstructured":"T. P.Lillicrap J. J.Hunt A.Pritzel N.Heess T.Erez Y.Tassa D.Silver D.Wierstra presented atProc. Int. Conf. Learning Representations (ICLR) San Juan Puerto Rico2016."},{"key":"e_1_2_9_24_1","unstructured":"S.Fujimoto H.Hoof D.Meger presented atProc. Int. Conf. Machine Learning (ICML) Stockholm Sweden2018."},{"key":"e_1_2_9_25_1","unstructured":"J.Schulman F.Wolski P.Dhariwal A.Radford O.Klimov (preprint) arXiv:1707.06347 v1 submitted: Jul.2017."},{"key":"e_1_2_9_26_1","first-page":"8","volume":"52","author":"Liu Y.\u2010C.","year":"2021","journal-title":"IEEE Trans. Cybern."},{"key":"e_1_2_9_27_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2020.03.063"},{"key":"e_1_2_9_28_1","first-page":"7","volume":"52","author":"Gu Y.","year":"2021","journal-title":"IEEE Trans. Syst., Man, Cybern., Syst."},{"key":"e_1_2_9_29_1","unstructured":"K. W. R.Quan Vuong Y.Zhang presented atProc. Int. Conf. Learning Representations (ICLR) Vancouver Canada2018."},{"key":"e_1_2_9_30_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.engappai.2018.11.006"},{"key":"e_1_2_9_31_1","first-page":"3","volume":"70","author":"Hua H.","year":"2022","journal-title":"IEEE Trans. Ind. Electron."},{"key":"e_1_2_9_32_1","first-page":"6","volume":"38","author":"Gong C.","year":"2023","journal-title":"IEEE Trans. Power Electron."},{"key":"e_1_2_9_33_1","first-page":"8","volume":"52","author":"Wen G.","year":"2021","journal-title":"IEEE Trans. Syst., Man, Cybern., Syst."},{"key":"e_1_2_9_34_1","first-page":"107","volume":"126","author":"Zhang W.","year":"2022","journal-title":"IEEE Aerosp. Sci. Technol."},{"key":"e_1_2_9_35_1","unstructured":"J.Schulman S.Levine P.Abbeel M.Jordan P.Moritz presented atInt. Conf. Machine Learning (ICML) Lille France2015."},{"key":"e_1_2_9_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/TII.2022.3143175"},{"key":"e_1_2_9_37_1","unstructured":"V.Mnih A. P.Badia M.Mirza A.Graves T.Lillicrap T.Harley D.Silver K.Kavukcuoglu presented atInt. Conf. Machine Learning (ICML) New York NY2016."},{"key":"e_1_2_9_38_1","unstructured":"J.Schulman P.Moritz S.Levine M.Jordan P.Abbeel presented atProc. Int. Conf. Learning Representations (ICLR) San Juan Puerto Rico2016."},{"key":"e_1_2_9_39_1","first-page":"5","volume":"70","author":"Yim J.","year":"2022","journal-title":"IEEE Trans. Ind. Electron."}],"container-title":["Advanced Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/advanced.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/aisy.202400427","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,7]],"date-time":"2025-10-07T19:57:06Z","timestamp":1759867026000},"score":1,"resource":{"primary":{"URL":"https:\/\/advanced.onlinelibrary.wiley.com\/doi\/10.1002\/aisy.202400427"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,2]]},"references-count":38,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2025,3]]}},"alternative-id":["10.1002\/aisy.202400427"],"URL":"https:\/\/doi.org\/10.1002\/aisy.202400427","archive":["Portico"],"relation":{},"ISSN":["2640-4567","2640-4567"],"issn-type":[{"value":"2640-4567","type":"print"},{"value":"2640-4567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,2,2]]},"assertion":[{"value":"2024-05-28","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-02-02","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"2400427"}}