{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,13]],"date-time":"2025-12-13T06:07:30Z","timestamp":1765606050071,"version":"3.48.0"},"reference-count":48,"publisher":"Wiley","issue":"12","license":[{"start":{"date-parts":[[2025,7,10]],"date-time":"2025-07-10T00:00:00Z","timestamp":1752105600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62403240"],"award-info":[{"award-number":["62403240"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004608","name":"Natural Science Foundation of Jiangsu Province","doi-asserted-by":"publisher","award":["BK20241396"],"award-info":[{"award-number":["BK20241396"]}],"id":[{"id":"10.13039\/501100004608","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["advanced.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Advanced Intelligent Systems"],"published-print":{"date-parts":[[2025,12]]},"abstract":"<jats:p>Multiple unmanned aerial vehicle (multi\u2010UAV) formation plays a pivotal role in both military and civilian applications. However, enabling multi\u2010UAV to efficiently acquire collaborative formation policies is challenging, primarily due to the complex kinematics and dynamic environmental factors involved. Herein, a deep reinforcement learning\u2010based multi\u2010UAV formation control approach is proposed. The multi\u2010UAV formation control problem as a Markov decision process to capture the inherent sequential decision\u2010making process is first modeled. Furthermore, an enhanced actor\u2010critic algorithm called offline sample correction actor\u2010critic (OSCAC) to learn the formation control policy is proposed. The core insight behind OSCAC is to optimize the utilization of historical data through correcting offline samples. Specifically, OSCAC adjusts the importance weights of the samples based on the discrepancies between the current policy and previous policies during each update step, making better use of past experience and improving the learning performance. The effectiveness and reliability of the proposed approach are validated through numerical simulations, software\u2010in\u2010the\u2010loop simulations, and real\u2010world experiments.<\/jats:p>","DOI":"10.1002\/aisy.202500208","type":"journal-article","created":{"date-parts":[[2025,7,10]],"date-time":"2025-07-10T13:28:48Z","timestamp":1752154128000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Multiple Unmanned Aerial Vehicle Formation Control through Deep Reinforcement Learning with Offline Sample Correction"],"prefix":"10.1002","volume":"7","author":[{"given":"Zhongkai","family":"Chen","sequence":"first","affiliation":[{"name":"College of Intelligence Science and Technology National University of Defense Technology  Changsha 410073 China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Han","family":"Zhou","sequence":"additional","affiliation":[{"name":"College of Intelligence Science and Technology National University of Defense Technology  Changsha 410073 China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9995-4239","authenticated-orcid":false,"given":"Chao","family":"Yan","sequence":"additional","affiliation":[{"name":"College of Intelligence Science and Technology National University of Defense Technology  Changsha 410073 China"},{"name":"College of Automation Engineering Nanjing University of Aeronautics and Astronautics  Nanjing 211106 China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yihao","family":"Sun","sequence":"additional","affiliation":[{"name":"College of Intelligence Science and Technology National University of Defense Technology  Changsha 410073 China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiaojia","family":"Xiang","sequence":"additional","affiliation":[{"name":"College of Intelligence Science and Technology National University of Defense Technology  Changsha 410073 China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"311","published-online":{"date-parts":[[2025,7,10]]},"reference":[{"key":"e_1_2_10_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVT.2023.3318641"},{"key":"e_1_2_10_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/TII.2023.3248616"},{"key":"e_1_2_10_4_1","doi-asserted-by":"publisher","DOI":"10.1002\/aisy.202300761"},{"key":"e_1_2_10_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2023.3331370"},{"key":"e_1_2_10_6_1","doi-asserted-by":"publisher","DOI":"10.1002\/aisy.202300709"},{"key":"e_1_2_10_7_1","doi-asserted-by":"publisher","DOI":"10.1007\/s40747-022-00891-7"},{"key":"e_1_2_10_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVT.2020.2999374"},{"key":"e_1_2_10_9_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2007.09.019"},{"key":"e_1_2_10_10_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1008814708459"},{"key":"e_1_2_10_11_1","doi-asserted-by":"crossref","unstructured":"C. W.Reynolds inProc. 14th Annual Conf. on Computer Graphics and Interactive Techniques(Ed:M. C.Stone) ACM Anaheim USA1987 p.25.","DOI":"10.1145\/37401.37406"},{"key":"e_1_2_10_12_1","doi-asserted-by":"publisher","DOI":"10.1002\/rob.10057"},{"key":"e_1_2_10_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCST.2023.3240286"},{"key":"e_1_2_10_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cja.2020.03.020"},{"key":"e_1_2_10_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2023.3330149"},{"key":"e_1_2_10_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ast.2021.106736"},{"key":"e_1_2_10_17_1","first-page":"1","volume":"12","author":"Tanveer F.","year":"2024","journal-title":"IEEE Access"},{"key":"e_1_2_10_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/TII.2021.3094207"},{"key":"e_1_2_10_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIV.2024.3372625"},{"key":"e_1_2_10_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2023.3245124"},{"key":"e_1_2_10_21_1","doi-asserted-by":"crossref","unstructured":"C.Pan X.Nian X.Dai H.Wang H.Xiong inInt. Conf. Autonomous Unmanned Systems(Ed:W.Fu M.Gu Y.Niu Springer Singapore2022 p.1149.","DOI":"10.1007\/978-981-99-0479-2_104"},{"key":"e_1_2_10_22_1","unstructured":"R.Lowe Y. I.Wu A.Tamar J.Harb P.Abbeel I.Mordatch inAdvances in Neural Information Processing Systems Vol.30(Ed:U.von Luxburg I.Guyon S.Bengio H.Wallach R.Fergus) Curran Associates Long Beach USA2017 p.1."},{"key":"e_1_2_10_23_1","doi-asserted-by":"crossref","unstructured":"B.Li S.Li C.Wang R.Fan J.Shao G.Xie inChina Automation Congress (CAC) IEEE Beijing China2021 p.4750.","DOI":"10.1109\/CAC53003.2021.9727879"},{"key":"e_1_2_10_24_1","doi-asserted-by":"crossref","unstructured":"Y.Shi J.Song Y.Hua J.Yu X.Dong Z.Ren inChinese Control Conf. (CCC) IEEE Hefei China2022 p.3456.","DOI":"10.23919\/CCC55666.2022.9901799"},{"key":"e_1_2_10_25_1","unstructured":"K.Wang R.Xing W.Feng B.Huang inInt. Con. Wireless Communications Networking and Applications(Ed:Z.Qian M.Jabbar X.Li) Springer Singapore2021 p.187."},{"key":"e_1_2_10_26_1","doi-asserted-by":"publisher","DOI":"10.3390\/drones7010028"},{"key":"e_1_2_10_27_1","unstructured":"J.Schulman F.Wolski P.Dhariwal A.Radford O.Klimov(Preprint) \u00a0arXiv:1707.06347 v1 Submitted: Jul.2017."},{"key":"e_1_2_10_28_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.aej.2024.05.029"},{"key":"e_1_2_10_29_1","unstructured":"T.Schaul J.Quan I.Antonoglou D.Silver inInt. Conf. Learning Representations OpenReview.net Puerto Rico USA2016 p.322."},{"key":"e_1_2_10_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2024.3375083"},{"key":"e_1_2_10_31_1","first-page":"13985","volume":"72","author":"Hu Z.","year":"2023","journal-title":"IEEE Trans. Veh. Technol."},{"key":"e_1_2_10_32_1","doi-asserted-by":"crossref","unstructured":"J.Zhong T.Long J.Sun J.Li Y.Cao inInt. Conf. Control Science and Systems Engineering (ICCSSE) IEEE Shenzhen China2023 p.172.","DOI":"10.1109\/ICCSSE59359.2023.10245929"},{"key":"e_1_2_10_33_1","unstructured":"T.Haarnoja A.Zhou P.Abbeel S.Levine inInt. Conf. Machine Learning(Ed:D.Jennifer K.Andreas) PMLR Stockholm Sweden2018 p.1861."},{"key":"e_1_2_10_34_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cja.2023.07.006"},{"key":"e_1_2_10_35_1","unstructured":"D.Silver G.Lever N.Heess T.Degris D.Wierstra M.Riedmiller inInt. Conf. Machine Learning(Ed:P.Eric J.Tony) PMLR Beijing China2014 p.387."},{"key":"e_1_2_10_36_1","unstructured":"J. J. H.Ackermann V.Gabler T.Osa M.Sugiyama(Preprint) arXiv:1910.01465 v1 Submitted: Oct.2019."},{"key":"e_1_2_10_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2020.3004893"},{"key":"e_1_2_10_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVT.2021.3129504"},{"key":"e_1_2_10_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2021.3127873"},{"key":"e_1_2_10_40_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.oceaneng.2023.113811"},{"key":"e_1_2_10_41_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2024.123202"},{"key":"e_1_2_10_42_1","first-page":"1","volume":"5","author":"Zhang Q.","year":"2023","journal-title":"IET Cyber\u2010Syst. Robot."},{"key":"e_1_2_10_43_1","unstructured":"J.Lu M.Henchion B.Mac Namee inProc. 12th Conf. on Language Resources and Evaluation (LREC)(Ed:N.Calzolari F.B\u00e9chet P.Blache K.Choukri C.Cieri T.Declerck S.Goggi H.Isahara B.Maegaard J.Mariani H.Mazo A.Moreno J.Odijk S.Piperidis) European Language Resources Association Marseille France2020 p.6740."},{"key":"e_1_2_10_44_1","unstructured":"S.Fujimoto H.Hoof D.Meger inInt. Conf. Machine Learning (Ed:D.Jennifer K.Andreas) PMLR Stockholm Sweden2018 p.1587."},{"key":"e_1_2_10_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2005.846556"},{"key":"e_1_2_10_46_1","doi-asserted-by":"crossref","unstructured":"H.Van Hasselt A.Guez D.Silver inProc. AAAI Conf. Artificial Intelligence AAAI Press Phoenix AZ2016 p.30.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"e_1_2_10_47_1","first-page":"1129","volume":"69","author":"Pan Z.","year":"2021","journal-title":"IEEE Trans. Circuits Syst. II, Exp. Briefs"},{"key":"e_1_2_10_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/LCSYS.2020.3003418"},{"key":"e_1_2_10_49_1","doi-asserted-by":"crossref","unstructured":"K.Xiao L.Ma S.Tan Y.Cong X.Wang inAdvances in Guidance Navigation and Control: Proc. 2020 Int. Conf. Guidance Navigation and Control(Ed:L.Yan H.Duan X.Yu) Springer Singapore2022 p.5131.","DOI":"10.1007\/978-981-15-8155-7_423"}],"container-title":["Advanced Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/advanced.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/aisy.202500208","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,13]],"date-time":"2025-12-13T06:03:06Z","timestamp":1765605786000},"score":1,"resource":{"primary":{"URL":"https:\/\/advanced.onlinelibrary.wiley.com\/doi\/10.1002\/aisy.202500208"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,10]]},"references-count":48,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2025,12]]}},"alternative-id":["10.1002\/aisy.202500208"],"URL":"https:\/\/doi.org\/10.1002\/aisy.202500208","archive":["Portico"],"relation":{},"ISSN":["2640-4567","2640-4567"],"issn-type":[{"type":"print","value":"2640-4567"},{"type":"electronic","value":"2640-4567"}],"subject":[],"published":{"date-parts":[[2025,7,10]]},"assertion":[{"value":"2025-02-20","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-07-10","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e202500208"}}