{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,10]],"date-time":"2026-05-10T00:28:49Z","timestamp":1778372929987,"version":"3.51.4"},"reference-count":38,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2025,11,14]],"date-time":"2025-11-14T00:00:00Z","timestamp":1763078400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>Cooperative path planning for unmanned aerial vehicle (UAV) swarms has attracted considerable research attention, yet it remains challenging in complex, uncertain environments. To tackle this problem, we model the cooperative path planning task as a heterogeneous decentralized Markov Decision Process (MDP), emphasizing the symmetry-inspired role assignment between leader and wingmen UAVs, which ensures balanced and coordinated behaviors in dynamic settings. We address the problem using a Multi-Agent Soft Actor-Critic (MASAC) framework enhanced with a symmetry-aware reward mechanism designed to optimize multiple cooperative objectives: simultaneous arrival, formation topology preservation, dynamic obstacle avoidance, trajectory smoothness, and inter-agent collision avoidance. This design promotes behavioral symmetry among agents, enhancing both coordination efficiency and system robustness. Simulation results demonstrate that our method achieves efficient swarm coordination and reliable obstacle avoidance. Quantitative evaluations show that our MASAC-CA algorithm achieves a Mission Success Rate (MSR) of 99.0% with 2\u20135 wingmen, representing approximately 13% improvement over baseline MASAC, while maintaining Formation Keeping Rates (FKR) of 59.68\u201326.29% across different swarm sizes. The method also reduces collisions to near zero in cluttered environments while keeping flight duration, path length, and energy consumption at levels comparable to baseline algorithms. Finally, the proposed model\u2019s robustness and effectiveness are validated in complex uncertain environments, underscoring the value of symmetry principles in multi-agent system design.<\/jats:p>","DOI":"10.3390\/sym17111970","type":"journal-article","created":{"date-parts":[[2025,11,14]],"date-time":"2025-11-14T14:37:52Z","timestamp":1763131072000},"page":"1970","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Cooperative Path Planning for Autonomous UAV Swarms Using MASAC-CA Algorithm"],"prefix":"10.3390","volume":"17","author":[{"given":"Wenli","family":"Hu","sequence":"first","affiliation":[{"name":"National Key Laboratory of Electromagnetic Energy Technology, Naval University of Engineering, Wuhan 430033, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mingyuan","family":"Zhang","sequence":"additional","affiliation":[{"name":"National Key Laboratory of Electromagnetic Energy Technology, Naval University of Engineering, Wuhan 430033, China"},{"name":"East Lake Laboratory, Wuhan 430033, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xinhua","family":"Xu","sequence":"additional","affiliation":[{"name":"National Key Laboratory of Electromagnetic Energy Technology, Naval University of Engineering, Wuhan 430033, China"},{"name":"East Lake Laboratory, Wuhan 430033, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4018-5826","authenticated-orcid":false,"given":"Shaohua","family":"Qiu","sequence":"additional","affiliation":[{"name":"National Key Laboratory of Electromagnetic Energy Technology, Naval University of Engineering, Wuhan 430033, China"},{"name":"East Lake Laboratory, Wuhan 430033, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-0863-8917","authenticated-orcid":false,"given":"Tao","family":"Liao","sequence":"additional","affiliation":[{"name":"National Key Laboratory of Electromagnetic Energy Technology, Naval University of Engineering, Wuhan 430033, China"},{"name":"East Lake Laboratory, Wuhan 430033, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4618-5955","authenticated-orcid":false,"given":"Longfei","family":"Yue","sequence":"additional","affiliation":[{"name":"National Key Laboratory of Electromagnetic Energy Technology, Naval University of Engineering, Wuhan 430033, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,11,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"415","DOI":"10.1016\/j.isatra.2019.08.018","article-title":"Efficient path planning for UAV formation via comprehensively improved particle swarm optimization","volume":"97","author":"Shao","year":"2020","journal-title":"ISA Trans."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"106332","DOI":"10.1016\/j.ast.2020.106332","article-title":"A new consensus theory-based method for formation control and obstacle avoidance of UAVs","volume":"107","author":"Wu","year":"2020","journal-title":"Aerosp. Sci. Technol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"106099","DOI":"10.1016\/j.asoc.2020.106099","article-title":"A novel reinforcement learning based grey wolf optimizer algorithm for unmanned aerial vehicles (UAVs) path planning","volume":"89","author":"Qu","year":"2020","journal-title":"Appl. Soft Comput."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1871","DOI":"10.1360\/SSI-2024-0050","article-title":"Multi-UAV cooperative path planning based on MASAC reinforcement learning algorithm","volume":"54","author":"Fang","year":"2024","journal-title":"Sci. Sin. Informationis"},{"key":"ref_5","unstructured":"Northwestern Polytechnical University Shenzhen Research Institute (2024). UAV Path Planning Method and Device Based on Maximum Entropy Safe Reinforcement Learning. (CN202410423432.7)."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2657","DOI":"10.1007\/s42405-025-00946-2","article-title":"Multiresolution approximation MDP for multi-target reconnaissance online planning","volume":"26","author":"Kim","year":"2025","journal-title":"Int. J. Aeronaut. Space Sci."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"9381","DOI":"10.1007\/s10586-024-04452-0","article-title":"Autonomous UAV-based surveillance system for multi-target detection using reinforcement learning","volume":"27","author":"Hussienat","year":"2024","journal-title":"Clust. Comput."},{"key":"ref_8","first-page":"89","article-title":"MDP formulation for multi-UAVs mission planning with refueling constraints","volume":"Volume 642","author":"Jo","year":"2023","journal-title":"Robot Intelligence Technology and Applications 7, Proceedings of the 10th International Conference on Robot Intelligence Technology and Applications (RiTA 2022), Kuala Lumpur, Malaysia, 7\u20139 December 2022"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"19667","DOI":"10.1007\/s00521-024-10261-8","article-title":"Mean policy-based proximal policy optimization for maneuvering decision in multi-UAV air combat","volume":"36","author":"Zheng","year":"2024","journal-title":"Neural Comput. Appl."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"9892","DOI":"10.1007\/s10489-024-05674-w","article-title":"Research on heterogeneous multi-UAV collaborative decision-making method based on improved PPO","volume":"54","author":"Xu","year":"2024","journal-title":"Appl. Intell."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1007\/s10458-025-09700-0","article-title":"Double mixing networks based monotonic value function decomposition algorithm for swarm intelligence in UAVs","volume":"39","author":"Qu","year":"2025","journal-title":"Auton. Agent Multi-Agent Syst."},{"key":"ref_12","first-page":"154","article-title":"Towards cost-efficient federated multi-agent RL with learnable aggregation","volume":"Volume 14646","author":"Yang","year":"2024","journal-title":"Advances in Knowledge Discovery and Data Mining, Proceedings of the 28th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD 2024), Taipei, Taiwan, 7\u201310 May 2024"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1007\/BF01386390","article-title":"A note on two problems in connexion with graphs","volume":"1","author":"Dijkstra","year":"1959","journal-title":"Numer. Math."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1109\/TSSC.1968.300136","article-title":"A formal basis for the heuristic determination of minimum cost paths","volume":"4","author":"Hart","year":"1968","journal-title":"IEEE Trans. Syst. Sci. Cybern."},{"key":"ref_15","unstructured":"Stentz, A. (1994, January 8\u201313). Optimal and efficient path planning for partially-known environments. Proceedings of the IEEE International Conference on Robotics and Automation, San Diego, CA, USA."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2201","DOI":"10.1007\/s10489-018-1384-y","article-title":"Three dimensional path planning using grey wolf optimizer for UAVs","volume":"49","author":"Dewangan","year":"2019","journal-title":"Appl. Intell."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1371","DOI":"10.1109\/TCDS.2022.3212062","article-title":"Path planning of unmanned autonomous helicopter based on hybrid satisficing decision-enhanced swarm intelligence algorithm","volume":"15","author":"Han","year":"2023","journal-title":"IEEE Trans. Cogn. Dev. Syst."},{"key":"ref_18","first-page":"2381","article-title":"Dynamic path planning based on improved constrained differential evolution algorithm","volume":"35","author":"Wu","year":"2020","journal-title":"Control Decis."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"119327","DOI":"10.1016\/j.eswa.2022.119327","article-title":"A hybrid algorithm based on grey wolf optimizer and differential evolution for UAV path planning","volume":"215","author":"Yu","year":"2023","journal-title":"Expert Syst. Appl."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"110164","DOI":"10.1016\/j.knosys.2022.110164","article-title":"Cooperative path planning optimization for multiple UAVs with communication constraints","volume":"260","author":"Xu","year":"2023","journal-title":"Knowl.-Based Syst."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1109\/TIV.2021.3082151","article-title":"Bio-inspired neural network-based optimal path planning for UUVs under the effect of ocean currents","volume":"7","author":"Zhu","year":"2022","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_22","unstructured":"Lin, L., Zhang, X.S., Han, C.L., and Ma, H. (2022). UAV maneuvering target tracking based on Kalman filter and DDQN algorithm. Tactical Missile Technol., 98\u2013104."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"457","DOI":"10.1016\/j.dt.2020.11.014","article-title":"Maneuvering target tracking of UAV based on MN-DDPG and transfer learning","volume":"17","author":"Li","year":"2021","journal-title":"Def. Technol."},{"key":"ref_24","first-page":"1","article-title":"Vision-based end-to-end target tracking control technology for UAV","volume":"56","author":"Hua","year":"2022","journal-title":"J. Zhejiang Univ."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Zhang, H.H., He, P.K., and Zhang, M. (2022, January 18\u201321). UAV target tracking method based on deep reinforcement learning. Proceedings of the 2022 International Conference on Cyber-Physical Social Intelligence (ICCSI), Beijing, China.","DOI":"10.1109\/ICCSI55536.2022.9970588"},{"key":"ref_26","first-page":"1","article-title":"Coordinated formation control for fixed-wing UAVs based on deep reinforcement learning","volume":"42","author":"Xiang","year":"2021","journal-title":"Acta Aeronaut. Astronaut. Sin."},{"key":"ref_27","first-page":"689","article-title":"A reinforcement learning path planning approach for range-only underwater target localization with autonomous vehicles","volume":"47","author":"Masmitja","year":"2022","journal-title":"IEEE J. Ocean. Eng."},{"key":"ref_28","first-page":"24","article-title":"Autonomous tracking and obstacle avoidance for UAV swarm based on decoupled MADDPG","volume":"40","author":"Wen","year":"2022","journal-title":"Flight Dyn."},{"key":"ref_29","first-page":"449","article-title":"UAV Path Planning Based on Multi-Agent Deep Reinforcement Learning","volume":"49","author":"Si","year":"2023","journal-title":"J. Beijing Univ. Technol."},{"key":"ref_30","unstructured":"Galilici, M., Martin, M., and Masmitja, I. (June, January 29). TransfQMix: Transformers for Leveraging the Graph Structure of Multi-Agent Reinforcement Learning Problems. Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems (AAMAS), London, UK."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"2397","DOI":"10.1109\/TAES.2013.6621824","article-title":"UAV path planning in a dynamic environment via partially observable Markov decision process","volume":"49","author":"Ragi","year":"2013","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"ref_32","first-page":"457","article-title":"Autonomous coordination saturation attacks method for loitering munitions in urban scenarios based on reinforcement learning","volume":"9","author":"Zhang","year":"2023","journal-title":"J. Command Control"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1109\/TCCN.2020.3027695","article-title":"Multi-agent deep reinforcement learning-based trajectory planning for multi-UAV assisted mobile edge computing","volume":"7","author":"Wang","year":"2020","journal-title":"IEEE Trans. Cogn. Commun. Netw."},{"key":"ref_34","first-page":"679","article-title":"A Markovian decision process","volume":"6","author":"Bellman","year":"1957","journal-title":"J. Math. Mech."},{"key":"ref_35","unstructured":"Bertsekas, D. (2019). Reinforcement Learning and Optimal Control, Athena Scientific."},{"key":"ref_36","first-page":"739","article-title":"Cooperative planning for an unmanned combat aerial vehicle fleet using reinforcement learning","volume":"18","author":"Yuksek","year":"2021","journal-title":"J. Aerosp. Inf. Syst."},{"key":"ref_37","unstructured":"Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. (2018, January 10\u201315). Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"479","DOI":"10.1016\/j.cja.2020.05.011","article-title":"UAV Navigation in High Dynamic Environments: A Deep Reinforcement Learning Approach","volume":"34","author":"Guo","year":"2021","journal-title":"Chin. J. Aeronaut."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/17\/11\/1970\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,18]],"date-time":"2025-11-18T05:18:23Z","timestamp":1763443103000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/17\/11\/1970"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,14]]},"references-count":38,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2025,11]]}},"alternative-id":["sym17111970"],"URL":"https:\/\/doi.org\/10.3390\/sym17111970","relation":{},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,14]]}}}