{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T05:20:15Z","timestamp":1784092815493,"version":"3.55.0"},"reference-count":32,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2025,8,11]],"date-time":"2025-08-11T00:00:00Z","timestamp":1754870400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Natural Science Foundation of Hunan Province","award":["2025JJ70706"],"award-info":[{"award-number":["2025JJ70706"]}]},{"name":"Natural Science Foundation of Hunan Province","award":["25BSQD07"],"award-info":[{"award-number":["25BSQD07"]}]},{"name":"Doctoral Research Start-up Fund Project of Hunan University of Arts and Science","award":["2025JJ70706"],"award-info":[{"award-number":["2025JJ70706"]}]},{"name":"Doctoral Research Start-up Fund Project of Hunan University of Arts and Science","award":["25BSQD07"],"award-info":[{"award-number":["25BSQD07"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>Multi-objective Unmanned Aerial Vehicle (UAV) path planning in complex 3D environments presents a fundamental challenge requiring the simultaneous optimization of conflicting objectives such as path length, safety, altitude constraints, and smoothness. This study proposes a novel hybrid framework, termed QL-MOPSO, that integrates reinforcement learning with metaheuristic optimization through a three-stage hierarchical architecture. The framework employs Q-learning to generate a global guidance path in a discretized 2D grid environment using an eight-directional symmetric action space that embodies rotational symmetry at \u03c0\/4 intervals, ensuring uniform exploration capabilities and unbiased path planning. A crucial intermediate stage transforms the discrete 2D path into a 3D initial trajectory, bridging the gap between discrete learning and continuous optimization domains. The MOPSO algorithm then performs fine-grained refinement in continuous 3D space, guided by a novel Q-learning path deviation objective that ensures continuous knowledge transfer throughout the optimization process. Experimental results demonstrate that the symmetric action space design yields 20.6% shorter paths compared to asymmetric alternatives, while the complete QL-MOPSO framework achieves 5% path length reduction and significantly faster convergence compared to standard MOPSO. The proposed method successfully generates Pareto-optimal solutions that balance multiple objectives while leveraging the symmetry-aware guidance mechanism to avoid local optima and accelerate convergence, offering a robust solution for complex multi-objective UAV path planning problems.<\/jats:p>","DOI":"10.3390\/sym17081292","type":"journal-article","created":{"date-parts":[[2025,8,11]],"date-time":"2025-08-11T09:59:13Z","timestamp":1754906353000},"page":"1292","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Reinforcement Learning-Guided Particle Swarm Optimization for Multi-Objective Unmanned Aerial Vehicle Path Planning"],"prefix":"10.3390","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-8035-5626","authenticated-orcid":false,"given":"Wuke","family":"Li","sequence":"first","affiliation":[{"name":"School of Computer and Electrical Engineering, Hunan University of Arts and Science, Changde 415000, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ying","family":"Xiong","sequence":"additional","affiliation":[{"name":"School of Information Technology, Zhangjiajie Institute of Aeronautical Engineering, Zhangjiajie 427000, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0396-4836","authenticated-orcid":false,"given":"Qi","family":"Xiong","sequence":"additional","affiliation":[{"name":"School of Automation Science and Engineering, Xi\u2019an Jiaotong University, Xi\u2019an 710049, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,8,11]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Yang, Y., Fu, Y., Xin, R., Feng, W., and Xu, K. (2025). Multi-UAV Trajectory Planning Based on a Two-Layer Algorithm Under Four-Dimensional Constraints. Drones, 9.","DOI":"10.20944\/preprints202505.2202.v1"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Chen, H., Liang, Y., and Meng, X. (2023). A UAV Path Planning Method for Building Surface Information Acquisition Utilizing Opposition-Based Learning Artificial Bee Colony Algorithm. Remote Sens., 15.","DOI":"10.3390\/rs15174312"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Lv, F., Jian, Y., Yuan, K., and Lu, Y. (2025). Unmanned Aerial Vehicle Path Planning Method Based on Improved Dung Beetle Optimization Algorithm. Symmetry, 17.","DOI":"10.3390\/sym17030367"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Haidar Ahmad, A., Zahwe, O., Nasser, A., and Clement, B. (2024). Path Planning for Unmanned Aerial Vehicles in Dynamic Environments: A Novel Approach Using Improved A* and Grey Wolf Optimizer. World Electr. Veh. J., 15.","DOI":"10.3390\/wevj15110531"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Li, W., Zhang, K., Xiong, Q., and Chen, X. (2025). Three-Dimensional Unmanned Aerial Vehicle Path Planning in Simulated Rugged Mountainous Terrain Using Improved Enhanced Snake Optimizer (IESO). World Electr. Veh. J., 16.","DOI":"10.3390\/wevj16060295"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"103520","DOI":"10.1016\/j.adhoc.2024.103520","article-title":"Multi-objective path planning for multi-UAV connectivity and area coverage","volume":"160","author":"Yanmaz","year":"2024","journal-title":"Ad Hoc Netw."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"121862","DOI":"10.1016\/j.eswa.2023.121862","article-title":"A novel multi-objective evolutionary algorithm with a two-fold constraint-handling mechanism for multiple UAV path planning","volume":"238","author":"Zhang","year":"2024","journal-title":"Expert Syst. Appl."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"119977","DOI":"10.1016\/j.ins.2023.119977","article-title":"A multi-objective evolutionary algorithm based on dimension exploration and discrepancy evolution for UAV path planning problem","volume":"657","author":"Xu","year":"2024","journal-title":"Inf. Sci."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"108022","DOI":"10.1016\/j.cie.2022.108022","article-title":"A review of Pareto pruning methods for multi-objective optimization","volume":"167","author":"Petchrompo","year":"2022","journal-title":"Comput. Ind. Eng."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1007\/s10661-022-10590-y","article-title":"Path planning optimization in unmanned aerial vehicles using meta-heuristic algorithms: A systematic review","volume":"195","author":"Yahia","year":"2023","journal-title":"Environ. Monit. Assess."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Hooshyar, M., and Huang, Y.M. (2023). Meta-heuristic algorithms in UAV path planning optimization: A systematic review (2018\u20132022). Drones, 7.","DOI":"10.3390\/drones7120687"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"3924","DOI":"10.1109\/TAES.2023.3234455","article-title":"Heterogeneous mission planning for multiple UAV formations via metaheuristic algorithms","volume":"59","author":"Wu","year":"2023","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Xia, S., and Zhang, X. (2021). Constrained path planning for unmanned aerial vehicle in 3D terrain using modified multi-objective particle swarm optimization. Actuators, 10.","DOI":"10.3390\/act10100255"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"15217","DOI":"10.1007\/s10462-023-10526-z","article-title":"A comprehensive survey on NSGA-II for multi-objective optimization and applications","volume":"56","author":"Ma","year":"2023","journal-title":"Artif. Intell. Rev."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"995","DOI":"10.1007\/s11831-024-10178-4","article-title":"Multi-objective ant colony optimization","volume":"32","author":"Awadallah","year":"2025","journal-title":"Arch. Comput. Methods Eng."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"111418","DOI":"10.1016\/j.oceaneng.2022.111418","article-title":"A comparative study on Ant Colony Optimization algorithm approaches for solving multi-objective path planning problems in case of unmanned surface vehicles","volume":"255","author":"Ntakolia","year":"2022","journal-title":"Ocean. Eng."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"19723","DOI":"10.1007\/s00521-022-07704-5","article-title":"Recent advances in multi-objective grey wolf optimizer, its versions and applications","volume":"34","author":"Makhadmeh","year":"2022","journal-title":"Neural Comput. Appl."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Tang, R., Qi, L., Ye, S., Li, C., Ni, T., Guo, J., Liu, H., Li, Y., Zuo, D., and Shi, J. (2025). Three-Dimensional Path Planning for AUVs Based on Interval Multi-Objective Secretary Bird Optimization Algorithm. Symmetry, 17.","DOI":"10.3390\/sym17070993"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Xiong, Q., Zhang, X., He, S., and Shen, J. (2021). A fractional-order chaotic sparrow search algorithm for enhancement of long distance iris image. Mathematics, 9.","DOI":"10.3390\/math9212790"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"109075","DOI":"10.1016\/j.knosys.2022.109075","article-title":"Multi-objective particle swarm optimization with multi-mode collaboration based on reinforcement learning for path planning of unmanned air vehicles","volume":"250","author":"Zhang","year":"2022","journal-title":"Knowl.-Based Syst."},{"key":"ref_21","first-page":"34","article-title":"Research on robot motion state estimation based on deep reinforcement learning","volume":"35","author":"Zheng","year":"2023","journal-title":"J. Hunan Univ. Arts Sci. (Nat. Sci.)"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Lyu, L., Shen, Y., and Zhang, S. (2022, January 25\u201327). The advance of reinforcement learning and deep reinforcement learning. Proceedings of the IEEE International Conference on Electrical Engineering, Big Data and Algorithms (EEBDA), Changchun, China.","DOI":"10.1109\/EEBDA53927.2022.9744760"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"121240","DOI":"10.1016\/j.eswa.2023.121240","article-title":"Q-learning based system for path planning with unmanned aerial vehicles swarms in obstacle environments","volume":"235","author":"Rivero","year":"2024","journal-title":"Expert Syst. Appl."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Souto, A., Alfaia, R., Cardoso, E., Ara\u00fajo, J., and Franc\u00eas, C. (2023). UAV Path Planning Optimization Strategy: Considerations of Urban Morphology, Microclimate, and Energy Efficiency Using Q-Learning Algorithm. Drones, 7.","DOI":"10.3390\/drones7020123"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"110773","DOI":"10.1016\/j.asoc.2023.110773","article-title":"Q-learning-based unmanned aerial vehicle path planning with dynamic obstacle avoidance","volume":"147","author":"Sonny","year":"2023","journal-title":"Appl. Soft Comput."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"15391","DOI":"10.1109\/TVT.2023.3297837","article-title":"An adaptive conversion speed Q-learning algorithm for search and rescue UAV path planning in unknown environments","volume":"72","author":"Wu","year":"2023","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"20220667","DOI":"10.1098\/rspa.2022.0667","article-title":"Extending Q-learning to continuous and mixed strategy games based on spatial reciprocity","volume":"479","author":"Wang","year":"2023","journal-title":"Proc. R. Soc. A"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"e7","DOI":"10.1017\/S0269888921000163","article-title":"An online scalarization multi-objective reinforcement learning algorithm: TOPSIS Q-learning","volume":"37","author":"Mirzanejad","year":"2022","journal-title":"Knowl. Eng. Rev."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"119246","DOI":"10.1016\/j.eswa.2022.119246","article-title":"QQLMPA: A quasi-opposition learning and Q-learning based marine predators algorithm","volume":"213","author":"Zhao","year":"2023","journal-title":"Expert Syst. Appl."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"5605","DOI":"10.1007\/s11831-022-09778-9","article-title":"A comprehensive review on multi-objective optimization techniques: Past, present and future","volume":"29","author":"Sharma","year":"2022","journal-title":"Arch. Comput. Methods Eng."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"2285","DOI":"10.1007\/s11831-021-09663-x","article-title":"A review of multi-objective optimization: Methods and algorithms in mechanical engineering problems","volume":"29","author":"Pereira","year":"2022","journal-title":"Arch. Comput. Methods Eng."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"123803","DOI":"10.1016\/j.eswa.2024.123803","article-title":"OPSBC: A method to sort Pareto-optimal sets of solutions in multi-objective problems","volume":"250","author":"Dosantos","year":"2024","journal-title":"Expert Syst. Appl."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/17\/8\/1292\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:24:32Z","timestamp":1760034272000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/17\/8\/1292"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,11]]},"references-count":32,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2025,8]]}},"alternative-id":["sym17081292"],"URL":"https:\/\/doi.org\/10.3390\/sym17081292","relation":{},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,11]]}}}