{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T22:19:30Z","timestamp":1775081970807,"version":"3.50.1"},"reference-count":24,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2025,10,16]],"date-time":"2025-10-16T00:00:00Z","timestamp":1760572800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ministry of Digital Development, Communications and Mass Media of the Russian Federation","award":["071-03-2025-005"],"award-info":[{"award-number":["071-03-2025-005"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>With the arrival of sixth-generation (6G) wireless systems comes radical potential for the deployment of autonomous aerial vehicle (AAV) swarms in mission-critical applications, ranging from disaster rescue to intelligent transportation. However, 6G-supporting AAV environments present challenges such as dynamic three-dimensional topologies, highly restrictive energy constraints, and extremely low latency demands, which substantially degrade the efficiency of conventional routing protocols. To this end, this work presents a Q-learning-enhanced ad hoc on-demand distance vector (QL-AODV). This intelligent routing protocol uses reinforcement learning within the AODV protocol to support adaptive, data-driven route selection in highly dynamic aerial networks. QL-AODV offers four novelties, including a multipath route set collection methodology that retains up to ten candidate routes for each destination using an extended route reply (RREP) waiting mechanism, a more detailed RREP message format with cumulative node buffer usage, enabling informed decision-making, a normalized 3D state space model recording hop count, average buffer occupancy, and peak buffer saturation, optimized to adhere to aerial network dynamics, and a light-weighted distributed Q-learning approach at the source node that uses an \u03b5-greedy policy to balance exploration and exploitation. Large-scale simulations conducted with NS-3.34 for various node densities and mobility conditions confirm the better performance of QL-AODV compared to conventional AODV. In high-mobility environments, QL-AODV offers up to 9.8% improvement in packet delivery ratio and up to 12.1% increase in throughput, while remaining persistently scalable for various network sizes. The results prove that QL-AODV is a reliable, scalable, and intelligent routing method for next-generation AAV networks that will operate in intensive environments that are expected for 6G.<\/jats:p>","DOI":"10.3390\/fi17100473","type":"journal-article","created":{"date-parts":[[2025,10,17]],"date-time":"2025-10-17T07:33:50Z","timestamp":1760686430000},"page":"473","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["QL-AODV: Q-Learning-Enhanced Multi-Path Routing Protocol for 6G-Enabled Autonomous Aerial Vehicle Networks"],"prefix":"10.3390","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1610-9612","authenticated-orcid":false,"given":"Abdelhamied A.","family":"Ateya","sequence":"first","affiliation":[{"name":"EIAS Data Science Lab, College of Computer and Information Sciences, Prince Sultan University, Riyadh 11586, Saudi Arabia"},{"name":"Department of Electronics and Communications Engineering, Zagazig University, Zagazig 44519, Egypt"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-0055-1526","authenticated-orcid":false,"given":"Nguyen Duc","family":"Tu","sequence":"additional","affiliation":[{"name":"Department of Telecommunication Networks and Data Transmission, The Bonch-Bruevich Saint-Petersburg State University of Telecommunications, 193232 Saint Petersburg, Russia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0213-8145","authenticated-orcid":false,"given":"Ammar","family":"Muthanna","sequence":"additional","affiliation":[{"name":"Department of Telecommunication Networks and Data Transmission, The Bonch-Bruevich Saint-Petersburg State University of Telecommunications, 193232 Saint Petersburg, Russia"}]},{"given":"Andrey","family":"Koucheryavy","sequence":"additional","affiliation":[{"name":"Department of Telecommunication Networks and Data Transmission, The Bonch-Bruevich Saint-Petersburg State University of Telecommunications, 193232 Saint Petersburg, Russia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0538-8430","authenticated-orcid":false,"given":"Dmitry","family":"Kozyrev","sequence":"additional","affiliation":[{"name":"Department of Probability Theory and Cybersecurity, Peoples\u2019 Friendship University of Russia Named After Patrice Lumumba (RUDN University), 117198 Moscow, Russia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5303-818X","authenticated-orcid":false,"given":"J\u00e1nos","family":"Sztrik","sequence":"additional","affiliation":[{"name":"Department of Informatics Systems and Networks, Faculty of Informatics, University of Debrecen, Egyetem ter 1, 4032 Debrecen, Hungary"}]}],"member":"1968","published-online":{"date-parts":[[2025,10,16]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Ning, Z., Li, T., Wu, Y., Wang, X., Wu, Q., Yu, F.R., and Guo, S. (2025). 6G Communication New Paradigm: The Integration of Unmanned Aerial Vehicles and Intelligent Reflecting Surfaces. IEEE Commun. Surv. Tutor., 1.","DOI":"10.1109\/COMST.2025.3526251"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Du, Z., Luo, C., Min, G., Wu, J., Luo, C., Pu, J., and Li, S. (2025). A Survey on Autonomous and Intelligent Swarms of Uncrewed Aerial Vehicles (UAVs). IEEE Trans. Intell. Transp. Syst., 1\u201324.","DOI":"10.1109\/TITS.2025.3569500"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Wheeb, A.H., Nordin, R., Samah, A.A., Alsharif, M.H., and Khan, M.A. (2021). Topology-Based Routing Protocols and Mobility Models for Flying Ad Hoc Networks: A Contemporary Review and Future Research Directions. Drones, 6.","DOI":"10.3390\/drones6010009"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1109\/MNET.124.2200241","article-title":"6G Network AI Architecture for Everyone-Centric Customized Services","volume":"37","author":"Yang","year":"2023","journal-title":"IEEE Netw."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"6611","DOI":"10.1109\/TVT.2022.3232815","article-title":"Learning to Routing in UAV Swarm Network: A Multi-Agent Reinforcement Learning Approach","volume":"72","author":"Wang","year":"2023","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"3778","DOI":"10.1109\/TAES.2022.3232322","article-title":"A Cross-Layer, Mobility, and Congestion-Aware Routing Protocol for UAV Networks","volume":"59","author":"Garg","year":"2023","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"66289","DOI":"10.1109\/ACCESS.2023.3290871","article-title":"A Fresh Look at Routing Protocols in Unmanned Aerial Vehicular Networks: A Survey","volume":"11","author":"Mansoor","year":"2023","journal-title":"IEEE Access"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"102790","DOI":"10.1016\/j.adhoc.2022.102790","article-title":"A Review of AI-Enabled Routing Protocols for UAV Networks: Trends, Challenges, and Future Outlook","volume":"130","author":"Razi","year":"2022","journal-title":"Ad Hoc Netw."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2483","DOI":"10.32604\/csse.2023.032737","article-title":"3D Path Optimisation of Unmanned Aerial Vehicles Using Q Learning-Controlled GWO-AOA","volume":"45","author":"Sreelakshmy","year":"2023","journal-title":"Comput. Syst. Sci. Eng."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"e1079","DOI":"10.7717\/peerj-cs.1079","article-title":"Systematic Review on Modification to the Ad-Hoc on-Demand Distance Vector Routing Discovery Mechanics","volume":"8","author":"Alameri","year":"2022","journal-title":"PeerJ Comput. Sci."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"7786","DOI":"10.1109\/ACCESS.2023.3349208","article-title":"A Comprehensive Survey on 5G-and-beyond Networks with UAVs: Applications, Emerging Technologies, Regulatory Aspects, Research Trends and Challenges","volume":"12","author":"Banafaa","year":"2024","journal-title":"IEEE Access"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Gupta, V., Kumar Yadav, D., and Agarwal, M. (2024, January 15\u201316). Evaluation of Routing Protocol Performance for Enhanced Operations of Unmanned Aerial Vehicles (UAVs). Proceedings of the 2024 2nd International Conference on Device Intelligence, Computing and Communication Technologies (DICCT), Dehradun, India.","DOI":"10.1109\/DICCT61038.2024.10532952"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"155014771986639","DOI":"10.1177\/1550147719866392","article-title":"Latency and Energy-Efficient Multi-Hop Routing Protocol for Unmanned Aerial Vehicle Networks","volume":"15","author":"Ateya","year":"2019","journal-title":"Int. J. Distrib. Sens. Netw."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2250008","DOI":"10.1142\/S0218126622500086","article-title":"Performance Evaluation of AODV and DSR Routing Protocols for Flying Ad Hoc Network Using Highway Mobility Model","volume":"31","author":"Maakar","year":"2022","journal-title":"J. Circuits Syst. Comput."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"3327","DOI":"10.1109\/JSAC.2024.3492720","article-title":"Space-Air-Ground Integrated Wireless Networks for 6G: Basics, Key Technologies, and Future Trends","volume":"42","author":"Xiao","year":"2024","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Raj, K., Patel, S., and Shukla, A.N. (2024, January 24\u201328). Evaluating AODV Routing Protocol Performance in UAV Networks for Search-and-Rescue Operations. Proceedings of the 2024 15th International Conference on Computing Communication and Networking Technologies (ICCCNT), Kamand, India.","DOI":"10.1109\/ICCCNT61001.2024.10724498"},{"key":"ref_17","unstructured":"Ali, M.H., Ferdian, H., and Sari, R.F. (2024, January 12\u201313). Performance Improvement of Ad-Hoc on-Demand Distance Vector (AODV) Routing Protocol Using K-Means Clustering in Flying Ad-Hoc Network. Proceedings of the 2024 International Conference on Radar, Antenna, Microwave, Electronics, and Telecommunications (ICRAMET), Bandung, Indonesia."},{"key":"ref_18","first-page":"3671","article-title":"Q-Learning Based Routing Protocol for Congestion Avoidance","volume":"68","author":"Godfrey","year":"2021","journal-title":"Comput. Mater. Contin."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"El-Basioni, B.M.M. (2024). Intensive Study, Tuning and Modification of Reactive Routing Approach to Improve Flat FANET Performance in Data Collection Scenario. Sci. Rep., 14.","DOI":"10.1038\/s41598-024-72983-y"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Li, X., Bian, X., and Li, M. (2024). Routing Selection Algorithm for Mobile Ad Hoc Networks Based on Neighbor Node Density. Sensors, 24.","DOI":"10.3390\/s24020325"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Chandrasekar, V., Shanmugavalli, V., Mahesh, T.R., Shashikumar, R., Borah, N., Kumar, V.V., and Guluwadi, S. (2024). Secure Malicious Node Detection in Flying Ad-Hoc Networks Using Enhanced AODV Algorithm. Sci. Rep., 14.","DOI":"10.1038\/s41598-024-57480-6"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Dong, H., Yu, B., and Wu, W. (2025). Routing Protocol for Intelligent Unmanned Cluster Network Based on Node Energy Consumption and Mobility Optimization. Sensors, 25.","DOI":"10.3390\/s25020500"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Campanile, L., Gribaudo, M., Iacono, M., Marulli, F., and Mastroianni, M. (2020). Computer Network Simulation with Ns-3: A Systematic Literature Review. Electronics, 9.","DOI":"10.3390\/electronics9020272"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Xie, H., Zou, G., and Ma, L. (2022, January 4\u20136). A Hierarchical Routing Protocol Based on AODV for Unmanned Aerial Vehicle Swarm Network. Proceedings of the 2022 IEEE International Conference on Unmanned Systems (ICUS), Guangzhou, China.","DOI":"10.1109\/ICUS55513.2022.9987222"}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/17\/10\/473\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,17]],"date-time":"2025-10-17T07:47:13Z","timestamp":1760687233000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/17\/10\/473"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,16]]},"references-count":24,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2025,10]]}},"alternative-id":["fi17100473"],"URL":"https:\/\/doi.org\/10.3390\/fi17100473","relation":{},"ISSN":["1999-5903"],"issn-type":[{"value":"1999-5903","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,16]]}}}