{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,9]],"date-time":"2026-03-09T22:10:57Z","timestamp":1773094257304,"version":"3.50.1"},"reference-count":45,"publisher":"MDPI AG","issue":"13","license":[{"start":{"date-parts":[[2022,7,3]],"date-time":"2022-07-03T00:00:00Z","timestamp":1656806400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Science Foundation of Guangxi Province","award":["2022GXNSFDA035070"],"award-info":[{"award-number":["2022GXNSFDA035070"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Unmanned aerial vehicles (UAVs) are important equipment for efficiently executing search and rescue missions in disaster or air-crash scenarios. Each node can communicate with the others by a routing protocol in UAV ad hoc networks (UANETs). However, UAV routing protocols are faced with the challenges of high mobility and limited node energy, which hugely lead to unstable link and sparse network topology due to premature node death. Eventually, this severely affects network performance. In order to solve these problems, we proposed the deep-reinforcement-learning-based geographical routing protocol of considering link stability and energy prediction (DSEGR) for UANETs. First of all, we came up with the link stability evaluation indicator and utilized the autoregressive integrated moving average (ARIMA) model to predict the residual energy of neighbor nodes. Then, the packet forward process was modeled as a Markov Decision Process, and according to a deep double Q network with prioritized experience replay to learn the routing-decision process. Meanwhile, a reward function was designed to obtain a better convergence rate, and the analytic hierarchy process (AHP) was used to analyze the weights of the considered factors in the reward function. Finally, to verify the effectiveness of DSEGR, we conducted simulation experiments to analyze network performance. The simulation results demonstrate that our proposed routing protocol remarkably outperforms others in packet delivery ratio and has a faster convergence rate.<\/jats:p>","DOI":"10.3390\/s22135020","type":"journal-article","created":{"date-parts":[[2022,7,4]],"date-time":"2022-07-04T23:38:55Z","timestamp":1656977935000},"page":"5020","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["DDQN with Prioritized Experience Replay-Based Optimized Geographical Routing Protocol of Considering Link Stability and Energy Prediction for UANET"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9865-5555","authenticated-orcid":false,"given":"Yanan","family":"Zhang","sequence":"first","affiliation":[{"name":"School of Information and Communications, Guilin University of Electronic Technology, Guilin 541004, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hongbing","family":"Qiu","sequence":"additional","affiliation":[{"name":"School of Information and Communications, Guilin University of Electronic Technology, Guilin 541004, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,7,3]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"606","DOI":"10.1109\/TIE.2018.2823658","article-title":"Kalman filter-based large-scale wildfire monitoring with a system of UAVs","volume":"66","author":"Lin","year":"2018","journal-title":"IEEE T. Ind. Electron."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"9063232","DOI":"10.1155\/2019\/9063232","article-title":"Optimization of base location and patrol routes for unmanned aerial vehicles in border intelligence, surveillance, and reconnaissance","volume":"2019","author":"Liu","year":"2019","journal-title":"J. Adv. Transport."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Saif, A., Dimyati, K., Noordin, K.A., Shah, N.S.M., Abdullah, Q., and Mukhlif, F. (2020, January 9). Unmanned aerial vehicles for post-disaster communication networks. Proceedings of the 2020 IEEE 10th International Conference on System Engineering and Technology (ICSET), Shah Alam, Malaysia.","DOI":"10.1109\/ICSET51301.2020.9265369"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"105100","DOI":"10.1109\/ACCESS.2019.2932119","article-title":"Unmanned aerial vehicles in agriculture: A review of perspective of platform, control, and applications","volume":"7","author":"Kim","year":"2019","journal-title":"IEEE Access"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Haider, S.K., Jiang, A., Almogren, A., Rehman, A.U., Ahmed, A., Khan, W.U., and Hamam, H. (2021). Energy efficient UAV flight path model for cluster head selection in next-generation wireless sensor networks. Sensors, 21.","DOI":"10.3390\/s21248445"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Peng, J., Gao, H., Liu, L., Li, N., and Xu, X. (2020, January 14\u201316). Tbm: An efficient trajectory-based multicast routing protocol for sparse UAV networks. Proceedings of the 2020 IEEE 22nd International Conference on High Performance Computing and Communications; IEEE 18th International Conference on Smart City; IEEE 6th International Conference on Data Science and Systems (HPCC\/SmartCity\/DSS), Cuvu, Fiji.","DOI":"10.1109\/HPCC-SmartCity-DSS50907.2020.00115"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"302","DOI":"10.23919\/JCC.2022.05.005","article-title":"Adaptive 3d routing protocol for flying ad hoc networks based on prediction-driven q-learning","volume":"19","author":"Zhang","year":"2022","journal-title":"China Commun."},{"key":"ref_8","unstructured":"Jacquet, P., Muhlethaler, P., Clausen, T., Laouiti, A., Qayyum, A., and Viennot, L. (2001, January 30). Optimized link state routing protocol for ad hoc networks. Proceedings of the IEEE International Multi Topic Conference, 2001. IEEE INMIC 2001. Technology for the 21st Century, Lahore, Pakistan."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Perkins, C.E., and Royer, E.M. (1999, January 25\u201326). Ad-hoc on-demand distance vector routing. Proceedings of the WMCSA\u201999. Second IEEE Workshop on Mobile Computing Systems and Applications, New Orleans, LA, USA.","DOI":"10.1109\/MCSA.1999.749281"},{"key":"ref_10","unstructured":"Sinha, P., Krishnamurthy, S.V., and Dao, S. (2000, January 23\u201328). Scalable unidirectional routing with zone routing protocol (zrp) extensions for mobile ad-hoc networks. Proceedings of the 2000 IEEE Wireless Communications and Networking Conference. Conference Record (Cat. No. 00TH8540), Chicago, IL, USA."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Karp, B., and Kung, H.-T. (2000, January 6\u201311). GPSR: Greedy perimeter stateless routing for wireless networks. Proceedings of the 6th Annual International Conference on Mobile Computing and Networking, Boston, MA, USA.","DOI":"10.1145\/345910.345953"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2109","DOI":"10.1109\/TWC.2017.2789293","article-title":"Joint trajectory and communication design for multi-UAV enabled wireless networks","volume":"17","author":"Wu","year":"2018","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Lin, L., Sun, Q., Wang, S., and Yang, F. (2012, January 3\u20137). A geographic mobility prediction routing protocol for ad hoc UAV network. Proceedings of the 2012 IEEE Globecom Workshops, Anaheim, CA, USA.","DOI":"10.1109\/GLOCOMW.2012.6477824"},{"key":"ref_14","first-page":"941","article-title":"Genetic algorithm routing protocol for mobile ad hoc network","volume":"68","author":"Alsaqour","year":"2021","journal-title":"Cmc-Comput. Mater. Contin."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Biradar, A., and Thool, R.C. (2014, January 17\u201319). Reliable genetic algorithm based intelligent routing for manet. Proceedings of the 2014 World Congress on Computer Applications and Information Systems (WCCAIS), Hammamet, Tunisia.","DOI":"10.1109\/WCCAIS.2014.6916649"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Ramamoorthy, R., and Thangavelu, M. (2021). An enhanced hybrid ant colony optimization routing protocol for vehicular ad-hoc networks. J. Ambient. Humaniz. Comput., 1\u201332.","DOI":"10.1007\/s12652-021-03176-y"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"4007","DOI":"10.1007\/s12652-020-01767-9","article-title":"Multi constraints applied energy efficient routing technique based on ant colony optimization used for disaster resilient location detection in mobile ad-hoc network","volume":"12","author":"Malar","year":"2021","journal-title":"J. Ambient. Humaniz. Comput."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Shilova, Y., Kavalerov, M., and Bezukladnikov, I. (2016, January 2\u20133). Full echo q-routing with adaptive learning rates: A reinforcement learning approach to network routing. Proceedings of the 2016 IEEE NW Russia Young Researchers in Electrical and Electronic Engineering Conference (EIConRusNW), St. Petersburg, Russia.","DOI":"10.1109\/EIConRusNW.2016.7448188"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Liu, J., Wang, Q., He, C., and Xu, Y. (2020, January 16\u201319). Ardeep: Adaptive and reliable routing protocol for mobile robotic networks with deep reinforcement learning. Proceedings of the 2020 IEEE 45th Conference on Local Computer Networks (LCN), Sydney, NSW, Australia.","DOI":"10.1109\/LCN48667.2020.9314848"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1985","DOI":"10.1109\/JIOT.2021.3089759","article-title":"A Q-learning-based topology-aware routing protocol for flying ad hoc networks","volume":"9","author":"Arafat","year":"2022","journal-title":"IEEE Internet Things"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"304","DOI":"10.1016\/j.comcom.2019.11.011","article-title":"Qmr: Q-learning based multi-objective optimization routing protocol for flying ad hoc networks","volume":"150","author":"Liu","year":"2020","journal-title":"Comput. Commun."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Minhas, H.I., Ahmad, R., Ahmed, W., Waheed, M., Alam, M.M., and Gul, S.T. (2021). A reinforcement learning routing protocol for UAV aided public safety networks. Sensors, 21.","DOI":"10.3390\/s21124121"},{"key":"ref_23","first-page":"92212","article-title":"Performance analysis of routing protocols for UAV communication networks","volume":"8","author":"Tan","year":"2020","journal-title":"IEEE Access"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1690","DOI":"10.1109\/TVT.2015.2414819","article-title":"Dynamic routing for flying ad hoc networks","volume":"65","author":"Rosati","year":"2015","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_25","unstructured":"Zheng, Y., Wang, Y., Li, Z., Dong, L., Jiang, Y., and Zhang, H. (2014, January 15\u201317). A mobility and load aware olsr routing protocol for UAV mobile ad-hoc networks. Proceedings of the 2014 International Conference on Information and Communications Technologies (ICT 2014), Nanjing, China."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"11297","DOI":"10.1109\/ACCESS.2020.2965616","article-title":"The novel mobility models based on spiral line for aerial backbone networks","volume":"8","author":"He","year":"2020","journal-title":"IEEE Access"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1023\/A:1019106118419","article-title":"Location-aided routing (lar) in mobile ad hoc networks","volume":"6","author":"Ko","year":"2000","journal-title":"Wirel. Netw."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Jing, Z., Binbin, S., Wei, F., and Chengmin, W. (2021, January 23\u201325). Gpsr protocol perimeter forwarding optimization algorithm based on game model in UAV network. Proceedings of the 2021 International Conference on Computer Technology and Media Convergence Design (CTMCD), Sanya, China.","DOI":"10.1109\/CTMCD53128.2021.00039"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Costa, R., Rosario, D., Cerqueira, E., and Santos, A. (2014, January 12\u201314). Enhanced connectivity for robust multimedia transmission in UAV networks. Proceedings of the 2014 IFIP Wireless Days (WD), Rio de Janeiro, Brazil.","DOI":"10.1109\/WD.2014.7020797"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Urquiza-Aguiar, L., Tripp-Barba, C., and Igartua, M.A. (2016). A geographical heuristic routing protocol for vanets. Sensors, 16.","DOI":"10.3390\/s16101567"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"14451","DOI":"10.1007\/s11042-015-3240-y","article-title":"Ant colony optimization based polymorphism-aware routing algorithm for ad hoc UAV network","volume":"75","author":"Yu","year":"2016","journal-title":"Multimed. Tools Appl."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Zhao, B., and Ding, Q. (2019, January 29\u201331). Route discovery in flying ad-hoc network based on bee colony algorithm. Proceedings of the 2019 IEEE International Conference on Artificial Intelligence and Computer Applications (ICAICA), Dalian, China.","DOI":"10.1109\/ICAICA.2019.8873436"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Patel, J., and El-Ocla, H. (2021). Energy efficient routing protocol in sensor networks using genetic algorithm. Sensors, 21.","DOI":"10.3390\/s21217060"},{"key":"ref_34","first-page":"671","article-title":"Packet routing in dynamically changing networks: A reinforcement learning approach","volume":"Volume 6","author":"Boyan","year":"1993","journal-title":"Proceedings of the 7th NIPS Conference"},{"key":"ref_35","unstructured":"Littman, M., and Boyan, J. (1993, January 18\u201320). A Distributed Reinforcement learning scheme for network routing. Proceedings of the International Workshop on Applications of Neural Networks to Telecommunications, Princeton, NJ, USA."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Mutombo, V.K., Shin, S.Y., and Hong, J. (2021, January 22\u201326). Ebr-rl: Energy balancing routing protocol based on reinforcement learning for wsn. Proceedings of the 36th Annual ACM Symposium on Applied Computing, Virtual, Korea.","DOI":"10.1145\/3412841.3442063"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"108379","DOI":"10.1016\/j.comnet.2021.108379","article-title":"Q-FANET: Improved q-learning based routing protocol for FANETs","volume":"198","author":"Kunst","year":"2021","journal-title":"Comput. Netw."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Li, R., Li, F., Li, X., and Wang, Y. (2014, January 5\u20137). Qgrid: Q-learning based routing protocol for vehicular ad hoc networks. Proceedings of the 2014 IEEE 33rd International Performance Computing and Communications Conference (IPCCC), Austin, TX, USA.","DOI":"10.1109\/PCCC.2014.7017079"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"2258","DOI":"10.1109\/LCOMM.2017.2656879","article-title":"Qgeo: Q-learning-based geographic ad hoc routing protocol for unmanned robotic networks","volume":"21","author":"Jung","year":"2017","journal-title":"IEEE Commun. Lett."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"136","DOI":"10.1109\/MCOM.2017.1700323","article-title":"Adaptive communication protocols in flying ad hoc network","volume":"56","author":"Zheng","year":"2018","journal-title":"IEEE Commun. Mag."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"2223","DOI":"10.1109\/TNSE.2021.3085514","article-title":"Fully-echoed q-routing with simulated annealing inference for flying adhoc networks","volume":"8","author":"Afghah","year":"2021","journal-title":"IEEE Trans. Netw. Sci. Eng."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Lyu, N., Song, G., Yang, B., and Cheng, Y. (2018, January 27\u201330). Qngpsr: A q-network enhanced geographic ad-hoc routing protocol based on gpsr. Proceedings of the 2018 IEEE 88th Vehicular Technology Conference (VTC-Fall), Chicago, IL, USA.","DOI":"10.1109\/VTCFall.2018.8690651"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1308","DOI":"10.1631\/FITEE.1900401","article-title":"A traffic-aware q-network enhanced routing protocol based on gpsr for unmanned aerial vehicle ad-hoc networks","volume":"21","author":"Chen","year":"2020","journal-title":"Front. Inf. Technol. Electron. Eng."},{"key":"ref_44","unstructured":"Hasselt, H.V., Guez, A., and Silver, D. (2016, January 12\u201317). Deep reinforcement learning with double q-learning. Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA."},{"key":"ref_45","unstructured":"Schaul, T., Quan, J., Antonoglou, I., and Silver, D. (2015). Prioritized experience replay. arXiv."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/13\/5020\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:42:14Z","timestamp":1760139734000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/13\/5020"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,3]]},"references-count":45,"journal-issue":{"issue":"13","published-online":{"date-parts":[[2022,7]]}},"alternative-id":["s22135020"],"URL":"https:\/\/doi.org\/10.3390\/s22135020","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,3]]}}}