{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T21:35:42Z","timestamp":1775079342429,"version":"3.50.1"},"reference-count":24,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2020,10,5]],"date-time":"2020-10-05T00:00:00Z","timestamp":1601856000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100005073","name":"Agency for Defense Development","doi-asserted-by":"publisher","award":["-"],"award-info":[{"award-number":["-"]}],"id":[{"id":"10.13039\/501100005073","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Although various unmanned aerial vehicle (UAV)-assisted routing protocols have been proposed for vehicular ad hoc networks, few studies have investigated load balancing algorithms to accommodate future traffic growth and deal with complex dynamic network environments simultaneously. In particular, owing to the extended coverage and clear line-of-sight relay link on a UAV relay node (URN), the possibility of a bottleneck link is high. To prevent problems caused by traffic congestion, we propose Q-learning based load balancing routing (Q-LBR) through a combination of three key techniques, namely, a low-overhead technique for estimating the network load through the queue status obtained from each ground vehicular node by the URN, a load balancing scheme based on Q-learning and a reward control function for rapid convergence of Q-learning. Through diverse simulations, we demonstrate that Q-LBR improves the packet delivery ratio, network utilization and latency by more than 8, 28 and 30%, respectively, compared to the existing protocol.<\/jats:p>","DOI":"10.3390\/s20195685","type":"journal-article","created":{"date-parts":[[2020,10,5]],"date-time":"2020-10-05T10:43:48Z","timestamp":1601894628000},"page":"5685","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":27,"title":["Q-LBR: Q-Learning Based Load Balancing Routing for UAV-Assisted VANET"],"prefix":"10.3390","volume":"20","author":[{"given":"Bong-Soo","family":"Roh","sequence":"first","affiliation":[{"name":"Agency for Defense Development, Daejeon 34186, Korea"},{"name":"Department of Computer Science and Engineering, Chungnam National University, Daejeon 34134, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Myoung-Hun","family":"Han","sequence":"additional","affiliation":[{"name":"Agency for Defense Development, Daejeon 34186, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jae-Hyun","family":"Ham","sequence":"additional","affiliation":[{"name":"Agency for Defense Development, Daejeon 34186, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8366-3533","authenticated-orcid":false,"given":"Ki-Il","family":"Kim","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, Chungnam National University, Daejeon 34134, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,10,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Kashyap, A., Ghose, D., Menon, P.P., Sujit, P., and Das, K. (2019, January 11\u201314). UAV aided dynamic routing of resources in a flood scenario. Proceedings of the 2019 International Conference on Unmanned Aircraft Systems (ICUAS), Atlanta, GA, USA.","DOI":"10.1109\/ICUAS.2019.8798357"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Zeng, F., Zhang, R., Cheng, X., and Yang, L. (2018, January 20\u201324). UAV-assisted data dissemination scheduling in VANETs. Proceedings of the 2018 IEEE International Conference on Communications (ICC), Kansas City, MO, USA.","DOI":"10.1109\/ICC.2018.8422219"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Cheng, C.-M., Hsiao, P.-H., Kung, H.T., and Vlah, D. (2007, January 11\u201315). Maximizing throughput of UAV-relaying networks with the load-carry-and-deliver paradigm. Proceedings of the 2007 IEEE Wireless Communications and Networking Conference, Kowloon, China.","DOI":"10.1109\/WCNC.2007.805"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1016\/j.comcom.2017.04.001","article-title":"Intelligent UAV-assisted routing protocol for urban VANETs","volume":"107","author":"Oubbati","year":"2017","journal-title":"Comput. Commun."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"e4104","DOI":"10.1002\/dac.4104","article-title":"U2RV: UAV-assisted reactive routing protocol for VANETs","volume":"33","author":"Oubbati","year":"2019","journal-title":"Int. J. Commun. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/JIOT.2020.2971645","article-title":"Multi-UAV Enabled Load-Balance Mobile Edge Computing for IoT Networks","volume":"7","author":"Yang","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Guo, Y., Li, X., Yousefi\u2019Zadeh, H., and Jafarkhani, H. (2012, January 1\u20134). UAV-aided cross-layer routing for MANETs. Proceedings of the 2012 IEEE Wireless Communications and Networking Conference (WCNC), Paris, France.","DOI":"10.1109\/WCNC.2012.6214304"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"73774","DOI":"10.1109\/ACCESS.2018.2884708","article-title":"A hierarchical routing scheme with load balancing in software defined vehicular ad hoc networks","volume":"6","author":"Gao","year":"2018","journal-title":"IEEE Access."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Yao, H., Yuan, X., Zhang, P., Wang, J., Jiang, C., and Guizani, M. (2019, January 24\u201328). A machine learning approach of load balance routing to support next-generation wireless networks. Proceedings of the 2019 15th International Wireless Communications & Mobile Computing Conference (IWCMC), Tangier, Morocco.","DOI":"10.1109\/IWCMC.2019.8766546"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"133653","DOI":"10.1109\/ACCESS.2019.2941229","article-title":"Q-Learning algorithms: A comprehensive classification and applications","volume":"7","author":"Jang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Simon, P. (2013). Too Big to Ignore: The Business Case for Big Data, Wiley.","DOI":"10.1002\/9781119204039"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"55916","DOI":"10.1109\/ACCESS.2019.2913776","article-title":"Reinforcement learning based routing in networks: Review and classification of approaches","volume":"7","author":"Mammeri","year":"2019","journal-title":"IEEE Access"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"445","DOI":"10.1038\/nature14540","article-title":"Reinforcement learning improves behavior from evaluative feedback","volume":"521","author":"Littman","year":"2015","journal-title":"Nature"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2258","DOI":"10.1109\/LCOMM.2017.2656879","article-title":"QGeo: Q-Learning-based geographic ad hoc routing protocol for unmanned robotic networks","volume":"21","author":"Jung","year":"2017","journal-title":"IEEE Commun. Lett."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Jafarzadeh, S.Z., and Yaghmaee, M.H. (2014, January 4\u20137). Design of energy-aware QoS routing protocol in wireless sensor networks using reinforcement learning. Proceedings of the 2014 IEEE 27th Canadian Conference on Electrical and Computer Engineering (CCECE), Toronto, ON, Canada.","DOI":"10.1109\/CCECE.2014.6900988"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Dong, S., Agrawal, P., and Sivalingam, K.M. (2007, January 26\u201330). Reinforcement learning based geographic routing protocol for uwb wireless sensor network. Proceedings of the IEEE GLOBECOM 2007\u2014IEEE Global Telecommunications Conference, Washington, DC, USA.","DOI":"10.1109\/GLOCOM.2007.127"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1007\/s11277-020-07181-w","article-title":"Q-learning-based fuzzy logic for multi-objective routing algorithm in flying ad hoc networks","volume":"113","author":"Yang","year":"2020","journal-title":"Wirel. Pers. Commun."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Al-Hourani, A., Kandeepan, S., and Jamalipour, A. (2014, January 8\u201312). Modeling air-to-ground path loss for low altitude platforms in urban environments. Proceedings of the 2014 IEEE Global Communications Conference, Austin, TX, USA.","DOI":"10.1109\/GLOCOM.2014.7037248"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1647","DOI":"10.1109\/LCOMM.2016.2578312","article-title":"Efficient deployment of multiple unmanned aerial vehicles for optimal wireless coverage","volume":"20","author":"Mozaffari","year":"2016","journal-title":"IEEE Commun. Lett."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"132806","DOI":"10.1109\/ACCESS.2019.2941727","article-title":"Joint Resource Allocation and Trajectory Control for UAV-Enabled Vehicular Communications","volume":"7","author":"Lijun","year":"2019","journal-title":"IEEE Access."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Jiang, D., and Delgrossi, L. (2008, January 11\u201314). IEEE 802.11p: Towards an international standard for wireless access in vehicular environments. Proceedings of the VTC Spring 2008\u2014IEEE Vehicular Technology Conference, Singapore.","DOI":"10.1109\/VETECS.2008.458"},{"key":"ref_22","unstructured":"Ahmed, A., Sidi-Mohammed, S., Samira, M., Hichem, S., and Mohamed-Ayoub, M. (2018). Efficient Data Processing In Software-Defined Uav-Assisted Vehicular Networks: A Sequential Game Approach, Springer Wireless Personal Comm."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Jobaer, S., Zhang, Y., Hussain, M.A.I., and Ahmed, F. (2020). UAV-assisted hybrid scheme for urban road safety based on VANETs. Electronics, 9.","DOI":"10.3390\/electronics9091499"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1016\/j.knosys.2019.03.018","article-title":"A novel multi-step Q-learning method to improve data efficiency for deep reinforcement learning","volume":"175","author":"Yuan","year":"2019","journal-title":"Knowl. Based Syst."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/19\/5685\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:16:42Z","timestamp":1760177802000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/19\/5685"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,5]]},"references-count":24,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2020,10]]}},"alternative-id":["s20195685"],"URL":"https:\/\/doi.org\/10.3390\/s20195685","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,10,5]]}}}