{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,14]],"date-time":"2026-07-14T14:57:58Z","timestamp":1784041078911,"version":"3.55.0"},"reference-count":36,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2022,2,5]],"date-time":"2022-02-05T00:00:00Z","timestamp":1644019200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["2020R1A2C2008447"],"award-info":[{"award-number":["2020R1A2C2008447"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100010418","name":"Institute for Information and Communications Technology Promotion","doi-asserted-by":"publisher","award":["IITP-2021-2015-0-00742, IITP-2021-2020-0-01821"],"award-info":[{"award-number":["IITP-2021-2015-0-00742, IITP-2021-2020-0-01821"]}],"id":[{"id":"10.13039\/501100010418","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The applications of Unmanned Aerial Vehicles (UAVs) are rapidly growing in domains such as surveillance, logistics, and entertainment and require continuous connectivity with cellular networks to ensure their seamless operations. However, handover policies in current cellular networks are primarily designed for ground users, and thus are not appropriate for UAVs due to frequent fluctuations of signal strength in the air. This paper presents a novel handover decision scheme deploying Deep Reinforcement Learning (DRL) to prevent unnecessary handovers while maintaining stable connectivity. The proposed DRL framework takes the UAV state as an input for a proximal policy optimization algorithm and develops a Received Signal Strength Indicator (RSSI) based on a reward function for the online learning of UAV handover decisions. The proposed scheme is evaluated in a 3D-emulated UAV mobility environment where it reduces up to 76 and 73% of unnecessary handovers compared to greedy and Q-learning-based UAV handover decision schemes, respectively. Furthermore, this scheme ensures reliable communication with the UAV by maintaining the RSSI above \u221275 dBm more than 80% of the time.<\/jats:p>","DOI":"10.3390\/s22031200","type":"journal-article","created":{"date-parts":[[2022,2,6]],"date-time":"2022-02-06T20:40:18Z","timestamp":1644180018000},"page":"1200","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":30,"title":["Proactive Handover Decision for UAVs with Deep Reinforcement Learning"],"prefix":"10.3390","volume":"22","author":[{"given":"Younghoon","family":"Jang","sequence":"first","affiliation":[{"name":"Department of Electrical and Computer Engineering, Sungkyunkwan University, Suwon 16419, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6580-3232","authenticated-orcid":false,"given":"Syed M.","family":"Raza","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, Sungkyunkwan University, Suwon 16419, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2692-6883","authenticated-orcid":false,"given":"Moonseong","family":"Kim","sequence":"additional","affiliation":[{"name":"Department of IT Convergence Software, Seoul Theological University, Bucheon 14754, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hyunseung","family":"Choo","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, Sungkyunkwan University, Suwon 16419, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,2,5]]},"reference":[{"key":"ref_1","unstructured":"Abhay, K., Nadeem, A., and Mahima, M. (2017). Mobility Management in LTE Heterogeneous Networks, Springer."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1109\/MVT.2019.2917363","article-title":"Mobile Network-Connected Drones: Field Trials, Simulations, and Design Insights","volume":"14","author":"Lin","year":"2019","journal-title":"IEEE Veh. Technol. Mag."},{"key":"ref_3","first-page":"3120","article-title":"Adaptive Success Rate-based Sensor Relocation for IoT Applications","volume":"15","author":"Kim","year":"2021","journal-title":"KSII Trans. Internet Inf. Syst. (TIIS)"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Euler, S., Maattanen, H.L., Lin, X., Zou, Z., Bergstr\u00f6m, M., and Sedin, J. (2019, January 15\u201318). Mobility Support for Cellular Connected Unmanned Aerial Vehicles: Performance and Analysis. Proceedings of the 2019 IEEE Wireless Communications and Networking Conference (WCNC), Marrakesh, Morocco.","DOI":"10.1109\/WCNC.2019.8885820"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Huang, W., Zhang, H., and Zhou, M. (2019, January 8\u201311). Analysis of Handover Probability Based on Equivalent Model for 3D UAV Networks. Proceedings of the 2019 IEEE 30th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), Istanbul, Turkey.","DOI":"10.1109\/PIMRC.2019.8904320"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"4994","DOI":"10.1109\/TWC.2020.2988633","article-title":"Performance Characterization of Canonical Mobility Models in Drone Cellular Networks","volume":"19","author":"Banagar","year":"2020","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Chowdhury, M.M.U., Saad, W., and G\u00fcven\u00e7, I. (2020, January 7\u201311). Mobility Management for Cellular-Connected UAVs: A Learning-Based Approach. Proceedings of the 2020 IEEE International Conference on Communications Workshops (ICC Workshops), Dublin, Ireland.","DOI":"10.1109\/ICCWorkshops49005.2020.9145089"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Chen, Y., Lin, X., Khan, T., and Mozaffari, M. (2020, January 25\u201328). Efficient Drone Mobility Support Using Reinforcement Learning. Proceedings of the 2020 IEEE Wireless Communications and Networking Conference (WCNC), Seoul, Korea.","DOI":"10.1109\/WCNC45663.2020.9120595"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Jang, Y., Raza, S.M., Kim, M., and Choo, H. (2022, January 3\u20135). UAVs Handover Decision using Deep Reinforcement Learning. Proceedings of the 2022 International Conference on Ubiquitous Information Management and Communication (IMCOM), Seoul, Korea.","DOI":"10.1109\/IMCOM53663.2022.9721627"},{"key":"ref_10","unstructured":"(2022, January 27). Unity 3D Framework. Available online: https:\/\/unity3d.com\/unity\/whats-new\/2019.4.13."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Azari, M.M., Solanki, S., Chatzinotas, S., Kodheli, O., Sallouha, H., Colpaert, A., Montoya, J.F.M., Pollin, S., Haqiqatnejad, A., and Mostaani, A. (2021). Evolution of Non-Terrestrial Networks From 5G to 6G: A Survey. arXiv, Available online: http:\/\/xxx.lanl.gov\/abs\/2107.06881.","DOI":"10.1109\/COMST.2022.3199901"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"8384","DOI":"10.1109\/TVT.2019.2911986","article-title":"Modeling and Performance Analysis of UAV-Assisted Vehicular Networks","volume":"68","author":"Khabbaz","year":"2019","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"102242","DOI":"10.1016\/j.adhoc.2020.102242","article-title":"Drone assisted Flying Ad-Hoc Networks: Mobility and Service oriented modeling using Neuro-fuzzy","volume":"106","author":"Kumar","year":"2020","journal-title":"Ad Hoc Netw."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"19530","DOI":"10.1109\/ACCESS.2018.2817799","article-title":"Flying Drone Base Stations for Macro Hotspots","volume":"6","author":"Fotouhi","year":"2018","journal-title":"IEEE Access"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"102895","DOI":"10.1016\/j.jnca.2020.102895","article-title":"DroneCells: Improving spectral efficiency using drone-mounted flying base stations","volume":"174","author":"Fotouhi","year":"2021","journal-title":"J. Netw. Comput. Appl."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2679","DOI":"10.1109\/TNET.2018.2878287","article-title":"Aerial Channel Prediction and User Scheduling in Mobile Drone Hotspots","volume":"26","author":"Chowdhery","year":"2018","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Banagar, M., and Dhillon, H.S. (2019, January 9\u201313). 3GPP-Inspired Stochastic Geometry-Based Mobility Model for a Drone Cellular Network. Proceedings of the 2019 IEEE Global Communications Conference (GLOBECOM), Waikoloa, HI, USA.","DOI":"10.1109\/GLOBECOM38437.2019.9013645"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Banagar, M., and Dhillon, H.S. (2019, January 9\u201313). Fundamentals of Drone Cellular Network Analysis under Random Waypoint Mobility Model. Proceedings of the 2019 IEEE Global Communications Conference (GLOBECOM), Waikoloa, HI, USA.","DOI":"10.1109\/GLOBECOM38437.2019.9013341"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"90555","DOI":"10.1109\/ACCESS.2021.3089253","article-title":"Drone Mobile Networks: Performance Analysis Under 3D Tractable Mobility Models","volume":"9","author":"Huang","year":"2021","journal-title":"IEEE Access"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Yang, H., Raza, S.M., Kim, M., Le, D.T., Van Vo, V., and Choo, H. (2020, January 3\u20135). Next Point-of-Attachment Selection Based on Long Short Term Memory Model in Wireless Networks. Proceedings of the 2020 14th International Conference on Ubiquitous Information Management and Communication (IMCOM), Taichung, Taiwan.","DOI":"10.1109\/IMCOM48794.2020.9001672"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Jang, B., Raza, S.M., Kim, M., and Choo, H. (2020, January 13\u201316). MoGAN: GAN based Next PoA Selection for Proactive Mobility Management. Proceedings of the 2020 IEEE 28th International Conference on Network Protocols (ICNP), Madrid, Spain.","DOI":"10.1109\/ICNP49622.2020.9259368"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Guo, M., Wang, P., Chan, C.Y., and Askary, S. (2019, January 27\u201330). A Reinforcement Learning Approach for Intelligent Traffic Signal Control at Urban Intersections. Proceedings of the 2019 IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, New Zealand.","DOI":"10.1109\/ITSC.2019.8917268"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Khoshkholgh, M.G., and Yanikomeroglu, H. (2021). RSS-Based UAV-BS 3-D Mobility Management via Policy Gradient Deep Reinforcement Learning. arXiv, Available online: http:\/\/xxx.lanl.gov\/abs\/2103.08034.","DOI":"10.1109\/ICC42927.2021.9500827"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Fakhreddine, A., Bettstetter, C., Hayat, S., Muzaffar, R., and Emini, D. (2019). Handover Challenges for Cellular-Connected Drones, DroNet\u201919; Association for Computing Machinery.","DOI":"10.1145\/3325421.3329770"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1007\/BF00992698","article-title":"Q-learning","volume":"8","author":"Peter","year":"1992","journal-title":"Mach. Learn."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Van Hasselt, H., Guez, A., and Silver, D. (2016, January 12\u201317). Deep Reinforcement Learning with Double Q-Learning. Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI\u201916), Phoenix, AZ, USA.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"ref_28","first-page":"1928","article-title":"Asynchronous Methods for Deep Reinforcement Learning","volume":"Volume 48","author":"Balcan","year":"2016","journal-title":"Machine Learning Research, Proceedings of the 33rd International Conference on Machine Learning, New York, NY, USA, 20\u201322 June 2016"},{"key":"ref_29","unstructured":"John, S., Filip, W., Prafulla, D., Alec, R., and Oleg, K. (2017). Proximal Policy Optimization Algorithms. arXiv, Available online: http:\/\/xxx.lanl.gov\/abs\/1707.06347."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Anpalagan, A., Bennis, M., and Vannithamby, R. (2015). Mobility performance optimization for 3GPP LTE HetNets. Design and Deployment of Small Cell Networks, Cambridge University Press.","DOI":"10.1017\/CBO9781107297333"},{"key":"ref_31","unstructured":"Schulman, J., Moritz, P., Levine, S., Jordan, M., and Abbeel, P. (2018). High-Dimensional Continuous Control Using Generalized Advantage Estimation. arXiv, Available online: http:\/\/xxx.lanl.gov\/abs\/1506.02438."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1109\/TAP.2019.2935091","article-title":"A Low-Profile Differentially Fed Dual-Polarized Antenna With High Gain and Isolation for 5G Microcell Communications","volume":"68","author":"Feng","year":"2020","journal-title":"IEEE Trans. Antennas Propag."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"2568","DOI":"10.1109\/COMST.2019.2911558","article-title":"A Survey of Indoor Localization Systems and Technologies","volume":"21","author":"Zafari","year":"2019","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"3485","DOI":"10.1109\/JSYST.2017.2777866","article-title":"Mobile Target Coverage and Tracking on Drone-Be-Gone UAV Cyber-Physical Testbed","volume":"12","author":"Khan","year":"2018","journal-title":"IEEE Syst. J."},{"key":"ref_35","unstructured":"Miguel, A.F., Chad, T., and Polina, P. (2018, January 8\u201311). Drone Deliveries Logistics, Efficiency, Safety and Last Mile Trade-offs. Proceedings of the 7th International Conference on Information Systems, Logistics and Supply Chain, Lyon, France."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Ahmadi, S. (2014). Chapter 4\u2014System Operation and UE States. LTE-Advanced, Academic Press.","DOI":"10.1016\/B978-0-12-405162-1.00004-6"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/3\/1200\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:14:24Z","timestamp":1760134464000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/3\/1200"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,2,5]]},"references-count":36,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2022,2]]}},"alternative-id":["s22031200"],"URL":"https:\/\/doi.org\/10.3390\/s22031200","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,2,5]]}}}