{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,31]],"date-time":"2026-03-31T13:48:12Z","timestamp":1774964892001,"version":"3.50.1"},"reference-count":26,"publisher":"MDPI AG","issue":"24","license":[{"start":{"date-parts":[[2021,12,9]],"date-time":"2021-12-09T00:00:00Z","timestamp":1639008000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"the Defense Acquisition Program Administration and Ministry of Trade, Industry and Energy of Korean government","award":["18-CM-SW-19"],"award-info":[{"award-number":["18-CM-SW-19"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>A network composed of unmanned aerial vehicles (UAVs), serving as base stations (UAV-BS network), is emerging as a promising component in next-generation communication systems. In the UAV-BS network, the optimal positioning of a UAV-BS is an essential requirement to establish line-of-sight (LoS) links for ground users. A novel deep Q-network (DQN)-based learning model enabling the optimal deployment of a UAV-BS is proposed. Moreover, without re-learning of the model and the acquisition of the path information of ground users, the proposed model presents the optimal UAV-BS trajectory while ground users move. Specifically, the proposed model optimizes the trajectory of a UAV-BS by maximizing the mean opinion score (MOS) for ground users who move to various paths. Furthermore, the proposed model is highly practical because, instead of the locations of individual mobile users, an average channel power gain is used as an input parameter. The accuracy of the proposed model is validated by comparing the results of the model with those of a mathematical optimization solver.<\/jats:p>","DOI":"10.3390\/s21248239","type":"journal-article","created":{"date-parts":[[2021,12,9]],"date-time":"2021-12-09T21:46:58Z","timestamp":1639086418000},"page":"8239","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":22,"title":["Deep Reinforcement Learning for UAV Trajectory Design Considering Mobile Ground Users"],"prefix":"10.3390","volume":"21","author":[{"given":"Wonseok","family":"Lee","sequence":"first","affiliation":[{"name":"School of Information and Communication Engineering, Chungbuk National University, Chungju 28644, Korea"}]},{"given":"Young","family":"Jeon","sequence":"additional","affiliation":[{"name":"School of Information and Communication Engineering, Chungbuk National University, Chungju 28644, Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6326-2559","authenticated-orcid":false,"given":"Taejoon","family":"Kim","sequence":"additional","affiliation":[{"name":"School of Information and Communication Engineering, Chungbuk National University, Chungju 28644, Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0395-4507","authenticated-orcid":false,"given":"Young-Il","family":"Kim","sequence":"additional","affiliation":[{"name":"Electronics and Telecommunications Research Institute, Daejeon 34129, Korea"}]}],"member":"1968","published-online":{"date-parts":[[2021,12,9]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"50131","DOI":"10.1109\/ACCESS.2020.2980359","article-title":"Monitoring ancient buildings: Real deployment of an IoT system enhanced by UAVs and virtual reality","volume":"8","author":"Bacco","year":"2020","journal-title":"IEEE Access"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1070","DOI":"10.1109\/TVT.2019.2949634","article-title":"Social-aware UAV-assisted mobile crowd sensing in stochastic and dynamic environments for disaster relief networks","volume":"69","author":"Wang","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Costa, F.-G., Ueyama, J., Braun, T., Pessin, G., Osorio, F.-S., and Vargas, P.-A. (2012, January 22\u201327). The use of unmanned aerial vehicles and wireless sensor network in agricultural applications. Proceedings of the 2012 IEEE International Geoscience and Remote Sensing Symposium, Munich, Germany.","DOI":"10.1109\/IGARSS.2012.6352477"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Hayat, S., Yanmaz, E., Brown, T.-X., and Bettstetter, C. (June, January 29). Multi-objective UAV path planning for search and rescue. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989656"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"102242","DOI":"10.1016\/j.adhoc.2020.102242","article-title":"Drone assisted Flying Ad-Hoc Networks: Mobility and Service oriented modeling using Neuro-fuzzy","volume":"106","author":"Kumar","year":"2020","journal-title":"Ad Hoc Netw."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Kumar, K., Kumar, S., Kaiwartya, O., Sikandar, A., Kharel, R., and Mauri, J.L. (2020). Internet of Unmanned Aerial Vehicles: QoS Provisioning in Aerial Ad-Hoc Networks. Sensors, 20.","DOI":"10.3390\/s20113160"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Cicek, C.T., Gultekin, H., Tavli, B., and Yanikomeroglu, H. (2019, January 5\u20137). UAV Base Station Location Optimization for Next Generation Wireless Networks: Overview and Future Research Directions. Proceedings of the 2019 1st International Conference on Unmanned Vehicle Systems-Oman (UVS) 2019, Muscat, Oman.","DOI":"10.1109\/UVS.2019.8658363"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"913","DOI":"10.1109\/LWC.2019.2899599","article-title":"On-Demand Density-Aware UAV Base Station 3D Placement for Arbitrarily Distributed Users With Guaranteed Data Rates","volume":"8","author":"Lai","year":"2019","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Cherif, N., Jaafar, W., Yanikomeroglu, H., and Yongacoglu, A. (2020, January 7\u201311). On the Optimal 3D Placement of a UAV Base Station for Maximal Coverage of UAV Users. Proceedings of the GLOBECOM 2020\u20142020 IEEE Global Communications Conference, Taipei, Taiwan.","DOI":"10.1109\/GLOBECOM42002.2020.9322569"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"434","DOI":"10.1109\/LWC.2017.2700840","article-title":"3-D Placement of an Unmanned Aerial Vehicle Base Station (UAV-BS) for Energy-Efficient Maximal Coverage","volume":"6","author":"Alzenad","year":"2017","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Li, X., Wang, Q., Liu, J., and Zhang, W. (2020, January 9\u201311). 3D Deployment with machine learning and system performance analysis of UAV-enabled networks. Proceedings of the 2020 IEEE\/CIC International Conference on Communications in China (ICCC), Chongqing, China.","DOI":"10.1109\/ICCC49849.2020.9238795"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Hou, M.-C., Deng, D.-J., and Wu, C.-L. (2019, January 9\u201313). Optimum aerial base station deployment for UAV networks: A reinforcement learning approach. Proceedings of the 019 IEEE Globecom Workshops (GC Wkshps), Waikoloa, HI, USA.","DOI":"10.1109\/GCWkshps45667.2019.9024648"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1101","DOI":"10.1109\/TCCN.2019.2948324","article-title":"Optimal UAV base station trajectories using flow-level models for reinforcement learning","volume":"5","author":"Saxena","year":"2019","journal-title":"IEEE Trans. Cogn. Commun. Netw."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Hao, G., Ni, W., Tian, H., and Cao, L. (2020, January 21\u201323). Mobility-aware trajectory design for aerial base station using deep reinforcement learning. Proceedings of the 2020 International Conference on Wireless Communications and Signal Processing (WCSP), Nanjing, China.","DOI":"10.1109\/WCSP49889.2020.9299676"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"8036","DOI":"10.1109\/TVT.2019.2922849","article-title":"Reinforcement learning in multiple-UAV networks: Deployment and movement design","volume":"68","author":"Liu","year":"2019","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"8227","DOI":"10.1109\/TVT.2019.2923214","article-title":"Intelligent trajectory design in UAV-aided communications with reinforcement learning","volume":"68","author":"Yin","year":"2019","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_17","unstructured":"Wu, J., Yu, P., Feng, L., Zhou, F., Li, W., and Qiu, X. (2019, January 8\u201312). 3D aerial base station position planning based on deep Q-network for capacity enhancement. Proceedings of the 2019 IFIP\/IEEE Symposium on Integrated Network and Service Management (IM), Washington, DC, USA."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1007\/BF00992698","article-title":"Q-learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Mach. Learn."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"McClelland, J.L., Rumelhart, D.E., and Group, T.P.R. (1986). Parallel Distributed Processing: Explorations in the Microstructure of Cognition, MIT Press.","DOI":"10.7551\/mitpress\/5236.001.0001"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Bayerlein, H., De Kerret, P., and Gesbert, D. (2018, January 25\u201328). Trajectory optimization for autonomous flying base station via reinforcement learning. Proceedings of the IEEE 19th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), Kalamata, Greece.","DOI":"10.1109\/SPAWC.2018.8445768"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"101872","DOI":"10.1016\/j.adhoc.2019.101872","article-title":"Q-SQUARE: A Q-learning approach to provide a QoE aware UAV flight path in cellular networks","volume":"91","author":"Colonnese","year":"2019","journal-title":"Ad Hoc Netw."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1109\/OJVT.2020.2971594","article-title":"A Reinforcement Learning Approach for Fair User Coverage Using UAV Mounted Base Stations Under Energy Constraints","volume":"1","author":"Abeywickrama","year":"2020","journal-title":"IEEE Open J. Veh. Technol."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Wright, S., and Nocedal, J. (1999). Numerical Optimization, Springer.","DOI":"10.1007\/b98874"},{"key":"ref_25","first-page":"3196","article-title":"Novel cross-layer QoE-aware radio resource allocation algorithms in multiuser OFDMA systems","volume":"62","author":"Rugelj","year":"2014","journal-title":"IEEE Commun. Mag."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"6160","DOI":"10.1109\/TWC.2018.2855130","article-title":"QoE-Based Resource Allocation for Multi-Cell NOMA Networks","volume":"17","author":"Cui","year":"2018","journal-title":"IEEE Trans. Wirel. Commun."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/24\/8239\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:44:18Z","timestamp":1760168658000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/24\/8239"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,9]]},"references-count":26,"journal-issue":{"issue":"24","published-online":{"date-parts":[[2021,12]]}},"alternative-id":["s21248239"],"URL":"https:\/\/doi.org\/10.3390\/s21248239","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,12,9]]}}}