{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T19:57:37Z","timestamp":1775073457746,"version":"3.50.1"},"reference-count":53,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2021,10,21]],"date-time":"2021-10-21T00:00:00Z","timestamp":1634774400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>A communication system based on unmanned aerial vehicles (UAVs) is a viable alternative for meeting the coverage and capacity needs of future wireless networks. However, because of the limitations of UAV-enabled communications in terms of coverage, energy consumption, and flying laws, the number of studies focused on the sustainability element of UAV-assisted networking in the literature was limited thus far. We present a solution to this problem in this study; specifically, we design a Q-learning-based UAV placement strategy for long-term wireless connectivity while taking into account major constraints such as altitude regulations, nonflight zones, and transmit power. The goal is to determine the best location for the UAV base station (BS) while reducing energy consumption and increasing the number of users covered. Furthermore, a weighting method is devised, allowing energy usage and the number of users served to be prioritized based on network\/battery circumstances. The suggested Q-learning-based solution is contrasted to the standard k-means clustering method, in which the UAV BS is positioned at the centroid location with the shortest cumulative distance between it and the users. The results demonstrate that the proposed solution outperforms the baseline k-means clustering-based method in terms of the number of users covered while achieving the desired minimization of the energy consumption.<\/jats:p>","DOI":"10.3390\/a14110302","type":"journal-article","created":{"date-parts":[[2021,10,21]],"date-time":"2021-10-21T14:37:48Z","timestamp":1634827068000},"page":"302","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Multi-Objective UAV Positioning Mechanism for Sustainable Wireless Connectivity in Environments with Forbidden Flying Zones"],"prefix":"10.3390","volume":"14","author":[{"given":"\u0130brahim","family":"Atli","sequence":"first","affiliation":[{"name":"Faculty of Engineering and Natural Sciences, Ankara Y\u0131ld\u0131r\u0131m Beyaz\u0131t University, Ankara 06760, Turkey"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8665-5291","authenticated-orcid":false,"given":"Metin","family":"Ozturk","sequence":"additional","affiliation":[{"name":"Faculty of Engineering and Natural Sciences, Ankara Y\u0131ld\u0131r\u0131m Beyaz\u0131t University, Ankara 06760, Turkey"}]},{"given":"Gianluca C.","family":"Valastro","sequence":"additional","affiliation":[{"name":"National Inter-University Consortium for Telecommunications (CNIT), 43124 Parma, Italy"}]},{"given":"Muhammad Zeeshan","family":"Asghar","sequence":"additional","affiliation":[{"name":"Department of Communications and Networking, Aalto University, 02150 Espoo, Finland"}]}],"member":"1968","published-online":{"date-parts":[[2021,10,21]]},"reference":[{"key":"ref_1","unstructured":"Ericsson (2021). Ericsson Mobility Report, Ericsson. Technical report."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2347","DOI":"10.1109\/COMST.2015.2444095","article-title":"Internet of Things: A Survey on Enabling Technologies, Protocols, and Applications","volume":"17","author":"Guizani","year":"2015","journal-title":"IEEE Commun. Surv. Tutorials"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1201","DOI":"10.1109\/JSAC.2017.2692307","article-title":"5G: A Tutorial Overview of Standards, Trials, Challenges, Deployment, and Practice","volume":"35","author":"Shafi","year":"2017","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Jameel, F., Haider, M.A.A., and Butt, A.A. (2017, January 24\u201326). Massive MIMO: A survey of recent advances, research issues and future directions. Proceedings of the 2017 International Symposium on Recent Advances in Electrical Engineering (RAEE), Islamabad, Pakistan.","DOI":"10.1109\/RAEE.2017.8246040"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"836","DOI":"10.1109\/COMST.2017.2787460","article-title":"Millimeter-Wave Massive MIMO Communication for Future Wireless Systems: A Survey","volume":"20","author":"Busari","year":"2018","journal-title":"IEEE Commun. Surv. Tutorials"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"6213","DOI":"10.1109\/TAP.2017.2734243","article-title":"Overview of Millimeter Wave Communications for Fifth-Generation (5G) Wireless Networks\u2014With a Focus on Propagation Models","volume":"65","author":"Rappaport","year":"2017","journal-title":"IEEE Trans. Antennas Propag."},{"key":"ref_7","unstructured":"3GPP (2021, September 01). 5G; NR; Base Station (BS) Radio Transmission and Reception. Available online: https:\/\/portal.3gpp.org\/desktopmodules\/Specifications\/SpecificationDetails.aspx?specificationId=3202."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2334","DOI":"10.1109\/COMST.2019.2902862","article-title":"A Tutorial on UAVs for Wireless Networks: Applications, Challenges, and Open Problems","volume":"21","author":"Mozaffari","year":"2019","journal-title":"IEEE Commun. Surv. Tutorials"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2392","DOI":"10.1109\/COMST.2017.2727878","article-title":"A Survey of Machine Learning Techniques Applied to Self-Organizing Cellular Networks","volume":"19","author":"Klaine","year":"2017","journal-title":"IEEE Commun. Surv. Tutorials"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2224","DOI":"10.1109\/COMST.2019.2904897","article-title":"Deep Learning in Mobile and Wireless Networking: A Survey","volume":"21","author":"Zhang","year":"2019","journal-title":"IEEE Commun. Surv. Tutorials"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Morocho Cayamcela, M.E., and Lim, W. (2018, January 17\u201319). Artificial Intelligence in 5G Technology: A Survey. Proceedings of the 2018 International Conference on Information and Communication Technology Convergence (ICTC), Jeju Island, Korea.","DOI":"10.1109\/ICTC.2018.8539642"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"3072","DOI":"10.1109\/COMST.2019.2924243","article-title":"Application of Machine Learning in Wireless Networks: Key Techniques and Open Issues","volume":"21","author":"Sun","year":"2019","journal-title":"IEEE Commun. Surv. Tutorials"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"802","DOI":"10.1109\/ACCESS.2018.2885821","article-title":"Mobility Prediction: A Survey on State-of-the-Art Schemes and Future Applications","volume":"7","author":"Zhang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Dogra, A., Jha, R.K., and Jain, S. (2020). A Survey on beyond 5G network with the advent of 6G: Architecture and Emerging Technologies. IEEE Access.","DOI":"10.1109\/ACCESS.2020.3031234"},{"key":"ref_15","first-page":"100158","article-title":"6G: A survey on technologies, scenarios, challenges, and the related issues","volume":"19","author":"Lu","year":"2020","journal-title":"J. Ind. Inf. Integr."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Ozturk, M., Nadas, J.P.B., Klaine, P.H.V., Hussain, S., and Imran, M.A. (2019, January 8\u201310). Clustering Based UAV Base Station Positioning for Enhanced Network Capacity. Proceedings of the 2019 International Conference on Advances in the Emerging Computing Technologies (AECT), Medina, Saudi Arabia.","DOI":"10.1109\/AECT47998.2020.9194188"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"10320","DOI":"10.1109\/TVT.2020.3004855","article-title":"Energy-Saving Deployment Algorithms of UAV Swarm for Sustainable Wireless Coverage","volume":"69","author":"Zhang","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"de Paula Parisotto, R., Klaine, P.V., Nadas, J.P.B., Souza, R.D., Brante, G., and Imran, M.A. (2019, January 27\u201330). Drone Base Station Positioning and Power Allocation using Reinforcement Learning. Proceedings of the 2019 16th International Symposium on Wireless Communication Systems (ISWCS), Oulu, Finland.","DOI":"10.1109\/ISWCS.2019.8877247"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"2010","DOI":"10.1109\/LWC.2018.2808957","article-title":"Energy Efficient Placement of a Drone Base Station for Minimum Required Transmit Power","volume":"9","author":"Wang","year":"2020","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Donevski, I., and Nielsen, J.J. (2020, January 16\u201317). Dynamic Standalone Drone-Mounted Small Cells. Proceedings of the 2020 European Conference on Networks and Communications (EuCNC), Dubrovnik, Croatia.","DOI":"10.1109\/EuCNC48522.2020.9200918"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"569","DOI":"10.1109\/LWC.2014.2342736","article-title":"Optimal LAP Altitude for Maximum Coverage","volume":"3","author":"Kandeepan","year":"2014","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"3949","DOI":"10.1109\/TWC.2016.2531652","article-title":"Unmanned Aerial Vehicle With Underlaid Device-to-Device Communications: Performance and Tradeoffs","volume":"15","author":"Mozaffari","year":"2016","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Becvar, Z., Mach, P., and Nikooroo, M. (2020, January 25\u201328). Reducing Energy Consumed by Repositioning of Flying Base Stations Serving Mobile Users. Proceedings of the 2020 IEEE Wireless Communications and Networking Conference (WCNC), Seoul, Korea.","DOI":"10.1109\/WCNC45663.2020.9120682"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"53172","DOI":"10.1109\/ACCESS.2020.2981403","article-title":"Energy Efficient 3-D UAV Control for Persistent Communication Service and Fairness: A Deep Reinforcement Learning Approach","volume":"8","author":"Qi","year":"2020","journal-title":"IEEE Access"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1109\/TSMC.2016.2582745","article-title":"Vehicle Routing Problems for Drone Delivery","volume":"47","author":"Dorling","year":"2017","journal-title":"IEEE Trans. Syst. Man, Cybern. Syst."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"74175","DOI":"10.1109\/ACCESS.2020.2987622","article-title":"Recent Research Progress of Unmanned Aerial Vehicle Regulation Policies and Technologies in Urban Low Altitude","volume":"8","author":"Xu","year":"2020","journal-title":"IEEE Access"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"3417","DOI":"10.1109\/COMST.2019.2906228","article-title":"Survey on UAV Cellular Communications: Practical Aspects, Standardization Advancements, Regulation, and Security Challenges","volume":"21","author":"Fotouhi","year":"2019","journal-title":"IEEE Commun. Surv. Tutorials"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Patan\u00e9, G.M.M., Valastro, G.C., Sambo, Y.A., Ozturk, M., Hussain, S., Imran, M.A., and Panno, D. (, January 7\u20139). Flexible SDN\/NFV-based SON testbed for 5G mobile networks. Proceedings of the 2019 IEEE\/ACM 23rd International Symposium on Distributed Simulation and Real Time Applications (DS-RT), Cosenza, Italy.","DOI":"10.1109\/DS-RT47707.2019.8958683"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Sambo, Y.A., Valastro, G.C., Patan\u00e9, G.M.M., Ozturk, M., Hussain, S., Imran, M.A., and Panno, D. (2019, January 11\u201313). Motion Sensor-Based Small Cell Sleep Scheduling for 5G Networks. Proceedings of the 2019 IEEE 24th International Workshop on Computer Aided Modeling and Design of Communication Links and Networks (CAMAD), Limassol, Cyprus.","DOI":"10.1109\/CAMAD.2019.8858435"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"7574","DOI":"10.1109\/TWC.2017.2751045","article-title":"Mobile Unmanned Aerial Vehicles (UAVs) for Energy-Efficient Internet of Things Communications","volume":"16","author":"Mozaffari","year":"2017","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Pang, Y., Zhang, Y., Gu, Y., Pan, M., Han, Z., and Li, P. (2014, January 8\u201312). Efficient data collection for wireless rechargeable sensor clusters in Harsh terrains using UAVs. Proceedings of the 2014 IEEE Global Communications Conference, Austin, TX, USA.","DOI":"10.1109\/GLOCOM.2014.7036813"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"580","DOI":"10.1109\/LWC.2018.2880467","article-title":"Resource Allocation in UAV-Assisted M2M Communications for Disaster Rescue","volume":"8","author":"Liu","year":"2019","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Bithas, P.S., Michailidis, E.T., Nomikos, N., Vouyioukas, D., and Kanatas, A.G. (2019). A Survey on Machine-Learning Techniques for UAV-Based Communications. Sensors, 19.","DOI":"10.3390\/s19235170"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Mozaffari, M., Saad, W., Bennis, M., and Debbah, M. (2015, January 6\u201310). Drone Small Cells in the Clouds: Design, Deployment and Performance Analysis. Proceedings of the 2015 IEEE Global Communications Conference (GLOBECOM), San Diego, CA, USA.","DOI":"10.1109\/GLOCOM.2015.7417609"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1647","DOI":"10.1109\/LCOMM.2016.2578312","article-title":"Efficient Deployment of Multiple Unmanned Aerial Vehicles for Optimal Wireless Coverage","volume":"20","author":"Mozaffari","year":"2016","journal-title":"IEEE Commun. Lett."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Hayajneh, A.M., Zaidi, S.A.R., McLernon, D.C., and Ghogho, M. (2016, January 27\u201330). Drone Empowered Small Cellular Disaster Recovery Networks for Resilient Smart Cities. Proceedings of the 2016 IEEE International Conference on Sensing, Communication and Networking (SECON Workshops), London, UK.","DOI":"10.1109\/SECONW.2016.7746806"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Bor-Yaliniz, R.I., El-Keyi, A., and Yanikomeroglu, H. (2016, January 23\u201327). Efficient 3-D placement of an aerial base station in next generation cellular networks. Proceedings of the 2016 IEEE International Conference on Communications (ICC), Kuala Lumpur, Malaysia.","DOI":"10.1109\/ICC.2016.7510820"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Kalantari, E., Yanikomeroglu, H., and Yongacoglu, A. (2016, January 18\u201321). On the Number and 3D Placement of Drone Base Stations in Wireless Cellular Networks. Proceedings of the 2016 IEEE 84th Vehicular Technology Conference (VTC-Fall), Montr\u00e9al, QC, Canada.","DOI":"10.1109\/VTCFall.2016.7881122"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Ko\u0161merl, J., and Vilhar, A. (2014, January 10\u201314). Base stations placement optimization in wireless networks for emergency communications. Proceedings of the 2014 IEEE International Conference on Communications Workshops (ICC), Sydney, NSW, Australia.","DOI":"10.1109\/ICCW.2014.6881196"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Zhan, P., Yu, K., and Lee Swindlehurst, A. (2006, January 2\u20135). Wireless Relay Communications using an Unmanned Aerial Vehicle. Proceedings of the 2006 IEEE 7th Workshop on Signal Processing Advances in Wireless Communications, Cannes, France.","DOI":"10.1109\/SPAWC.2006.346492"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Mohamed, E.M., Hashima, S., Aldosary, A., Hatano, K., and Abdelghany, M.A. (2020). Gateway Selection in Millimeter Wave UAV Wireless Networks Using Multi-Player Multi-Armed Bandit. Sensors, 20.","DOI":"10.3390\/s20143947"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"2109","DOI":"10.1109\/TWC.2017.2789293","article-title":"Joint Trajectory and Communication Design for Multi-UAV Enabled Wireless Networks","volume":"17","author":"Wu","year":"2018","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"993","DOI":"10.1109\/JSAC.2012.120614","article-title":"Optimization of UAV Heading for the Ground-to-Air Uplink","volume":"30","author":"Jiang","year":"2012","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"4983","DOI":"10.1109\/TCOMM.2016.2611512","article-title":"Throughput Maximization for UAV-Enabled Mobile Relaying Systems","volume":"64","author":"Zeng","year":"2016","journal-title":"IEEE Trans. Commun."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"3662","DOI":"10.1109\/LRA.2018.2854967","article-title":"Energy-Aware Spiral Coverage Path Planning for UAV Photogrammetric Applications","volume":"3","author":"Cabreira","year":"2018","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Shao, X., Gong, Y.J., Zhan, Z.H., and Zhang, J. (2021). Bipartite Cooperative Coevolution for Energy-Aware Coverage Path Planning of UAVs. IEEE Trans. Artif. Intell.","DOI":"10.1109\/TAI.2021.3103143"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"136","DOI":"10.23919\/JCC.2021.02.009","article-title":"An efficient trajectory planning for cellular-connected UAV under the connectivity constraint","volume":"18","author":"Yang","year":"2021","journal-title":"China Commun."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Sun, G., Li, J., Liu, Y., Liang, S., and Kang, H. (2021). Time and Energy Minimization Communications Based on Collaborative Beamforming for UAV Networks: A Multi-objective Optimization Method. IEEE J. Sel. Areas Commun.","DOI":"10.1109\/JSAC.2021.3088720"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Yu, Y., Tang, J., Huang, J., Zhang, X., So, D.K.C., and Wong, K.K. (2021). Multi-Objective Optimization for UAV-Assisted Wireless Powered IoT Networks Based on Extended DDPG Algorithm. IEEE Trans. Commun.","DOI":"10.1109\/TCOMM.2021.3089476"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"2329","DOI":"10.1109\/TWC.2019.2902559","article-title":"Energy Minimization for Wireless Communication With Rotary-Wing UAV","volume":"18","author":"Zeng","year":"2019","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Arani, A.H., Hu, P., and Zhu, Y. (2021, January 14\u201323). Re-envisioning Space-Air-Ground Integrated Networks: Reinforcement Learning for Link Optimization. Proceedings of the ICC 2021\u2014IEEE International Conference on Communications, Montreal, QC, Canada.","DOI":"10.1109\/ICC42927.2021.9500978"},{"key":"ref_52","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press."},{"key":"ref_53","unstructured":"Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., and Zaremba, W. OpenAI Gym. arXiv preprint."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/14\/11\/302\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:20:30Z","timestamp":1760167230000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/14\/11\/302"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,21]]},"references-count":53,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2021,11]]}},"alternative-id":["a14110302"],"URL":"https:\/\/doi.org\/10.3390\/a14110302","relation":{"has-preprint":[{"id-type":"doi","id":"10.20944\/preprints202109.0177.v1","asserted-by":"object"}]},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,21]]}}}