{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,26]],"date-time":"2026-03-26T15:34:47Z","timestamp":1774539287655,"version":"3.50.1"},"reference-count":34,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2024,10,30]],"date-time":"2024-10-30T00:00:00Z","timestamp":1730246400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>Recent advancements in Unmanned Aerial Vehicle (UAV) technology have made them effective platforms for data capture in applications like environmental monitoring. UAVs, acting as mobile data ferries, can significantly improve ground network performance by involving ground network representatives in data collection. These representatives communicate opportunistically with accessible UAVs. Emerging technologies such as Software Defined Wireless Sensor Networks (SDWSN), wherein the role\/function of sensor nodes is defined via software, can offer a flexible operation for UAV data-gathering approaches. In this paper, we introduce the \u201cUAV Fuzzy Travel Path\u201d, a novel approach that utilizes Deep Reinforcement Learning (DRL) algorithms, which is a subfield of machine learning, for optimal UAV trajectory planning. The approach also involves the integration between UAV and SDWSN wherein nodes acting as gateways (GWs) receive data from the flexibly formulated group members via software definition. A UAV is then dispatched to capture data from GWs along a planned trajectory within a fuzzy span. Our dual objectives are to minimize the total energy consumption of the UAV system during each data collection round and to enhance the communication bit rate on the UAV-Ground connectivity. We formulate this problem as a constrained combinatorial optimization problem, jointly planning the UAV path with improved communication performance. To tackle the NP-hard nature of this problem, we propose a novel DRL technique based on Deep Q-Learning. By learning from UAV path policy experiences, our approach efficiently reduces energy consumption while maximizing packet delivery.<\/jats:p>","DOI":"10.3390\/fi16110398","type":"journal-article","created":{"date-parts":[[2024,10,30]],"date-time":"2024-10-30T09:11:24Z","timestamp":1730279484000},"page":"398","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Deep Reinforcement Learning for UAV-Based SDWSN Data Collection"],"prefix":"10.3390","volume":"16","author":[{"given":"Pejman A.","family":"Karegar","sequence":"first","affiliation":[{"name":"Department of Electrical and Electronic Engineering, Auckland University of Technology (AUT), Auckland 1010, New Zealand"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2832-7190","authenticated-orcid":false,"given":"Duaa Zuhair","family":"Al-Hamid","sequence":"additional","affiliation":[{"name":"Department of Electrical and Electronic Engineering, Auckland University of Technology (AUT), Auckland 1010, New Zealand"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5375-8961","authenticated-orcid":false,"given":"Peter Han Joo","family":"Chong","sequence":"additional","affiliation":[{"name":"Department of Electrical and Electronic Engineering, Auckland University of Technology (AUT), Auckland 1010, New Zealand"}]}],"member":"1968","published-online":{"date-parts":[[2024,10,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Fascista, A. (2022). Toward integrated large-scale environmental monitoring using WSN\/UAV\/Crowdsensing: A review of applications, signal processing, and future perspectives. Sensors, 22.","DOI":"10.3390\/s22051824"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Mohsan, S.A.H., Khan, M.A., Noor, F., Ullah, I., and Alsharif, M.H. (2022). Towards the unmanned aerial vehicles (UAVs): A comprehensive review. Drones, 6.","DOI":"10.3390\/drones6060147"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"145813","DOI":"10.1109\/ACCESS.2023.3346299","article-title":"Internet of things and wireless sensor networks for smart agriculture applications-a survey","volume":"11","author":"Mowla","year":"2023","journal-title":"IEEE Access"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"23560","DOI":"10.1109\/ACCESS.2022.3153521","article-title":"A survey on machine learning software-defined wireless sensor networks (ml-SDWSNS): Current status and major challenges","volume":"10","author":"Marchegiani","year":"2022","journal-title":"IEEE Access"},{"key":"ref_5","first-page":"156","article-title":"Softwarization, virtualization, machine learning for intelligent effective v2x communications","volume":"14","author":"Moubayed","year":"2020","journal-title":"IEEE Access"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Al-Hamid, D.Z., and Al-Anbuky, A. (2023). Vehicular Networks Dynamic Grouping and Re-Orchestration Scenarios. Information, 14.","DOI":"10.3390\/info14010032"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1016\/j.comcom.2022.07.021","article-title":"A survey on the role of UAVs in the communication process: A technological perspective","volume":"194","author":"Alsuhli","year":"2022","journal-title":"Comput. Commun."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"6397","DOI":"10.1109\/TITS.2024.3375893","article-title":"Comprehensive Review of Drones Collision Avoidance Schemes: Challenges and Open Issues","volume":"25","author":"Rezaee","year":"2024","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Azar, A.T., Koubaa, A., Ali Mohamed, N., Ibrahim, H.A., Ibrahim, Z.F., Kazim, M., and Casalino, G. (2021). Drone deep reinforcement learning: A review. Electronics, 10.","DOI":"10.3390\/electronics10090999"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"e5423","DOI":"10.1002\/dac.5423","article-title":"Classical versus reinforcement learning algorithms for unmanned aerial vehicle network communication and coverage path planning: A systematic literature review","volume":"36","author":"Mannan","year":"2023","journal-title":"Int. J. Commun. Syst."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"21061","DOI":"10.1109\/JIOT.2024.3350525","article-title":"A lightweight reinforcement learning-based real-time path planning method for unmanned aerial vehicles","volume":"11","author":"Xi","year":"2024","journal-title":"IEEE Internet Things J."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Luo, X., Chen, C., Zeng, C., Li, C., Xu, J., and Gong, S. (2023). Deep reinforcement learning for joint trajectory planning, transmission scheduling, and access control in UAV-assisted wireless sensor networks. Sensors, 23.","DOI":"10.3390\/s23104691"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Al-Hamid, D.Z., Karegar, P.A., and Chong, P.H.J. (2023). Modelling and Implementation Tools for SDWSN Smart Applications. 2023 28th Asia Pacific Conference on Communications (APCC), IEEE.","DOI":"10.1109\/APCC60132.2023.10460734"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Karegar, P.A., Al-Hamid, D.Z., and Chong, P.H.J. (2024). UAV-enabled software defined data collection from an adaptive WSN. Wireless Networks, Springer.","DOI":"10.1007\/s11276-024-03744-y"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"6829","DOI":"10.1007\/s11042-023-15956-z","article-title":"Localization in wireless sensor networks and wireless multimedia sensor networks using clustering techniques","volume":"83","author":"Wajgi","year":"2024","journal-title":"Multimed. Tools Appl."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1016\/j.jnca.2018.06.005","article-title":"Reliable location aware and Cluster-Tap Root based data collection protocol for large scale wireless sensor networks","volume":"118","author":"Kalaivanan","year":"2018","journal-title":"J. Netw. Comput. Appl."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"2377","DOI":"10.1109\/JSEN.2014.2377200","article-title":"An Efficient Cluster-Tree Based Data Collection Scheme for Large Mobile Wireless Sensor Networks","volume":"15","author":"Velmani","year":"2015","journal-title":"IEEE Sens."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Tazibt, C.Y., Bekhti, M., Djamah, T., Achir, N., and Boussetta, K. (2017, January 29\u201331). Wireless sensor network clustering for UAV-based data gathering. Proceedings of the Wireless Days, Porto, Portugal.","DOI":"10.1109\/WD.2017.7918154"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"4324","DOI":"10.1002\/dac.4324","article-title":"Prolong network lifetime and improve efficiency in WSN-UAV systems using new clustering parameters and CSMA modification","volume":"33","author":"Tarighi","year":"2020","journal-title":"Int. J. Commun. Syst."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Popescu, D., Stoican, F., Stamatescu, G., Ichim, L., and Dragana, C. (2020). Advanced UAV\u2013WSN System for Intelligent Monitoring in Precision Agriculture. Sensors, 20.","DOI":"10.3390\/s20030817"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1893","DOI":"10.1109\/JIOT.2018.2878834","article-title":"UAV-Aided ProjectionBased Compressive Data Gathering in Wireless Sensor Networks","volume":"6","author":"Ebrahimi","year":"2019","journal-title":"IEEE Internet Things"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"80716","DOI":"10.1109\/ACCESS.2020.2988796","article-title":"Unsupervised K-means clustering algorithm","volume":"8","author":"Sinaga","year":"2020","journal-title":"IEEE Access"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"553","DOI":"10.1109\/COMST.2015.2412971","article-title":"Wireless sensor network virtualization: A survey","volume":"18","author":"Khan","year":"2016","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Al-Hamid, D.Z., Karegar, P.A., and Chong, P.H.J. (2023). A novel SDWSN-based testbed for IoT smart applications. Future Internet, 15.","DOI":"10.3390\/fi15090291"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1367","DOI":"10.1007\/s11276-022-03194-4","article-title":"UAV-assisted data gathering from a sparse wireless sensor adaptive networks","volume":"29","author":"Karegar","year":"2022","journal-title":"Wirel. Netw."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"12382","DOI":"10.1109\/TVT.2020.3023861","article-title":"Age of Information Aware Trajectory Planning of UAVs in Intelligent Transportation Systems: A Deep Learning Approach","volume":"69","author":"Samir","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"2109","DOI":"10.1109\/TWC.2017.2789293","article-title":"Joint Trajectory and Communication Design for Multi-UAV Enabled Wireless Networks","volume":"17","author":"Wu","year":"2018","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1464","DOI":"10.1109\/JIOT.2019.2955732","article-title":"Joint Optimization on Trajectory, Altitude, Velocity, and Link Scheduling for Minimum Mission Time in UAV-Aided Data Collection","volume":"7","author":"Li","year":"2020","journal-title":"IEEE Internet Things"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2124","DOI":"10.1109\/TVT.2018.2890773","article-title":"Autonomous Navigation of UAVs in Large-Scale Complex Environments: A Deep Reinforcement Learning Approach","volume":"68","author":"Wang","year":"2019","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1666","DOI":"10.1109\/TII.2017.2783439","article-title":"Learning-based energy-efficient data collection by unmanned vehicles in smart cities","volume":"14","author":"Zhang","year":"2018","journal-title":"IEEE Trans. Ind. Informat."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"2804943","DOI":"10.1155\/2023\/2804943","article-title":"Deep Reinforcement Learning-Based UAV Path Planning for Energy-Efficient Multitier Cooperative Computing in Wireless Sensor Networks","volume":"2023","author":"Guo","year":"2023","journal-title":"J. Sens."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"9572","DOI":"10.1109\/JIOT.2023.3323921","article-title":"UAV Path Planning for Data Gathering in Wireless Sensor Networks: Spatial and Temporal Substate-Based Q-Learning","volume":"11","author":"Beishenalieva","year":"2023","journal-title":"IEEE Internet Things J."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"29705","DOI":"10.1109\/JSEN.2023.3324845","article-title":"Optimizing Energy-Efficiency in UAV-assisted Wireless Sensor Networks with Reinforcement Learning PPO2 Algorithm","volume":"23","author":"Sun","year":"2023","journal-title":"IEEE Sens. J."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"3747","DOI":"10.1109\/TWC.2017.2688328","article-title":"Energy-efficient UAV communication with trajectory optimization","volume":"16","author":"Zeng","year":"2017","journal-title":"IEEE Trans. Wirel. Commun."}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/16\/11\/398\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:24:14Z","timestamp":1760113454000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/16\/11\/398"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,30]]},"references-count":34,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2024,11]]}},"alternative-id":["fi16110398"],"URL":"https:\/\/doi.org\/10.3390\/fi16110398","relation":{},"ISSN":["1999-5903"],"issn-type":[{"value":"1999-5903","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,30]]}}}