{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,12]],"date-time":"2025-11-12T14:26:08Z","timestamp":1762957568381,"version":"build-2065373602"},"reference-count":42,"publisher":"MDPI AG","issue":"23","license":[{"start":{"date-parts":[[2024,11,23]],"date-time":"2024-11-23T00:00:00Z","timestamp":1732320000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key R&amp;D Program of China","doi-asserted-by":"publisher","award":["2021YFB2900200","2021ZYA05003","JCYJ20220818102209020","2023B1515120093"],"award-info":[{"award-number":["2021YFB2900200","2021ZYA05003","JCYJ20220818102209020","2023B1515120093"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Ministry of Education of China University Innovation Funds","award":["2021YFB2900200","2021ZYA05003","JCYJ20220818102209020","2023B1515120093"],"award-info":[{"award-number":["2021YFB2900200","2021ZYA05003","JCYJ20220818102209020","2023B1515120093"]}]},{"name":"Key Natural Science Foundation of Shenzhen","award":["2021YFB2900200","2021ZYA05003","JCYJ20220818102209020","2023B1515120093"],"award-info":[{"award-number":["2021YFB2900200","2021ZYA05003","JCYJ20220818102209020","2023B1515120093"]}]},{"name":"Guangdong Basic and Applied Basic Research Fund Regional Joint Fund","award":["2021YFB2900200","2021ZYA05003","JCYJ20220818102209020","2023B1515120093"],"award-info":[{"award-number":["2021YFB2900200","2021ZYA05003","JCYJ20220818102209020","2023B1515120093"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>UAV-assisted communication facilitates efficient data collection from IoT nodes by exploiting UAVs\u2019 flexible deployment and wide coverage capabilities. In this paper, we consider a scenario in which UAVs equipped with high-precision sensors collect sensing data from ground terminals (GTs) in real-time over a wide geographic area and transmit the collected data to a ground base station (BS). Our research aims to jointly optimize the trajectory scheduling and the allocation of collection time slots for multiple UAVs, to maximize the system\u2019s data collection rates and fairness while minimizing energy consumption within the task deadline. Due to UAVs\u2019 limited sensing distance and battery energy, ensuring timely data processing in target areas presents a challenge. To address this issue, we propose a novel constraint optimization-based deep reinforcement learning\u2013Lagrangian UAV real-time data collection management (CDRLL\u2014RDCM) framework utilizing centralized training and distributed execution. In this framework, a CNN\u2013GRU network units extract spatial and temporal features of the environmental information. We then introduce the PPO\u2013Lagrangian algorithm to iteratively update the policy network and Lagrange multipliers at different time scales, enabling the learning of more effective collaborative policies for real-time UAV decision-making. Extensive simulations show that our proposed framework significantly improves the efficiency of multi-UAV collaboration and substantially reduces data staleness.<\/jats:p>","DOI":"10.3390\/rs16234378","type":"journal-article","created":{"date-parts":[[2024,11,25]],"date-time":"2024-11-25T08:38:24Z","timestamp":1732523904000},"page":"4378","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Real-Time Data Collection and Trajectory Scheduling Using a DRL\u2013Lagrangian Framework in Multiple UAVs Collaborative Communication Systems"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-8707-5065","authenticated-orcid":false,"given":"Shanshan","family":"Wang","sequence":"first","affiliation":[{"name":"School of Electronics and Communication Engineering, Sun Yat-sen University, Shenzhen 518107, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4084-4027","authenticated-orcid":false,"given":"Zhiyong","family":"Luo","sequence":"additional","affiliation":[{"name":"School of Electronics and Communication Engineering, Sun Yat-sen University, Shenzhen 518107, China"},{"name":"Peng Cheng Laboratory, Shenzhen 518055, China"}]}],"member":"1968","published-online":{"date-parts":[[2024,11,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1327","DOI":"10.1109\/JIOT.2019.2954530","article-title":"Optimal UAV route in wireless charging sensor networks","volume":"7","author":"Baek","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"24300","DOI":"10.1109\/JIOT.2022.3189214","article-title":"UAV trajectory optimization for time-constrained data collection in UAV-enabled environmental monitoring systems","volume":"9","author":"Liu","year":"2022","journal-title":"IEEE Internet Things J."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Chen, X., Li, Q., Li, R., Cai, X., Wei, J., and Zhao, H. (2023). UAV network path planning and optimization using a vehicle routing model. Remote Sens., 15.","DOI":"10.3390\/rs15092227"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"6898","DOI":"10.1109\/TVT.2021.3086556","article-title":"Energy-constrained UAV data collection systems: NOMA and OMA","volume":"70","author":"Mu","year":"2021","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Zhao, L., Shi, Y., Liu, B., Hovis, C., Duan, Y., and Shi, Z. (2019). Finer classification of crops by fusing UAV images and Sentinel-2A data. Remote Sens., 11.","DOI":"10.3390\/rs11243012"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Yang, M.D., Tseng, H.H., Hsu, Y.C., Yang, C.Y., Lai, M.H., and Wu, D.H. (2021). A UAV open dataset of rice paddies for deep learning practice. Remote Sens., 13.","DOI":"10.3390\/rs13071358"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1109\/TII.2023.3256375","article-title":"Robust trajectory and offloading for energy-efficient UAV edge computing in industrial Internet of Things","volume":"20","author":"Tang","year":"2024","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2059","DOI":"10.1109\/JSAC.2018.2864373","article-title":"Energy-efficient UAV control for effective and fair communication coverage: A deep reinforcement learning approach","volume":"36","author":"Liu","year":"2018","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1355","DOI":"10.1109\/TCOMM.2023.3331029","article-title":"Computing over the sky: Joint UAV trajectory and task offloading scheme based on optimization-embedding multi-agent deep reinforcement learning","volume":"72","author":"Li","year":"2024","journal-title":"IEEE Trans. Commun."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"5725","DOI":"10.1109\/TWC.2023.3235997","article-title":"Energy efficient computation offloading in aerial edge networks with multi-agent cooperation","volume":"22","author":"Liu","year":"2023","journal-title":"IEEE Trans. Wireless Commun."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Jia, Z., Qin, X., Wang, Z., and Liu, B. (2019, January 20\u201324). Age-based path planning and data acquisition in UAV-assisted IoT networks. Proceedings of the 2019 IEEE International Conference on Communications Workshops (ICC Workshops), Shanghai, China.","DOI":"10.1109\/ICCW.2019.8756751"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1250","DOI":"10.1109\/JSAC.2021.3065049","article-title":"Neural combinatorial deep reinforcement learning for age-optimal joint trajectory and scheduling design in UAV-assisted networks","volume":"39","author":"Ferdowsi","year":"2021","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"11214","DOI":"10.1109\/JIOT.2021.3126329","article-title":"Joint optimization of UAV trajectory and sensor uploading powers for UAV-assisted data collection in wireless sensor networks","volume":"9","author":"Wang","year":"2022","journal-title":"IEEE Internet Things J."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Wang, D., Wu, M., He, Y., Pang, L., Xu, Q., and Zhang, R. (2022). An HAP and UAVs Collaboration Framework for Uplink Secure Rate Maximization in NOMA-Enabled IoT Networks. Remote Sens., 14.","DOI":"10.3390\/rs14184501"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1122","DOI":"10.1109\/JIOT.2019.2952364","article-title":"Energy-efficient data collection and device positioning in UAV-assisted IoT","volume":"7","author":"Wang","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"182302","DOI":"10.1007\/s11432-021-3343-7","article-title":"Time and energy efficient data collection via UAV","volume":"65","author":"Wang","year":"2022","journal-title":"Sci. China Inf. Sci."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1109\/MWC.2018.1800023","article-title":"Cellular-Connected UAV: Potential, Challenges, and Promising Technologies","volume":"26","author":"Zeng","year":"2019","journal-title":"IEEE Wireless Commun."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"4095","DOI":"10.1109\/TVT.2023.3323455","article-title":"UAV-Aided Information Collection Method Based on Joint Optimization of Node Access and Flight Strategy","volume":"73","author":"Han","year":"2024","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"2740","DOI":"10.1109\/TCOMM.2023.3255240","article-title":"Cooperative Data Collection With Multiple UAVs for Information Freshness in the Internet of Things","volume":"71","author":"Wang","year":"2023","journal-title":"IEEE Trans. Commun."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"17700","DOI":"10.1109\/JIOT.2024.3362321","article-title":"Joint cooperative computation offloading and trajectory optimization in heterogeneous UAV-swarm-enabled aerial edge computing networks","volume":"11","author":"Yu","year":"2024","journal-title":"IEEE Internet Things J."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"3884","DOI":"10.1109\/TWC.2022.3222067","article-title":"Joint User Scheduling and UAV Trajectory Design on Completion Time Minimization for UAV-Aided Data Collection","volume":"22","author":"Yuan","year":"2023","journal-title":"IEEE Trans. Wireless Commun."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"158","DOI":"10.1109\/TCOMM.2022.3224751","article-title":"Trajectory Design and Power Control for Joint Radar and Communication Enabled Multi-UAV Cooperative Detection Systems","volume":"71","author":"Zhang","year":"2023","journal-title":"IEEE Trans. Commun."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"7957","DOI":"10.1109\/TVT.2019.2920284","article-title":"Trajectory Design and Power Control for Multi-UAV Assisted Wireless Networks: A Machine Learning Approach","volume":"68","author":"Liu","year":"2019","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"15460","DOI":"10.1109\/JIOT.2022.3176903","article-title":"UAV-Assisted Data Collection for Internet of Things: A Survey","volume":"9","author":"Wei","year":"2022","journal-title":"IEEE Internet Things J."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"16663","DOI":"10.1109\/JIOT.2022.3153585","article-title":"Learning-Based UAV Path Planning for Data Collection With Integrated Collision Avoidance","volume":"9","author":"Wang","year":"2022","journal-title":"IEEE Internet Things J."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"8812","DOI":"10.1109\/TVT.2023.3244812","article-title":"Joint Resource and Trajectory Optimization for Heterogeneous-UAVs Enabled Aerial-Ground Cooperative Computing Networks","volume":"72","author":"Hu","year":"2023","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Liu, W., Jin, Z., Zhang, X., Zang, W., Wang, S., and Shen, Y. (2023, January 10\u201312). AoI-Aware UAV-Enabled Marine MEC Networks with Integrated Sensing, Computation, and Communication. Proceedings of the 2023 IEEE International Conference on Communications Workshops (ICC Workshops), Dalian, China.","DOI":"10.1109\/ICCCWorkshops57813.2023.10233771"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Wang, S., and Luo, Z. (2024). An Energy-Efficient Scheme Design for NOMA-Based UAV-Assisted MEC Systems. Electronics, 13.","DOI":"10.3390\/electronics13214240"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"16830","DOI":"10.1109\/JIOT.2023.3270332","article-title":"Unmanned-Aerial-Vehicle-Aided Integrated Sensing and Computation With Mobile-Edge Computing","volume":"10","author":"Huang","year":"2023","journal-title":"IEEE Internet Things J."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1109\/TCCN.2020.3027695","article-title":"Multi-agent deep reinforcement learning-based trajectory planning for multi-UAV assisted mobile edge computing","volume":"7","author":"Wang","year":"2021","journal-title":"IEEE Trans. Cogn. Commun. Networking"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"21215","DOI":"10.1109\/JIOT.2024.3366506","article-title":"Periodic Collaboration and Real-Time Dispatch Using an Actor\u2013Critic Framework for UAV Movement in Mobile Edge Computing","volume":"11","author":"Zeng","year":"2024","journal-title":"IEEE Internet Things J."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1495","DOI":"10.1109\/TWC.2022.3204794","article-title":"Cooperative Trajectory Design of Multiple UAV Base Stations With Heterogeneous Graph Neural Networks","volume":"22","author":"Zhang","year":"2023","journal-title":"IEEE Trans. Wireless Commun."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"10754","DOI":"10.1109\/JIOT.2023.3327121","article-title":"Parameterized Deep Reinforcement Learning With Hybrid Action Space for Edge Task Offloading","volume":"11","author":"Wang","year":"2024","journal-title":"IEEE Internet Things J."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1109\/TMC.2019.2938509","article-title":"Distributed and Energy-Efficient Mobile Crowdsensing with Charging Stations by Deep Reinforcement Learning","volume":"20","author":"Liu","year":"2021","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"124967","DOI":"10.1016\/j.energy.2022.124967","article-title":"A multi-energy load forecasting method based on parallel architecture CNN-GRU and transfer learning for data deficient integrated energy systems","volume":"259","author":"Li","year":"2022","journal-title":"Energy"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1101","DOI":"10.1109\/JSAC.2020.3018804","article-title":"Constrained Deep Reinforcement Learning for Energy Sustainable Multi-UAV Based Random Access IoT Networks With NOMA","volume":"39","author":"Khairy","year":"2021","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"450","DOI":"10.1109\/TWC.2023.3278460","article-title":"Minimizing the AoI in Resource-Constrained Multi-Source Relaying Systems: Dynamic and Learning-Based Scheduling","volume":"23","author":"Zakeri","year":"2024","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"114814","DOI":"10.1109\/ACCESS.2022.3218216","article-title":"Deep Reinforcement Learning Under Signal Temporal Logic Constraints Using Lagrangian Relaxation","volume":"10","author":"Ikemoto","year":"2022","journal-title":"IEEE Access"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"119191","DOI":"10.1016\/j.eswa.2022.119191","article-title":"Proximal Policy Optimization Algorithm for Dynamic Pricing with Online Reviews","volume":"213","author":"Chao","year":"2023","journal-title":"Expert Syst. Appl."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"140","DOI":"10.1109\/MCOMSTD.0001.1900021","article-title":"An Overview of 3GPP Release-15 Study on Enhanced LTE Support for Connected Drones","volume":"5","author":"Muruganathan","year":"2021","journal-title":"IEEE Commun. Stand. Mag."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.compeleceng.2019.01.028","article-title":"Data Collection Using Unmanned Aerial Vehicles for Internet of Things Platforms","volume":"75","author":"Goudarzi","year":"2019","journal-title":"Comput. Electr. Eng."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"11599","DOI":"10.1109\/TVT.2020.3014788","article-title":"UAV-Enabled Secure Communications by Multi-Agent Deep Reinforcement Learning","volume":"69","author":"Zhang","year":"2020","journal-title":"IEEE Trans. Veh. Technol."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/23\/4378\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:38:04Z","timestamp":1760114284000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/23\/4378"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,23]]},"references-count":42,"journal-issue":{"issue":"23","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["rs16234378"],"URL":"https:\/\/doi.org\/10.3390\/rs16234378","relation":{},"ISSN":["2072-4292"],"issn-type":[{"type":"electronic","value":"2072-4292"}],"subject":[],"published":{"date-parts":[[2024,11,23]]}}}