{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,9]],"date-time":"2026-05-09T16:53:46Z","timestamp":1778345626790,"version":"3.51.4"},"reference-count":45,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2024,4,9]],"date-time":"2024-04-09T00:00:00Z","timestamp":1712620800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Natural Science Foundation of China","award":["62062047"],"award-info":[{"award-number":["62062047"]}]},{"name":"National Natural Science Foundation of China","award":["202202AD080006"],"award-info":[{"award-number":["202202AD080006"]}]},{"name":"National Natural Science Foundation of China","award":["202105AG070010"],"award-info":[{"award-number":["202105AG070010"]}]},{"name":"major scientific and technological projects in Yunnan Province","award":["62062047"],"award-info":[{"award-number":["62062047"]}]},{"name":"major scientific and technological projects in Yunnan Province","award":["202202AD080006"],"award-info":[{"award-number":["202202AD080006"]}]},{"name":"major scientific and technological projects in Yunnan Province","award":["202105AG070010"],"award-info":[{"award-number":["202105AG070010"]}]},{"name":"Yuxi Normal University","award":["62062047"],"award-info":[{"award-number":["62062047"]}]},{"name":"Yuxi Normal University","award":["202202AD080006"],"award-info":[{"award-number":["202202AD080006"]}]},{"name":"Yuxi Normal University","award":["202105AG070010"],"award-info":[{"award-number":["202105AG070010"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>With the emergence of wireless rechargeable sensor networks (WRSNs), the possibility of wirelessly recharging nodes using mobile charging vehicles (MCVs) has become a reality. However, existing approaches overlook the effective integration of node energy replenishment and mobile data collection processes. In this paper, we propose a joint energy replenishment and data collection scheme (D-JERDG) for WRSNs based on deep reinforcement learning. By capitalizing on the high mobility of unmanned aerial vehicles (UAVs), D-JERDG enables continuous visits to the cluster head nodes in each cluster, facilitating data collection and range-based charging. First, D-JERDG utilizes the K-means algorithm to partition the network into multiple clusters, and a cluster head selection algorithm is proposed based on an improved dynamic routing protocol, which elects cluster head nodes based on the remaining energy and geographical location of the cluster member nodes. Afterward, the simulated annealing (SA) algorithm determines the shortest flight path. Subsequently, the DRL model multiobjective deep deterministic policy gradient (MODDPG) is employed to control and optimize the UAV instantaneous heading and speed, effectively planning UAV hover points. By redesigning the reward function, joint optimization of multiple objectives such as node death rate, UAV throughput, and average flight energy consumption is achieved. Extensive simulation results show that the proposed D-JERDG achieves joint optimization of multiple objectives and exhibits significant advantages over the baseline in terms of throughput, time utilization, and charging cost, among other indicators.<\/jats:p>","DOI":"10.3390\/s24082386","type":"journal-article","created":{"date-parts":[[2024,4,9]],"date-time":"2024-04-09T04:37:27Z","timestamp":1712637447000},"page":"2386","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Deep-Reinforcement-Learning-Based Joint Energy Replenishment and Data Collection Scheme for WRSN"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-6188-0941","authenticated-orcid":false,"given":"Jishan","family":"Li","sequence":"first","affiliation":[{"name":"Yunnan Key Laboratory of Computer Technology Applications, Kunming University of Science and Technology, Kunming 650500, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-0588-3731","authenticated-orcid":false,"given":"Zhichao","family":"Deng","sequence":"additional","affiliation":[{"name":"Yunnan Key Laboratory of Computer Technology Applications, Kunming University of Science and Technology, Kunming 650500, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0908-1623","authenticated-orcid":false,"given":"Yong","family":"Feng","sequence":"additional","affiliation":[{"name":"Yunnan Key Laboratory of Computer Technology Applications, Kunming University of Science and Technology, Kunming 650500, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3014-1889","authenticated-orcid":false,"given":"Nianbo","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,4,9]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1126\/science.1143254","article-title":"Wireless power transfer via strongly coupled magnetic resonances","volume":"317","author":"Kurs","year":"2007","journal-title":"Science"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"044102","DOI":"10.1063\/1.3284651","article-title":"Simultaneous mid-range power transfer to multiple devices","volume":"96","author":"Kurs","year":"2010","journal-title":"Appl. Phys. Lett."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"949","DOI":"10.1109\/TNET.2022.3199389","article-title":"Robust Wireless Rechargeable Sensor Networks","volume":"31","author":"Yang","year":"2023","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Rodi\u0107, A., and Mester, G. (2010, January 10\u201311). Virtual WRSN\u2014Modeling and simulation of wireless robot-sensor networked systems. Proceedings of the IEEE 8th International Symposium on Intelligent Systems and Informatics, Subotica, Serbia.","DOI":"10.1109\/SISY.2010.5647245"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"3877","DOI":"10.1109\/TVT.2021.3066216","article-title":"Early Wake-Up Ahead Node for Fast Code Dissemination in Wireless Sensor Networks","volume":"70","author":"Deng","year":"2021","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_6","unstructured":"Schurgers, C., and Srivastava, M. (2001, January 28\u201331). Energy efficient routing in wireless sensor networks. Proceedings of the 2001 MILCOM Proceedings Communications for Network-Centric Operations: Creating the Information Force (Cat. No.01CH37277), McLean, VA, USA."},{"key":"ref_7","first-page":"275","article-title":"Data aggregation in underwater wireless sensor network: Recent approaches and issues","volume":"31","author":"Goyal","year":"2019","journal-title":"J. King Saud Univ.-Comput. Inf. Sci."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"4483","DOI":"10.1109\/TMC.2023.3294391","article-title":"Omnidirectional Chargability with Directional Antennas","volume":"23","author":"Dai","year":"2023","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"3589","DOI":"10.1109\/TMC.2021.3136967","article-title":"Placing Wireless Chargers With Limited Mobility","volume":"22","author":"Dai","year":"2023","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1109\/MCOM.2016.7470933","article-title":"Wireless communications with unmanned aerial vehicles: Opportunities and challenges","volume":"54","author":"Zeng","year":"2016","journal-title":"IEEE Commun. Mag."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Pedditi, R.B., and Debasis, K. (2023). Energy Efficient Routing Protocol for an IoT-Based WSN System to Detect Forest Fires. Appl. Sci., 13.","DOI":"10.3390\/app13053026"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"da Silva, R.I., Del Duca Almeida, V., Poersch, A.M., and Nogueira, J.M.S. (2010, January 19\u201323). Wireless sensor network for disaster management. Proceedings of the 2010 IEEE Network Operations and Management Symposium\u2014NOMS 2010, Osaka, Japan.","DOI":"10.1109\/NOMS.2010.5488351"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Yang, J., Wang, X., Li, Z., Yang, P., Luo, X., Zhang, K., Zhang, S., and Chen, L. (2016, January 12\u201315). Path planning of unmanned aerial vehicles for farmland information monitoring based on WSN. Proceedings of the 2016 12th World Congress on Intelligent Control and Automation (WCICA), Guilin, China.","DOI":"10.1109\/WCICA.2016.7578794"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Liu, Y., Pan, H., Sun, G., and Wang, A. (2021, January 5\u20138). Scheduling Optimization of Charging UAV in Wireless Rechargeable Sensor Networks. Proceedings of the 2021 IEEE Symposium on Computers and Communications (ISCC), Athens, Greece.","DOI":"10.1109\/ISCC53001.2021.9631448"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Park, S.Y., Jeong, D., Shin, C.S., and Lee, H. (2017, January 4\u20138). DroneNet+: Adaptive Route Recovery Using Path Stitching of UAVs in Ad-Hoc Networks. Proceedings of the GLOBECOM 2017\u20142017 IEEE Global Communications Conference, Singapore.","DOI":"10.1109\/GLOCOM.2017.8253970"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"3899","DOI":"10.1109\/JIOT.2021.3102185","article-title":"Trajectory Design for UAV-Based Internet of Things Data Collection: A Deep Reinforcement Learning Approach","volume":"9","author":"Wang","year":"2022","journal-title":"IEEE Internet Things J."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"2109","DOI":"10.1109\/TWC.2017.2789293","article-title":"Joint Trajectory and Communication Design for Multi-UAV Enabled Wireless Networks","volume":"17","author":"Wu","year":"2018","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1741","DOI":"10.1109\/TVT.2019.2959808","article-title":"Energy-Efficient UAV Routing for Wireless Sensor Networks","volume":"69","author":"Baek","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Gendreau, M., and Potvin, J.Y. (2019). Handbook of Metaheuristics, Springer International Publishing.","DOI":"10.1007\/978-3-319-91086-4"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"6361","DOI":"10.1109\/TCOMM.2021.3089476","article-title":"Multi-Objective Optimization for UAV-Assisted Wireless Powered IoT Networks Based on Extended DDPG Algorithm","volume":"69","author":"Yu","year":"2021","journal-title":"IEEE Trans. Commun."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"109196","DOI":"10.1016\/j.comnet.2022.109196","article-title":"A many-objective optimization charging scheme for wireless rechargeable sensor networks via mobile charging vehicles","volume":"215","author":"Li","year":"2022","journal-title":"Comput. Netw."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"10306","DOI":"10.1109\/JSEN.2020.2990641","article-title":"An Efficient Combined Charging Strategy for Large-Scale Wireless Rechargeable Sensor Networks","volume":"20","author":"Dong","year":"2020","journal-title":"IEEE Sens. J."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"990","DOI":"10.1109\/TVT.2008.926073","article-title":"Hausdorff Clustering and Minimum Energy Routing for Wireless Sensor Networks","volume":"58","author":"Zhu","year":"2009","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2117","DOI":"10.1007\/s11277-014-1629-y","article-title":"Heterogeneous HEED protocol for wireless sensor networks","volume":"77","author":"Chand","year":"2014","journal-title":"Wirel. Pers. Commun."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Wu, Q., Sun, P., and Boukerche, A. (2020, January 7\u201311). A Novel Joint Data Gathering and Wireless Charging Scheme for Sustainable Wireless Sensor Networks. Proceedings of the ICC 2020\u20142020 IEEE International Conference on Communications (ICC), Dublin, Ireland.","DOI":"10.1109\/ICC40277.2020.9148735"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Li, B., Xiao, X., Tang, S., and Ning, W. (2021, January 14\u201316). An Energy Efficiency-oriented Routing Protocol for Wireless Rechargeable Sensor Networks. Proceedings of the 2021 IEEE Asia-Pacific Conference on Image Processing, Electronics and Computers (IPEC), Dalian, China.","DOI":"10.1109\/IPEC51340.2021.9421204"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"2596","DOI":"10.1109\/JIOT.2017.2784478","article-title":"A Joint Energy Replenishment and Data Collection Algorithm in Wireless Rechargeable Sensor Networks","volume":"5","author":"Han","year":"2018","journal-title":"IEEE Internet Things J."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"2689","DOI":"10.1109\/TMC.2014.2307335","article-title":"A Framework of Joint Mobile Energy Replenishment and Data Gathering in Wireless Rechargeable Sensor Networks","volume":"13","author":"Zhao","year":"2014","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1016\/j.inffus.2020.07.005","article-title":"Renewable energy harvesting schemes in wireless sensor networks: A Survey","volume":"63","author":"Sah","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1109\/MNET.2019.1800250","article-title":"UAV Network and IoT in the Sky for Future Smart Cities","volume":"33","author":"Qi","year":"2019","journal-title":"IEEE Netw."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"5092","DOI":"10.1109\/TWC.2018.2838134","article-title":"UAV-Enabled Wireless Power Transfer: Trajectory Design and Energy Optimization","volume":"17","author":"Xu","year":"2018","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"11796","DOI":"10.1109\/JIOT.2021.3132015","article-title":"Joint Scheduling and Trajectory Optimization of Charging UAV in Wireless Rechargeable Sensor Networks","volume":"9","author":"Liu","year":"2022","journal-title":"IEEE Internet Things J."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"4207","DOI":"10.1109\/TVT.2020.2969220","article-title":"Trajectory Optimization for UAVs\u2019 Efficient Charging in Wireless Rechargeable Sensor Networks","volume":"69","author":"Wu","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1109\/TSUSC.2022.3224442","article-title":"UAV Dispatch Planning for a Wireless Rechargeable Sensor Network for Bridge Monitoring","volume":"8","author":"Zhao","year":"2023","journal-title":"IEEE Trans. Sustain. Comput."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1327","DOI":"10.1109\/JIOT.2019.2954530","article-title":"Optimal UAV Route in Wireless Charging Sensor Networks","volume":"7","author":"Baek","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Lin, C., Guo, C., Du, W., Deng, J., Wang, L., and Wu, G. (2019, January 10\u201313). Maximizing Energy Efficiency of Period-Area Coverage with UAVs for Wireless Rechargeable Sensor Networks. Proceedings of the 2019 16th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON), Boston, MA, USA.","DOI":"10.1109\/SAHCN.2019.8824918"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1211","DOI":"10.1109\/JIOT.2020.3012835","article-title":"AoI-Minimal Trajectory Planning and Data Collection in UAV-Assisted Wireless Powered IoT Networks","volume":"8","author":"Hu","year":"2021","journal-title":"IEEE Internet Things J."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"110446","DOI":"10.1109\/ACCESS.2020.3002538","article-title":"A UAV-Assisted Data Collection for Wireless Sensor Networks: Autonomous Navigation and Scheduling","volume":"8","author":"Bouhamed","year":"2020","journal-title":"IEEE Access"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1204","DOI":"10.1109\/TIV.2022.3213703","article-title":"DRL-UTPS: DRL-Based Trajectory Planning for Unmanned Aerial Vehicles for Data Collection in Dynamic IoT Network","volume":"8","author":"Liu","year":"2023","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"4179","DOI":"10.1109\/JIOT.2021.3102831","article-title":"LSTM-Characterized Deep Reinforcement Learning for Continuous Flight Control and Resource Allocation in UAV-Assisted Sensor Network","volume":"9","author":"Li","year":"2022","journal-title":"IEEE Internet Things J."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1016\/j.comcom.2023.02.021","article-title":"Multi-UAV WRSN charging path planning based on improved heed and IA-DRL","volume":"203","author":"Shan","year":"2023","journal-title":"Comput. Commun."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Liu, N., Zhang, J., Luo, C., Cao, J., Hong, Y., Chen, Z., and Chen, T. (2023). Dynamic Charging Strategy Optimization for UAV-Assisted Wireless Rechargeable Sensor Networks Based On Deep Q-network. IEEE Internet Things J., 11.","DOI":"10.1109\/JIOT.2023.3345311"},{"key":"ref_43","first-page":"1485","article-title":"Dynamic Spatiotemporal Charging Scheduling Based on Deep Reinforcement Learning for WRSN","volume":"35","author":"Wang","year":"2023","journal-title":"J. Softw."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J."},{"key":"ref_45","unstructured":"Heinzelman, W., Chandrakasan, A., and Balakrishnan, H. (2000, January 4\u20137). Energy-efficient communication protocol for wireless microsensor networks. Proceedings of the 33rd Annual Hawaii International Conference on System Sciences, Maui, Hawaii."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/8\/2386\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:25:18Z","timestamp":1760106318000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/8\/2386"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,9]]},"references-count":45,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2024,4]]}},"alternative-id":["s24082386"],"URL":"https:\/\/doi.org\/10.3390\/s24082386","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,9]]}}}