{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T13:59:37Z","timestamp":1784123977265,"version":"3.55.0"},"reference-count":38,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2023,4,12]],"date-time":"2023-04-12T00:00:00Z","timestamp":1681257600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62062047"],"award-info":[{"award-number":["62062047"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["202202AD080006"],"award-info":[{"award-number":["202202AD080006"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Major scientific and technological projects in Yunnan Province","award":["62062047"],"award-info":[{"award-number":["62062047"]}]},{"name":"Major scientific and technological projects in Yunnan Province","award":["202202AD080006"],"award-info":[{"award-number":["202202AD080006"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Wireless rechargeable sensor networks (WRSN) have been emerging as an effective solution to the energy constraint problem of wireless sensor networks (WSN). However, most of the existing charging schemes use Mobile Charging (MC) to charge nodes one-to-one and do not optimize MC scheduling from a more comprehensive perspective, leading to difficulties in meeting the huge energy demand of large-scale WSNs; therefore, one-to-multiple charging which can charge multiple nodes simultaneously may be a more reasonable choice. To achieve timely and efficient energy replenishment for large-scale WSN, we propose an online one-to-multiple charging scheme based on Deep Reinforcement Learning, which utilizes Double Dueling DQN (3DQN) to jointly optimize the scheduling of both the charging sequence of MC and the charging amount of nodes. The scheme cellularizes the whole network based on the effective charging distance of MC and uses 3DQN to determine the optimal charging cell sequence with the objective of minimizing dead nodes and adjusting the charging amount of each cell being recharged according to the nodes\u2019 energy demand in the cell, the network survival time, and MC\u2019s residual energy. To obtain better performance and timeliness to adapt to the varying environments, our scheme further utilizes Dueling DQN to improve the stability of training and uses Double DQN to reduce overestimation. Extensive simulation experiments show that our proposed scheme achieves better charging performance compared with several existing typical works, and it has significant advantages in terms of reducing node dead ratio and charging latency.<\/jats:p>","DOI":"10.3390\/s23083903","type":"journal-article","created":{"date-parts":[[2023,4,12]],"date-time":"2023-04-12T02:08:11Z","timestamp":1681265291000},"page":"3903","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Deep Reinforcement Learning\u2013Based Online One-to-Multiple Charging Scheme in Wireless Rechargeable Sensor Network"],"prefix":"10.3390","volume":"23","author":[{"given":"Zheng","family":"Gong","sequence":"first","affiliation":[{"name":"Yunnan Key Laboratory of Computer Technology Applications, Kunming University of Science and Technology, Kunming 650500, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hao","family":"Wu","sequence":"additional","affiliation":[{"name":"Yunnan Key Laboratory of Computer Technology Applications, Kunming University of Science and Technology, Kunming 650500, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0908-1623","authenticated-orcid":false,"given":"Yong","family":"Feng","sequence":"additional","affiliation":[{"name":"Yunnan Key Laboratory of Computer Technology Applications, Kunming University of Science and Technology, Kunming 650500, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nianbo","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,4,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2155","DOI":"10.1109\/TMC.2018.2793222","article-title":"\u03c0-Splicer: Perceiving Accurate CSI Phases with Commodity WiFi Devices","volume":"17","author":"Zhu","year":"2018","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1770","DOI":"10.1109\/TPDS.2015.2467392","article-title":"POST: Exploiting Dynamic Sociality for Mobile Advertising in Vehicular Networks","volume":"27","author":"Qin","year":"2016","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1931","DOI":"10.1109\/TMC.2012.161","article-title":"Energy Provisioning in Wireless Rechargeable Sensor Networks","volume":"12","author":"He","year":"2013","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2250","DOI":"10.1109\/TNET.2017.2684159","article-title":"Joint charging tour planning and depot positioning for wireless sensor networks using mobile chargers","volume":"25","author":"Jiang","year":"2017","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1591","DOI":"10.1109\/TNET.2018.2841420","article-title":"Charging utility maximization in wireless rechargeable sensor networks by charging multiple sensors simultaneously","volume":"26","author":"Ma","year":"2018","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Xu, W., Liang, W., Kan, H., Xu, Y., and Zhang, X. (2019, January 7\u20139). Minimizing the longest charge delay of multiple mobile chargers for wireless rechargeable sensor networks by charging multiple sensors simultaneously. Proceedings of the IEEE 39th International Conference on Distributed Computing Systems, Dallas, TX, USA.","DOI":"10.1109\/ICDCS.2019.00092"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2478","DOI":"10.1109\/TNET.2021.3095280","article-title":"Minimizing charging delay for directional charging","volume":"29","author":"Lin","year":"2021","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"13842","DOI":"10.1109\/JSEN.2020.3004758","article-title":"Wireless mobile charger excursion optimization algorithm in wireless rechargeable sensor networks","volume":"20","author":"Malebary","year":"2020","journal-title":"IEEE Sens. J."},{"key":"ref_9","unstructured":"Wu, T., Yang, P., Dai, H., Xu, W., and Xu, M. (May, January 29). Collaborated tasks-driven mobile charging and scheduling: A near optimal result. Proceedings of the IEEE INFOCOM, Paris, France."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1016\/j.comcom.2018.05.002","article-title":"Adaptive online mobile charging for node failure avoidance in wireless rechargeable sensor networks","volume":"126","author":"Zhu","year":"2018","journal-title":"Comput. Commun."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1016\/j.comnet.2018.10.023","article-title":"Double warning thresholds for preemptive charging scheduling in wireless rechargeable sensor networks","volume":"148","author":"Lin","year":"2019","journal-title":"Comput. Netw."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1109\/TMC.2017.2703094","article-title":"TSCA: A Temporal-Spatial Real-Time Charging Scheduling Algorithm for On-Demand Architecture in Wireless Rechargeable Sensor Networks","volume":"17","author":"Lin","year":"2018","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1016\/j.jnca.2018.02.017","article-title":"An efficient scheduling scheme for mobile charger in on-demand wireless rechargeable sensor networks","volume":"114","author":"Kaswan","year":"2018","journal-title":"J. Netw. Comput. Appl."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"101074","DOI":"10.1016\/j.pmcj.2019.101074","article-title":"An efficient scheduling scheme for on-demand mobile charging in wireless rechargeable sensor networks","volume":"59","author":"Tomar","year":"2019","journal-title":"Pervasive Mob. Comput."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Wei, Z., Liu, F., Lyu, Z., Ding, X., Shi, L., and Xia, C. (2018, January 20\u201322). Reinforcement learning for a novel mobile charging strategy in wireless rechargeable sensor networks. Proceedings of the International Conference on Wireless Algorithms, Systems, and Applications, Tianjin, China.","DOI":"10.1007\/978-3-319-94268-1_40"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"102278","DOI":"10.1016\/j.adhoc.2020.102278","article-title":"A deep reinforcement learning-based on-demand charging algorithm for wireless rechargeable sensor networks","volume":"110","author":"Cao","year":"2021","journal-title":"Ad Hoc Netw."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"370","DOI":"10.1109\/JIOT.2020.3005598","article-title":"Dynamic Charging Scheme Problem With Actor\u2013Critic Reinforcement Learning","volume":"8","author":"Yang","year":"2021","journal-title":"IEEE Internet Things J."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1861","DOI":"10.1109\/TMC.2014.2368557","article-title":"Evaluating the on-demand mobile charging in wireless sensor networks","volume":"14","author":"He","year":"2014","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"388","DOI":"10.1016\/j.asoc.2018.11.022","article-title":"Periodic charging planning for a mobile WCE in wireless rechargeable sensor networks based on hybrid PSO and GA algorithm","volume":"75","author":"Lyu","year":"2019","journal-title":"Appl. Soft Comput."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Xie, L., Shi, Y., Hou, Y.T., Lou, W., Sherali, H.D., and Midkiff, S.F. (2012, January 18\u201321). On renewable sensor networks with wireless energy transfer: The multi-node case. Proceedings of the 2012 9th Annual IEEE Communications Society Conference on Sensor, Mesh and ad hoc Communications and Networks (SECON), Seoul, Republic of Korea.","DOI":"10.1109\/SECON.2012.6275766"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.swevo.2016.12.005","article-title":"A survey of swarm intelligence for dynamic optimization: Algorithms and applications","volume":"33","author":"Mavrovouniotis","year":"2017","journal-title":"Swarm Evol. Comput."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"431","DOI":"10.1109\/TEVC.2021.3104343","article-title":"Dynamic Optimization in Fast-Changing Environments via Offline Evolutionary Search","volume":"26","author":"Lu","year":"2021","journal-title":"IEEE Trans. Evol. Comput."},{"key":"ref_23","first-page":"86","article-title":"Research on reinforcement learning technology: A review","volume":"30","author":"Gao","year":"2004","journal-title":"Acta Autom. Sin."},{"key":"ref_24","first-page":"643","article-title":"Deep learning for control: The state of the art and prospects","volume":"42","author":"Duan","year":"2016","journal-title":"Acta Autom. Sin."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Chen, T.S., Chen, J.J., Gao, X.Y., and Chen, T.C. (2022). Mobile Charging Strategy for Wireless Rechargeable Sensor Networks. Sensors, 22.","DOI":"10.3390\/s22010359"},{"key":"ref_26","first-page":"2377","article-title":"Scheduling on-demand charging request in wireless rechargeable sensor network with fruit fly optimization-based path selection","volume":"14","author":"Susan","year":"2022","journal-title":"Int. J. Inf. Technol."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"15267","DOI":"10.1007\/s00521-021-06146-9","article-title":"Dynamic mobile charger scheduling with partial charging strategy for WSNs using deep-Q-networks","volume":"33","author":"Banoth","year":"2021","journal-title":"Neural Comput. Appl."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Dudyala, A.K., and Ram, L.K. (2021, January 11\u201312). Improving the Lifetime of Wireless Rechargeable Sensors Using Mobile Charger in On-Demand Charging Environment Based on Energy Consumption Rate Prediction. Proceedings of the First International Conference on Computational Electronics for Wireless Communications: ICCWC 2021, Haryana, India.","DOI":"10.1007\/978-981-16-6246-1_58"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Liu, T., Wu, B., Zhang, S., Peng, J., and Xu, W. (2020, January 6\u20139). An Effective Multi-node Charging Scheme for Wireless Rechargeable Sensor Networks. Proceedings of the IEEE Conference on Computer Communications, Toronto, ON, Canada.","DOI":"10.1109\/INFOCOM41043.2020.9155262"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Lin, C., Wang, Z., Deng, J., Wang, L., Ren, J., and Wu, G. (2018, January 16\u201319). mTS: Temporal-and spatial-collaborative charging for wireless rechargeable sensor networks with multiple vehicles. Proceedings of the IEEE Conference on Computer Communications, Honolulu, HI, USA.","DOI":"10.1109\/INFOCOM.2018.8486402"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"679","DOI":"10.1109\/TSUSC.2020.3003014","article-title":"Utility-aware charging scheduling for multiple mobile chargers in large-scale wireless rechargeable sensor networks","volume":"6","author":"Ouyang","year":"2020","journal-title":"IEEE Trans. Sustain. Comput."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"437","DOI":"10.1109\/TNET.2014.2303979","article-title":"Multi-node wireless energy charging in sensor networks","volume":"23","author":"Xie","year":"2014","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"2608","DOI":"10.1109\/TIM.2008.925019","article-title":"Design of an RFID-Based Battery-Free Programmable Sensing Platform","volume":"57","author":"Sample","year":"2008","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Hu, W., Jin, Y., Wen, Y., Wang, Z., and Sun, L. (2017). Towards Wi-Fi AP-Assisted Content Prefetching for On-Demand TV Series: A Reinforcement Learning Approach. arXiv.","DOI":"10.1109\/TCSVT.2017.2684302"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Van Hasselt, H., Guez, A., and Silver, D. (2016, January 12\u201317). Deep reinforcement learning with double q-learning. Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"ref_37","unstructured":"Wang, Z., Schaul, T., Hessel, M., Hasselt, H., Lanctot, M., and Freitas, N. (2016, January 19\u201324). Dueling network architectures for deep reinforcement learning. Proceedings of the International Conference on Machine Learning, New York, NY, USA."},{"key":"ref_38","unstructured":"Schaul, T., Quan, J., Antonoglou, I., and Silver, D. (2015). Prioritized experience replay. arXiv."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/8\/3903\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:14:15Z","timestamp":1760123655000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/8\/3903"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,12]]},"references-count":38,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2023,4]]}},"alternative-id":["s23083903"],"URL":"https:\/\/doi.org\/10.3390\/s23083903","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,12]]}}}