{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T13:59:40Z","timestamp":1784123980056,"version":"3.55.0"},"reference-count":44,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2022,7,12]],"date-time":"2022-07-12T00:00:00Z","timestamp":1657584000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Natural Science Foundation","award":["62173032"],"award-info":[{"award-number":["62173032"]}]},{"name":"National Natural Science Foundation","award":["62173028"],"award-info":[{"award-number":["62173028"]}]},{"name":"National Natural Science Foundation","award":["BK20AF005"],"award-info":[{"award-number":["BK20AF005"]}]},{"name":"Foshan Science and Technology Innovation Special Project","award":["62173032"],"award-info":[{"award-number":["62173032"]}]},{"name":"Foshan Science and Technology Innovation Special Project","award":["62173028"],"award-info":[{"award-number":["62173028"]}]},{"name":"Foshan Science and Technology Innovation Special Project","award":["BK20AF005"],"award-info":[{"award-number":["BK20AF005"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>The breakthrough of wireless energy transmission (WET) technology has greatly promoted the wireless rechargeable sensor networks (WRSNs). A promising method to overcome the energy constraint problem in WRSNs is mobile charging by employing a mobile charger to charge sensors via WET. Recently, more and more studies have been conducted for mobile charging scheduling under dynamic charging environments, ignoring the consideration of the joint charging sequence scheduling and charging ratio control (JSSRC) optimal design. This paper will propose a novel attention-shared multi-agent actor\u2013critic-based deep reinforcement learning approach for JSSRC (AMADRL-JSSRC). In AMADRL-JSSRC, we employ two heterogeneous agents named charging sequence scheduler and charging ratio controller with an independent actor network and critic network. Meanwhile, we design the reward function for them, respectively, by considering the tour length and the number of dead sensors. The AMADRL-JSSRC trains decentralized policies in multi-agent environments, using a centralized computing critic network to share an attention mechanism, and it selects relevant policy information for each agent at every charging decision. Simulation results demonstrate that the proposed AMADRL-JSSRC can efficiently prolong the lifetime of the network and reduce the number of death sensors compared with the baseline algorithms.<\/jats:p>","DOI":"10.3390\/e24070965","type":"journal-article","created":{"date-parts":[[2022,7,12]],"date-time":"2022-07-12T20:52:41Z","timestamp":1657659161000},"page":"965","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":19,"title":["Attention-Shared Multi-Agent Actor\u2013Critic-Based Deep Reinforcement Learning Approach for Mobile Charging Dynamic Scheduling in Wireless Rechargeable Sensor Networks"],"prefix":"10.3390","volume":"24","author":[{"given":"Chengpeng","family":"Jiang","sequence":"first","affiliation":[{"name":"School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing 100083, China"},{"name":"Beijing Engineering Research Center of Industrial Spectrum Imaging, Beijing 100083, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ziyang","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Automation, Tsinghua University, Beijing 100084, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shuai","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing 100083, China"},{"name":"Beijing Engineering Research Center of Industrial Spectrum Imaging, Beijing 100083, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jinglin","family":"Li","sequence":"additional","affiliation":[{"name":"School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing 100083, China"},{"name":"Beijing Engineering Research Center of Industrial Spectrum Imaging, Beijing 100083, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Haoran","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing 100083, China"},{"name":"Beijing Engineering Research Center of Industrial Spectrum Imaging, Beijing 100083, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jinwei","family":"Xiang","sequence":"additional","affiliation":[{"name":"School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing 100083, China"},{"name":"Beijing Engineering Research Center of Industrial Spectrum Imaging, Beijing 100083, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2270-7889","authenticated-orcid":false,"given":"Wendong","family":"Xiao","sequence":"additional","affiliation":[{"name":"School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing 100083, China"},{"name":"Beijing Engineering Research Center of Industrial Spectrum Imaging, Beijing 100083, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,7,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Liu, G., Su, X., Hong, F., Zhong, X., Liang, Z., Wu, X., and Huang, Z. (2022). A Novel Epidemic Model Base on Pulse Charging in Wireless Rechargeable Sensor Networks. Entropy, 24.","DOI":"10.3390\/e24020302"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1109\/JSEN.2017.2766364","article-title":"Wireless Sensor\u2019s Civil Applications, Prototypes, and Future Integration Possibilities: A Review","volume":"18","author":"Ayaz","year":"2018","journal-title":"IEEE Sens. J."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1109\/COMST.2017.2759725","article-title":"A Critical Analysis of Research Potential, Challenges, and Future Directives in Industrial Wireless Sensor Networks","volume":"20","author":"Raza","year":"2018","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Liu, G., Peng, Z., Liang, Z., Li, J., and Cheng, L. (2021). Dynamics Analysis of a Wireless Rechargeable Sensor Network for Virus Mutation Spreading. Entropy, 23.","DOI":"10.3390\/e23050572"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Liu, G., Huang, Z., Wu, X., Liang, Z., Hong, F., and Su, X. (2021). Modelling and Analysis of the Epidemic Model under Pulse Charging in Wireless Rechargeable Sensor Networks. Entropy, 23.","DOI":"10.3390\/e23080927"},{"key":"ref_6","unstructured":"Liang, H., Yu, G., Pan, J., and Zhu, T. (2013, January 14\u201316). On-Demand Charging in Wireless Sensor Networks: Theories and Applications. Proceedings of the IEEE International Conference on Mobile Ad-Hoc & Sensor Systems, Hangzhou, China."},{"key":"ref_7","unstructured":"Wang, C., Yang, Y., and Li, J. (2013, January 7\u201310). Stochastic Mobile Energy Replenishment and Adaptive Sensor Activation for Perpetual Wireless Rechargeable Sensor Networks. Proceedings of the 2013 IEEE Wireless Communications and Networking Conference (WCNC), Shanghai, China."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Feng, Y., Liu, N., Wang, F., Qian, Q., and Li, X. (2016, January 22\u201327). Starvation Avoidance Mobile Energy Replenishment for Wireless Rechargeable Sensor Networks. Proceedings of the IEEE International Conference on Communications (ICC), Kuala Lumpur, Malaysia.","DOI":"10.1109\/ICC.2016.7510769"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"3161","DOI":"10.1109\/TNET.2017.2723605","article-title":"Approximation Algorithms for Charging Reward Maximization in Rechargeable Sensor Networks via a Mobile Charger","volume":"25","author":"Liang","year":"2017","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Peng, Y., Li, Z., Zhang, W., and Qiao, D. (December, January 30). Prolonging Sensor Network Lifetime through Wireless Charging. Proceedings of the 2010 31st IEEE Real-Time Systems Symposium, RTSS 2010, San Diego, CA, USA.","DOI":"10.1109\/RTSS.2010.35"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Li, Z., Peng, Y., Zhang, W., and Qiao, D. (2011, January 17\u201320). J-RoC: A Joint Routing and Charging Scheme to Prolong Sensor Network Lifetime. Proceedings of the 2011 19th IEEE International Conference on Network Protocols, Vancouver, BC, Canada.","DOI":"10.1109\/ICNP.2011.6089076"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Chen, F., Zhao, Z., Min, G., and Wu, Y. (2016, January 16\u201318). A Novel Approach for Path Plan of Mobile Chargers in Wireless Rechargeable Sensor Networks. Proceedings of the 2016 12th International Conference on Mobile Ad-Hoc and Sensor Networks (MSN), Hefei, China.","DOI":"10.1109\/MSN.2016.018"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ping, Z., Yiwen, Z., Shuaihua, M., Xiaoyan, K., and Jianliang, G. (2018). RCSS: A Real-Time on-Demand Charging Scheduling Scheme for Wireless Rechargeable Sensor Networks. Sensors, 18.","DOI":"10.3390\/s18051601"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1861","DOI":"10.1109\/TMC.2014.2368557","article-title":"Evaluating the on-Demand Mobile Charging in Wireless Sensor Networks","volume":"14","author":"He","year":"2015","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"8047","DOI":"10.1109\/TVT.2017.2683493","article-title":"P2S: A Primary and Passer-By Scheduling Algorithm for On-Demand Charging Architecture in Wireless Rechargeable Sensor Networks","volume":"66","author":"Lin","year":"2017","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1109\/TMC.2017.2703094","article-title":"TSCA: A Temporal-Spatial Real-Time Charging Scheduling Algorithm for on-Demand Architecture in Wireless Rechargeable Sensor Networks","volume":"17","author":"Chi","year":"2018","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"378","DOI":"10.1016\/j.ijleo.2018.12.055","article-title":"Low-Energy PSO-Based Node Positioning in Optical Wireless Sensor Networks","volume":"181","author":"Yan","year":"2018","journal-title":"Opt.-Int. J. Light Electron Opt."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1109\/TII.2016.2586028","article-title":"Joint Energy Replenishment and Operation Scheduling in Wireless Rechargeable Sensor Networks","volume":"13","author":"Shu","year":"2017","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"11014","DOI":"10.1109\/JSEN.2020.2994113","article-title":"A Newborn Particle Swarm Optimization Algorithm for Charging-Scheduling Algorithm in Industrial Rechargeable Sensor Networks","volume":"20","author":"Feng","year":"2020","journal-title":"IEEE Sens. J."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1316","DOI":"10.1007\/s12083-021-01090-w","article-title":"Correction to: Hybrid Meta-Heuristic Techniques Based Efficient Charging Scheduling Scheme for Multiple Mobile Wireless Chargers Based Wireless Rechargeable Sensor Networks","volume":"14","author":"Chawra","year":"2021","journal-title":"Peer-Peer Netw. Appl."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"654","DOI":"10.1109\/TC.2013.2297926","article-title":"Collaborative Mobile Charging","volume":"64","author":"Zhang","year":"2015","journal-title":"IEEE Trans. Comput."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2898357","article-title":"Maintaining Large-Scale Rechargeable Sensor Networks Perpetually via Multiple Mobile Charging Vehicles","volume":"12","author":"Liang","year":"2016","journal-title":"ACM Trans. Sens. Netw."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"550","DOI":"10.1007\/s11390-014-1449-2","article-title":"Collaborative Mobile Charging and Coverage","volume":"29","author":"Wu","year":"2014","journal-title":"J. Comp. Sci. Technol."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Madhja, A., Nikoletseas, S., and Raptis, T.P. (2015, January 9\u201312). Hierarchical, Collaborative Wireless Charging in Sensor Networks. Proceedings of the 2015 IEEE Wireless Communications and Networking Conference (WCNC), New Orleans, LA, USA.","DOI":"10.1109\/WCNC.2015.7127654"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"10131","DOI":"10.1109\/JSEN.2019.2928169","article-title":"Efficient Mobile Energy Replenishment Scheme Based on Hybrid Mode for Wireless Rechargeable Sensor Networks","volume":"19","author":"Feng","year":"2019","journal-title":"IEEE Sens. J."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1016\/j.jnca.2018.02.017","article-title":"An Efficient Scheduling Scheme for Mobile Charger in on-Demand Wireless Rechargeable Sensor Networks","volume":"114","author":"Kaswan","year":"2018","journal-title":"J. Netw. Comput. Appl."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"2715","DOI":"10.1109\/TMC.2020.2990419","article-title":"A Fuzzy Logic-Based On-Demand Charging Algorithm for Wireless Rechargeable Sensor Networks with Multiple Chargers","volume":"20","author":"Tomar","year":"2021","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"102278","DOI":"10.1016\/j.adhoc.2020.102278","article-title":"A Deep Reinforcement Learning-Based on-Demand Charging Algorithm for Wireless Rechargeable Sensor Networks","volume":"110","author":"Cao","year":"2021","journal-title":"Ad Hoc Netw."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Chellappan, S., Cheng, W., and Li, W. (2018). Reinforcement Learning for a Novel Mobile Charging Strategy in Wireless Rechargeable Sensor Networks. Wireless Algorithms, Systems, and Applications, Springer International Publishing. Lecture Notes in Computer Science.","DOI":"10.1007\/978-3-319-94268-1"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1052","DOI":"10.1007\/s42452-019-1091-2","article-title":"Novel Wireless Charging Algorithms to Charge Mobile Wireless Sensor Network by Using Reinforcement Learning","volume":"1","author":"Soni","year":"2019","journal-title":"SN Appl. Sci."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"370","DOI":"10.1109\/JIOT.2020.3005598","article-title":"Dynamic Charging Scheme Problem with Actor-Critic Reinforcement Learning","volume":"8","author":"Yang","year":"2021","journal-title":"IEEE Internet Things J."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1748","DOI":"10.1109\/TNET.2012.2185831","article-title":"Making Sensor Networks Immortal: An Energy-Renewal Approach with Wireless energy transmission","volume":"20","author":"Xie","year":"2012","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1109\/TNET.2007.900407","article-title":"Rate Allocation and Network Lifetime Problems for Wireless Sensor Networks","volume":"16","author":"Hou","year":"2008","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1699","DOI":"10.1109\/TMC.2015.2473163","article-title":"Near-Optimal Velocity Control for Mobile Charging in Wireless Rechargeable Sensor Networks","volume":"15","author":"Shu","year":"2016","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_35","unstructured":"Cohen, W.W., and Hirsh, H. (1994). Markov Games as a Framework for Multi-Agent Reinforcement Learning. Machine Learning Proceeding 1994, Morgan Kaufmann."},{"key":"ref_36","unstructured":"Lowe, R., Wu, Y., Tamar, A., and Harb, J. (2017, June 07). Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments. Available online: https:\/\/doi.org\/10.48550\/arXiv.1706.02275."},{"key":"ref_37","unstructured":"Yu, C., Velu, A., Vinitsky, E., Wang, Y., and Wu, Y. (2021, March 02). The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games. Available online: https:\/\/arxiv.org\/abs\/2103.01955."},{"key":"ref_38","unstructured":"Graves, A., Wayne, G., and Danihelka, I. (2014, October 20). Neural Turing Machines. Available online: https:\/\/arxiv.org\/abs\/1410.5401v1."},{"key":"ref_39","unstructured":"Oh, J., Chockalingam, V., Singh, S., and Lee, H. (2016, May 30). Control of Memory, Active Perception, and Action in Minecraft. Available online: https:\/\/arxiv.org\/abs\/1605.09128."},{"key":"ref_40","unstructured":"Foerster, J., Farquhar, G., Afouras, T., Nardelli, N., and Whiteson, S. (2017, May 24). Counterfactual Multi-Agent Policy Gradients. Available online: https:\/\/arxiv.org\/abs\/1705.08926."},{"key":"ref_41","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017, June 12). Attention Is All You Need. Available online: https:\/\/arxiv.org\/abs\/1706.03762."},{"key":"ref_42","unstructured":"Iqbal, S., and Sha, F. (2018, October 05). Actor-Attention-Critic for Multi-Agent Reinforcement Learning. Available online: https:\/\/arxiv.org\/abs\/1810.02912."},{"key":"ref_43","unstructured":"Wei, E., Wicke, D., Freelan, D., and Luke, S. (2018, April 25). Multiagent Soft Q-Learning. Available online: https:\/\/arxiv.org\/abs\/1804.09817."},{"key":"ref_44","unstructured":"Kingma, D.P., and Ba, J. (2014, December 22). Adam: A Method for Stochastic Optimization. Available online: https:\/\/arxiv.org\/abs\/1412.6980."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/7\/965\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:48:54Z","timestamp":1760140134000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/7\/965"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,12]]},"references-count":44,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2022,7]]}},"alternative-id":["e24070965"],"URL":"https:\/\/doi.org\/10.3390\/e24070965","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,12]]}}}