{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T01:25:47Z","timestamp":1760232347672,"version":"build-2065373602"},"reference-count":19,"publisher":"MDPI AG","issue":"21","license":[{"start":{"date-parts":[[2022,10,25]],"date-time":"2022-10-25T00:00:00Z","timestamp":1666656000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Science Foundation of China","award":["U19B2014"],"award-info":[{"award-number":["U19B2014"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In this paper, in order to solve the problem of wireless sensor networks\u2019 reliable transmission in intelligent malicious jamming, we propose a Distributed Anti-Jamming Algorithm (DAJA) based on an actor\u2013critic algorithm for a multi-agent system. The Multi-Agent Markov Decision Process (MAMPD) is introduced to model the progress of wireless sensor networks\u2019 anti-jamming communication, and the multi-agent system learns the intelligent jamming from the external environment by using an actor\u2013critic algorithm. On the basis of coping with the influence of external and internal factors effectively, each sensor in networks selects the appropriate channels for transmission and finally realizes the optimal transmission of the system overall in a unit time period. In the environment of probabilistic intelligent jamming with tracking properties set in this paper, the simulation shows that the algorithm proposed can outperform the algorithm based on joint Q-learning and the conventional scheme based on orthogonal frequency hopping in terms of transmission. In addition, the proposed algorithm completes two updates of strategy evaluation and action selection in one iteration, which means that the system has higher efficiency of action selection and better adaptability to the environment through the interaction with the external environment, resulting in the better performance of transmission and convergence.<\/jats:p>","DOI":"10.3390\/s22218159","type":"journal-article","created":{"date-parts":[[2022,10,26]],"date-time":"2022-10-26T07:17:48Z","timestamp":1666768668000},"page":"8159","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["A Distributed Anti-Jamming Algorithm Based on Actor\u2013Critic Countering Intelligent Malicious Jamming for WSN"],"prefix":"10.3390","volume":"22","author":[{"given":"Yuheng","family":"Chen","sequence":"first","affiliation":[{"name":"Fundamentals Department, Air Force Engineering University of People\u2019s Liberation Army, Xi\u2019an 710051, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6366-3774","authenticated-orcid":false,"given":"Yingtao","family":"Niu","sequence":"additional","affiliation":[{"name":"The Sixty-Third Research Institute, National University of Defense Technology, Nanjing 210007, China"}]},{"given":"Changxing","family":"Chen","sequence":"additional","affiliation":[{"name":"Fundamentals Department, Air Force Engineering University of People\u2019s Liberation Army, Xi\u2019an 710051, China"}]},{"given":"Quan","family":"Zhou","sequence":"additional","affiliation":[{"name":"College of Communications Engineering, Army Engineering University of People\u2019s Liberation Army, Nanjing 210007, China"}]},{"given":"Peng","family":"Xiang","sequence":"additional","affiliation":[{"name":"College of Communications Engineering, Army Engineering University of People\u2019s Liberation Army, Nanjing 210007, China"}]}],"member":"1968","published-online":{"date-parts":[[2022,10,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1504\/IJAHUC.2014.066419","article-title":"Jamming and anti-jamming techniques in wireless networks: A survey","volume":"17","author":"Grover","year":"2014","journal-title":"Int. J. Ad Hoc Ubiquitous Comput."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1024","DOI":"10.1109\/LWC.2019.2904486","article-title":"A Collaborative Multi-Agent Reinforcement Learning Anti-Jamming Algorithm in Wireless Networks","volume":"8","author":"Yao","year":"2019","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Yuan, L., and Ishii, H. (2021, January 14\u201317). Resilient Consensus with Multi-hop Communication. Proceedings of the 2021 60th IEEE Conference on Decision and Control (CDC), Austin, TX, USA.","DOI":"10.1109\/CDC45484.2021.9683181"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2834","DOI":"10.1109\/LCOMM.2021.3096388","article-title":"Enhancing the Security of Chaotic Direct Sequence Spread Spectrum Communication Through WFRFT","volume":"25","author":"Yuan","year":"2021","journal-title":"IEEE Commun. Lett."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Shalev-Shwartz, S. (2014). Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press.","DOI":"10.1017\/CBO9781107298019"},{"key":"ref_6","first-page":"639","article-title":"On Using Multi Agent Systems in Cognitive Radio Networks: A Survey","volume":"4","author":"Trigui","year":"2013","journal-title":"Int. J. Wirel. Mob. Netw."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Chen, Y., Niu, Y., Chen, C., and Zhou, Q. (2022). Conservative but Stable: A SARSA-Based Algorithm for Random Pulse Jamming in the Time Domain. Electronics, 11.","DOI":"10.3390\/electronics11091456"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1109\/TSMCC.2007.913919","article-title":"A Comprehensive Survey of Multiagent Reinforcement Learning","volume":"38","author":"Busoniu","year":"2008","journal-title":"IEEE Trans. Syst. Man Cybern. Part C"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Slimeni, F., Scheers, B., Chtourou, Z., and le Nir, V. (2015, January 18\u201319). Jamming mitigation in cognitive radio networks using a modified Q-learning algorithm. Proceedings of the 2015 International Conference on Military Communications and Information Systems (ICMCIS), Cracow, Poland.","DOI":"10.1109\/ICMCIS.2015.7158697"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"97164","DOI":"10.1109\/ACCESS.2020.2996804","article-title":"A Countermeasure Against Random Pulse Jamming in Time Domain Based on Reinforcement Learning","volume":"8","author":"Zhou","year":"2020","journal-title":"IEEE Access"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Aref, M.A., Jayaweera, S.K., and Machuzak, S. (2017, January 19\u201322). Multi-Agent Reinforcement Learning Based Cognitive Anti-Jamming. Proceedings of the 2017 IEEE Wireless Communications and Networking Conference (WCNC), San Francisco, CA, USA.","DOI":"10.1109\/WCNC.2017.7925694"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zhou, Q., Li, Y., Niu, Y., Qin, Z., Zhao, L., and Wang, J. (2020, January 11\u201314). \u201cOne Plus One is Greater Than Two\u201d: Defeating Intelligent Dynamic Jamming with Collaborative Multi-agent Reinforcement Learning. Proceedings of the 2020 IEEE 6th International Conference on Computer and Communications (ICCC), Chengdu, China.","DOI":"10.1109\/ICCC51575.2020.9345127"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Zhang, G., Li, Y., Niu, Y., and Zhou, Q. (2022). Anti-Jamming Path Selection Method in a Wireless Communication Network Based on Dyna-Q. Electronics, 11.","DOI":"10.3390\/electronics11152397"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Shan, Q., Xiong, J., Ma, D., Li, J., and Hu, T. (2018, January 12\u201314). Distributed Multi-agent Q-learning for Anti-dynamic Jamming and Collision-avoidance Spectrum Access in Cognitive Radio System. Proceedings of the 2018 24th Asia-Pacific Conference on Communications (APCC), Ningbo, China.","DOI":"10.1109\/APCC.2018.8633463"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zhang, K., Yang, Z., and Baar, T. (2021). Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms, Springer.","DOI":"10.1007\/978-3-030-60990-0_12"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1717","DOI":"10.1109\/COMST.2016.2539923","article-title":"A Survey on Applications of Model-Free Strategy Learning in Cognitive Wireless Networks","volume":"18","author":"Wang","year":"2016","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_17","unstructured":"Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., and Mordatch, I. (2017). Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"409","DOI":"10.1007\/s42979-021-00796-1","article-title":"Security Threat Analysis and Countermeasures on Consensus-Based Time Synchronization Algorithms for Wireless Sensor Network","volume":"2","author":"Kumar","year":"2021","journal-title":"SN Comput. Sci."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"2395","DOI":"10.1007\/s11277-021-09000-2","article-title":"Architectural Design, Improvement, and Challenges of Distributed Software-Defined Wireless Sensor Networks","volume":"122","author":"Bukar","year":"2021","journal-title":"Wirel. Pers. Commun."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/21\/8159\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:02:13Z","timestamp":1760144533000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/21\/8159"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,25]]},"references-count":19,"journal-issue":{"issue":"21","published-online":{"date-parts":[[2022,11]]}},"alternative-id":["s22218159"],"URL":"https:\/\/doi.org\/10.3390\/s22218159","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2022,10,25]]}}}