{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T14:31:37Z","timestamp":1774621897195,"version":"3.50.1"},"reference-count":26,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2021,12,26]],"date-time":"2021-12-26T00:00:00Z","timestamp":1640476800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Modern adaptive radars can switch work modes to perform various missions and simultaneously use pulse parameter agility in each mode to improve survivability, which leads to a multiplicative increase in the decision-making complexity and declining performance of the existing jamming methods. In this paper, a two-level jamming decision-making framework is developed, based on which a dual Q-learning (DQL) model is proposed to optimize the jamming strategy and a dynamic method for jamming effectiveness evaluation is designed to update the model. Specifically, the jamming procedure is modeled as a finite Markov decision process. On this basis, the high-dimensional jamming action space is disassembled into two low-dimensional subspaces containing jamming mode and pulse parameters respectively, then two specialized Q-learning models with interaction are built to obtain the optimal solution. Moreover, the jamming effectiveness is evaluated through indicator vector distance measuring to acquire the feedback for the DQL model, where indicators are dynamically weighted to adapt to the environment. The experiments demonstrate the advantage of the proposed method in learning radar joint strategy of mode switching and parameter agility, shown as improving the average jamming-to-signal radio (JSR) by 4.05% while reducing the convergence time by 34.94% compared with the normal Q-learning method.<\/jats:p>","DOI":"10.3390\/s22010145","type":"journal-article","created":{"date-parts":[[2021,12,27]],"date-time":"2021-12-27T01:06:54Z","timestamp":1640567214000},"page":"145","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":35,"title":["Jamming Strategy Optimization through Dual Q-Learning Model against Adaptive Radar"],"prefix":"10.3390","volume":"22","author":[{"given":"Hongdi","family":"Liu","sequence":"first","affiliation":[{"name":"Key Lab of Universal Wireless Communications, Ministry of Education of China, Beijing University of Posts and Telecommunications, Beijing 100876, China"}],"role":[{"role":"author","vocab":"crossref"}]},{"given":"Hongtao","family":"Zhang","sequence":"additional","affiliation":[{"name":"Key Lab of Universal Wireless Communications, Ministry of Education of China, Beijing University of Posts and Telecommunications, Beijing 100876, China"}],"role":[{"role":"author","vocab":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7578-8515","authenticated-orcid":false,"given":"Yuan","family":"He","sequence":"additional","affiliation":[{"name":"Key Lab of Universal Wireless Communications, Ministry of Education of China, Beijing University of Posts and Telecommunications, Beijing 100876, China"}],"role":[{"role":"author","vocab":"crossref"}]},{"given":"Yong","family":"Sun","sequence":"additional","affiliation":[{"name":"Key Lab of Universal Wireless Communications, Ministry of Education of China, Beijing University of Posts and Telecommunications, Beijing 100876, China"}],"role":[{"role":"author","vocab":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,12,26]]},"reference":[{"key":"ref_1","first-page":"7449","article-title":"Jamming decision under condition of incomplete jamming rule library","volume":"2019","author":"Xing","year":"2019","journal-title":"J. Eng."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1109\/MSP.2006.1593335","article-title":"Cognitive Radar: A Way of the Future","volume":"23","author":"Haykin","year":"2006","journal-title":"IEEE Signal Procesing Mag."},{"key":"ref_3","first-page":"7958","article-title":"Performance analysis of one-step prediction-based cognitive jamming in jammer-radar countermeasure model","volume":"2019","author":"Gao","year":"2019","journal-title":"J. Eng."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Zhang, B., and Zhu, W. (2019, January 20\u201322). Research on Decision-making System of Cognitive Jamming against Multifunctional Radar. Proceedings of the 2019 IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC), Dalian, China.","DOI":"10.1109\/ICSPCC46631.2019.8960757"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"702","DOI":"10.1016\/S1004-4132(08)60142-5","article-title":"Research on the self-defence electronic jamming decision-making based on the discrete dynamic Bayesian network","volume":"19","author":"Zheng","year":"2008","journal-title":"J. Syst. Eng. Electron."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Pan, W., Jin, X., Xie, H., and Xia, Y. (2020, January 22\u201324). Radar Jamming Strategy Allocation Algorithm based on Improved Chaos Genetic Algorithm. Proceedings of the 2020 Chinese Control And Decision Conference (CCDC), Hefei, China.","DOI":"10.1109\/CCDC49329.2020.9164855"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Wang, L., Zeng, Y., Li, Y., and Wang, M. (2016, January 10\u201313). An Optimal Jamming Strategy Aiming at Cognitive MIMO Radar. Proceedings of the 2016 CIE International Conference on Radar (RADAR), Guangzhou, China.","DOI":"10.1109\/RADAR.2016.8059297"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Slimeni, F., Scheers, B., Chtourou, Z., and Nir, V.L. (2015, January 18\u201319). Jamming mitigation in cognitive radio networks using a modified Q-learning algorithm. Proceedings of the 2015 International Conference on Military Communications and Information Systems (ICMCIS), Cracow, Poland.","DOI":"10.1109\/ICMCIS.2015.7158697"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Machuzak, S., and Jayaweera, S.K. (2016, January 27\u201329). Reinforcement Learning Based Anti-jamming with Wideband Autonomous Cognitive Radios. Proceedings of the 2016 IEEE\/CIC International Conference on Communications in China (ICCC), Chengdu, China.","DOI":"10.1109\/ICCChina.2016.7636793"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"180532","DOI":"10.1109\/ACCESS.2019.2958328","article-title":"Anti-Jamming Communications in UAV Swarms A Reinforcement Learning Approach","volume":"7","author":"Peng","year":"2019","journal-title":"IEEE Access."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1109\/MWC.001.1900207","article-title":"UAV-aided cellular communications with deep reinforcement learning against jamming","volume":"27","author":"Lu","year":"2020","journal-title":"IEEE Wirel. Commun."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1024","DOI":"10.1109\/LWC.2019.2904486","article-title":"A collaborative multi-agent reinforcement learning anti-jamming algorithm in wireless networks","volume":"8","author":"Yao","year":"2019","journal-title":"Wirel. Commun. Lett."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Sutton, R.S., and Barto, A.G. (1998). Reinforcement Learning: An Introduction, MIT Press. [1st ed.].","DOI":"10.1109\/TNN.1998.712192"},{"key":"ref_14","unstructured":"Xing, Q., Zhu, W., and Jia, X. (2017, January 8\u201311). Research on method of intelligent radar confrontation based on reinforcement learning. Proceedings of the 2017 2nd IEEE International Conference on Computational Intelligence and Applications (ICCIA), Beijing, China."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Wang, L., Peng, J., Xie, Z., and Zhang, Y. (2019, January 28\u201330). Optimal jamming frequency selection for cognitive jammer based on reinforcement learning. Proceedings of the 2019 IEEE 2nd International Conference on Information Communication and Signal Processing (ICICSP), Weihai, China.","DOI":"10.1109\/ICICSP48821.2019.8958575"},{"key":"ref_16","unstructured":"Li, K., Jiu, B., Liu, H., and Liang, S. (2018, January 14\u201316). Reinforcement learning based anti-jamming frequency hopping strategies design for cognitive radar. Proceedings of the 2018 IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC), Qingdao, China."},{"key":"ref_17","unstructured":"Lei, M., and Zhang, J. (2016, January 14\u201317). Study on anti-jamming frequency selection in radar netting. Proceedings of the 2016 2nd IEEE International Conference on Computer and Communications (ICCC), Chengdu, China."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Ak, S., and Br\u00fcggenwirth, S. (2020, January 21\u201325). Avoiding Jammers: A Reinforcement Learning Approach. Proceedings of the 2020 IEEE International Radar Conference (RADAR), Florence, Italy.","DOI":"10.1109\/RADAR42522.2020.9114797"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Li, K., Jiu, B., Liu, H., and Pu, W. (2021). Robust antijamming strategy design for frequency-agile radar against main lobe jamming. Remote Sens., 13.","DOI":"10.3390\/rs13153043"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"348","DOI":"10.1049\/iet-rsn.2017.0421","article-title":"Range-Doppler reconstruction for frequency agile and PRF-jittering radar","volume":"12","author":"Quan","year":"2018","journal-title":"IET Radar Sonar Navig."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Ou, J., Zhao, F., Ai, X., Liu, J., and Xiao, S. (2016, January 10\u201313). Quantitative evaluation for self-screening jamming effectiveness based on the changing characteristics of intercepted radar signals. Proceedings of the 2016 CIE International Conference on Radar (RADAR), Guangzhou, China.","DOI":"10.1109\/RADAR.2016.8059366"},{"key":"ref_22","first-page":"46","article-title":"Jamming effectiveness evaluation from the jamming side","volume":"23","author":"Li","year":"2008","journal-title":"Electron. Inf. Warf. Technol."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Peng, X., Yu, J., Ren, W., and Weng, X. (2020, January 4\u20136). Radar jamming effectiveness evaluation method based on feature space weighting. Proceedings of the IET International Radar Conference (IET IRC 2020), Chongqing, China.","DOI":"10.1049\/icp.2021.0697"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1109\/5.18626","article-title":"A tutorial on hidden Markov models and selected applications in speech recognition","volume":"77","author":"Rabiner","year":"1989","journal-title":"Proc. IEEE"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"459","DOI":"10.1049\/iet-rsn.2016.0277","article-title":"Threat evaluation and jamming allocation","volume":"11","author":"Osner","year":"2017","journal-title":"IET Radar Sonar Navig."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1039","DOI":"10.1049\/iet-rsn.2019.0433","article-title":"Ground threat evaluation and jamming allocation model with Markov chain for aircraft","volume":"14","author":"Han","year":"2020","journal-title":"IET Radar Sonar Navig."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/1\/145\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:53:46Z","timestamp":1760169226000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/1\/145"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,26]]},"references-count":26,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,1]]}},"alternative-id":["s22010145"],"URL":"https:\/\/doi.org\/10.3390\/s22010145","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,12,26]]}}}