{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,16]],"date-time":"2026-05-16T17:53:52Z","timestamp":1778954032056,"version":"3.51.4"},"reference-count":28,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2024,2,19]],"date-time":"2024-02-19T00:00:00Z","timestamp":1708300800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Natural Science Foundation of China","award":["62171455"],"award-info":[{"award-number":["62171455"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Optimizing jamming strategies is crucial for enhancing the performance of cognitive jamming systems in dynamic electromagnetic environments. The emergence of frequency-agile radars, capable of changing the carrier frequency within or between pulses, poses significant challenges for the jammer to make intelligent decisions and adapt to the dynamic environment. This paper focuses on researching intelligent jamming decision-making algorithms for Intra-Pulse Frequency Agile Radar using deep reinforcement learning. Intra-Pulse Frequency Agile Radar achieves frequency agility at the sub-pulse level, creating a significant frequency agility space. This presents challenges for traditional jamming decision-making methods to rapidly learn its changing patterns through interactions. By employing Gated Recurrent Units (GRU) to capture long-term dependencies in sequence data, together with the attention mechanism, this paper proposes a GA-Dueling DQN (GRU-Attention-based Dueling Deep Q Network) method for jamming frequency selection. Simulation results indicate that the proposed method outperforms traditional Q-learning, DQN, and Dueling DQN methods in terms of jamming effectiveness. It exhibits the fastest convergence speed and reduced reliance on prior knowledge, highlighting its significant advantages in jamming the subpulse-level frequency-agile radar.<\/jats:p>","DOI":"10.3390\/s24041325","type":"journal-article","created":{"date-parts":[[2024,2,19]],"date-time":"2024-02-19T03:18:38Z","timestamp":1708312718000},"page":"1325","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["GA-Dueling DQN Jamming Decision-Making Method for Intra-Pulse Frequency Agile Radar"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-0496-1702","authenticated-orcid":false,"given":"Liqun","family":"Xia","sequence":"first","affiliation":[{"name":"National Innovation Institute of Defense Technology, Academy of Military Science, Beijing 100010, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lulu","family":"Wang","sequence":"additional","affiliation":[{"name":"Intelligent Game and Decision Laboratory, Academy of Military Science, Beijing 100091, China"},{"name":"Chinese People\u2019s Liberation Army 32806 Unit, Academy of Military Science, Beijing 100091, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhidong","family":"Xie","sequence":"additional","affiliation":[{"name":"Intelligent Game and Decision Laboratory, Academy of Military Science, Beijing 100091, China"},{"name":"Chinese People\u2019s Liberation Army 32806 Unit, Academy of Military Science, Beijing 100091, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xin","family":"Gao","sequence":"additional","affiliation":[{"name":"The 85th Detachment, Chinese People\u2019s Liberation Army 95969 Unit, Wuhan 430000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,2,19]]},"reference":[{"key":"ref_1","unstructured":"Haigh, K., and Andrusenko, J. (2021). Cognitive Electronic Warfare: An Artificial Intelligence Approach, Artech House."},{"key":"ref_2","unstructured":"Ruixue, Z., Guifen, X., Yue, Z., and Hengze, L. (2015, January 16\u201318). Coherent signal processing method for frequency-agile radar. Proceedings of the 2015 12th IEEE International Conference on Electronic Measurement & Instruments (ICEMI), Qingdao, China."},{"key":"ref_3","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"619","DOI":"10.1049\/cmu2.12339","article-title":"Joint channel and power optimisation for multi-user anti-jamming communications: A dual mode Q-learning approach","volume":"16","author":"Zhang","year":"2022","journal-title":"IET Commun."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"983","DOI":"10.1109\/TIFS.2020.3027145","article-title":"Jamming resilient tracking using POMDP-based detection of hidden targets","volume":"16","author":"Jiang","year":"2020","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1109\/MWC.001.1900207","article-title":"UAV-aided cellular communications with deep reinforcement learning against jamming","volume":"27","author":"Lu","year":"2020","journal-title":"IEEE Wirel. Commun."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1963","DOI":"10.1109\/TWC.2020.3037767","article-title":"Intelligent reflecting surface assisted anti-jamming communications: A fast reinforcement learning approach","volume":"20","author":"Yang","year":"2020","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Zheng, Z., Li, W., and Zou, K. (2022). Airborne Radar Anti-Jamming Waveform Design Based on Deep Reinforcement Learning. Sensors, 22.","DOI":"10.3390\/s22228689"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1024","DOI":"10.1109\/LWC.2019.2904486","article-title":"A collaborative multi-agent reinforcement learning anti-jamming algorithm in wireless networks","volume":"8","author":"Yao","year":"2019","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_10","first-page":"1031","article-title":"Intelligent radar countermeasure based on Q-learning","volume":"40","author":"Qiang","year":"2018","journal-title":"Syst. Eng. Electron."},{"key":"ref_11","first-page":"819","article-title":"DQN based decision-making method of cognitive jamming against multifunctional radar","volume":"42","author":"Bokai","year":"2020","journal-title":"Syst. Eng. Electron."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zhang, B., and Zhu, W. (2019, January 20\u201322). Research on decision-making system of cognitive jamming against multifunctional radar. Proceedings of the 2019 IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC), Dalian, China.","DOI":"10.1109\/ICSPCC46631.2019.8960757"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Yan, P., Chu, F., and Jia, L. (2021, January 13\u201315). A novel collaborative jamming method based on decoy strategy. Proceedings of the 2021 International Conference on Wireless Communications and Smart Grid (ICWCSG), Hangzhou, China.","DOI":"10.1109\/ICWCSG53609.2021.00044"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"108965","DOI":"10.1016\/j.sigpro.2023.108965","article-title":"Performance analysis of deep reinforcement learning-based intelligent cooperative jamming method confronting multi-functional networked radar","volume":"207","author":"Zhang","year":"2023","journal-title":"Signal Process."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Wang, L., Peng, J., Xie, Z., and Zhang, Y. (2019, January 28\u201330). Optimal jamming frequency selection for cognitive jammer based on reinforcement learning. Proceedings of the 2019 IEEE 2nd International Conference on Information Communication and Signal Processing (ICICSP), Weihai, China.","DOI":"10.1109\/ICICSP48821.2019.8958575"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Liu, H., Zhang, H., He, Y., and Sun, Y. (2021). Jamming strategy optimization through dual Q-learning model against adaptive radar. Sensors, 22.","DOI":"10.3390\/s22010145"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"4651","DOI":"10.1109\/TAES.2023.3272307","article-title":"Joint Optimization of Jamming Type Selection and Power Control for Countering Multi-function Radar Based on Deep Reinforcement Learning","volume":"59","author":"Pan","year":"2023","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"548","DOI":"10.1109\/LSP.2023.3270762","article-title":"DRL-Based Joint Path Planning and Jamming Power Allocation Optimization for Suppressing Netted Radar System","volume":"30","author":"Li","year":"2023","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Li, H., Han, Z., Pu, W., Liu, L., Li, K., and Jiu, B. (2022, January 20\u201323). Counterfactual Regret Minimization for Anti-Jamming Game of Frequency Agile Radar. Proceedings of the 2022 IEEE 12th Sensor Array and Multichannel Signal Processing Workshop (SAM), Trondheim, Norway.","DOI":"10.1109\/SAM53842.2022.9827883"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2792","DOI":"10.1109\/TWC.2015.2510643","article-title":"Jamming bandits A novel learning method for optimal jamming","volume":"15","author":"Amuru","year":"2015","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"412","DOI":"10.1109\/TWC.2016.2624291","article-title":"On jamming against wireless networks","volume":"16","author":"Amuru","year":"2016","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1186\/s13638-019-1414-4","article-title":"An algorithm for jamming strategy using OMP and MAB","volume":"2019","author":"ZhuanSun","year":"2019","journal-title":"EURASIP J. Wirel. Commun. Netw."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"4389","DOI":"10.1109\/TSP.2016.2566610","article-title":"Generalized multicarrier radar: Models and performance","volume":"64","author":"Koivunen","year":"2016","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_24","unstructured":"Wang, Z., Schaul, T., Hessel, M., Hasselt, H., Lanctot, M., and Freitas, N. (2016, January 20\u201322). Dueling network architectures for deep reinforcement learning. Proceedings of the International Conference on Machine Learning, New York, NY, USA."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Van Hasselt, H., Guez, A., and Silver, D. (2016, January 12). Deep reinforcement learning with double q-learning. Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Cho, K., Van Merri\u00ebnboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv.","DOI":"10.3115\/v1\/D14-1179"},{"key":"ref_27","first-page":"5998","article-title":"Attention is all you need","volume":"30","author":"Vaswani","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_28","unstructured":"Fortunato, M., Azar, M.G., Piot, B., Menick, J., Osband, I., Graves, A., Mnih, V., Munos, R., Hassabis, D., and Pietquin, O. (2017). Noisy networks for exploration. arXiv."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/4\/1325\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:01:57Z","timestamp":1760104917000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/4\/1325"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,19]]},"references-count":28,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2024,2]]}},"alternative-id":["s24041325"],"URL":"https:\/\/doi.org\/10.3390\/s24041325","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,19]]}}}