{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,17]],"date-time":"2026-03-17T23:55:03Z","timestamp":1773791703538,"version":"3.50.1"},"reference-count":24,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2024,3,28]],"date-time":"2024-03-28T00:00:00Z","timestamp":1711584000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key R&amp;D Program of China","doi-asserted-by":"publisher","award":["2021YFA0716600"],"award-info":[{"award-number":["2021YFA0716600"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key R&amp;D Program of China","doi-asserted-by":"publisher","award":["JCYJ20180307151430655"],"award-info":[{"award-number":["JCYJ20180307151430655"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key R&amp;D Program of China","doi-asserted-by":"publisher","award":["KQTD20190929172704911"],"award-info":[{"award-number":["KQTD20190929172704911"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100017607","name":"Shenzhen Fundamental Research Program","doi-asserted-by":"publisher","award":["2021YFA0716600"],"award-info":[{"award-number":["2021YFA0716600"]}],"id":[{"id":"10.13039\/501100017607","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100017607","name":"Shenzhen Fundamental Research Program","doi-asserted-by":"publisher","award":["JCYJ20180307151430655"],"award-info":[{"award-number":["JCYJ20180307151430655"]}],"id":[{"id":"10.13039\/501100017607","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100017607","name":"Shenzhen Fundamental Research Program","doi-asserted-by":"publisher","award":["KQTD20190929172704911"],"award-info":[{"award-number":["KQTD20190929172704911"]}],"id":[{"id":"10.13039\/501100017607","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Shenzhen Science and Technology Program","award":["2021YFA0716600"],"award-info":[{"award-number":["2021YFA0716600"]}]},{"name":"Shenzhen Science and Technology Program","award":["JCYJ20180307151430655"],"award-info":[{"award-number":["JCYJ20180307151430655"]}]},{"name":"Shenzhen Science and Technology Program","award":["KQTD20190929172704911"],"award-info":[{"award-number":["KQTD20190929172704911"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>In contemporary warfare, radar countermeasures have become multifunctional and intelligent, rendering the conventional jamming method and platform unsuitable for the modern radar countermeasures battlefield due to their limited efficiency. Reinforcement learning has been proven to be a practical solution for cognitive jamming decision-making in the cognitive electronic warfare. In this paper, we proposed a radar-jamming decision-making algorithm based on an improved Q-Learning algorithm. This improved Q-Learning algorithm ameliorated the problem of overestimating the Q-value that exists in the Q-Learning algorithm by introducing a second Q-table. At the same time, we performed a comprehensive design and implementation based on the classical Q-Learning algorithm, deploying it to a Field Programmable Gate Array (FPGA) hardware. We decomposed the implementation of the reinforcement learning algorithm into individual steps and described each step using a hardware description language. Then, the reinforcement learning algorithm can be computed on FPGA by linking the logic modules with valid signals. Experiments show that the proposed Q-Learning algorithm obtains considerable improvement in performance over the classical Q-Learning algorithm. Additionally, they confirm that the FPGA hardware can achieve great efficiency improvement on the radar-jamming decision-making algorithm implementation.<\/jats:p>","DOI":"10.3390\/rs16071190","type":"journal-article","created":{"date-parts":[[2024,3,28]],"date-time":"2024-03-28T12:09:40Z","timestamp":1711627780000},"page":"1190","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Radar-Jamming Decision-Making Based on Improved Q-Learning and FPGA Hardware Implementation"],"prefix":"10.3390","volume":"16","author":[{"given":"Shujian","family":"Zheng","sequence":"first","affiliation":[{"name":"School of Electronic and Communication Engineering, Sun Yat-sen University, Shenzhen 518107, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0839-3908","authenticated-orcid":false,"given":"Chudi","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Electronic and Communication Engineering, Sun Yat-sen University, Shenzhen 518107, China"}]},{"given":"Jun","family":"Hu","sequence":"additional","affiliation":[{"name":"School of Electronic and Communication Engineering, Sun Yat-sen University, Shenzhen 518107, China"}]},{"given":"Shiyou","family":"Xu","sequence":"additional","affiliation":[{"name":"School of Electronic and Communication Engineering, Sun Yat-sen University, Shenzhen 518107, China"}]}],"member":"1968","published-online":{"date-parts":[[2024,3,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1109\/MSP.2006.1593335","article-title":"Cognitive radar: A way of the future","volume":"23","author":"Haykin","year":"2006","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_2","first-page":"V3-153","article-title":"A radar emitter identification method based on pulse match template sequence","volume":"3","author":"Gong","year":"2008","journal-title":"Mod. Def. Technol."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Hao, H., Zeng, D., and Ge, P. (2015, January 18\u201320). Research on the Method of Smart Noise Jamming on Pulse Radar. Proceedings of the 2015 Fifth International Conference on Instrumentation and Measurement, Computer, Communication and Control (IMCCC), Qinhuangdao, China.","DOI":"10.1109\/IMCCC.2015.287"},{"key":"ref_4","unstructured":"Morris, G., and Kastle, T. (1991, January 26\u201327). Trends in electronic counter-countermeasures. Proceedings of the NTC \u201991\u2014National Telesystems Conference Proceedings, Atlanta, GA, USA."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1054","DOI":"10.1109\/TNN.1998.712192","article-title":"Reinforcement Learning: An Introduction","volume":"9","author":"Sutton","year":"1998","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"11383","DOI":"10.1109\/JSEN.2023.3267068","article-title":"Radar Jamming Decision-Making in Cognitive Electronic Warfare: A Review","volume":"23","author":"Zhang","year":"2023","journal-title":"IEEE Sens. J."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Liu, H., Zhang, H., He, Y., and Sun, Y. (2022). Jamming Strategy Optimization through Dual Q-Learning Model against Adaptive Radar. Sensors, 22.","DOI":"10.3390\/s22010145"},{"key":"ref_8","first-page":"83","article-title":"Research on Self-Learning Model of Intelligent Radar Jamming System","volume":"37","author":"Li","year":"2009","journal-title":"Mod. Def. Technol."},{"key":"ref_9","unstructured":"Xing, Q., Zhu, W.-G., and Jia, X. (2017, January 22\u201325). Intelligent countermeasure design of radar working-modes unknown. Proceedings of the 2017 IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC), Xiamen, China."},{"key":"ref_10","unstructured":"Mao, S. (2021). Research on Intelligent Jamming Decision-Making Methods Based on Reinforcement Learning. [Master\u2019s Thesis, Harbin Institute of Technology]."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"7958","DOI":"10.1049\/joe.2019.0916","article-title":"Performance analysis of one-step prediction-based cognitive jamming in jammer-radar countermeasure model","volume":"2019","author":"Gao","year":"2019","journal-title":"J. Eng."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zhang, B., and Zhu, W. (2019, January 20\u201323). Research on Decision-making System of Cognitive Jamming against Multifunctional Radar. Proceedings of the 2019 IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC), Dalian, China.","DOI":"10.1109\/ICSPCC46631.2019.8960757"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Zhang, C., Song, Y., Jiang, R., Hu, J., and Xu, S. (2023). A Cognitive Electronic Jamming Decision-Making Method Based on Q-Learning and Ant Colony Fusion Algorithm. Remote Sens., 15.","DOI":"10.3390\/rs15123108"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"8647386","DOI":"10.1155\/2021\/8647386","article-title":"Cognitive Electronic Jamming Decision-Making Method Based on Improved Q-Learning Algorithm","volume":"2021","author":"Li","year":"2021","journal-title":"Int. J. Aerosp. Eng."},{"key":"ref_15","first-page":"52","article-title":"A Review on Reinforcement Learning Based Radar Jamming Decision-Making Technology","volume":"29","author":"Zhu","year":"2022","journal-title":"Electron. Opt. Control."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1603","DOI":"10.1360\/N112018-00153","article-title":"The architecture and technology of cognitive electronic warfare","volume":"48","author":"Wang","year":"2018","journal-title":"Sci. Sin. Inf."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Ye, F., Che, F., and Tian, H. (2017, January 19\u201322). Cognitive cooperative-jamming decision method based on bee colony algorithm. Proceedings of the 2017 Progress in Electromagnetics Research Symposium\u2014Fall (PIERS-FALL), Singapore.","DOI":"10.1109\/PIERS-FALL.2017.8293195"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Shao, S., Zhang, T., and Ye, F. (2017, January 19\u201322). Jamming resource allocation aimed to data link based on simulant annealing algorithm. Proceedings of the 2017 Progress in Electromagnetics Research Symposium-Fall (PIERS-FALL), Singapore.","DOI":"10.1109\/PIERS-FALL.2017.8293276"},{"key":"ref_19","first-page":"1031","article-title":"Intelligence radar countermeasure based on Q-learning","volume":"40","author":"Xing","year":"2018","journal-title":"Syst. Eng. Electron."},{"key":"ref_20","first-page":"819","article-title":"DQN Based Decision-making method of congnitive jamming against multifunctional radar","volume":"42","author":"Zhang","year":"2020","journal-title":"Syst. Eng. Electron."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"427","DOI":"10.1587\/nolta.13.427","article-title":"An investigation of the relationship between numerical precision and performance of Q-learning for hardware implementation","volume":"13","author":"Oguchi","year":"2022","journal-title":"Nonlinear Theory Its Appl. IEICE"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Cardarilli, G.C., Di Nunzio, L., Fazzolari, R., Giardino, D., Matta, M., Nannarelli, A., Re, M., and Span\u00f2, S. (2020, January 1\u20134). FPGA Implementation of Q-RTS for Real-Time Swarm Intelligence Systems. Proceedings of the Conference Record\u2014Asilomar Conference on Signals, Systems and Computers, Virtual.","DOI":"10.1109\/IEEECONF51394.2020.9443368"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2782","DOI":"10.1109\/ACCESS.2018.2885950","article-title":"Parallel Implementation of Reinforcement Learning Q-Learning Technique for FPGA","volume":"7","author":"Torquato","year":"2019","journal-title":"IEEE Access"},{"key":"ref_24","first-page":"1194","article-title":"Design of Cognitive Radar Jamming Based on Q-Learning Algorithm","volume":"35","author":"Li","year":"2015","journal-title":"Trans. Beijing Inst. Technol."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/7\/1190\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:20:20Z","timestamp":1760106020000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/7\/1190"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,28]]},"references-count":24,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2024,4]]}},"alternative-id":["rs16071190"],"URL":"https:\/\/doi.org\/10.3390\/rs16071190","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,28]]}}}