{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,9]],"date-time":"2026-04-09T14:25:44Z","timestamp":1775744744372,"version":"3.50.1"},"reference-count":59,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2023,6,14]],"date-time":"2023-06-14T00:00:00Z","timestamp":1686700800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Key R&amp;D Program of China","award":["2021YFA0716600"],"award-info":[{"award-number":["2021YFA0716600"]}]},{"name":"National Key R&amp;D Program of China","award":["JCYJ20180307151430655"],"award-info":[{"award-number":["JCYJ20180307151430655"]}]},{"name":"National Key R&amp;D Program of China","award":["KQTD20190929172704911"],"award-info":[{"award-number":["KQTD20190929172704911"]}]},{"name":"Shenzhen Fundamental Research Programunder","award":["2021YFA0716600"],"award-info":[{"award-number":["2021YFA0716600"]}]},{"name":"Shenzhen Fundamental Research Programunder","award":["JCYJ20180307151430655"],"award-info":[{"award-number":["JCYJ20180307151430655"]}]},{"name":"Shenzhen Fundamental Research Programunder","award":["KQTD20190929172704911"],"award-info":[{"award-number":["KQTD20190929172704911"]}]},{"name":"Shenzhen Science and Technology Program","award":["2021YFA0716600"],"award-info":[{"award-number":["2021YFA0716600"]}]},{"name":"Shenzhen Science and Technology Program","award":["JCYJ20180307151430655"],"award-info":[{"award-number":["JCYJ20180307151430655"]}]},{"name":"Shenzhen Science and Technology Program","award":["KQTD20190929172704911"],"award-info":[{"award-number":["KQTD20190929172704911"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>In order to improve the efficiency and adaptability of cognitive radar jamming decision-making, a fusion algorithm (Ant-QL) based on ant colony and Q-Learning is proposed in this paper. The algorithm does not rely on a priori information and enhances adaptability through real-time interactions between the jammer and the target radar. At the same time, it can be applied to single jammer and multiple jammer countermeasure scenarios with high jamming effects. First, traditional Q-Learning and DQN algorithms are discussed, and a radar jamming decision-making model is built for the simulation verification of each algorithm. Then, an improved Q-Learning algorithm is proposed to address the shortcomings of both algorithms. By introducing the pheromone mechanism of ant colony algorithms in Q-Learning and using the \u03b5-greedy algorithm to balance the contradictory relationship between exploration and exploitation, the algorithm greatly avoids falling into a local optimum, thus accelerating the convergence speed of the algorithm with good stability and robustness in the convergence process. In order to better adapt to the cluster countermeasure environment in future battlefields, the algorithm and model are extended to cluster cooperative jamming decision-making. We map each jammer in the cluster to an intelligent ant searching for the optimal path, and multiple jammers interact with each other to obtain information. During the process of confrontation, the method greatly improves the convergence speed and stability and reduces the need for hardware and power resources of the jammer. Assuming that the number of jammers is three, the experimental simulation results of the convergence speed of the Ant-QL algorithm improve by 85.4%, 80.56% and 72% compared with the Q-Learning, DQN and improved Q-Learning algorithms, respectively. During the convergence process, the Ant-QL algorithm is very stable and efficient, and the algorithm complexity is low. After the algorithms converge, the average response times of the four algorithms are 6.99 \u00d7 10\u22124 s, 2.234 \u00d7 10\u22123 s, 2.21 \u00d7 10\u22124 s and 1.7 \u00d7 10\u22124 s, respectively. The results show that the improved Q-Learning algorithm and Ant-QL algorithm also have more advantages in terms of average response time after convergence.<\/jats:p>","DOI":"10.3390\/rs15123108","type":"journal-article","created":{"date-parts":[[2023,6,15]],"date-time":"2023-06-15T02:03:19Z","timestamp":1686794599000},"page":"3108","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":30,"title":["A Cognitive Electronic Jamming Decision-Making Method Based on Q-Learning and Ant Colony Fusion Algorithm"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0839-3908","authenticated-orcid":false,"given":"Chudi","family":"Zhang","sequence":"first","affiliation":[{"name":"School of Electronics and Communication Engineering, Sun Yat-sen University, Shenzhen 528406, China"}]},{"given":"Yunqi","family":"Song","sequence":"additional","affiliation":[{"name":"School of Electronics and Communication Engineering, Sun Yat-sen University, Shenzhen 528406, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9036-6553","authenticated-orcid":false,"given":"Rundong","family":"Jiang","sequence":"additional","affiliation":[{"name":"School of Electronics and Communication Engineering, Sun Yat-sen University, Shenzhen 528406, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6816-9417","authenticated-orcid":false,"given":"Jun","family":"Hu","sequence":"additional","affiliation":[{"name":"School of Electronics and Communication Engineering, Sun Yat-sen University, Shenzhen 528406, China"}]},{"given":"Shiyou","family":"Xu","sequence":"additional","affiliation":[{"name":"School of Electronics and Communication Engineering, Sun Yat-sen University, Shenzhen 528406, China"}]}],"member":"1968","published-online":{"date-parts":[[2023,6,14]]},"reference":[{"key":"ref_1","unstructured":"Haigh, K., and Andrusenko, J. (2021). Cognitive Electronic Warfare: An Artificial Intelligence Approach, Artech House."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Haykin, S. (2010, January 10\u201314). New generation of radar systems enabled with cognition. Proceedings of the 2010 IEEE Radar Conference, Arlington, VA, USA.","DOI":"10.1109\/RADAR.2010.5494676"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1109\/MSP.2006.1593335","article-title":"Cognitive radar: A way of the future","volume":"23","author":"Haykin","year":"2006","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_4","unstructured":"Darpa, A. (2010). Behavioral Learning for Adaptive Electronic Warfare, Defense Advanced Research Projects Agency. Darpa-BAA-10-79."},{"key":"ref_5","first-page":"16","article-title":"DARPA seeks proposals for adaptive radar countermeasures J","volume":"2012","author":"Haystead","year":"2012","journal-title":"J. Electron. Def."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1109\/MAES.2019.2953762","article-title":"An overview of cognitive radar: Past, present, and future","volume":"34","author":"Gurbuz","year":"2019","journal-title":"IEEE Aerosp. Electron. Syst. Mag."},{"key":"ref_7","unstructured":"Zhou, H. (2020). Communications, Signal Processing, and Systems, Proceedings of the 2018 CSPS, Dalian, China, 14\u201316 July 2018, Springer. Volume III: Systems 7th."},{"key":"ref_8","unstructured":"du Plessis, W.P., and Osner, N.R. (2018, January 13\u201316). Cognitive electronic warfare (EW) systems as a training aid. Proceedings of the Electronic Warfare International Conference (EWCI), Bangalore, India."},{"key":"ref_9","first-page":"1603","article-title":"Cognitive Electronic Warfare Architecture and Technology","volume":"48","author":"Shafei","year":"2018","journal-title":"China Sci. Inf. Sci."},{"key":"ref_10","first-page":"52","article-title":"A Review of Reinforcement Learning Based Radar Jamming Decision Making Techniques","volume":"29","author":"Kun","year":"2022","journal-title":"Electro-Opt. Control."},{"key":"ref_11","first-page":"306","article-title":"Review of electronic countermeasure jamming effect evaluation techniques","volume":"15","author":"Songtao","year":"2020","journal-title":"J. Chin. Acad. Electron. Sci."},{"key":"ref_12","first-page":"36","article-title":"New field of electronic warfare-AI","volume":"2","author":"Gong","year":"1986","journal-title":"Aerosp. Shanghai"},{"key":"ref_13","first-page":"27","article-title":"Application of AI technology in EW","volume":"2","author":"Li","year":"1988","journal-title":"Electron. Warf. Technol."},{"key":"ref_14","unstructured":"Wang, X., Zhu, M., and Cheng, Y. (2014). The Principle of Reinforcement Learning and Its Application, Science Press."},{"key":"ref_15","first-page":"103","article-title":"Reinforcement learning: An introduction","volume":"21","author":"Thrun","year":"2000","journal-title":"AI Mag."},{"key":"ref_16","first-page":"10","article-title":"Modeling and simulation of cognitive electronic attack operations under system countermeasures","volume":"5","author":"Yang","year":"2019","journal-title":"J. Chin. Acad. Electron. Sci."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Wonderley, D., Selee, T., and Chakravarthy, V. (2016, January 2\u20136). Game theoretic decision support framework for electronic warfare applications. Proceedings of the 2016 IEEE Radar Conference (RadarConf), Philadelphia, PA, USA.","DOI":"10.1109\/RADAR.2016.7485124"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"5314","DOI":"10.1109\/TWC.2018.2841921","article-title":"Game theory-based anti-jamming strategies for frequency hopping wireless communications","volume":"17","author":"Gao","year":"2018","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_19","unstructured":"Hongwei, S., Ningning, T., and Fujun, S. (2003). Electronic interference pattern selection based on DS evidence theory. J. Ballist. Arrow Guid., 218\u2013220."},{"key":"ref_20","first-page":"83","article-title":"Research on self-learning model of intelligent radar jamming system","volume":"1","author":"Gong","year":"2009","journal-title":"Mod. Def. Technol."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"7490895","DOI":"10.1155\/2018\/7490895","article-title":"Multiobjective cognitive cooperative jamming decision-making method based on Tabu search-artificial bee colony algorithm","volume":"2018","author":"Ye","year":"2018","journal-title":"Int. J. Aerosp. Eng."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Ye, F., Che, F., and Tian, H. (2017, January 19\u201322). Cognitive cooperative-jamming decision method based on bee colony algorithm. Proceedings of the 2017 Progress in Electromagnetics Research Symposium-Fall (PIERS-FALL), Singapore.","DOI":"10.1109\/PIERS-FALL.2017.8293195"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Pan, W., Jin, X., Xie, H., and Xia, Y. (2020, January 22\u201324). Radar jamming strategy allocation algorithm based on improved chaos genetic algorithm. Proceedings of the 2020 Chinese Control And Decision Conference (CCDC), Hefei, China.","DOI":"10.1109\/CCDC49329.2020.9164855"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Tan, Y., Pan, W., Han, Y., and Xu, S. (2019, January 3\u20135). Research on force assignment of radar jamming system based on chaos genetic algorithm. Proceedings of the 2019 Chinese Control And Decision Conference (CCDC), Nanchang, China.","DOI":"10.1109\/CCDC.2019.8832823"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Gu, W., Zhu, L., Bu, Y., Yue, W., Cai, X., and Fan, Y. (2020, January 28\u201331). Collaborative jamming decision-making mechanism using ant colony algorithm in electromagnetic antagonism. Proceedings of the 2020 IEEE 20th International Conference on Communication Technology (ICCT), Nanning, China.","DOI":"10.1109\/ICCT50939.2020.9295683"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"5094","DOI":"10.1109\/TITS.2019.2948596","article-title":"Coordinated charging scheduling of electric vehicles: A mixed-variable differential evolution approach","volume":"21","author":"Liu","year":"2019","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1430","DOI":"10.1109\/TCYB.2019.2939219","article-title":"A self-adaptive differential evolution algorithm for scheduling a single batch-processing machine with arbitrary job sizes and release times","volume":"51","author":"Zhou","year":"2019","journal-title":"IEEE Trans. Cybern."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"113678","DOI":"10.1016\/j.eswa.2020.113678","article-title":"An ensemble discrete differential evolution for the distributed blocking flowshop scheduling with minimizing makespan criterion","volume":"160","author":"Zhao","year":"2020","journal-title":"Expert Syst. Appl."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"113166","DOI":"10.1016\/j.eswa.2019.113166","article-title":"A hybrid discrete water wave optimization algorithm for the no-idle flowshop scheduling problem with total tardiness criterion","volume":"146","author":"Zhao","year":"2020","journal-title":"Expert Syst. Appl."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"548473","DOI":"10.1155\/2014\/548473","article-title":"Cognitive radio transceivers: RF, spectrum sensing, and learning algorithms review","volume":"2014","author":"Safatly","year":"2014","journal-title":"Int. J. Antennas Propag."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Akanksha, E., Sharma, N., and Gulati, K. (2021, January 8\u201310). Review on reinforcement learning, research evolution and scope of application. Proceedings of the 2021 5th International Conference on Computing Methodologies and Communication (ICCMC), Erode, India.","DOI":"10.1109\/ICCMC51019.2021.9418283"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"8647386","DOI":"10.1155\/2021\/8647386","article-title":"Cognitive electronic jamming decision-making method based on improved Q-learning algorithm","volume":"2021","author":"Li","year":"2021","journal-title":"Int. J. Aerosp. Eng."},{"key":"ref_33","first-page":"2779","article-title":"An efficient computing of correlated equilibrium for cooperative Q-learning-based multi-robot planning","volume":"50","author":"Sadhu","year":"2018","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"3803","DOI":"10.1109\/TNNLS.2019.2899311","article-title":"Kinodynamic motion planning with continuous-time Q-learning: An online, model-free, and safe navigation framework","volume":"30","author":"Kontoudis","year":"2019","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1308","DOI":"10.1109\/TNNLS.2018.2861945","article-title":"Off-policy interleaved Q-learning: Optimal control for affine nonlinear discrete-time systems","volume":"30","author":"Li","year":"2018","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"4109","DOI":"10.1109\/TSMC.2019.2957000","article-title":"Reinforcement Q-learning algorithm for H\u221e tracking control of unknown discrete-time linear systems","volume":"50","author":"Peng","year":"2019","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"2714","DOI":"10.1109\/TIE.2020.2975466","article-title":"Efficient hybrid central processing unit\/input\u2013output resource scheduling for virtual machines","volume":"68","author":"Wang","year":"2020","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"2773","DOI":"10.1109\/JSAC.2020.3005495","article-title":"Deep reinforcement learning for dynamic uplink\/downlink resource allocation in high mobility 5G HetNet","volume":"38","author":"Tang","year":"2020","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Qiang, X., Wei-gang, Z., and Xin, J. (2017, January 22\u201325). Intelligent countermeasure design of radar working-modes unknown. Proceedings of the 2017 IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC), Xiamen, China.","DOI":"10.1109\/ICSPCC.2017.8242558"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Qiang, X., Weigang, Z., and Xin, J. (2017, January 8\u201311). Research on method of intelligent radar confrontation based on reinforcement learning. Proceedings of the 2017 2nd IEEE International Conference on Computational Intelligence and Applications (ICCIA), Beijing, China.","DOI":"10.1109\/CIAPP.2017.8167262"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Kang, L., Bo, J., Hongwei, L., and Siyuan, L. (2018, January 14\u201316). Reinforcement learning based anti-jamming frequency hopping strategies design for cognitive radar. Proceedings of the 2018 IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC), Qingdao, China.","DOI":"10.1109\/ICSPCC.2018.8567751"},{"key":"ref_42","first-page":"2488","article-title":"Markov-based multifunctional radar cognitive jamming decision modeling study","volume":"44","author":"Kun","year":"2022","journal-title":"Syst. Eng. Electron. Technol."},{"key":"ref_43","first-page":"3685","article-title":"A priori knowledge-based intelligent jamming decision method for multifunctional radar","volume":"44","author":"Kun","year":"2022","journal-title":"Syst. Eng. Electron. Technol."},{"key":"ref_44","first-page":"1194","article-title":"Q-learning algorithm-based design of cognitive radar countermeasure process","volume":"35","author":"Yunjie","year":"2015","journal-title":"J. Beijing Univ. Technol."},{"key":"ref_45","first-page":"819","article-title":"DQN cognitive interference decision-making approach for multifunctional radars","volume":"42","author":"Kai","year":"2020","journal-title":"Syst. Eng. Electron. Technol."},{"key":"ref_46","first-page":"129","article-title":"A cognitive jamming decision method for multi-functional radar based on Q-learning","volume":"60","author":"Zhang","year":"2020","journal-title":"Telecommun. Eng."},{"key":"ref_47","unstructured":"Visnevski, N., Krishnamurthy, V., Haykin, S., Currie, B., Dilkes, F., and Lavoie, P. (2003, January 9\u201312). Multi-function radar emitter modelling: A stochastic discrete event system approach. Proceedings of the 42nd IEEE International Conference on Decision and Control (IEEE Cat. No. 03CH37475), Maui, HI, USA."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"1000","DOI":"10.1109\/JPROC.2007.893252","article-title":"Syntactic modeling and signal processing of multifunction radars: A stochastic context-free grammar approach","volume":"95","author":"Visnevski","year":"2007","journal-title":"Proc. IEEE"},{"key":"ref_49","first-page":"86","article-title":"A3C-based multifunctional radar cognitive jamming decision method","volume":"45","author":"Zou","year":"2023","journal-title":"Syst. Eng. Electron. Technol."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"4637","DOI":"10.1109\/TWC.2021.3131580","article-title":"Multi-UAV aided millimeter-wave networks: Positioning, clustering, and beamforming","volume":"21","author":"Zhu","year":"2021","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Wang, R., Gu, Y., Zhou, Z., Wang, Z., Xu, F., Luo, J., Ma, L., and Qiu, H. (2021, January 21\u201325). A Dynamic Task Assignment Strategy for Emitter Reconnaissance and Positioning through Use of UAV Swarms. Proceedings of the 11th International Conference on Computer Engineering and Networks, Hechi, China.","DOI":"10.1007\/978-981-16-6554-7_182"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Han, B., Qu, X., Yang, X., Li, W., and Zhang, Z. (2023). DRFM-Based Repeater Jamming Reconstruction and Cancellation Method with Accurate Edge Detection. Remote Sens., 15.","DOI":"10.3390\/rs15071759"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1007\/s11432-007-2017-y","article-title":"Mathematic principles of interrupted-sampling repeater jamming (ISRJ)","volume":"50","author":"Wang","year":"2007","journal-title":"Sci. China Ser. F Inf. Sci."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Sun, Z., Quan, Y., and Liu, Z. (2023). A Non-Uniform Interrupted-Sampling Repeater Jamming Method for Intra-Pulse Frequency Agile Radar. Remote Sens., 15.","DOI":"10.3390\/rs15071851"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"3685","DOI":"10.1109\/TAES.2020.2981268","article-title":"Performance of chirp parameter estimation in the fractional Fourier domains and an algorithm for fast chirp-rate estimation","volume":"56","author":"Aldimashki","year":"2020","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"580","DOI":"10.1109\/TEVC.2022.3175517","article-title":"A stochastic simulation optimization based range gate pull-off jamming method","volume":"27","author":"Wang","year":"2022","journal-title":"IEEE Trans. Evol. Comput."},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Jia, R., Zhang, T., Wang, Y., Deng, Y., and Kong, L. (2020, January 20\u201321). An intelligent range gate pull-off (RGPO) jamming method. Proceedings of the 2020 International Conference on UK-China Emerging Technologies (UCET), Glasgow, UK.","DOI":"10.1109\/UCET51115.2020.9205386"},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"108130","DOI":"10.1016\/j.sigpro.2021.108130","article-title":"Radar active antagonism through deep reinforcement learning: A way to address the challenge of mainlobe jamming","volume":"186","author":"Li","year":"2021","journal-title":"Signal Process."},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Geng, J., Jiu, B., Li, K., Zhao, Y., Liu, H., and Li, H. (2023). Radar and Jammer Intelligent Game under Jamming Power Dynamic Allocation. Remote Sens., 15.","DOI":"10.3390\/rs15030581"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/12\/3108\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:54:38Z","timestamp":1760126078000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/12\/3108"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,14]]},"references-count":59,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2023,6]]}},"alternative-id":["rs15123108"],"URL":"https:\/\/doi.org\/10.3390\/rs15123108","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,14]]}}}