{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,21]],"date-time":"2026-05-21T10:41:23Z","timestamp":1779360083477,"version":"3.51.4"},"reference-count":49,"publisher":"Wiley","issue":"1","license":[{"start":{"date-parts":[[2024,6,5]],"date-time":"2024-06-05T00:00:00Z","timestamp":1717545600000},"content-version":"vor","delay-in-days":156,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["72001214"],"award-info":[{"award-number":["72001214"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62106283"],"award-info":[{"award-number":["62106283"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["52175282"],"award-info":[{"award-number":["52175282"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100017596","name":"Natural Science Basic Research Program of Shaanxi Province","doi-asserted-by":"publisher","award":["2021JM-226"],"award-info":[{"award-number":["2021JM-226"]}],"id":[{"id":"10.13039\/501100017596","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["International Journal of Intelligent Systems"],"published-print":{"date-parts":[[2024,1]]},"abstract":"<jats:p>Intelligent decision\u2010making in air defense operations has attracted wide attention from researchers. Facing complex battlefield environments, existing decision\u2010making algorithms fail to make targeted decisions according to the hierarchical decision\u2010making characteristics of air defense operational command and control. What\u2019s worse, in the process of problem\u2010solving, these algorithms are beset by defects such as dimensional disaster and poor real\u2010time performance. To address these problems, a new hierarchical reinforcement learning algorithm named Hierarchy Asynchronous Advantage Actor\u2010Critic (H\u2010A3C) is developed. This algorithm is designed to have a hierarchical decision\u2010making framework considering the characteristics of air defense operations and employs the hierarchical reinforcement learning method for problem\u2010solving. With a hierarchical decision\u2010making capability similar to that of human commanders in decision\u2010making, the developed algorithm produces many new policies during the learning process. The features of air situation information are extracted using the bidirectional\u2010gated recurrent unit (Bi\u2010GRU) network, and then the agent is trained using the H\u2010A3C algorithm. In the training process, the multihead attention mechanism and the event\u2010based reward mechanism are introduced to facilitate the training. In the end, the proposed H\u2010A3C algorithm is verified in a digital battlefield environment, and the results prove its advantages over existing algorithms.<\/jats:p>","DOI":"10.1155\/2024\/7777050","type":"journal-article","created":{"date-parts":[[2024,6,6]],"date-time":"2024-06-06T15:16:02Z","timestamp":1717686962000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Intelligent Decision\u2010Making System of Air Defense Resource Allocation via Hierarchical Reinforcement Learning"],"prefix":"10.1155","volume":"2024","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1898-9828","authenticated-orcid":false,"given":"Minrui","family":"Zhao","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gang","family":"Wang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1456-4216","authenticated-orcid":false,"given":"Qiang","family":"Fu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wen","family":"Quan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Quan","family":"Wen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3783-1268","authenticated-orcid":false,"given":"Xiaoqiang","family":"Wang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tengda","family":"Li","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yu","family":"Chen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shan","family":"Xue","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jiaozhi","family":"Han","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"311","published-online":{"date-parts":[[2024,6,5]]},"reference":[{"key":"e_1_2_10_1_2","doi-asserted-by":"publisher","DOI":"10.1109\/tsmcb.2012.2231673"},{"key":"e_1_2_10_2_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cor.2018.10.015"},{"key":"e_1_2_10_3_2","doi-asserted-by":"publisher","DOI":"10.1117\/12.438322"},{"key":"e_1_2_10_4_2","doi-asserted-by":"publisher","DOI":"10.26599\/tst.2021.9010013"},{"key":"e_1_2_10_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/tgcn.2021.3067555"},{"key":"e_1_2_10_6_2","doi-asserted-by":"publisher","DOI":"10.3390\/e24121825"},{"key":"e_1_2_10_7_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.dt.2021.01.005"},{"key":"e_1_2_10_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/tvt.2020.2982508"},{"key":"e_1_2_10_9_2","doi-asserted-by":"publisher","DOI":"10.3390\/sym13020271"},{"key":"e_1_2_10_10_2","doi-asserted-by":"crossref","unstructured":"ZhangY. WangG. HuangX. XiJ. DangY. andMiaoH. Research on task assignment of cruise ammunition cooperative attack based on dragonfly algorithm Proceedings of the International Conference on Algorithms High Performance Computing and Artificial Intelligence (AHPCAI ) in 2021 December 2021 Sanya China.","DOI":"10.1117\/12.2626422"},{"key":"e_1_2_10_11_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10479-020-03919-8"},{"key":"e_1_2_10_12_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cor.2020.104890"},{"key":"e_1_2_10_13_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cie.2021.107717"},{"key":"e_1_2_10_14_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.omega.2019.102138"},{"key":"e_1_2_10_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/tsmc.2017.2784187"},{"key":"e_1_2_10_16_2","doi-asserted-by":"crossref","unstructured":"GengZ. HuangY. ZhangH. andChenT. Improved sparrow search algorithm applied to multi-stage weapon target assignment Proceedings of the 2022 International Conference on Cyber-Physical Social Intelligence (ICCSI) November 2022 Nanjing China https:\/\/doi.org\/10.1109\/ICCSI55536.2022.9970663.","DOI":"10.1109\/ICCSI55536.2022.9970663"},{"key":"e_1_2_10_17_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-019-1724-z"},{"key":"e_1_2_10_18_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.dt.2022.04.001"},{"key":"e_1_2_10_19_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-021-04357-7"},{"key":"e_1_2_10_20_2","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-021-00431-x"},{"key":"e_1_2_10_21_2","doi-asserted-by":"publisher","DOI":"10.1162\/neco_a_01454"},{"key":"e_1_2_10_22_2","volume-title":"Reinforcement Learning: An Introduction","author":"Sutton R. S.","year":"2018"},{"key":"e_1_2_10_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/tcyb.2020.2977374"},{"key":"e_1_2_10_24_2","doi-asserted-by":"crossref","unstructured":"XuC. T.andSongH. B. Mixed initiative balance of human-swarm teaming in surveillance via reinforcement learning Proceedings of the IEEE-AIAA Digital Avionics Systems Conference June 2021 San Antonio TX USA.","DOI":"10.1109\/DASC52595.2021.9594355"},{"key":"e_1_2_10_25_2","doi-asserted-by":"publisher","DOI":"10.1038\/nature16961"},{"key":"e_1_2_10_26_2","doi-asserted-by":"publisher","DOI":"10.1038\/nature24270"},{"key":"e_1_2_10_27_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-019-1724-z"},{"key":"e_1_2_10_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/tits.2022.3229518"},{"key":"e_1_2_10_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/tpami.2023.3322426"},{"key":"e_1_2_10_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/tetc.2019.2902661"},{"key":"e_1_2_10_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/jiot.2019.2961707"},{"key":"e_1_2_10_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/tetc.2018.2805718"},{"key":"e_1_2_10_33_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2022.117796"},{"key":"e_1_2_10_34_2","doi-asserted-by":"publisher","DOI":"10.3233\/jifs-189081"},{"key":"e_1_2_10_35_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2022.119205"},{"key":"e_1_2_10_36_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2012.07.050"},{"key":"e_1_2_10_37_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.engappai.2020.104112"},{"key":"e_1_2_10_38_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2018.11.014"},{"key":"e_1_2_10_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/access.2020.2993459"},{"key":"e_1_2_10_40_2","doi-asserted-by":"publisher","DOI":"10.3390\/electronics11111796"},{"key":"e_1_2_10_41_2","doi-asserted-by":"crossref","unstructured":"LuoP. C. XieJ. J. andCheW. F. Q-Learning based air combat target assignment algorithm Proceedings of the IEEE International Conference on Systems Man and Cybernetics Conference Proceedings June 2016 San Antonio TX USA.","DOI":"10.1109\/SMC.2016.7844336"},{"key":"e_1_2_10_42_2","doi-asserted-by":"publisher","DOI":"10.1002\/aisy.202300151"},{"key":"e_1_2_10_43_2","unstructured":"BabaeizadehM. FrosioI. TyreeS. ClemonsJ. andKautzJ. Reinforcement learning through asynchronous advantage actor-critic on a GPU Proceedings of the International Conference on Learning Representations (ICLR) in 2016 May 2016 San Juan Puerto Rico."},{"key":"e_1_2_10_44_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cor.2021.105400"},{"key":"e_1_2_10_45_2","doi-asserted-by":"publisher","DOI":"10.1561\/2200000086"},{"key":"e_1_2_10_46_2","volume-title":"Proximal Policy Optimization Algorithms","author":"Schulman J.","year":"2017"},{"key":"e_1_2_10_47_2","doi-asserted-by":"publisher","DOI":"10.3390\/app11114948"},{"key":"e_1_2_10_48_2","doi-asserted-by":"publisher","DOI":"10.1155\/2023\/8569161"},{"key":"e_1_2_10_49_2","unstructured":"GlorotX.andBengioY. Understanding the difficulty of training deep feedforward neural networks Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (PMLR) in 2010 March 2010 Haifa Israel."}],"container-title":["International Journal of Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1155\/2024\/7777050","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,6]],"date-time":"2024-06-06T15:16:12Z","timestamp":1717686972000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1155\/2024\/7777050"}},"subtitle":[],"editor":[{"given":"Riccardo","family":"Ortale","sequence":"additional","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2024,1]]},"references-count":49,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,1]]}},"alternative-id":["10.1155\/2024\/7777050"],"URL":"https:\/\/doi.org\/10.1155\/2024\/7777050","archive":["Portico"],"relation":{},"ISSN":["0884-8173","1098-111X"],"issn-type":[{"value":"0884-8173","type":"print"},{"value":"1098-111X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,1]]},"assertion":[{"value":"2022-09-06","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-05-16","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-06-05","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"7777050"}}