{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,22]],"date-time":"2026-01-22T18:24:20Z","timestamp":1769106260080,"version":"3.49.0"},"reference-count":37,"publisher":"Springer Science and Business Media LLC","issue":"6","license":[{"start":{"date-parts":[[2023,7,3]],"date-time":"2023-07-03T00:00:00Z","timestamp":1688342400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,7,3]],"date-time":"2023-07-03T00:00:00Z","timestamp":1688342400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62103375"],"award-info":[{"award-number":["62103375"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Key project of Philosophy and Social Science of Zhejiang Province","award":["22NDJC009Z"],"award-info":[{"award-number":["22NDJC009Z"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Complex Intell. Syst."],"published-print":{"date-parts":[[2023,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Existing research shows that cooperative multi-agent deep reinforcement learning (c-MADRL) is vulnerable to adversarial attacks, and c-MADRL is increasingly being applied to safety-critical domains. However, the robustness of c-MADRL against adversarial attacks has not been fully studied. In the setting of c-MADRL, unlike the single-agent scenario, an adversary can attack multiple agents or all agents at each time step, but the attacker needs more computation to generate adversarial examples and will be more easily detected. Therefore, how the attacker chooses one or several agents instead of all agents to attack is a significant issue in the setting of c-MADRL. Aiming to address this issue, this paper proposes a novel adversarial attack approach, which dynamically groups the agents according to relevant features and selects a group to attack based on the group\u2019s contribution to the overall reward, thus effectively reducing the cost and number of attacks, as well as improving attack efficiency and decreasing the chance of attackers being detected. Moreover, we exploit the transferability of adversarial examples to greatly reduce the computational cost of generating adversarial examples. Our method is tested in multi-agent particle environments (MPE) and in StarCraft II. Experimental results demonstrate that our proposed method can effectively degrade the performance of multi-agent deep reinforcement learning algorithms with fewer attacks and lower computational costs.<\/jats:p>","DOI":"10.1007\/s40747-023-01145-w","type":"journal-article","created":{"date-parts":[[2023,7,3]],"date-time":"2023-07-03T09:02:36Z","timestamp":1688374956000},"page":"7439-7450","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Adversarial attacks on cooperative multi-agent deep reinforcement learning: a dynamic group-based adversarial example transferability method"],"prefix":"10.1007","volume":"9","author":[{"given":"Lixia","family":"Zan","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7281-6680","authenticated-orcid":false,"given":"Xiangbin","family":"Zhu","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4548-8674","authenticated-orcid":false,"given":"Zhao-Long","family":"Hu","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,7,3]]},"reference":[{"key":"1145_CR1","doi-asserted-by":"crossref","unstructured":"Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G, et al. (2015) Human-level control through deep reinforcement learning 518(7540):529\u2013533","DOI":"10.1038\/nature14236"},{"key":"1145_CR2","doi-asserted-by":"crossref","unstructured":"Sharif A, Marijan D (2021) Evaluating the robustness of deep reinforcement learning for autonomous and adversarial policies in a multi-agent urban driving environment","DOI":"10.1109\/QRS57517.2022.00084"},{"key":"1145_CR3","unstructured":"Shalev-Shwartz S, Shammah S, Shashua A (2016) Safe, multi-agent, reinforcement learning for autonomous driving"},{"key":"1145_CR4","doi-asserted-by":"crossref","unstructured":"Gu S, Holly E, Lillicrap T, Levine S (2017) Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 3389\u20133396","DOI":"10.1109\/ICRA.2017.7989385"},{"key":"1145_CR5","unstructured":"Goodfellow IJ, Shlens J, Szegedy C (2014) Explaining and harnessing adversarial examples"},{"key":"1145_CR6","unstructured":"Carlini N, Wagner D (2016) Defensive distillation is not robust to adversarial examples"},{"key":"1145_CR7","unstructured":"Kos J, Song D (2017) Delving into adversarial attacks on deep policies"},{"key":"1145_CR8","unstructured":"Lin Y-C, Liu M-Y, Sun M, Huang J-B (2017) Detecting adversarial attacks on neural network policies with visual foresight"},{"key":"1145_CR9","unstructured":"Ma X, Liang Z, Xia L, Zhang J, Blanchet J, Liu M, Zhao Q, Zhou Z (2022) Distributionally robust offline reinforcement learning with linear function approximation"},{"key":"1145_CR10","doi-asserted-by":"crossref","unstructured":"Bravo M, Mertikopoulos P (2017) On the robustness of learning in games with stochastically perturbed payoff observations 103:41\u201366","DOI":"10.1016\/j.geb.2016.06.004"},{"issue":"2","key":"1145_CR11","first-page":"90","volume":"3","author":"I Ilahi","year":"2021","unstructured":"Ilahi I, Usama M, Qadir J, Janjua MU, Al-Fuqaha A, Hoang DT, Niyato D (2021) Challenges and countermeasures for adversarial attacks on deep reinforcement learning 3(2):90\u2013109","journal-title":"Challenges and countermeasures for adversarial attacks on deep reinforcement learning"},{"key":"1145_CR12","doi-asserted-by":"crossref","unstructured":"Hernandez-Leal P, Kartal B, Taylor ME (2019) A survey and critique of multiagent deep reinforcement learning 33(6): 750\u2013797","DOI":"10.1007\/s10458-019-09421-1"},{"key":"1145_CR13","unstructured":"Christianos F, Papoudakis G, Rahman MA, Albrecht SV (2021) Scaling multi-agent reinforcement learning with selective parameter sharing. In: International Conference on Machine Learning, pp. 1989\u20131998"},{"key":"1145_CR14","doi-asserted-by":"crossref","unstructured":"Lin J, Dzeparoska K, Zhang SQ, Leon-Garcia A, Papernot N (2020) On the robustness of cooperative multi-agent reinforcement learning. In: 2020 IEEE Security and Privacy Workshops (SPW), pp. 62\u201368","DOI":"10.1109\/SPW50608.2020.00027"},{"key":"1145_CR15","unstructured":"Xue W, Qiu W, An B, Rabinovich Z, Obraztsova S, Yeo CK (2021) Mis-spoke or mis-lead: achieving robustness in multi-agent communicative reinforcement learning"},{"key":"1145_CR16","unstructured":"Vinyals O Ewalds T, Bartunov S, Georgiev P, Vezhnevets AS, Yeo M, Makhzani A, K\u00fcttler H, Agapiou J, Schrittwieser J, et al. (2017) Starcraft ii: a new challenge for reinforcement learning"},{"key":"1145_CR17","unstructured":"Papernot N, McDaniel P, Goodfellow I (2016) Transferability in machine learning: from phenomena to black-box attacks using adversarial samples"},{"key":"1145_CR18","doi-asserted-by":"crossref","unstructured":"Mordatch I, Abbeel P (2018) Emergence of grounded compositional language in multi-agent populations. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32","DOI":"10.1609\/aaai.v32i1.11492"},{"key":"1145_CR19","unstructured":"Huang S, Papernot N, Goodfellow I, Duan Y, Abbeel P (2017) Adversarial attacks on neural network policies"},{"key":"1145_CR20","doi-asserted-by":"crossref","unstructured":"Lin Y-C, Hong Z-W, Liao Y-H, Shih M-L, Liu M-Y, Sun M (2017) Tactics of adversarial attack on deep reinforcement learning agents","DOI":"10.24963\/ijcai.2017\/525"},{"key":"1145_CR21","doi-asserted-by":"crossref","unstructured":"Carlini N, Wagner D (2017) Towards evaluating the robustness of neural networks. In: 2017 Ieee Symposium on Security and Privacy (sp), pp. 39\u201357","DOI":"10.1109\/SP.2017.49"},{"key":"1145_CR22","doi-asserted-by":"crossref","unstructured":"Sun J, Zhang T, Xie X, Ma L, Zheng Y, Chen K, Liu Y (2020) Stealthy and efficient adversarial attacks against deep reinforcement learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 5883\u20135891","DOI":"10.1609\/aaai.v34i04.6047"},{"key":"1145_CR23","unstructured":"Pattanaik A, Tang Z, Liu S, Bommannan G, Chowdhary G (2017) Robust deep reinforcement learning with adversarial attacks"},{"key":"1145_CR24","unstructured":"Zhang H, Chen H, Xiao C, Li B, Liu M, Boning D, Hsieh C-J (2020) Robust deep reinforcement learning against adversarial perturbations on state observations 33:21024\u201321037"},{"key":"1145_CR25","unstructured":"Gleave A, Dennis M, Wild C, Kant N, Levine S, Russell S (2019) Adversarial policies: attacking deep reinforcement learning"},{"key":"1145_CR26","unstructured":"Nisioti E, Bloembergen D, Kaisers M (2021) Robust multi-agent q-learning in cooperative games with adversaries"},{"key":"1145_CR27","unstructured":"Hu Y, Zhang Z (2022) Sparse adversarial attack in multi-agent reinforcement learning"},{"key":"1145_CR28","doi-asserted-by":"crossref","unstructured":"Li S, Wu Y, Cui X, Dong H, Fang F, Russell S (2019) Robust multi-agent reinforcement learning via minimax deep deterministic policy gradient. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 4213\u20134220","DOI":"10.1609\/aaai.v33i01.33014213"},{"key":"1145_CR29","doi-asserted-by":"crossref","unstructured":"Guo J, Chen Y, Hao Y, Yin Z, Yu Y, Li S (2022) Towards comprehensive testing on the robustness of cooperative multi-agent reinforcement learning. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, pp. 115\u2013122","DOI":"10.1109\/CVPRW56347.2022.00022"},{"key":"1145_CR30","unstructured":"Pham NH, Nguyen LM, Chen J, Lam HT, Das S, Weng T-W (2022) Evaluating robustness of cooperative MARL: a model-based approach"},{"key":"1145_CR31","doi-asserted-by":"crossref","unstructured":"Tu J, Wang T, Wang J, Manivasagam S, Ren M, Urtasun R (2021) Adversarial attacks on multi-agent communication. In: Proceedings of the IEEE\/CVF International Conference on Computer Vision, pp. 7768\u20137777","DOI":"10.1109\/ICCV48922.2021.00767"},{"key":"1145_CR32","doi-asserted-by":"crossref","unstructured":"Oliehoek FA, Amato C (2016) A Concise Introduction to Decentralized POMDPs. Springer","DOI":"10.1007\/978-3-319-28929-8"},{"key":"1145_CR33","unstructured":"Lowe R, Wu YI, Tamar A, Harb J, Pieter\u00a0Abbeel O, Mordatch I (2017) Multi-agent actor-critic for mixed cooperative-competitive environments 30"},{"key":"1145_CR34","unstructured":"Lillicrap TP, Hunt JJ, Pritzel A, Heess N, Erez T, Tassa Y, Silver D, Wierstra D (2015) Continuous control with deep reinforcement learning"},{"key":"1145_CR35","unstructured":"Rashid T, Samvelyan M, Schroeder C, Farquhar G, Foerster J, Whiteson S (2018) Qmix: monotonic value function factorisation for deep multi-agent reinforcement learning. In: International Conference on Machine Learning, pp. 4295\u20134304"},{"key":"1145_CR36","doi-asserted-by":"crossref","unstructured":"Foerster J, Farquhar G, Afouras T, Nardelli N, Whiteson S (2018) Counterfactual multi-agent policy gradients. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32","DOI":"10.1609\/aaai.v32i1.11794"},{"key":"1145_CR37","doi-asserted-by":"crossref","unstructured":"Yang C-HH, Qi J, Chen P-Y, Ouyang Y, Hung I-TD, Lee C-H, Ma X (2020) Enhanced adversarial strategically-timed attacks against deep reinforcement learning. In: ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3407\u20133411","DOI":"10.1109\/ICASSP40776.2020.9053342"}],"container-title":["Complex &amp; Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-023-01145-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s40747-023-01145-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-023-01145-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,27]],"date-time":"2023-10-27T19:08:50Z","timestamp":1698433730000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s40747-023-01145-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,3]]},"references-count":37,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2023,12]]}},"alternative-id":["1145"],"URL":"https:\/\/doi.org\/10.1007\/s40747-023-01145-w","relation":{},"ISSN":["2199-4536","2198-6053"],"issn-type":[{"value":"2199-4536","type":"print"},{"value":"2198-6053","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,3]]},"assertion":[{"value":"18 October 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 June 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 July 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}}]}}