{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,7]],"date-time":"2026-07-07T06:11:23Z","timestamp":1783404683050,"version":"3.54.6"},"reference-count":34,"publisher":"Association for Computing Machinery (ACM)","issue":"3","funder":[{"DOI":"10.13039\/501100003995","name":"Natural Science Foundation of Anhui Province","doi-asserted-by":"crossref","award":["2308085MF203"],"award-info":[{"award-number":["2308085MF203"]}],"id":[{"id":"10.13039\/501100003995","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Advanced Materials\u2013National Science and Technology Major Project","award":["2025ZD0610101"],"award-info":[{"award-number":["2025ZD0610101"]}]},{"name":"China University Industry\u2013Academia\u2013Research Innovation Fund","award":["2024ZY005"],"award-info":[{"award-number":["2024ZY005"]}]},{"name":"University Synergy Innovation Program of Anhui Province","award":["GXXT-2022-055"],"award-info":[{"award-number":["GXXT-2022-055"]}]},{"name":"Open Fund of the Key Laboratory of Flight Techniques and Flight Safety, CAAC","award":["FZ2022KF09 and FZ2022ZZ02"],"award-info":[{"award-number":["FZ2022KF09 and FZ2022ZZ02"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2026,6,30]]},"abstract":"<jats:p>Large-scale multi-agent systems face two core challenges: inefficient policy learning and the explosion of state dimensions. Existing methods often rely on manually designed task sequences to guide agents\u2019 learning in stages, but these designs lack adaptability to agents\u2019 learning abilities, making it difficult to ensure the rationality of task difficulty. Moreover, the representation capability of current network structures is limited, making it challenging to efficiently handle high-dimensional state information and complex interaction relationships. To address these issues, we propose a Progressive Multi-Agent Reinforcement Learning (PMARL) framework. PMARL introduces a task adapter that adaptively selects task difficulty based on agents\u2019 learning abilities, eliminating reliance on manual experience. Additionally, a Dynamic Dimension Adaptive Network (DDAN) is designed, incorporating hypernetwork and self-attention mechanisms to achieve adaptive feature extraction of high-dimensional states and efficient representation of agent interaction relationships. Experimental results demonstrate that PMARL exhibits higher efficiency and better adaptability compared to existing methods when addressing large-scale multi-agent tasks.<\/jats:p>","DOI":"10.1145\/3799229","type":"journal-article","created":{"date-parts":[[2026,3,7]],"date-time":"2026-03-07T15:21:30Z","timestamp":1772896890000},"page":"1-19","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["PMARL: Multi-Agent Reinforcement Learning in Large-Scale Systems"],"prefix":"10.1145","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1742-0250","authenticated-orcid":false,"given":"Baofu","family":"Fang","sequence":"first","affiliation":[{"name":"School of Computer Science and Information, Hefei University of Technology, Hefei, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-9434-3420","authenticated-orcid":false,"given":"Qiong","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Information, Hefei University of Technology, Hefei, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-3916-0822","authenticated-orcid":false,"given":"Hao","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Information, Hefei University of Technology, Hefei, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2442-4572","authenticated-orcid":false,"given":"Kui","family":"Yu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Information, Hefei University of Technology, Hefei, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-5683-7195","authenticated-orcid":false,"given":"Zaijun","family":"Wang","sequence":"additional","affiliation":[{"name":"CAAC Academy of Flight Technology and Safety, Civil Aviation Flight University of China, Guanghan, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,4,27]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.5555\/3398761.3398967"},{"key":"e_1_3_1_3_2","first-page":"1478","volume-title":"Conference on Robot Learning","author":"Beck Jacob","year":"2023","unstructured":"Jacob Beck, Matthew Thomas Jackson, Risto Vuorio, and Shimon Whiteson. 2023. Hypernetworks in meta-reinforcement learning. In Conference on Robot Learning. PMLR, 1478\u20131487."},{"key":"e_1_3_1_4_2","first-page":"9681","article-title":"Variational automatic curriculum learning for sparse-reward cooperative multi-agent problems","volume":"34","author":"Chen Jiayu","year":"2021","unstructured":"Jiayu Chen, Yuanxin Zhang, Yuanfan Xu, Huimin Ma, Huazhong Yang, Jiaming Song, Yu Wang, and Yi Wu. 2021. Variational automatic curriculum learning for sparse-reward cooperative multi-agent problems. Advances in Neural Information Processing Systems 34 (2021), 9681\u20139693.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_5_2","first-page":"6851","article-title":"Multiagent reinforcement learning with heterogeneous graph attention network","author":"Du Wei","year":"2022","unstructured":"Wei Du, Shifei Ding, Chenglong Zhang, and Zhongzhi Shi. 2022. Multiagent reinforcement learning with heterogeneous graph attention network. IEEE Transactions on Neural Networks and Learning Systems 34, 10 (2022), 6851\u20136860.","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"key":"e_1_3_1_6_2","first-page":"6614","volume-title":"Proceedings of the 36th AAAI Conference on Artificial Intelligence","author":"Feng Aosong","year":"2022","unstructured":"Aosong Feng, Chenyu You, Shiqiang Wang, and Leandros Tassiulas. 2022. Kergnns: Interpretable graph neural networks with graph kernels. In Proceedings of the 36th AAAI Conference on Artificial Intelligence, 6614\u20136622."},{"key":"e_1_3_1_7_2","first-page":"9341","volume-title":"Proceedings of the 36th AAAI Conference on Artificial Intelligence","author":"Fu Qingxu","year":"2022","unstructured":"Qingxu Fu, Tenghai Qiu, Jianqiang Yi, Zhiqiang Pu, and Shiguang Wu. 2022. Concentration network for reinforcement learning of large-scale multi-agent systems. In Proceedings of the 36th AAAI Conference on Artificial Intelligence, 9341\u20139349."},{"key":"e_1_3_1_8_2","first-page":"2829","volume-title":"International Conference on Machine Learning","author":"Gu Shixiang","year":"2016","unstructured":"Shixiang Gu, Timothy Lillicrap, Ilya Sutskever, and Sergey Levine. 2016. Continuous deep q-learning with model-based acceleration. In International Conference on Machine Learning. PMLR, 2829\u20132838."},{"key":"e_1_3_1_9_2","unstructured":"Jian Hu Haibin Wu Seth Austin Harding Siyang Jiang and Shih-Wei Liao. 2021. Riit: Rethinking the importance of implementation tricks in multi-agent reinforcement learning. arXiv:2102.03479. Retrieved from https:\/\/arxiv.org\/abs\/2102.03479"},{"key":"e_1_3_1_10_2","first-page":"1","volume-title":"Proceedings of the 9th International Conference on Learning Representations (ICLR \u201921)","author":"Hu Siyi","year":"2021","unstructured":"Siyi Hu, Fengda Zhu, Xiaojun Chang, and Xiaodan Liang. 2021. UPDET: Universal multi-agent reinforcement learning via policy decoupling with transformers. In Proceedings of the 9th International Conference on Learning Representations (ICLR \u201921). International Conference on Representation Learning, 1\u201315."},{"key":"e_1_3_1_11_2","volume-title":"The 11th International Conference on Learning Representations","author":"Jianye Hao","year":"2022","unstructured":"Hao Jianye, Xiaotian Hao, Hangyu Mao, Weixun Wang, Yaodong Yang, Dong Li, Yan Zheng, and Zhen Wang. 2022. Boosting multiagent reinforcement learning via permutation invariant and permutation equivariant networks. In The 11th International Conference on Learning Representations."},{"issue":"3","key":"e_1_3_1_12_2","doi-asserted-by":"crossref","first-page":"1015","DOI":"10.1007\/s10479-022-04904-z","article-title":"A systematic literature review on the use of big data analytics in humanitarian and disaster operations","volume":"335","author":"Kondraganti Abhilash","year":"2024","unstructured":"Abhilash Kondraganti, Gopalakrishnan Narayanamurthy, and Hossein Sharifi. 2024. A systematic literature review on the use of big data analytics in humanitarian and disaster operations. Annals of Operations Research 335, 3 (2024), 1015\u20131052.","journal-title":"Annals of Operations Research"},{"key":"e_1_3_1_13_2","volume-title":"The 11th International Conference on Learning Representations","author":"Laskin Michael","year":"2022","unstructured":"Michael Laskin, Luyu Wang, Junhyuk Oh, Emilio Parisotto, Stephen Spencer, Richie Steigerwald, D. J. Strouse, Steven Stenberg Hansen, Angelos Filos, Ethan Brooks, et al. 2022. In-context reinforcement learning with algorithm distillation. In The 11th International Conference on Learning Representations."},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.5555\/3091574.3091594"},{"key":"e_1_3_1_15_2","volume-title":"International Conference on Learning Representations","author":"Long Qian","year":"2019","unstructured":"Qian Long, Zihan Zhou, Abhinav Gupta, Fei Fang, Yi Wu, and Xiaolong Wang. 2019. Evolutionary population curriculum for scaling multi-agent reinforcement learning. In International Conference on Learning Representations."},{"key":"e_1_3_1_16_2","first-page":"6379","article-title":"Multi-agent actor-critic for mixed cooperative-competitive environments","volume":"30","author":"Lowe Ryan","year":"2017","unstructured":"Ryan Lowe, Yi. I. Wu, Aviv Tamar, Jean Harb, OpenAI Pieter Abbeel, and Igor Mordatch. 2017. Multi-agent actor-critic for mixed cooperative-competitive environments. Advances in Neural Information Processing Systems 30 (2017), 6379\u20136390.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_17_2","first-page":"1495","volume-title":"Proceedings of the 32nd AAAI Conference on Artificial Intelligence","author":"Mordatch Igor","year":"2018","unstructured":"Igor Mordatch and Pieter Abbeel. 2018. Emergence of grounded compositional language in multi-agent populations. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence, 1495\u20131502."},{"issue":"12","key":"e_1_3_1_18_2","doi-asserted-by":"crossref","first-page":"7341","DOI":"10.1109\/TSMC.2022.3153030","article-title":"Learning-based cooperative multiagent formation control with collision avoidance","volume":"52","author":"Mu Chaoxu","year":"2022","unstructured":"Chaoxu Mu and Jiangwen Peng. 2022. Learning-based cooperative multiagent formation control with collision avoidance. IEEE Transactions on Systems, Man, and Cybernetics: Systems 52, 12 (2022), 7341\u20137352.","journal-title":"IEEE Transactions on Systems, Man, and Cybernetics: Systems"},{"key":"e_1_3_1_19_2","first-page":"25817","volume-title":"International Conference on Machine Learning","author":"Nayak Siddharth","year":"2023","unstructured":"Siddharth Nayak, Kenneth Choi, Wenqi Ding, Sydney Dolan, Karthik Gopalakrishnan, and Hamsa Balakrishnan. 2023. Scalable multi-agent reinforcement learning through intelligent information aggregation. In International Conference on Machine Learning. PMLR, 25817\u201325833."},{"key":"e_1_3_1_20_2","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1016\/j.jai.2024.02.003","article-title":"A survey on multi-agent reinforcement learning and its application","author":"Ning Zepeng","year":"2024","unstructured":"Zepeng Ning and Lihua Xie. 2024. A survey on multi-agent reinforcement learning and its application. Journal of Automation and Intelligence 3, 2 (2024), 73\u201391.","journal-title":"Journal of Automation and Intelligence"},{"key":"e_1_3_1_21_2","volume-title":"Deep Reinforcement Learning Workshop NeurIPS 2022","author":"Qin Rong-Jun","year":"2022","unstructured":"Rong-Jun Qin, Feng Chen, Tonghan Wang, Lei Yuan, Xiaoran Wu, Yipeng Kang, Zongzhang Zhang, Chongjie Zhang, and Yang Yu. 2022. Multi-agent policy transfer via task relationship modeling. In Deep Reinforcement Learning Workshop NeurIPS 2022."},{"key":"e_1_3_1_22_2","first-page":"10199","article-title":"Weighted qmix: Expanding monotonic value function factorisation for deep multi-agent reinforcement learning","volume":"33","author":"Rashid Tabish","year":"2020","unstructured":"Tabish Rashid, Gregory Farquhar, Bei Peng, and Shimon Whiteson. 2020. Weighted qmix: Expanding monotonic value function factorisation for deep multi-agent reinforcement learning. Advances in Neural Information Processing Systems 33 (2020), 10199\u201310210.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_23_2","doi-asserted-by":"crossref","unstructured":"Mikayel Samvelyan Tabish Rashid Christian Schroeder De Witt Gregory Farquhar Nantas Nardelli Tim G. J. Rudner Chia-Man Hung Philip H. S. Torr Jakob Foerster and Shimon Whiteson. 2019. The starcraft multi-agent challenge. arXiv:1902.04043. Retrieved from https:\/\/arxiv.org\/abs\/1902.04043","DOI":"10.65109\/LVZZ5205"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/TETCI.2018.2823329"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.39.10.1095"},{"key":"e_1_3_1_26_2","first-page":"8707","volume-title":"International Conference on Machine Learning","author":"Shen Qianli","year":"2020","unstructured":"Qianli Shen, Yan Li, Haoming Jiang, Zhaoran Wang, and Tuo Zhao. 2020. Deep reinforcement learning with robust and smooth policy. In International Conference on Machine Learning. PMLR, 8707\u20138718."},{"key":"e_1_3_1_27_2","first-page":"776","volume-title":"2021 IEEE International Conference on Real-Time Computing and Robotics (RCAR)","author":"Shi Yinbin","year":"2021","unstructured":"Yinbin Shi, Biao Hu, and Ran Huang. 2021. Task allocation and path planning of many robots with motion uncertainty in a warehouse environment. In 2021 IEEE International Conference on Real-Time Computing and Robotics (RCAR). IEEE, 776\u2013781."},{"key":"e_1_3_1_28_2","first-page":"5998","volume-title":"Advances in Neural Information Processing Systems","volume":"30","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in Neural Information Processing Systems 30 (2017), 5998\u20136008."},{"key":"e_1_3_1_29_2","first-page":"7293","volume-title":"Proceedings of the 34th AAAI Conference on Artificial Intelligence","author":"Wang Weixun","year":"2020","unstructured":"Weixun Wang, Tianpei Yang, Yong Liu, Jianye Hao, Xiaotian Hao, Yujing Hu, Yingfeng Chen, Changjie Fan, and Yang Gao. 2020. From few to more: Large-scale dynamic multiagent curriculum learning. In Proceedings of the 34th AAAI Conference on Artificial Intelligence, 7293\u20137300."},{"key":"e_1_3_1_30_2","first-page":"10905","volume-title":"International Conference on Machine Learning","author":"Wang Xiangjun","year":"2021","unstructured":"Xiangjun Wang, Junxiao Song, Penghui Qi, Peng Peng, Zhenkun Tang, Wei Zhang, Weimin Li, Xiongjun Pi, Jujie He, Chao Gao, et al. 2021. SCC: An efficient deep reinforcement learning agent mastering the game of StarCraft II. In International Conference on Machine Learning. PMLR, 10905\u201310915."},{"key":"e_1_3_1_31_2","first-page":"4649","article-title":"Hierarchical hybrid multi-agent deep reinforcement learning for peer-to-peer energy trading among multiple heterogeneous microgrids","author":"Wu Yuxin","year":"2023","unstructured":"Yuxin Wu, Tianyang Zhao, Haoyuan Yan, Min Liu, and Nian Liu. 2023. Hierarchical hybrid multi-agent deep reinforcement learning for peer-to-peer energy trading among multiple heterogeneous microgrids. IEEE Transactions on Smart Grid 14, 6 (2023), 4649\u20134665.","journal-title":"IEEE Transactions on Smart Grid"},{"key":"e_1_3_1_32_2","first-page":"669","volume-title":"Proceedings of the 35th AAAI Conference on Artificial Intelligence","author":"Xu Bingyu","year":"2021","unstructured":"Bingyu Xu, Yaowei Wang, Zhaozhi Wang, Huizhu Jia, and Zongqing Lu. 2021. Hierarchically and cooperatively learning traffic signal control. In Proceedings of the 35th AAAI Conference on Artificial Intelligence, 669\u2013677."},{"key":"e_1_3_1_33_2","first-page":"5571","volume-title":"International Conference on Machine Learning","author":"Yang Yaodong","year":"2018","unstructured":"Yaodong Yang, Rui Luo, Minne Li, Ming Zhou, Weinan Zhang, and Jun Wang. 2018. Mean field multi-agent reinforcement learning. In International Conference on Machine Learning. PMLR, 5571\u20135580."},{"key":"e_1_3_1_34_2","first-page":"912","article-title":"Automatic curriculum learning for large-scale cooperative multiagent systems","author":"Zhang Tianle","year":"2022","unstructured":"Tianle Zhang, Zhen Liu, Zhiqiang Pu, and Jianqiang Yi. 2022. Automatic curriculum learning for large-scale cooperative multiagent systems. IEEE Transactions on Emerging Topics in Computational Intelligence 7, 3 (2022), 912\u2013930.","journal-title":"IEEE Transactions on Emerging Topics in Computational Intelligence"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.aiopen.2021.01.001"}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3799229","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,27]],"date-time":"2026-04-27T14:11:43Z","timestamp":1777299103000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3799229"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,27]]},"references-count":34,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2026,6,30]]}},"alternative-id":["10.1145\/3799229"],"URL":"https:\/\/doi.org\/10.1145\/3799229","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"value":"2157-6904","type":"print"},{"value":"2157-6912","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,27]]},"assertion":[{"value":"2024-11-27","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-02-04","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-04-27","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}