{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,8]],"date-time":"2026-01-08T16:58:45Z","timestamp":1767891525772,"version":"3.49.0"},"reference-count":48,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,11,12]],"date-time":"2024-11-12T00:00:00Z","timestamp":1731369600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,11,12]],"date-time":"2024-11-12T00:00:00Z","timestamp":1731369600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62076203"],"award-info":[{"award-number":["62076203"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Complex Intell. Syst."],"published-print":{"date-parts":[[2025,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>Swarm systems consist of a large number of interacting individuals, which exhibit complex behavior despite having simple interaction rules. However, crafting individual motion policies that can manifest desired collective behaviors poses a significant challenge due to the intricate relationship between individual policies and swarm dynamics. This paper addresses this issue by proposing an imitation learning method, which derives individual policies from collective behavior data. The approach leverages an adversarial imitation learning framework, with a deep attention network serving as the individual policy network. Our method successfully imitates three distinct collective behaviors. Utilizing the ease of analysis provided by the deep attention network, we have verified that the individual policies underlying a certain collective behavior are not unique. Additionally, we have analyzed the different individual policies discovered. Lastly, we validate the applicability of the proposed method in designing policies for swarm robots through practical implementation on swarm robots.<\/jats:p>","DOI":"10.1007\/s40747-024-01662-2","type":"journal-article","created":{"date-parts":[[2024,11,12]],"date-time":"2024-11-12T04:58:47Z","timestamp":1731387527000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Adversarial imitation learning with deep attention network for swarm systems"],"prefix":"10.1007","volume":"11","author":[{"given":"Yapei","family":"Wu","sequence":"first","affiliation":[]},{"given":"Tao","family":"Wang","sequence":"additional","affiliation":[]},{"given":"Tong","family":"Liu","sequence":"additional","affiliation":[]},{"given":"Zhicheng","family":"Zheng","sequence":"additional","affiliation":[]},{"given":"Demin","family":"Xu","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8483-5432","authenticated-orcid":false,"given":"Xingguang","family":"Peng","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,11,12]]},"reference":[{"issue":"6","key":"1662_CR1","doi-asserted-by":"publisher","first-page":"693","DOI":"10.1098\/rsfs.2012.0033","volume":"2","author":"U Lopez","year":"2012","unstructured":"Lopez U, Gautrais J, Couzin ID, Theraulaz G (2012) From behavioural analyses to models of collective motion in fish schools. Interface focus 2(6):693\u2013707","journal-title":"Interface focus"},{"issue":"15","key":"1662_CR2","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.98.158102","volume":"98","author":"A Sokolov","year":"2007","unstructured":"Sokolov A, Aranson IS, Kessler JO, Goldstein RE (2007) Concentration dependence of the collective dynamics of swimming bacteria. Phys Rev Lett 98(15):158102","journal-title":"Phys Rev Lett"},{"key":"1662_CR3","doi-asserted-by":"crossref","unstructured":"Zou H, Su H, Song S, Zhu J (2018) Understanding human behaviors in crowds by imitating the decision-making process. In: Proceedings of the AAAI conference on artificial intelligence, vol 32","DOI":"10.1609\/aaai.v32i1.12316"},{"key":"1662_CR4","doi-asserted-by":"publisher","first-page":"205","DOI":"10.1007\/BF01298460","volume":"8","author":"NJ Vriend","year":"1995","unstructured":"Vriend NJ (1995) Self-organization of markets: an example of a computational approach. Comput Econ 8:205\u2013231","journal-title":"Comput Econ"},{"issue":"6","key":"1662_CR5","doi-asserted-by":"publisher","first-page":"385","DOI":"10.1080\/00963402.2018.1533209","volume":"74","author":"P Scharre","year":"2018","unstructured":"Scharre P (2018) How swarming will change warfare. Bull Atom Sci 74(6):385\u2013389","journal-title":"Bull Atom Sci"},{"key":"1662_CR6","doi-asserted-by":"publisher","DOI":"10.1016\/j.compag.2021.106608","volume":"193","author":"D Albiero","year":"2022","unstructured":"Albiero D, Garcia AP, Umezu CK, de Paulo RL (2022) Swarm robots in mechanized agricultural operations: a review about challenges for research. Comput Electron Agric 193:106608","journal-title":"Comput Electron Agric"},{"key":"1662_CR7","doi-asserted-by":"crossref","unstructured":"Duarte M, Gomes J, Costa V, Rodrigues T, Silva F, Lobo V, Marques MM, Oliveira SM, Christensen AL (2016), Application of swarm robotics systems to marine environmental monitoring. In: OCEANS 2016-Shanghai, IEEE, pp 1\u20138","DOI":"10.1109\/OCEANSAP.2016.7485429"},{"key":"1662_CR8","doi-asserted-by":"crossref","unstructured":"Hsu A, Wong-Foy A, McCoy B, Cowan C, Marlow J, Chavez B, Kobayashi T, Shockey D, Pelrine R (2016) Application of micro-robots for building carbon fiber trusses. In: International conference on manipulation, automation and robotics at small scales (MARSS). IEEE 2016, pp 1\u20136","DOI":"10.1109\/MARSS.2016.7561729"},{"issue":"1","key":"1662_CR9","doi-asserted-by":"publisher","first-page":"8601","DOI":"10.1038\/s41598-022-12525-6","volume":"12","author":"Z Zheng","year":"2022","unstructured":"Zheng Z, Lei X, Peng X (2022) Selective interaction and its effect on collective motion. Sci Rep 12(1):8601","journal-title":"Sci Rep"},{"issue":"54","key":"1662_CR10","first-page":"1","volume":"20","author":"M H\u00fcttenrauch","year":"2019","unstructured":"H\u00fcttenrauch M, \u0160o\u0161i\u0107 A, Neumann G (2019) Deep reinforcement learning for swarm systems. J Mach Learn Res 20(54):1\u201331","journal-title":"J Mach Learn Res"},{"key":"1662_CR11","doi-asserted-by":"publisher","DOI":"10.1016\/j.swevo.2024.101475","volume":"85","author":"T Wang","year":"2024","unstructured":"Wang T, Peng X, Wang T, Liu T, Xu D (2024) Automated design of action advising trigger conditions for multiagent reinforcement learning: a genetic programming-based approach. Swarm Evol Comput 85:101475","journal-title":"Swarm Evol Comput"},{"key":"1662_CR12","doi-asserted-by":"crossref","unstructured":"Bezcioglu M. B, Lennox B, Arvin F (2021) Self-organised swarm flocking with deep reinforcement learning. In: 2021 7th International conference on automation, robotics and applications (ICARA), IEEE, pp 226\u2013230","DOI":"10.1109\/ICARA51699.2021.9376509"},{"issue":"1","key":"1662_CR13","doi-asserted-by":"publisher","first-page":"4345","DOI":"10.1038\/s41467-021-24642-3","volume":"12","author":"K Hasselmann","year":"2021","unstructured":"Hasselmann K, Ligot A, Ruddick J, Birattari M (2021) Empirical assessment and comparison of neuro-evolutionary methods for the automatic off-line design of robot swarms. Nat Commun 12(1):4345","journal-title":"Nat Commun"},{"key":"1662_CR14","doi-asserted-by":"publisher","DOI":"10.1016\/j.robot.2019.103293","volume":"122","author":"TK Kaiser","year":"2019","unstructured":"Kaiser TK, Hamann H (2019) Engineered self-organization for resilient robot self-assembly with minimal surprise. Robot Auton Syst 122:103293","journal-title":"Robot Auton Syst"},{"key":"1662_CR15","unstructured":"Pomerleau DA (1988) Alvinn: an autonomous land vehicle in a neural network. In: Proceedings of the advances in neural information processing systems, Vienna, pp 1\u201320"},{"key":"1662_CR16","unstructured":"Ng AY, Russell SJ (2000) Algorithms for inverse reinforcement learning. In: Proceedings of the seventeenth international conference on machine learning, ICML \u201900, Morgan Kaufmann Publishers Inc., San Francisco, pp 663\u2013670"},{"key":"1662_CR17","doi-asserted-by":"crossref","unstructured":"Zhou S, Phielipp MJ, Sefair JA, Walker SI, Amor HB (2019), Clone swarms: learning to predict and control multi-robot systems by imitation. In: 2019 IEEE\/RSJ international conference on intelligent robots and systems (IROS), IEEE, pp 4092\u20134099","DOI":"10.1109\/IROS40897.2019.8967824"},{"issue":"19","key":"1662_CR18","doi-asserted-by":"publisher","first-page":"9055","DOI":"10.3390\/app11199055","volume":"11","author":"C Guo","year":"2021","unstructured":"Guo C, Zhu P, Zhou Z, Lang L, Zeng Z, Lu H (2021) Imitation learning with graph neural networks for improving swarm robustness under restricted communications. Appl Sci 11(19):9055","journal-title":"Appl Sci"},{"issue":"212","key":"1662_CR19","doi-asserted-by":"publisher","first-page":"20230630","DOI":"10.1098\/rsif.2023.0630","volume":"21","author":"V Papaspyros","year":"2024","unstructured":"Papaspyros V, Escobedo R, Alahi A, Theraulaz G, Sire C, Mondada F (2024) Predicting the long-term collective behaviour of fish pairs with deep learning. J R Soc Interface 21(212):20230630","journal-title":"J R Soc Interface"},{"key":"1662_CR20","unstructured":"Ross S, Gordon G, Bagnell D (2011) A reduction of imitation learning and structured prediction to no-regret online learning. In: Proceedings of the fourteenth international conference on artificial intelligence and statistics, JMLR Workshop and Conference Proceedings, pp 627\u2013635"},{"key":"1662_CR21","unstructured":"\u0160o\u0161i\u0107 A, KhudaBukhsh WR, Zoubir AM, Koeppl H (2017) Inverse reinforcement learning in swarm systems. In: Proceedings of the 16th conference on autonomous agents and multiagent systems, pp 1413\u20131421"},{"key":"1662_CR22","doi-asserted-by":"crossref","unstructured":"Yu X, Wu W, Feng P, Tian Y (2021) Swarm inverse reinforcement learning for biological systems. In: 2021 IEEE International conference on bioinformatics and biomedicine (BIBM), IEEE, pp 274\u2013279","DOI":"10.1109\/BIBM52615.2021.9669656"},{"key":"1662_CR23","unstructured":"Pinsler R, Maag M, Arenz O, Neumann G (2018) Inverse reinforcement learning of bird flocking behavior. In: ICRA swarms workshop"},{"issue":"9","key":"1662_CR24","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1007354","volume":"15","author":"FJ Heras","year":"2019","unstructured":"Heras FJ, Romero-Ferrero F, Hinz RC, de Polavieja GG (2019) Deep attention networks reveal the rules of collective motion in zebrafish. PLoS Comput Biol 15(9):e1007354","journal-title":"PLoS Comput Biol"},{"key":"1662_CR25","first-page":"1","volume":"29","author":"J Ho","year":"2016","unstructured":"Ho J, Ermon S (2016) Generative adversarial imitation learning. Proc Adv Neural Inf Process Syst 29:1\u201320","journal-title":"Proc Adv Neural Inf Process Syst"},{"key":"1662_CR26","unstructured":"Tolstaya E, Gama F, Paulos J, Pappas G, Kumar V, Ribeiro A (2020) Learning decentralized controllers for robot swarms with graph neural networks. In: Conference on robot learning, PMLR, pp 671\u2013682"},{"issue":"4","key":"1662_CR27","doi-asserted-by":"publisher","first-page":"1034","DOI":"10.1109\/TSP.2018.2887403","volume":"67","author":"F Gama","year":"2018","unstructured":"Gama F, Marques AG, Leus G, Ribeiro A (2018) Convolutional neural network architectures for signals supported on graphs. IEEE Trans Signal Process 67(4):1034\u20131049","journal-title":"IEEE Trans Signal Process"},{"issue":"4","key":"1662_CR28","doi-asserted-by":"publisher","first-page":"4523","DOI":"10.1109\/LRA.2019.2935377","volume":"4","author":"F Schilling","year":"2019","unstructured":"Schilling F, Lecoeur J, Schiano F, Floreano D (2019) Learning vision-based flight in drone swarms by imitation. IEEE Robot Autom Lett 4(4):4523\u20134530","journal-title":"IEEE Robot Autom Lett"},{"issue":"6","key":"1662_CR29","doi-asserted-by":"publisher","first-page":"1226","DOI":"10.1103\/PhysRevLett.75.1226","volume":"75","author":"T Vicsek","year":"1995","unstructured":"Vicsek T, Czir\u00f3k A, Ben-Jacob E, Cohen I, Shochet O (1995) Novel type of phase transition in a system of self-driven particles. Phys Rev Lett 75(6):1226","journal-title":"Phys Rev Lett"},{"issue":"1874","key":"1662_CR30","doi-asserted-by":"publisher","first-page":"20220073","DOI":"10.1098\/rstb.2022.0073","volume":"378","author":"F Romero-Ferrero","year":"2023","unstructured":"Romero-Ferrero F, Heras FJ, Rance D, de Polavieja GG (2023) A study of transfer of information in animal collectives using deep learning tools. Philos Trans R Soc B 378(1874):20220073","journal-title":"Philos Trans R Soc B"},{"issue":"4","key":"1662_CR31","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1009293","volume":"18","author":"J LaChance","year":"2022","unstructured":"LaChance J, Suh K, Clausen J, Cohen DJ (2022) Learning the rules of collective cell migration using deep attention networks. PLoS Comput Biol 18(4):e1009293","journal-title":"PLoS Comput Biol"},{"key":"1662_CR32","doi-asserted-by":"publisher","first-page":"200","DOI":"10.3389\/fphy.2020.00200","volume":"8","author":"T Costa","year":"2020","unstructured":"Costa T, Laan A, Heras FJ, De Polavieja GG (2020) Automated discovery of local rules for desired collective-level behavior through reinforcement learning. Front Phys 8:200","journal-title":"Front Phys"},{"key":"1662_CR33","unstructured":"Salimans T, Ho J, Chen X, Sidor S, Sutskever I (2017) Evolution strategies as a scalable alternative to reinforcement learning. arXiv preprint arXiv:1703.03864"},{"issue":"4","key":"1662_CR34","doi-asserted-by":"publisher","first-page":"1431","DOI":"10.1007\/s10994-022-06144-5","volume":"111","author":"L Blond\u00e9","year":"2022","unstructured":"Blond\u00e9 L, Strasser P, Kalousis A (2022) Lipschitzness is all you need to tame off-policy generative adversarial imitation learning. Mach Learn 111(4):1431\u20131521","journal-title":"Mach Learn"},{"key":"1662_CR35","unstructured":"Jena R, Agrawal S, Sycara K (2020) Addressing reward bias in adversarial imitation learning with neutral reward functions. arXiv preprint arXiv:2009.09467"},{"key":"1662_CR36","unstructured":"Orsini M, Raichuk A, Hussenot L, Vincent D, Dadashi R, Girgin S, Geist M, Bachem O, Pietquin O, Andrychowicz M (2021) What matters for adversarial imitation learning? In: Proceedings of the advances in neural information processing systems, vol 34, Virtual Conference, pp 14656\u201314668"},{"key":"1662_CR37","doi-asserted-by":"crossref","unstructured":"Zhang M, Wang Y, Ma X, Xia L, Yang J, Li Z, Li X (2020) Wasserstein distance guided adversarial imitation learning with reward shape exploration. In: Proceedings of the IEEE 9th data driven control and learning systems conference, IEEE, Liuzhou, pp 1165\u20131170","DOI":"10.1109\/DDCLS49620.2020.9275169"},{"key":"1662_CR38","unstructured":"Shin W, Kang H, Hong S (2019) Mature gail: imitation learning for low-level and high-dimensional input using global encoder and cost transformation. arXiv preprint arXiv:1909.03200"},{"key":"1662_CR39","doi-asserted-by":"publisher","first-page":"365","DOI":"10.1016\/j.neucom.2021.06.053","volume":"457","author":"G Zuo","year":"2021","unstructured":"Zuo G, Zhao Q, Huang S, Li J, Gong D (2021) Adversarial imitation learning with mixed demonstrations from multiple demonstrators. Neurocomputing 457:365\u2013376","journal-title":"Neurocomputing"},{"key":"1662_CR40","doi-asserted-by":"publisher","DOI":"10.1016\/j.swevo.2021.100974","volume":"68","author":"P Yang","year":"2022","unstructured":"Yang P, Zhang H, Yu Y, Li M, Tang K (2022) Evolutionary reinforcement learning via cooperative coevolutionary negatively correlated search. Swarm Evol Comput 68:100974","journal-title":"Swarm Evol Comput"},{"key":"1662_CR41","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2021.107951","volume":"113","author":"J Liang","year":"2021","unstructured":"Liang J, Chen G, Qu B, Yue C, Yu K, Qiao K (2021) Niche-based cooperative co-evolutionary ensemble neural network for classification. Appl Soft Comput 113:107951","journal-title":"Appl Soft Comput"},{"key":"1662_CR42","doi-asserted-by":"crossref","unstructured":"Peng F, Liu S, Lu N, Tang K (2022) Training quantized deep neural networks via cooperative coevolution. In: International conference on sensing and imaging. Springer, pp 81\u201393","DOI":"10.1007\/978-3-031-09726-3_8"},{"key":"1662_CR43","unstructured":"Huning A (1976) Evolutionsstrategie. optimierung technischer systeme nach prinzipien der biologischen evolution"},{"key":"1662_CR44","doi-asserted-by":"crossref","unstructured":"Majid AY, Saaybi S, Francois-Lavet V, Prasad RV, Verhoeven C (2023) Deep reinforcement learning versus evolution strategies: a comparative survey. In: IEEE transactions on neural networks and learning systems","DOI":"10.1109\/TNNLS.2023.3264540"},{"key":"1662_CR45","doi-asserted-by":"publisher","first-page":"229","DOI":"10.1007\/BF00992696","volume":"8","author":"RJ Williams","year":"1992","unstructured":"Williams RJ (1992) Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach Learn 8:229\u2013256","journal-title":"Mach Learn"},{"key":"1662_CR46","doi-asserted-by":"crossref","unstructured":"Brockhoff D, Auger A, Hansen N, Arnold D. V, Hohm T (2010) Mirrored sampling and sequential selection for evolution strategies. In: Parallel problem solving from nature, PPSN XI: 11th international conference, Krak\u00f3w, Poland, September 11\u201315, 2010, Proceedings, Part I 11, Springer, pp 11\u201321","DOI":"10.1007\/978-3-642-15844-5_2"},{"key":"1662_CR47","doi-asserted-by":"crossref","unstructured":"Chrabaszcz P, Loshchilov I, Hutter F (2018) Back to basics: benchmarking canonical evolution strategies for playing atari. arXiv preprint arXiv:1802.08842","DOI":"10.24963\/ijcai.2018\/197"},{"key":"1662_CR48","doi-asserted-by":"crossref","unstructured":"Couzin ID, Krause J, James R, Ruxton GD, Franks NR (2002) Collective memory and spatial sorting in animal groups. J Theor Biol 218(1):1\u201311","DOI":"10.1006\/jtbi.2002.3065"}],"container-title":["Complex &amp; Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-024-01662-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s40747-024-01662-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-024-01662-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,30]],"date-time":"2025-01-30T20:16:00Z","timestamp":1738268160000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s40747-024-01662-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,12]]},"references-count":48,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,1]]}},"alternative-id":["1662"],"URL":"https:\/\/doi.org\/10.1007\/s40747-024-01662-2","relation":{},"ISSN":["2199-4536","2198-6053"],"issn-type":[{"value":"2199-4536","type":"print"},{"value":"2198-6053","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,12]]},"assertion":[{"value":"23 June 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 October 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 November 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"On behalf of all authors, the corresponding author states that there is no Conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}],"article-number":"26"}}