{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,16]],"date-time":"2025-07-16T13:53:48Z","timestamp":1752674028803,"version":"3.37.3"},"reference-count":25,"publisher":"American Institute of Aeronautics and Astronautics (AIAA)","issue":"7","funder":[{"name":"BAE Systems","award":["2454254"],"award-info":[{"award-number":["2454254"]}]},{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["2454254"],"award-info":[{"award-number":["2454254"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["arc.aiaa.org"],"crossmark-restriction":true},"short-container-title":["Journal of Aerospace Information Systems"],"published-print":{"date-parts":[[2023,7]]},"abstract":"<jats:p> Reinforcement learning tree-based planning methods have been gaining popularity in the last few years due to their success in single-agent domains, where a perfect simulator model is available: for example, Go and chess strategic board games. This paper pretends to extend tree search algorithms to the multiagent setting in a decentralized structure, dealing with scalability issues and exponential growth of computational resources. The [Formula: see text] dynamic tree search combines forward planning and direct temporal-difference updates, outperforming markedly conventional tabular algorithms such as [Formula: see text] learning and state-action-reward-state-action (SARSA). Future state transitions and rewards are predicted with a model built and learned from real interactions between agents and the environment. This paper analyzes the developed algorithm in the hunter\u2013pursuit cooperative game against stochastic and intelligent evaders. The [Formula: see text] dynamic tree search aims to adapt single-agent tree search learning methods to the multiagent boundaries and is demonstrated to be a remarkable advance as compared to conventional temporal-difference techniques. <\/jats:p>","DOI":"10.2514\/1.i011086","type":"journal-article","created":{"date-parts":[[2023,5,3]],"date-time":"2023-05-03T09:50:40Z","timestamp":1683107440000},"page":"418-425","update-policy":"https:\/\/doi.org\/10.2514\/aiaa_crossmarkpolicy","source":"Crossref","is-referenced-by-count":2,"title":["Swarm Intelligence in Cooperative Environments: n-Step Dynamic Tree Search Algorithm Overview"],"prefix":"10.2514","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7916-9383","authenticated-orcid":false,"given":"Marc","family":"Espin\u00f3s Longa","sequence":"first","affiliation":[{"name":"Cranfield University, Cranfield, England MK43 0AL United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4490-8358","authenticated-orcid":false,"given":"Antonios","family":"Tsourdos","sequence":"additional","affiliation":[{"name":"Cranfield University, Cranfield, England MK43 0AL United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3966-7633","authenticated-orcid":false,"given":"Gokhan","family":"Inalhan","sequence":"additional","affiliation":[{"name":"Cranfield University, Cranfield, England MK43 0AL United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1387","reference":[{"key":"r1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-75171-7_2"},{"key":"r2","doi-asserted-by":"publisher","DOI":"10.1109\/LGRS.2015.2478256"},{"key":"r3","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0409746102"},{"key":"r4","doi-asserted-by":"publisher","DOI":"10.1016\/j.proeng.2012.06.357"},{"key":"r5","doi-asserted-by":"publisher","DOI":"10.1007\/s10458-005-2631-2"},{"key":"r7","first-page":"249","volume-title":"Parallel Problem Solving from Nature","author":"Potter M. A.","year":"1994"},{"key":"r8","first-page":"467","volume-title":"Parallel Problem Solving from Nature","author":"Ficici S. G.","year":"2000"},{"key":"r11","first-page":"871","volume-title":"Proceedings of the 16th International Conference on Neural Information Processing Systems (NIPS\u201903)","author":"Tesauro G.","year":"2003"},{"key":"r12","doi-asserted-by":"publisher","DOI":"10.1007\/BF00992698"},{"key":"r13","doi-asserted-by":"publisher","DOI":"10.1023\/A:1007570708568"},{"key":"r14","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2019.2919614"},{"key":"r17","unstructured":"SuttonR. S.BartoA. G., Reinforcement Learning: An Introduction, MIT Press, Cambridge, MA, 2018, pp.\u00a0185\u2013188, 152, 190."},{"key":"r18","first-page":"1146","volume-title":"Proceedings of the 34th International Conference on Machine Learning","author":"Foerster J.","year":"2017"},{"key":"r21","doi-asserted-by":"publisher","DOI":"10.1038\/nature24270"},{"key":"r22","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-020-03051-4"},{"key":"r24","doi-asserted-by":"publisher","DOI":"10.1049\/joe.2019.1200"},{"key":"r26","first-page":"487","volume-title":"Readings in Agents","author":"Tan M.","year":"1997"},{"key":"r27","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-92910-9_48"},{"key":"r28","doi-asserted-by":"publisher","DOI":"10.1007\/s10846-022-01690-5"},{"key":"r29","doi-asserted-by":"publisher","DOI":"10.2514\/6.2022-1839"},{"key":"r31","doi-asserted-by":"publisher","DOI":"10.1007\/s40595-015-0045-x"},{"key":"r34","doi-asserted-by":"publisher","DOI":"10.1016\/S0921-8890(03)00040-X"},{"key":"r35","doi-asserted-by":"publisher","DOI":"10.3390\/robotics2030149"},{"key":"r36","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2020.06.031"},{"key":"r37","doi-asserted-by":"publisher","DOI":"10.1155\/2010\/659432"}],"container-title":["Journal of Aerospace Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/arc.aiaa.org\/doi\/pdf\/10.2514\/1.I011086","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,29]],"date-time":"2023-06-29T13:53:43Z","timestamp":1688046823000},"score":1,"resource":{"primary":{"URL":"https:\/\/arc.aiaa.org\/doi\/10.2514\/1.I011086"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7]]},"references-count":25,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2023,7]]}},"alternative-id":["10.2514\/1.I011086"],"URL":"https:\/\/doi.org\/10.2514\/1.i011086","relation":{},"ISSN":["1940-3151","2327-3097"],"issn-type":[{"type":"print","value":"1940-3151"},{"type":"electronic","value":"2327-3097"}],"subject":[],"published":{"date-parts":[[2023,7]]},"assertion":[{"value":"2021-12-16","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-02-17","order":1,"name":"revised","label":"Revised","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-03-24","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-05-03","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}