{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,25]],"date-time":"2025-10-25T01:53:24Z","timestamp":1761357204063,"version":"build-2065373602"},"reference-count":39,"publisher":"Springer Science and Business Media LLC","issue":"11","license":[{"start":{"date-parts":[[2025,8,30]],"date-time":"2025-08-30T00:00:00Z","timestamp":1756512000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,8,30]],"date-time":"2025-08-30T00:00:00Z","timestamp":1756512000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62176225 and 62276168"],"award-info":[{"award-number":["62176225 and 62276168"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003392","name":"Natural Science Foundation of Fujian Province","doi-asserted-by":"crossref","award":["2022J05176"],"award-info":[{"award-number":["2022J05176"]}],"id":[{"id":"10.13039\/501100003392","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["EP\/S011609\/1"],"award-info":[{"award-number":["EP\/S011609\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Artif Intell Rev"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Interactive dynamic influence diagrams\u00a0(I-DIDs) are a general decision framework for a subject agent who interacts with other agents\u00a0(of either collaborative or competitive) in a common environment with partial observability. The subject agent aims to optimize its decision-making (response strategy) while other agents concurrently adapt their behaviors over time. The I-DID model has faced a long-term challenge when other agents exhibit unknown behaviors that go beyond what the subject agent has planned for prior to their interactions. This is because the subject agent does not hold the capability of modeling unknown behaviours of other agents in traditional I-DID techniques. In this article, we adapt two different swarm intelligence\u00a0(SI) techniques to develop new behaviours for other agents in I-DIDs. The SI-based algorithms have the strength of generating a collective set of behaviours that could potentially contain various types of agents\u2019 behaviours. We theoretically analyze how the two algorithms impact the subject agent\u2019s decision quality, and empirically demonstrate the algorithm performance in two commonly used problem domains.<\/jats:p>","DOI":"10.1007\/s10462-025-11355-y","type":"journal-article","created":{"date-parts":[[2025,8,30]],"date-time":"2025-08-30T03:35:03Z","timestamp":1756524903000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Improved decisions for unknown behaviours in interactive dynamic influence diagrams"],"prefix":"10.1007","volume":"58","author":[{"given":"Yinghui","family":"Pan","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mengen","family":"Zhou","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Biyang","family":"Ma","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yifeng","family":"Zeng","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yew-soon","family":"Ong","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Guoquan","family":"Liu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2025,8,30]]},"reference":[{"issue":"6","key":"11355_CR1","doi-asserted-by":"publisher","first-page":"5567","DOI":"10.1109\/LRA.2024.3396097","volume":"9","author":"X Cao","year":"2024","unstructured":"Cao X, Li M, Tao Y, Lu P (2024) Hma-sar: multi-agent search and rescue for unknown located dynamic targets in completely unknown environments. IEEE Robot Autom Lett 9(6):5567\u20135574","journal-title":"IEEE Robot Autom Lett"},{"issue":"11","key":"11355_CR2","doi-asserted-by":"publisher","first-page":"936","DOI":"10.3390\/jmse8110936","volume":"8","author":"J Xie","year":"2020","unstructured":"Xie J, Zhou R, Luo J, Peng Y, Liu Y, Xie S, Pu H (2020) Hybrid partition-based patrolling scheme for maritime area patrol with multiple cooperative unmanned surface vehicles. J Marine Sci Eng 8(11):936","journal-title":"J Marine Sci Eng"},{"issue":"2","key":"11355_CR3","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1002\/adc2.27","volume":"2","author":"X Wang","year":"2020","unstructured":"Wang X, Xuan S, Ke L (2020) Cooperatively pursuing a target unmanned aerial vehicle by multiple unmanned aerial vehicles based on multiagent reinforcement learning. Adv Control Appl 2(2):27","journal-title":"Adv Control Appl"},{"issue":"2","key":"11355_CR4","doi-asserted-by":"publisher","first-page":"190","DOI":"10.1007\/s10458-007-9026-5","volume":"17","author":"S Seuken","year":"2008","unstructured":"Seuken S, Zilberstein S (2008) Formal models and algorithms for decentralized decision making under uncertainty. Auton Agents Multi-Agent Syst 17(2):190\u2013250","journal-title":"Auton Agents Multi-Agent Syst"},{"key":"11355_CR5","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1613\/jair.1579","volume":"24","author":"PJ Gmytrasiewicz","year":"2005","unstructured":"Gmytrasiewicz PJ (2005) A framework for sequential planning in multiagent settings. J Artif Intell Res 24:49\u201379","journal-title":"J Artif Intell Res"},{"issue":"3","key":"11355_CR6","doi-asserted-by":"publisher","first-page":"376","DOI":"10.1007\/s10458-008-9064-7","volume":"18","author":"P Doshi","year":"2009","unstructured":"Doshi P, Zeng Y, Chen Q (2009) Graphical models for interactive pomdps: representations and solutions. Auton Agents Multi-Agent Syst 18(3):376\u2013416","journal-title":"Auton Agents Multi-Agent Syst"},{"key":"11355_CR7","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1613\/jair.3461","volume":"43","author":"Y Zeng","year":"2012","unstructured":"Zeng Y, Doshi P (2012) Exploiting model equivalences for solving interactive dynamic influence diagrams. J Artif Intell Res 43:211\u2013255","journal-title":"J Artif Intell Res"},{"key":"11355_CR8","doi-asserted-by":"publisher","first-page":"103202","DOI":"10.1016\/j.artint.2019.103202","volume":"279","author":"P Doshi","year":"2020","unstructured":"Doshi P, Gmytrasiewicz PJ, Durfee EH (2020) Recursively modeling other agents for decision making: a research perspective. Artif Intell 279:103202","journal-title":"Artif Intell"},{"key":"11355_CR9","unstructured":"Pynadath V, Marsella S (2007) Minimal mental models. In: Proceedings of the Twenty-Second Conference on Artificial Intelligence (AAAI\u201907), pp. 1038\u20131044"},{"key":"11355_CR10","doi-asserted-by":"publisher","first-page":"103292","DOI":"10.1016\/j.artint.2020.103292","volume":"285","author":"SV Albrecht","year":"2020","unstructured":"Albrecht SV, Stone P, Wellman MP (2020) Special issue on autonomous agents modelling other agents: guest editorial. Artif Intell 285:103292","journal-title":"Artif Intell"},{"key":"11355_CR11","doi-asserted-by":"publisher","first-page":"495","DOI":"10.1016\/j.ins.2021.09.039","volume":"582","author":"Y Pan","year":"2022","unstructured":"Pan Y, Ma B, Tang J, Zeng Y (2022) Behavioral model summarisation for other agents under uncertainty. Inf Sci 582:495\u2013508","journal-title":"Inf Sci"},{"issue":"9","key":"11355_CR12","doi-asserted-by":"publisher","first-page":"2431","DOI":"10.1007\/s10115-021-01600-5","volume":"63","author":"Y Pan","year":"2021","unstructured":"Pan Y, Tang J, Ma B, Zeng Y, Ming Z (2021) Towards data-driven solutions to interactive dynamic influence diagrams. Knowl Inf Syst 63(9):2431\u20132453","journal-title":"Knowl Inf Syst"},{"issue":"4","key":"11355_CR13","doi-asserted-by":"publisher","first-page":"1276","DOI":"10.1109\/TETCI.2022.3229308","volume":"7","author":"Y Pan","year":"2023","unstructured":"Pan Y, Ma B, Zeng Y, Tang J, Zeng B, Ming Z (2023) An evolutionary framework for modelling unknown behaviours of other agents. IEEE Trans Emerg Top Comput Intell 7(4):1276\u20131289","journal-title":"IEEE Trans Emerg Top Comput Intell"},{"issue":"1","key":"11355_CR14","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1007\/s12293-021-00343-8","volume":"14","author":"Y Zeng","year":"2022","unstructured":"Zeng Y, Ran Q, Ma B, Pan Y (2022) Modelling other agents through evolutionary behaviours. Meme Comput 14(1):19\u201330","journal-title":"Meme Comput"},{"issue":"2","key":"11355_CR15","doi-asserted-by":"publisher","first-page":"192","DOI":"10.1109\/TEVC.2000.850661","volume":"4","author":"AE Smith","year":"2020","unstructured":"Smith AE (2020) Swarm intelligence: from natural to artificial systems. IEEE Trans Evol Comput 4(2):192\u2013193","journal-title":"IEEE Trans Evol Comput"},{"key":"11355_CR16","doi-asserted-by":"crossref","unstructured":"Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of International Conference on Neural Networks (ICNN\u201995), pp. 1942\u20131948","DOI":"10.1109\/ICNN.1995.488968"},{"issue":"4","key":"11355_CR17","doi-asserted-by":"publisher","first-page":"28","DOI":"10.1109\/MCI.2006.329691","volume":"1","author":"M Dorigo","year":"2006","unstructured":"Dorigo M, Birattari M, Stutzle T (2006) Ant colony optimization. IEEE Comput Intell Mag 1(4):28\u201339","journal-title":"IEEE Comput Intell Mag"},{"issue":"2","key":"11355_CR18","doi-asserted-by":"publisher","first-page":"365","DOI":"10.1109\/21.52548","volume":"20","author":"A Tatman","year":"1990","unstructured":"Tatman A, Shachter RD (1990) Dynamic programming and influence diagrams. IEEE Trans Syst Man Cybern 20(2):365\u2013379","journal-title":"IEEE Trans Syst Man Cybern"},{"key":"11355_CR19","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1016\/S0004-3702(98)00023-X","volume":"101","author":"LP Kaelbling","year":"1998","unstructured":"Kaelbling LP, Littman ML, Cassandra AR (1998) Planning and acting in partially observable stochastic domains. Artif Intell 101:99\u2013134","journal-title":"Artif Intell"},{"key":"11355_CR20","unstructured":"Conroy R, Zeng Y, Cavazza M, Chen Y (2015) Learning behaviors in agents systems with interactive dynamic influence diagrams. In: Proceedings of the 24th International Conference on Artificial Intelligence (IJCAI\u201915), pp. 39\u201345"},{"issue":"2","key":"11355_CR21","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1017\/S0269888914000289","volume":"30","author":"A Byrski","year":"2015","unstructured":"Byrski A, Dre\u017cewski R, Siwik L, Kisiel-Dorohinicki M (2015) Evolutionary multi-agent systems. Knowl Eng Rev 30(2):171\u2013186","journal-title":"Knowl Eng Rev"},{"issue":"1","key":"11355_CR22","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1007\/s10458-012-9204-y","volume":"27","author":"B Eker","year":"2013","unstructured":"Eker B, Ak\u0131n HL (2013) Solving decentralized pomdp problems using genetic algorithms. Auton Agent Multi-Agent Syst 27(1):161\u2013196","journal-title":"Auton Agent Multi-Agent Syst"},{"key":"11355_CR23","doi-asserted-by":"crossref","unstructured":"Klijn D, Eiben AE (2021) A coevolutionary approach to deep multi-agent reinforcement learning. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion (GECCO\u201921), pp. 283\u2013284","DOI":"10.1145\/3449726.3459576"},{"issue":"1","key":"11355_CR24","doi-asserted-by":"publisher","first-page":"102","DOI":"10.1109\/TEVC.2016.2577593","volume":"21","author":"Y Zeng","year":"2017","unstructured":"Zeng Y, Chen X, Ong Y-S, Tang J, Xiang Y (2017) Structured memetic automation for online human-like social behavior learning. IEEE Trans Evol Comput 21(1):102\u2013115","journal-title":"IEEE Trans Evol Comput"},{"issue":"2","key":"11355_CR25","doi-asserted-by":"publisher","first-page":"531","DOI":"10.1109\/TEVC.2023.3268076","volume":"28","author":"Y Hou","year":"2023","unstructured":"Hou Y, Sun M, Zeng Y, Ong Y-S, Jin Y, Ge H, Zhang Q (2023) A multi-agent cooperative learning system with evolution of social roles. IEEE Trans Evol Comput 28(2):531\u2013543","journal-title":"IEEE Trans Evol Comput"},{"issue":"1","key":"11355_CR26","doi-asserted-by":"publisher","first-page":"402","DOI":"10.1016\/j.jnca.2012.04.018","volume":"36","author":"D Rosaci","year":"2013","unstructured":"Rosaci D, Sarn\u00c3 GML (2013) Cloning mechanisms to improve agent performances. J Netw Comput Appl 36(1):402\u2013408","journal-title":"J Netw Comput Appl"},{"key":"11355_CR27","doi-asserted-by":"publisher","first-page":"110062","DOI":"10.1016\/j.asoc.2023.110062","volume":"136","author":"W Li","year":"2023","unstructured":"Li W, Hu Y, Jiang C, Wu S, Bai Q, Lai E (2023) Abem: an adaptive agent-based evolutionary approach for influence maximization in dynamic social networks. Appl Soft Comput 136:110062","journal-title":"Appl Soft Comput"},{"key":"11355_CR28","unstructured":"Marcus GF (2020) The next decade in ai: Four steps towards robust artificial intelligence. ArXiv abs\/2002.06177 [cs.AI]"},{"key":"11355_CR29","doi-asserted-by":"publisher","first-page":"3218","DOI":"10.1109\/LRA.2022.3145517","volume":"7","author":"S Rabiee","year":"2021","unstructured":"Rabiee S, Basich C, Wray KH, Zilberstein S, Biswas J (2021) Competence-aware path planning via introspective perception. IEEE Robot Autom Lett 7:3218\u20133225","journal-title":"IEEE Robot Autom Lett"},{"key":"11355_CR30","unstructured":"Papoudakis G, Albrecht SV(2020) Variational autoencoders for opponent modeling in multi-agent systems. In: Proceedings of the 2019 International Conference on Autonomous Agents and Multiagent System (AAMAS\u201919), pp. 1\u20138"},{"key":"11355_CR31","unstructured":"Stephen C (2020) The achilles heel hypothesis: Pitfalls for ai systems via decision theoretic adversaries. arXiv:2010.05418"},{"issue":"6","key":"11355_CR32","doi-asserted-by":"publisher","first-page":"103371","DOI":"10.1016\/j.im.2020.103371","volume":"58","author":"L Wang","year":"2021","unstructured":"Wang L, Schuetz CG, Cai D (2021) Choosing response strategies in social media crisis communication: an evolutionary game theory perspective. Inf Manag 58(6):103371","journal-title":"Inf Manag"},{"key":"11355_CR33","unstructured":"Druzdzel MJ (1999) Smile: Structural modeling, inference, and learning engine and genie: A development environment for graphical decision-theoretic models. In: Proceedings of the AAAI Conference on Artificial Intelligence (AAAI\u201999), pp. 902\u2013903"},{"key":"11355_CR34","unstructured":"Andersen SK, Olesen KG, Jensen FV (1989) Hugin\u2014a shell for building bayesian belief universes for expert systems. In: Proceedings of the 11th International Joint Conference on Artificial Intelligence (IJCAI\u201989), pp. 332\u2013337"},{"key":"11355_CR35","unstructured":"Chen Y, Doshi P, Zeng Y (2015) Iterative online planning in multiagent settings with limited model spaces and PAC guarantees. In: Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems (AAMAS\u201915), pp. 1161\u20131169"},{"issue":"1","key":"11355_CR36","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1007\/s10458-024-09640-1","volume":"38","author":"K He","year":"2024","unstructured":"He K, Doshi P, Banerjee B (2024) Modeling and reinforcement learning in partially observable many-agent systems. Auton Agent Multi-Agent Syst 38(1):10","journal-title":"Auton Agent Multi-Agent Syst"},{"key":"11355_CR37","doi-asserted-by":"publisher","first-page":"100777","DOI":"10.1016\/j.swevo.2020.100777","volume":"60","author":"G Pereira","year":"2021","unstructured":"Pereira G, de Lacerda M, de Araujo Pessoa LF, de Lima B, Neto F, Ludermir TB, Kuchen H (2021) A systematic literature review on general parameter control for evolutionary and swarm-based algorithms. Swarm Evol Comput 60:100777","journal-title":"Swarm Evol Comput"},{"issue":"11","key":"11355_CR38","doi-asserted-by":"publisher","first-page":"1717","DOI":"10.1093\/comjnl\/bxx072","volume":"60","author":"L Fotia","year":"2017","unstructured":"Fotia L, Messina F, Rosaci D, Sarn\u00c3 GML (2017) Using local trust for forming cohesive social structures in virtual communities. Comput J 60(11):1717\u20131727","journal-title":"Comput J"},{"issue":"1","key":"11355_CR39","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1007\/s10458-024-09640-1","volume":"38","author":"K He","year":"2024","unstructured":"He K, Doshi P, Banerjee B (2024) Modeling and reinforcement learning in partially observable many-agent systems. Auton Agent Multi-Agent Syst 38(1):12","journal-title":"Auton Agent Multi-Agent Syst"}],"container-title":["Artificial Intelligence Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10462-025-11355-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10462-025-11355-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10462-025-11355-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,25]],"date-time":"2025-10-25T01:47:21Z","timestamp":1761356841000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10462-025-11355-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,30]]},"references-count":39,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2025,11]]}},"alternative-id":["11355"],"URL":"https:\/\/doi.org\/10.1007\/s10462-025-11355-y","relation":{},"ISSN":["1573-7462"],"issn-type":[{"type":"electronic","value":"1573-7462"}],"subject":[],"published":{"date-parts":[[2025,8,30]]},"assertion":[{"value":"8 August 2025","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"30 August 2025","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no Conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}],"article-number":"361"}}