{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,7]],"date-time":"2026-07-07T12:35:02Z","timestamp":1783427702208,"version":"3.54.6"},"reference-count":73,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,7,28]],"date-time":"2023-07-28T00:00:00Z","timestamp":1690502400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:p>Resilience in autonomous agent systems is about having the capacity to anticipate, respond to, adapt to, and recover from adverse and dynamic conditions in complex environments. It is associated with the intelligence possessed by the agents to preserve the functionality or to minimize the impact on functionality through a transformation, reconfiguration, or expansion performed across the system. Enhancing the resilience of systems could pave way toward higher autonomy allowing them to tackle intricate dynamic problems. The state-of-the-art systems have mostly focussed on improving the redundancy of the system, adopting decentralized control architectures, and utilizing distributed sensing capabilities. While machine learning approaches for efficient distribution and allocation of skills and tasks have enhanced the potential of these systems, they are still limited when presented with dynamic environments. To move beyond the current limitations, this paper advocates incorporating counterfactual learning models for agents to enable them with the ability to predict possible future conditions and adjust their behavior. Counterfactual learning is a topic that has recently been gaining attention as a model-agnostic and post-hoc technique to improve explainability in machine learning models. Using counterfactual causality can also help gain insights into unforeseen circumstances and make inferences about the probability of desired outcomes. We propose that this can be used in agent systems as a means to guide and prepare them to cope with unanticipated environmental conditions. This supplementary support for adaptation can enable the design of more intelligent and complex autonomous agent systems to address the multifaceted characteristics of real-world problem domains.<\/jats:p>","DOI":"10.3389\/frai.2023.1212336","type":"journal-article","created":{"date-parts":[[2023,7,28]],"date-time":"2023-07-28T17:53:32Z","timestamp":1690566812000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Counterfactual learning in enhancing resilience in autonomous agent systems"],"prefix":"10.3389","volume":"6","author":[{"given":"Dilini","family":"Samarasinghe","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2023,7,28]]},"reference":[{"key":"B1","volume-title":"Adaptive Control","author":"\u00c5str\u00f6m","year":"2013"},{"key":"B2","article-title":"\u201cSanity checks for saliency maps,\u201d","volume-title":"Advances in Neural Information Processing Systems, Vol. 31","author":"Adebayo","year":"2018"},{"key":"B3","doi-asserted-by":"publisher","first-page":"3412","DOI":"10.3390\/app13063412","article-title":"Fuzzy cognitive maps: their role in explainable artificial intelligence","volume":"13","author":"Apostolopoulos","year":"2023","journal-title":"Appl. Sci"},{"key":"B4","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1145\/3457183","article-title":"Explaining in time: Meeting interactive standards of explanation for robotic systems","volume":"10","author":"Arnold","year":"2021","journal-title":"J. Hum. Robot Interact"},{"key":"B5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/SSCI50451.2021.9660058","article-title":"\u201cEvaluating robustness of counterfactual explanations,\u201d","volume-title":"2021 IEEE Symposium Series on Computational Intelligence (SSCI)","author":"Artelt","year":"2021"},{"key":"B6","doi-asserted-by":"publisher","first-page":"104304","DOI":"10.1016\/j.robot.2022.104304","article-title":"A survey on the autonomous exploration of confined subterranean spaces: perspectives from real-word and industrial robotic deployments","volume":"160","author":"Azp\u00farua","year":"2023","journal-title":"Robot. Auton. Syst"},{"key":"B7","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1080\/095281300409793","article-title":"Dynamic adaptive autonomy in multi-agent systems","volume":"12","author":"Barber","year":"2000","journal-title":"J. Exp. Theor. Artif. Intell"},{"key":"B8","first-page":"115","article-title":"A review of studies in swarm robotics","volume":"15","author":"Bayindir","year":"2007","journal-title":"Turk. J. Electr. Eng. Comput. Sci"},{"key":"B9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/978-3-540-30552-1_1","article-title":"\u201cFrom swarm intelligence to swarm robotics,\u201d","volume-title":"Swarm Robotics","author":"Beni","year":"2005"},{"key":"B10","doi-asserted-by":"crossref","first-page":"172","DOI":"10.4018\/978-1-59140-581-8.ch007","article-title":"\u201cEngineering adaptive multi-agent systems: the Adelfe methodology,\u201d","volume-title":"Agent-Oriented Methodologies","author":"Bernon","year":"2005"},{"key":"B11","doi-asserted-by":"crossref","first-page":"8772","DOI":"10.1109\/ICRA46639.2022.9811744","article-title":"\u201cA framework for real-world multi-robot systems running decentralized GNN-based policies,\u201d","volume-title":"2022 International Conference on Robotics and Automation (ICRA)","author":"Blumenkamp","year":"2022"},{"key":"B12","doi-asserted-by":"publisher","first-page":"326","DOI":"10.1108\/IR-01-2019-0023","article-title":"Strong prospects for robots in retail","volume":"46","author":"Bogue","year":"2019","journal-title":"Indus. Robot"},{"key":"B13","doi-asserted-by":"crossref","first-page":"2943","DOI":"10.1109\/IROS.2015.7353783","article-title":"\u201cCounterfactual reasoning about intent for interactive navigation in dynamic environments,\u201d","volume-title":"2015 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)","author":"Bordallo","year":"2015"},{"key":"B14","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1007\/s43154-022-00079-4","article-title":"Resilient robot teams: a review integrating decentralised control, change-detection, and learning","volume":"3","author":"Bossens","year":"2022","journal-title":"Curr. Robot. Rep"},{"key":"B15","first-page":"793","article-title":"\u201cDeep counterfactual regret minimization,\u201d","volume-title":"Proceedings of the 36th International Conference on Machine Learning, Vol. 97 of Proceedings of Machine Learning Research","author":"Brown","year":"2019"},{"key":"B16","doi-asserted-by":"publisher","first-page":"735","DOI":"10.3390\/jmse10060735","article-title":"Biomimetic aquatic robots based on fluid-driven actuators: a review","volume":"10","author":"Bu","year":"2022","journal-title":"J. Mar. Sci. Eng"},{"key":"B17","first-page":"6276","article-title":"\u201cCounterfactuals in explainable artificial intelligence (XAI): evidence from human reasoning,\u201d","volume-title":"IJCAI","author":"Byrne","year":"2019"},{"key":"B18","doi-asserted-by":"publisher","first-page":"9156","DOI":"10.3390\/su14159156","article-title":"A comprehensive review of path planning for agricultural ground robots","volume":"14","author":"Chakraborty","year":"2022","journal-title":"Sustainability"},{"key":"B19","doi-asserted-by":"publisher","first-page":"59","DOI":"10.1016\/j.inffus.2021.11.003","article-title":"Counterfactuals and causability in explainable artificial intelligence: theory, algorithms, and applications","volume":"81","author":"Chou","year":"2022","journal-title":"Inform. Fus"},{"key":"B20","article-title":"\u201cScalability & multi-agent systems,\u201d","author":"Deters","year":"2001","journal-title":"2nd International Workshop Infrastructure for Agents, MAS and Scalable MAS, Vol. 10"},{"key":"B21","doi-asserted-by":"publisher","first-page":"23","DOI":"10.5840\/monist20099212","article-title":"Absences, possible causation, and the problem of non-locality","volume":"92","author":"Dowe","year":"2009","journal-title":"Monist"},{"key":"B22","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/INFOTEH57020.2023.10094113","article-title":"\u201cProposal of multi-agent robotic fire extinguishing model for industrial premises,\u201d","volume-title":"2023 22nd International Symposium INFOTEH-JAHORINA (INFOTEH)","author":"Filipovi\u0107","year":"2023"},{"key":"B23","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1609\/aaai.v32i1.11794","article-title":"Counterfactual multi-agent policy gradients","volume":"32","author":"Foerster","year":"2018","journal-title":"Proc. AAAI Conf. Artif. Intell"},{"key":"B24","doi-asserted-by":"publisher","first-page":"1208","DOI":"10.1109\/TRO.2005.853487","article-title":"Swarm aggregations using artificial potentials and sliding-mode control","volume":"21","author":"Gazi","year":"2005","journal-title":"IEEE Trans. Robot"},{"key":"B25","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1007\/978-3-540-71541-2_6","article-title":"\u201cCoordination and control of multi-agent dynamic systems: models and approaches,\u201d","volume-title":"Swarm Robotics","author":"Gazi","year":"2007"},{"key":"B26","first-page":"9","volume-title":"A Decentralized Control Strategy for Resilient Connectivity Maintenance in Multi-Robot Systems Subject to Failures","author":"Ghedini","year":"2018"},{"key":"B27","doi-asserted-by":"publisher","first-page":"26339137221114874","DOI":"10.1177\/26339137221114874","article-title":"Collective intelligence for deep learning: a survey of recent developments","volume":"1","author":"Ha","year":"2022","journal-title":"Collect. Intell"},{"key":"B28","first-page":"21","article-title":"\u201cExplanation in human-agent teamwork,\u201d","volume-title":"Coordination, Organizations, Institutions, and Norms in Agent System VII","author":"Harbers","year":"2012"},{"key":"B29","doi-asserted-by":"crossref","first-page":"735","DOI":"10.1109\/ROBOT.2005.1570205","article-title":"\u201cRedundancy, efficiency and robustness in multi-robot coverage,\u201d","volume-title":"Proceedings of the 2005 IEEE International Conference on Robotics and Automation","author":"Hazon","year":"2005"},{"key":"B30","first-page":"93","volume-title":"Robustness in the Presence of Task Differentiation in Robot Ensembles","author":"Hsieh","year":"2013"},{"key":"B31","first-page":"1","article-title":"\u201cLand-mobile robots for rescue and search: a technological and systematic review,\u201d","volume-title":"2022 IEEE International IOT, Electronics and Mechatronics Conference (IEMTRONICS)","author":"Huamanchahua","year":"2022"},{"key":"B32","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1007\/3-540-44568-4_2","article-title":"\u201cReactivity and deliberation: a survey on multi-robot systems,\u201d","volume-title":"Balancing Reactivity and Social Deliberation in Multi-Agent Systems","author":"Iocchi","year":"2001"},{"key":"B33","first-page":"8","article-title":"\u201cA survey and analysis of cooperative multi-agent robot systems: challenges and directions,\u201d","volume-title":"Applications of Mobile Robots","author":"Ismail","year":"2018"},{"key":"B34","first-page":"3616","article-title":"\u201cOffline learning of counterfactual predictions for real-world robotic reinforcement learning","author":"Jin","year":"2022"},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.1145\/3422841.3423533","article-title":"Gender slopes: counterfactual fairness for computer vision models by attribute manipulation","year":"2020","journal-title":"arXiv:2005.10430"},{"key":"B36","doi-asserted-by":"publisher","first-page":"798","DOI":"10.1109\/LRA.2020.2965855","article-title":"Cooperative aerial-ground multi-robot system for automated construction tasks","volume":"5","author":"Krizmancic","year":"2020","journal-title":"IEEE Robot. Autom. Lett"},{"key":"B37","doi-asserted-by":"crossref","first-page":"3306","DOI":"10.1109\/ICRA46639.2022.9812401","article-title":"\u201cAutonomous teamed exploration of subterranean environments using legged and aerial robots,\u201d","volume-title":"2022 International Conference on Robotics and Automation (ICRA)","author":"Kulkarni","year":"2022"},{"key":"B38","doi-asserted-by":"publisher","first-page":"8","DOI":"10.3390\/technologies9010008","article-title":"A survey of robots in healthcare","volume":"9","author":"Kyrarini","year":"2021","journal-title":"Technologies"},{"key":"B39","doi-asserted-by":"publisher","first-page":"294","DOI":"10.1016\/j.neucom.2021.03.030","article-title":"Learning locomotion skills in evolvable robots","volume":"452","author":"Lan","year":"2021","journal-title":"Neurocomputing"},{"key":"B40","doi-asserted-by":"publisher","first-page":"556","DOI":"10.2307\/2025310","article-title":"Causation","volume":"70","author":"Lewis","year":"1974","journal-title":"J. Philos"},{"key":"B41","doi-asserted-by":"crossref","DOI":"10.15607\/RSS.2020.XVI.021","article-title":"\u201cResilient distributed diffusion for multi-robot systems using centerpoint,\u201d","volume-title":"Robotics: Science and Systems","author":"Li","year":"2020"},{"key":"B42","doi-asserted-by":"publisher","first-page":"102510","DOI":"10.1016\/j.rcim.2022.102510","article-title":"Proactive human robot collaboration: mutual-cognitive, predictable, and self-organising perspectives","volume":"81","author":"Li","year":"2023","journal-title":"Robot. Comput. Integr. Manufact"},{"key":"B43","doi-asserted-by":"publisher","DOI":"10.2514\/6.2010-8160","article-title":"\u201cChance constrained RRT for probabilistic robustness to environmental uncertainty,\u201d","author":"Luders","year":"2010","journal-title":"AIAA Guidance, Navigation, and Control Conference"},{"key":"B44","first-page":"3643","article-title":"\u201cGenerating multi-agent potential functions using counterfactual estimates,\u201d","volume-title":"Proceedings of Learning, Inference and Control of Multi-Agent Systems (at NIPS 2016)","author":"Mannion","year":"2016"},{"key":"B45","article-title":"\u201cInteractive constrained learning from demonstration using visual robot behavior counterfactuals,\u201d","volume-title":"Proceedings of the Accessibility of Robot Programming and Work of the Future Workshop at RSS, Vol. 2021","author":"Mueller","year":"2021"},{"key":"B46","doi-asserted-by":"crossref","first-page":"625","DOI":"10.1007\/978-981-16-9770-8_41","article-title":"\u201cRobots in mining,\u201d","volume-title":"Proceedings of Geotechnical Challenges in Mining, Tunneling and Underground Infrastructures","author":"Nanadrekar","year":"2022"},{"key":"B47","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-01231-1_38","article-title":"\u201cOpen set learning with counterfactual images,\u201d","volume-title":"Proceedings of the European Conference on Computer Vision (ECCV)","author":"Neal","year":"2018"},{"key":"B48","doi-asserted-by":"crossref","first-page":"204","DOI":"10.1007\/978-3-319-91122-9_18","article-title":"\u201cUsing perceptual and cognitive explanations for enhanced human-agent team performance,\u201d","volume-title":"International Conference on Engineering Psychology and Cognitive Ergonomics","author":"Neerincx","year":"2018"},{"key":"B49","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1007\/978-3-319-14484-9_4","article-title":"\u201cImproving the design and modularity of BDI agents with capability relationships,\u201d","volume-title":"Engineering Multi-Agent Systems","author":"Nunes","year":"2014"},{"key":"B50","doi-asserted-by":"publisher","first-page":"1520","DOI":"10.1109\/TAC.2004.834113","article-title":"Consensus problems in networks of agents with switching topology and time-delays","volume":"49","author":"Olfati-Saber","year":"2004","journal-title":"IEEE Trans. Autom. Control"},{"key":"B51","doi-asserted-by":"publisher","first-page":"220","DOI":"10.1109\/70.681242","article-title":"Alliance: an architecture for fault tolerant multirobot cooperation","volume":"14","author":"Parker","year":"1998","journal-title":"IEEE Trans. Robot. Autom"},{"key":"B52","article-title":"\u201cRobust multi-agent counterfactual prediction,\u201d","volume-title":"Advances in Neural Information Processing Systems, Vol. 32","author":"Peysakhovich","year":"2019"},{"key":"B53","article-title":"Beyond robustness: a taxonomy of approaches towards resilient multi-robot systems","author":"Prorok","year":"2021","journal-title":"CoRR, abs\/2109.12343"},{"key":"B54","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1016\/S0921-8890(98)00080-3","article-title":"Multi-agent-based agile scheduling","volume":"27","author":"Rabelo","year":"1999","journal-title":"Robot. Auton. Syst"},{"key":"B55","doi-asserted-by":"publisher","first-page":"1314","DOI":"10.5897\/IJPS11.1745","article-title":"Optimal path planning of mobile robots: a review","volume":"7","author":"Raja","year":"2012","journal-title":"Int. J. Phys. Sci"},{"key":"B56","doi-asserted-by":"publisher","first-page":"109384","DOI":"10.1016\/j.automatica.2020.109384","article-title":"Resiliency in dynamic leader follower multiagent systems","volume":"125","author":"Rezaee","year":"2021","journal-title":"Automatica"},{"key":"B57","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1007\/978-3-540-30552-1_2","article-title":"\u201cSwarm robotics: from sources of inspiration to domains of application,\u201d","volume-title":"Swarm Robotics","author":"\u015eahin","year":"2005"},{"key":"B58","doi-asserted-by":"publisher","first-page":"828","DOI":"10.1145\/3460231.3473320","article-title":"\u201cCounterfactual learning and evaluation for recommender systems: Foundations, implementations, and recent advances,\u201d","author":"Saito","year":"2021","journal-title":"Proceedings of the 15th ACM Conference on Recommender Systems, RecSys '21"},{"key":"B59","doi-asserted-by":"publisher","first-page":"102247","DOI":"10.1109\/ACCESS.2022.3209260","article-title":"Flow-based reinforcement learning","volume":"10","author":"Samarasinghe","year":"","journal-title":"IEEE Access"},{"key":"B60","doi-asserted-by":"publisher","first-page":"101017","DOI":"10.1016\/j.swevo.2021.101017","article-title":"Grammar-based cooperative learning for evolving collective behaviours in multi-agent systems","volume":"69","author":"Samarasinghe","year":"","journal-title":"Swarm Evol. Comput"},{"key":"B61","doi-asserted-by":"publisher","first-page":"1109","DOI":"10.1109\/TRO.2019.2922493","article-title":"Distributed learning of decentralized control policies for articulated mobile robots","volume":"35","author":"Sartoretti","year":"2019","journal-title":"IEEE Trans. Robot"},{"key":"B62","doi-asserted-by":"publisher","DOI":"10.6028\/NIST.IR.8022","author":"Shneier","year":"2015","journal-title":"Literature Review of Mobile Robots for Manufacturing"},{"key":"B63","first-page":"1655","article-title":"\u201cApproximate difference rewards for scalable multigent reinforcement learning,\u201d","volume-title":"Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS '21","author":"Singh","year":"2021"},{"key":"B64","first-page":"763","article-title":"Evolution of industrial robots and their applications","volume":"3","author":"Singh","year":"2013","journal-title":"Int. J. Emerg. Technol. Adv. Eng"},{"key":"B65","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/ICDL-EpiRob48136.2020.9278061","article-title":"\u201cCounterfactual explanation and causal inference in service of robustness in robot control,\u201d","volume-title":"2020 Joint IEEE 10th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob)","author":"Smith","year":"2020"},{"key":"B66","first-page":"223","article-title":"\u201cDirected stigmergy-based control for multi-robot systems,\u201d","volume-title":"2007 2nd ACM\/IEEE International Conference of Human-Robot Interaction (HRI)","author":"Steele","year":"2007"},{"key":"B67","doi-asserted-by":"crossref","first-page":"5403","DOI":"10.1109\/ICRA.2015.7139954","article-title":"\u201cPlanning for multi-agent teams with leader switching,\u201d","volume-title":"2015 IEEE International Conference on Robotics and Automation (ICRA)","author":"Swaminathan","year":"2015"},{"key":"B68","doi-asserted-by":"publisher","first-page":"102576","DOI":"10.1016\/j.mechatronics.2021.102576","article-title":"Active learning in robotics: a review of control principles","volume":"77","author":"Taylor","year":"2021","journal-title":"Mechatronics"},{"key":"B69","doi-asserted-by":"publisher","first-page":"11376","DOI":"10.1609\/aaai.v36i10.21389","article-title":"Debiasing nlu models via causal intervention and counterfactual reasoning","volume":"36","author":"Tian","year":"2022","journal-title":"Proc. AAAI Conf. Artif. Intell"},{"key":"B70","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1109\/MIS.2004.30","article-title":"A new generation of military robots","volume":"19","author":"Voth","year":"2004","journal-title":"IEEE Intell. Syst"},{"key":"B71","doi-asserted-by":"publisher","first-page":"841","DOI":"10.2139\/ssrn.3063289","article-title":"Counterfactual explanations without opening the black box: automated decisions and the GDPR","volume":"31","author":"Wachter","year":"2017","journal-title":"Harv. JL Tech"},{"key":"B72","doi-asserted-by":"publisher","first-page":"184109","DOI":"10.1109\/ACCESS.2020.3025287","article-title":"Decentralized control of multi-robot system in cooperative object transportation using deep reinforcement learning","volume":"8","author":"Zhang","year":"2020","journal-title":"IEEE Access"},{"key":"B73","doi-asserted-by":"publisher","first-page":"3438","DOI":"10.18653\/v1\/2020.emnlp-main.276","article-title":"\u201cCounterfactual off-policy training for neural dialogue generation,\u201d","author":"Zhu","year":"2020","journal-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2023.1212336\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,7,28]],"date-time":"2023-07-28T17:54:08Z","timestamp":1690566848000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2023.1212336\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,28]]},"references-count":73,"alternative-id":["10.3389\/frai.2023.1212336"],"URL":"https:\/\/doi.org\/10.3389\/frai.2023.1212336","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,28]]},"article-number":"1212336"}}