{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,9]],"date-time":"2026-01-09T00:15:49Z","timestamp":1767917749960,"version":"3.49.0"},"reference-count":102,"publisher":"Association for Computing Machinery (ACM)","issue":"9","license":[{"start":{"date-parts":[[2024,4,24]],"date-time":"2024-04-24T00:00:00Z","timestamp":1713916800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001602","name":"Science Foundation Ireland","doi-asserted-by":"crossref","award":["18\/CRT\/6223"],"award-info":[{"award-number":["18\/CRT\/6223"]}],"id":[{"id":"10.13039\/501100001602","id-type":"DOI","asserted-by":"crossref"}]},{"name":"SFI Frontiers for the Future","award":["21\/FFP-A\/8957"],"award-info":[{"award-number":["21\/FFP-A\/8957"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Comput. Surv."],"published-print":{"date-parts":[[2024,10,31]]},"abstract":"<jats:p>While AI algorithms have shown remarkable success in various fields, their lack of transparency hinders their application to real-life tasks. Although explanations targeted at non-experts are necessary for user trust and human-AI collaboration, the majority of explanation methods for AI are focused on developers and expert users. Counterfactual explanations are local explanations that offer users advice on what can be changed in the input for the output of the black-box model to change. Counterfactuals are user-friendly and provide actionable advice for achieving the desired output from the AI system. While extensively researched in supervised learning, there are few methods applying them to reinforcement learning (RL). In this work, we explore the reasons for the underrepresentation of a powerful explanation method in RL. We start by reviewing the current work in counterfactual explanations in supervised learning. Additionally, we explore the differences between counterfactual explanations in supervised learning and RL and identify the main challenges that prevent the adoption of methods from supervised in reinforcement learning. Finally, we redefine counterfactuals for RL and propose research directions for implementing counterfactuals in RL.<\/jats:p>","DOI":"10.1145\/3648472","type":"journal-article","created":{"date-parts":[[2024,2,17]],"date-time":"2024-02-17T10:04:40Z","timestamp":1708164280000},"page":"1-33","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":15,"title":["Redefining Counterfactual Explanations for Reinforcement Learning: Overview, Challenges and Opportunities"],"prefix":"10.1145","volume":"56","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8731-1236","authenticated-orcid":false,"given":"Jasmina","family":"Gajcin","sequence":"first","affiliation":[{"name":"Trinity College Dublin, Dublin, Ireland"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0621-5400","authenticated-orcid":false,"given":"Ivana","family":"Dusparic","sequence":"additional","affiliation":[{"name":"Trinity College Dublin, Dublin, Ireland"}]}],"member":"320","published-online":{"date-parts":[[2024,4,24]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.heliyon.2018.e00938"},{"key":"e_1_3_2_3_2","first-page":"1168","volume-title":"Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems","author":"Amir Dan","year":"2018","unstructured":"Dan Amir and Ofra Amir. 2018. Highlights: Summarizing agent behavior to people. In Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems. 1168\u20131176."},{"key":"e_1_3_2_4_2","article-title":"Concrete problems in AI safety","author":"Amodei Dario","year":"2016","unstructured":"Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Man\u00e9. 2016. Concrete problems in AI safety. arXiv preprint arXiv:1606.06565 (2016).","journal-title":"arXiv preprint arXiv:1606.06565"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2020.3024655"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2017.2743240"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/3357236.3395525"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1137\/080716542"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01270-0_28"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/IROS40897.2019.8968488"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1613\/jair.1.12228"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2019\/876"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1037\/h0085791"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.gpb.2017.07.003"},{"key":"e_1_3_2_15_2","article-title":"ReLACE: Reinforcement learning agent for counterfactual explanations of arbitrary predictive models","author":"Chen Ziheng","year":"2021","unstructured":"Ziheng Chen, Fabrizio Silvestri, Gabriele Tolomei, He Zhu, Jia Wang, and Hongshik Ahn. 2021. ReLACE: Reinforcement learning agent for counterfactual explanations of arbitrary predictive models. arXiv preprint arXiv:2110.11960 (2021).","journal-title":"arXiv preprint arXiv:2110.11960"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00916"},{"key":"e_1_3_2_17_2","first-page":"1","volume-title":"Proceedings of the IJCAI 2019 Workshop on Explainable Artificial Intelligence","author":"Coppens Youri","year":"2019","unstructured":"Youri Coppens, Kyriakos Efthymiadis, Tom Lenaerts, Ann Now\u00e9, Tim Miller, Rosina Weber, and Daniele Magazzeni. 2019. Distilling deep reinforcement learning policies in soft decision trees. In Proceedings of the IJCAI 2019 Workshop on Explainable Artificial Intelligence. 1\u20136."},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.artmed.2020.101964"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3514094.3534144"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58112-1_31"},{"key":"e_1_3_2_21_2","first-page":"1","article-title":"Explainable reinforcement learning for Broad-XAI: A conceptual framework and survey","author":"Dazeley Richard","year":"2023","unstructured":"Richard Dazeley, Peter Vamplew, and Francisco Cruz. 2023. Explainable reinforcement learning for Broad-XAI: A conceptual framework and survey. Neural Computing and Applications (2023), 1\u201324.","journal-title":"Neural Computing and Applications"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2021.103525"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/4235.996017"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3341216.3342210"},{"key":"e_1_3_2_25_2","article-title":"Challenges of real-world reinforcement learning","author":"Dulac-Arnold Gabriel","year":"2019","unstructured":"Gabriel Dulac-Arnold, Daniel Mankowitz, and Todd Hester. 2019. Challenges of real-world reinforcement learning. arXiv preprint arXiv:1904.12901 (2019).","journal-title":"arXiv preprint arXiv:1904.12901"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1037\/a0027947"},{"key":"e_1_3_2_27_2","article-title":"Distilling a neural network into a soft decision tree","author":"Frosst Nicholas","year":"2017","unstructured":"Nicholas Frosst and Geoffrey Hinton. 2017. Distilling a neural network into a soft decision tree. arXiv preprint arXiv:1711.09784 (2017).","journal-title":"arXiv preprint arXiv:1711.09784"},{"key":"e_1_3_2_28_2","article-title":"ReCCoVER: Detecting causal confusion for explainable reinforcement learning","author":"Gajcin Jasmina","year":"2022","unstructured":"Jasmina Gajcin and Ivana Dusparic. 2022. ReCCoVER: Detecting causal confusion for explainable reinforcement learning. arXiv preprint arXiv:2203.11211 (2022).","journal-title":"arXiv preprint arXiv:2203.11211"},{"key":"e_1_3_2_29_2","article-title":"Contrastive explanations for comparing preferences of reinforcement learning agents","author":"Gajcin Jasmina","year":"2021","unstructured":"Jasmina Gajcin, Rahul Nair, Tejaswini Pedapati, Radu Marinescu, Elizabeth Daly, and Ivana Dusparic. 2021. Contrastive explanations for comparing preferences of reinforcement learning agents. arXiv preprint arXiv:2112.09462 (2021).","journal-title":"arXiv preprint arXiv:2112.09462"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.5555\/2789272.2886795"},{"key":"e_1_3_2_31_2","article-title":"Quantifying differences in reward functions","author":"Gleave Adam","year":"2020","unstructured":"Adam Gleave, Michael Dennis, Shane Legg, Stuart Russell, and Jan Leike. 2020. Quantifying differences in reward functions. arXiv preprint arXiv:2006.13900 (2020).","journal-title":"arXiv preprint arXiv:2006.13900"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1609\/aimag.v38i3.2741"},{"key":"e_1_3_2_33_2","first-page":"1792","volume-title":"International Conference on Machine Learning","author":"Greydanus Samuel","year":"2018","unstructured":"Samuel Greydanus, Anurag Koul, Jonathan Dodge, and Alan Fern. 2018. Visualizing and understanding Atari agents. In International Conference on Machine Learning. PMLR, 1792\u20131801."},{"key":"e_1_3_2_34_2","first-page":"1","article-title":"Counterfactual explanations and how to find them: Literature review and benchmarking","author":"Guidotti Riccardo","year":"2022","unstructured":"Riccardo Guidotti. 2022. Counterfactual explanations and how to find them: Literature review and benchmarking. Data Mining and Knowledge Discovery (2022), 1\u201355.","journal-title":"Data Mining and Knowledge Discovery"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/MIS.2019.2957223"},{"key":"e_1_3_2_36_2","unstructured":"Anna Harutyunyan Will Dabney Thomas Mesnard Mohammad Gheshlaghi Azar Bilal Piot Nicolas Heess Hado P. van Hasselt Gregory Wayne Satinder Singh Doina Precup and R\u00e9mi Munos. 2019. Hindsight credit assignment. Advances in Neural Information Processing Systems 32 (2019)."},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2020.106685"},{"key":"e_1_3_2_38_2","article-title":"GANterfactual-RL: Understanding reinforcement learning agents\u2019 strategies through visual counterfactual explanations","author":"Huber Tobias","year":"2023","unstructured":"Tobias Huber, Maximilian Demmler, Silvan Mertes, Matthew L. Olson, and Elisabeth Andr\u00e9. 2023. GANterfactual-RL: Understanding reinforcement learning agents\u2019 strategies through visual counterfactual explanations. arXiv preprint arXiv:2302.12689 (2023).","journal-title":"arXiv preprint arXiv:2302.12689"},{"key":"e_1_3_2_39_2","first-page":"arXiv\u20132101","article-title":"Benchmarking perturbation-based saliency maps for explaining deep reinforcement learning agents","author":"Huber Tobias","year":"2021","unstructured":"Tobias Huber, Benedikt Limmer, and Elisabeth Andr\u00e9. 2021. Benchmarking perturbation-based saliency maps for explaining deep reinforcement learning agents. arXiv e-prints (2021), arXiv\u20132101.","journal-title":"arXiv e-prints"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3306618.3314267"},{"key":"e_1_3_2_41_2","article-title":"Towards realistic individual recourse and actionable explanations in black-box decision making systems","author":"Joshi Shalmali","year":"2019","unstructured":"Shalmali Joshi, Oluwasanmi Koyejo, Warut Vijitbenjaronk, Been Kim, and Joydeep Ghosh. 2019. Towards realistic individual recourse and actionable explanations in black-box decision making systems. arXiv preprint arXiv:1907.09615 (2019).","journal-title":"arXiv preprint arXiv:1907.09615"},{"key":"e_1_3_2_42_2","volume-title":"IJCAI\/ECAI Workshop on Explainable Artificial Intelligence","author":"Juozapaitis Zoe","year":"2019","unstructured":"Zoe Juozapaitis, Anurag Koul, Alan Fern, Martin Erwig, and Finale Doshi-Velez. 2019. Explainable reinforcement learning via reward decomposition. In IJCAI\/ECAI Workshop on Explainable Artificial Intelligence."},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCIAIG.2012.2188528"},{"key":"e_1_3_2_44_2","article-title":"A survey of algorithmic recourse: Definitions, formulations, solutions, and prospects","author":"Karimi Amir-Hossein","year":"2020","unstructured":"Amir-Hossein Karimi, Gilles Barthe, Bernhard Sch\u00f6lkopf, and Isabel Valera. 2020. A survey of algorithmic recourse: Definitions, formulations, solutions, and prospects. arXiv preprint arXiv:2010.04050 (2020).","journal-title":"arXiv preprint arXiv:2010.04050"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/3442188.3445899"},{"key":"e_1_3_2_46_2","first-page":"265","article-title":"Algorithmic recourse under imperfect causal knowledge: A probabilistic approach","volume":"33","author":"Karimi Amir-Hossein","year":"2020","unstructured":"Amir-Hossein Karimi, Julius Von K\u00fcgelgen, Bernhard Sch\u00f6lkopf, and Isabel Valera. 2020. Algorithmic recourse under imperfect causal knowledge: A probabilistic approach. Advances in Neural Information Processing Systems 33 (2020), 265\u2013277.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_47_2","article-title":"Deep reinforcement learning for autonomous driving: A survey","author":"Kiran B. Ravi","year":"2021","unstructured":"B. Ravi Kiran, Ibrahim Sobh, Victor Talpaert, Patrick Mannion, Ahmad A. Al Sallab, Senthil Yogamani, and Patrick P\u00e9rez. 2021. Deep reinforcement learning for autonomous driving: A survey. IEEE Transactions on Intelligent Transportation Systems (2021).","journal-title":"IEEE Transactions on Intelligent Transportation Systems"},{"key":"e_1_3_2_48_2","unstructured":"Will Knight. 2018. What Uber\u2019s Fatal Accident Could Mean for the Autonomous-Car Industry. technologyreview.com\/2018\/03\/19\/241022\/what-ubers-fatal-accident-could-mean-for-the-autonomous-car-industry"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1177\/0278364913495721"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-021-01461-z"},{"key":"e_1_3_2_51_2","article-title":"Interpretable & explorable approximations of black box models","author":"Lakkaraju Himabindu","year":"2017","unstructured":"Himabindu Lakkaraju, Ece Kamar, Rich Caruana, and Jure Leskovec. 2017. Interpretable & explorable approximations of black box models. arXiv preprint arXiv:1707.01154 (2017).","journal-title":"arXiv preprint arXiv:1707.01154"},{"key":"e_1_3_2_52_2","first-page":"5979","volume-title":"International Conference on Machine Learning","author":"Landajuela Mikel","year":"2021","unstructured":"Mikel Landajuela, Brenden K. Petersen, Sookyung Kim, Claudio P. Santiago, Ruben Glatt, Nathan Mundhenk, Jacob F. Pettit, and Daniel Faissol. 2021. Discovering symbolic policies with deep reinforcement learning. In International Conference on Machine Learning. PMLR, 5979\u20135989."},{"key":"e_1_3_2_53_2","article-title":"Inverse classification for comparison-based interpretability in machine learning","author":"Laugel Thibault","year":"2017","unstructured":"Thibault Laugel, Marie-Jeanne Lesot, Christophe Marsala, Xavier Renard, and Marcin Detyniecki. 2017. Inverse classification for comparison-based interpretability in machine learning. arXiv preprint arXiv:1712.08443 (2017).","journal-title":"arXiv preprint arXiv:1712.08443"},{"key":"e_1_3_2_54_2","article-title":"Reinforcement learning applications","author":"Li Yuxi","year":"2019","unstructured":"Yuxi Li. 2019. Reinforcement learning applications. arXiv preprint arXiv:1908.06973 (2019).","journal-title":"arXiv preprint arXiv:1908.06973"},{"key":"e_1_3_2_55_2","article-title":"Continuous control with deep reinforcement learning","author":"Lillicrap Timothy P.","year":"2015","unstructured":"Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. 2015. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971 (2015).","journal-title":"arXiv preprint arXiv:1509.02971"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1017\/S1358246100005130"},{"issue":"3","key":"e_1_3_2_57_2","doi-asserted-by":"crossref","first-page":"385","DOI":"10.1109\/TSMC.2014.2358639","article-title":"Multiobjective reinforcement learning: A comprehensive overview","volume":"45","author":"Liu Chunming","year":"2014","unstructured":"Chunming Liu, Xin Xu, and Dewen Hu. 2014. Multiobjective reinforcement learning: A comprehensive overview. IEEE Transactions on Systems, Man, and Cybernetics: Systems 45, 3 (2014), 385\u2013398.","journal-title":"IEEE Transactions on Systems, Man, and Cybernetics: Systems"},{"key":"e_1_3_2_58_2","first-page":"414","volume-title":"Joint European Conference on Machine Learning and Knowledge Discovery in Databases","author":"Liu Guiliang","year":"2018","unstructured":"Guiliang Liu, Oliver Schulte, Wang Zhu, and Qingcan Li. 2018. Toward interpretable deep reinforcement learning with linear model u-trees. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 414\u2013429."},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-86520-7_40"},{"key":"e_1_3_2_60_2","article-title":"Estimating dynamic treatment regimes in mobile health using v-learning","author":"Luckett Daniel J.","year":"2019","unstructured":"Daniel J. Luckett, Eric B. Laber, Anna R. Kahkoska, David M. Maahs, Elizabeth Mayer-Davis, and Michael R. Kosorok. 2019. Estimating dynamic treatment regimes in mobile health using v-learning. J. Amer. Statist. Assoc. (2019).","journal-title":"J. Amer. Statist. Assoc."},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.5555\/3295222.3295230"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i03.5631"},{"key":"e_1_3_2_63_2","article-title":"Preserving causal constraints in counterfactual explanations for machine learning classifiers","author":"Mahajan Divyat","year":"2019","unstructured":"Divyat Mahajan, Chenhao Tan, and Amit Sharma. 2019. Preserving causal constraints in counterfactual explanations for machine learning classifiers. arXiv preprint arXiv:1912.03277 (2019).","journal-title":"arXiv preprint arXiv:1912.03277"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2018.07.007"},{"key":"e_1_3_2_65_2","article-title":"Playing Atari with deep reinforcement learning","author":"Mnih Volodymyr","year":"2013","unstructured":"Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. 2013. Playing Atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013).","journal-title":"arXiv preprint arXiv:1312.5602"},{"key":"e_1_3_2_66_2","unstructured":"Christoph Molnar. 2020. Interpretable machine learning. Lulu.com."},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372850"},{"key":"e_1_3_2_68_2","article-title":"Counterfactual states for Atari agents via generative deep learning","author":"Olson Matthew L.","year":"2019","unstructured":"Matthew L. Olson, Lawrence Neal, Fuxin Li, and Weng-Keen Wong. 2019. Counterfactual states for Atari agents via generative deep learning. arXiv preprint arXiv:1909.12969 (2019).","journal-title":"arXiv preprint arXiv:1909.12969"},{"key":"e_1_3_2_69_2","article-title":"The effects of reward misspecification: Mapping and mitigating misaligned models","author":"Pan Alexander","year":"2022","unstructured":"Alexander Pan, Kush Bhatia, and Jacob Steinhardt. 2022. The effects of reward misspecification: Mapping and mitigating misaligned models. arXiv preprint arXiv:2201.03544 (2022).","journal-title":"arXiv preprint arXiv:2201.03544"},{"key":"e_1_3_2_70_2","volume-title":"The Book of Why: The New Science of Cause and Effect","author":"Pearl Judea","year":"2018","unstructured":"Judea Pearl and Dana Mackenzie. 2018. The Book of Why: The New Science of Cause and Effect. Basic books."},{"issue":"4","key":"e_1_3_2_71_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3197517.3201311","article-title":"DeepMimic: Example-guided deep reinforcement learning of physics-based character skills","volume":"37","author":"Peng Xue Bin","year":"2018","unstructured":"Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel van de Panne. 2018. DeepMimic: Example-guided deep reinforcement learning of physics-based character skills. ACM Transactions on Graphics (TOG) 37, 4 (2018), 1\u201314.","journal-title":"ACM Transactions on Graphics (TOG)"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.1145\/3234150"},{"key":"e_1_3_2_73_2","doi-asserted-by":"publisher","DOI":"10.1145\/3375627.3375850"},{"key":"e_1_3_2_74_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-57321-8_5"},{"key":"e_1_3_2_75_2","article-title":"Explain your move: Understanding agent actions using specific and relevant feature attribution","author":"Puri Nikaash","year":"2019","unstructured":"Nikaash Puri, Sukriti Verma, Piyush Gupta, Dhruv Kayastha, Shripad Deshmukh, Balaji Krishnamurthy, and Sameer Singh. 2019. Explain your move: Understanding agent actions using specific and relevant feature attribution. arXiv preprint arXiv:1912.12191 (2019).","journal-title":"arXiv preprint arXiv:1912.12191"},{"key":"e_1_3_2_76_2","article-title":"Challenges for reinforcement learning in healthcare","author":"Riachi Elsa","year":"2021","unstructured":"Elsa Riachi, Muhammad Mamdani, Michael Fralick, and Frank Rudzicz. 2021. Challenges for reinforcement learning in healthcare. arXiv preprint arXiv:2103.05612 (2021).","journal-title":"arXiv preprint arXiv:2103.05612"},{"key":"e_1_3_2_77_2","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939778"},{"key":"e_1_3_2_78_2","article-title":"Model-agnostic and scalable counterfactual explanations via reinforcement learning","author":"Samoilescu Robert-Florian","year":"2021","unstructured":"Robert-Florian Samoilescu, Arnaud Van Looveren, and Janis Klaise. 2021. Model-agnostic and scalable counterfactual explanations via reinforcement learning. arXiv preprint arXiv:2106.02597 (2021).","journal-title":"arXiv preprint arXiv:2106.02597"},{"key":"e_1_3_2_79_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2020.103367"},{"key":"e_1_3_2_80_2","article-title":"CERTIFAI: Counterfactual explanations for robustness, transparency, interpretability, and fairness of artificial intelligence models","author":"Sharma Shubham","year":"2019","unstructured":"Shubham Sharma, Jette Henderson, and Joydeep Ghosh. 2019. CERTIFAI: Counterfactual explanations for robustness, transparency, interpretability, and fairness of artificial intelligence models. arXiv preprint arXiv:1905.07857 (2019).","journal-title":"arXiv preprint arXiv:1905.07857"},{"key":"e_1_3_2_81_2","article-title":"Deep inside convolutional networks: Visualising image classification models and saliency maps","author":"Simonyan Karen","year":"2013","unstructured":"Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2013. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034 (2013).","journal-title":"arXiv preprint arXiv:1312.6034"},{"key":"e_1_3_2_82_2","article-title":"Counterfactual explanations of machine learning predictions: Opportunities and challenges for AI safety","author":"Sokol Kacper","year":"2019","unstructured":"Kacper Sokol and Peter A. Flach. 2019. Counterfactual explanations of machine learning predictions: Opportunities and challenges for AI safety. SafeAI@AAAI (2019).","journal-title":"SafeAI@AAAI"},{"key":"e_1_3_2_83_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2021.3051315"},{"key":"e_1_3_2_84_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10115-013-0679-x"},{"key":"e_1_3_2_85_2","doi-asserted-by":"publisher","DOI":"10.1109\/RO-MAN47096.2020.9223614"},{"key":"e_1_3_2_86_2","volume-title":"Reinforcement Learning: An Introduction","author":"Sutton Richard S.","year":"2018","unstructured":"Richard S. Sutton and Andrew G. Barto. 2018. Reinforcement Learning: An Introduction. MIT Press."},{"key":"e_1_3_2_87_2","article-title":"Intriguing properties of neural networks","author":"Szegedy Christian","year":"2013","unstructured":"Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2013. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 (2013).","journal-title":"arXiv preprint arXiv:1312.6199"},{"key":"e_1_3_2_88_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33012514"},{"key":"e_1_3_2_89_2","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287566"},{"key":"e_1_3_2_90_2","article-title":"Contrastive explanations for reinforcement learning in terms of expected consequences","author":"Waa Jasper van der","year":"2018","unstructured":"Jasper van der Waa, Jurriaan van Diggelen, Karel van den Bosch, and Mark Neerincx. 2018. Contrastive explanations for reinforcement learning in terms of expected consequences. arXiv preprint arXiv:1807.08706 (2018).","journal-title":"arXiv preprint arXiv:1807.08706"},{"key":"e_1_3_2_91_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33019902"},{"key":"e_1_3_2_92_2","first-page":"5045","volume-title":"International Conference on Machine Learning","author":"Verma Abhinav","year":"2018","unstructured":"Abhinav Verma, Vijayaraghavan Murali, Rishabh Singh, Pushmeet Kohli, and Swarat Chaudhuri. 2018. Programmatically interpretable reinforcement learning. In International Conference on Machine Learning. PMLR, 5045\u20135054."},{"key":"e_1_3_2_93_2","article-title":"Counterfactual explanations for machine learning: A review","author":"Verma Sahil","year":"2020","unstructured":"Sahil Verma, John Dickerson, and Keegan Hines. 2020. Counterfactual explanations for machine learning: A review. arXiv preprint arXiv:2010.10596 (2020).","journal-title":"arXiv preprint arXiv:2010.10596"},{"key":"e_1_3_2_94_2","unstructured":"Sahil Verma John Dickerson and Keegan Hines. 2021. Counterfactual explanations for machine learning: Challenges revisited. arXiv preprint arXiv:2106.07756 (2021)."},{"key":"e_1_3_2_95_2","unstructured":"Oriol Vinyals Timo Ewalds Sergey Bartunov Petko Georgiev Alexander Sasha Vezhnevets Michelle Yeo Alireza Makhzani Heinrich K\u00fcttler John Agapiou Julian Schrittwieser John Quan Stephen Gaffney Stig Petersen Karen Simonyan Tom Schaul Hado van Hasselt David Silver Timothy Lillicrap Kevin Calderone Paul Keet Anthony Brunasso David Lawrence Anders Ekermo Jacob Repp and Rodney Tsing. 2017. Starcraft ii: A new challenge for reinforcement learning. arXiv preprint arXiv:1708.04782 (2017)."},{"key":"e_1_3_2_96_2","doi-asserted-by":"publisher","DOI":"10.1145\/3527448"},{"key":"e_1_3_2_97_2","first-page":"841","article-title":"Counterfactual explanations without opening the black box: Automated decisions and the GDPR","volume":"31","author":"Wachter Sandra","year":"2017","unstructured":"Sandra Wachter, Brent Mittelstadt, and Chris Russell. 2017. Counterfactual explanations without opening the black box: Automated decisions and the GDPR. Harv. JL & Tech. 31 (2017), 841.","journal-title":"Harv. JL & Tech."},{"key":"e_1_3_2_98_2","first-page":"1995","volume-title":"International Conference on Machine Learning","author":"Wang Ziyu","year":"2016","unstructured":"Ziyu Wang, Tom Schaul, Matteo Hessel, Hado Hasselt, Marc Lanctot, and Nando Freitas. 2016. Dueling network architectures for deep reinforcement learning. In International Conference on Machine Learning. PMLR, 1995\u20132003."},{"key":"e_1_3_2_99_2","doi-asserted-by":"publisher","DOI":"10.3389\/frai.2021.550030"},{"key":"e_1_3_2_100_2","doi-asserted-by":"publisher","DOI":"10.3390\/designs2020013"},{"key":"e_1_3_2_101_2","doi-asserted-by":"publisher","DOI":"10.1109\/EI250167.2020.9347147"},{"key":"e_1_3_2_102_2","doi-asserted-by":"publisher","DOI":"10.1002\/sim.3720"},{"key":"e_1_3_2_103_2","article-title":"Visualizing dynamics: From t-SNE to SEMI-MDPs","author":"Zrihem Nir Ben","year":"2016","unstructured":"Nir Ben Zrihem, Tom Zahavy, and Shie Mannor. 2016. Visualizing dynamics: From t-SNE to SEMI-MDPs. arXiv preprint arXiv:1606.07112 (2016).","journal-title":"arXiv preprint arXiv:1606.07112"}],"container-title":["ACM Computing Surveys"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3648472","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3648472","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:50:19Z","timestamp":1750287019000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3648472"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,24]]},"references-count":102,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2024,10,31]]}},"alternative-id":["10.1145\/3648472"],"URL":"https:\/\/doi.org\/10.1145\/3648472","relation":{},"ISSN":["0360-0300","1557-7341"],"issn-type":[{"value":"0360-0300","type":"print"},{"value":"1557-7341","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,24]]},"assertion":[{"value":"2022-10-14","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-01-31","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-04-24","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}