{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,19]],"date-time":"2026-11-19T05:12:02Z","timestamp":1795065122805,"version":"4.10.2"},"publisher-location":"California","reference-count":0,"publisher":"International Joint Conferences on Artificial Intelligence Organization","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,8]]},"abstract":"<jats:p>In recent years, there has been an explosion of AI research on counterfactual explanations as a solution to the problem of eXplainable AI (XAI).  These explanations seem to offer technical, psychological and legal benefits over other explanation techniques. We survey 100 distinct counterfactual explanation methods reported in the literature. This survey addresses the extent to which these methods have been adequately evaluated, both psychologically and computationally, and quantifies the shortfalls occurring. For instance, only 21% of these methods have been user tested. Five key deficits in the evaluation of these methods are detailed and a roadmap, with standardised benchmark evaluations, is proposed to resolve the issues arising; issues, that currently effectively block scientific progress in this field.<\/jats:p>","DOI":"10.24963\/ijcai.2021\/609","type":"proceedings-article","created":{"date-parts":[[2021,8,11]],"date-time":"2021-08-11T07:00:49Z","timestamp":1628665249000},"page":"4466-4474","source":"Crossref","is-referenced-by-count":132,"title":["If Only We Had Better Counterfactual Explanations: Five Key Deficits to Rectify in the Evaluation of Counterfactual XAI Techniques"],"prefix":"10.24963","author":[{"given":"Mark T.","family":"Keane","sequence":"first","affiliation":[{"name":"School of Computer Science, University College Dublin, Belfield, Dublin 4, Ireland"},{"name":"VistaMilk SFI Research Centre, Belfield, Dublin 4, Ireland"},{"name":"Insight Centre for Data Analytics, Belfield, Dublin 4, Ireland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Eoin M.","family":"Kenny","sequence":"additional","affiliation":[{"name":"School of Computer Science, University College Dublin, Belfield, Dublin 4, Ireland"},{"name":"VistaMilk SFI Research Centre, Belfield, Dublin 4, Ireland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Eoin","family":"Delaney","sequence":"additional","affiliation":[{"name":"School of Computer Science, University College Dublin, Belfield, Dublin 4, Ireland"},{"name":"VistaMilk SFI Research Centre, Belfield, Dublin 4, Ireland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Barry","family":"Smyth","sequence":"additional","affiliation":[{"name":"School of Computer Science, University College Dublin, Belfield, Dublin 4, Ireland"},{"name":"Insight Centre for Data Analytics, Belfield, Dublin 4, Ireland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"10584","event":{"name":"Thirtieth International Joint Conference on Artificial Intelligence {IJCAI-21}","theme":"Artificial Intelligence","location":"Montreal, Canada","acronym":"IJCAI-2021","number":"30","sponsor":["International Joint Conferences on Artificial Intelligence Organization (IJCAI)"],"start":{"date-parts":[[2021,8,19]]},"end":{"date-parts":[[2021,8,27]]}},"container-title":["Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence"],"original-title":[],"deposited":{"date-parts":[[2021,8,11]],"date-time":"2021-08-11T07:04:18Z","timestamp":1628665458000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.ijcai.org\/proceedings\/2021\/609"}},"subtitle":[],"proceedings-subject":"Artificial Intelligence Research Articles","short-title":[],"issued":{"date-parts":[[2021,8]]},"references-count":0,"URL":"https:\/\/doi.org\/10.24963\/ijcai.2021\/609","relation":{},"subject":[],"published":{"date-parts":[[2021,8]]}}}