{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,26]],"date-time":"2026-02-26T11:08:39Z","timestamp":1772104119447,"version":"3.50.1"},"reference-count":22,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2021,1,20]],"date-time":"2021-01-20T00:00:00Z","timestamp":1611100800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["EP\/R511559\/1,EP\/S027238\/1"],"award-info":[{"award-number":["EP\/R511559\/1,EP\/S027238\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Digit. Gov.: Res. Pract."],"published-print":{"date-parts":[[2021,4,30]]},"abstract":"<jats:p>AI-based automated decisions are increasingly used as part of new services being deployed to the general public. This approach to building services presents significant potential benefits, such as the reduced speed of execution, increased accuracy, lower cost, and ability to adapt to a wide variety of situations. However, equally significant concerns have been raised and are now well documented such as concerns about privacy, fairness, bias, and ethics. On the consumer side, more often than not, the users of those services are provided with no or inadequate explanations for decisions that may impact their lives. In this article, we report the experience of developing a socio-technical approach to constructing explanations for such decisions from their audit trails, or provenance, in an automated manner. The work has been carried out in collaboration with the UK Information Commissioner\u2019s Office. In particular, we have implemented an automated Loan Decision scenario, instrumented its decision pipeline to record provenance, categorized relevant explanations according to their audience and their regulatory purposes, built an explanation-generation prototype, and deployed the whole system in an online demonstrator.<\/jats:p>","DOI":"10.1145\/3436897","type":"journal-article","created":{"date-parts":[[2020,11,26]],"date-time":"2020-11-26T18:15:56Z","timestamp":1606414556000},"page":"1-14","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":12,"title":["Addressing Regulatory Requirements on Explanations for Automated Decisions with Provenance\u2014A Case Study"],"prefix":"10.1145","volume":"2","author":[{"given":"Trung Dong","family":"Huynh","sequence":"first","affiliation":[{"name":"King\u2019s College London, United Kingdom"}]},{"given":"Niko","family":"Tsakalakis","sequence":"additional","affiliation":[{"name":"University of Southampton, Southampton, United Kingdom"}]},{"given":"Ayah","family":"Helal","sequence":"additional","affiliation":[{"name":"King\u2019s College London, United Kingdom"}]},{"given":"Sophie","family":"Stalla-Bourdillon","sequence":"additional","affiliation":[{"name":"University of Southampton, Southampton, United Kingdom"}]},{"given":"Luc","family":"Moreau","sequence":"additional","affiliation":[{"name":"King\u2019s College London, United Kingdom"}]}],"member":"320","published-online":{"date-parts":[[2021,1,20]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2870052"},{"key":"e_1_2_1_2_1","first-page":"1","article-title":"Combining explanation and argumentation in dialogue","volume":"7","author":"Bex Floris","year":"2016","unstructured":"Floris Bex and Douglas Walton . 2016 . Combining explanation and argumentation in dialogue . Argument Comput. 7 , 1 (Jul. 2016), 55--68. DOI:https:\/\/doi.org\/10.3233\/AAC-160001 10.3233\/AAC-160001 Floris Bex and Douglas Walton. 2016. Combining explanation and argumentation in dialogue. Argument Comput. 7, 1 (Jul. 2016), 55--68. DOI:https:\/\/doi.org\/10.3233\/AAC-160001","journal-title":"Argument Comput."},{"key":"e_1_2_1_3_1","volume-title":"Proceedings of Machine Learning Research: Conference on Fairness, Accountability and Transparency","volume":"81","author":"Binns Reuben","year":"2017","unstructured":"Reuben Binns . 2017 . Fairness in machine learning: Lessons from political philosophy . In Proceedings of Machine Learning Research: Conference on Fairness, Accountability and Transparency , Vol. 81 . 149--159. Reuben Binns. 2017. Fairness in machine learning: Lessons from political philosophy. In Proceedings of Machine Learning Research: Conference on Fairness, Accountability and Transparency, Vol. 81. 149--159."},{"key":"e_1_2_1_4_1","volume-title":"Beyond Explainability: A Practical Guide to Managing Risk in Machine Learning Models. Technical Report. Future of Privacy Forum.","author":"Burt Andrew","year":"2018","unstructured":"Andrew Burt , Stuart Shirrell , Brenda Leong , and Xiangnong George Wang . 2018 . Beyond Explainability: A Practical Guide to Managing Risk in Machine Learning Models. Technical Report. Future of Privacy Forum. Andrew Burt, Stuart Shirrell, Brenda Leong, and Xiangnong George Wang. 2018. Beyond Explainability: A Practical Guide to Managing Risk in Machine Learning Models. Technical Report. Future of Privacy Forum."},{"key":"e_1_2_1_5_1","unstructured":"Finale Doshi-Velez Mason Kortz Ryan Budish Chris Bavitz Sam Gershman David O\u2019Brien Kate Scott Stuart Schieber James Waldo David Weinberger Adrian Weller and Alexandra Wood. 2019. Accountability of AI Under the Law: The Role of Explanation. arxiv:1711.01134. Retrieved from https:\/\/arxiv.org\/abs\/1711.01134.  Finale Doshi-Velez Mason Kortz Ryan Budish Chris Bavitz Sam Gershman David O\u2019Brien Kate Scott Stuart Schieber James Waldo David Weinberger Adrian Weller and Alexandra Wood. 2019. Accountability of AI Under the Law: The Role of Explanation. arxiv:1711.01134. Retrieved from https:\/\/arxiv.org\/abs\/1711.01134."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2018.2701152"},{"key":"e_1_2_1_7_1","article-title":"Regulation 2016\/679 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95\/46\/EC (General Data Protection Regulation)","volume":"59","author":"Union European","year":"2016","unstructured":"European Union . 2016 . Regulation 2016\/679 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95\/46\/EC (General Data Protection Regulation) . Off. J. Eur. Union 59 , L 199 (2016), 1--88. European Union. 2016. Regulation 2016\/679 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95\/46\/EC (General Data Protection Regulation). Off. J. Eur. Union 59, L 199 (2016), 1--88.","journal-title":"Off. J. Eur. Union"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.3115\/1610195.1610208"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025171.3025193"},{"key":"e_1_2_1_10_1","first-page":"5","article-title":"A survey of methods for explaining black box models","volume":"51","author":"Guidotti Riccardo","year":"2019","unstructured":"Riccardo Guidotti , Anna Monreale , Salvatore Ruggieri , Franco Turini , Fosca Giannotti , and Dino Pedreschi . 2019 . A survey of methods for explaining black box models . Comput. Surv. 51 , 5 (Jan. 2019), 1--42. DOI:https:\/\/doi.org\/10.1145\/3236009 arxiv:1802.01933 10.1145\/3236009 Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. 2019. A survey of methods for explaining black box models. Comput. Surv. 51, 5 (Jan. 2019), 1--42. DOI:https:\/\/doi.org\/10.1145\/3236009 arxiv:1802.01933","journal-title":"Comput. Surv."},{"key":"e_1_2_1_12_1","first-page":"189","article-title":"The right to explanation, explained","volume":"34","author":"Kaminski Margot E.","year":"2019","unstructured":"Margot E. Kaminski . 2019 . The right to explanation, explained . Berk. Technol. Law J. 34 , 1 (2019), 189 -- 218 . DOI:https:\/\/doi.org\/10.15779\/Z38TD9N83H 10.15779\/Z38TD9N83H Margot E. Kaminski. 2019. The right to explanation, explained. Berk. Technol. Law J. 34, 1 (2019), 189--218. DOI:https:\/\/doi.org\/10.15779\/Z38TD9N83H","journal-title":"Berk. Technol. Law J."},{"key":"e_1_2_1_13_1","volume-title":"Proceedings of the 2nd ICAPS Workshop on Explainable Planning (XAIP\u201919)","author":"Krarup Benjamin","year":"2019","unstructured":"Benjamin Krarup , Michael Cashmore , Daniele Magazzeni , and Tim Miller . 2019 . Towards model-based contrastive explanations for explainable planning . In Proceedings of the 2nd ICAPS Workshop on Explainable Planning (XAIP\u201919) . 21--29. Benjamin Krarup, Michael Cashmore, Daniele Magazzeni, and Tim Miller. 2019. Towards model-based contrastive explanations for explainable planning. In Proceedings of the 2nd ICAPS Workshop on Explainable Planning (XAIP\u201919). 21--29."},{"key":"e_1_2_1_14_1","first-page":"I","article-title":"A unified approach to interpreting model predictions","volume":"30","author":"Lundberg Scott M.","year":"2017","unstructured":"Scott M. Lundberg and Su-In Lee . 2017 . A unified approach to interpreting model predictions . In Advances in Neural Information Processing Systems 30 , I . Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.). Curran Associates, Inc., 4765--4774. Scott M. Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems 30, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.). Curran Associates, Inc., 4765--4774.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_1_15_1","first-page":"737","article-title":"Implementing the FACT act: Self-executing provisions","volume":"60","author":"McEneney Michael F.","year":"2005","unstructured":"Michael F. McEneney and Karl F. Kaufmann . 2005 . Implementing the FACT act: Self-executing provisions . Bus. Law. 60 , 2 (2005), 737 -- 747 . Michael F. McEneney and Karl F. Kaufmann. 2005. Implementing the FACT act: Self-executing provisions. Bus. Law. 60, 2 (2005), 737--747.","journal-title":"Bus. Law."},{"key":"e_1_2_1_16_1","volume-title":"Explanation in artificial intelligence: Insights from the social sciences. Artif. Intell. 267 (Feb","author":"Miller Tim","year":"2019","unstructured":"Tim Miller . 2019. Explanation in artificial intelligence: Insights from the social sciences. Artif. Intell. 267 (Feb . 2019 ), 1--38. DOI:https:\/\/doi.org\/10.1016\/j.artint.2018.07.007 arxiv:1706.07269 10.1016\/j.artint.2018.07.007 Tim Miller. 2019. Explanation in artificial intelligence: Insights from the social sciences. Artif. Intell. 267 (Feb. 2019), 1--38. DOI:https:\/\/doi.org\/10.1016\/j.artint.2018.07.007 arxiv:1706.07269"},{"key":"e_1_2_1_17_1","unstructured":"Christoph Molnar. 2019. Interpretable Machine Learning. Retrieved from https:\/\/christophm.github.io\/interpretable-ml-book\/.  Christoph Molnar. 2019. Interpretable Machine Learning. Retrieved from https:\/\/christophm.github.io\/interpretable-ml-book\/."},{"key":"e_1_2_1_18_1","first-page":"2","article-title":"A templating system to generate provenance","volume":"44","author":"Moreau Luc","year":"2018","unstructured":"Luc Moreau , Belfrit Victor Batlajery , Trung Dong Huynh , Danius Michaelides , and Heather Packer . 2018 . A templating system to generate provenance . IEEE Trans. Softw. Eng. 44 , 2 (Feb. 2018), 103--121. DOI:https:\/\/doi.org\/10.1109\/TSE.2017.2659745 10.1109\/TSE.2017.2659745 Luc Moreau, Belfrit Victor Batlajery, Trung Dong Huynh, Danius Michaelides, and Heather Packer. 2018. A templating system to generate provenance. IEEE Trans. Softw. Eng. 44, 2 (Feb. 2018), 103--121. DOI:https:\/\/doi.org\/10.1109\/TSE.2017.2659745","journal-title":"IEEE Trans. Softw. Eng."},{"key":"e_1_2_1_20_1","volume-title":"Proceedings of the International Conference on Provenance and Annotation of Data and Processes (IPAW\u201916)","volume":"9672","author":"Darren","unstructured":"Darren P. Richardson and Luc Moreau. 2016. Towards the domain agnostic generation of natural language explanations from provenance graphs for casual users . In Proceedings of the International Conference on Provenance and Annotation of Data and Processes (IPAW\u201916) , Marta Mattoso and Boris Glavic (Eds.). Lecture Notes in Computer Science , Vol. 9672 . Springer, Cham, 95--106. DOI:https:\/\/doi.org\/10.1007\/978-3-319-40593-3_8 10.1007\/978-3-319-40593-3_8 Darren P. Richardson and Luc Moreau. 2016. Towards the domain agnostic generation of natural language explanations from provenance graphs for casual users. In Proceedings of the International Conference on Provenance and Annotation of Data and Processes (IPAW\u201916), Marta Mattoso and Boris Glavic (Eds.). Lecture Notes in Computer Science, Vol. 9672. Springer, Cham, 95--106. DOI:https:\/\/doi.org\/10.1007\/978-3-319-40593-3_8"},{"key":"e_1_2_1_21_1","volume-title":"Robinson","author":"Rieke Aaron","year":"2018","unstructured":"Aaron Rieke , Miranda Bogen , and David G . Robinson . 2018 . Public Scrutiny of Automated Decisions: Early Lessons and Emerging Methods. Technical Report. Upturn and Omidyar Network . Aaron Rieke, Miranda Bogen, and David G. Robinson. 2018. Public Scrutiny of Automated Decisions: Early Lessons and Emerging Methods. Technical Report. Upturn and Omidyar Network."},{"key":"e_1_2_1_22_1","volume-title":"Explainable AI: The Basics.","author":"The Royal Society","year":"2019","unstructured":"The Royal Society . 2019 . Explainable AI: The Basics. Retrieved from https:\/\/royalsociety.org\/topics-policy\/projects\/explainable-ai\/. The Royal Society. 2019. Explainable AI: The Basics. Retrieved from https:\/\/royalsociety.org\/topics-policy\/projects\/explainable-ai\/."},{"key":"e_1_2_1_24_1","article-title":"Counterfactual explanations without opening the black box: Automated decisions and the GDPR","volume":"31","author":"Wachter Sandra","year":"2017","unstructured":"Sandra Wachter , Brent Mittelstadt , and Chris Russell . 2017 . Counterfactual explanations without opening the black box: Automated decisions and the GDPR . Harv. J. Law Technol. 31 , 2 (2017). DOI:https:\/\/doi.org\/10.2139\/ssrn.3063289 10.2139\/ssrn.3063289 Sandra Wachter, Brent Mittelstadt, and Chris Russell. 2017. Counterfactual explanations without opening the black box: Automated decisions and the GDPR. Harv. J. Law Technol. 31, 2 (2017). DOI:https:\/\/doi.org\/10.2139\/ssrn.3063289","journal-title":"Harv. J. Law Technol."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1177\/0162243915605575"}],"container-title":["Digital Government: Research and Practice"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3436897","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3436897","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:02:20Z","timestamp":1750197740000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3436897"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,1,20]]},"references-count":22,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2021,4,30]]}},"alternative-id":["10.1145\/3436897"],"URL":"https:\/\/doi.org\/10.1145\/3436897","relation":{},"ISSN":["2691-199X","2639-0175"],"issn-type":[{"value":"2691-199X","type":"print"},{"value":"2639-0175","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,1,20]]},"assertion":[{"value":"2020-02-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-11-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-01-20","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}