{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,6]],"date-time":"2026-05-06T00:52:24Z","timestamp":1778028744330,"version":"3.51.4"},"reference-count":51,"publisher":"MIT Press - Journals","license":[{"start":{"date-parts":[[2021,4,6]],"date-time":"2021-04-06T00:00:00Z","timestamp":1617667200000},"content-version":"vor","delay-in-days":95,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,3,31]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>We find that the requirement of model interpretations to be faithful is vague and incomplete. With interpretation by textual highlights as a case study, we present several failure cases. Borrowing concepts from social science, we identify that the problem is a misalignment between the causal chain of decisions (causal attribution) and the attribution of human behavior to the interpretation (social attribution). We reformulate faithfulness as an accurate attribution of causality to the model, and introduce the concept of aligned faithfulness: faithful causal chains that are aligned with their expected social behavior. The two steps of causal attribution and social attribution together complete the process of explaining behavior. With this formalization, we characterize various failures of misaligned faithful highlight interpretations, and propose an alternative causal chain to remedy the issues. Finally, we implement highlight explanations of the proposed causal format using contrastive explanations.<\/jats:p>","DOI":"10.1162\/tacl_a_00367","type":"journal-article","created":{"date-parts":[[2021,4,15]],"date-time":"2021-04-15T00:40:58Z","timestamp":1618447258000},"page":"294-310","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":24,"title":["Aligning Faithful Interpretations with their Social Attribution"],"prefix":"10.1162","volume":"9","author":[{"given":"Alon","family":"Jacovi","sequence":"first","affiliation":[{"name":"Bar Ilan University, Israel. alonjacovi@gmail.com"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yoav","family":"Goldberg","sequence":"additional","affiliation":[{"name":"Bar Ilan University and, Allen Institute for AI, Israel. yoav.goldberg@gmail.com"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","published-online":{"date-parts":[[2021,3,31]]},"reference":[{"key":"2021060823400174500_bib1","article-title":"On the robustness of interpretability methods","author":"Alvarez-Melis","year":"2018","journal-title":"CoRR"},{"key":"2021060823400174500_bib2","article-title":"Learning to compose neural networks for question answering","author":"Andreas","year":"2016","journal-title":"CoRR"},{"key":"2021060823400174500_bib3","article-title":"Do transformer attention heads provide transparency in abstractive summarization?","author":"Baan","year":"2019","journal-title":"CoRR"},{"key":"2021060823400174500_bib4","first-page":"1903","article-title":"Deriving machine attention from human rationales","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018","author":"Bao","year":"2018"},{"key":"2021060823400174500_bib5","doi-asserted-by":"crossref","first-page":"2963","DOI":"10.18653\/v1\/P19-1284","article-title":"Interpretable neural predictions with differentiable binary variables","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Bastings","year":"2019"},{"key":"2021060823400174500_bib6","article-title":"Contrastive algorithmic fairness: Part 1 (theory)","author":"Chakraborti","year":"2019","journal-title":"ArXiv"},{"key":"2021060823400174500_bib7","first-page":"10055","article-title":"A game theoretic approach to class-wise selective rationalization","volume-title":"Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, 8\u201314 December 2019, Vancouver, BC, Canada","author":"Chang","year":"2019"},{"key":"2021060823400174500_bib8","article-title":"Invariant rationalization","author":"Chang","year":"2020","journal-title":"CoRR"},{"key":"2021060823400174500_bib9","article-title":"Towards transparent robotic planning via contrastive explanations","author":"Chen","year":"2020","journal-title":"ArXiv"},{"issue":"6","key":"2021060823400174500_bib10","doi-asserted-by":"crossref","first-page":"1645","DOI":"10.1111\/cogs.12405","article-title":"Contrastive constraints guide explanation-based category learning","volume":"41","author":"Chin-Parker","year":"2017","journal-title":"Cognitive Science"},{"key":"2021060823400174500_bib11","doi-asserted-by":"crossref","first-page":"page 510\u2013page 518","DOI":"10.1145\/3377325.3377512","article-title":"Leveraging rationales to improve human task performance","volume-title":"Proceedings of the 25th International Conference on Intelligent User Interfaces","author":"Das","year":"2020"},{"key":"2021060823400174500_bib12","article-title":"BERT: Pre-training of deep bidirectional transformers for language understanding","author":"Devlin","year":"2018","journal-title":"CoRR"},{"key":"2021060823400174500_bib13","article-title":"Eraser: A benchmark to evaluate rationalized NLP models","author":"DeYoung","year":"2019"},{"key":"2021060823400174500_bib14","article-title":"Explanations based on the missing: Towards contrastive explanations with pertinent negatives","volume-title":"NeurIPS","author":"Dhurandhar","year":"2018"},{"key":"2021060823400174500_bib15","article-title":"Towards a rigorous science of interpretable machine learning","author":"Doshi-Velez","year":"2017","journal-title":"arXiv preprint arXiv:1702.08608"},{"key":"2021060823400174500_bib16","article-title":"Causalm: Causal model explanation through counterfactual language models","author":"Feder","year":"2020","journal-title":"CoRR"},{"key":"2021060823400174500_bib17","first-page":"3719","article-title":"Pathologies of neural models make interpretation difficult","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018","author":"Feng","year":"2018"},{"key":"2021060823400174500_bib18","first-page":"1780","article-title":"Why attention is not explanation: Surgical intervention and causal reasoning about neural models","volume-title":"Proceedings of the 12th Language Resources and Evaluation Conference","author":"Grimsley","year":"2020"},{"key":"2021060823400174500_bib19","article-title":"Evaluating explainable AI: Which algorithmic explanations help users predict model behavior?","author":"Hase","year":"2020","journal-title":"CoRR"},{"key":"2021060823400174500_bib20","doi-asserted-by":"crossref","DOI":"10.1037\/10628-000","volume-title":"The Psychology of Interpersonal Relations","author":"Heider","year":"1958"},{"issue":"2","key":"2021060823400174500_bib21","doi-asserted-by":"crossref","first-page":"243","DOI":"10.2307\/1416950","article-title":"An experimental study of apparent behavior","volume":"57","author":"Heider","year":"1944","journal-title":"The American Journal of Psychology"},{"key":"2021060823400174500_bib22","article-title":"The promise and peril of human evaluation for model interpretability","author":"Herman","year":"2017","journal-title":"CoRR"},{"key":"2021060823400174500_bib23","article-title":"The problem of causal selection","volume-title":"Contemporary Science and Natural Explanation: Commonsense Conceptions of Causality","author":"Hesslow","year":"1988"},{"key":"2021060823400174500_bib24","first-page":"33","article-title":"Logic and causal attribution.","volume-title":"Contemporary Science and Natural Explanation: Commonsense Conceptions of Causality","author":"Hilton","year":"1988"},{"key":"2021060823400174500_bib25","volume-title":"The Psychology of Counterfactual Thinking","author":"Hilton","year":"2005"},{"issue":"1","key":"2021060823400174500_bib26","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1037\/0033-295X.93.1.75","article-title":"Knowledge-based causal attribution: The abnormal conditions focus model.","volume":"93","author":"Hilton","year":"1986","journal-title":"Psychological Review"},{"key":"2021060823400174500_bib27","article-title":"Towards faithfully interpretable NLP systems: How should we define and evaluate faithfulness?","author":"Jacovi","year":"2020","journal-title":"CoRR"},{"key":"2021060823400174500_bib28","article-title":"Learning to faithfully rationalize by construction","author":"Jain","year":"2020","journal-title":"CoRR"},{"key":"2021060823400174500_bib29","article-title":"Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav)","author":"Kim","year":"2017"},{"key":"2021060823400174500_bib30","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1007\/978-3-030-28954-6_14","article-title":"The (un)reliability of saliency methods","volume-title":"Explainable AI: Interpreting, Explaining and Visualizing Deep Learning","author":"Kindermans","year":"2019"},{"key":"2021060823400174500_bib31","article-title":"Auto-encoding variational bayes","volume-title":"2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14\u201316, 2014, Conference Track Proceedings","author":"Kingma","year":"2014"},{"key":"2021060823400174500_bib32","article-title":"Problems with shapley-value-based explanations as feature importance measures","author":"Elizabeth Kumar","year":"2020","journal-title":"CoRR"},{"key":"2021060823400174500_bib33","article-title":"Rationalizing neural predictions","volume":"abs\/1606.04155","author":"Lei","year":"2016","journal-title":"CoRR"},{"key":"2021060823400174500_bib34","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1017\/S1358246100005130","article-title":"Contrastive explanation","volume":"27","author":"Lipton","year":"1990","journal-title":"Royal Institute of Philosophy Supplements"},{"issue":"10","key":"2021060823400174500_bib35","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1145\/3233231","article-title":"The mythos of model interpretability","volume":"61","author":"Lipton","year":"2018","journal-title":"Communications of ACM"},{"issue":"10","key":"2021060823400174500_bib36","doi-asserted-by":"crossref","first-page":"464","DOI":"10.1016\/j.tics.2006.08.004","article-title":"The structure and function of explanations","volume":"10","author":"Lombrozo","year":"2006","journal-title":"Trends in Cognitive Sciences"},{"key":"2021060823400174500_bib37","first-page":"4765","article-title":"A unified approach to interpreting model predictions","volume-title":"Advances in Neural Information Processing Systems 30","author":"Lundberg","year":"2017"},{"key":"2021060823400174500_bib38","first-page":"2493","article-title":"Explainable reinforcement learning through a causal lens","volume-title":"The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7\u201312, 2020","author":"Madumal","year":"2020"},{"key":"2021060823400174500_bib39","doi-asserted-by":"crossref","first-page":"897","DOI":"10.1037\/0022-3514.64.6.897","article-title":"Contrastive and counterfactual thinking in causal judgment","volume":"64","author":"Mcgill","year":"1993","journal-title":"Journal of Personality and Social Psychology"},{"key":"2021060823400174500_bib40","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.artint.2018.07.007","article-title":"Explanation in artificial intelligence: Insights from the social sciences","volume":"267","author":"Miller","year":"2019","journal-title":"Artificial Intelligence"},{"key":"2021060823400174500_bib41","article-title":"Contrastive explanation: A structural-model approach","author":"Miller","year":"2020"},{"key":"2021060823400174500_bib42","article-title":"Learning to deceive with attention-based explanations","author":"Pruthi","year":"2019","journal-title":"CoRR"},{"key":"2021060823400174500_bib43","doi-asserted-by":"crossref","first-page":"1135","DOI":"10.1145\/2939672.2939778","article-title":"\u201cWhy Should I Trust You?\u201d: Explaining the predictions of any classifier","volume-title":"Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","author":"Ribeiro","year":"2016"},{"issue":"5","key":"2021060823400174500_bib44","doi-asserted-by":"crossref","first-page":"206","DOI":"10.1038\/s42256-019-0048-x","article-title":"Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead","volume":"1","author":"Rudin","year":"2019","journal-title":"Nature Machine Intelligence"},{"key":"2021060823400174500_bib45","first-page":"2931","article-title":"Is attention interpretable?","volume-title":"Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28\u2013August 2, 2019, Volume 1: Long Papers","author":"Serrano","year":"2019"},{"key":"2021060823400174500_bib46","doi-asserted-by":"crossref","first-page":"1679","DOI":"10.18653\/v1\/P19-1164","article-title":"Evaluating gender bias in machine translation","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Stanovsky","year":"2019"},{"key":"2021060823400174500_bib47","article-title":"Obtaining faithful interpretations from compositional neural networks","author":"Subramanian","year":"2020","journal-title":"CoRR"},{"key":"2021060823400174500_bib48","first-page":"11","article-title":"Attention is not not explanation","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3\u20137, 2019","author":"Wiegreffe","year":"2019"},{"issue":"4","key":"2021060823400174500_bib49","doi-asserted-by":"crossref","first-page":"1006","DOI":"10.1037\/a0030996","article-title":"The hazards of explanation: Overgeneralization in the face of exceptions.","volume":"142","author":"Williams","year":"2013","journal-title":"Journal of Experimental Psychology: General"},{"issue":"3\u20134","key":"2021060823400174500_bib50","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1007\/BF00992696","article-title":"Simple statistical gradient-following algorithms for connectionist reinforcement learning","volume":"8","author":"Williams","year":"1992","journal-title":"Machine Learning"},{"key":"2021060823400174500_bib51","first-page":"260","article-title":"Using \u201cannotator rationales\u201d to improve machine learning for text categorization","volume-title":"Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, Proceedings, April 22\u201327, 2007, Rochester, New York, USA","author":"Zaidan","year":"2007"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00367\/1923972\/tacl_a_00367.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00367\/1923972\/tacl_a_00367.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,6,9]],"date-time":"2021-06-09T09:33:29Z","timestamp":1623231209000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/doi\/10.1162\/tacl_a_00367\/98620\/Aligning-Faithful-Interpretations-with-their"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021]]},"references-count":51,"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00367","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021]]},"published":{"date-parts":[[2021]]}}}