{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T08:44:48Z","timestamp":1790757888051,"version":"4.1.0"},"reference-count":33,"publisher":"MIT Press","license":[{"start":{"date-parts":[[2022,4,7]],"date-time":"2022-04-07T00:00:00Z","timestamp":1649289600000},"content-version":"vor","delay-in-days":96,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,4,6]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>While many methods purport to explain predictions by highlighting salient features, what aims these explanations serve and how they ought to be evaluated often go unstated. In this work, we introduce a framework to quantify the value of explanations via the accuracy gains that they confer on a student model trained to simulate a teacher model. Crucially, the explanations are available to the student during training, but are not available at test time. Compared with prior proposals, our approach is less easily gamed, enabling principled, automatic, model-agnostic evaluation of attributions. Using our framework, we compare numerous attribution methods for text classification and question answering, and observe quantitative differences that are consistent (to a moderate to high degree) across different student model architectures and learning strategies.1<\/jats:p>","DOI":"10.1162\/tacl_a_00465","type":"journal-article","created":{"date-parts":[[2022,4,7]],"date-time":"2022-04-07T10:22:45Z","timestamp":1649326965000},"page":"359-375","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":14,"title":["Evaluating Explanations: How Much Do Explanations from the Teacher Aid Students?"],"prefix":"10.1162","volume":"10","author":[{"given":"Danish","family":"Pruthi","sequence":"first","affiliation":[{"name":"Carnegie Mellon University, USA. ddanish@cs.cmu.edu"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rachit","family":"Bansal","sequence":"additional","affiliation":[{"name":"Delhi Technological University, India. racbansa@gmail.com"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bhuwan","family":"Dhingra","sequence":"additional","affiliation":[{"name":"Google Research, USA. bdhingra@google.com"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Livio Baldini","family":"Soares","sequence":"additional","affiliation":[{"name":"Google Research, USA. liviobs@google.com"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael","family":"Collins","sequence":"additional","affiliation":[{"name":"Google Research, USA. mjcollins@google.com"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zachary C.","family":"Lipton","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, USA. zlipton@cs.cmu.edu"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Graham","family":"Neubig","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, USA. gneubig@cs.cmu.edu"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"William W.","family":"Cohen","sequence":"additional","affiliation":[{"name":"Google Research, USA. wcohen@google.com"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","published-online":{"date-parts":[[2022,4,6]]},"reference":[{"key":"2022040714222643100_bib1","article-title":"Neural machine translation by jointly learning to align and translate","author":"Bahdanau","year":"2015","journal-title":"3rd International Conference on Learning Representations, ICLR"},{"key":"2022040714222643100_bib2","doi-asserted-by":"publisher","first-page":"1903","DOI":"10.18653\/v1\/D18-1216","article-title":"Deriving machine attention from human rationales","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Bao","year":"2018"},{"key":"2022040714222643100_bib3","doi-asserted-by":"publisher","first-page":"1036","DOI":"10.18653\/v1\/D18-1128","article-title":"Do explanations make VQA models more predictable to a human?","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Chandrasekaran","year":"2018"},{"key":"2022040714222643100_bib4","article-title":"Reading tea leaves: How humans interpret topic models","volume-title":"Advances in Neural Information Processing Systems","author":"Chang","year":"2009"},{"key":"2022040714222643100_bib5","first-page":"1448","article-title":"Invariant rationalization","volume-title":"Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event","author":"Chang","year":"2020"},{"key":"2022040714222643100_bib6","first-page":"4171","article-title":"BERT: Pre-training of deep bidirectional transformers for language understanding","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Devlin","year":"2019"},{"key":"2022040714222643100_bib7","doi-asserted-by":"publisher","first-page":"4443","DOI":"10.18653\/v1\/2020.acl-main.408","article-title":"ERASER: A benchmark to evaluate rationalized NLP models","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"DeYoung","year":"2020"},{"key":"2022040714222643100_bib8","article-title":"How important is a neuron","volume-title":"7th International Conference on Learning Representations, ICLR","author":"Dhamdhere","year":"2019"},{"key":"2022040714222643100_bib9","article-title":"Towards a rigorous science of interpretable machine learning","author":"Doshi-Velez","year":"2017","journal-title":"arXiv preprint arXiv:1702.08608"},{"key":"2022040714222643100_bib10","doi-asserted-by":"publisher","first-page":"5540","DOI":"10.18653\/v1\/2020.acl-main.491","article-title":"Evaluating explainable AI: Which algorithmic explanations help users predict model behavior?","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Hase","year":"2020"},{"key":"2022040714222643100_bib11","doi-asserted-by":"crossref","first-page":"4351","DOI":"10.18653\/v1\/2020.findings-emnlp.390","article-title":"Leakage-adjusted simulatability: Can models generate non-trivial explanations of their behavior in natural language?","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2020","author":"Hase","year":"2020"},{"key":"2022040714222643100_bib12","doi-asserted-by":"publisher","first-page":"9737","DOI":"10.18653\/v1\/2020.findings-emnlp.390","article-title":"A benchmark for interpretability methods in deep neural networks","volume-title":"Advances in Neural Information Processing Systems","author":"Hooker","year":"2019"},{"key":"2022040714222643100_bib13","doi-asserted-by":"publisher","first-page":"294","DOI":"10.1162\/tacl_a_00367","article-title":"Aligning Faithful Interpretations with their Social Attribution","volume":"9","author":"Jacovi","year":"2021","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2022040714222643100_bib14","doi-asserted-by":"publisher","first-page":"64","DOI":"10.1162\/tacl_a_00300","article-title":"SpanBERT: Improving pre-training by representing and predicting spans","volume":"8","author":"Joshi","year":"2020","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2022040714222643100_bib15","doi-asserted-by":"publisher","first-page":"453","DOI":"10.1162\/tacl_a_00276","article-title":"Natural questions: a benchmark for question answering research","volume":"7","author":"Kwiatkowski","year":"2019","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2022040714222643100_bib16","first-page":"282","article-title":"Conditional random fields: Probabilistic models for segmenting and labeling sequence data","author":"Lafferty","year":"2001","journal-title":"18th International Conference on Machine Learning 2001 (ICML 2001)"},{"key":"2022040714222643100_bib17","doi-asserted-by":"publisher","first-page":"790","DOI":"10.1162\/tacl_a_00398","article-title":"QED: A framework and dataset for explanations in question answering","volume":"9","author":"Lamm","year":"2021","journal-title":"Transactions of the Association for Computational Linguistics"},{"issue":"3","key":"2022040714222643100_bib18","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1145\/3236386.3241340","article-title":"The mythos of model interpretability","volume":"16","author":"Lipton","year":"2016","journal-title":"ACM Queue"},{"key":"2022040714222643100_bib19","first-page":"142","article-title":"Learning word vectors for sentiment analysis","volume-title":"Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies","author":"Maas","year":"2011"},{"key":"2022040714222643100_bib20","volume-title":"The Enigma of Reason","author":"Mercier","year":"2017"},{"key":"2022040714222643100_bib21","doi-asserted-by":"publisher","first-page":"340","DOI":"10.18653\/v1\/P18-1032","article-title":"Evaluating neural network explanation methods using hybrid documents and morphosyntactic agreement","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Poerner","year":"2018"},{"key":"2022040714222643100_bib22","doi-asserted-by":"publisher","first-page":"3965","DOI":"10.18653\/v1\/2020.findings-emnlp.353","article-title":"Weakly-and semi-supervised evidence extraction","author":"Pruthi","year":"2020","journal-title":"Findings of the Association for Computational Linguistics: EMNLP 2020"},{"key":"2022040714222643100_bib23","first-page":"113","article-title":"Meta-learning with implicit gradients","volume-title":"Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems","author":"Rajeswaran","year":"2019"},{"key":"2022040714222643100_bib24","doi-asserted-by":"crossref","first-page":"1135","DOI":"10.1145\/2939672.2939778","article-title":"\u201cwhy should I trust you?\u201d: Explaining the predictions of any classifier","volume-title":"Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","author":"Ribeiro","year":"2016"},{"key":"2022040714222643100_bib25","first-page":"3145","article-title":"Learning important features through propagating activation differences","volume-title":"Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6\u201311 August 2017","author":"Shrikumar","year":"2017"},{"key":"2022040714222643100_bib26","article-title":"Deep inside convolutional networks: Visualising image classification models and saliency maps","volume-title":"2nd International Conference on Learning Representations, ICLR","author":"Simonyan","year":"2014"},{"key":"2022040714222643100_bib27","first-page":"3319","article-title":"Axiomatic attribution for deep networks","volume-title":"Proceedings of the 34th International Conference on Machine Learning, ICML","author":"Sundararajan","year":"2017"},{"key":"2022040714222643100_bib28","doi-asserted-by":"publisher","first-page":"107","DOI":"10.18653\/v1\/2020.blackboxnlp-1.10","article-title":"The explanation game: Towards prediction explainability through sparse communication","volume-title":"Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, BlackboxNLP@EMNLP 2020, Online, November 2020","author":"Treviso","year":"2020"},{"key":"2022040714222643100_bib29","first-page":"5998","article-title":"Attention is all you need","volume-title":"Advances in neural information processing systems","author":"Vaswani","year":"2017"},{"key":"2022040714222643100_bib30","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.acl-long.65","article-title":"Do context-aware translation models pay the right attention?","volume-title":"Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-IJCNLP)","author":"Yin","year":"2021"},{"key":"2022040714222643100_bib31","doi-asserted-by":"publisher","first-page":"31","DOI":"10.3115\/1613715.1613721","article-title":"Modeling annotators: A generative approach to learning from annotator rationales","volume-title":"Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing","author":"Zaidan","year":"2008"},{"key":"2022040714222643100_bib32","first-page":"260","article-title":"Using \u201cannotator rationales\u201d to improve machine learning for text categorization","volume-title":"Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference","author":"Zaidan","year":"2007"},{"key":"2022040714222643100_bib33","article-title":"Fine-grained sentiment analysis with faithful attention","author":"Zhong","year":"2019","journal-title":"arXiv preprint arXiv:1908.06870"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00465\/2006971\/tacl_a_00465.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00465\/2006971\/tacl_a_00465.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,4,7]],"date-time":"2022-04-07T10:23:04Z","timestamp":1649326984000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/doi\/10.1162\/tacl_a_00465\/110436\/Evaluating-Explanations-How-Much-Do-Explanations"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022]]},"references-count":33,"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00465","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022]]},"published":{"date-parts":[[2022]]}}}