{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,21]],"date-time":"2026-02-21T04:38:11Z","timestamp":1771648691754,"version":"3.50.1"},"reference-count":56,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2021,10,22]],"date-time":"2021-10-22T00:00:00Z","timestamp":1634860800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"US National Science Foundation","doi-asserted-by":"crossref","award":["IIS-1707498 and IIS-1838730"],"award-info":[{"award-number":["IIS-1707498 and IIS-1838730"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Amazon AWS credits"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Knowl. Discov. Data"],"published-print":{"date-parts":[[2022,6,30]]},"abstract":"<jats:p>Using attention weights to identify information that is important for models\u2019 decision making is a popular approach to interpret attention-based neural networks. This is commonly realized in practice through the generation of a heat-map for every single document based on attention weights. However, this interpretation method is fragile and it is easy to find contradictory examples. In this article, we propose a corpus-level explanation approach, which aims at capturing causal relationships between keywords and model predictions via learning the importance of keywords for predicted labels across a training corpus based on attention weights. Based on this idea, we further propose a concept-based explanation method that can automatically learn higher level concepts and their importance to model prediction tasks. Our concept-based explanation method is built upon a novel Abstraction-Aggregation Network (AAN), which can automatically cluster important keywords during an end-to-end training process. We apply these methods to the document classification task and show that they are powerful in extracting semantically meaningful keywords and concepts. Our consistency analysis results based on an attention-based Na\u00efve Bayes classifier (NBC) also demonstrate that these keywords and concepts are important for model predictions.<\/jats:p>","DOI":"10.1145\/3477539","type":"journal-article","created":{"date-parts":[[2021,10,23]],"date-time":"2021-10-23T04:28:40Z","timestamp":1634963320000},"page":"1-17","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["Corpus-level and Concept-based Explanations for Interpretable Document Classification"],"prefix":"10.1145","volume":"16","author":[{"given":"Tian","family":"Shi","sequence":"first","affiliation":[{"name":"Virginia Tech, Blacksburg, VA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xuchao","family":"Zhang","sequence":"additional","affiliation":[{"name":"Virginia Tech, Blacksburg, VA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ping","family":"Wang","sequence":"additional","affiliation":[{"name":"Virginia Tech, Blacksburg, VA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chandan K.","family":"Reddy","sequence":"additional","affiliation":[{"name":"Virginia Tech, Blacksburg, VA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,10,22]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"crossref","unstructured":"Diego Antognini and Boi Faltings. 2021. Rationalization through concepts. In Proceedings of the Findings of the Association for Computational Linguistics (ACL-IJCNLP\u201921) . Association for Computational Linguistics 761\u2013775. DOI:10.18653\/v1\/2021.findings-acl.68","DOI":"10.18653\/v1\/2021.findings-acl.68"},{"key":"e_1_3_2_3_2","unstructured":"Dzmitry Bahdanau Kyunghyun Cho and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv:1409.0473. Retrieved from https:\/\/arxiv.org\/abs\/1409.0473."},{"key":"e_1_3_2_4_2","unstructured":"Iz Beltagy Matthew E. Peters and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv:2004.05150. Retrieved from https:\/\/arxiv.org\/abs\/2004.05150."},{"key":"e_1_3_2_5_2","unstructured":"Francesco Bodria Fosca Giannotti Riccardo Guidotti Francesca Naretto Dino Pedreschi and Salvatore Rinzivillo. 2021. Benchmarking and survey of explanation methods for black box models. arXiv:2102.13076. Retrieved from https:\/\/arxiv.org\/abs\/2102.13076."},{"key":"e_1_3_2_6_2","unstructured":"Diane Bouchacourt and Ludovic Denoyer. 2019. EDUCE: Explaining model decisions through unsupervised concepts extraction. arXiv:1905.11852. Retrieved from https:\/\/arxiv.org\/abs\/1905.11852."},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-020-00265-z"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.5555\/1953048.2078186"},{"key":"e_1_3_2_9_2","unstructured":"Arun Das and Paul Rad. 2020. Opportunities and challenges in explainable artificial intelligence (XAI): A survey. arXiv:2006.11371. Retrieved from https:\/\/arxiv.org\/abs\/2006.11371."},{"key":"e_1_3_2_10_2","first-page":"4171","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 4171\u20134186. DOI:https:\/\/doi.org\/10.18653\/v1\/N19-1423"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1023\/A:1007465528199"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1537"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33013681"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.5555\/3454287.3455119"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/DSAA.2018.00018"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-1065"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/2872427.2883037"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.5555\/2969239.2969428"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3369873"},{"key":"e_1_3_2_21_2","first-page":"3543","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Jain Sarthak","year":"2019","unstructured":"Sarthak Jain and Byron C. Wallace. 2019. Attention is not explanation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 3543\u20133556. DOI:https:\/\/doi.org\/10.18653\/v1\/N19-1357"},{"key":"e_1_3_2_22_2","first-page":"2668","volume-title":"Proceedings of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research)","author":"Kim Been","year":"2018","unstructured":"Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, and Rory Sayres. 2018. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV). In Proceedings of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research), Jennifer Dy and Andreas Krause (Eds.). PMLR, 2668\u20132677. Retrieved from http:\/\/proceedings.mlr.press\/v80\/kim18d.html."},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1181"},{"key":"e_1_3_2_24_2","first-page":"267","volume-title":"The (Un)reliability of Saliency Methods","author":"Kindermans Pieter-Jan","year":"2019","unstructured":"Pieter-Jan Kindermans, Sara Hooker, Julius Adebayo, Maximilian Alber, Kristof T. Sch\u00fctt, Sven D\u00e4hne, Dumitru Erhan, and Been Kim. 2019. The (Un)reliability of Saliency Methods. Springer International Publishing, Cham, 267\u2013280. DOI:https:\/\/doi.org\/10.1007\/978-3-030-28954-6_14"},{"key":"e_1_3_2_25_2","unstructured":"Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv:1412.6980. Retrieved from https:\/\/arxiv.org\/abs\/1412.6980."},{"key":"e_1_3_2_26_2","volume-title":"Information theory and statistics","author":"Kullback Solomon","year":"1997","unstructured":"Solomon Kullback. 1997. Information theory and statistics. Courier Corporation."},{"key":"e_1_3_2_27_2","unstructured":"Zhouhan Lin Minwei Feng Cicero Nogueira dos Santos Mo Yu Bing Xiang Bowen Zhou and Yoshua Bengio. 2017. A structured self-attentive sentence embedding. arXiv:1703.03130. Retrieved from https:\/\/arxiv.org\/abs\/1703.03130."},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1631"},{"key":"e_1_3_2_29_2","unstructured":"Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy Mike Lewis Luke Zettlemoyer and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized bert pretraining approach. arXiv:1907.11692. Retrieved from https:\/\/arxiv.org\/abs\/1907.11692."},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.5555\/3295222.3295230"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D15-1166"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.5555\/2002472.2002491"},{"key":"e_1_3_2_33_2","volume-title":"Proceedings of the NeurIPS Autodiff Workshop","author":"Paszke Adam","year":"2017","unstructured":"Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in PyTorch. In Proceedings of the NeurIPS Autodiff Workshop."},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1162"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-1202"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/S16-1002"},{"key":"e_1_3_2_37_2","unstructured":"Marco Tulio Ribeiro Sameer Singh and Carlos Guestrin. 2016. Model-agnostic interpretability of machine learning. arXiv:1606.05386. Retrieved from https:\/\/arxiv.org\/abs\/1606.05386."},{"key":"e_1_3_2_38_2","unstructured":"Victor Sanh Lysandre Debut Julien Chaumond and Thomas Wolf. 2019. DistilBERT a distilled version of BERT: smaller faster cheaper and lighter. arXiv:1910.01108. Retrieved from https:\/\/arxiv.org\/abs\/1701.00133."},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1282"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3178876.3186009"},{"key":"e_1_3_2_41_2","first-page":"13815","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Shi Tian","year":"2021","unstructured":"Tian Shi, Liuqing Li, Ping Wang, and Chandan K Reddy. 2021. A simple and effective self-supervised contrastive learning framework for aspect detection. In Proceedings of the AAAI Conference on Artificial Intelligence. 13815\u201313824."},{"key":"e_1_3_2_42_2","first-page":"66","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations)","author":"Shi Tian","year":"2019","unstructured":"Tian Shi, Ping Wang, and Chandan K Reddy. 2019. LeafNATS: An open-source toolkit and live demo system for neural abstractive text summarization. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations). 66\u201371."},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2018.2865044"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.5555\/3305890.3306024"},{"key":"e_1_3_2_45_2","doi-asserted-by":"crossref","unstructured":"Erico Tjoa and Cuntai Guan. 2020. A survey on explainable artificial intelligence (XAI): Toward medical XAI. IEEE Transactions on Neural Networks and Learning Systems PP (October 2020). DOI:https:\/\/doi.org\/10.1109\/tnnls.2020.3027314","DOI":"10.1109\/TNNLS.2020.3027314"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-3007"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.5555\/2969442.2969550"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.5555\/3297863.3297977"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1002"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-demos.6"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00868"},{"key":"e_1_3_2_52_2","unstructured":"Zhilin Yang Zihang Dai Yiming Yang Jaime Carbonell Ruslan Salakhutdinov and Quoc V Le. 2019. XLNet: Generalized autoregressive pretraining for language understanding. Advances in Neural Information Processing Systems 32 (2019)."},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N16-1174"},{"key":"e_1_3_2_54_2","first-page":"20554","volume-title":"Advances in Neural Information Processing Systems","author":"Yeh Chih-Kuan","year":"2020","unstructured":"Chih-Kuan Yeh, Been Kim, Sercan Arik, Chun-Liang Li, Tomas Pfister, and Pradeep Ravikumar. 2020. On completeness-aware concept-based explanations in deep neural networks. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (Eds.), Vol. 33. Curran Associates, Inc., 20554\u201320565. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2020\/file\/ecb287ff763c169694f682af52c1f309-Paper.pdf."},{"key":"e_1_3_2_55_2","unstructured":"Mohammad Nokhbeh Zaeem and Majid Komeili. 2021. Cause and effect: Concept-based explanation of neural networks. arXiv:2105.07033. Retrieved from https:\/\/arxiv.org\/abs\/2105.07033."},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N19-1316"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01237-3_8"}],"container-title":["ACM Transactions on Knowledge Discovery from Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3477539","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3477539","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T18:10:37Z","timestamp":1750183837000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3477539"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,22]]},"references-count":56,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2022,6,30]]}},"alternative-id":["10.1145\/3477539"],"URL":"https:\/\/doi.org\/10.1145\/3477539","relation":{},"ISSN":["1556-4681","1556-472X"],"issn-type":[{"value":"1556-4681","type":"print"},{"value":"1556-472X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,22]]},"assertion":[{"value":"2020-12-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-07-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-10-22","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}