{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T03:59:30Z","timestamp":1777435170087,"version":"3.51.4"},"publisher-location":"New York, NY, USA","reference-count":33,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,9,15]],"date-time":"2021-09-15T00:00:00Z","timestamp":1631664000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-sa\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,9,15]]},"DOI":"10.1145\/3488466.3488479","type":"proceedings-article","created":{"date-parts":[[2022,1,8]],"date-time":"2022-01-08T17:29:17Z","timestamp":1641662957000},"page":"110-116","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":16,"title":["Explaining transformer-based models for automatic short answer grading"],"prefix":"10.1145","author":[{"given":"Andrew","family":"Poulton","sequence":"first","affiliation":[{"name":"Data&Research, Alef Education, UAE"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sebas","family":"Eliens","sequence":"additional","affiliation":[{"name":"Data&Research, Alef Education, UAE"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,1,8]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"Semeval-2013 task 7: The joint student response analysis and 8th recognizing textual entailment challenge. Technical report","author":"Dzikovska O","year":"2013","unstructured":"[ 1 ] Myroslava\u00a0 O Dzikovska , Rodney\u00a0 D Nielsen , Chris Brew , Claudia Leacock , Danilo Giampiccolo , Luisa Bentivogli , Peter Clark , Ido Dagan , and Hoa\u00a0 T Dang . Semeval-2013 task 7: The joint student response analysis and 8th recognizing textual entailment challenge. Technical report , 2013 . [1] Myroslava\u00a0O Dzikovska, Rodney\u00a0D Nielsen, Chris Brew, Claudia Leacock, Danilo Giampiccolo, Luisa Bentivogli, Peter Clark, Ido Dagan, and Hoa\u00a0T Dang. Semeval-2013 task 7: The joint student response analysis and 8th recognizing textual entailment challenge. Technical report, 2013."},{"key":"e_1_3_2_1_2_1","volume-title":"Autotutor and affective autotutor: Learning by talking with cognitively and emotionally intelligent computers that talk back. ACM Transactions on Interactive Intelligent Systems (TiiS), 2(4):1\u201339","author":"D\u2019mello Sidney","year":"2013","unstructured":"[ 2 ] Sidney D\u2019mello and Art Graesser . Autotutor and affective autotutor: Learning by talking with cognitively and emotionally intelligent computers that talk back. ACM Transactions on Interactive Intelligent Systems (TiiS), 2(4):1\u201339 , 2013 . [2] Sidney D\u2019mello and Art Graesser. Autotutor and affective autotutor: Learning by talking with cognitively and emotionally intelligent computers that talk back. ACM Transactions on Interactive Intelligent Systems (TiiS), 2(4):1\u201339, 2013."},{"key":"e_1_3_2_1_3_1","first-page":"12","volume-title":"International Conference on Artificial Intelligence in Education","author":"Albacete Patricia","unstructured":"[ 3 ] Patricia Albacete , Pamela Jordan , and Sandra Katz . Is a dialogue-based tutoring system that emulates helpful co-constructed relations during human tutoring effective ? In International Conference on Artificial Intelligence in Education , pages 3\u2013 12 . Springer, 2015. [3] Patricia Albacete, Pamela Jordan, and Sandra Katz. Is a dialogue-based tutoring system that emulates helpful co-constructed relations during human tutoring effective? In International Conference on Artificial Intelligence in Education, pages 3\u201312. Springer, 2015."},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2556325.2567885"},{"key":"e_1_3_2_1_5_1","first-page":"483","volume-title":"International Conference on Artificial Intelligence in Education","author":"Ventura Matthew","unstructured":"[ 5 ] Matthew Ventura , Maria Chang , Peter Foltz , Nirmal Mukhi , Jessica Yarbro , Anne\u00a0Pier Salverda , John Behrens , Jae-wook Ahn, Tengfei Ma , Tejas\u00a0 I Dhamecha , et\u00a0al. Preliminary evaluations of a dialogue-based digital tutor . In International Conference on Artificial Intelligence in Education , pages 480\u2013 483 . Springer, 2018. [5] Matthew Ventura, Maria Chang, Peter Foltz, Nirmal Mukhi, Jessica Yarbro, Anne\u00a0Pier Salverda, John Behrens, Jae-wook Ahn, Tengfei Ma, Tejas\u00a0I Dhamecha, et\u00a0al. Preliminary evaluations of a dialogue-based digital tutor. In International Conference on Artificial Intelligence in Education, pages 480\u2013483. Springer, 2018."},{"key":"e_1_3_2_1_6_1","volume-title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Technical report","author":"Devlin Jacob","year":"2018","unstructured":"[ 6 ] Jacob Devlin , Ming-Wei Chang , Kenton Lee , and Kristina Toutanova . BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Technical report , 2018 . arXiv:1810.04805. [6] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Technical report, 2018. arXiv:1810.04805."},{"key":"e_1_3_2_1_7_1","first-page":"589","volume-title":"Direct Transfer of Learned Information Among Neural Networks. AAAI-91 Proceedings","year":"1991","unstructured":"[ 7 ] Direct Transfer of Learned Information Among Neural Networks. AAAI-91 Proceedings , pages 584\u2013 589 , 1991 . URL: www.aaai.org. [7] Direct Transfer of Learned Information Among Neural Networks. AAAI-91 Proceedings, pages 584\u2013589, 1991. URL: www.aaai.org."},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1111\/jcal.12398"},{"key":"e_1_3_2_1_9_1","first-page":"190","volume-title":"International Conference on Artificial Intelligence in Education","author":"Filighera Anna","unstructured":"[ 9 ] Anna Filighera , Tim Steuer , and Christoph Rensing . Fooling automatic short answer grading systems . In International Conference on Artificial Intelligence in Education , pages 177\u2013 190 . Springer, 2020. [9] Anna Filighera, Tim Steuer, and Christoph Rensing. Fooling automatic short answer grading systems. In International Conference on Artificial Intelligence in Education, pages 177\u2013190. Springer, 2020."},{"key":"e_1_3_2_1_10_1","first-page":"481","volume-title":"International Conference on Artificial Intelligence in Education","author":"Sung Chul","unstructured":"[ 10 ] Chul Sung , Tejas\u00a0Indulal Dhamecha , and Nirmal Mukhi . Improving short answer grading using transformer-based pre-training . In International Conference on Artificial Intelligence in Education , pages 469\u2013 481 . Springer, 2019. [10] Chul Sung, Tejas\u00a0Indulal Dhamecha, and Nirmal Mukhi. Improving short answer grading using transformer-based pre-training. In International Conference on Artificial Intelligence in Education, pages 469\u2013481. Springer, 2019."},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-52240-7_8"},{"key":"e_1_3_2_1_12_1","volume-title":"apr","author":"Narang Sharan","year":"2020","unstructured":"[ 12 ] Sharan Narang , Colin Raffel , Katherine Lee , Adam Roberts , Noah Fiedel , and Karishma Malkan . WT5?! training text-to-text models to explain their predictions , apr 2020 . URL : https:\/\/arxiv.org\/abs\/2004.14546, arXiv:2004.14546. [12] Sharan Narang, Colin Raffel, Katherine Lee, Adam Roberts, Noah Fiedel, and Karishma Malkan. WT5?! training text-to-text models to explain their predictions, apr 2020. URL: https:\/\/arxiv.org\/abs\/2004.14546, arXiv:2004.14546."},{"key":"e_1_3_2_1_13_1","volume-title":"Comparative evaluation of pretrained transfer learning models on automatic short answer grading. arXiv preprint arXiv:2009.01303","author":"Gaddipati Sasi\u00a0Kiran","year":"2020","unstructured":"[ 13 ] Sasi\u00a0Kiran Gaddipati , Deebul Nair , and Paul\u00a0 G Pl\u00f6ger . Comparative evaluation of pretrained transfer learning models on automatic short answer grading. arXiv preprint arXiv:2009.01303 , 2020 . [13] Sasi\u00a0Kiran Gaddipati, Deebul Nair, and Paul\u00a0G Pl\u00f6ger. Comparative evaluation of pretrained transfer learning models on automatic short answer grading. arXiv preprint arXiv:2009.01303, 2020."},{"key":"e_1_3_2_1_14_1","first-page":"762","volume-title":"Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies","author":"Mohler Michael","year":"2011","unstructured":"[ 14 ] Michael Mohler , Razvan Bunescu , and Rada Mihalcea . Learning to grade short answer questions using semantic similarity measures and dependency graph alignments . In Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies , pages 752\u2013 762 , 2011 . [14] Michael Mohler, Razvan Bunescu, and Rada Mihalcea. Learning to grade short answer questions using semantic similarity measures and dependency graph alignments. In Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies, pages 752\u2013762, 2011."},{"key":"e_1_3_2_1_15_1","volume-title":"A diagnostic study of explainability techniques for text classification. arXiv preprint arXiv:2009.13295","author":"Atanasova Pepa","year":"2020","unstructured":"[ 15 ] Pepa Atanasova , Jakob\u00a0Grue Simonsen , Christina Lioma , and Isabelle Augenstein . A diagnostic study of explainability techniques for text classification. arXiv preprint arXiv:2009.13295 , 2020 . [15] Pepa Atanasova, Jakob\u00a0Grue Simonsen, Christina Lioma, and Isabelle Augenstein. A diagnostic study of explainability techniques for text classification. arXiv preprint arXiv:2009.13295, 2020."},{"key":"e_1_3_2_1_16_1","unstructured":"[\n  16\n  ]  Dan Hendrycks and Kevin Gimpel. Gaussian Error Linear Units (GELUS). Technical report. arXiv:1606.08415v3.  [16] Dan Hendrycks and Kevin Gimpel. Gaussian Error Linear Units (GELUS). Technical report. arXiv:1606.08415v3."},{"key":"e_1_3_2_1_17_1","volume-title":"Technical report","author":"Ba Jimmy\u00a0Lei","year":"2016","unstructured":"[ 17 ] Jimmy\u00a0Lei Ba , Jamie\u00a0Ryan Kiros , and Geoffrey\u00a0 E. Hinton . Layer Normalization . Technical report , 2016 . URL : http:\/\/arxiv.org\/abs\/1607.06450, arXiv:1607.06450. [17] Jimmy\u00a0Lei Ba, Jamie\u00a0Ryan Kiros, and Geoffrey\u00a0E. Hinton. Layer Normalization. Technical report, 2016. URL: http:\/\/arxiv.org\/abs\/1607.06450, arXiv:1607.06450."},{"key":"e_1_3_2_1_18_1","unstructured":"[\n  18\n  ]  Ashish Vaswani Google Brain Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan\u00a0N Gomez \u0141ukasz Kaiser and Illia Polosukhin. Attention Is All You Need. Technical report. URL: https:\/\/arxiv.org\/pdf\/1706.03762.pdf arXiv:1706.03762v5.  [18] Ashish Vaswani Google Brain Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan\u00a0N Gomez \u0141ukasz Kaiser and Illia Polosukhin. Attention Is All You Need. Technical report. URL: https:\/\/arxiv.org\/pdf\/1706.03762.pdf arXiv:1706.03762v5."},{"key":"e_1_3_2_1_19_1","volume-title":"RoBERTa: A Robustly Optimized BERT Pretraining Approach. Technical report","author":"Liu Yinhan","year":"2019","unstructured":"[ 19 ] Yinhan Liu , Myle Ott , Naman Goyal , Jingfei Du , Mandar Joshi , Danqi Chen , Omer Levy , Mike Lewis , Luke Zettlemoyer , Veselin Stoyanov , and Paul\u00a0 G Allen . RoBERTa: A Robustly Optimized BERT Pretraining Approach. Technical report , 2019 . URL : https:\/\/github.com\/pytorch\/fairseq, arXiv:1907.11692v1. [19] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, Veselin Stoyanov, and Paul\u00a0G Allen. RoBERTa: A Robustly Optimized BERT Pretraining Approach. Technical report, 2019. URL: https:\/\/github.com\/pytorch\/fairseq, arXiv:1907.11692v1."},{"key":"e_1_3_2_1_20_1","volume-title":"a distilled version of bert: smaller, faster, cheaper and lighter","author":"Sanh Victor","year":"2020","unstructured":"[ 20 ] Victor Sanh , Lysandre Debut , Julien Chaumond , and Thomas Wolf . Distilbert , a distilled version of bert: smaller, faster, cheaper and lighter , 2020 . arXiv:1910.01108. [20] Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter, 2020. arXiv:1910.01108."},{"key":"e_1_3_2_1_21_1","unstructured":"[\n  21\n  ]  Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut and Google Research. ALBERT: A Lite Bert for Self-Supervised Learning of Language Representations. Technical report. arXiv:1909.11942v3.  [21] Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut and Google Research. ALBERT: A Lite Bert for Self-Supervised Learning of Language Representations. Technical report. arXiv:1909.11942v3."},{"key":"e_1_3_2_1_22_1","volume-title":"NIPS Deep Learning and Representation Learning Workshop","author":"Hinton Geoffrey","year":"2015","unstructured":"[ 22 ] Geoffrey Hinton , Oriol Vinyals , and Jeffrey Dean . Distilling the knowledge in a neural network . In NIPS Deep Learning and Representation Learning Workshop , 2015 . URL: http:\/\/arxiv.org\/abs\/1503.02531. [22] Geoffrey Hinton, Oriol Vinyals, and Jeffrey Dean. Distilling the knowledge in a neural network. In NIPS Deep Learning and Representation Learning Workshop, 2015. URL: http:\/\/arxiv.org\/abs\/1503.02531."},{"key":"e_1_3_2_1_23_1","first-page":"789","volume-title":"Know what you don\u2019t know: Unanswerable questions for squad","author":"Rajpurkar Pranav","year":"2018","unstructured":"[ 23 ] Pranav Rajpurkar , Robin Jia , and Percy Liang . Know what you don\u2019t know: Unanswerable questions for squad . pages 784\u2013 789 , 01 2018 . doi:10.18653\/v1\/P18-2124. [23] Pranav Rajpurkar, Robin Jia, and Percy Liang. Know what you don\u2019t know: Unanswerable questions for squad. pages 784\u2013789, 01 2018. doi:10.18653\/v1\/P18-2124."},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P16-1162"},{"key":"e_1_3_2_1_25_1","unstructured":"[\n  25\n  ]  Karen Simonyan Andrea Vedaldi and Andrew Zisserman. Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps. Technical report. URL: http:\/\/code.google.com\/p\/cuda-convnet\/ arXiv:1312.6034v2.  [25] Karen Simonyan Andrea Vedaldi and Andrew Zisserman. Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps. Technical report. URL: http:\/\/code.google.com\/p\/cuda-convnet\/ arXiv:1312.6034v2."},{"key":"e_1_3_2_1_26_1","unstructured":"[\n  26\n  ]  Avanti Shrikumar Peyton Greenside and Anshul Kundaje. Learning Important Features Through Propagating Activation Differences. Technical report. URL: http:\/\/goo.gl\/qKb7pL arXiv:1704.02685v2.  [26] Avanti Shrikumar Peyton Greenside and Anshul Kundaje. Learning Important Features Through Propagating Activation Differences. Technical report. URL: http:\/\/goo.gl\/qKb7pL arXiv:1704.02685v2."},{"key":"e_1_3_2_1_27_1","first-page":"3328","volume-title":"International Conference on Machine Learning","author":"Sundararajan Mukund","unstructured":"[ 27 ] Mukund Sundararajan , Ankur Taly , and Qiqi Yan . Axiomatic attribution for deep networks . In International Conference on Machine Learning , pages 3319\u2013 3328 . PMLR, 2017. [27] Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International Conference on Machine Learning, pages 3319\u20133328. PMLR, 2017."},{"key":"e_1_3_2_1_28_1","first-page":"833","volume-title":"Computer Vision \u2013 ECCV","author":"D.","year":"2014","unstructured":"[ 28 ] Matthew\u00a0 D. Zeiler and Rob Fergus. Visualizing and understanding convolutional networks . In David Fleet, Tomas Pajdla, Bernt Schiele, and Tinne Tuytelaars, editors, Computer Vision \u2013 ECCV 2014 , pages 818\u2013 833 , Cham, 2014. Springer International Publishing . [28] Matthew\u00a0D. Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In David Fleet, Tomas Pajdla, Bernt Schiele, and Tinne Tuytelaars, editors, Computer Vision \u2013 ECCV 2014, pages 818\u2013833, Cham, 2014. Springer International Publishing."},{"key":"e_1_3_2_1_29_1","volume-title":"I.\u00a0Guyon, U.\u00a0V. Luxburg, S.\u00a0Bengio, H.\u00a0Wallach, R.\u00a0Fergus, S.\u00a0Vishwanathan, and R.\u00a0Garnett","author":"Lundberg M","year":"2017","unstructured":"[ 29 ] Scott\u00a0 M Lundberg and Su-In Lee . A unified approach to interpreting model predictions . In I.\u00a0Guyon, U.\u00a0V. Luxburg, S.\u00a0Bengio, H.\u00a0Wallach, R.\u00a0Fergus, S.\u00a0Vishwanathan, and R.\u00a0Garnett , editors, Advances in Neural Information Processing Systems, volume\u00a030. Curran Associates, Inc ., 2017 . [29] Scott\u00a0M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In I.\u00a0Guyon, U.\u00a0V. Luxburg, S.\u00a0Bengio, H.\u00a0Wallach, R.\u00a0Fergus, S.\u00a0Vishwanathan, and R.\u00a0Garnett, editors, Advances in Neural Information Processing Systems, volume\u00a030. Curran Associates, Inc., 2017."},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939778"},{"key":"e_1_3_2_1_31_1","first-page":"3153","volume-title":"International Conference on Machine Learning","author":"Shrikumar Avanti","unstructured":"[ 31 ] Avanti Shrikumar , Peyton Greenside , and Anshul Kundaje . Learning important features through propagating activation differences . In International Conference on Machine Learning , pages 3145\u2013 3153 . PMLR, 2017. [31] Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. In International Conference on Machine Learning, pages 3145\u20133153. PMLR, 2017."},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0130140"},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-demos.6"}],"event":{"name":"ICDTE 2021: 2021 5th International Conference on Digital Technology in Education","location":"Busan Republic of Korea","acronym":"ICDTE 2021"},"container-title":["2021 5th International Conference on Digital Technology in Education"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3488466.3488479","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3488466.3488479","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:30:25Z","timestamp":1750188625000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3488466.3488479"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,15]]},"references-count":33,"alternative-id":["10.1145\/3488466.3488479","10.1145\/3488466"],"URL":"https:\/\/doi.org\/10.1145\/3488466.3488479","relation":{},"subject":[],"published":{"date-parts":[[2021,9,15]]},"assertion":[{"value":"2022-01-08","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}