{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T16:39:47Z","timestamp":1781887187652,"version":"3.54.5"},"reference-count":76,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2024,6,24]],"date-time":"2024-06-24T00:00:00Z","timestamp":1719187200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"SoBigData++","award":["871042"],"award-info":[{"award-number":["871042"]}]},{"name":"AI4Media","award":["951911"],"award-info":[{"award-number":["951911"]}]},{"DOI":"10.13039\/501100000780","name":"European Commission","doi-asserted-by":"crossref","award":["871042"],"award-info":[{"award-number":["871042"]}],"id":[{"id":"10.13039\/501100000780","id-type":"DOI","asserted-by":"crossref"}]},{"name":"ERC-2018-ADG","award":["834756"],"award-info":[{"award-number":["834756"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["J. Comput. Cult. Herit."],"published-print":{"date-parts":[[2024,9,30]]},"abstract":"<jats:p>While a substantial amount of work has recently been devoted to improving the accuracy of computational Authorship Identification (AId) systems for textual data, little to no attention has been paid to endowing AId systems with the ability to explain the reasons behind their predictions. This substantially hinders the practical application of AId methods, since the predictions returned by such systems are hardly useful unless they are supported by suitable explanations. In this article, we explore the applicability of existing general-purpose eXplainable Artificial Intelligence (XAI) techniques to AId, with a focus on explanations addressed to scholars working in cultural heritage. In particular, we assess the relative merits of three different types of XAI techniques (feature ranking, probing, factual and counterfactual selection) on three different AId tasks (authorship attribution, authorship verification and same-authorship verification) by running experiments on real AId textual data. Our analysis shows that, while these techniques make important first steps towards XAI, more work remains to be done to provide tools that can be profitably integrated into the workflows of scholars.<\/jats:p>","DOI":"10.1145\/3654675","type":"journal-article","created":{"date-parts":[[2024,4,20]],"date-time":"2024-04-20T09:18:56Z","timestamp":1713604736000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Explainable Authorship Identification in Cultural Heritage Applications"],"prefix":"10.1145","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8351-9999","authenticated-orcid":false,"given":"Mattia","family":"Setzu","sequence":"first","affiliation":[{"name":"Universit\u00e0 di Pisa, Pisa, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5284-1771","authenticated-orcid":false,"given":"Silvia","family":"Corbara","sequence":"additional","affiliation":[{"name":"Scuola Normale Superiore, Pisa, Italy and Istituto di Scienza e Tecnologie dell\u2019Informazione, Consiglio Nazionale delle Ricerche, Pisa, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8541-0284","authenticated-orcid":false,"given":"Anna","family":"Monreale","sequence":"additional","affiliation":[{"name":"Universit\u00e0 di Pisa, Pisa, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0377-1025","authenticated-orcid":false,"given":"Alejandro","family":"Moreo","sequence":"additional","affiliation":[{"name":"Consiglio Nazionale delle Ricerche, Pisa, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4221-6427","authenticated-orcid":false,"given":"Fabrizio","family":"Sebastiani","sequence":"additional","affiliation":[{"name":"Consiglio Nazionale delle Ricerche, Pisa, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,6,24]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4614-3223-4_6"},{"key":"e_1_3_2_3_2","first-page":"309","volume-title":"Seminario Dantesco Internazionale\/International Dante Seminar 1","author":"Ascoli Albert R.","year":"1997","unstructured":"Albert R. Ascoli. 1997. Access to authority: Dante in the epistle to cangrande. In Seminario Dantesco Internazionale\/International Dante Seminar 1. Zygmunt G. Baranski (Ed.), Le Lettere, Firenze, 309\u201352."},{"key":"e_1_3_2_4_2","first-page":"271","volume-title":"Nuova edizione commentata delle opere di Dante","author":"Azzetta Luca","year":"2016","unstructured":"Luca Azzetta. 2016. Nuova edizione commentata delle opere di Dante, Vol. 5. Salerno Editrice, Roma, IT, Chapter \u201cEpistola XIII\u201d, 271\u2013487."},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","unstructured":"Shane Barratt. 2017. Interpnet: Neural introspection for interpretable deep learning. Retrieved from 10.48550\/arXiv.1710.09511","DOI":"10.48550\/arXiv.1710.09511"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1162\/coli_a_00422"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-24403-7_10"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1093\/llc\/fqad019"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1080\/09332480.2003.10554843"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","unstructured":"Sebastian Bischoff Niklas Deckers Marcel Schliebs Ben Thies Matthias Hagen Efstathios Stamatatos Benno Stein and Martin Potthast. 2020. The importance of suppressing domain style in authorship analysis. Retrieved from 10.48550\/arXiv.2005.14714","DOI":"10.48550\/arXiv.2005.14714"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/BigData47090.2019.9005650"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.18848\/1836-6236\/CGP\/v02i02\/51247"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.3390\/electronics8080832"},{"key":"e_1_3_2_14_2","first-page":"215","article-title":"Sempre contro l\u2019autenticita dell\u2019Epistola a Cangrande","author":"Casadei Alberto","year":"2016","unstructured":"Alberto Casadei. 2016. Sempre contro l\u2019autenticita dell\u2019Epistola a Cangrande. Studi danteschi LXXXI (2016), 215\u201346.","journal-title":"Studi danteschi"},{"issue":"1","key":"e_1_3_2_15_2","first-page":"1","article-title":"Who\u2019s at the keyboard? Authorship attribution in digital evidence investigations","volume":"4","author":"Chaski Carole E.","year":"2005","unstructured":"Carole E. Chaski. 2005. Who\u2019s at the keyboard? Authorship attribution in digital evidence investigations. International Journal of Digital Evidence 4, 1 (2005), 1\u201313.","journal-title":"International Journal of Digital Evidence"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3609226"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1002\/asi.24660"},{"key":"e_1_3_2_18_2","first-page":"153","volume-title":"Atti del Seminario \u201cNuove Inchieste sull\u2019Epistola a Cangrande\u201d","author":"Corbara Silvia","year":"2020","unstructured":"Silvia Corbara, Alejandro Moreo, Fabrizio Sebastiani, and Mirko Tavoni. 2020. L\u2019epistola a Cangrande al vaglio della computational authorship verification: Risultati preliminari (con una postilla sulla cosiddetta \u201cXIV Epistola di Dante Alighieri\u201d). In Atti del Seminario \u201cNuove Inchieste sull\u2019Epistola a Cangrande\u201d. Alberto Casadei (Ed.), Pisa University Press, Pisa, IT, 153\u2013192."},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3485822"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.aacl-main.46"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1093\/llc\/fqr029"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1093\/llc\/fqm020"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","unstructured":"Yuling Gu Bhavana D. Mishra and Peter Clark. 2021. DREAM: Uncovering mental models behind language models. Retrieved from 10.48550\/arXiv.2112.08656","DOI":"10.48550\/arXiv.2112.08656"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3236009"},{"key":"e_1_3_2_25_2","first-page":"89","article-title":"\u2018Cursus\u2019 in the can grande epistle: A forger shows his hand?","volume":"5","author":"Hall Ralph G.","year":"1989","unstructured":"Ralph G. Hall and Madison U. Sowell. 1989. \u2018Cursus\u2019 in the can grande epistle: A forger shows his hand? Lectura Dantis 5 (1989), 89\u2013104.","journal-title":"Lectura Dantis"},{"key":"e_1_3_2_26_2","volume-title":"Practice-Oriented Authorship Verification","author":"Halvani Oren","year":"2021","unstructured":"Oren Halvani. 2021. Practice-Oriented Authorship Verification. Ph. D. Dissertation. Technische Universit\u00e4t Darmstadt, Darmstadt, DE."},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/3339252.3340508"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.2760\/57493"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1093\/llc\/13.3.111"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","unstructured":"Giles Hooker and Lucas Mentch. 2019. Please stop permuting features: An explanation and alternatives. Retrieved from 10.48550\/arXiv.1905.03151","DOI":"10.48550\/arXiv.1905.03151"},{"key":"e_1_3_2_31_2","first-page":"2733","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP \u201919)","author":"Hwitt John","year":"2019","unstructured":"John Hwitt and Percy Liang. 2019. Designing and interpreting probes with control tasks. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP \u201919). 2733\u20132743."},{"key":"e_1_3_2_32_2","first-page":"234","volume-title":"Proceedings of the 33rd International Conference of the Florida Artificial Intelligence Research Society (FLAIRS \u201920)","author":"Jafariakinabad Fereshteh","year":"2020","unstructured":"Fereshteh Jafariakinabad, Sansiri Tarnpradab, and Kien A. Hua. 2020. Syntactic neural model for authorship attribution. In Proceedings of the 33rd International Conference of the Florida Artificial Intelligence Research Society (FLAIRS \u201920). Virtual Event, 234\u2013239."},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","unstructured":"Mael Jullien Marco Valentino and Andre Freitas. 2022. Do transformers encode a foundational ontology? Probing abstract classes in natural language. Retrieved from 10.48550\/arXiv.2201.10262","DOI":"10.48550\/arXiv.2201.10262"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1561\/1500000005"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10579-018-9424-0"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/W14-0908"},{"key":"e_1_3_2_37_2","first-page":"1743","volume-title":"Proceedings of the Working Notes of the 2021 Conference and Labs of the Evaluation Forum (CLEF \u201921)","author":"Kestemont Mike","year":"2021","unstructured":"Mike Kestemont, Enrique Manjavacas, Ilia Markov, Janek Bevendorff, Matti Wiegmann, Efstathios Stamatatos, Benno Stein, and Martin Potthast. 2021. Overview of the cross-domain authorship verification task at PAN 2021. In Proceedings of the Working Notes of the 2021 Conference and Labs of the Evaluation Forum (CLEF \u201921). 1743\u20131759."},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1093\/llc\/fqt063"},{"key":"e_1_3_2_39_2","first-page":"1","volume-title":"Proceedings of the Working Notes of the 2019 Conference and Labs of the Evaluation Forum (CLEF \u201919)","author":"Kestemont Mike","year":"2019","unstructured":"Mike Kestemont, Efstathios Stamatatos, Enrique Manjavacas, Walter Daelemans, Martin Potthast, and Benno Stein. 2019. Overview of the cross-domain authorship attribution task at PAN 2019. In Proceedings of the Working Notes of the 2019 Conference and Labs of the Evaluation Forum (CLEF \u201919). 1\u201315."},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2016.06.029"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1002\/asi.20961"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-022-06150-7"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.5555\/2755227"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394486.3403066"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1523"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.557"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.3390\/e23010018"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1560"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N19-1112"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","unstructured":"Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy Mike Lewis Luke Zettlemoyer and Veselin Stoyanov. 2019b. RoBERTa: A robustly optimized BERT pretraining approach. Retrieved from 10.48550\/arXiv.1907.11692","DOI":"10.48550\/arXiv.1907.11692"},{"key":"e_1_3_2_51_2","volume-title":"Proceedings of the 6th International Conference on Learning Representations (ICLR \u201919)","author":"Loshchilov Ilya","year":"2018","unstructured":"Ilya Loshchilov and Frank Hutter. 2018. Decoupled weight decay regularization. In Proceedings of the 6th International Conference on Learning Representations (ICLR \u201919). DOI: https:\/\/openreview.net\/forum?id=Bkg6RiCqY7"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-019-0138-9"},{"key":"e_1_3_2_53_2","first-page":"4765","volume-title":"Proceedings of the 31st Annual Conference on Neural Information Processing Systems (NIPS \u201917)","author":"Lundberg Scott M.","year":"2017","unstructured":"Scott M. Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Proceedings of the 31st Annual Conference on Neural Information Processing Systems (NIPS \u201917). 4765\u20134774."},{"key":"e_1_3_2_54_2","first-page":"214","article-title":"The characteristic curves of composition","volume":"9","author":"Mendenhall Thomas C.","year":"1887","unstructured":"Thomas C. Mendenhall. 1887. The characteristic curves of composition. Science 9, 214 (1887), 237\u2013249.","journal-title":"Science"},{"key":"e_1_3_2_55_2","first-page":"302","article-title":"Inference in an authorship problem: A comparative study of discrimination methods applied to the authorship of the disputed Federalist Articles","volume":"58","author":"Mosteller Frederick","year":"1963","unstructured":"Frederick Mosteller and David L. Wallace. 1963. Inference in an authorship problem: A comparative study of discrimination methods applied to the authorship of the disputed Federalist Articles. Journal of the American Statistical Association 58, 302 (1963), 275\u2013309.","journal-title":"Journal of the American Statistical Association"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.5555\/1953048.2078195"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.4018\/978-1-4666-8345-7.ch012"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.64"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939778"},{"key":"e_1_3_2_60_2","volume-title":"Proceedings of the ICLR 2020 Workshop on AI for Affordable Healthcare","author":"Rieger Laura","year":"2020","unstructured":"Laura Rieger and Lars K. Hansen. 2020. IROF: A low-resource evaluation metric for explanation methods. In Proceedings of the ICLR 2020 Workshop on AI for Affordable Healthcare. DOI: https:\/\/orbit.dtu.dk\/en\/publications\/irof-a-low-resource-evaluation-metric-for-explanation-methods"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIFS.2016.2603960"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/N15-1010"},{"key":"e_1_3_2_63_2","first-page":"343","volume-title":"Proceedings of the 27th International Conference on Computational Linguistics (COLING \u201918)","author":"Sari Yunita","year":"2018","unstructured":"Yunita Sari, Mark Stevenson, and Andreas Vlachos. 2018. Topic or style? Exploring the most useful features for authorship attribution. In Proceedings of the 27th International Conference on Computational Linguistics (COLING \u201918). 343\u2013353."},{"key":"e_1_3_2_64_2","first-page":"359","article-title":"Sull\u2019Epistola a Cangrande","volume":"3","author":"Sasso Gennaro","year":"2013","unstructured":"Gennaro Sasso. 2013. Sull\u2019Epistola a Cangrande. La Cultura 3 (2013), 359\u2013446.","journal-title":"La Cultura"},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1002\/asi.24176"},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.diin.2015.05.012"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/E17-2106"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.1002\/asi.21001"},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.13053\/rcs-123-1-1"},{"key":"e_1_3_2_70_2","first-page":"2301","volume-title":"Proceedings of the Working Notes of the Conference and Labs of the Evaluation Forum (CLEF \u201922)","author":"Stamatatos Efstathios","year":"2022","unstructured":"Efstathios Stamatatos, Mike Kestemont, Krzysztof Kredens, Piotr Pezik, Annina Heini, Janek Bevendorff, Benno Stein, and Martin Potthast. 2022. Overview of the authorship verification task at PAN 2022. In Proceedings of the Working Notes of the Conference and Labs of the Evaluation Forum (CLEF \u201922). 2301\u20132313."},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1002\/asi.23460"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP43922.2022.9746262"},{"key":"e_1_3_2_73_2","doi-asserted-by":"publisher","DOI":"10.1093\/llc\/fqw001"},{"key":"e_1_3_2_74_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1002"},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-demos.6"},{"key":"e_1_3_2_76_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2021.107815"},{"key":"e_1_3_2_77_2","doi-asserted-by":"publisher","DOI":"10.5555\/1115657.1115669"}],"container-title":["Journal on Computing and Cultural Heritage"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3654675","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3654675","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:49:04Z","timestamp":1750286944000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3654675"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,24]]},"references-count":76,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,9,30]]}},"alternative-id":["10.1145\/3654675"],"URL":"https:\/\/doi.org\/10.1145\/3654675","relation":{},"ISSN":["1556-4673","1556-4711"],"issn-type":[{"value":"1556-4673","type":"print"},{"value":"1556-4711","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,24]]},"assertion":[{"value":"2023-08-03","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-03-15","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-06-24","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}