{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,21]],"date-time":"2026-02-21T21:19:28Z","timestamp":1771708768517,"version":"3.50.1"},"reference-count":67,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,9,25]],"date-time":"2023-09-25T00:00:00Z","timestamp":1695600000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100018693","name":"HORIZON EUROPE Framework Programme","doi-asserted-by":"publisher","award":["860621"],"award-info":[{"award-number":["860621"]}],"id":[{"id":"10.13039\/100018693","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:p>When applied to Image-to-text models, explainability methods have two challenges. First, they often provide token-by-token explanations namely, they compute a visual explanation for each token of the generated sequence. This makes explanations expensive to compute and unable to comprehensively explain the model's output. Second, for models with visual inputs, explainability methods such as SHAP typically consider superpixels as features. Since superpixels do not correspond to semantically meaningful regions of an image, this makes explanations harder to interpret. We develop a framework based on SHAP, that allows for generating comprehensive, meaningful explanations leveraging the meaning representation of the output sequence as a whole. Moreover, by exploiting semantic priors in the visual backbone, we extract an arbitrary number of features that allows the efficient computation of Shapley values on large-scale models, generating at the same time highly meaningful visual explanations. We demonstrate that our method generates semantically more expressive explanations than traditional methods at a lower compute cost and that it can be generalized to a large family of vision-language models.<\/jats:p>","DOI":"10.3389\/frai.2023.1220476","type":"journal-article","created":{"date-parts":[[2023,9,26]],"date-time":"2023-09-26T06:09:48Z","timestamp":1695708588000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Interpreting vision and language generative models with semantic visual priors"],"prefix":"10.3389","volume":"6","author":[{"given":"Michele","family":"Cafagna","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lina M.","family":"Rojas-Barahona","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kees","family":"van Deemter","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Albert","family":"Gatt","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2023,9,25]]},"reference":[{"key":"B1","article-title":"\u201cPost-hoc explanations may be ineffective for detecting unknown spurious correlation,\u201d","author":"Adebayo","year":"2022","journal-title":"Proceedings of the 10th International Conference on Learning Representations (ICLR'22)"},{"key":"B2","doi-asserted-by":"crossref","DOI":"10.1109\/FUZZ45933.2021.9494423","article-title":"\u201cFuzzy explainable attention-based deep active learning on mental-health data,\u201d","volume-title":"2021 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE)","author":"Ahmed","year":"2021"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00636","article-title":"\u201cBottom-up and top-down attention for image captioning and visual question answering,\u201d","author":"Anderson","year":"2018","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B4","doi-asserted-by":"publisher","author":"Antol","year":"2015","DOI":"10.1109\/ICCV.2015.279"},{"key":"B5","doi-asserted-by":"crossref","DOI":"10.1007\/978-981-10-0557-2_87","article-title":"\u201cLayer-wise relevance propagation for deep neural network architectures,\u201d","volume-title":"Information Science and Applications (ICISA)","author":"Binder","year":"2016"},{"key":"B6","doi-asserted-by":"crossref","DOI":"10.1109\/BigData47090.2019.9005650","article-title":"\u201cExplainable authorship verification in social media via attention-based similarity learning,\u201d","volume-title":"2019 IEEE International Conference on Big Data (Big Data)","author":"Boenninghoff","year":"2019"},{"key":"B7","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn"},{"key":"B8","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2023.inlg-main.21","article-title":"\u201cHL dataset: visually-grounded description of scenes, actions and rationales,\u201d","volume-title":"Proceedings of the 16th International Natural Language Generation Conference (INLG'23)","author":"Cafagna","year":"2023"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2022\/759","article-title":"\u201cImage-text retrieval: A survey on recent research and development,\u201d","author":"Cao","year":"2022","journal-title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22"},{"key":"B10","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2020.acl-main.494","article-title":"\u201cGenerating Hierarchical Explanations on Text Classification via Feature Interaction Detection,\u201d","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Chen","year":"2020"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-00296-0_5","article-title":"\u201cPearson correlation coefficient,\u201d","author":"Cohen","year":"2009","journal-title":"Noise Reduction In Speech Processing"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01264-9_21","article-title":"\u201cDeep feature factorization for concept discovery,\u201d","author":"Collins","year":"2018","journal-title":"Proceedings of the European Conference on Computer Vision (ECCV)"},{"key":"B13","doi-asserted-by":"publisher","first-page":"111","DOI":"10.3233\/AIC-210172","article-title":"Explaining transformer-based image captioning models: An empirical analysis","volume":"35","author":"Cornia","year":"2022","journal-title":"AI Commun"},{"key":"B14","article-title":"An image is worth 16x16 words: Transformers for image recognition at scale","author":"Dosovitskiy","year":"2020","journal-title":"arXiv preprint arXiv:2010.11929"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.705","article-title":"Capwap: Captioning with a purpose","author":"Fisch","year":"2020","journal-title":"arXiv preprint arXiv:2011.04264"},{"key":"B16","doi-asserted-by":"publisher","first-page":"1","DOI":"10.48550\/arXiv.1801.01489","article-title":"All models are wrong, but many are useful: Learning a variable's importance by studying an entire class of prediction models simultaneously","volume":"20","author":"Fisher","year":"2019","journal-title":"J. Mach. Learn. Res"},{"key":"B17","article-title":"Unsupervised semantic segmentation by distilling feature correspondences","author":"Hamilton","year":"2022","journal-title":"arXiv preprint arXiv:2203.08414"},{"key":"B18","article-title":"Metrics for explainable ai: Challenges and prospects","author":"Hoffman","year":"2018","journal-title":"arXiv preprint arXiv:1812.04608"},{"key":"B19","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3295748","article-title":"A comprehensive survey of deep learning for image captioning","volume":"51","author":"Hossain","year":"2019","journal-title":"ACM Comput. Surv"},{"key":"B20","first-page":"4211","article-title":"How can i explain this to you? An empirical study of deep neural network explanation methods","volume":"33","author":"Jeyakumar","year":"2020","journal-title":"Adv. Neur. Inf. Proc. Syst"},{"key":"B21","article-title":"Segment anything","author":"Kirillov","year":"2023","journal-title":"arXiv preprint arXiv:2304.02643"},{"key":"B22","first-page":"16","article-title":"\u201cBert meets shapley: Extending shap explanations to transformer-based classifiers,\u201d","author":"Kokalj","year":"2021","journal-title":"Proceedings of the EACL Hackashop on News Media Content Analysis and Automated Report Generation"},{"key":"B23","doi-asserted-by":"publisher","first-page":"411","DOI":"10.1111\/j.1468-2958.2004.tb00738.x","article-title":"Reliability in content analysis: Some common misconceptions and recommendations","volume":"30","author":"Krippendorff","year":"2004","journal-title":"Hum. Commun. Res"},{"key":"B24","doi-asserted-by":"publisher","first-page":"1094","DOI":"10.1080\/01621459.2017.1307116","article-title":"Distribution-free predictive inference for regression","volume":"113","author":"Lei","year":"2018","journal-title":"J. Am. Statist. Assoc"},{"key":"B25","article-title":"SEED-bench: benchmarking multimodal LLMs with generative comprehension","author":"Li","year":"2023","journal-title":"arXiv:2307.16125"},{"key":"B26","first-page":"12888","article-title":"\u201cBlip: Bootstrapping language-image pre-training for unified vision-language understanding and generation,\u201d","volume-title":"International Conference on Machine Learning","author":"Li","year":""},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00108","article-title":"\u201cScouter: Slot attention-based classifier for explainable image recognition,\u201d","author":"Li","year":"2021","journal-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision"},{"key":"B28","doi-asserted-by":"publisher","first-page":"609","DOI":"10.1007\/s11831-021-09591-w","article-title":"A comprehensive review of markov random field and conditional random field approaches in pathology image analysis","volume":"29","author":"Li","year":"","journal-title":"Arch. Comput. Methods Eng"},{"key":"B29","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-10602-1_48","article-title":"\u201cMicrosoft coco: Common objects in context,\u201d","volume-title":"Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6\u201312, 2014, Proceedings, Part V 13","author":"Lin","year":"2014"},{"key":"B30","doi-asserted-by":"publisher","first-page":"154096","DOI":"10.1109\/ACCESS.2019.2949286","article-title":"Black-box vs. white-box: Understanding their advantages and weaknesses from a practical point of view","volume":"7","author":"Loyola-Gonzalez","year":"2019","journal-title":"IEEE Access"},{"key":"B31","article-title":"\u201cA unified approach to interpreting model predictions,\u201d","author":"Lundberg","year":"2017","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B32","article-title":"Clipcap: Clip prefix for image captioning","author":"Mokady","year":"2021","journal-title":"arXiv preprint arXiv:2111.09734"},{"key":"B33","volume-title":"Interpretable Machine Learning","author":"Molnar","year":"2020"},{"key":"B34","first-page":"4593","article-title":"\u201cShap-based explanation methods: A review for nlp interpretability,\u201d","author":"Mosca","year":"","journal-title":"Proceedings of the 29th International Conference on Computational Linguistics"},{"key":"B35","first-page":"4593","article-title":"\u201cSHAP-based explanation methods: A review for NLP interpretability,\u201d","volume-title":"Proceedings of the 29th International Conference on Computational Linguistics","author":"Mosca","year":""},{"key":"B36","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3583558","article-title":"From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI","volume":"55","author":"Nauta","year":"2023","journal-title":"ACM Comput Surv"},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.3115\/1073083.1073135","article-title":"\u201cBleu: a method for automatic evaluation of machine translation,\u201d","author":"Papineni","year":"2002","journal-title":"Proceedings of the 40th annual meeting of the Association for Computational Linguistics"},{"key":"B38","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2022.acl-long.567","article-title":"\u201cVALSE: A task-independent benchmark for vision and language models centered on linguistic phenomena,\u201d","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Parcalabescu","year":"2022"},{"key":"B39","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.acl-long.223","article-title":"Mm-shap: A performance-agnostic metric for measuring multimodal contributions in vision and language models &tasks","author":"Parcalabescu","year":"2022","journal-title":"arXiv preprint arXiv:2212.08158"},{"key":"B40","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-58558-7_30","article-title":"\u201cVisualcomet: Reasoning about the dynamic context of a still image,\u201d","volume-title":"Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23\u201328, 2020, Proceedings, Part V 16","author":"Park","year":"2020"},{"key":"B41","article-title":"Rise: Randomized input sampling for explanation of black-box models","author":"Petsiuk","year":"2018","journal-title":"arXiv preprint arXiv:1806.07421"},{"key":"B42","first-page":"8748","article-title":"\u201cLearning transferable visual models from natural language supervision,\u201d","volume-title":"International Conference on Machine Learning","author":"Radford","year":"2021"},{"key":"B43","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1410","article-title":"Sentence-bert: Sentence embeddings using siamese bert-networks","author":"Reimers","year":"2019","journal-title":"arXiv preprint arXiv:1908.10084"},{"key":"B44","article-title":"\u201cFaster r-cnn: Towards real-time object detection with region proposal networks,\u201d","author":"Ren","year":"2015","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B45","article-title":"Model-agnostic interpretability of machine learning","author":"Ribeiro","year":"2016","journal-title":"arXiv preprint arXiv:1606.05386"},{"key":"B46","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.74","article-title":"\u201cGrad-cam: Visual explanations from deep networks via gradient-based localization,\u201d","author":"Selvaraju","year":"2017","journal-title":"Proceedings of the IEEE International Conference on Computer Vision"},{"key":"B47","volume-title":"A Value for n-Person Games","author":"Shapley","year":"1953"},{"key":"B48","doi-asserted-by":"crossref","DOI":"10.1109\/PARC49193.2020.236619","article-title":"\u201cImage captioning: a comprehensive survey,\u201d","volume-title":"2020 International Conference on Power Electronics &IoT Applications in Renewable Energy and its Control (PARC)","author":"Sharma","year":"2020"},{"key":"B49","first-page":"3145","article-title":"\u201cLearning important features through propagating activation differences,\u201d","volume-title":"International Conference on Machine Learning","author":"Shrikumar","year":"2017"},{"key":"B50","article-title":"Not just a black box: Learning important features through propagating activation differences","author":"Shrikumar","year":"2016","journal-title":"arXiv preprint arXiv:1605.01713"},{"key":"B51","article-title":"Deep inside convolutional networks: Visualising image classification models and saliency maps","author":"Simonyan","year":"2013","journal-title":"arXiv preprint arXiv:1312.6034"},{"key":"B52","article-title":"Striving for simplicity: The all convolutional net","author":"Springenberg","year":"2014","journal-title":"arXiv preprint arXiv:1412.6806"},{"key":"B53","doi-asserted-by":"crossref","DOI":"10.1007\/978-981-16-1092-9_7","article-title":"\u201cVisual question answering using deep learning: A survey and performance analysis,\u201d","volume-title":"Computer Vision and Image Processing: 5th International Conference, CVIP 2020, Prayagraj, India, December 4\u20136, 2020, Revised Selected Papers, Part II 5","author":"Srivastava","year":"2021"},{"key":"B54","first-page":"3319","article-title":"\u201cAxiomatic attribution for deep networks,\u201d","volume-title":"International Conference on Machine Learning","author":"Sundararajan","year":"2017"},{"key":"B55","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1514","article-title":"Lxmert: Learning cross-modality encoder representations from transformers","author":"Tan","year":"2019","journal-title":"arXiv preprint arXiv:1908.07490"},{"key":"B56","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2022.3189849","article-title":"\u201cFast hierarchical games for image explanations,\u201d","author":"Teneggi","year":"2022","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"B57","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.00517","article-title":"\u201cWinoground: Probing vision and language models for visio-linguistic compositionality,\u201d","author":"Thrush","year":"2022","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B58","article-title":"Unifying architectures, tasks, and modalities through a simple sequence-to-sequence learning framework","author":"Wang","year":"2022","journal-title":"arXiv preprint arXiv:2202.03052"},{"key":"B59","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/1852102.1852106","article-title":"A similarity measure for indefinite rankings","volume":"28","author":"Webber","year":"2010","journal-title":"ACM Trans. Inf. Syst"},{"key":"B60","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-46475-6_5","article-title":"\u201cModeling context in referring expressions,\u201d","volume-title":"Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11\u201314, 2016, Proceedings, Part II 14","author":"Yu","year":"2016"},{"key":"B61","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-10590-1_53","article-title":"\u201cVisualizing and understanding convolutional networks,\u201d","volume-title":"Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6\u201312, 2014, Proceedings, Part I 13","author":"Zeiler","year":"2014"},{"key":"B62","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00688","article-title":"\u201cFrom recognition to cognition: Visual commonsense reasoning,\u201d","author":"Zellers","year":"2019","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B63","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00546","article-title":"\u201cRaven: A dataset for relational and analogical visual reasoning,\u201d","author":"Zhang","year":"","journal-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition"},{"key":"B64","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00553","article-title":"\u201cVinvl: Revisiting visual representations in vision-language models,\u201d","author":"Zhang","year":"2021","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B65","doi-asserted-by":"crossref","DOI":"10.1109\/WACV.2019.00043","article-title":"\u201cInterpretable visual question answering by visual grounding from attention supervision mining,\u201d","volume-title":"2019 IEEE Winter Conference on Applications of Computer Vision (WACV)","author":"Zhang","year":""},{"key":"B66","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1016\/j.ins.2022.03.010","article-title":"Attention-based explainable friend link prediction with heterogeneous context information","volume":"597","author":"Zheng","year":"2022","journal-title":"Inf. Sci"},{"key":"B67","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.540","article-title":"\u201cVisual7w: Grounded question answering in images,\u201d","author":"Zhu","year":"2016","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2023.1220476\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,22]],"date-time":"2023-12-22T17:04:04Z","timestamp":1703264644000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2023.1220476\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,25]]},"references-count":67,"alternative-id":["10.3389\/frai.2023.1220476"],"URL":"https:\/\/doi.org\/10.3389\/frai.2023.1220476","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,25]]},"article-number":"1220476"}}