{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,8]],"date-time":"2026-02-08T06:51:55Z","timestamp":1770533515264,"version":"3.49.0"},"reference-count":23,"publisher":"Wiley","issue":"1","license":[{"start":{"date-parts":[[2023,6,3]],"date-time":"2023-06-03T00:00:00Z","timestamp":1685750400000},"content-version":"vor","delay-in-days":153,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100011033","name":"Agencia Estatal de Investigaci\u00f3n","doi-asserted-by":"publisher","award":["PID2020-119478GBI00"],"award-info":[{"award-number":["PID2020-119478GBI00"]}],"id":[{"id":"10.13039\/501100011033","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100006393","name":"Universidad de Granada","doi-asserted-by":"publisher","award":["6717"],"award-info":[{"award-number":["6717"]}],"id":[{"id":"10.13039\/501100006393","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100006393","name":"Universidad de Granada","doi-asserted-by":"publisher","award":["5987"],"award-info":[{"award-number":["5987"]}],"id":[{"id":"10.13039\/501100006393","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100011011","name":"Junta de Andaluc\u00eda","doi-asserted-by":"publisher","award":["P18-FR-4961"],"award-info":[{"award-number":["P18-FR-4961"]}],"id":[{"id":"10.13039\/501100011011","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004837","name":"Ministerio de Ciencia e Innovaci\u00f3n","doi-asserted-by":"publisher","award":["EQC2018-005084-P"],"award-info":[{"award-number":["EQC2018-005084-P"]}],"id":[{"id":"10.13039\/501100004837","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100008530","name":"European Regional Development Fund","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100008530","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100018943","name":"Consejer\u00eda de Conocimiento, Investigaci\u00f3n y Universidad, Junta de Andaluc\u00eda","doi-asserted-by":"publisher","award":["SOMM17\/6110\/UGR"],"award-info":[{"award-number":["SOMM17\/6110\/UGR"]}],"id":[{"id":"10.13039\/501100018943","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["International Journal of Intelligent Systems"],"published-print":{"date-parts":[[2023,1]]},"abstract":"<jats:p>Explainable artificial intelligence is proposed to provide explanations for reasoning performed by artificial intelligence. There is no consensus on how to evaluate the quality of these explanations, since even the definition of explanation itself is not clear in the literature. In particular, for the widely known local linear explanations, there are qualitative proposals for the evaluation of explanations, although they suffer from theoretical inconsistencies. The case of image is even more problematic, where a visual explanation seems to explain a decision while detecting edges is what it really does. There are a large number of metrics in the literature specialized in quantitatively measuring different qualitative aspects, so we should be able to develop metrics capable of measuring in a robust and correct way the desirable aspects of the explanations. Some previous papers have attempted to develop new measures for this purpose. However, these measures suffer from lack of objectivity or lack of mathematical consistency, such as saturation or lack of smoothness. In this paper, we propose a procedure called REVEL to evaluate different aspects concerning the quality of explanations with a theoretically coherent development which do not have the problems of the previous measures. This procedure has several advances in the state of the art: it standardizes the concepts of explanation and develops a series of metrics not only to be able to compare between them but also to obtain absolute information regarding the explanation itself. The experiments have been carried out on four image datasets as benchmark where we show REVEL\u2019s descriptive and analytical power.<\/jats:p>","DOI":"10.1155\/2023\/8068569","type":"journal-article","created":{"date-parts":[[2023,6,3]],"date-time":"2023-06-03T21:20:05Z","timestamp":1685827205000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["REVEL Framework to Measure Local Linear Explanations for Black\u2010Box Models: Deep Learning Image Classification Case Study"],"prefix":"10.1155","volume":"2023","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5029-9106","authenticated-orcid":false,"given":"Iv\u00e1n","family":"Sevillano-Garc\u00eda","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3952-3629","authenticated-orcid":false,"given":"Juli\u00e1n","family":"Luengo","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7283-312X","authenticated-orcid":false,"given":"Francisco","family":"Herrera","sequence":"additional","affiliation":[]}],"member":"311","published-online":{"date-parts":[[2023,6,3]]},"reference":[{"key":"e_1_2_11_1_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2019.12.012"},{"key":"e_1_2_11_2_2","doi-asserted-by":"publisher","DOI":"10.1002\/int.23022"},{"key":"e_1_2_11_3_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2016.11.010"},{"key":"e_1_2_11_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/3236009"},{"key":"e_1_2_11_5_2","doi-asserted-by":"publisher","DOI":"10.1002\/widm.1391"},{"key":"e_1_2_11_6_2","doi-asserted-by":"publisher","DOI":"10.7717\/peerj-cs.479"},{"key":"e_1_2_11_7_2","doi-asserted-by":"crossref","unstructured":"GilpinL. H. BauD. YuanB. Z. BajwaA. SpecterM. andKagalL. Explaining explanations: an overview of interpretability of machine learning Proceedings of the 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) October 2018 Turin Italy IEEE 80\u201389.","DOI":"10.1109\/DSAA.2018.00018"},{"key":"e_1_2_11_8_2","unstructured":"Sevillano-Garc\u00edaI. Luengo-Mart\u00ednJ. andHerreraF. Revel framework to measure local linear explanations for black-box models: Deep learning image classification case of study 2022 https:\/\/arxiv.org\/abs\/2211.06154."},{"key":"e_1_2_11_9_2","first-page":"9391","volume-title":"Advances in Neural Information Processing Systems","author":"Slack D.","year":"2021"},{"key":"e_1_2_11_10_2","doi-asserted-by":"crossref","unstructured":"RibeiroM. T. SinghS. andGuestrinC. \u201cWhy should i trust you?\u201d: explaining the predictions of any classifier Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining KDD \u201916 October 2016 New York NY USA Association for Computing Machinery 1135\u20131144.","DOI":"10.1145\/2939672.2939778"},{"key":"e_1_2_11_11_2","first-page":"4765","volume-title":"Advances in Neural Information Processing Systems","author":"Lundberg S. M.","year":"2017"},{"key":"e_1_2_11_12_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-33850-3_5"},{"key":"e_1_2_11_13_2","first-page":"147","volume-title":"Joint European Conference on Machine Learning and Knowledge Discovery in Databases","author":"Schallner L.","year":"2019"},{"key":"e_1_2_11_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.2980581"},{"key":"e_1_2_11_15_2","unstructured":"RosenfeldA. Better metrics for evaluating explainable artificial intelligence Proceedings of the 20th International Conference on Autonomous Agents and Multiagent Systems May 2021 London UK 45\u201350."},{"key":"e_1_2_11_16_2","first-page":"225","article-title":"Linear hinge loss and average margin","volume":"11","author":"Gentile C.","year":"1998","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_11_17_2","unstructured":"KrizhevskyA. NairV. andHintonG. Cifar-10 (canadian institute for advanced research) 2010 5 no. 4."},{"key":"e_1_2_11_18_2","volume-title":"Learning Multiple Layers of Features From Tiny Images","author":"Krizhevsky A.","year":"2009"},{"key":"e_1_2_11_19_2","unstructured":"XiaoH. RasulK. andVollgrafR. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms 2017 https:\/\/arxiv.org\/abs\/1708.07747."},{"key":"e_1_2_11_20_2","doi-asserted-by":"crossref","unstructured":"CohenG. AfsharS. TapsonJ. andVan SchaikA. Emnist: extending mnist to handwritten letters Proceedings of the 2017 International Joint Conference on Neural Networks (IJCNN) May 2017 Anchorage AL USA IEEE 2921\u20132926.","DOI":"10.1109\/IJCNN.2017.7966217"},{"key":"e_1_2_11_21_2","unstructured":"TanM.andLeQ. Efficientnet: rethinking model scaling for convolutional neural networks Proceedings of the International Conference on Machine Learning June 2019 Montreal Canada 6105\u20136114."},{"key":"e_1_2_11_22_2","unstructured":"KingmaD. P.andBaJ. Adam: a method for stochastic optimization 2014 https:\/\/arxiv.org\/abs\/1412.6980."},{"key":"e_1_2_11_23_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41592-019-0686-2"}],"container-title":["International Journal of Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/downloads.hindawi.com\/journals\/ijis\/2023\/8068569.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.hindawi.com\/journals\/ijis\/2023\/8068569.xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1155\/2023\/8068569","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,31]],"date-time":"2024-12-31T05:37:24Z","timestamp":1735623444000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1155\/2023\/8068569"}},"subtitle":[],"editor":[{"given":"Alexander","family":"Ho\u0161ovsk\u00fd","sequence":"additional","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2023,1]]},"references-count":23,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023,1]]}},"alternative-id":["10.1155\/2023\/8068569"],"URL":"https:\/\/doi.org\/10.1155\/2023\/8068569","archive":["Portico"],"relation":{},"ISSN":["0884-8173","1098-111X"],"issn-type":[{"value":"0884-8173","type":"print"},{"value":"1098-111X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1]]},"assertion":[{"value":"2022-11-15","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-05-08","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-06-03","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"8068569"}}