{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,20]],"date-time":"2026-02-20T06:52:50Z","timestamp":1771570370535,"version":"3.50.1"},"reference-count":30,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2022,7,14]],"date-time":"2022-07-14T00:00:00Z","timestamp":1657756800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,7,14]],"date-time":"2022-07-14T00:00:00Z","timestamp":1657756800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100004442","name":"Narodowym Centrum Nauki","doi-asserted-by":"publisher","award":["2019\/34\/E\/ST6\/00052"],"award-info":[{"award-number":["2019\/34\/E\/ST6\/00052"]}],"id":[{"id":"10.13039\/501100004442","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004281","name":"Narodowe Centrum Nauki","doi-asserted-by":"publisher","award":["2019\/34\/E\/ST6\/00052"],"award-info":[{"award-number":["2019\/34\/E\/ST6\/00052"]}],"id":[{"id":"10.13039\/501100004281","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Mach Learn"],"published-print":{"date-parts":[[2024,5]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The increased interest in deep learning applications, and their hard-to-detect biases result in the\u00a0need to validate and explain complex models. However, current explanation methods are limited as far as both the\u00a0explanation of the\u00a0reasoning process and prediction results are concerned. They usually only show the\u00a0location in the\u00a0image that was important for model prediction. The\u00a0lack of possibility to interact with explanations makes it difficult to verify and understand exactly how the\u00a0model works. This creates a\u00a0significant risk when using the\u00a0model. The risk is compounded by the\u00a0fact that explanations do not take into account the\u00a0semantic meaning of the\u00a0explained objects. To escape from the\u00a0trap of static and meaningless explanations, we propose a\u00a0tool and a\u00a0process called LIMEcraft. LIMEcraft enhances the\u00a0process of explanation by allowing a\u00a0user to interactively select semantically consistent areas and thoroughly examine the\u00a0prediction for the\u00a0image instance in case of many image features. Experiments on several models show that our tool improves model safety by inspecting model fairness for image pieces that may indicate model bias. The\u00a0code is available at: <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"http:\/\/github.com\/MI2DataLab\/LIMEcraft\">http:\/\/github.com\/MI2DataLab\/LIMEcraft<\/jats:ext-link>.<\/jats:p>","DOI":"10.1007\/s10994-022-06204-w","type":"journal-article","created":{"date-parts":[[2022,7,14]],"date-time":"2022-07-14T15:04:59Z","timestamp":1657811099000},"page":"3143-3160","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["LIMEcraft: handcrafted superpixel selection and inspection for Visual eXplanations"],"prefix":"10.1007","volume":"113","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2903-6050","authenticated-orcid":false,"given":"Weronika","family":"Hryniewska","sequence":"first","affiliation":[]},{"given":"Adrianna","family":"Grudzie\u0144","sequence":"additional","affiliation":[]},{"given":"Przemys\u0142aw","family":"Biecek","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,7,14]]},"reference":[{"key":"6204_CR1","unstructured":"Ahern, I., Noack, A., Guzman-Nateras, L., Dou, D., Li, B., & Huan, J. (2019). Normlime: A new feature importance metric for explaining deep neural networks. CoRR abs\/1909.04200 . Retrieved from http:\/\/arxiv.org\/abs\/1909.04200"},{"key":"6204_CR2","unstructured":"Alvarez-Melis, D., & Jaakkola, T. S. (2018). On the robustness of interpretability methods. In Proceedings of the 2018 icml workshop on human interpretability in machine learning. Retrieved from http:\/\/arxiv.org\/abs\/1806.08049"},{"key":"6204_CR3","unstructured":"Burns, C., Thomason, J., & Tansey, W. (2019). Interpreting black box models via hypothesis testing (pp. 47\u201357). Association for Computing Machinery, Inc. Retrieved from https:\/\/arxiv.org\/abs\/1904.00045v3. 10.1145\/3412815.3416889"},{"key":"6204_CR4","doi-asserted-by":"crossref","unstructured":"Chattopadhay, A., Sarkar, A., Howlader, P., & Balasubramanian, V. N. (2018). Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks. In 2018 IEEE winter conference on applications of computer vision (WACV). Retrieved from http:\/\/dx.doi.org\/10.1109\/WACV.2018.00097. 10.1109\/wacv.2018.00097","DOI":"10.1109\/WACV.2018.00097"},{"key":"6204_CR5","doi-asserted-by":"crossref","unstructured":"Damelin, S. B., & Hoang, N. S. (2018). On surface completion and image inpainting by biharmonic functions: Numerical aspects, vol. 2018. Hindawi Limited 10.1155\/2018\/3950312","DOI":"10.1155\/2018\/3950312"},{"key":"6204_CR6","doi-asserted-by":"crossref","unstructured":"Fong, R. C., & Vedaldi, A. (2017). Interpretable explanations of black boxes by meaningful perturbation. In 2017 IEEE international conference on computer vision (ICCV) (pp. 3449\u20133457). 10.1109\/ICCV.2017.371","DOI":"10.1109\/ICCV.2017.371"},{"key":"6204_CR7","unstructured":"Greydanus, S., Koul, A., Dodge, J., & Fern, A. (2018). Visualizing and understanding Atari agents. In J. Dy & A. Krause (Eds.), Proceedings of the 35th international conference on machine learning, vol. 80 (pp. 1792\u20131801). PMLR. Retrieved from https:\/\/proceedings.mlr.press\/v80\/greydanus18a.html"},{"key":"6204_CR8","doi-asserted-by":"crossref","unstructured":"Hryniewska, W., Bombinski, P., Szatkowski, P., Tomaszewska, P., Przelaskowski, A., & Biecek, P. (2021). Checklist for responsible deep learning modeling of medical images based on COVID-19 detection studies, vol. 118 (p. 108035). Pergamon. Retrieved from https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0031320321002223. 10.1016\/j.patcog.2021.108035","DOI":"10.1016\/j.patcog.2021.108035"},{"key":"6204_CR9","unstructured":"Hu, L., Chen, J., Nair, V. N., & Sudjianto, A. (2018). Locally interpretable models and effects based on supervised partitioning (LIME-SUP). CoRR abs\/1806.00663. Retrieved from http:\/\/arxiv.org\/abs\/1806.0066"},{"key":"6204_CR10","unstructured":"Huang, Q., Yamada, M., Tian, Y., Singh, D., Yin, D., & Chang, Y. (2020). Graphlime: Local interpretable model explanations for graph neural networks. CoRR abs\/2001.06216. Retrieved from https:\/\/arxiv.org\/abs\/2001.06216"},{"key":"6204_CR11","doi-asserted-by":"crossref","unstructured":"Li, H., Fan, W., Shi, S., & Chou, Q. (2019). A modified lime and its application to explain service supply chain forecasting. In Natural language processing and Chinese computing (pp. 637\u2013644). Springer International Publishing.","DOI":"10.1007\/978-3-030-32236-6_58"},{"key":"6204_CR12","unstructured":"Mader, K. (2019). Deep learning skin lesion classification\u2014kaggle. Retrieved from https:\/\/www.kaggle.com\/kmader\/deep-learningskin-lesion-classification"},{"key":"6204_CR13","unstructured":"Peltola, T. (2018). Local interpretable model-agnostic explanations of bayesian predictive models via kullback-leibler projections. In Proceedings of the 2nd workshop on explainable artificial intelligence (XAI 2018) at IJCAI\/ECAI 2018."},{"key":"6204_CR14","unstructured":"Petsiuk, V., Das, A., & Saenko, K. (2018). Rise: Randomized input sampling for explanation of black-box models. In Proceedings of the British machine vision conference (BMVC)."},{"key":"6204_CR15","doi-asserted-by":"crossref","unstructured":"Rabold, J., Deininger, H., Siebers, M., & Schmid, U. (2020). Enriching visual with verbal explanations for relational concepts\u2013combining LIME with aleph, vol. 1167 CCIS (pp. 180\u2013192). 10.1007\/978-3-030-43823-4_16","DOI":"10.1007\/978-3-030-43823-4_16"},{"key":"6204_CR16","unstructured":"Rahnama, A. H .A., & Bostr\u00f6m, H. (2019). A study of data and label shift in the LIME framework. CoRR abs\/1910.14421. Retrieved from http:\/\/arxiv.org\/abs\/1910.14421"},{"key":"6204_CR17","doi-asserted-by":"crossref","unstructured":"Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). Why should i trust you?: Explaining the predictions of any classifier. In Proceedings of the 2016 conference of the North American chapter of the Association for Computational Linguistics: Demonstrations (pp. 97\u2013101). Association for Computational Linguistics (ACL). 10.18653\/v1\/n16-3020","DOI":"10.18653\/v1\/N16-3020"},{"key":"6204_CR18","doi-asserted-by":"crossref","unstructured":"Ribeiro, M. T., Singh, S., & Guestrin, C. (2018). Anchors: High-precision model-agnostic explanations. In S. A. McIlraith & K. Q. Weinberger (Eds.), Proceedings of the thirty-second AAAI conference on artificial intelligence, the 30th IAAI-18, and the 8th AAAI symposium on EAAI-18 (pp. 1527\u20131535). AAAI Press.","DOI":"10.1609\/aaai.v32i1.11491"},{"key":"6204_CR19","doi-asserted-by":"crossref","unstructured":"Samek, W., Montavon, G., Vedaldi, A., Hansen, L., & M\u00fcller, K. R. (2019). Explainable AI: Interpreting, explaining and visualizing deep learning. 10.1007\/978-3-030-28954-6","DOI":"10.1007\/978-3-030-28954-6"},{"key":"6204_CR20","doi-asserted-by":"crossref","unstructured":"Schallner, L., Rabold, J., Scholz, O., & Schmid, U. (2020). Effect of superpixel aggregation on explanations in LIME-A case study with biological data. In Communications in computer and information science, vol. 1167 CCIS (pp. 147\u2013158). 10.1007\/978-3-030-43823-4_13","DOI":"10.1007\/978-3-030-43823-4_13"},{"issue":"2","key":"6204_CR21","doi-asserted-by":"publisher","first-page":"336","DOI":"10.1007\/s11263-019-01228-7","volume":"128","author":"RR Selvaraju","year":"2019","unstructured":"Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2019). Grad-cam: Visual explanations from deep networks via gradient-based localization. International Journal of Computer Vision, 128(2), 336\u2013359.","journal-title":"International Journal of Computer Vision"},{"key":"6204_CR22","doi-asserted-by":"crossref","unstructured":"Shankaranarayana, S. M., & Runje, D. (2019). ALIME: Autoencoder based approach for local interpretability. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11871 LNCS (pp. 454\u2013463). Springer. 10.1007\/978-3-030-33607-3_49","DOI":"10.1007\/978-3-030-33607-3_49"},{"key":"6204_CR23","unstructured":"Shi, S., Zhang, X., & Fan, W. (2020). A modified perturbed sampling method for local interpretable model-agnostic explanation. CoRR abs\/2002.07434. Retrieved from https:\/\/arxiv.org\/abs\/2002.07434"},{"key":"6204_CR24","doi-asserted-by":"crossref","unstructured":"Slack, D., Hilgard, S., Jia, E., Singh, S., & Lakkaraju, H. (2020). Fooling lime and shap: Adversarial attacks on post hoc explanation methods. In Proceedings of the AAAI\/ACM conference on AI, ethics, and society (pp. 180\u2013186). New York, NY, USA: Association for Computing Machinery. 10.1145\/ 3375627.3375830","DOI":"10.1145\/3375627.3375830"},{"key":"6204_CR25","doi-asserted-by":"crossref","unstructured":"Stieler, F., Rabe, F., & Bauer, B. (2021). Towards domain-specific explainable ai: Model interpretation of a skin image classifier using a human approach. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (CVPR) workshops (pp. 1802\u20131809).","DOI":"10.1109\/CVPRW53098.2021.00199"},{"key":"6204_CR26","doi-asserted-by":"crossref","unstructured":"Vargas, P., C\u00e1rdenas, R., Cullen, R., & Figueroa, A. (2020). Eruptive disseminated spitz nevi-case report, vol. 95 (pp. 71\u201374). Retrieved from www.sciencedirect.com\/science\/article\/pii\/S0365059619301540. 10.1016\/j.abd.2019.01.010","DOI":"10.1016\/j.abd.2019.01.010"},{"key":"6204_CR27","doi-asserted-by":"crossref","unstructured":"Vedaldi, A., & Soatto, S. (2008). Quick shift and kernel methods for mode seeking. In D. Forsyth, P. Torr, & A. Zisserman (Eds.), Computer vision\u2014ECCV 2008 (pp. 705\u2013718). Berlin, Heidelberg: Springer Berlin Heidelberg.","DOI":"10.1007\/978-3-540-88693-8_52"},{"key":"6204_CR28","unstructured":"Zafar, M. R., & Khan, N. M. (2019). Dlime: A deterministic local interpretable model-agnostic explanations approach for computer-aided diagnosis systems. In Proceeding of ACM SIGKDD workshop on explainable AI\/ML (XAI) for accountability, fairness, and transparency. Anchorage, Alaska: ACM."},{"key":"6204_CR29","doi-asserted-by":"crossref","unstructured":"Zeiler, M.D., & Fergus, R. (2014). Visualizing and understanding convolutional networks. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 8689 LNCS (pp. 818\u2013833). Springer, Cham. Retrieved from https:\/\/link.springer.com\/chapter\/10.1007\/978-3-319-10590-1_53. 10.1007\/978-3-319-10590-1_53","DOI":"10.1007\/978-3-319-10590-1_53"},{"key":"6204_CR30","unstructured":"Zintgraf, L. M., Cohen, T. S., Adel, T., & Welling, M. (2017). Visualizing deep neural network decisions: Prediction difference analysis. In 5th international conference on learning representations, ICLR 2017\u2014Conference track proceedings. International conference on learning representations, ICLR. Retrieved from https:\/\/arxiv.org\/abs\/1702.04595v"}],"container-title":["Machine Learning"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-022-06204-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10994-022-06204-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-022-06204-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,5,2]],"date-time":"2024-05-02T18:08:41Z","timestamp":1714673321000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10994-022-06204-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,14]]},"references-count":30,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2024,5]]}},"alternative-id":["6204"],"URL":"https:\/\/doi.org\/10.1007\/s10994-022-06204-w","relation":{},"ISSN":["0885-6125","1573-0565"],"issn-type":[{"value":"0885-6125","type":"print"},{"value":"1573-0565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,14]]},"assertion":[{"value":"18 November 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 April 2022","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 May 2022","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 July 2022","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"All participants of the\u00a0user study gave their informed consent that they: received and understood the\u00a0information concerning the\u00a0test, understood the\u00a0purpose of the\u00a0test and their involvement in it, understood that they may withdraw from participation at any stage and understood that their personal results will remain confidential and they will not be quoted or damaged if the\u00a0information will be made public.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to participate"}}]}}