{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,26]],"date-time":"2026-02-26T15:29:11Z","timestamp":1772119751373,"version":"3.50.1"},"reference-count":25,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2024,9,5]],"date-time":"2024-09-05T00:00:00Z","timestamp":1725494400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,9,5]],"date-time":"2024-09-05T00:00:00Z","timestamp":1725494400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100014374","name":"Universitat Polit\u00e8cnica de Catalunya","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100014374","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Neural Process Lett"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Although Deep Learning networks generally outperform traditional machine learning approaches based on tailored features, they often lack explainability. To address this issue, numerous methods have been proposed, particularly for image-related tasks such as image classification or object segmentation. These methods generate a heatmap that visually explains the classification problem by identifying the most important regions for the classifier. However, these explanations remain purely visual. To overcome this limitation, we introduce a novel CNN explainability method that identifies the most relevant regions in an image and generates a decision tree based on meaningful regional features, providing a rule-based explanation of the classification model. We evaluated the proposed method on a synthetic blob\u2019s dataset and subsequently applied it to two cell image classification datasets with healthy and pathological patterns.<\/jats:p>","DOI":"10.1007\/s11063-024-11678-x","type":"journal-article","created":{"date-parts":[[2024,9,5]],"date-time":"2024-09-05T15:02:05Z","timestamp":1725548525000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Generation of Rule-Based Explanations of CNN Classifiers Using Regional Features"],"prefix":"10.1007","volume":"56","author":[{"given":"William","family":"Philipp","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"R.","family":"Yashwanthika","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"O. K.","family":"Sikha","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Raul","family":"Benitez","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,9,5]]},"reference":[{"issue":"10","key":"11678_CR1","doi-asserted-by":"publisher","first-page":"11936","DOI":"10.2196\/11936","volume":"20","author":"TJ Brinker","year":"2018","unstructured":"Brinker TJ, Hekler A, Utikal JS, Grabe N, Schadendorf D, Klode J, Berking C, Steeb T, Enk AH, Von Kalle C (2018) Skin cancer classification using convolutional neural networks: systematic review. J Med Internet Res 20(10):11936","journal-title":"J Med Internet Res"},{"key":"11678_CR2","unstructured":"Sakinis T, Milletari F, Roth H, Korfiatis P, Kostandy P, Philbrick K, Akkus Z, Xu Z, Xu D, Erickson BJ (2019) Interactive segmentation of medical images through fully convolutional neural networks. arXiv:1903.08205"},{"key":"11678_CR3","unstructured":"Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556"},{"key":"11678_CR4","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-28954-6","volume-title":"Explainable AI: interpreting, explaining and visualizing deep learning","author":"W Samek","year":"2019","unstructured":"Samek W, Montavon G, Vedaldi A, Hansen LK, M\u00fcller K-R (2019) Explainable AI: interpreting, explaining and visualizing deep learning. Springer, Berlin"},{"key":"11678_CR5","doi-asserted-by":"crossref","unstructured":"Ribeiro MT, Singh S, Guestrin C (2016) \" why should i trust you?\" explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135\u20131144","DOI":"10.1145\/2939672.2939778"},{"key":"11678_CR6","unstructured":"Lundberg SM, Lee S-I (2017) A unified approach to interpreting model predictions. Adv Neural Inf Process Syst 30"},{"key":"11678_CR7","doi-asserted-by":"crossref","unstructured":"Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-CAM: Visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE international conference on computer vision, pp. 618\u2013626","DOI":"10.1109\/ICCV.2017.74"},{"key":"11678_CR8","doi-asserted-by":"crossref","unstructured":"Chattopadhay A, Sarkar A, Howlader P, Balasubramanian VN (2018) Grad-CAM++: Generalized gradient-based visual explanations for deep convolutional networks. In: 2018 IEEE winter conference on applications of computer vision (WACV), pp. 839\u2013847. IEEE","DOI":"10.1109\/WACV.2018.00097"},{"issue":"11","key":"11678_CR9","first-page":"2579","volume":"9","author":"L Maaten","year":"2008","unstructured":"Maaten L, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9(11):2579\u20132605","journal-title":"J Mach Learn Res"},{"issue":"1","key":"11678_CR10","first-page":"27","volume":"41","author":"A Mead","year":"1992","unstructured":"Mead A (1992) Review of the development of multidimensional scaling methods. J R Stat Soc Ser D (Stat) 41(1):27\u201339","journal-title":"J R Stat Soc Ser D (Stat)"},{"key":"11678_CR11","doi-asserted-by":"crossref","unstructured":"McInnes L, Healy J, Melville J (2018) UMAP: uniform manifold approximation and projection for dimension reduction. arXiv:1802.03426","DOI":"10.21105\/joss.00861"},{"key":"11678_CR12","doi-asserted-by":"publisher","first-page":"351","DOI":"10.1016\/j.jocs.2017.07.007","volume":"25","author":"O Sikha","year":"2018","unstructured":"Sikha O, Kumar SS, Soman K (2018) Salient region detection and object segmentation in color images using dynamic mode decomposition. J Comput Sci 25:351\u2013366","journal-title":"J Comput Sci"},{"issue":"6","key":"11678_CR13","doi-asserted-by":"publisher","first-page":"14","DOI":"10.1109\/MIS.2019.2957223","volume":"34","author":"R Guidotti","year":"2019","unstructured":"Guidotti R, Monreale A, Giannotti F, Pedreschi D, Ruggieri S, Turini F (2019) Factual and counterfactual explanations for black box decision making. IEEE Intell Syst 34(6):14\u201323","journal-title":"IEEE Intell Syst"},{"key":"11678_CR14","unstructured":"Hailesilassie T (2016) Rule extraction algorithm for deep neural networks: a review. arXiv:1610.05267"},{"key":"11678_CR15","doi-asserted-by":"crossref","unstructured":"Zhang Q, Yang Y, Ma H, Wu YN (2019) Interpreting CNNs via decision trees. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp. 6261\u20136270","DOI":"10.1109\/CVPR.2019.00642"},{"key":"11678_CR16","unstructured":"Shi S, Zhang X, Fan W (2019) Explaining the predictions of any image classifier via decision trees. arXiv:1911.01058"},{"key":"11678_CR17","unstructured":"Sokol K, Flach P (2020) LIMEtree: interactively customisable explanations based on local surrogate multi-output regression trees. arXiv:2005.01427"},{"key":"11678_CR18","doi-asserted-by":"publisher","unstructured":"Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-CAM: visual explanations from deep networks via gradient-based localization. In: 2017 IEEE international conference on computer vision (ICCV), pp. 618\u2013626. https:\/\/doi.org\/10.1109\/ICCV.2017.74","DOI":"10.1109\/ICCV.2017.74"},{"key":"11678_CR19","doi-asserted-by":"publisher","unstructured":"Simonyan K, Vedaldi A, Zisserman A (2013) Deep inside convolutional networks: visualising image classification models and saliency maps. https:\/\/doi.org\/10.48550\/arxiv.1312.6034","DOI":"10.48550\/arxiv.1312.6034"},{"key":"11678_CR20","doi-asserted-by":"publisher","unstructured":"Chattopadhay A, Sarkar A, Howlader P, Balasubramanian VN (2018) Grad-CAM++: improved visual explanations for deep convolutional networks. In: 2018 IEEE winter conference on applications of computer vision (WACV), IEEE. https:\/\/doi.org\/10.1109\/wacv.2018.00097","DOI":"10.1109\/wacv.2018.00097"},{"key":"11678_CR21","unstructured":"Kayhan OS, Gemert JC (2020) On translation invariance in CNNs: convolutional layers can exploit absolute spatial location. CoRR, arxiv:2003.07064"},{"key":"11678_CR22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/1471-2105-10-213","volume":"10","author":"BH Menze","year":"2009","unstructured":"Menze BH, Kelm BM, Masuch R, Himmelreich U, Bachert P, Petrich W, Hamprecht FA (2009) A comparison of random forest and its Gini importance with standard chemometric methods for the feature selection and classification of spectral data. BMC Bioinf 10:1\u201316","journal-title":"BMC Bioinf"},{"issue":"1","key":"11678_CR23","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","volume":"45","author":"L Breiman","year":"2001","unstructured":"Breiman L (2001) Random forests. Mach Learn 45(1):5\u201332. https:\/\/doi.org\/10.1023\/A:1010933404324","journal-title":"Mach Learn"},{"key":"11678_CR24","doi-asserted-by":"publisher","first-page":"6977","DOI":"10.7717\/peerj.6977","volume":"7","author":"S Rajaraman","year":"2019","unstructured":"Rajaraman S, Jaeger S, Antani SK (2019) Performance evaluation of deep neural ensembles toward malaria parasite detection in thin-blood smear images. PeerJ 7:6977. https:\/\/doi.org\/10.7717\/peerj.6977","journal-title":"PeerJ"},{"key":"11678_CR25","doi-asserted-by":"crossref","unstructured":"Labati RD, Piuri V, Scotti F (2011) All-IDB: the acute lymphoblastic leukemia image database for image processing. In: 2011 18th IEEE international conference on image processing, pp. 2045\u20132048. IEEE","DOI":"10.1109\/ICIP.2011.6115881"}],"container-title":["Neural Processing Letters"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11063-024-11678-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11063-024-11678-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11063-024-11678-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,30]],"date-time":"2024-10-30T11:55:16Z","timestamp":1730289316000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11063-024-11678-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,5]]},"references-count":25,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2024,10]]}},"alternative-id":["11678"],"URL":"https:\/\/doi.org\/10.1007\/s11063-024-11678-x","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-4216588\/v1","asserted-by":"object"}]},"ISSN":["1573-773X"],"issn-type":[{"value":"1573-773X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,9,5]]},"assertion":[{"value":"23 July 2024","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 September 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no Conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}],"article-number":"230"}}