{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,20]],"date-time":"2026-08-20T16:22:35Z","timestamp":1787242955101,"version":"build-2736575974"},"reference-count":36,"publisher":"Springer Science and Business Media LLC","issue":"11","license":[{"start":{"date-parts":[[2023,8,21]],"date-time":"2023-08-21T00:00:00Z","timestamp":1692576000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,8,21]],"date-time":"2023-08-21T00:00:00Z","timestamp":1692576000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["390621612"],"award-info":[{"award-number":["390621612"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100007210","name":"RWTH Aachen University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100007210","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Mach Learn"],"published-print":{"date-parts":[[2023,11]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Convolutional Neural Networks (CNN) have become a common choice for industrial quality control, as well as other critical applications in the Industry 4.0. When these CNNs behave in ways unexpected to human users or developers, severe consequences can arise, such as economic losses or an increased risk to human life. Concept extraction techniques can be applied to increase the reliability and transparency of CNNs through generating global explanations for trained neural network models. The decisive features of image datasets in quality control often depend on the feature\u2019s scale; for example, the size of a hole or an edge. However, existing concept extraction methods do not correctly represent scale, which leads to problems interpreting these models as we show herein. To address this issue, we introduce the Scale-Preserving Automatic Concept Extraction (SPACE) algorithm, as a state-of-the-art alternative concept extraction technique for CNNs, focused on industrial applications. SPACE is specifically designed to overcome the aforementioned problems by avoiding scale changes throughout the concept extraction process. SPACE proposes an approach based on square slices of input images, which are selected and then tiled before being clustered into concepts. Our method provides explanations of the models\u2019 decision-making process in the form of human-understandable concepts. We evaluate SPACE on three image classification datasets in the context of industrial quality control. Through experimental results, we illustrate how SPACE outperforms other methods and provides actionable insights on the decision mechanisms of CNNs. Finally, code for the implementation of SPACE is provided.<\/jats:p>","DOI":"10.1007\/s10994-023-06373-2","type":"journal-article","created":{"date-parts":[[2023,8,21]],"date-time":"2023-08-21T14:02:19Z","timestamp":1692626539000},"page":"4495-4525","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Scale-preserving automatic concept extraction (SPACE)"],"prefix":"10.1007","volume":"112","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3751-0680","authenticated-orcid":false,"given":"Andr\u00e9s Felipe","family":"Posada-Moreno","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lukas","family":"Kreisk\u00f6ther","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tassilo","family":"Glander","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sebastian","family":"Trimpe","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,8,21]]},"reference":[{"key":"6373_CR1","doi-asserted-by":"crossref","unstructured":"Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., & S\u00fcsstrunk, S. (2012). Slic superpixels compared to state-of-the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(11).","DOI":"10.1109\/TPAMI.2012.120"},{"key":"6373_CR2","doi-asserted-by":"crossref","unstructured":"Ankerst, M., Breunig, M.\u00a0M., Kriegel, H.-P., & Sander, J. (1999). Optics: Ordering points to identify the clustering structure. In Proceedings of the 1999 ACM SIGMOD International Conference on Management of Data, SIGMOD \u201999, page 49-60, New York, NY, USA, Association for Computing Machinery. ISBN 1581130848.","DOI":"10.1145\/304182.304187"},{"key":"6373_CR3","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1016\/j.inffus.2019.12.012","volume":"58","author":"AB Arrieta","year":"2020","unstructured":"Arrieta, A. B., D\u00edaz-Rodr\u00edguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garc\u00eda, S., Gil-L\u00f3pez, S., Molina, D., Benjamins, R., et al. (2020). Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai. Information Fusion, 58, 82\u2013115.","journal-title":"Information Fusion"},{"issue":"2","key":"6373_CR4","doi-asserted-by":"publisher","first-page":"131","DOI":"10.1007\/s11063-011-9207-8","volume":"35","author":"MG Augasta","year":"2012","unstructured":"Augasta, M. G., & Kathirvalavakumar, T. (2012). Reverse engineering the neural networks for rule extraction in classification problems. Neural Processing Letters, 35(2), 131\u2013150, 4. ISSN 1573-773X.","journal-title":"Neural Processing Letters"},{"key":"6373_CR5","doi-asserted-by":"crossref","unstructured":"Bergmann, P., Fauser, M., Sattlegger, D., & Steger, C. (2019). Mvtec ad - a comprehensive real-world dataset for unsupervised anomaly detection. In 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9584\u20139592.","DOI":"10.1109\/CVPR.2019.00982"},{"key":"6373_CR6","first-page":"245","volume":"70","author":"N Burkart","year":"2021","unstructured":"Burkart, N., & Huber, M. F. (2021). A survey on the explainability of supervised machine learning. J. Artif. Int. Res., 70, 245\u2013317. ISSN 1076-9757.","journal-title":"J. Artif. Int. Res."},{"key":"6373_CR7","doi-asserted-by":"publisher","unstructured":"\u00c7a\u011flar F\u0131rat\u00a0\u00d6zgenel. (2019). Concrete crack images for classification. 2, https:\/\/doi.org\/10.17632\/5Y9WDSG2ZT.2.","DOI":"10.17632\/5Y9WDSG2ZT.2"},{"key":"6373_CR8","unstructured":"Chen, C., Li, O., Tao, D., Barnett, A., Rudin, C., & Su, J. (2019). This looks like that: Deep learning for interpretable image recognition. In Wallach, H.\u00a0M., Larochelle, H., Beygelzimer, A., d\u2019Alch\u00e9-Buc, F., Fox, E.\u00a0B., & Garnett, R. (eds), Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 8928\u20138939."},{"key":"6373_CR9","unstructured":"Chen, Z., Bei, Y., & Rudin, C. (2020). Concept whitening for interpretable image recognition. CoRR, arXiv:abs\/2002.01650."},{"key":"6373_CR10","unstructured":"Das, A., & Rad, P. (2020). Opportunities and challenges in explainable artificial intelligence (XAI): A survey. CoRR, arXiv:abs\/2006.11371"},{"key":"6373_CR11","doi-asserted-by":"crossref","unstructured":"Dikmen, M., & Burns, C.\u00a0M. (2016). Autonomous driving in the real world: Experiences with tesla autopilot and summon. In Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Automotive\u2019UI 16, page 225-228, New York, NY, USA, Association for Computing Machinery. ISBN 9781450345330.","DOI":"10.1145\/3003715.3005465"},{"key":"6373_CR12","doi-asserted-by":"crossref","unstructured":"Ge, Y., Xiao, Y., Xu, Z., Zheng, M., Karanam, S., Chen, T., Itti, L., & Wu, Z. (2021). A peek into the reasoning of neural networks: Interpreting with structural visual concepts. CoRR, arXiv:abs\/2105.00290.","DOI":"10.1109\/CVPR46437.2021.00223"},{"key":"6373_CR13","unstructured":"Ghorbani, A., Wexler, J., Zou, J.\u00a0Y., & Kim, B. (2019). Towards automatic concept-based explanations. In Wallach, H., Larochelle, H., Beygelzimer, A., d\u2019Alch\u00e9-Buc, F., Fox, E., & Garnett, R. (eds), Advances in Neural Information Processing Systems, volume\u00a032. Curran Associates, Inc."},{"key":"6373_CR14","unstructured":"Goyal, Y., Feder, A., Shalit, U., & Kim, B. (2019). Explaining classifiers with causal concept effect (cace). arXiv preprint arXiv:1907.07165."},{"key":"6373_CR15","unstructured":"Gu, X., & Cheng, X. (2020). Distilling a deep neural network into a takagi-sugeno-kang fuzzy inference system. CoRR, arXiv:abs\/2010.04974"},{"key":"6373_CR16","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., & Sun, J. (2015). Deep residual learning for image recognition. CoRR, arXiv:abs\/1512.03385.","DOI":"10.1109\/CVPR.2016.90"},{"key":"6373_CR17","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., & Weinberger, K.\u00a0Q. (2016). Densely connected convolutional networks. CoRR, arXiv:abs\/1608.06993.","DOI":"10.1109\/CVPR.2017.243"},{"key":"6373_CR18","doi-asserted-by":"crossref","unstructured":"Kapishnikov, A., Bolukbasi, T., Viegas, F., & Terry, M. (2019). Xrai: Better attributions through regions. In 2019 IEEE\/CVF International Conference on Computer Vision (ICCV), pages 4947\u20134956","DOI":"10.1109\/ICCV.2019.00505"},{"key":"6373_CR19","doi-asserted-by":"crossref","unstructured":"Kaufman, S., Rosset, S., Perlich, C., & Stitelman, O. (2012). Leakage in data mining: Formulation, detection, and avoidance. ACM Trans. Knowl. Discov. Data, 6(4), Dec. 2012. ISSN 1556-4681.","DOI":"10.1145\/2382577.2382579"},{"key":"6373_CR20","unstructured":"Kim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., & Viegas, F. et\u00a0al. (2018a). Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In International conference on machine learning, pages 2668\u20132677. PMLR."},{"key":"6373_CR21","doi-asserted-by":"crossref","unstructured":"Kim, S.-W., Kook, H.-K., Sun, J.-Y., Kang, M.-C., & Ko, S.-J. (2018b). Parallel feature pyramid network for object detection. In Proceedings of the European Conference on Computer Vision (ECCV), pages 234\u2013250.","DOI":"10.1007\/978-3-030-01228-1_15"},{"key":"6373_CR22","unstructured":"Koh, P.\u00a0W., Nguyen, T., Tang, Y.\u00a0S., Mussmann, S., Pierson, E., Kim, B., & Liang, P. (2020). Concept bottleneck models. In H.\u00a0D. III and A.\u00a0Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research. PMLR, 7."},{"key":"6373_CR23","unstructured":"Landing AI. 2020 State of AI-Based Machine Vision High confidence, growing adoption and many challenges-trends and insights based on a survey of 110 companies. Technical report, 2020. URL https:\/\/landing.ai\/wp-content\/uploads\/2020\/11\/MachineVisionSurvey.pdf."},{"key":"6373_CR24","doi-asserted-by":"crossref","unstructured":"Lin, H.\u00a0W., Tegmark, M., & Rolnick, D. (2017). Why does deep and cheap learning work so well? Journal of Statistical Physics, 168(6), 1223\u20131247, 9 ISSN 1572-9613.","DOI":"10.1007\/s10955-017-1836-5"},{"key":"6373_CR25","doi-asserted-by":"crossref","unstructured":"Sato, M., & Tsukimoto, H. (2001). Rule extraction from neural networks via decision tree induction. In IJCNN\u201901. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222), volume\u00a03, pages 1870\u20131875 vol.3.","DOI":"10.1109\/IJCNN.2001.938448"},{"key":"6373_CR26","doi-asserted-by":"crossref","unstructured":"Schaaf, N., Huber, M., & Maucher, J. (2019). Enhancing decision tree based interpretation of deep neural networks through l1-orthogonal regularization. In 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA), pages 42\u201349.","DOI":"10.1109\/ICMLA.2019.00016"},{"key":"6373_CR27","doi-asserted-by":"crossref","unstructured":"Selvaraju, R.\u00a0R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017). Grad-cam: Visual explanations from deep networks via gradient-based localization. In 2017 IEEE International Conference on Computer Vision (ICCV), pages 618\u2013626.","DOI":"10.1109\/ICCV.2017.74"},{"key":"6373_CR28","unstructured":"Simonyan, K., & Zisserman, A. (2015). Very deep convolutional networks for large-scale image recognition. In Bengio, Y. & LeCun, Y. (eds), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings."},{"key":"6373_CR29","doi-asserted-by":"crossref","unstructured":"Utkin, L.\u00a0V., Drobintsev, P.\u00a0D., Kovalev, M. & Konstantinov, A.\u00a0V. (2021). Combining an autoencoder and a variational autoencoder for explaining the machine learning model predictions. In 28th Conference of Open Innovations Association, FRUCT 2021, Moscow, Russia, January 27-29, 2021. IEEE.","DOI":"10.23919\/FRUCT50888.2021.9347612"},{"key":"6373_CR30","doi-asserted-by":"publisher","first-page":"583","DOI":"10.1016\/j.patcog.2016.06.005","volume":"61","author":"N Van Noord","year":"2017","unstructured":"Van Noord, N., & Postma, E. (2017). Learning scale-variant and scale-invariant features for deep image classification. Pattern Recognition, 61, 583\u2013592.","journal-title":"Pattern Recognition"},{"key":"6373_CR31","doi-asserted-by":"crossref","unstructured":"Wang, P., Kong, X., Guo, W. & Zhang, X. (2021). Exclusive feature constrained class activation mapping for better visual explanation. IEEE Access, 9.","DOI":"10.1109\/ACCESS.2021.3073465"},{"key":"6373_CR32","unstructured":"Xu, Y., Xiao, T., Zhang, J., Yang, K. & Zhang, Z. (2014). Scale-invariant convolutional neural networks. arXiv preprint arXiv:1411.6369."},{"key":"6373_CR33","volume-title":"Advances in Neural Information Processing Systems","author":"C.-K Yeh","year":"2020","unstructured":"Yeh, C. .-K., Kim, B., Arik, S., Li, C. .-L., Pfister, T., & Ravikumar, P. (2020). On completeness-aware concept-based explanations in deep neural networks. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, & H. Lin (Eds.), Advances in Neural Information Processing Systems.  (Vol. 33). Curran Associates, Inc."},{"key":"6373_CR34","doi-asserted-by":"crossref","unstructured":"Zhang, Q., Yang, Y., Ma, H. & Wu, Y.\u00a0N. (2019). Interpreting cnns via decision trees. In 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6254\u20136263.","DOI":"10.1109\/CVPR.2019.00642"},{"key":"6373_CR35","first-page":"3","volume":"16","author":"Z Zhou","year":"2003","unstructured":"Zhou, Z., Jiang, Y., & Chen, S. (2003). Extracting symbolic rules from trained neural network ensembles. AI Communications, 16, 3\u201315.","journal-title":"AI Communications"},{"key":"6373_CR36","doi-asserted-by":"crossref","unstructured":"Zilke, J.\u00a0R., Loza\u00a0Menc\u00eda, E., & Janssen. F. (2016). Deepred \u2013 rule extraction from deep neural networks. In Calders, T., Ceci, M. & Malerba, D. (eds), Discovery Science, pages 457\u2013473, Cham, Springer International Publishing. ISBN 978-3-319-46307-0.","DOI":"10.1007\/978-3-319-46307-0_29"}],"container-title":["Machine Learning"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-023-06373-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10994-023-06373-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-023-06373-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,25]],"date-time":"2023-10-25T13:14:57Z","timestamp":1698239697000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10994-023-06373-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,21]]},"references-count":36,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2023,11]]}},"alternative-id":["6373"],"URL":"https:\/\/doi.org\/10.1007\/s10994-023-06373-2","relation":{},"ISSN":["0885-6125","1573-0565"],"issn-type":[{"value":"0885-6125","type":"print"},{"value":"1573-0565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,21]]},"assertion":[{"value":"18 October 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 May 2023","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 July 2023","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 August 2023","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflicts of interest\/competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}