{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,19]],"date-time":"2026-08-19T17:42:34Z","timestamp":1787161354214,"version":"build-2736575974"},"publisher-location":"Cham","reference-count":27,"publisher":"Springer Nature Switzerland","isbn-type":[{"value":"9783032083166","type":"print"},{"value":"9783032083173","type":"electronic"}],"license":[{"start":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T00:00:00Z","timestamp":1760227200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T00:00:00Z","timestamp":1760227200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Convolutional neural networks (CNNs) for time series classification (TSC) are being increasingly used in applications ranging from quality prediction to medical diagnosis. The black box nature of these models makes understanding their prediction process difficult. This issue is crucial because CNNs are prone to learning shortcuts and biases, compromising their robustness and alignment with human expectations. To assess whether such mechanisms are being used and the associated risk, it is essential to provide model explanations that reflect the inner workings of the model. Concept Extraction (CE) methods offer such explanations, but have mostly been developed for the image domain so far, leaving a gap in the time series domain. In this work, we present a CE and localization method tailored to the time series domain, based on the ideas of CE methods for images. We propose the novel method ECLAD-ts, which provides post-hoc global explanations based on how the models encode subsets of the input at different levels of abstraction. For this, concepts are produced by clustering timestep-wise aggregations of CNN activation maps, and their importance is computed based on their impact on the prediction process. We evaluate our method on synthetic and natural datasets. Furthermore, we assess the advantages and limitations of CE in\u00a0time series through empirical results. Our results show that ECLAD-ts effectively explains models\u00a0by leveraging their internal representations, providing useful insights about their prediction process.<\/jats:p>","DOI":"10.1007\/978-3-032-08317-3_5","type":"book-chapter","created":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T03:36:55Z","timestamp":1760153815000},"page":"90-112","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Concept Extraction for\u00a0Time Series with\u00a0ECLAD-ts"],"prefix":"10.1007","author":[{"given":"Antonia","family":"Holzapfel","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andres Felipe","family":"Posada Moreno","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sebastian","family":"Trimpe","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,10,12]]},"reference":[{"key":"5_CR1","doi-asserted-by":"crossref","unstructured":"Burkart, N., Huber, M.F.: A survey on the explainability of supervised machine learning. JAIR (2021)","DOI":"10.1613\/jair.1.12228"},{"key":"5_CR2","doi-asserted-by":"crossref","unstructured":"Crabb\u00e9, J., van der Schaar, M.: Concept activation regions: a generalized framework for concept-based explanations. In: NeurIPS, vol. 35, pp. 2590\u20132607 (2022)","DOI":"10.52202\/068431-0188"},{"key":"5_CR3","unstructured":"Das, A., Rad, P.: Opportunities and challenges in explainable artificial intelligence (XAI): a survey. arXiv preprint arXiv:2006.11371 (2020)"},{"key":"5_CR4","doi-asserted-by":"crossref","unstructured":"Das, S., Xu, P., Dai, Z., Endert, A., Ren, L.: Interpreting deep neural networks through prototype factorization. In: International Conference on Data Mining Workshops (ICDMW), pp. 448\u2013457. IEEE (2020)","DOI":"10.1109\/ICDMW51313.2020.00068"},{"key":"5_CR5","unstructured":"Dau, H.A., et al.: The UCR time series classification archive (2018)"},{"key":"5_CR6","doi-asserted-by":"publisher","unstructured":"Decker, T., Lebacher, M., Tresp, V.: Explaining deep neural networks for bearing fault detection with vibration concepts. In: INDIN, pp.\u00a01\u20136 (2023). https:\/\/doi.org\/10.1109\/INDIN51400.2023.10218170","DOI":"10.1109\/INDIN51400.2023.10218170"},{"issue":"11","key":"5_CR7","doi-asserted-by":"publisher","first-page":"665","DOI":"10.1038\/s42256-020-00257-z","volume":"2","author":"R Geirhos","year":"2020","unstructured":"Geirhos, R., et al.: Shortcut learning in deep neural networks. Nat. Mach. Intell. 2(11), 665\u2013673 (2020). https:\/\/doi.org\/10.1038\/s42256-020-00257-z","journal-title":"Nat. Mach. Intell."},{"key":"5_CR8","unstructured":"Ghorbani, A., Wexler, J., Zou, J.Y., Kim, B.: Towards automatic concept-based explanations. In: NeurIPS. vol.\u00a032. Curran Associates, Inc. (2019)"},{"key":"5_CR9","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770\u2013778 (2016)","DOI":"10.1109\/CVPR.2016.90"},{"key":"5_CR10","doi-asserted-by":"crossref","unstructured":"Hills, J., Lines, J., Baranauskas, E., Mapp, J., Bagnall, A.: Classification of time series by shapelet transformation. Data Mining and Knowledge Discovery (2014)","DOI":"10.1007\/s10618-013-0322-1"},{"key":"5_CR11","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., Van Der\u00a0Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: CVPR, pp. 4700\u20134708 (2017)","DOI":"10.1109\/CVPR.2017.243"},{"issue":"6","key":"5_CR12","doi-asserted-by":"publisher","first-page":"1936","DOI":"10.1007\/s10618-020-00710-y","volume":"34","author":"H Ismail Fawaz","year":"2020","unstructured":"Ismail Fawaz, H., et al.: Inceptiontime: finding alexnet for time series classification. Data Min. Knowl. Disc. 34(6), 1936\u20131962 (2020)","journal-title":"Data Min. Knowl. Disc."},{"key":"5_CR13","doi-asserted-by":"crossref","unstructured":"Jiang, X., Hu, Z., Wang, S., Zhang, Y.: Deep learning for medical image-based cancer diagnosis. Cancers 15(14) (2023)","DOI":"10.3390\/cancers15143608"},{"key":"5_CR14","doi-asserted-by":"publisher","unstructured":"Kamakshi, V., Gupta, U., Krishnan, N.C.: PACE: posthoc architecture-agnostic concept extractor for explaining CNNs. In: IJCNN, pp.\u00a01\u20138 (2021). https:\/\/doi.org\/10.1109\/IJCNN52387.2021.9534369","DOI":"10.1109\/IJCNN52387.2021.9534369"},{"key":"5_CR15","unstructured":"Kim, B., et al.: Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV). In: ICML, pp. 2668\u20132677. PMLR (2018)"},{"key":"5_CR16","doi-asserted-by":"crossref","unstructured":"Kraus, M., Steinmann, D., W\u00fcst, A., Kokozinski, A., Kersting, K.: Right on time: revising time series models by constraining their explanations. arXiv preprint arXiv:2402.12921 (2024)","DOI":"10.1007\/978-3-032-06109-6_28"},{"key":"5_CR17","doi-asserted-by":"publisher","unstructured":"K\u00fcsters, F., Schichtel, P., Ahmed, S., Dengel, A.: Conceptual explanations of neural network prediction for time series. In: IJCNN, pp.\u00a01\u20136 (2020). https:\/\/doi.org\/10.1109\/IJCNN48605.2020.9207341","DOI":"10.1109\/IJCNN48605.2020.9207341"},{"key":"5_CR18","unstructured":"Lundberg, S.M., Lee, S.I.: A unified approach to interpreting model predictions. In: NeurIPS, pp. 4768\u20134777. NIPS, Curran Associates Inc. (2017)"},{"key":"5_CR19","doi-asserted-by":"crossref","unstructured":"Madsen, A.G., Lehn-Schi\u00f8ler, W.T., J\u00f3nsd\u00f3ttir, \u00c1., Arnard\u00f3ttir, B., Hansen, L.K.: Concept-based explainability for an EEG transformer model. In: MLSP, pp.\u00a01\u20136. IEEE (2023)","DOI":"10.1109\/MLSP55844.2023.10285992"},{"key":"5_CR20","unstructured":"Obermair, C., Fuchs, A., Pernkopf, F., Felsberger, L., Apollonio, A., Wollmann, D.: Example or Prototype? Learning Concept-Based Explanations in Time-Series. In: ACML, pp. 816\u2013831. PMLR (2023)"},{"issue":"11","key":"5_CR21","doi-asserted-by":"publisher","first-page":"4495","DOI":"10.1007\/s10994-023-06373-2","volume":"112","author":"AF Posada-Moreno","year":"2023","unstructured":"Posada-Moreno, A.F., Kreisk\u00f6ther, L., Glander, T., Trimpe, S.: Scale-preserving automatic concept extraction (SPACE). Mach. Learn. 112(11), 4495\u20134525 (2023). https:\/\/doi.org\/10.1007\/s10994-023-06373-2","journal-title":"Mach. Learn."},{"key":"5_CR22","doi-asserted-by":"crossref","unstructured":"Posada-Moreno, A.F., M\u00fcller, K., Brillowski, F., Solowjow, F., Gries, T., Trimpe, S.: Scalable Concept Extraction in\u00a0Industry\u00a04.0. In: Longo, L. (ed.) xAI 2023. CCIS, vol. 1903, pp. 512\u2013535. Springer, Cham (2023)","DOI":"10.1007\/978-3-031-44070-0_26"},{"key":"5_CR23","doi-asserted-by":"crossref","unstructured":"Posada-Moreno, A.F., Surya, N., Trimpe, S.: ECLAD: extracting concepts with local aggregated descriptors. Pattern Recogn. 147, 110146 (2023)","DOI":"10.1016\/j.patcog.2023.110146"},{"key":"5_CR24","doi-asserted-by":"crossref","unstructured":"Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-cam: visual explanations from deep networks via gradient-based localization. In: ICCV, pp. 618\u2013626 (2017)","DOI":"10.1109\/ICCV.2017.74"},{"key":"5_CR25","unstructured":"Yeh, C.K., Kim, B., Arik, S., Li, C.L., Pfister, T., Ravikumar, P.: On completeness-aware concept-based explanations in deep neural networks. In: NeurIPS, vol.\u00a033, pp. 20554\u201320565 (2020)"},{"key":"5_CR26","doi-asserted-by":"crossref","unstructured":"Younis, R., Zerr, S., Ahmadi, Z.: Multivariate time series analysis: an interpretable CNN-based model. In: International Conference on Data Science and Advanced Analytics (DSAA), pp. 1\u201310. IEEE (2022)","DOI":"10.1109\/DSAA54385.2022.10032335"},{"key":"5_CR27","doi-asserted-by":"crossref","unstructured":"Yuan, X., et al.: Quality prediction modeling for industrial processes using multiscale attention-based convolutional neural network. IEEE Trans. Cybern. (2024)","DOI":"10.1109\/TCYB.2024.3365068"}],"container-title":["Communications in Computer and Information Science","Explainable Artificial Intelligence"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/978-3-032-08317-3_5","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,19]],"date-time":"2026-08-19T17:13:05Z","timestamp":1787159585000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/978-3-032-08317-3_5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,12]]},"ISBN":["9783032083166","9783032083173"],"references-count":27,"URL":"https:\/\/doi.org\/10.1007\/978-3-032-08317-3_5","relation":{},"ISSN":["1865-0929","1865-0937"],"issn-type":[{"value":"1865-0929","type":"print"},{"value":"1865-0937","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,12]]},"assertion":[{"value":"12 October 2025","order":1,"name":"first_online","label":"First Online","group":{"name":"ChapterHistory","label":"Chapter History"}},{"value":"This work is partially funded by\u00a0the Deutsche Forschungsgemeinschaft (DFG) within the Priority Program SPP 2422 (TR 1433\/3-1) and Germany\u2019s Excellence Strategy\u2013EXC-2023 Internet of Production \u2013 390621612.","order":1,"name":"Ethics","label":"Disclosure of Interests","group":{"name":"EthicsHeading","label":"Ethics"}},{"value":"xAI","order":1,"name":"conference_acronym","label":"Conference Acronym","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"World Conference on Explainable Artificial Intelligence","order":2,"name":"conference_name","label":"Conference Name","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Istanbul","order":3,"name":"conference_city","label":"Conference City","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"T\u00fcrkiye","order":4,"name":"conference_country","label":"Conference Country","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"2025","order":5,"name":"conference_year","label":"Conference Year","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"9 July 2025","order":7,"name":"conference_start_date","label":"Conference Start Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"11 July 2025","order":8,"name":"conference_end_date","label":"Conference End Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"3","order":9,"name":"conference_number","label":"Conference Number","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"xai2025","order":10,"name":"conference_id","label":"Conference ID","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"https:\/\/xaiworldconference.com\/2025\/","order":11,"name":"conference_url","label":"Conference URL","group":{"name":"ConferenceInfo","label":"Conference Information"}}]}}