{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T04:42:20Z","timestamp":1759984940492,"version":"build-2065373602"},"reference-count":25,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T00:00:00Z","timestamp":1759968000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["499552394"],"award-info":[{"award-number":["499552394"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002347","name":"Bundesministerium f\u00fcr Bildung und Forschung","doi-asserted-by":"publisher","award":["1IS19077C"],"award-info":[{"award-number":["1IS19077C"]}],"id":[{"id":"10.13039\/501100002347","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Robot. AI"],"abstract":"<jats:p>An important but unresolved question in deep learning for EEG decoding is which features neural networks learn to solve the task. Prior interpretability studies have mainly explained individual predictions, analyzed the use of established EEG features, or examined subnetworks of larger models. In contrast, we apply interpretability methods to uncover features learned by the complete network. Specifically, we introduce two complementary architectures with dedicated visualization techniques to obtain an approximate understanding of the full network trained on binary classification into nonpathological and pathological EEG. First, we use invertible networks\u2014networks that are designed to be invertible\u2014to generate prototypical input signals for each class. Second, we design a very compact network that is fully visualizable, while still retaining reasonable decoding performance. Through these visualizations, we find both expected features like higher-amplitude oscillations in the delta and theta frequency bands in the temporal region for the pathological class as well as surprising differences in the very low sub-delta frequencies below 0.5 Hz. Closer investigation reveals higher spectral amplitudes for the healthy class at the frontal sensors in these sub-delta frequencies, an unexpected feature that the proposed visualizations helped identify. Overall, the study shows the potential of visualizations to understand the network prediction function without relying on specific predefined features.<\/jats:p>","DOI":"10.3389\/frobt.2025.1625732","type":"journal-article","created":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T04:23:08Z","timestamp":1759983788000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["New avenues for understanding what deep networks learn from EEG"],"prefix":"10.3389","volume":"12","author":[{"given":"Robin T.","family":"Schirrmeister","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tonio","family":"Ball","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2025,10,9]]},"reference":[{"key":"B1","article-title":"Training normalizing flows with the information bottleneck for competitive generative classification","volume-title":"Advances in neural information processing Systems 33: annual conference on neural information processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual","author":"Ardizzone","year":"2020"},{"key":"B2","doi-asserted-by":"publisher","first-page":"e0130140","DOI":"10.1371\/journal.pone.0130140","article-title":"On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation","volume":"10","author":"Bach","year":"2015","journal-title":"PloS one"},{"key":"B3","doi-asserted-by":"publisher","first-page":"108117","DOI":"10.1016\/j.biopsycho.2021.108117","article-title":"Deep learning applied to electroencephalogram data in mental disorders: a systematic review","volume":"162","author":"de Bardeci","year":"2021","journal-title":"Biol. Psychol."},{"key":"B4","doi-asserted-by":"publisher","first-page":"251","DOI":"10.3389\/fnins.2020.00251","article-title":"Deep learning convolutional neural networks discriminate adult adhd from healthy individuals on the basis of event-related spectral eeg","volume":"14","author":"Dubreuil-Vall","year":"2020","journal-title":"Front. Neurosci."},{"key":"B5","doi-asserted-by":"publisher","first-page":"117021","DOI":"10.1016\/j.neuroimage.2020.117021","article-title":"Machine-learning-based diagnostics of eeg pathology","volume":"220","author":"Gemein","year":"2020","journal-title":"NeuroImage"},{"key":"B6","doi-asserted-by":"publisher","first-page":"e745","DOI":"10.1016\/s2589-7500(21)00208-9","article-title":"The false hope of current approaches to explainable artificial intelligence in health care","volume":"3","author":"Ghassemi","year":"2021","journal-title":"Lancet Digital Health"},{"key":"B7","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/SPMB.2014.7002953","article-title":"The tuh eeg corpus: a big data resource for automated eeg interpretation","volume-title":"2014 IEEE signal processing in medicine and biology symposium (SPMB)","author":"Harati","year":"2014"},{"key":"B8","doi-asserted-by":"publisher","first-page":"96","DOI":"10.1016\/j.neuroimage.2013.10.067","article-title":"On the interpretation of weight vectors of linear models in multivariate neuroimaging","volume":"87","author":"Haufe","year":"2014","journal-title":"NeuroImage"},{"key":"B9","first-page":"4615","article-title":"Semi-supervised learning with normalizing flows","volume-title":"Proceedings of the 37th international conference on machine learning, ICML 2020, 13-18 July 2020, virtual event","author":"Izmailov","year":"2020"},{"key":"B10","doi-asserted-by":"publisher","first-page":"60141","DOI":"10.1109\/ACCESS.2022.3176367","article-title":"An interpretable deep learning classifier for epileptic seizure prediction using eeg data","volume":"10","author":"Jemal","year":"2022","journal-title":"IEEE Access"},{"key":"B11","first-page":"10236","article-title":"Glow: generative flow with invertible 1x1 convolutions","volume-title":"Advances in neural information Processing systems 31: annual conference on neural Information processing systems 2018, NeurIPS 2018, December 3-8, 2018, Montr\u00e9al, Canada","author":"Kingma","year":"2018"},{"key":"B12","volume-title":"Automated interpretation of abnormal adult electroencephalography","author":"L\u00f3pez de Diego","year":"2017"},{"key":"B13","article-title":"SGDR: stochastic gradient descent with warm restarts","volume-title":"5th international conference on learning representations, ICLR 2017, toulon, France, April 24-26, 2017, conference track proceedings","author":"Loshchilov","year":"2017"},{"key":"B14","article-title":"Decoupled weight decay regularization","volume-title":"7th international conference on learning representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019","author":"Loshchilov","year":"2019"},{"key":"B15","doi-asserted-by":"publisher","first-page":"196","DOI":"10.3389\/fnins.2016.00196","article-title":"The temple university hospital EEG data corpus","volume":"10","author":"Obeid","year":"2016","journal-title":"Front. Neurosci."},{"key":"B16","doi-asserted-by":"publisher","first-page":"e214","DOI":"10.1016\/s2589-7500(22)00029-2","article-title":"Explainability and artificial intelligence in medicine","volume":"4","author":"Reddy","year":"2022","journal-title":"Lancet Digital Health"},{"key":"B17","doi-asserted-by":"publisher","first-page":"36672","DOI":"10.1109\/access.2022.3161489","article-title":"Eeg-itnet: an explainable inception temporal convolutional network for motor imagery classification","volume":"10","author":"Salami","year":"2022","journal-title":"IEEE Access"},{"key":"B18","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/spmb.2017.8257015","article-title":"Deep learning with convolutional neural networks for decoding and visualization of EEG pathology","author":"Schirrmeister","year":"2017","journal-title":"arXiv Prepr. arXiv:1708.08012"},{"key":"B19","first-page":"618","article-title":"Grad-cam: visual explanations from deep networks via gradient-based localization","volume-title":"IEEE international Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017","author":"Selvaraju","year":"2017"},{"key":"B20","article-title":"A note on the evaluation of generative models","volume-title":"4th international conference on learning representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, conference track proceedings","author":"Theis","year":"2016"},{"key":"B21","doi-asserted-by":"publisher","first-page":"4793","DOI":"10.1109\/TNNLS.2020.3027314","article-title":"A survey on explainable artificial intelligence (xai): toward medical xai","volume":"32","author":"Tjoa","year":"2021","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"B22","article-title":"Interpretable epilepsy detection in routine, interictal eeg data using deep learning","volume-title":"Proceedings of the machine learning for health NeurIPS workshop","author":"Uyttenhove","year":"2020"},{"key":"B23","doi-asserted-by":"publisher","first-page":"1055","DOI":"10.3390\/jcm8071055","article-title":"Deep learning based on event-related eeg differentiates children with adhd from healthy controls","volume":"8","author":"Vahid","year":"2019","journal-title":"J. Clin. Med."},{"key":"B24","doi-asserted-by":"publisher","first-page":"102470","DOI":"10.1016\/j.media.2022.102470","article-title":"Explainable artificial intelligence (xai) in deep learning-based medical image analysis","volume":"79","author":"van der Velden","year":"2022","journal-title":"Med. Image Anal."},{"key":"B25","first-page":"128","article-title":"Eeg-based depression detection using convolutional neural network with demographic attention mechanism","volume-title":"2020 42nd annual international conference of the IEEE engineering in medicine and biology society (EMBC)","author":"Zhang","year":"2020"}],"container-title":["Frontiers in Robotics and AI"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2025.1625732\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T04:23:12Z","timestamp":1759983792000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2025.1625732\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,9]]},"references-count":25,"alternative-id":["10.3389\/frobt.2025.1625732"],"URL":"https:\/\/doi.org\/10.3389\/frobt.2025.1625732","relation":{},"ISSN":["2296-9144"],"issn-type":[{"value":"2296-9144","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,9]]},"article-number":"1625732"}}