{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,24]],"date-time":"2026-10-24T19:09:10Z","timestamp":1792868950383,"version":"4.5.0"},"reference-count":49,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,10,21]],"date-time":"2025-10-21T00:00:00Z","timestamp":1761004800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Hum. Neurosci."],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>\n                      Inner speech\u2014the covert articulation of words in one\u2019s mind\u2014is a fundamental phenomenon in human cognition with growing interest across BCI. This pilot study evaluates and compares deep learning models for inner-speech classification using non-invasive EEG derived from a bimodal EEG-fMRI dataset (4 participants, 8 words). The study assesses a compact CNN (EEGNet) and a spectro-temporal Transformer using leave-one-subject-out validation, reporting accuracy. Macro-F\n                      <jats:sub>1<\/jats:sub>\n                      , precision, and recall.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>This study aims to evaluate and compare deep learning models for inner speech classification using non-invasive electroencephalography (EEG) data, derived from a bimodal EEG-fMRI dataset. The goal is to assess the performance and generalizability of two architectures: the compact convolutional EEGNet and a novel spectro-temporal Transformer.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>\n                      Data were obtained from four healthy participants who performed structured inner speech tasks involving eight target words. EEG signals were preprocessed and segmented into epochs for each imagined word. EEGNet and Transformer models were trained using a leave-one-subject-out (LOSO) cross-validation strategy. Performance metrics included accuracy, macro-averaged F\n                      <jats:sub>1<\/jats:sub>\n                      score, precision, and recall. An ablation study examined the contribution of Transformer components, including wavelet decomposition and self-attention mechanisms.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      The spectro-temporal Transformer achieved the highest classification accuracy (82.4%) and macro-F\n                      <jats:sub>1<\/jats:sub>\n                      score (0.70), outperforming both the standard and improved EEGNet models. Discriminative power was also substantially improved by using wavelet-based time-frequency features and attention mechanisms. Results showed that confusion patterns of social word categories outperformed those of number concepts, corresponding to different mental processing strategies.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Deep learning models, in particular attention-based Transformers, demonstrate great promise in decoding internal speech from EEG. These findings lay the groundwork for non-invasive, real-time BCIs for communication rehabilitation in severely disabled patients. Future work will take into account vocabulary expansion, wider participant variety, and real-time validation in clinical settings.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fnhum.2025.1668935","type":"journal-article","created":{"date-parts":[[2025,10,21]],"date-time":"2025-10-21T05:37:40Z","timestamp":1761025060000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["Deep learning for inner speech recognition: a pilot comparative study of EEGNet and a spectro-temporal Transformer on bimodal EEG-fMRI data"],"prefix":"10.3389","volume":"19","author":[{"given":"Ahmad H.","family":"Milyani","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Eyad Talal","family":"Attar","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2025,10,21]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","first-page":"874","DOI":"10.1038\/s41598-018-37359-z","article-title":"Towards reconstructing intelligible speech from the human auditory cortex","volume":"9","author":"Akbari","year":"2019","journal-title":"Sci. Rep."},{"key":"ref2","doi-asserted-by":"publisher","first-page":"931","DOI":"10.1037\/bul0000021","article-title":"Inner speech: development, cognitive functions, phenomenology, and neurobiology","volume":"141","author":"Alderson-Day","year":"2015","journal-title":"Psychol. Bull."},{"key":"ref3","doi-asserted-by":"publisher","first-page":"036019","DOI":"10.1088\/1741-2552\/ab0c59","article-title":"Speech synthesis from ECoG using densely connected 3D convolutional neural networks","volume":"16","author":"Angrick","year":"2019","journal-title":"J. Neural Eng."},{"key":"ref9004","doi-asserted-by":"publisher","first-page":"217","DOI":"10.4103\/jmss.jmss_24_22","article-title":"Integrated biosignal analysis to provide biomarkers for recognizing time perception difficulties","volume":"13","author":"Attar","year":"2023","journal-title":"Journal of Medical Signals & Sensors"},{"key":"ref9003","doi-asserted-by":"publisher","first-page":"86025","DOI":"10.1007\/s11042-024-19212-w","article-title":"The consequences of eye tracking on brain and heart coherence","volume":"83","author":"Attar","year":"2024","journal-title":"Multimedia Tools and Applications"},{"key":"ref9005","doi-asserted-by":"publisher","first-page":"2525","DOI":"10.3390\/diagnostics14222525","article-title":"Enhancing epilepsy seizure detection through advanced EEG preprocessing techniques and peak-to-peak amplitude fluctuation analysis","volume":"14","author":"Bahhah","year":"2024","journal-title":"Diagnostics"},{"key":"ref4","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1016\/S0304-3940(01)02535-6","article-title":"Event-related theta power increases in the human EEG during online sentence processing","volume":"383","author":"Bastiaansen","year":"2005","journal-title":"Neurosci. Lett."},{"key":"ref5","doi-asserted-by":"publisher","first-page":"634","DOI":"10.1097\/WCO.0b013e328315ee2d","article-title":"Brain\u2013computer interface in paralysis","volume":"21","author":"Birbaumer","year":"2008","journal-title":"Curr. Opin. Neurol."},{"key":"ref6","first-page":"782","article-title":"Transformer interpretability beyond attention visualization","author":"Chefer","year":"2021"},{"key":"ref19","doi-asserted-by":"publisher","first-page":"7084","DOI":"10.3390\/s24217084","article-title":"Steady-State Visual Evoked Potential-Based Brain-Computer Interface System for Enhanced Human Activity Monitoring and Assessment","volume":"24","author":"Chen","year":"2024","journal-title":"Sensors (Basel, Switzerland)"},{"key":"ref9","doi-asserted-by":"publisher","first-page":"031001","DOI":"10.1088\/1741-2552\/ab0ab5","article-title":"Deep learning for electroencephalogram (EEG) classification tasks: a review","volume":"16","author":"Craik","year":"2019","journal-title":"J. Neural Eng."},{"key":"ref10","doi-asserted-by":"publisher","first-page":"970","DOI":"10.1126\/science.284.5416.970","article-title":"Sources of mathematical thinking: behavioral and brain-imaging evidence","volume":"284","author":"Dehaene","year":"1999","journal-title":"Science"},{"key":"ref11","doi-asserted-by":"publisher","first-page":"26","DOI":"10.1109\/RBME.2024.3449790","article-title":"Non-invasive brain\u2013computer interfaces: state of the art and trends","volume":"18","author":"Edelman","year":"2025","journal-title":"IEEE Rev. Biomed. Eng."},{"key":"ref9008","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s11042-023-15052-2","article-title":"EEG seizure detection: concepts, techniques, challenges, and future trends","author":"Ein Shoka","year":"2023","journal-title":"Multimedia tools and applications"},{"key":"ref12","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR46437.2021.00072","article-title":"Removing diffraction image artifacts in under-display camera via dynamic skip connection network","author":"Feng","year":"2021"},{"key":"ref9009","doi-asserted-by":"publisher","DOI":"10.18112\/openneuro.ds006033.v1.0.1","article-title":"Synchronous EEG and fMRI dataset on inner speech","author":"Foteini","year":"2025","journal-title":"OpenNeuro."},{"key":"ref9010","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1016\/j.isci.2018.09.017","volume":"8","author":"Goodhill","year":"2018","journal-title":"Theoretical Models of Neural Development. iScience"},{"key":"ref13","doi-asserted-by":"publisher","first-page":"267","DOI":"10.3389\/fnins.2013.00267","article-title":"MEG and EEG data analysis with MNE-Python","volume":"7","author":"Gramfort","year":"2013","journal-title":"Front. Neurosci."},{"key":"ref14","article-title":"Deep compression: compressing deep neural networks with pruning, trained quantization and Huffman coding","author":"Han","year":"2015"},{"key":"ref15","doi-asserted-by":"publisher","first-page":"399","DOI":"10.1109\/TBME.2019.2913914","article-title":"Transfer learning for brain\u2013computer interfaces: a Euclidean space data alignment approach","volume":"67","author":"He","year":"2019","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref16","doi-asserted-by":"publisher","first-page":"217","DOI":"10.3389\/fnins.2015.00217","article-title":"Brain-to-text: decoding spoken sentences from phone representations in the brain","volume":"9","author":"Herff","year":"2015","journal-title":"Front. Neurosci."},{"key":"ref17","article-title":"Decoding covert speech from EEG using a functional areas spatio-temporal Transformer (FAST)","author":"Jiang","year":"2025"},{"key":"ref18","doi-asserted-by":"publisher","first-page":"653659","DOI":"10.3389\/fnhum.2021.653659","article-title":"BENDR: using Transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data","volume":"15","author":"Kostas","year":"2021","journal-title":"Front. Hum. Neurosci."},{"key":"ref20","doi-asserted-by":"publisher","first-page":"056013","DOI":"10.1088\/1741-2552\/aace8c","article-title":"EEGNet: a compact convolutional neural network for EEG-based brain\u2013computer interfaces","volume":"15","author":"Lawhern","year":"2018","journal-title":"J. Neural Eng."},{"key":"ref21","article-title":"EEG-Transformer: self-attention from Transformer architecture for decoding EEG of imagined speech","author":"Lee","year":"2021"},{"key":"ref38","doi-asserted-by":"publisher","first-page":"1517273","DOI":"10.3389\/fnhum.2025.1517273","article-title":"A spatial and temporal transformer-based EEG emotion recognition in VR environment","volume":"19","author":"Li","year":"2025","journal-title":"Frontiers in human neuroscience"},{"key":"ref22","doi-asserted-by":"publisher","first-page":"867281","DOI":"10.3389\/fnhum.2022.867281","article-title":"A state-of-the-art review of EEG-based imagined speech decoding","volume":"16","author":"L\u00f3pez-Bernal","year":"2022","journal-title":"Front. Hum. Neurosci."},{"key":"ref23","doi-asserted-by":"publisher","first-page":"031005","DOI":"10.1088\/1741-2552\/aab2f2","article-title":"A review of classification algorithms for EEG-based brain\u2013computer interfaces: a 10 year update","volume":"15","author":"Lotte","year":"2018","journal-title":"J. Neural Eng."},{"key":"ref24","doi-asserted-by":"publisher","first-page":"575","DOI":"10.1038\/s41593-020-0608-8","article-title":"Machine translation of cortical activity to text with an encoder-decoder framework","volume":"23","author":"Makin","year":"2020","journal-title":"Nat. Neurosci."},{"key":"ref25","doi-asserted-by":"publisher","first-page":"14","DOI":"10.3389\/fneng.2014.00014","article-title":"Decoding spectrotemporal features of overt and covert speech from the human cortex","volume":"7","author":"Martin","year":"2014","journal-title":"Front. Neuroeng."},{"key":"ref27","doi-asserted-by":"publisher","first-page":"387","DOI":"10.1016\/j.brainresbull.2007.06.013","article-title":"Self-awareness and the left inferior frontal gyrus: inner speech use during self-related processing","volume":"74","author":"Morin","year":"2007","journal-title":"Brain Res. Bull."},{"key":"ref8008","doi-asserted-by":"publisher","DOI":"10.1186\/s13634-017-0483-z","article-title":"Classification of imagined speech using EEG signals","volume":"43","author":"Nguyen","year":"2017","journal-title":"EURASIP J. Adv. Signal Process."},{"key":"ref9007","doi-asserted-by":"publisher","first-page":"8837954","DOI":"10.1155\/2020\/8837954","article-title":"A Novel Bayesian Approach for EEG Source Localization","volume":"2020","author":"Oikonomou","year":"2020","journal-title":"Computational intelligence and neuroscience"},{"key":"ref30","doi-asserted-by":"publisher","first-page":"1186594","DOI":"10.3389\/fnhum.2023.1186594","article-title":"Multiclass classification of imagined speech EEG using noise-assisted multivariate empirical mode decomposition and multireceptive field convolutional neural network","volume":"17","author":"Park","year":"2023","journal-title":"Front. Hum. Neurosci."},{"key":"ref31","doi-asserted-by":"publisher","first-page":"220","DOI":"10.1016\/j.bbr.2013.12.034","article-title":"What is that little voice inside my head? Inner speech phenomenology, its role in cognitive performance, and its relation to self-monitoring","volume":"261","author":"Perrone-Bertolotti","year":"2014","journal-title":"Behav. Brain Res."},{"key":"ref32","doi-asserted-by":"publisher","first-page":"816","DOI":"10.1016\/j.neuroimage.2012.04.062","article-title":"A review and synthesis of the first 20 years of PET and fMRI studies of heard speech, spoken language, and reading","volume":"62","author":"Price","year":"2012","journal-title":"NeuroImage"},{"key":"ref34","article-title":"Inner speech EEG-fMRI dataset for covert speech decoding","volume-title":"OpenNeuro","author":"Rezazadeh Sereshkeh","year":"2021"},{"key":"ref35","article-title":"Explainable artificial intelligence: understanding, visualizing and interpreting deep learning models","author":"Samek","year":"2017"},{"key":"ref36","doi-asserted-by":"publisher","first-page":"5391","DOI":"10.1002\/hbm.23730","article-title":"Deep learning with convolutional neural networks for EEG decoding and visualization","volume":"38","author":"Schirrmeister","year":"2017","journal-title":"Hum. Brain Mapp."},{"key":"ref37","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1016\/j.neubiorev.2014.01.009","article-title":"Fractionating theory of mind: a meta-analysis of functional brain imaging studies","volume":"42","author":"Schurz","year":"2014","journal-title":"Neurosci. Biobehav. Rev."},{"key":"ref41","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3530811","article-title":"Efficient Transformers: a survey","volume":"55","author":"Tay","year":"2022","journal-title":"ACM Comput. Surv."},{"key":"ref9011","doi-asserted-by":"publisher","first-page":"e2222","DOI":"10.7717\/peerj-cs.2222","article-title":"Natural language processing with transformers: a review","volume":"10","author":"Tucudean","year":"2024","journal-title":"PeerJ. Computer science"},{"key":"ref42","doi-asserted-by":"publisher","first-page":"68","DOI":"10.1016\/j.neuroimage.2017.06.061","article-title":"Cross-validation failure: small sample sizes lead to large error bars","volume":"180","author":"Varoquaux","year":"2018","journal-title":"NeuroImage"},{"key":"ref44","doi-asserted-by":"publisher","first-page":"19212","DOI":"10.1523\/JNEUROSCI.2649-13.2013","article-title":"Simultaneous EEG-fMRI reveals temporal evolution of coupling between supramodal cortical attention networks and the brainstem","volume":"33","author":"Walz","year":"2013","journal-title":"The Journal of neuroscience: the official journal of the Society for Neuroscience"},{"key":"ref45","doi-asserted-by":"publisher","first-page":"791","DOI":"10.1016\/j.bpsc.2019.05.011","article-title":"Speaking-induced suppression of the auditory cortex in humans and its relevance to schizophrenia","volume":"4","author":"Whitford","year":"2019","journal-title":"Biol. Psychiatry Cogn. Neurosci. Neuroimaging"},{"key":"ref46","doi-asserted-by":"crossref","DOI":"10.1093\/acprof:oso\/9780195388855.001.0001","volume-title":"Brain\u2013computer interfaces: principles and practice","author":"Wolpaw","year":"2012"},{"key":"ref9012","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1109\/TCDS.2020.3007453","article-title":"Transfer Learning for EEG-Based Brain\u2013Computer Interfaces: A Review of Progress Made Since 2016","volume":"14","author":"Wu","year":"2016","journal-title":"IEEE Transactions on Cognitive and Developmental Systems"},{"key":"ref9013","doi-asserted-by":"publisher","first-page":"bbad467","DOI":"10.1093\/bib\/bbad467","article-title":"Attention is all you need: utilizing attention in AI-enabled drug discovery","volume":"25","author":"Zhang","year":"2023","journal-title":"Briefings in bioinformatics"},{"key":"ref9014","doi-asserted-by":"publisher","first-page":"3464","DOI":"10.3390\/s24113464","article-title":"EEG Emotion Recognition Network Based on Attention and Spatiotemporal Convolution","volume":"24","author":"Zhu","year":"2024","journal-title":"Sensors"}],"container-title":["Frontiers in Human Neuroscience"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnhum.2025.1668935\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,23]],"date-time":"2025-10-23T06:19:24Z","timestamp":1761200364000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnhum.2025.1668935\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,21]]},"references-count":49,"alternative-id":["10.3389\/fnhum.2025.1668935"],"URL":"https:\/\/doi.org\/10.3389\/fnhum.2025.1668935","relation":{},"ISSN":["1662-5161"],"issn-type":[{"value":"1662-5161","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,21]]},"article-number":"1668935"}}