{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,21]],"date-time":"2026-04-21T15:32:48Z","timestamp":1776785568082,"version":"3.51.2"},"reference-count":35,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,12,10]],"date-time":"2024-12-10T00:00:00Z","timestamp":1733788800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurorobot."],"abstract":"<jats:p>Non-invasive brain-computer interfaces (BCI) hold great promise in the field of neurorehabilitation. They are easy to use and do not require surgery, particularly in the area of motor imagery electroencephalography (EEG). However, motor imagery EEG signals often have a low signal-to-noise ratio and limited spatial and temporal resolution. Traditional deep neural networks typically only focus on the spatial and temporal features of EEG, resulting in relatively low decoding and accuracy rates for motor imagery tasks. To address these challenges, this paper proposes a 3D Convolutional Neural Network (P-3DCNN) decoding method that jointly learns spatial-frequency feature maps from the frequency and spatial domains of the EEG signals. First, the Welch method is used to calculate the frequency band power spectrum of the EEG, and a 2D matrix representing the spatial topology distribution of the electrodes is constructed. These spatial-frequency representations are then generated through cubic interpolation of the temporal EEG data. Next, the paper designs a 3DCNN network with 1D and 2D convolutional layers in series to optimize the convolutional kernel parameters and effectively learn the spatial-frequency features of the EEG. Batch normalization and dropout are also applied to improve the training speed and classification performance of the network. Finally, through experiments, the proposed method is compared to various classic machine learning and deep learning techniques. The results show an average decoding accuracy rate of 86.69%, surpassing other advanced networks. This demonstrates the effectiveness of our approach in decoding motor imagery EEG and offers valuable insights for the development of BCI.<\/jats:p>","DOI":"10.3389\/fnbot.2024.1485640","type":"journal-article","created":{"date-parts":[[2024,12,10]],"date-time":"2024-12-10T10:37:48Z","timestamp":1733827068000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["3D convolutional neural network based on spatial-spectral feature pictures learning for decoding motor imagery EEG signal"],"prefix":"10.3389","volume":"18","author":[{"given":"Xiaoguang","family":"Li","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yaqi","family":"Chu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xuejian","family":"Wu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2024,12,10]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","first-page":"102172","DOI":"10.1016\/j.bspc.2020.102172","article-title":"Deep learning for motor imagery EEG-based classification: a review","volume":"63","author":"Al-Saegh","year":"2021","journal-title":"Biomed Signal Process Control"},{"key":"ref2","doi-asserted-by":"publisher","first-page":"2917","DOI":"10.1007\/s11063-021-10533-7","article-title":"Deep learning with ConvNet predicts imagery tasks through EEG","volume":"53","author":"Altan","year":"2021","journal-title":"Neural. Process. Lett."},{"key":"ref3","doi-asserted-by":"publisher","first-page":"102831","DOI":"10.1016\/j.bspc.2021.102831","article-title":"EEG-based motor imagery classification using digraph Fourier transforms and extreme learning machines","volume":"69","author":"Annaby","year":"2021","journal-title":"Biomed Signal Process Control"},{"key":"ref4","doi-asserted-by":"publisher","first-page":"7596","DOI":"10.3390\/s22197596","article-title":"Classification of EEG using adaptive SVM classifier with CSP and online recursive independent component analysis","volume":"22","author":"Antony","year":"2022","journal-title":"Sensors"},{"key":"ref5","doi-asserted-by":"publisher","first-page":"2470","DOI":"10.3390\/electronics10202470","article-title":"CNN variants for computer vision: history, architecture, application, challenges and future scope","volume":"10","author":"Bhatt","year":"2021","journal-title":"Electronics"},{"key":"ref6","doi-asserted-by":"publisher","first-page":"4494","DOI":"10.1109\/JSEN.2019.2899645","article-title":"Convolutional neural network based approach towards motor imagery tasks EEG signals classification","volume":"19","author":"Chaudhary","year":"2019","journal-title":"IEEE Sensors J."},{"key":"ref7","doi-asserted-by":"publisher","first-page":"1393","DOI":"10.3390\/bioengineering10121393","article-title":"fNIRS-EEG BCIs for motor rehabilitation: a review","volume":"10","author":"Chen","year":"2023","journal-title":"Bioengineering"},{"key":"ref8","doi-asserted-by":"publisher","first-page":"026012","DOI":"10.1088\/1741-2552\/aaf594","article-title":"Combination of high-frequency SSVEP-based BCI and computer vision for controlling a robotic arm","volume":"16","author":"Chen","year":"2019","journal-title":"J. Neural. Eng."},{"key":"ref9","doi-asserted-by":"publisher","first-page":"104944","DOI":"10.1016\/j.bspc.2023.104944","article-title":"A VR-based BCI interactive system for UAV swarm control","volume":"85","author":"Deng","year":"2023","journal-title":"Biomed. Signal Process Control"},{"key":"ref10","doi-asserted-by":"publisher","first-page":"45891","DOI":"10.1007\/s11042-023-15468-w","article-title":"Convolutional neural network with support vector machine for motor imagery EEG signal classification","volume":"82","author":"Echtioui","year":"2023","journal-title":"Multimed. Tools Appl."},{"key":"ref11","doi-asserted-by":"publisher","first-page":"455","DOI":"10.1177\/15500594221148285","article-title":"Classification of BCI multiclass motor imagery task based on artificial neural network","volume":"55","author":"Echtioui","year":"2024","journal-title":"Clin. EEG Neurosci."},{"key":"ref12","doi-asserted-by":"publisher","first-page":"1403","DOI":"10.1007\/s12541-023-00835-2","article-title":"BCI-VR-based hand soft rehabilitation system with its applications in hand rehabilitation after stroke","volume":"24","author":"Gao","year":"2023","journal-title":"Int. J. Precis. Eng. Man."},{"key":"ref13","doi-asserted-by":"publisher","first-page":"e215","DOI":"10.1161\/01.cir.101.23.e215","article-title":"PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals","volume":"101","author":"Goldberger","year":"2000","journal-title":"Circulation"},{"key":"ref14","doi-asserted-by":"publisher","first-page":"056013","DOI":"10.1088\/1741-2552\/aace8c","article-title":"EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces","volume":"15","author":"Lawhern","year":"2018","journal-title":"J. Neural Eng."},{"key":"ref15","doi-asserted-by":"publisher","first-page":"103342","DOI":"10.1016\/j.bspc.2021.103342","article-title":"Motor imagery EEG classification algorithm based on CNN-LSTM feature fusion network","volume":"72","author":"Li","year":"2022","journal-title":"Biomed Signal Process Control"},{"key":"ref16","doi-asserted-by":"publisher","first-page":"1613","DOI":"10.3390\/s21051613","article-title":"The MindGomoku: an online P300 BCI game based on Bayesian deep learning","volume":"21","author":"Li","year":"2021","journal-title":"Sensors"},{"key":"ref17","doi-asserted-by":"publisher","first-page":"046029","DOI":"10.1088\/1741-2552\/abf68b","article-title":"A novel decoding method for motor imagery tasks with 4D data representation and 3D convolutional neural networks","volume":"18","author":"Li","year":"2021","journal-title":"J. Neural Eng."},{"key":"ref18","doi-asserted-by":"publisher","first-page":"198","DOI":"10.1016\/j.irbm.2021.01.002","article-title":"Motor imagery EEG spectral-spatial feature optimization using dual-tree complex wavelet and neighbourhood component analysis","volume":"43","author":"Malan","year":"2022","journal-title":"IRBM"},{"key":"ref19","doi-asserted-by":"publisher","first-page":"107013","DOI":"10.1016\/j.engappai.2023.107013","article-title":"CTRAN: CNN-transformer-based network for natural language understanding","volume":"126","author":"Mehrdad","year":"2023","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref20","doi-asserted-by":"publisher","first-page":"98275","DOI":"10.1109\/ACCESS.2021.3091399","article-title":"Shallow convolutional network excel for classifying motor imagery EEG in BCI applications","volume":"9","author":"Milanes Hermosilla","year":"2021","journal-title":"IEEE Access"},{"key":"ref21","volume-title":"Machine learning: A probabilistic perspective","author":"Murphy","year":"2012"},{"key":"ref22","doi-asserted-by":"publisher","first-page":"465","DOI":"10.1109\/TNSRE.2021.3125386","article-title":"A tensor-based frequency features combination method for brain-computer interfaces","volume":"30","author":"Pei","year":"2021","journal-title":"IEEE T. Neur. Sys. Reh."},{"key":"ref23","doi-asserted-by":"publisher","first-page":"103496","DOI":"10.1016\/j.bspc.2022.103496","article-title":"An efficient multi-scale CNN model with intrinsic feature integration for motor imagery EEG subject classification in brain-machine interfaces","volume":"74","author":"Roy","year":"2022","journal-title":"Biomed Signal Process Control"},{"key":"ref24","doi-asserted-by":"publisher","first-page":"1962","DOI":"10.1111\/ejn.14629","article-title":"Dynamics of movement-related cortical potentials and sensorimotor oscillations during palmar grasp movements","volume":"51","author":"Savi\u0107","year":"2020","journal-title":"Eur. J. Neurosci."},{"key":"ref25","doi-asserted-by":"publisher","first-page":"109425","DOI":"10.1016\/j.jneumeth.2021.109425","article-title":"Feature selection using extreme gradient boosting Bayesian optimization to upgrade the classification performance of motor imagery signals for BCI","volume":"366","author":"Thenmozhi","year":"2022","journal-title":"J. Neurosci. Methods"},{"key":"ref26","doi-asserted-by":"publisher","first-page":"15632","DOI":"10.1109\/TPAMI.2023.3299568","article-title":"Sparse Bayesian learning for end-to-end EEG decoding","volume":"45","author":"Wang","year":"2023","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref27","doi-asserted-by":"publisher","first-page":"6281","DOI":"10.48550\/arXiv.2006.04735","article-title":"Minibatch vs local sgd for heterogeneous distributed learning","volume":"33","author":"Woodworth","year":"2020","journal-title":"Adv. Neural. Inf. Process Syst."},{"key":"ref28","doi-asserted-by":"publisher","first-page":"2767","DOI":"10.3390\/electronics13142767","article-title":"An analysis of traditional methods and deep learning methods in SSVEP-based BCI: a survey","volume":"13","author":"Wu","year":"2024","journal-title":"Electronics"},{"key":"ref29","doi-asserted-by":"publisher","first-page":"015010","DOI":"10.1088\/2057-1976\/acaca2","article-title":"A novel ANN adaptive Riemannian-based kernel classification for motor imagery","volume":"9","author":"Yacine","year":"2022","journal-title":"Biomed Phys. Eng. Express"},{"key":"ref30","doi-asserted-by":"publisher","first-page":"026012","DOI":"10.1088\/1741-2560\/10\/2\/026012","article-title":"A novel hybrid BCI speller based on the incorporation of SSVEP into the P300 paradigm","volume":"10","author":"Yin","year":"","journal-title":"J. Neural Eng."},{"key":"ref31","doi-asserted-by":"publisher","first-page":"473","DOI":"10.1109\/TBME.2013.2281976","article-title":"A speedy hybrid BCI spelling approach combining P300 and SSVEP","volume":"61","author":"Yin","year":"","journal-title":"IEEE T. Biomed. Eng."},{"key":"ref32","doi-asserted-by":"publisher","first-page":"1447","DOI":"10.1109\/TBME.2014.2320948","article-title":"A dynamically optimized SSVEP brain\u2013computer interface (BCI) speller","volume":"62","author":"Yin","year":"2014","journal-title":"IEEE T. Biomed. Eng."},{"key":"ref33","doi-asserted-by":"publisher","first-page":"2570","DOI":"10.1109\/JBHI.2020.2967128","article-title":"Motor imagery classification via temporal attention cues of graph embedded EEG signals","volume":"24","author":"Zhang","year":"2020","journal-title":"IEEE J. Biomed Health"},{"key":"ref34","doi-asserted-by":"publisher","first-page":"1129049","DOI":"10.3389\/fnins.2023.1129049","article-title":"Recognition of single upper limb motor imagery tasks from EEG using multi-branch fusion convolutional neural network","volume":"17","author":"Zhang","year":"2023","journal-title":"Front. Neurosci."},{"key":"ref35","doi-asserted-by":"publisher","first-page":"103825","DOI":"10.1016\/j.bspc.2022.103825","article-title":"Overall optimization of CSP based on ensemble learning for motor imagery EEG decoding","volume":"77","author":"Zhang","year":"2022","journal-title":"Biomed Signal Process Control"}],"container-title":["Frontiers in Neurorobotics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2024.1485640\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,10]],"date-time":"2024-12-10T10:37:59Z","timestamp":1733827079000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2024.1485640\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,10]]},"references-count":35,"alternative-id":["10.3389\/fnbot.2024.1485640"],"URL":"https:\/\/doi.org\/10.3389\/fnbot.2024.1485640","relation":{},"ISSN":["1662-5218"],"issn-type":[{"value":"1662-5218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,10]]},"article-number":"1485640"}}