{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,7]],"date-time":"2026-11-07T05:40:37Z","timestamp":1794030037264,"version":"4.8.0"},"reference-count":55,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,12,15]],"date-time":"2023-12-15T00:00:00Z","timestamp":1702598400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurosci."],"abstract":"<jats:p>The classification of electroencephalogram (EEG) motor imagery signals has emerged as a prominent research focus within the realm of brain-computer interfaces. Nevertheless, the conventional, limited categories (typically just two or four) offered by brain-computer interfaces fail to provide an extensive array of control modes. To address this challenge, we propose the Time-Spatial Parallel Network (TSPNet) for recognizing six distinct categories of upper limb motor imagery. Within TSPNet, temporal and spatial features are extracted separately, with the time dimension feature extractor and spatial dimension feature extractor performing their respective functions. Following this, the Time-Spatial Parallel Feature Extractor is employed to decouple the connection between temporal and spatial features, thus diminishing feature redundancy. The Time-Spatial Parallel Feature Extractor deploys a gating mechanism to optimize weight distribution and parallelize time-spatial features. Additionally, we introduce a feature visualization algorithm based on signal occlusion frequency to facilitate a qualitative analysis of TSPNet. In a six-category scenario, TSPNet achieved an accuracy of 49.1% \u00b1 0.043 on our dataset and 49.7% \u00b1 0.029 on a public dataset. Experimental results conclusively establish that TSPNet outperforms other deep learning methods in classifying data from these two datasets. Moreover, visualization results vividly illustrate that our proposed framework can generate distinctive classifier patterns for multiple categories of upper limb motor imagery, discerned through signals of varying frequencies. These findings underscore that, in comparison to other deep learning methods, TSPNet excels in intention recognition, which bears immense significance for non-invasive brain-computer interfaces.<\/jats:p>","DOI":"10.3389\/fnins.2023.1303242","type":"journal-article","created":{"date-parts":[[2023,12,15]],"date-time":"2023-12-15T10:53:15Z","timestamp":1702637595000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":12,"title":["TSPNet: a time-spatial parallel network for classification of EEG-based multiclass upper limb motor imagery BCI"],"prefix":"10.3389","volume":"17","author":[{"given":"Jingfeng","family":"Bi","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ming","family":"Chu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gang","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaoshan","family":"Gao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2023,12,15]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1109\/TBME.2015.2487738","article-title":"Separable common spatio-spectral patterns for motor imagery BCI systems","volume":"63","author":"Aghaei","year":"2016","journal-title":"IEEE Trans. Biomed. Eng"},{"key":"B2","doi-asserted-by":"publisher","first-page":"944","DOI":"10.1109\/JPROC.2015.2415800","article-title":"Brain-computer interface for neurorehabilitation of upper limb after stroke","volume":"103","author":"Ang","year":"2015","journal-title":"Proc. IEEE"},{"key":"B3","doi-asserted-by":"publisher","first-page":"392","DOI":"10.1109\/TNSRE.2016.2646763","article-title":"EEG-based strategies to detect motor imagery for control and rehabilitation","volume":"25","author":"Ang","year":"2017","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng"},{"key":"B4","doi-asserted-by":"publisher","first-page":"3366","DOI":"10.1109\/JSEN.2019.2958210","article-title":"A single-channel consumer-grade EEG device for brain-computer interface: enhancing detection of SSVEP and its amplitude modulation","volume":"20","author":"Autthasan","year":"2020","journal-title":"IEEE Sens. J"},{"key":"B5","doi-asserted-by":"publisher","first-page":"920","DOI":"10.1109\/TBME.2011.2172210","article-title":"Multiclass brain-computer interface classification by Riemannian geometry","volume":"59","author":"Barachant","year":"2012","journal-title":"IEEE Trans. Biomed. Eng"},{"key":"B6","doi-asserted-by":"publisher","first-page":"433","DOI":"10.1109\/TPAMI.2010.125","article-title":"Convolutional neural networks for P300 detection with application to brain-computer interfaces","volume":"33","author":"Cecotti","year":"2011","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B7","doi-asserted-by":"publisher","first-page":"1142892","DOI":"10.3389\/fnins.2023.1142892","article-title":"SSVEP detection assessment by combining visual stimuli paradigms and no-training detection methods","volume":"17","author":"Chailloux Peguero","year":"2023","journal-title":"Front. Neurosci"},{"key":"B8","doi-asserted-by":"publisher","first-page":"13776","DOI":"10.1109\/JSEN.2020.3005968","article-title":"Decoding EEG rhythms during action observation, motor imagery, and execution for standing and sitting","volume":"20","author":"Chaisaen","year":"2020","journal-title":"IEEE Sens. J"},{"key":"B9","doi-asserted-by":"publisher","first-page":"513","DOI":"10.1038\/nrneurol.2016.113","article-title":"Brain-computer interfaces for communication and rehabilitation","volume":"12","author":"Chaudhary","year":"2016","journal-title":"Nat. Rev. Neurol"},{"key":"B10","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TIM.2022.3168927","article-title":"Fusing frequency-domain features and brain connectivity features for cross-subject emotion recognition","volume":"71","author":"Chen","year":"2022","journal-title":"IEEE Trans. Instrum. Meas"},{"key":"B11","doi-asserted-by":"publisher","first-page":"1389","DOI":"10.1007\/s12559-021-09941-7","article-title":"Decoding premovement patterns with task-related component analysis","volume":"13","author":"Duan","year":"2021","journal-title":"Cogn. Comput"},{"key":"B12","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1109\/TBME.2015.2467312","article-title":"EEG source imaging enhances the decoding of complex right-hand motor imagery tasks","volume":"63","author":"Edelman","year":"2016","journal-title":"IEEE Trans. Biomed. Eng"},{"key":"B13","doi-asserted-by":"publisher","first-page":"1250991","DOI":"10.3389\/fnins.2023.1250991","article-title":"Temporal-frequency-phase feature classification using 3D-convolutional neural networks for motor imagery and movement","volume":"17","author":"Fan","year":"2023","journal-title":"Front. Neurosci"},{"key":"B14","doi-asserted-by":"publisher","first-page":"786","DOI":"10.1109\/TBME.2019.2921198","article-title":"Assessment of the efficacy of EEG-based MI-BCI with visual feedback and EEG correlates of mental fatigue for upper-limb stroke rehabilitation","volume":"67","author":"Foong","year":"2020","journal-title":"IEEE Trans. Biomed. Eng"},{"key":"B15","doi-asserted-by":"publisher","first-page":"801","DOI":"10.1109\/TAFFC.2018.2890636","article-title":"A review on nonlinear methods using electroencephalographic recordings for emotion recognition","volume":"12","author":"Garcia-Martinez","year":"2021","journal-title":"IEEE Trans. Affect. Comput"},{"key":"B16","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TIM.2021.3051996","article-title":"A sliding window common spatial pattern for enhancing motor imagery classification in EEG-BCI","volume":"70","author":"Gaur","year":"2021","journal-title":"IEEE Trans. Instrum. Meas"},{"key":"B17","doi-asserted-by":"publisher","first-page":"549","DOI":"10.1016\/j.image.2016.05.018","article-title":"EEG-based prediction of driver's cognitive performance by deep convolutional neural network","volume":"47","author":"Hajinoroozi","year":"2016","journal-title":"Signal Process. Image Commun"},{"key":"B18","doi-asserted-by":"publisher","first-page":"843","DOI":"10.1109\/TSMC.2015.2450680","article-title":"Common Bayesian network for classification of EEG-based multiclass motor imagery BCI","volume":"46","author":"He","year":"2016","journal-title":"IEEE Trans. Syst. Man Cybern. Syst"},{"key":"B19","doi-asserted-by":"publisher","first-page":"1100","DOI":"10.1109\/TBME.2012.2215960","article-title":"Simultaneous design of FIR filter banks and spatial patterns for EEG signal classification","volume":"60","author":"Higashi","year":"2013","journal-title":"IEEE Trans. Biomed. Eng"},{"key":"B20","doi-asserted-by":"publisher","first-page":"1173778","DOI":"10.3389\/fnins.2023.1173778","article-title":"MSATNet: multi-scale adaptive transformer network for motor imagery classification","volume":"17","author":"Hu","year":"2023","journal-title":"Front. Neurosci"},{"key":"B21","doi-asserted-by":"publisher","first-page":"3171","DOI":"10.1109\/TCYB.2015.2498974","article-title":"A classification algorithm for electroencephalography signals by self-induced emotional stimuli","volume":"46","author":"Iacoviello","year":"2016","journal-title":"IEEE Trans. Cybern"},{"key":"B22","doi-asserted-by":"publisher","first-page":"687","DOI":"10.1109\/TNSRE.2020.2966826","article-title":"Decoding movement-related cortical potentials based on subject-dependent and section-wise spectral filtering","volume":"28","author":"Jeong","year":"2020","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng"},{"key":"B23","doi-asserted-by":"publisher","first-page":"3867","DOI":"10.1109\/JBHI.2023.3278747","article-title":"Multi-class classification of upper limb movements with filter bank task-related component analysis","volume":"27","author":"Jia","year":"2023","journal-title":"IEEE J. Biomed. Health Inform"},{"key":"B24","doi-asserted-by":"publisher","first-page":"066012","DOI":"10.1088\/1741-2552\/ac9e75","article-title":"Improving pre-movement pattern detection with filter bank selection","volume":"19","author":"Jia","year":"2022","journal-title":"J. Neural Eng"},{"key":"B25","doi-asserted-by":"publisher","first-page":"631","DOI":"10.1109\/JBHI.2018.2832538","article-title":"Sparse group representation model for motor imagery EEG classification","volume":"23","author":"Jiao","year":"2019","journal-title":"IEEE J. Biomed. Health Inform"},{"key":"B26","doi-asserted-by":"publisher","first-page":"262","DOI":"10.1016\/j.neunet.2019.07.008","article-title":"Correlation-based channel selection and regularized feature optimization for MI-based BCI","volume":"118","author":"Jin","year":"2019","journal-title":"Neural Netw"},{"key":"B27","doi-asserted-by":"publisher","first-page":"6601","DOI":"10.1007\/s00521-018-3735-3","article-title":"EEG classification using sparse Bayesian extreme learning machine for brain-computer interface","volume":"32","author":"Jin","year":"2020","journal-title":"Neural Comput. Appl"},{"key":"B28","first-page":"1","article-title":"\u201cAdam: a method for stochastic optimization,\u201d","volume-title":"Proc. 3rd Int. Conf. Learn. Represent","author":"Kingma","year":"2015"},{"key":"B29","doi-asserted-by":"publisher","first-page":"3654","DOI":"10.1109\/TCYB.2019.2924237","article-title":"Error correction regression framework for enhancing the decoding accuracies of Ear-EEG brain-computer interfaces","volume":"50","author":"Kwak","year":"2020","journal-title":"IEEE Trans. Cybern"},{"key":"B30","doi-asserted-by":"publisher","first-page":"056013","DOI":"10.1088\/1741-2552\/aace8c","article-title":"EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces","volume":"15","author":"Lawhern","year":"2018","journal-title":"J. Neural Eng"},{"key":"B31","doi-asserted-by":"publisher","first-page":"969","DOI":"10.1109\/JPROC.2015.2419736","article-title":"Towards independence: a BCI telepresence robot for people with severemotor disabilities","volume":"103","author":"Leeb","year":"2015","journal-title":"Proc. IEEE"},{"key":"B32","doi-asserted-by":"publisher","first-page":"54","DOI":"10.3389\/fnins.2020.00054","article-title":"Comparison of the ERP-based BCI performance among chromatic (RGB) semitransparent face patterns","volume":"14","author":"Li","year":"2020","journal-title":"Front. Neurosci"},{"key":"B33","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TIM.2022.3193407","article-title":"Iterative outlier removal clustering based time-frequency-spatial feature selection for binary EEG motor imagery decoding","volume":"71","author":"Ma","year":"2022","journal-title":"IEEE Trans. Instrum. Meas"},{"key":"B34","doi-asserted-by":"publisher","first-page":"e0182578","DOI":"10.1371\/journal.pone.0182578","article-title":"Upper limb movements can be decoded from the time-domain of low-frequency EEG","volume":"12","author":"Ofner","year":"2017","journal-title":"PLoS ONE"},{"key":"B35","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1016\/0167-8760(84)90014-X","article-title":"Low resolution electromagnetic tomography: a new method for localizing electrical activity in the brain","volume":"18","author":"Pascual-Marqui","year":"1994","journal-title":"Int. J. Psychophysiol"},{"key":"B36","doi-asserted-by":"publisher","first-page":"1766","DOI":"10.1109\/TNSRE.2022.3186442","article-title":"EEGSym: overcoming inter-subject variability in motor imagery based BCIs with deep learning","volume":"30","author":"Perez-Velasco","year":"2022","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng"},{"key":"B37","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TIM.2022.3218102","article-title":"CCA-based compressive sensing for SSVEP-based brain-computer interfaces to command a robotic wheelchair","volume":"71","author":"Rivera-Flor","year":"2022","journal-title":"IEEE Trans. Instrum. Meas"},{"key":"B38","doi-asserted-by":"publisher","first-page":"5619","DOI":"10.1109\/TNNLS.2018.2789927","article-title":"Learning temporal information for brain-computer interface using convolutional neural networks","volume":"29","author":"Sakhavi","year":"2018","journal-title":"IEEE Trans. Neural Netw. Learn. Syst"},{"key":"B39","doi-asserted-by":"publisher","first-page":"5391","DOI":"10.1002\/hbm.23730","article-title":"Deep learning with convolutional neural networks for EEG decoding and visualization","volume":"38","author":"Schirrmeister","year":"2017","journal-title":"Hum. Brain Mapp"},{"key":"B40","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TIM.2023.3277985","article-title":"Multisource associate domain adaptation for cross-subject and cross-session EEG emotion recognition","volume":"72","author":"She","year":"2023","journal-title":"IEEE Trans. Instrum. Meas"},{"key":"B41","doi-asserted-by":"publisher","first-page":"526","DOI":"10.1109\/TNSRE.2012.2184838","article-title":"Improving the separability of motor imagery EEG signals using a cross correlation-based least square support vector machine for brain-computer interface","volume":"20","author":"Siuly","year":"2012","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng"},{"key":"B42","doi-asserted-by":"publisher","first-page":"286","DOI":"10.1109\/TPAMI.2012.69","article-title":"A novel Bayesian framework for discriminative feature extraction in brain-computer interfaces","volume":"35","author":"Suk","year":"2013","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B43","doi-asserted-by":"publisher","first-page":"016003","DOI":"10.1088\/1741-2560\/14\/1\/016003","article-title":"A novel deep learning approach for classification of EEG motor imagery signals","volume":"14","author":"Tabar","year":"2017","journal-title":"J. Neural. Eng"},{"key":"B44","doi-asserted-by":"publisher","first-page":"879716","DOI":"10.1155\/2011\/879716","article-title":"Brainstorm: a user-friendly application for MEG\/EEG analysis","volume":"2011","author":"Tadel","year":"2011","journal-title":"Comput. Intell. Neurosci"},{"key":"B45","doi-asserted-by":"publisher","first-page":"964","DOI":"10.1016\/j.medengphy.2011.11.001","article-title":"A two-stage four-class BCI based on imaginary movements of the left and the right wrist","volume":"34","author":"Vuckovic","year":"2012","journal-title":"Med. Eng. Phys"},{"key":"B46","doi-asserted-by":"publisher","first-page":"5008","DOI":"10.1109\/TCYB.2020.2982901","article-title":"Common spatial pattern reformulated for regularizations in brain-computer interfaces","volume":"51","author":"Wang","year":"2021","journal-title":"IEEE Trans. Cybern"},{"key":"B47","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TIM.2022.3224991","article-title":"Decoding the continuous motion imagery trajectories of upper limb skeleton points for EEG-based brain-computer interface","volume":"72","author":"Wang","year":"2023","journal-title":"IEEE Trans. Instrum. Meas"},{"key":"B48","doi-asserted-by":"publisher","first-page":"066010","DOI":"10.1088\/1741-2552\/abc1b6","article-title":"An investigation of in-ear sensing for motor task classification","volume":"17","author":"Wu","year":"2020","journal-title":"J. Neural Eng"},{"key":"B49","doi-asserted-by":"publisher","first-page":"288","DOI":"10.1109\/TBME.2013.2294203","article-title":"Enhanced low-latency detection of motor intention from EEG for closed-loop Brain-computer interface applications","volume":"61","author":"Xu","year":"2014","journal-title":"IEEE Trans. Biomed. Eng"},{"key":"B50","doi-asserted-by":"publisher","first-page":"1161511","DOI":"10.3389\/fnins.2023.1161511","article-title":"SSVEP unsupervised adaptive feature recognition method based on self-similarity of same-frequency signals","volume":"17","author":"Yan","year":"2023","journal-title":"Front. Neurosci"},{"key":"B51","doi-asserted-by":"publisher","first-page":"3033","DOI":"10.1109\/TCYB.2019.2905157","article-title":"Making sense of spatio-temporal preserving representations for EEG-based human intention recognition","volume":"50","author":"Zhang","year":"2020","journal-title":"IEEE Trans. Cybern"},{"key":"B52","doi-asserted-by":"publisher","first-page":"1129049","DOI":"10.3389\/fnins.2023.1129049","article-title":"Recognition of single upper limb motor imagery tasks from EEG using multi-branch fusion convolutional neural network","volume":"17","author":"Zhang","year":"2023","journal-title":"Front. Neurosci"},{"key":"B53","doi-asserted-by":"publisher","first-page":"4000509","DOI":"10.1109\/TIM.2022.3146950","article-title":"Multi-objective optimization-based high-pass spatial filtering for SSVEP-based Brain-computer interfaces","volume":"71","author":"Zhang","year":"2022","journal-title":"IEEE Trans. Instrum. Meas"},{"key":"B54","doi-asserted-by":"publisher","first-page":"3322","DOI":"10.1109\/TCYB.2018.2841847","article-title":"Temporally constrained sparse group spatial patterns for motor imagery BCI","volume":"49","author":"Zhang","year":"2019","journal-title":"IEEE Trans Cybern"},{"key":"B55","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1109\/JBHI.2014.2370646","article-title":"Automatic identification of artifact-related independent components for artifact removal in EEG recordings","volume":"20","author":"Zou","year":"2016","journal-title":"IEEE J. Biomed. Health Inform"}],"container-title":["Frontiers in Neuroscience"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnins.2023.1303242\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,15]],"date-time":"2023-12-15T10:53:17Z","timestamp":1702637597000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnins.2023.1303242\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,15]]},"references-count":55,"alternative-id":["10.3389\/fnins.2023.1303242"],"URL":"https:\/\/doi.org\/10.3389\/fnins.2023.1303242","relation":{},"ISSN":["1662-453X"],"issn-type":[{"value":"1662-453X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,15]]},"article-number":"1303242"}}