{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T12:11:50Z","timestamp":1781611910782,"version":"3.54.5"},"reference-count":50,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,2,22]],"date-time":"2023-02-22T00:00:00Z","timestamp":1677024000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurosci."],"abstract":"<jats:p>Motor imagery-based brain-computer interfaces (MI-BCI) have important application values in the field of neurorehabilitation and robot control. At present, MI-BCI mostly use bilateral upper limb motor tasks, but there are relatively few studies on single upper limb MI tasks. In this work, we conducted studies on the recognition of motor imagery EEG signals of the right upper limb and proposed a multi-branch fusion convolutional neural network (MF-CNN) for learning the features of the raw EEG signals as well as the two-dimensional time-frequency maps at the same time. The dataset used in this study contained three types of motor imagery tasks: extending the arm, rotating the wrist, and grasping the object, 25 subjects were included. In the binary classification experiment between the grasping object and the arm-extending tasks, MF-CNN achieved an average classification accuracy of 78.52% and kappa value of 0.57. When all three tasks were used for classification, the accuracy and kappa value were 57.06% and 0.36, respectively. The comparison results showed that the classification performance of MF-CNN is higher than that of single CNN branch algorithms in both binary-class and three-class classification. In conclusion, MF-CNN makes full use of the time-domain and frequency-domain features of EEG, can improve the decoding accuracy of single limb motor imagery tasks, and it contributes to the application of MI-BCI in motor function rehabilitation training after stroke.<\/jats:p>","DOI":"10.3389\/fnins.2023.1129049","type":"journal-article","created":{"date-parts":[[2023,2,22]],"date-time":"2023-02-22T09:33:10Z","timestamp":1677058390000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":24,"title":["Recognition of single upper limb motor imagery tasks from EEG using multi-branch fusion convolutional neural network"],"prefix":"10.3389","volume":"17","author":[{"given":"Rui","family":"Zhang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yadi","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zongxin","family":"Xu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lipeng","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuxia","family":"Hu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mingming","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2023,2,22]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-018-04673-z","article-title":"Brain-actuated functional electrical stimulation elicits lasting arm motor recovery after stroke.","volume":"9","author":"Biasiucci","year":"2018","journal-title":"Nat. Commun."},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.3389\/fninf.2015.00016","article-title":"The PREP pipeline: Standardized preprocessing for large-scale EEG analysis.","volume":"9","author":"Bigdely-Shamlo","year":"2015","journal-title":"Front. Neuroinform."},{"key":"B3","doi-asserted-by":"crossref","first-page":"651","DOI":"10.1002\/acn3.544","article-title":"Brain-computer interfaces for post-stroke motor rehabilitation: A meta-analysis.","volume":"5","author":"Cervera","year":"2018","journal-title":"Ann. Clin. Transl. Neurol."},{"key":"B4","doi-asserted-by":"publisher","first-page":"13279","DOI":"10.1109\/tcyb.2021.3122969","article-title":"NeuroGrasp: Real-Time EEG classification of high-level motor imagery tasks using a dual-stage deep learning framework.","volume":"52","author":"Cho","year":"2021","journal-title":"IEEE Trans. Cybern."},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1511.07289","article-title":"Fast and accurate deep network learning by exponential linear units (ELUs).","author":"Clevert","year":"2015","journal-title":"arXiv"},{"key":"B6","doi-asserted-by":"crossref","DOI":"10.3390\/s19030551","article-title":"EEG classification of motor imagery using a novel deep learning framework.","volume":"19","author":"Dai","year":"2019","journal-title":"Sensors"},{"key":"B7","doi-asserted-by":"publisher","first-page":"993","DOI":"10.1109\/tbme.2004.827088","article-title":"Boosting bit rates in noninvasive EEG single-trial classifications by feature combination and multiclass paradigms.","volume":"51","author":"Dornhege","year":"2004","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"B8","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1109\/TBME.2015.2467312","article-title":"EEG source imaging enhances the decoding of complex right-hand motor imagery tasks.","volume":"63","author":"Edelman","year":"2016","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"B9","doi-asserted-by":"publisher","first-page":"353","DOI":"10.1515\/bmt-2020-0229","article-title":"A novel signal to image transformation and feature level fusion for multimodal emotion recognition.","volume":"66","author":"Hatipoglu Yilmaz","year":"2021","journal-title":"Biomed. Tech."},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.1088\/1741-2552\/aaa8c0","article-title":"Brain-machine interfaces for controlling lower-limb powered robotic systems.","volume":"15","author":"He","year":"2018","journal-title":"J. Neural Eng."},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1602.07360","article-title":"SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size.","author":"Iandola","year":"2016","journal-title":"arXiv"},{"key":"B12","doi-asserted-by":"publisher","first-page":"417","DOI":"10.1016\/j.neuroimage.2017.06.030","article-title":"Autoreject: Automated artifact rejection for MEG and EEG data.","volume":"159","author":"Jas","year":"2016","journal-title":"Neuroimage"},{"key":"B13","doi-asserted-by":"crossref","first-page":"66941","DOI":"10.1109\/ACCESS.2020.2983182","article-title":"EEG classification of forearm movement imagery using a hierarchical flow convolutional neural network.","volume":"8","author":"Jeong","year":"","journal-title":"IEEE Access."},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1093\/gigascience\/giaa098","article-title":"Multimodal signal dataset for 11 intuitive movement tasks from single upper extremity during multiple recording sessions.","volume":"9","author":"Jeong","year":"","journal-title":"Gigascience"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.1109\/EMBC.2019.8856312","article-title":"Trajectory decoding of arm reaching movement imageries for brain-controlled robot arm system","author":"Jeong","year":"2019","journal-title":"Paper Presented at the 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.1088\/1741-2560\/11\/5\/056004","article-title":"Beyond maximum speed-a novel two-stimulus paradigm for brain-computer interfaces based on event-related potentials (P300-BCI).","volume":"11","author":"Kaufmann","year":"2014","journal-title":"J. Neural Eng."},{"key":"B17","doi-asserted-by":"crossref","first-page":"5751","DOI":"10.1016\/j.eswa.2010.02.033","article-title":"A comparative study of DWT, CWT and DCT transformations in ECG arrhythmias classification.","volume":"37","author":"Khorrami","year":"2010","journal-title":"Expert Syst. Appl."},{"key":"B18","doi-asserted-by":"publisher","first-page":"654","DOI":"10.1109\/TNSRE.2016.2597854","article-title":"Commanding a brain-controlled wheelchair using steady-state somatosensory evoked potentials.","volume":"26","author":"Kim","year":"2018","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.1088\/1741-2560\/12\/5\/056009","article-title":"A lower limb exoskeleton control system based on steady state visual evoked potentials.","volume":"12","author":"Kwak","year":"2015","journal-title":"J. Neural Eng."},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.1088\/1741-2552\/aace8c","article-title":"EEGNet: A compact convolutional neural network for EEG-based brain\u2013computer interfaces.","volume":"15","author":"Lawhern","year":"2018","journal-title":"J. Neural Eng."},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.3390\/e21121199","article-title":"Application of continuous wavelet transform and convolutional neural network in decoding motor imagery brain-computer interface.","volume":"21","author":"Lee","year":"2019","journal-title":"Entropy"},{"key":"B22","doi-asserted-by":"publisher","first-page":"4843","DOI":"10.1109\/TIP.2017.2725580","article-title":"Going deeper with contextual CNN for hyperspectral image classification.","volume":"26","author":"Lee","year":"2016","journal-title":"IEEE Trans. Image Process."},{"key":"B23","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12984-016-0212-z","article-title":"A motion-classification strategy based on sEMG-EEG signal combination for upper-limb amputees.","volume":"14","author":"Li","year":"2017","journal-title":"J. Neuroeng. Rehabil."},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.1109\/EMBC.2018.8512711","article-title":"A hybrid EEG-EMG BMI improves the detection of movement intention in cortical stroke patients with complete hand paralysis","author":"Loopez-Larraz","year":"2018","journal-title":"Paper Presented at the 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society"},{"key":"B25","doi-asserted-by":"publisher","first-page":"177","DOI":"10.1023\/a:1023437823106","article-title":"Mu and beta rhythm topographies during motor imagery and actual movements.","volume":"12","author":"McFarland","year":"2000","journal-title":"Brain Topogr."},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0182578","article-title":"Upper limb movements can be decoded from the time-domain of low-frequency EEG.","volume":"12","author":"Ofner","year":"2017","journal-title":"PLoS One"},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.aat1228","article-title":"BMI control of a third arm for multitasking.","volume":"3","author":"Penaloza","year":"2018","journal-title":"Sci. Robot."},{"key":"B28","doi-asserted-by":"publisher","first-page":"1842","DOI":"10.1016\/s1388-2457(99)00141-8","article-title":"Event-related EEG\/MEG synchronization and desynchronization: Basic principles.","volume":"110","author":"Pfurtscheller","year":"1999","journal-title":"Clin. Neurophysiol."},{"key":"B29","doi-asserted-by":"publisher","first-page":"1123","DOI":"10.1109\/5.939829","article-title":"Motor imagery and direct brain-computer communication.","volume":"89","author":"Pfurtscheller","year":"2001","journal-title":"Proc. IEEE"},{"key":"B30","doi-asserted-by":"crossref","first-page":"441","DOI":"10.1109\/86.895946","article-title":"Optimal spatial filtering of single trial EEG during imagined hand movement.","volume":"8","author":"Ramoser","year":"2000","journal-title":"IEEE Trans. Rehabil. Eng."},{"key":"B31","doi-asserted-by":"publisher","first-page":"988","DOI":"10.1109\/TNSRE.2020.2974056","article-title":"A low-cost lower-limb brain-machine interface triggered by pedaling motor imagery for post-stroke patients rehabilitation.","volume":"28","author":"Romero-Laiseca","year":"2020","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"B32","doi-asserted-by":"publisher","first-page":"5391","DOI":"10.1002\/hbm.23730","article-title":"Deep learning with convolutional neural networks for EEG decoding and visualization.","volume":"38","author":"Schirrmeister","year":"2017","journal-title":"Hum. Brain Mapp."},{"key":"B33","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-72038-8_5","article-title":"Classification of motor imagery based EEG signals using sparsity approach","author":"Sreeja","year":"2017","journal-title":"Paper Presented at the 9th International Conference on Intelligent Human Computer Interaction (IHCI)"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.1088\/1741-2560\/14\/1\/016003","article-title":"A novel deep learning approach for classification of EEG motor imagery signals.","volume":"14","author":"Tabar","year":"2017","journal-title":"J. Neural Eng."},{"key":"B35","doi-asserted-by":"publisher","first-page":"992","DOI":"10.1109\/TBME.2020.3009053","article-title":"Unified expression of the quasi-static electromagnetic field: Demonstration with MEG and EEG signals.","volume":"68","author":"Taulu","year":"2021","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0174161","article-title":"Classifying three imaginary states of the same upper extremity using time-domain features.","volume":"12","author":"Tavakolan","year":"2017","journal-title":"PLoS One"},{"key":"B37","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12984-017-0219-0","article-title":"Classification of upper limb center-out reaching tasks by means of EEG-based continuous decoding techniques.","volume":"14","author":"Ubeda","year":"2017","journal-title":"J. Neuroeng. Rehabil."},{"key":"B38","doi-asserted-by":"crossref","first-page":"206","DOI":"10.1109\/T-AFFC.2011.12","article-title":"Exploring fusion methods for multimodal emotion recognition with missing data.","volume":"2","author":"Wagner","year":"2011","journal-title":"IEEE Trans. Affect. Comput."},{"key":"B39","doi-asserted-by":"crossref","first-page":"767","DOI":"10.1016\/S1388-2457(02)00057-3","article-title":"Brain-computer interfaces for communication and control.","volume":"113","author":"Wolpaw","year":"2002","journal-title":"Clin. Neurophysiol."},{"key":"B40","doi-asserted-by":"crossref","first-page":"59844","DOI":"10.1109\/ACCESS.2019.2914872","article-title":"Exploiting EEG signals and audiovisual feature fusion for video emotion recognition.","volume":"7","author":"Xing","year":"2019","journal-title":"IEEE Access."},{"key":"B41","doi-asserted-by":"publisher","DOI":"10.3390\/e24030376","article-title":"A two-branch CNN fusing temporal and frequency features for motor imagery EEG decoding.","volume":"24","author":"Yang","year":"2022","journal-title":"Entropy"},{"key":"B42","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1080\/27706710.2022.2120775","article-title":"From psychosomatic medicine, brain\u2013computer interface to brain\u2013apparatus communication.","volume":"1","author":"Yao","year":"2022","journal-title":"Brain Apparatus Commun."},{"key":"B43","doi-asserted-by":"publisher","first-page":"2367","DOI":"10.1109\/TNSRE.2018.2881215","article-title":"An asynchronous control paradigm based on sequential motor imagery and its application in wheelchair navigation.","volume":"26","author":"Yu","year":"2018","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"B44","doi-asserted-by":"publisher","DOI":"10.3390\/brainsci11010056","article-title":"BCI training effects on chronic stroke correlate with functional reorganization in motor-related regions: A concurrent EEG and fMRI study.","volume":"11","author":"Yuan","year":"2021","journal-title":"Brain Sci."},{"key":"B45","doi-asserted-by":"publisher","DOI":"10.1088\/1741-2552\/abca16","article-title":"Motor imagery recognition with automatic EEG channel selection and deep learning.","volume":"18","author":"Zhang","year":"2021","journal-title":"J. Neural Eng."},{"key":"B46","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1109\/tnsre.2018.2884641","article-title":"Learning spatial-spectral-temporal EEG features with recurrent 3D convolutional neural networks for cross-task mental workload assessment.","volume":"27","author":"Zhang","year":"2019","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"B47","doi-asserted-by":"publisher","DOI":"10.1088\/1741-2552\/ab3471","article-title":"A novel hybrid deep learning scheme for four-class motor imagery classification.","volume":"16","author":"Zhang","year":"2019","journal-title":"J. Neural Eng."},{"key":"B48","doi-asserted-by":"publisher","first-page":"2265","DOI":"10.1109\/JBHI.2019.2938247","article-title":"Multimodal depression detection: Fusion of electroencephalography and paralinguistic behaviors using a novel strategy for classifier ensemble.","volume":"23","author":"Zhang","year":"2019","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"B49","doi-asserted-by":"publisher","first-page":"679","DOI":"10.1007\/s40120-022-00333-z","article-title":"Effects of training with a brain\u2013computer interface-controlled robot on rehabilitation outcome in patients with subacute stroke: A randomized controlled trial.","volume":"11","author":"Zhao","year":"2022","journal-title":"Neurol. Ther."},{"key":"B50","first-page":"250","article-title":"Buildings segmentation of remote sensing images based on VGG16 pre-encoding.","volume":"19","author":"Zhao-Hong","year":"2019","journal-title":"Sci. Technol. Eng."}],"container-title":["Frontiers in Neuroscience"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnins.2023.1129049\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,2,22]],"date-time":"2023-02-22T09:33:19Z","timestamp":1677058399000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnins.2023.1129049\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,22]]},"references-count":50,"alternative-id":["10.3389\/fnins.2023.1129049"],"URL":"https:\/\/doi.org\/10.3389\/fnins.2023.1129049","relation":{},"ISSN":["1662-453X"],"issn-type":[{"value":"1662-453X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,22]]},"article-number":"1129049"}}