{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,11]],"date-time":"2026-09-11T16:17:12Z","timestamp":1789143432940,"version":"build-2803163510"},"reference-count":47,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2022,4,25]],"date-time":"2022-04-25T00:00:00Z","timestamp":1650844800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurosci."],"abstract":"<jats:p>A challenging task for the biological neural signal-based human-exoskeleton interface is to achieve accurate lower limb movement prediction of patients with hemiplegia in rehabilitation training scenarios. The human-exoskeleton interface based on single-modal biological signals such as electroencephalogram (EEG) is currently not mature in predicting movements, due to its unreliability. The multimodal human-exoskeleton interface is a very novel solution to this problem. This kind of interface normally combines the EEG signal with surface electromyography (sEMG) signal. However, their use for the lower limb movement prediction is still limited\u2014the connection between sEMG and EEG signals and the deep feature fusion between them are ignored. In this article, a Dense con-attention mechanism-based Multimodal Enhance Fusion Network (DMEFNet) is proposed for predicting lower limb movement of patients with hemiplegia. The DMEFNet introduces the con-attention structure to extract the common attention between sEMG and EEG signal features. To verify the effectiveness of DMEFNet, an sEMG and EEG data acquisition experiment and an incomplete asynchronous data collection paradigm are designed. The experimental results show that DMEFNet has a good movement prediction performance in both within-subject and cross-subject situations, reaching an accuracy of 82.96 and 88.44%, respectively.<\/jats:p>","DOI":"10.3389\/fnins.2022.796290","type":"journal-article","created":{"date-parts":[[2022,4,25]],"date-time":"2022-04-25T11:12:59Z","timestamp":1650885179000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":18,"title":["Multimodal Human-Exoskeleton Interface for Lower Limb Movement Prediction Through a Dense Co-Attention Symmetric Mechanism"],"prefix":"10.3389","volume":"16","author":[{"given":"Kecheng","family":"Shi","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fengjun","family":"Mu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rui","family":"Huang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ke","family":"Huang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhinan","family":"Peng","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chaobin","family":"Zou","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiao","family":"Yang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hong","family":"Cheng","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2022,4,25]]},"reference":[{"key":"B1","doi-asserted-by":"crossref","first-page":"2464","DOI":"10.1109\/IROS.2016.7759384","article-title":"\u201cA convolutional neural network for robotic arm guidance using semg based frequency-features,\u201d","volume-title":"2016 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)","author":"Allard","year":"2016"},{"key":"B2","doi-asserted-by":"crossref","first-page":"5431","DOI":"10.1109\/IROS.2018.8594110","article-title":"\u201cHuman motion classification based on multi-modal sensor data for lower limb exoskeletons,\u201d","volume-title":"2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)","author":"Beil","year":"2018"},{"key":"B3","doi-asserted-by":"publisher","first-page":"122","DOI":"10.3389\/fnins.2016.00122","article-title":"Design and optimization of an eeg-based brain machine interface (bmi) to an upper-limb exoskeleton for stroke survivors","volume":"10","author":"Bhagat","year":"2016","journal-title":"Front. Neurosci"},{"key":"B4","doi-asserted-by":"publisher","first-page":"31","DOI":"10.3389\/fnbot.2019.00031","article-title":"Svm-based classification of semg signals for upper-limb self-rehabilitation training","volume":"13","author":"Cai","year":"2019","journal-title":"Front. Neurorobot"},{"key":"B5","first-page":"59","article-title":"Robot-assisted gait training improves motor performances and modifies motor unit firing in poststroke patients","volume":"51","author":"Chisari","year":"2015","journal-title":"Eur. J. Phys. Rehabil. Med"},{"key":"B6","article-title":"Fast and accurate deep network learning by exponential linear units (elus)","author":"Clevert","year":"2015","journal-title":"arXiv preprint"},{"key":"B7","doi-asserted-by":"publisher","first-page":"760","DOI":"10.1109\/TNSRE.2019.2896269","article-title":"Deep learning for electromyographic hand gesture signal classification using transfer learning","volume":"27","author":"C\u00f4t\u00e9-Allard","year":"2019","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng"},{"key":"B8","doi-asserted-by":"publisher","first-page":"016025","DOI":"10.1088\/1741-2552\/ab405f","article-title":"Hs-cnn: a cnn with hybrid convolution scale for eeg motor imagery classification","volume":"17","author":"Dai","year":"2020","journal-title":"J. Neural Eng"},{"key":"B9","doi-asserted-by":"publisher","first-page":"3447","DOI":"10.1109\/TCSII.2020.2983389","article-title":"A deep learning method for improving the classification accuracy of ssmvep-based bci","volume":"67","author":"Gao","year":"2020","journal-title":"IEEE Trans. Circ. Syst. II"},{"key":"B10","doi-asserted-by":"publisher","first-page":"2054","DOI":"10.1109\/TNSRE.2017.2703586","article-title":"Toward multimodal human-robot interaction to enhance active participation of users in gait rehabilitation","volume":"25","author":"Gui","year":"2017","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng"},{"key":"B11","doi-asserted-by":"publisher","first-page":"98","DOI":"10.3389\/fnins.2021.704603","article-title":"Mcsnet: channel synergy-based human-exoskeleton interface with surface electromyogram","volume":"15","author":"Kecheng","year":"2021","journal-title":"Front. Neurorobot"},{"key":"B12","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12984-021-00804-8","article-title":"Robot enhanced stroke therapy optimizes rehabilitation (restore): a pilot study","volume":"18","author":"Keeling","year":"2021","journal-title":"J. Neuroeng. Rehabil"},{"key":"B13","doi-asserted-by":"publisher","first-page":"139","DOI":"10.1109\/TAI.2020.3046160","article-title":"Recursive multi-signal temporal fusions with attention mechanism improves emg feature extraction","volume":"1","author":"Khushaba","year":"2020","journal-title":"IEEE Trans. Artif. Intell"},{"key":"B14","article-title":"Adam: A method for stochastic optimization","author":"Kingma","year":"2014","journal-title":"arXiv preprint"},{"key":"B15","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1155\/2012\/901483","article-title":"Human-robot interfaces in exoskeletons for gait training after stroke: state of the art and challenges","volume":"9","author":"Lagoda","year":"2012","journal-title":"Appl. Bionics Biomech"},{"key":"B16","doi-asserted-by":"publisher","first-page":"726","DOI":"10.1109\/ICORR.2017.8009334","article-title":"The ewrist-a wearable wrist exoskeleton with semg-based force control for stroke rehabilitation","volume":"2017","author":"Lambelet","year":"2017","journal-title":"IEEE Int. Conf. Rehabil. Robot"},{"key":"B17","doi-asserted-by":"publisher","first-page":"1693","DOI":"10.1016\/S0140-6736(11)60325-5","article-title":"Stroke rehabilitation","volume":"377","author":"Langhorne","year":"2011","journal-title":"Lancet"},{"key":"B18","doi-asserted-by":"publisher","first-page":"611064","DOI":"10.3389\/fnhum.2020.611064","article-title":"Modulation of functional connectivity and low-frequency fluctuations after brain-computer interface-guided robot hand training in chronic stroke: a 6-month follow-up study","volume":"14","author":"Lau","year":"2020","journal-title":"Front. Hum. Neurosci"},{"key":"B19","doi-asserted-by":"publisher","first-page":"056013","DOI":"10.1088\/1741-2552\/aace8c","article-title":"Eegnet: a compact convolutional neural network for eeg-based brain-computer interfaces","volume":"15","author":"Lawhern","year":"2018","journal-title":"J. Neural Eng"},{"key":"B20","doi-asserted-by":"publisher","first-page":"190","DOI":"10.1016\/j.ins.2019.06.008","article-title":"Comparative analysis of features extracted from eeg spatial, spectral and temporal domains for binary and multiclass motor imagery classification","volume":"502","author":"Lee","year":"2019","journal-title":"Inf. Sci"},{"key":"B21","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41598-020-58630-2","article-title":"Comparisons between end-effector and exoskeleton rehabilitation robots regarding upper extremity function among chronic stroke patients with moderate-to-severe upper limb impairment","volume":"10","author":"Lee","year":"2020","journal-title":"Sci. Rep"},{"key":"B22","doi-asserted-by":"crossref","first-page":"458","DOI":"10.1109\/RCAR49640.2020.9303041","article-title":"\u201cA hierarchical fusion strategy based on eeg and semg for human-exoskeleton system,\u201d","volume-title":"2020 IEEE International Conference on Real-time Computing and Robotics (RCAR)","author":"Li","year":"2020"},{"key":"B23","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12984-016-0212-z","article-title":"A motion-classification strategy based on semg-eeg signal combination for upper-limb amputees","volume":"14","author":"Li","year":"2017","journal-title":"J. Neuroeng Rehabil"},{"key":"B24","doi-asserted-by":"publisher","first-page":"031005","DOI":"10.1088\/1741-2552\/aab2f2","article-title":"A review of classification algorithms for eeg-based brain-computer interfaces: a 10 year update","volume":"15","author":"Lotte","year":"2018","journal-title":"J. Neural Eng"},{"key":"B25","doi-asserted-by":"publisher","first-page":"161","DOI":"10.3389\/fnins.2010.00161","article-title":"Combining brain-computer interfaces and assistive technologies: State-of-the-art and challenges","volume":"4","author":"Mill\u00e1n","year":"2010","journal-title":"Front. Neurosci"},{"key":"B26","doi-asserted-by":"publisher","first-page":"293","DOI":"10.3233\/RNN-170799","article-title":"Clinical features of patients who might benefit more from walking robotic training","volume":"36","author":"Morone","year":"2018","journal-title":"Restor. Neurol Neurosci"},{"key":"B27","doi-asserted-by":"publisher","first-page":"2314","DOI":"10.1109\/JSYST.2020.3021485","article-title":"Restoring activities of daily living using an eeg\/eog-controlled semiautonomous and mobile whole-arm exoskeleton in chronic stroke","volume":"15","author":"Nann","year":"2020","journal-title":"IEEE Syst. J"},{"key":"B28","first-page":"6087","article-title":"\u201cImproved fusion of visual and language representations by dense symmetric co-attention for visual question answering,\u201d","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Nguyen","year":"2018"},{"key":"B29","article-title":"\u201cAutomatic differentiation in pytorch,\u201d","author":"Paszke","year":"2017","journal-title":"31st Conference on Neural Information Processing Systems (NIPS 2017)"},{"key":"B30","doi-asserted-by":"publisher","first-page":"7420","DOI":"10.1016\/j.eswa.2012.01.102","article-title":"Feature reduction and selection for emg signal classification","volume":"39","author":"Phinyomark","year":"2012","journal-title":"Expert. Syst. Appl"},{"key":"B31","doi-asserted-by":"publisher","first-page":"895","DOI":"10.1109\/ICORR.2017.8009362","article-title":"A hybrid brain-machine interface based on eeg and emg activity for the motor rehabilitation of stroke patients","volume":"2017","author":"Sarasola-Sanz","year":"2017","journal-title":"IEEE Int. Conf. Rehabil. Robot"},{"key":"B32","doi-asserted-by":"publisher","first-page":"137489","DOI":"10.1109\/ACCESS.2019.2941977","article-title":"Movements classification of multi-channel semg based on cnn and stacking ensemble learning","volume":"7","author":"Shen","year":"2019","journal-title":"IEEE Access"},{"key":"B33","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12984-018-0438-z","article-title":"On the design of eeg-based movement decoders for completely paralyzed stroke patients","volume":"15","author":"Sp\u00fcler","year":"2018","journal-title":"J. Neuroeng. Rehabil"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2020.3025777","article-title":"Eeg-based emotion recognition via channel-wise attention and self attention","author":"Tao","year":"2020","journal-title":"IEEE Trans. Affect. Comput"},{"key":"B35","doi-asserted-by":"publisher","first-page":"046011","DOI":"10.1088\/1741-2552\/ab9842","article-title":"Deep learning-based bci for gait decoding from eeg with lstm recurrent neural network","volume":"17","author":"Tortora","year":"","journal-title":"J. Neural Eng"},{"key":"B36","doi-asserted-by":"publisher","first-page":"89","DOI":"10.3389\/fnbot.2020.582728","article-title":"Hybrid human-machine interface for gait decoding through bayesian fusion of eeg and emg classifiers","volume":"14","author":"Tortora","year":"","journal-title":"Front. Neurorobot"},{"key":"B37","doi-asserted-by":"publisher","first-page":"943","DOI":"10.1109\/TNSRE.2020.2978273","article-title":"Quantitative assessment of upper-limb motor function for post-stroke rehabilitation based on motor synergy analysis and multi-modality fusion","volume":"28","author":"Wang","year":"2020","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng"},{"key":"B38","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1109\/ICInfA.2017.8078925","article-title":"\u201cA control system of lower limb exoskeleton robots based on motor imagery,\u201d","volume-title":"2017 IEEE International Conference on Information and Automation (ICIA)","author":"Wang","year":"2017"},{"key":"B39","doi-asserted-by":"publisher","first-page":"767","DOI":"10.1016\/S1388-2457(02)00057-3","article-title":"Brain-computer interfaces for communication and control","volume":"113","author":"Wolpaw","year":"2002","journal-title":"Clin. Neurophysiol"},{"key":"B40","doi-asserted-by":"publisher","first-page":"1706","DOI":"10.1109\/TBME.2015.2402283","article-title":"Enhanced motor imagery training using a hybrid bci with feedback","volume":"62","author":"Yu","year":"2015","journal-title":"IEEE Trans. Biomed. Eng"},{"key":"B41","first-page":"6281","article-title":"\u201cDeep modular co-attention networks for visual question answering,\u201d","author":"Yu","year":"2019","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B42","doi-asserted-by":"crossref","first-page":"2904","DOI":"10.1109\/ICRA.2017.7989337","article-title":"\u201cMaestro: an emg-driven assistive hhand exoskeleton for spinal cord injury patients,\u201d","volume-title":"2017 IEEE International Conference on Robotics and Automation (ICRA)","author":"Yun","year":"2017"},{"key":"B43","doi-asserted-by":"publisher","first-page":"115639","DOI":"10.1016\/j.eswa.2021.115639","article-title":"Mffnet: Multi-dimensional feature fusion network based on attention mechanism for semg analysis to detect muscle fatigue","volume":"185","author":"Zhang","year":"2021","journal-title":"Expert Syst. Appl"},{"key":"B44","doi-asserted-by":"publisher","first-page":"9967592","DOI":"10.1155\/2021\/9967592","article-title":"Expression eeg multimodal emotion recognition method based on the bidirectional lstm and attention mechanism","volume":"2021","author":"Zhao","year":"2021","journal-title":"Comput. Math. Methods Med"},{"key":"B45","doi-asserted-by":"publisher","first-page":"2176","DOI":"10.3390\/electronics9122176","article-title":"A novel motion intention recognition approach for soft exoskeleton via imu","volume":"9","author":"Zhu","year":"","journal-title":"Electronics"},{"key":"B46","doi-asserted-by":"publisher","first-page":"95","DOI":"10.3389\/fnbot.2020.583641","article-title":"A hybrid bci based on ssvep and eog for robotic arm control","volume":"14","author":"Zhu","year":"","journal-title":"Front. Neurorobot"},{"key":"B47","doi-asserted-by":"publisher","first-page":"3649","DOI":"10.1109\/LRA.2019.2926958","article-title":"Anyexo: A versatile and dynamic upper-limb rehabilitation robot","volume":"4","author":"Zimmermann","year":"2019","journal-title":"IEEE Rob. Autom. Lett"}],"container-title":["Frontiers in Neuroscience"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnins.2022.796290\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,4,25]],"date-time":"2022-04-25T11:13:12Z","timestamp":1650885192000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnins.2022.796290\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4,25]]},"references-count":47,"alternative-id":["10.3389\/fnins.2022.796290"],"URL":"https:\/\/doi.org\/10.3389\/fnins.2022.796290","relation":{},"ISSN":["1662-453X"],"issn-type":[{"value":"1662-453X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,4,25]]},"article-number":"796290"}}