{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T18:08:37Z","timestamp":1775066917160,"version":"3.50.1"},"reference-count":49,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,7,27]],"date-time":"2023-07-27T00:00:00Z","timestamp":1690416000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurorobot."],"abstract":"<jats:p>This research study proposes a unique framework that takes input from a surface electromyogram (sEMG) and functional near-infrared spectroscopy (fNIRS) bio-signals. These signals are trained using convolutional neural networks (CNN). The framework entails a real-time neuro-machine interface to decode the human intention of upper limb motions. The bio-signals from the two modalities are recorded for eight movements simultaneously for prosthetic arm functions focusing on trans-humeral amputees. The fNIRS signals are acquired from the human motor cortex, while sEMG is recorded from the human bicep muscles. The selected classification and command generation features are the peak, minimum, and mean \u0394HbO and \u0394HbR values within a 2-s moving window. In the case of sEMG, wavelength, peak, and mean were extracted with a 150-ms moving window. It was found that this scheme generates eight motions with an enhanced average accuracy of 94.5%. The obtained results validate the adopted research methodology and potential for future real-time neural-machine interfaces to control prosthetic arms.<\/jats:p>","DOI":"10.3389\/fnbot.2023.1174613","type":"journal-article","created":{"date-parts":[[2023,7,27]],"date-time":"2023-07-27T18:28:59Z","timestamp":1690482539000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["Deep learning-based framework for real-time upper limb motion intention classification using combined bio-signals"],"prefix":"10.3389","volume":"17","author":[{"given":"A. Usama","family":"Syed","sequence":"first","affiliation":[]},{"given":"Neelum Y.","family":"Sattar","sequence":"additional","affiliation":[]},{"given":"Ismaila","family":"Ganiyu","sequence":"additional","affiliation":[]},{"given":"Chintakindi","family":"Sanjay","sequence":"additional","affiliation":[]},{"given":"Soliman","family":"Alkhatib","sequence":"additional","affiliation":[]},{"given":"Bashir","family":"Salah","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2023,7,27]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"5354","DOI":"10.1364\/AO.47.005354","article-title":"Correction to the Beer\u2013Lambert\u2013Bouguer law for optical absorption","volume":"47","author":"Abitan","year":"2008","journal-title":"Appl. Opt"},{"key":"B2","doi-asserted-by":"publisher","first-page":"272","DOI":"10.3389\/fnagi.2014.00272","article-title":"Improved cerebral oxygenation response and executive performance as a function of cardiorespiratory fitness in older women: a fNIRS study","volume":"6","author":"Albinet","year":"2014","journal-title":"Front. Aging Neurosci"},{"key":"B3","doi-asserted-by":"publisher","first-page":"036015","DOI":"10.1088\/1741-2552\/ab0e2e","article-title":"Regression convolutional neural network for improved simultaneous EMG control","volume":"16","author":"Ameri","year":"2019","journal-title":"J. Neural Eng"},{"key":"B4","doi-asserted-by":"publisher","first-page":"1210","DOI":"10.1152\/japplphysiol.01115.2009","article-title":"Reaction time to peripheral visual stimuli during exercise under hypoxia","volume":"108","author":"Ando","year":"2010","journal-title":"J. Appl. Physiol"},{"key":"B5","doi-asserted-by":"publisher","first-page":"S24001","DOI":"10.1117\/1.NPh.9.S2.S24001","article-title":"Optical imaging and spectroscopy for the study of the human brain: status report","volume":"9","author":"Ayaz","year":"2022","journal-title":"Neurophotonics"},{"key":"B6","doi-asserted-by":"crossref","DOI":"10.1109\/EMBC.2018.8512678","article-title":"\u201cUpper limb prosthesis control: A hybrid EEG-EMG scheme for motion estimation in transhumeral subjects,\u201d","volume-title":"2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)","author":"Bakshi","year":"2018"},{"key":"B7","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/B978-0-12-817450-0.00001-8","article-title":"\u201cControl of transhumeral prostheses based on electromyography pattern recognition: from amputees to deep learning,\u201d","volume-title":"Powered Prostheses","author":"Barron","year":"2020"},{"key":"B8","doi-asserted-by":"publisher","first-page":"e15474","DOI":"10.1371\/journal.pone.0015474","article-title":"Speeded near infrared spectroscopy (NIRS) response detection","volume":"5","author":"Cui","year":"2010","journal-title":"PLoS ONE"},{"key":"B9","doi-asserted-by":"publisher","first-page":"056043","DOI":"10.1088\/1741-2552\/abc024","article-title":"Decoding of voluntary and involuntary upper-limb motor imagery based on graph fourier transform and cross-frequency coupling coefficients","volume":"17","author":"Feng","year":"2020","journal-title":"J. Neural Eng"},{"key":"B10","doi-asserted-by":"publisher","first-page":"466","DOI":"10.3390\/jcm7120466","article-title":"Applications of functional near-infrared spectroscopy (fNIRS) neuroimaging in exercise\u2013cognition science: a systematic, methodology-focused review","volume":"7","author":"Herold","year":"2018","journal-title":"J. Clin. Med"},{"key":"B11","doi-asserted-by":"publisher","first-page":"041403","DOI":"10.1117\/1.NPh.4.4.041403","article-title":"Functional near-infrared spectroscopy in movement science: a systematic review on cortical activity in postural and walking tasks","volume":"4","author":"Herold","year":"2017","journal-title":"Neurophotonics"},{"key":"B12","doi-asserted-by":"publisher","first-page":"246","DOI":"10.3389\/fnhum.2018.00246","article-title":"Feature extraction and classification methods for hybrid fNIRS-EEG brain-computer interfaces","volume":"12","author":"Hong","year":"2018","journal-title":"Front. Hum. Neurosci"},{"key":"B13","doi-asserted-by":"publisher","first-page":"2621","DOI":"10.1016\/j.neurobiolaging.2011.12.022","article-title":"Acute moderate exercise enhances compensatory brain activation in older adults","volume":"33","author":"Hyodo","year":"2012","journal-title":"Neurobiol. Aging"},{"key":"B14","doi-asserted-by":"publisher","first-page":"182","DOI":"10.1016\/j.dcn.2018.01.009","article-title":"Variability of the hemodynamic response in infants: influence of experimental design and stimulus complexity","volume":"33","author":"Issard","year":"2018","journal-title":"Dev. Cogn. Neurosci"},{"key":"B15","doi-asserted-by":"crossref","DOI":"10.1109\/ICORR.2017.8009419","article-title":"\u201cVoluntary phantom hand and finger movements in transhumerai amputees could be used to naturally control polydigital prostheses,\u201d","volume-title":"2017 International Conference on Rehabilitation Robotics (ICORR)","author":"Jarrasse","year":"2017"},{"key":"B16","doi-asserted-by":"publisher","first-page":"3198","DOI":"10.1038\/s41598-022-06805-4","article-title":"Novel fNIRS study on homogeneous symmetric feature-based transfer learning for brain\u2013computer interface","volume":"12","author":"Khalil","year":"2022","journal-title":"Sci. Rep"},{"key":"B17","doi-asserted-by":"publisher","first-page":"6","DOI":"10.3389\/fnbot.2017.00006","article-title":"Hybrid EEG\u2013fNIRS-based eight-command decoding for BCI: application to quadcopter control","volume":"11","author":"Khan","year":"2017","journal-title":"Front. Neurorobot"},{"key":"B18","doi-asserted-by":"publisher","first-page":"1335","DOI":"10.1111\/psyp.12688","article-title":"Effects of continuous and intermittent exercise on executive function in children aged 8\u201310 years","volume":"53","author":"Lambrick","year":"2016","journal-title":"Psychophysiology"},{"key":"B19","doi-asserted-by":"publisher","first-page":"046014","DOI":"10.1088\/1741-2552\/aa71c0","article-title":"A Gaussian mixture model based adaptive classifier for fNIRS brain\u2013computer interfaces and its testing via simulation","volume":"14","author":"Li","year":"2017","journal-title":"J. Neural Eng"},{"key":"B20","doi-asserted-by":"publisher","first-page":"036024","DOI":"10.1088\/1741-2552\/abe357","article-title":"Improving the performance of multisubject motor imagery-based BCIs using twin cascaded softmax CNNs","volume":"18","author":"Luo","year":"2021","journal-title":"J. Neural Eng"},{"key":"B21","doi-asserted-by":"publisher","first-page":"873239","DOI":"10.3389\/fnbot.2022.873239","article-title":"EEG-fNIRS-based hybrid image construction and classification using CNN-LSTM","volume":"16","author":"Mughal","year":"2022","journal-title":"Front. Neurorobot"},{"key":"B22","doi-asserted-by":"publisher","first-page":"122","DOI":"10.1049\/iet-csr.2020.0008","article-title":"Gesture recognition for transhumeral prosthesis control using EMG and NIR","volume":"2","author":"Nsugbe","year":"2020","journal-title":"IET Cyber-Syst. Robot"},{"key":"B23","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/LSENS.2019.2906386","article-title":"Time derivative moments based feature extraction approach for recognition of upper limb motions using EMG","volume":"3","author":"Pancholi","year":"2019","journal-title":"IEEE Sensors Lett"},{"key":"B24","doi-asserted-by":"publisher","first-page":"3819","DOI":"10.1109\/TCYB.2020.3016595","article-title":"Advanced energy kernel-based feature extraction scheme for improved EMG-PR-based prosthesis control against force variation","volume":"52","author":"Pancholi","year":"2020","journal-title":"IEEE Trans. Cybern"},{"key":"B25","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1007\/978-3-031-20784-6_10","article-title":"\u201cBrain-computer interface,\u201d","volume-title":"Humans and Electricity: Understanding Body Electricity and Applications","author":"Park","year":"2023"},{"key":"B26","doi-asserted-by":"crossref","DOI":"10.1109\/TNSRE.2023.3237583","article-title":"\u201cImproving performance of motor imagery-based brain-computer interface in poorly performing subjects using a hybrid-imagery method utilizing combined motor and somatosensory activity,\u201d","volume-title":"IEEE Transactions on Neural Systems and Rehabilitation Engineering","author":"Park","year":"2023"},{"key":"B27","doi-asserted-by":"publisher","first-page":"289","DOI":"10.1016\/j.ymeth.2008.04.005","article-title":"Non-invasive NIR spectroscopy of human brain function during exercise","volume":"45","author":"Perrey","year":"2008","journal-title":"Methods"},{"key":"B28","doi-asserted-by":"publisher","first-page":"641","DOI":"10.3389\/fnhum.2017.00641","article-title":"Signal processing in functional near-infrared spectroscopy (fNIRS): methodological differences lead to different statistical results","volume":"11","author":"Pfeifer","year":"2018","journal-title":"Front. Hum. Neurosci"},{"key":"B29","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1186\/s12984-023-01127-6","article-title":"Exploring high-density corticomuscular networks after stroke to enable a hybrid Brain-Computer Interface for hand motor rehabilitation","volume":"20","author":"Pichiorri","year":"2023","journal-title":"J. Neuroeng. Rehabil"},{"key":"B30","doi-asserted-by":"publisher","first-page":"46","DOI":"10.1177\/1094428116658959","article-title":"Functional near-infrared spectroscopy (fNIRS) for assessing cerebral cortex function during human behavior in natural\/social situations: a concise review","volume":"22","author":"Quaresima","year":"2019","journal-title":"Organ. Res. Methods"},{"key":"B31","doi-asserted-by":"publisher","first-page":"726","DOI":"10.3390\/s22030726","article-title":"fNIRS-based upper limb motion intention recognition using an artificial neural network for transhumeral amputees","volume":"22","author":"Sattar","year":"2022","journal-title":"Sensors"},{"key":"B32","doi-asserted-by":"publisher","first-page":"3522","DOI":"10.1007\/s12555-019-1058-5","article-title":"EMG-based control of transhumeral prosthesis using machine learning algorithms","volume":"19","author":"Sattar","year":"","journal-title":"Int. J. Control Autom. Syst"},{"key":"B33","doi-asserted-by":"publisher","first-page":"113246","DOI":"10.1109\/ACCESS.2021.3099973","article-title":"Enhancing classification accuracy of transhumeral prosthesis: a hybrid sEMG and fNIRS approach","volume":"9","author":"Sattar","year":"","journal-title":"IEEE Access"},{"key":"B34","doi-asserted-by":"crossref","DOI":"10.1109\/AIM.2019.8868796","article-title":"\u201cReal-time EMG signal processing with implementation of PID control for upper-limb prosthesis,\u201d","volume-title":"2019 IEEE\/ASME International Conference on Advanced Intelligent Mechatronics (AIM)","author":"Sattar","year":"2019"},{"key":"B35","doi-asserted-by":"publisher","first-page":"649","DOI":"10.1088\/0967-3334\/31\/5\/004","article-title":"How to detect and reduce movement artifacts in near-infrared imaging using moving standard deviation and spline interpolation","volume":"31","author":"Scholkmann","year":"2010","journal-title":"Physiol. Meas"},{"key":"B36","first-page":"6","article-title":"Measuring brain activity using functional near infrared spectroscopy: a short review","volume":"24","author":"Scholkmann","year":"2012","journal-title":"Spectrosc. Eur."},{"key":"B37","doi-asserted-by":"publisher","first-page":"679","DOI":"10.1016\/S0006-3223(02)01550-0","article-title":"Non-invasive neuroimaging using near-infrared light","volume":"52","author":"Strangman","year":"2002","journal-title":"Biol. Psychiatry"},{"key":"B38","doi-asserted-by":"publisher","first-page":"865","DOI":"10.1016\/S1053-8119(03)00021-1","article-title":"Factors affecting the accuracy of near-infrared spectroscopy concentration calculations for focal changes in oxygenation parameters","volume":"18","author":"Strangman","year":"2003","journal-title":"Neuroimage"},{"key":"B39","article-title":"\u201cControl of a prosthetic arm using fNIRS, a neural-machine interface,\u201d","volume-title":"Data Acquisition-Recent Advances and Applications in Biomedical Engineering","author":"Syed","year":"2020"},{"key":"B40","doi-asserted-by":"crossref","DOI":"10.1109\/ICRAI54018.2021.9651388","article-title":"\u201cUpper limb motion detection using deep neural network,\u201d","volume-title":"2021 International Conference on Robotics and Automation in Industry (ICRAI)","author":"Syed","year":"2021"},{"key":"B41","doi-asserted-by":"publisher","DOI":"10.1109\/TIM.2023.3277930","article-title":"Extracting spatial muscle activation patterns in facial and neck muscles for silent speech recognition using high-density sEMG","author":"Tan","year":"2023","journal-title":"IEEE Trans. Instrum. Meas"},{"key":"B42","doi-asserted-by":"publisher","first-page":"435","DOI":"10.1016\/S0166-2236(97)01132-6","article-title":"Non-invasive optical spectroscopy and imaging of human brain function","volume":"20","author":"Villringer","year":"1997","journal-title":"Trends Neurosci"},{"key":"B43","doi-asserted-by":"publisher","first-page":"1","DOI":"10.21307\/ijssis-2018-005","article-title":"Technical features and functionalities of Myo armband: an overview on related literature and advanced applications of myoelectric armbands mainly focused on arm prostheses","volume":"11","author":"Visconti","year":"2018","journal-title":"Int. J. Smart Sens. Intell. Syst."},{"key":"B44","doi-asserted-by":"publisher","first-page":"056003","DOI":"10.1088\/1741-2552\/abeead","article-title":"A convolutional neural network to identify motor units from high-density surface electromyography signals in real time","volume":"18","author":"Wen","year":"2021","journal-title":"J. Neural Eng"},{"key":"B45","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/1475-925X-13-8","article-title":"Hybrid soft computing systems for electromyographic signals analysis: a review","volume":"13","author":"Xie","year":"2014","journal-title":"Biomed. Eng. Online"},{"key":"B46","doi-asserted-by":"publisher","DOI":"10.1117\/12.2511595","article-title":"\u201cInitial-dip-based classification for fNIRS-BCI,\u201d","author":"Zafar","year":"2019","journal-title":"Neural Imaging and Sensing"},{"key":"B47","doi-asserted-by":"publisher","first-page":"066004","DOI":"10.1088\/1741-2552\/ab3471","article-title":"A novel hybrid deep learning scheme for four-class motor imagery classification","volume":"16","author":"Zhang","year":"2019","journal-title":"J. Neural Eng"},{"key":"B48","doi-asserted-by":"publisher","first-page":"031002","DOI":"10.1088\/1741-2552\/abc902","article-title":"A survey on deep learning-based non-invasive brain signals: recent advances and new frontiers","volume":"18","author":"Zhang","year":"2021","journal-title":"J. Neural Eng"},{"key":"B49","doi-asserted-by":"publisher","first-page":"2974","DOI":"10.1152\/jn.00178.2007","article-title":"Decoding a new neural\u2013machine interface to control artificial limbs","volume":"98","author":"Zhou","year":"2007","journal-title":"J. Neurophysiol"}],"container-title":["Frontiers in Neurorobotics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1174613\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,7,27]],"date-time":"2023-07-27T18:29:10Z","timestamp":1690482550000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1174613\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,27]]},"references-count":49,"alternative-id":["10.3389\/fnbot.2023.1174613"],"URL":"https:\/\/doi.org\/10.3389\/fnbot.2023.1174613","relation":{},"ISSN":["1662-5218"],"issn-type":[{"value":"1662-5218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,27]]},"article-number":"1174613"}}