{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,30]],"date-time":"2025-07-30T12:52:32Z","timestamp":1753879952240,"version":"3.41.2"},"reference-count":42,"publisher":"ASME International","issue":"5","license":[{"start":{"date-parts":[[2022,6,7]],"date-time":"2022-06-07T00:00:00Z","timestamp":1654560000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.asme.org\/publications-submissions\/publishing-information\/legal-policies"}],"content-domain":{"domain":["asmedigitalcollection.asme.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,10,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Efficient human\u2013robot collaboration during physical interaction requires estimating the human state for optimal role allocation and load sharing. Machine learning (ML) methods are gaining popularity for estimating the interaction parameters from physiological signals. However, due to individual differences, the ML models might not generalize well to new subjects. In this study, we present a convolution neural network (CNN) model to predict motor control difficulty using surface electromyography (sEMG) from human upper limb during physical human\u2013robot interaction (pHRI) task and present a transfer learning approach to transfer a learned model to new subjects. Twenty-six individuals participated in a pHRI experiment where a subject guides the robot's end-effector with different levels of motor control difficulty. The motor control difficulty is varied by changing the damping parameter of the robot from low to high and constraining the motion to gross and fine movements. A CNN network with raw sEMG as input is used to classify the motor control difficulty. The CNN's transfer learning approach is compared against Riemann geometry-based Procrustes analysis (RPA). With very few labeled samples from new subjects, we demonstrate that the CNN-based transfer learning approach (avg. 69.77%) outperforms the RPA transfer learning (avg. 59.20%). Moreover, we observe that the subject's skill level in the pre-trained model has no significant effect on the transfer learning performance of the new users.<\/jats:p>","DOI":"10.1115\/1.4054594","type":"journal-article","created":{"date-parts":[[2022,5,17]],"date-time":"2022-05-17T05:10:14Z","timestamp":1652764214000},"update-policy":"https:\/\/doi.org\/10.1115\/crossmarkpolicy-asme","source":"Crossref","is-referenced-by-count":7,"title":["Transfer Learning of Motor Difficulty Classification in Physical Human\u2013Robot Interaction Using Electromyography"],"prefix":"10.1115","volume":"22","author":[{"given":"Hemanth","family":"Manjunatha","sequence":"first","affiliation":[{"name":"University at Buffalo Human in the Loop Systems Lab, , Buffalo, NY 14260"}]},{"given":"Sri Sadhan","family":"Jujjavarapu","sequence":"additional","affiliation":[{"name":"University at Buffalo Human in the Loop Systems Lab, , Buffalo, NY 14260"}]},{"given":"Ehsan T.","family":"Esfahani","sequence":"additional","affiliation":[{"name":"University at Buffalo Human in the Loop Systems Lab, , Buffalo, NY 14260"}]}],"member":"33","published-online":{"date-parts":[[2022,6,7]]},"reference":[{"issue":"4","key":"2022060715261916800_CIT0001","doi-asserted-by":"publisher","first-page":"1729881417716010","DOI":"10.1177\/1729881417716010","article-title":"Human\u2013Robot Collaboration in Industrial Applications: Safety, Interaction and Trust","volume":"14","author":"Maurtua","year":"2017","journal-title":"Int. J. Adv. Robot. Syst."},{"key":"2022060715261916800_CIT0002","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1016\/j.cosrev.2016.09.001","article-title":"An Overview of Self-Adaptive Technologies Within Virtual Reality Training","volume":"22","author":"Vaughan","year":"2016","journal-title":"Comput. Sci. Rev."},{"issue":"4","key":"2022060715261916800_CIT0003","doi-asserted-by":"publisher","first-page":"389","DOI":"10.1038\/s41587-019-0045-y","article-title":"Wearable Biosensors for Healthcare Monitoring","volume":"37","author":"Kim","year":"2019","journal-title":"Nat. Biotechnol."},{"issue":"6","key":"2022060715261916800_CIT0004","doi-asserted-by":"publisher","first-page":"443","DOI":"10.3233\/THC-2010-0604","article-title":"Electrograms (ECG, EEG, EMG, EOG)","volume":"18","author":"Reilly","year":"2010","journal-title":"Technol. Health Care"},{"issue":"4","key":"2022060715261916800_CIT0005","doi-asserted-by":"publisher","first-page":"760","DOI":"10.1109\/TNSRE.2019.2896269","article-title":"Deep Learning for Electromyographic Hand Gesture Signal Classification Using Transfer Learning","volume":"27","author":"C\u00f4t\u00e9-Allard","year":"2019","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"issue":"2","key":"2022060715261916800_CIT0006","doi-asserted-by":"publisher","first-page":"370","DOI":"10.1109\/TNSRE.2019.2962189","article-title":"A Deep Transfer Learning Approach to Reducing the Effect of Electrode Shift in EMG Pattern Recognition-Based Control","volume":"28","author":"Ameri","year":"2019","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"issue":"2","key":"2022060715261916800_CIT0007","doi-asserted-by":"publisher","first-page":"371","DOI":"10.2466\/pms.106.2.371-386","article-title":"Effect of Physical Fatigue on Motor Control at Different Skill Levels","volume":"106","author":"Aune","year":"2008","journal-title":"Percept. Motor Skills"},{"issue":"1","key":"2022060715261916800_CIT0008","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1115\/1.3140702","article-title":"Impedance Control: An Approach to Manipulation: Part I\u2014Theory","volume":"107","author":"Hogan","year":"1985","journal-title":"J. Dyn. Syst. Meas. Control"},{"issue":"2","key":"2022060715261916800_CIT0009","doi-asserted-by":"publisher","first-page":"613","DOI":"10.1109\/TMECH.2015.2465849","article-title":"A Review of Algorithms for Compliant Control of Stiff and Fixed-Compliance Robots","volume":"21","author":"Calanca","year":"2015","journal-title":"IEEE\/ASME Trans. Mechatron."},{"key":"2022060715261916800_CIT0010","first-page":"001900","article-title":"Variable Admittance Control in pHRI Using EMG-Based Arm Muscles Co-Activation","author":"Grafakos","year":"2016"},{"key":"2022060715261916800_CIT0011","doi-asserted-by":"publisher","first-page":"5","DOI":"10.3389\/frobt.2017.00058","article-title":"A User Study on Personalized Stiffness Control and Task Specificity in Physical Human\u2013Robot Interaction","volume":"4","author":"Gopinathan","year":"2017","journal-title":"Front. Robot. AI"},{"issue":"4","key":"2022060715261916800_CIT0012","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1108\/ir-05-2018-0111","article-title":"Improving Stability in Physical Human\u2013Robot Interaction by Estimating Human Hand Stiffness and a Vibration Index","volume":"46","author":"Bian","year":"2018","journal-title":"Ind. Robot."},{"key":"2022060715261916800_CIT0013","doi-asserted-by":"crossref","DOI":"10.1115\/DETC2019-98506","article-title":"Stiffness Based Stability Enhancement in Human\u2013Robot Collaboration","author":"Jujjavarapu","year":"2019"},{"issue":"11","key":"2022060715261916800_CIT0014","doi-asserted-by":"publisher","first-page":"1421","DOI":"10.1177\/0278364918768950","article-title":"Admittance Control for Physical Human\u2013Robot Interaction","volume":"37","author":"Keemink","year":"2018","journal-title":"Int. J. Robot. Res."},{"key":"2022060715261916800_CIT0015","first-page":"229","article-title":"EEG Correlates of Motor Control Difficulty in Physical Human\u2013Robot Interaction: A Frequency Domain Analysis","author":"Memar","year":"2018"},{"issue":"6","key":"2022060715261916800_CIT0016","doi-asserted-by":"publisher","first-page":"616","DOI":"10.1093\/iwc\/iwu021","article-title":"Workload Estimation in Physical Human\u2013Robot Interaction Using Physiological Measurements","volume":"27","author":"Novak","year":"2014","journal-title":"Interact. Comput."},{"issue":"8","key":"2022060715261916800_CIT0017","doi-asserted-by":"publisher","first-page":"1304","DOI":"10.3390\/s16081304","article-title":"A Review of Classification Techniques of EMG Signals During Isotonic and Isometric Contractions","volume":"16","author":"Nazmi","year":"2016","journal-title":"Sensors"},{"issue":"1","key":"2022060715261916800_CIT0018","doi-asserted-by":"publisher","first-page":"38","DOI":"10.1080\/00221309.1930.9918189","article-title":"Individual Differences in Motor Skills","volume":"3","author":"Seashore","year":"1930","journal-title":"J. Gen. Psychol."},{"key":"2022060715261916800_CIT0019","first-page":"1","article-title":"Factorization of EMG Via Muscle Synergies in Walking Task: Evaluation of Intra-Subject and Inter-Subject Variability","author":"Taborri","year":"2017"},{"key":"2022060715261916800_CIT0020","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.cmpb.2018.04.005","article-title":"Deep Learning for Healthcare Applications Based on Physiological Signals: A Review","volume":"161","author":"Faust","year":"2018","journal-title":"Comput. Methods Programs Biomed."},{"key":"2022060715261916800_CIT0021","first-page":"536","article-title":"Deep Learning for Tactile Understanding From Visual and Haptic Data","author":"Gao","year":"2016"},{"issue":"10","key":"2022060715261916800_CIT0022","doi-asserted-by":"publisher","first-page":"e0206049","DOI":"10.1371\/journal.pone.0206049","article-title":"A Novel Attention-Based Hybrid CNN-RNN Architecture for sEMG-Based Gesture Recognition","volume":"13","author":"Hu","year":"2018","journal-title":"PLoS One"},{"key":"2022060715261916800_CIT0023","unstructured":"Manjunatha, H.\n          , 2021, \u201cAddressing Stability, Transferability, and Interpretability Issues in Physical Human\u2013Robot Interaction Using Physiological Data and Deep Learning,\u201d PhD thesis, State University of New York at Buffalo."},{"key":"2022060715261916800_CIT0024","first-page":"270","article-title":"A Survey on Deep Transfer Learning","author":"Tan","year":"2018"},{"issue":"3","key":"2022060715261916800_CIT0025","doi-asserted-by":"publisher","first-page":"458","DOI":"10.3390\/s17030458","article-title":"Surface EMG-Based Inter-Session Gesture Recognition Enhanced by Deep Domain Adaptation","volume":"17","author":"Du","year":"2017","journal-title":"Sensors"},{"key":"2022060715261916800_CIT0026","doi-asserted-by":"publisher","first-page":"699174.1","DOI":"10.3389\/fnbot.2021.699174","article-title":"Improvement of EMG Pattern Recognition Model Performance in Repeated Uses by Combining Feature Selection and Incremental Transfer Learning","volume":"15","author":"Li","year":"2021","journal-title":"Front. Neurorobot."},{"issue":"3","key":"2022060715261916800_CIT0027","doi-asserted-by":"publisher","first-page":"512","DOI":"10.1109\/JAS.2021.1003865","article-title":"Deep Learning for EMG-Based Human\u2013Machine Interaction: A Review","volume":"8","author":"Xiong","year":"2021","journal-title":"IEEE\/CAA J. Autom. Sin."},{"issue":"11","key":"2022060715261916800_CIT0028","doi-asserted-by":"publisher","first-page":"5391","DOI":"10.1002\/hbm.23730","article-title":"Deep Learning With Convolutional Neural Networks for EEG Decoding and Visualization","volume":"38","author":"Schirrmeister","year":"2017","journal-title":"Hum. Brain Mapp."},{"issue":"9","key":"2022060715261916800_CIT0029","doi-asserted-by":"crossref","first-page":"3760","DOI":"10.1109\/TNNLS.2019.2944933","article-title":"Deep Adaptive Input Normalization for Time Series Forecasting","volume":"31","author":"Passalis","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"issue":"4","key":"2022060715261916800_CIT0030","doi-asserted-by":"publisher","first-page":"920","DOI":"10.1109\/TBME.2011.2172210","article-title":"Multiclass Brain\u2013Computer Interface Classification by Riemannian Geometry","volume":"59","author":"Barachant","year":"2011","journal-title":"IEEE Trans. Biomed. Eng."},{"issue":"1","key":"2022060715261916800_CIT0031","doi-asserted-by":"publisher","first-page":"110","DOI":"10.1186\/s12984-015-0102-9","article-title":"Improving Robustness Against Electrode Shift of High Density EMG for Myoelectric Control Through Common Spatial Patterns","volume":"12","author":"Pan","year":"2015","journal-title":"J. NeuroEng. Rehabil."},{"issue":"3","key":"2022060715261916800_CIT0032","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1080\/2326263X.2017.1297192","article-title":"Riemannian Geometry for EEG-Based Brain\u2013Computer Interfaces; a Primer and a Review","volume":"4","author":"Congedo","year":"2017","journal-title":"Brain Comput. Interface"},{"key":"2022060715261916800_CIT0033","doi-asserted-by":"publisher","DOI":"10.3217\/978-3-85125-467-9-181","article-title":"Extraction of Motor Patterns From Joint EEG\/EMG Recording: A Riemannian Geometry Approach","author":"Barachant","year":"2016"},{"key":"2022060715261916800_CIT0034","first-page":"2708","article-title":"Classification of Motor Control Difficulty Using EMG in Physical Human\u2013Robot Interaction","author":"Manjunatha","year":"2020"},{"issue":"8","key":"2022060715261916800_CIT0035","doi-asserted-by":"publisher","first-page":"2390","DOI":"10.1109\/TBME.2018.2889705","article-title":"Riemannian Procrustes Analysis: Transfer Learning for Brain\u2013Computer Interfaces","volume":"66","author":"Rodrigues","year":"2019","journal-title":"IEEE Trans. Biomed. Eng."},{"issue":"10","key":"2022060715261916800_CIT0036","doi-asserted-by":"publisher","first-page":"1345","DOI":"10.1109\/TKDE.2009.191","article-title":"A Survey on Transfer Learning","volume":"22","author":"Pan","year":"2009","journal-title":"IEEE Trans. Knowl. Data Eng."},{"issue":"9","key":"2022060715261916800_CIT0037","doi-asserted-by":"publisher","first-page":"1790","DOI":"10.1109\/TPAMI.2015.2500224","article-title":"Factors of Transferability for a Generic Convnet Representation","volume":"38","author":"Azizpour","year":"2015","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"issue":"1","key":"2022060715261916800_CIT0038","doi-asserted-by":"publisher","first-page":"563","DOI":"10.1613\/jair.5756","article-title":"On the Behavior of Convolutional Nets for Feature Extraction","volume":"61","author":"Garcia-Gasulla","year":"2018","journal-title":"J. Artif. Intell. Res."},{"key":"2022060715261916800_CIT0039","doi-asserted-by":"publisher","first-page":"54789","DOI":"10.1109\/ACCESS.2020.2979074","article-title":"Cross-Domain MLP and CNN Transfer Learning for Biological Signal Processing: EEG and EMG","volume":"8","author":"Bird","year":"2020","journal-title":"IEEE Access"},{"issue":"2","key":"2022060715261916800_CIT0040","doi-asserted-by":"publisher","first-page":"2851","DOI":"10.1109\/LRA.2022.3142721","article-title":"Deep Heterogeneous Dilation of LSTM for Transient-Phase Gesture Prediction Through High-Density Electromyography: Towards Application in Neurorobotics","volume":"7","author":"Sun","year":"2022","journal-title":"IEEE Robot. Autom. Lett."},{"key":"2022060715261916800_CIT0041","first-page":"2579","article-title":"Visualizing Data Using t-SNE","volume":"9","author":"van der","year":"2008","journal-title":"J. Mach. Learn. Res."},{"issue":"2","key":"2022060715261916800_CIT0042","doi-asserted-by":"publisher","first-page":"267","DOI":"10.1109\/TOH.2016.2518670","article-title":"Online Stability in Human\u2013Robot Cooperation With Admittance Control","volume":"9","author":"Dimeas","year":"2016","journal-title":"IEEE Trans. Haptics"}],"container-title":["Journal of Computing and Information Science in Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/22\/5\/050908\/6885349\/jcise_22_5_050908.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/22\/5\/050908\/6885349\/jcise_22_5_050908.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,6,7]],"date-time":"2022-06-07T15:26:33Z","timestamp":1654615593000},"score":1,"resource":{"primary":{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article\/22\/5\/050908\/1141032\/Transfer-Learning-of-Motor-Difficulty"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,7]]},"references-count":42,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2022,10,1]]}},"URL":"https:\/\/doi.org\/10.1115\/1.4054594","relation":{},"ISSN":["1530-9827","1944-7078"],"issn-type":[{"type":"print","value":"1530-9827"},{"type":"electronic","value":"1944-7078"}],"subject":[],"published":{"date-parts":[[2022,6,7]]},"article-number":"050908"}}