{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,5]],"date-time":"2026-03-05T18:24:55Z","timestamp":1772735095923,"version":"3.50.1"},"reference-count":43,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2020,9,25]],"date-time":"2020-09-25T00:00:00Z","timestamp":1600992000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Key R&amp;D Program of China","award":["2017YFB1300204 and 2018YFF01012304"],"award-info":[{"award-number":["2017YFB1300204 and 2018YFF01012304"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In manufacturing, traditional task pre-programming methods limit the efficiency of human\u2013robot skill transfer. This paper proposes a novel task-learning strategy, enabling robots to learn skills from human demonstrations flexibly and generalize skills under new task situations. Specifically, we establish a markerless vision capture system to acquire continuous human hand movements and develop a threshold-based heuristic segmentation algorithm to segment the complete movements into different movement primitives (MPs) which encode human hand movements with task-oriented models. For movement primitive learning, we adopt a Gaussian mixture model and Gaussian mixture regression (GMM-GMR) to extract the optimal trajectory encapsulating sufficient human features and utilize dynamical movement primitives (DMPs) to learn for trajectory generalization. In addition, we propose an improved visuo-spatial skill learning (VSL) algorithm to learn goal configurations concerning spatial relationships between task-relevant objects. Only one multioperation demonstration is required for learning, and robots can generalize goal configurations under new task situations following the task execution order from demonstration. A series of peg-in-hole experiments demonstrate that the proposed task-learning strategy can obtain exact pick-and-place points and generate smooth human-like trajectories, verifying the effectiveness of the proposed strategy.<\/jats:p>","DOI":"10.3390\/s20195505","type":"journal-article","created":{"date-parts":[[2020,9,25]],"date-time":"2020-09-25T08:57:32Z","timestamp":1601024252000},"page":"5505","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["A Task-Learning Strategy for Robotic Assembly Tasks from Human Demonstrations"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3400-3037","authenticated-orcid":false,"given":"Guanwen","family":"Ding","sequence":"first","affiliation":[{"name":"State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin 150001, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yubin","family":"Liu","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin 150001, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xizhe","family":"Zang","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin 150001, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8138-2802","authenticated-orcid":false,"given":"Xuehe","family":"Zhang","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin 150001, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0078-0281","authenticated-orcid":false,"given":"Gangfeng","family":"Liu","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin 150001, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jie","family":"Zhao","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin 150001, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,9,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Roy, S., and Edan, Y. (2018). Investigating joint-action in short-cycle repetitive handover tasks: The role of giver versus receiver and its implications for human-robot collaborative system design. Int. J. Soc. Robot., 1\u201316.","DOI":"10.1007\/s12369-017-0424-9"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"248","DOI":"10.1016\/j.procir.2015.08.014","article-title":"Design considerations for safe human-robot collaborative workplaces","volume":"37","author":"Michalos","year":"2015","journal-title":"Procedia CIrP"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Someshwar, R., and Kerner, Y. (2013, January 13\u201316). Optimization of waiting time in HR coordination. Proceedings of the 2013 IEEE International Conference on Systems, Man, and Cybernetics, Manchester, UK.","DOI":"10.1109\/SMC.2013.330"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"957","DOI":"10.1007\/s10514-017-9677-2","article-title":"Progress and prospects of the human\u2013robot collaboration","volume":"42","author":"Ajoudani","year":"2018","journal-title":"Auton. Robot."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"698","DOI":"10.3182\/20120905-3-HR-2030.00134","article-title":"A timing control model for hr synchronization","volume":"45","author":"Someshwar","year":"2012","journal-title":"IFAC Proc. Vol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"701","DOI":"10.1016\/j.cirp.2019.05.002","article-title":"Symbiotic human-robot collaborative assembly","volume":"68","author":"Wang","year":"2019","journal-title":"CIRP Ann."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1272","DOI":"10.1080\/0951192X.2017.1307524","article-title":"On a human\u2013robot workplace design and task allocation system","volume":"30","author":"Tsarouchi","year":"2017","journal-title":"Int. J. Comput. Integr. Manuf."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3054912","article-title":"Imitation learning: A survey of learning methods","volume":"50","author":"Hussein","year":"2017","journal-title":"ACM Comput. Surv."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Zhu, Z., and Hu, H. (2018). Robot learning from demonstration in robotic assembly: A survey. Robotics, 7.","DOI":"10.3390\/robotics7020017"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1117","DOI":"10.1109\/TMECH.2019.2909081","article-title":"Robotic skill learning for precision assembly with microscopic vision and force feedback","volume":"24","author":"Qin","year":"2019","journal-title":"IEEE. ASME. Trans. Mechatron."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1007\/s10514-018-9725-6","article-title":"Robot learning of industrial assembly task via human demonstrations","volume":"43","author":"Kyrarini","year":"2019","journal-title":"Auton. Robot."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"292","DOI":"10.1016\/j.rcim.2018.12.007","article-title":"Trajectory generation for robotic assembly operations using learning by demonstration","volume":"57","author":"Duque","year":"2019","journal-title":"Robot. Comput. Integr. Manuf"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"765","DOI":"10.1007\/s12369-019-00595-y","article-title":"Imitation of Human Motion by Low Degree-of-Freedom Simulated Robots and Human Preference for Mappings Driven by Spinal, Arm, and Leg Activity","volume":"11","author":"Kaushik","year":"2019","journal-title":"Int. J. Soc. Robot."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Edmonds, M., Gao, F., Xie, X., Liu, H., Qi, S., Zhu, Y., Rothrock, B., and Zhu, S.C. (2017, January 24\u201328). Feeling the force: Integrating force and pose for fluent discovery through imitation learning to open medicine bottles. Proceedings of the 2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada.","DOI":"10.1109\/IROS.2017.8206196"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1153","DOI":"10.1109\/TII.2018.2826064","article-title":"A learning framework of adaptive manipulative skills from human to robot","volume":"15","author":"Yang","year":"2019","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Mao, R., Yang, Y., Ferm\u00fcller, C., Aloimonos, Y., and Baras, J.S. (2015, January 18\u201320). Learning hand movements from markerless demonstrations for humanoid tasks. Proceedings of the 2014 IEEE-RAS International Conference on Humanoid Robots, Madrid, Spain.","DOI":"10.1109\/HUMANOIDS.2014.7041476"},{"key":"ref_17","first-page":"3","article-title":"Efficient model-based 3D tracking of hand articulations using Kinect","volume":"1","author":"Oikonomidis","year":"2011","journal-title":"BmVC"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"561","DOI":"10.1007\/11008941_60","article-title":"Learning movement primitives","volume":"15","author":"Schaal","year":"2005","journal-title":"Springer Tracts Adv. Rob."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"713","DOI":"10.1007\/s10514-018-9745-2","article-title":"Learning from demonstration for semi-autonomous teleoperation","volume":"43","author":"Havoutis","year":"2019","journal-title":"Auton. Robot."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s11370-015-0187-9","article-title":"A tutorial on task-parameterized movement learning and retrieval","volume":"9","author":"Calinon","year":"2016","journal-title":"Intell. Serv. Robot."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1162\/NECO_a_00393","article-title":"Dynamical movement primitives: Learning attractor models for motor behaviors","volume":"25","author":"Ijspeert","year":"2013","journal-title":"Neural. Comput."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Kormushev, P., Calinon, S., and Caldwell, D.G. (2010, January 18\u201322). Robot motor skill coordination with EM-based reinforcement learning. Proceedings of the 2010 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Taipei, Taiwan.","DOI":"10.1109\/IROS.2010.5649089"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1177\/0278364912472380","article-title":"Learning to select and generalize striking movements in robot table tennis","volume":"32","author":"Kober","year":"2013","journal-title":"Int. J. Rob. Res."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1007\/s10514-017-9648-7","article-title":"Using probabilistic movement primitives in robotics","volume":"42","author":"Paraschos","year":"2018","journal-title":"Auton. Robot."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"670","DOI":"10.1109\/TSMC.2016.2635479","article-title":"Teaching a robot the semantics of assembly tasks","volume":"48","author":"Savarimuthu","year":"2018","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1177\/0278364914554471","article-title":"Learning grounded finite-state representations from unstructured demonstrations","volume":"34","author":"Niekum","year":"2015","journal-title":"Int. J. Rob. Res."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Lioutikov, R., Neumann, G., Maeda, G., and Peters, J. (2015, January 3\u20135). Probabilistic segmentation applied to an assembly task. Proceedings of the 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), Seoul, South Korea.","DOI":"10.1109\/HUMANOIDS.2015.7363584"},{"key":"ref_28","first-page":"1281","article-title":"Joint modeling of multiple time series via the beta process with application to motion capture segmentation","volume":"8","author":"Fox","year":"2014","journal-title":"Ann. Stat."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1111\/j.2517-6161.1977.tb01600.x","article-title":"Maximum likelihood from incomplete data via the EM algorithm","volume":"39","author":"Dempster","year":"1977","journal-title":"J. Roy. Statist. Soc. B Methodol."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Dantam, N., Essa, I., and Stilman, M. (2012, January 7\u201312). Linguistic transfer of human assembly tasks to robots. Proceedings of the 2012 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Vilamoura, Portugal.","DOI":"10.1109\/IROS.2012.6385749"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Ahmadzadeh, S.R., Kormushev, P., and Caldwell, D.G. (2013, January 3\u20137). Visuospatial skill learning for object reconfiguration tasks. Proceedings of the 2013 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Tokyo, Japan.","DOI":"10.1109\/IROS.2013.6696425"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Ahmadzadeh, S.R., Paikan, A., Mastrogiovanni, F., Natale, L., Kormushev, P., and Caldwell, D.G. (2015, January 26\u201330). Learning symbolic representations of actions from human demonstrations. Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA.","DOI":"10.1109\/ICRA.2015.7139728"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"224","DOI":"10.1016\/j.cviu.2010.10.002","article-title":"A survey of vision-based methods for action representation, segmentation and recognition","volume":"115","author":"Weinland","year":"2011","journal-title":"Comput. Vis. Image. Underst."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"286","DOI":"10.1109\/TSMCB.2006.886952","article-title":"On learning, representing, and generalizing a task in a humanoid robot","volume":"37","author":"Calinon","year":"2007","journal-title":"IEEE Trans. Syst. Man Cybern. B"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Murari, A., Peluso, E., Cianfrani, F., Gaudio, P., and Lungaroni, M. (2019). On the use of entropy to improve model selection criteria. Entropy, 21.","DOI":"10.3390\/e21040394"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1504\/IJCSM.2019.098745","article-title":"A novel computation method for 2D deformation of fish scale based on SURF and NR optimisation","volume":"10","author":"Li","year":"2019","journal-title":"Int. J. Comput. Math"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Xie, B., Li, J., and Zhao, X. (2020). Strain Measurement Based on Speeded-up Robust Feature Algorithm Applied to Microimages from a Smartphone-Based Microscope. Sensors, 20.","DOI":"10.3390\/s20102805"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"510","DOI":"10.1080\/10447318.2016.1275435","article-title":"Human robot engagement and acceptability in residential aged care","volume":"33","author":"Khosla","year":"2017","journal-title":"Hum. Comput. Interact."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1109\/MRA.2010.936947","article-title":"Learning and reproduction of gestures by imitation","volume":"17","author":"Calinon","year":"2010","journal-title":"IEEE. Robot. Autom. Mag."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1022","DOI":"10.1007\/s12555-014-0495-4","article-title":"Smooth trajectory planning for a parallel manipulator with joint friction and jerk constraints","volume":"14","author":"Liu","year":"2016","journal-title":"Int. J. Control. Autom. Syst."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Laursen, J.S., Schultz, U.P., and Ellekilde, L.P. (October, January 28). Automatic error recovery in robot assembly operations using reverse execution. Proceedings of the 2015 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany.","DOI":"10.1109\/IROS.2015.7353609"},{"key":"ref_42","unstructured":"Rana, M.A., Li, A., Ravichandar, H., Mukadam, M., Chernova, S., Fox, D., Boots, B., and Ratliff, N. (2020, January 16\u201318). Learning reactive motion policies in multiple task spaces from human demonstrations. Proceedings of the Conference on Robot Learning, Cambridge, MA, USA."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Cho, N.J., Lee, S.H., Kim, J.B., and Suh, I.H. (2020). Learning, Improving, and Generalizing Motor Skills for the Peg-in-Hole Tasks Based on Imitation Learning and Self-Learning. Appl. Sci., 10.","DOI":"10.3390\/app10082719"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/19\/5505\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:13:41Z","timestamp":1760177621000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/19\/5505"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,25]]},"references-count":43,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2020,10]]}},"alternative-id":["s20195505"],"URL":"https:\/\/doi.org\/10.3390\/s20195505","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,9,25]]}}}