{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:56:25Z","timestamp":1760144185246,"version":"build-2065373602"},"reference-count":47,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2024,3,28]],"date-time":"2024-03-28T00:00:00Z","timestamp":1711584000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Key Research and Development Program of China","award":["2022YFB4700501","2022YFB2302404-02"],"award-info":[{"award-number":["2022YFB4700501","2022YFB2302404-02"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>Although robots have been widely used in a variety of fields, the idea of enabling them to perform multiple tasks in the same way that humans do remains a difficulty. To solve this, we investigate the learning from demonstration (LFD) system with our independently designed symmetrical humanoid dual-arm robot. We present a novel action feature matching algorithm. This algorithm accurately transforms human demonstration data into task models that robots can directly execute, considerably improving LFD\u2019s generalization capabilities. In our studies, we used motion capture cameras to capture human demonstration actions, which included combinations of simple actions (the action layer) and a succession of complicated operational tasks (the task layer). For the action layer data, we employed Gaussian mixture models (GMM) for processing and constructing an action primitive library. As for the task layer data, we created a \u201ckeyframe\u201d segmentation method to transform this data into a series of action primitives and build another action primitive library. Guided by our algorithm, the robot successfully imitated complex human tasks. Results show its excellent task learning and execution, providing an effective solution for robots to learn from human demonstrations and significantly advancing robot technology.<\/jats:p>","DOI":"10.3390\/sym16040396","type":"journal-article","created":{"date-parts":[[2024,3,28]],"date-time":"2024-03-28T12:52:12Z","timestamp":1711630332000},"page":"396","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Research on LFD System of Humanoid Dual-Arm Robot"],"prefix":"10.3390","volume":"16","author":[{"given":"Ze","family":"Cui","sequence":"first","affiliation":[{"name":"School of Mechatronic Engineering and Automation, Shanghai University, Shanghai 200444, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lang","family":"Kou","sequence":"additional","affiliation":[{"name":"School of Mechatronic Engineering and Automation, Shanghai University, Shanghai 200444, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zenghao","family":"Chen","sequence":"additional","affiliation":[{"name":"Shanghai Aerospace Control Technology Institute, Shanghai 201109, China"},{"name":"Shanghai Engineering Research Center of Inertial, Shanghai 201109, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Peng","family":"Bao","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Shield Machine and Boring Technology, Zhengzhou 450001, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Donghai","family":"Qian","sequence":"additional","affiliation":[{"name":"School of Mechatronic Engineering and Automation, Shanghai University, Shanghai 200444, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lang","family":"Xie","sequence":"additional","affiliation":[{"name":"School of Mechatronic Engineering and Automation, Shanghai University, Shanghai 200444, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yue","family":"Tang","sequence":"additional","affiliation":[{"name":"School of Mechatronic Engineering and Automation, Shanghai University, Shanghai 200444, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,3,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"3523","DOI":"10.1109\/LRA.2022.3145088","article-title":"Learning with Dual Demonstration Domains: Random Domain-Adaptive Meta-Learning","volume":"7","author":"Hu","year":"2022","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"105618","DOI":"10.1016\/j.engappai.2022.105618","article-title":"Adaptive and intelligent robot task planning for home service: A review","volume":"117","author":"Li","year":"2023","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"11910","DOI":"10.1109\/LRA.2022.3207558","article-title":"Learning from Demonstrations Via Multi-Level and Multi-Attention Domain-Adaptive Me-ta-Learning","volume":"7","author":"Hu","year":"2022","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"469","DOI":"10.1016\/j.robot.2008.10.024","article-title":"A survey of robot learning from demonstration","volume":"57","author":"Argall","year":"2009","journal-title":"Robot. Auton. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1146\/annurev-control-100819-063206","article-title":"Recent Advances in Robot Learning from Demonstration","volume":"3","author":"Ravichandar","year":"2020","journal-title":"Annu. Rev. Control. Robot. Auton. Syst."},{"key":"ref_6","unstructured":"Jang, E., Irpan, A., Khansari, M., Kappler, D., Ebert, F., Lynch, C., Levine, S., and Finn, C. (2022, January 14\u201318). Bc-z: Zero-shot task generalization with robotic imitation learning. Proceedings of the Conference on Robot Learning, Auckland, New Zealand."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Barros, J.J.O., dos Santos, V.M.F., and da Silva, F.M.T.P. (2015, January 8\u201310). Bimanual haptics for humanoid robot teleoperation using ROS and V-REP. Proceedings of the 2015 IEEE International Conference on Autonomous Robot Systems and Competitions, Vila Real, Portugal.","DOI":"10.1109\/ICARSC.2015.27"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Cui, Z., Li, K., Chen, Z., Bao, P., Kou, L., Xie, L., Tang, Y., and Zhu, D. (2022, January 5\u20139). Research on Learning from Demonstration System of Manipulator Based on the Improved Soft Actor-Critic Algorithm. Proceedings of the 2022 IEEE International Conference on Robotics and Biomimetics (ROBIO), Jinghong, China.","DOI":"10.1109\/ROBIO55434.2022.10011646"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Mueller, C.L., and Hayes, B. (2020, January 23\u201326). Safe and robust robot learning from demonstration through conceptual constraints. Proceedings of the Companion of the 2020 ACM\/IEEE International Conference on Human-Robot Interaction, Cambridge, UK.","DOI":"10.1145\/3371382.3377428"},{"key":"ref_10","unstructured":"Illandara, T. (2022). Active Keyframe Learning (AKL): Learning Interaction and Constraint Keyframes from a Single Demonstration of a Task. [Ph.D. Thesis, Massachusetts Institute of Technology]."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1517","DOI":"10.1016\/j.robot.2014.05.010","article-title":"A morphable template framework for robot learning by demonstration: Integrating one-shot and in-cremental learning approaches","volume":"62","author":"Wu","year":"2014","journal-title":"Robot. Auton. Syst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"260","DOI":"10.1016\/j.neucom.2019.04.100","article-title":"A robot learning framework based on adaptive admittance control and generalizable motion mod-eling with neural network controller","volume":"390","author":"Wang","year":"2020","journal-title":"Neurocomputing"},{"key":"ref_13","unstructured":"Fitzgerald, T., Goel, A.K., and Thomaz, A.L. (2014, January 13\u201315). Representing skill demonstrations for adaptation and transfer. Proceedings of the 2014 AAAI Fall Sym-posium Series, Arlington, VA, USA."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Chen, Y., Paulius, D., Sun, Y., and Jia, Y. (2022, January 20\u201324). Robot Learning of Assembly Tasks from Non-expert Demonstrations using Functional Ob-ject-Oriented Network. Proceedings of the 2022 IEEE 18th International Conference on Automation Science and Engineering (CASE), Mexico City, Mexico.","DOI":"10.1109\/CASE49997.2022.9926527"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"266","DOI":"10.1109\/TFUZZ.2023.3297665","article-title":"A Framework of Robot Manipulability Learning and Control and Its Application in Telerobotics","volume":"32","author":"Sun","year":"2023","journal-title":"IEEE Trans. Fuzzy Syst."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"3259","DOI":"10.1109\/TRO.2023.3258669","article-title":"Robot Programming by Demonstration: Trajectory Learning Enhanced by sEMG-Based User Hand Stiffness Estimation","volume":"39","author":"Biagiotti","year":"2023","journal-title":"IEEE Trans. Robot."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"23441","DOI":"10.1007\/s00521-021-06449-x","article-title":"Human\u2013robot skill transmission for mobile robot via learning by demonstration","volume":"35","author":"Li","year":"2023","journal-title":"Neural Com-Puting Appl."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Mahalingam, D., and Chakraborty, N. (June, January 29). Human-guided planning for complex manipulation tasks using the screw geometry of mo-tion. Proceedings of the 2023 IEEE International Conference on Robotics and Automation (ICRA), London, UK.","DOI":"10.1109\/ICRA48891.2023.10161130"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"103036","DOI":"10.1016\/j.mechatronics.2023.103036","article-title":"Impedance adjustment for learning human variable impedance skill","volume":"95","author":"Wu","year":"2023","journal-title":"Mechatronics"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Evrard, P., Gribovskaya, E., Calinon, S., Billard, A., and Kheddar, A. (2009, January 7\u201310). Teaching physical collaborative tasks: Object-lifting case study with a humanoid. Proceedings of the 2009 9th IEEE-RAS International Conference on Humanoid Robots, Paris, France.","DOI":"10.1109\/ICHR.2009.5379513"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Sirintuna, D., Giammarino, A., and Ajoudani, A. (2023). An object deformation-agnostic framework for human\u2013robot collaborative transportation. IEEE Trans. Autom. Sci. Eng.","DOI":"10.1109\/TASE.2023.3259162"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"441","DOI":"10.1007\/s43154-021-00068-z","article-title":"Human-Humanoid Interaction and Cooperation: A Review","volume":"2","author":"Vianello","year":"2021","journal-title":"Curr. Robot. Rep."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"103515","DOI":"10.1016\/j.robot.2020.103515","article-title":"Skill transfer learning for autonomous robots and human\u2013robot cooperation: A survey","volume":"128","author":"Liu","year":"2020","journal-title":"Robot. Auton. Syst."},{"key":"ref_24","first-page":"V002T02A025","article-title":"Human grasping force prediction, measurement, and validation for human-robot lifting","volume":"Volume 86212","author":"Arefeen","year":"2022","journal-title":"Proceedings of the International Design Engineering Technical Conferences and Computers and Information in Engineering Conference"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"286","DOI":"10.1109\/TSMCB.2006.886952","article-title":"On Learning, Representing, and Generalizing a Task in a Humanoid Robot","volume":"37","author":"Calinon","year":"2007","journal-title":"IEEE Trans. Syst. Man Cybern. Part B (Cybernetics)"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Hua, J., Zeng, L., Li, G., and Ju, Z. (2021). Learning for a Robot: Deep Reinforcement Learning, Imitation Learning, Transfer Learning. Sensors, 21.","DOI":"10.3390\/s21041278"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Grimes, D.B., Chalodhorn, R., and Rao, R.P.N. (2006, January 16\u201319). Dynamic imitation in a humanoid robot through nonparametric probabilistic in-ference. Proceedings of the Robotics: Science and Systems, Philadelphia, PA, USA.","DOI":"10.15607\/RSS.2006.II.026"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Zhao, D., Song, S., Su, J., Jiang, Z., and Zhang, J. (2020, January 13\u201316). Learning bionic motions by imitating animals. Proceedings of the 2020 IEEE International Conference on Mechatronics and Automation (ICMA), Beijing, China.","DOI":"10.1109\/ICMA49215.2020.9233839"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1109\/TCDS.2023.3296166","article-title":"Learning Skills from Demonstrations: A Trend from Motion Primitives to Experience Abstraction","volume":"16","author":"Tavassoli","year":"2023","journal-title":"IEEE Trans. Cogn. Dev. Syst."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Franzese, G., Rosa, L.d.S., Verburg, T., Peternel, L., and Kober, J. (2023). Interactive Imitation Learning of Bimanual Movement Primitives. IEEE\/ASME Trans. Mechatron., 1\u201313.","DOI":"10.1109\/TMECH.2023.3295249"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Schneider, M., and Ertel, W. (2010, January 18\u201322). Robot learning by demonstration with local gaussian process regression. Proceedings of the 2010 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Taipei, Taiwan.","DOI":"10.1109\/IROS.2010.5650949"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2379","DOI":"10.1109\/TRO.2023.3238910","article-title":"Hybrid Robotic Grasping with a Soft Multimodal Gripper and a Deep Multistage Learning Scheme","volume":"39","author":"Liu","year":"2023","journal-title":"IEEE Trans. Robot."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1007\/s12555-020-0197-z","article-title":"Deep Robotic Grasping Prediction with Hierarchical RGB-D Fusion","volume":"20","author":"Song","year":"2022","journal-title":"Int. J. Control. Autom. Syst."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"209","DOI":"10.3390\/ai2020013","article-title":"Vision-Guided Hand\u2013Eye Coordination for Robotic Grasping and Its Application in Tangram Puzzles","volume":"2","author":"Wei","year":"2021","journal-title":"AI"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1109\/LRA.2020.3044029","article-title":"Task-Oriented Motion Mapping on Robots of Various Configuration Using Body Role Division","volume":"6","author":"Sasabuchi","year":"2020","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"3827","DOI":"10.1109\/LRA.2022.3148436","article-title":"Gaussian Process Constraint Learning for Scalable Chance-Constrained Motion Planning from Demonstrations","volume":"7","author":"Chou","year":"2022","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Lobbezoo, A., and Kwon, H.J. (2023). Simulated and Real Robotic Reach, Grasp, and Pick-and-Place Using Combined Reinforcement Learning and Traditional Controls. Robotics, 12.","DOI":"10.3390\/robotics12010012"},{"key":"ref_38","unstructured":"Radosavovic, I., Xiao, T., James, S., Abbeel, P., Malik, J., and Darrell, T. (2022, January 14\u201318). Real-world robot learning with masked visual pre-training. Proceedings of the Conference on Robot Learning, Auckland, New Zealand."},{"key":"ref_39","unstructured":"Chebotar, Y., Vuong, Q., Hausman, K., Xia, F., Lu, Y., Irpan, A., Kumar, A., Yu, T., Herzog, A., and Pertsch, K. (2023, January 6\u20139). Q-transformer: Scalable offline reinforcement learning via autoregressive q-functions. Proceedings of the Conference on Robot Learning, Atlanta, GA, USA."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Kehoe, B., Matsukawa, A., Candido, S., Kuffner, J., and Goldberg, K. (2013, January 17). Cloud-based robot grasping with the google object recognition engine. Proceedings of the 2013 IEEE International Conference on Robotics and Automation, Karlsruhe, Germany.","DOI":"10.1109\/ICRA.2013.6631180"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"P\u00e9rez-D\u2019Arpino, C., and Shah, J.A. (June, January 29). C-learn: Learning geometric constraints from demonstrations for multi-step manipulation in shared autonomy. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989466"},{"key":"ref_42","unstructured":"Ebert, F., Finn, C., Dasari, S., Xie, A., Lee, A., and Levine, S. (2018). Visual foresight: Model-based deep reinforcement learning for vision-based robotic control. arXiv, 6\u201325."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1049\/ccs2.12005","article-title":"A review on manipulation skill acquisition through teleoperation-based learning from demonstration","volume":"3","author":"Si","year":"2021","journal-title":"Cogn. Comput. Syst."},{"key":"ref_44","unstructured":"Cohen, P.R., Chang, Y.H., Morrison, C.T., and Beal, C.R. (2007, January 6\u201312). Learning and Transferring Action Schemas. Proceedings of the IJCAI 2007, Hyderabad, India."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"513","DOI":"10.1109\/TRO.2016.2540623","article-title":"Learning Physical Collaborative Robot Behaviors from Human Demonstrations","volume":"32","author":"Rozo","year":"2016","journal-title":"IEEE Trans. Robot."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Castro, A., Silva, F., and Santos, V. (2021). Trends of Human-Robot Collaboration in Industry Contexts: Handover, Learning, and Metrics. Sensors, 21.","DOI":"10.3390\/s21124113"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"483","DOI":"10.1111\/j.0006-341X.2000.00483.x","article-title":"Mixture Models, Robustness, and the Weighted Likelihood Methodology","volume":"56","author":"Markatou","year":"2000","journal-title":"Biometrics"}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/16\/4\/396\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:20:12Z","timestamp":1760106012000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/16\/4\/396"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,28]]},"references-count":47,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2024,4]]}},"alternative-id":["sym16040396"],"URL":"https:\/\/doi.org\/10.3390\/sym16040396","relation":{},"ISSN":["2073-8994"],"issn-type":[{"type":"electronic","value":"2073-8994"}],"subject":[],"published":{"date-parts":[[2024,3,28]]}}}