{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T20:15:29Z","timestamp":1790626529053,"version":"4.1.0"},"reference-count":26,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2025,1,23]],"date-time":"2025-01-23T00:00:00Z","timestamp":1737590400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China (NSFC)","doi-asserted-by":"publisher","award":["52025054"],"award-info":[{"award-number":["52025054"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China (NSFC)","doi-asserted-by":"publisher","award":["52105016"],"award-info":[{"award-number":["52105016"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Biosensors"],"abstract":"<jats:p>Supernumerary Robotic Limbs (SRLs) are designed to collaborate with the wearer, enhancing operational capabilities. When human limbs are occupied with primary tasks, controlling SRLs flexibly and naturally becomes a challenge. Existing methods such as electromyography (EMG) control and redundant limb control partially address SRL control issues. However, they still face limitations like restricted degrees of freedom and complex data requirements, which hinder their applicability in real-world scenarios. Additionally, fully autonomous control methods, while efficient, often lack the flexibility needed for complex tasks, as they do not allow for real-time user adjustments. In contrast, shared control combines machine autonomy with human input, enabling finer control and more intuitive task completion. Building on our previous work with the mouth-and-tongue interface, this paper integrates a mixed reality (MR) device to form an interactive system that enables shared control of the SRL. The system allows users to dynamically switch between voluntary and autonomous control, providing both flexibility and efficiency. A random forest model classifies 14 distinct tongue and mouth operations, mapping them to six-degree-of-freedom SRL control. In comparative experiments involving ten healthy subjects performing assembly tasks under three control modes (shared control, autonomous control, and voluntary control), shared control demonstrates a balance between machine autonomy and human input. While autonomous control offers higher task efficiency, shared control achieves greater task success rates and improves user experience by combining the advantages of both autonomous operation and voluntary control. This study validates the feasibility of shared control and highlights its advantages in providing flexible switching between autonomy and user intervention, offering new insights into SRL control.<\/jats:p>","DOI":"10.3390\/bios15020070","type":"journal-article","created":{"date-parts":[[2025,1,23]],"date-time":"2025-01-23T09:01:13Z","timestamp":1737622873000},"page":"70","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Shared Control of Supernumerary Robotic Limbs Using Mixed Realityand Mouth-and-Tongue Interfaces"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9842-472X","authenticated-orcid":false,"given":"Hongwei","family":"Jing","sequence":"first","affiliation":[{"name":"State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin 150006, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sikai","family":"Zhao","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin 150006, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tianjiao","family":"Zheng","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin 150006, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lele","family":"Li","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin 150006, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qinghua","family":"Zhang","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin 150006, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kerui","family":"Sun","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin 150006, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jie","family":"Zhao","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin 150006, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1960-6278","authenticated-orcid":false,"given":"Yanhe","family":"Zhu","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin 150006, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,1,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"929","DOI":"10.1109\/JAS.2021.1003961","article-title":"Review of research and development of supernumerary robotic limbs","volume":"8","author":"Tong","year":"2021","journal-title":"IEEE\/CAA J. Autom. Sin."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1345","DOI":"10.1038\/s41467-022-28725-7","article-title":"Principles of human movement augmentation and the challenges in making it a reality","volume":"13","author":"Eden","year":"2022","journal-title":"Nat. Commun."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Llorens-Bonilla, B., Parietti, F., and Asada, H.H. (2012, January 7\u201312). Based Control of Supernumerary Robotic Limbs. Proceedings of the 2012 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Vilamoura-Algarve, Portugal.","DOI":"10.1109\/IROS.2012.6386055"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Bonilla, B.L., and Asada, H.H. (June, January 31). A Robot on the Shoulder: Coordinated Human-Wearable Robot Control Using Coloured Petri Nets and Partial Least Squares Predictions. Proceedings of the 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China.","DOI":"10.1109\/ICRA.2014.6906598"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Wu, F.Y., and Asada, H. (2014). Bio-Artificial Synergies for Grasp Posture Control of Supernumerary Robotic Fingers. Robotics: Science and Systems X, University of California.","DOI":"10.15607\/RSS.2014.X.027"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1391","DOI":"10.1177\/0278364917714062","article-title":"Empirical characterization of modular variable stiffness inflatable structures for supernumerary grasp-assist devices","volume":"36","author":"Tiziani","year":"2017","journal-title":"Int. J. Robot. Res."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Cunningham, J., Hapsari, A., Guilleminot, P., Shafti, A., and Faisal, A.A. (2018, January 26\u201329). The Supernumerary Robotic 3rd Thumb for Skilled Music Tasks. Proceedings of the 2018 7th IEEE International Conference on Biomedical Robotics and Biomechatronics (Biorob), Enschede, The Netherlands.","DOI":"10.1109\/BIOROB.2018.8487609"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2300245","DOI":"10.1002\/aisy.202300245","article-title":"Reconfigurable design and modeling of an underwater superlimb for diving assistance","volume":"5","author":"Huo","year":"2023","journal-title":"Adv. Intell. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"eabo2418","DOI":"10.1126\/sciadv.abo2418","article-title":"A wearable textile-based pneumatic energy harvesting system for assistive robotics","volume":"8","author":"Shveda","year":"2022","journal-title":"Sci. Adv."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Sasaki, T., Saraiji, M.Y., Fernando, C.L., Minamizawa, K., and Inami, M. (2017). MetaLimbs: Multiple Arms Interaction Metamorphism. ACM SIGGRAPH 2017 Emerging Technologies, Association for Computing Machinery.","DOI":"10.1145\/3084822.3084837"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Shimobayashi, H., Sasaki, T., Horie, A., Arakawa, R., Kashino, Z., and Inami, M. (2021, January 22\u201324). Independent Control of Supernumerary Appendages Exploiting Upper Limb Redundancy. Proceedings of the Augmented Humans International Conference 2021, Rovaniemi, Finland.","DOI":"10.1145\/3458709.3458980"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"4505","DOI":"10.1109\/LRA.2023.3285474","article-title":"Task-based human-robot collaboration control of supernumerary robotic limbs for overhead tasks","volume":"8","author":"Tu","year":"2023","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Fan, Z., Lin, C., and Fu, C. (2020, January 26\u201329). A Gaze Signal Based Control Method for Supernumerary Robotic Limbs. Proceedings of the 2020 3rd International Conference on Control and Robots (ICCR), Tokyo, Japan.","DOI":"10.1109\/ICCR51572.2020.9344272"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.robot.2017.03.015","article-title":"A soft supernumerary robotic finger and mobile arm support for grasping compensation and hemiparetic upper limb rehabilitation","volume":"93","author":"Hussain","year":"2017","journal-title":"Robot. Auton. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Parietti, F., and Asada, H.H. (June, January 29). Independent, Voluntary Control of Extra Robotic Limbs. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989702"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1298","DOI":"10.1109\/TNSRE.2022.3172974","article-title":"Wearable supernumerary robotic limb system using a hybrid control approach based on motor imagery and object detection","volume":"30","author":"Tang","year":"2022","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"eaat1228","DOI":"10.1126\/scirobotics.aat1228","article-title":"BMI control of a third arm for multitasking","volume":"3","author":"Penaloza","year":"2018","journal-title":"Sci. Robot."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1646","DOI":"10.1109\/LRA.2021.3058926","article-title":"Integrated voluntary-reactive control of a human-superlimb hybrid system for hemiplegic patient support","volume":"6","author":"Song","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"2177","DOI":"10.1109\/LRA.2020.2970948","article-title":"Leveraging the human operator in the design and control of supernumerary robotic limbs","volume":"5","author":"Guggenheim","year":"2020","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"4457","DOI":"10.1109\/LRA.2023.3284363","article-title":"Voluntary-Redundant Hybrid Control of SuperLimb based on Redundant Muscle for On-Site Assembly Tasks","volume":"8","author":"Lin","year":"2023","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1109\/OJEMB.2021.3058036","article-title":"Semi-automated control system for reaching movements in EMG shoulder disarticulation prosthesis based on mixed reality device","volume":"2","author":"Togo","year":"2021","journal-title":"IEEE Open J. Eng. Med. Biol."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Jing, H., Zheng, T., Zhang, Q., Sun, K., Li, L., Lai, M., Zhao, J., and Zhu, Y. (2023). Human Operation Augmentation through Wearable Robotic Limb Integrated with Mixed Reality Device. Biomimetics, 8.","DOI":"10.3390\/biomimetics8060479"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Jing, H., Zheng, T., Zhang, Q., Liu, B., Sun, K., Li, L., Zhao, J., and Zhu, Y. (2024). A Mouth and Tongue Interactive Device to Control Wearable Robotic Limbs in Tasks where Human Limbs Are Occupied. Biosensors, 14.","DOI":"10.3390\/bios14050213"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"7407","DOI":"10.1109\/LRA.2022.3183251","article-title":"Rtsras: A series-parallel-reconfigurable tendon-driven supernumerary robotic arms","volume":"7","author":"Zhang","year":"2022","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_25","unstructured":"Ungureanu, D., Bogo, F., Galliani, S., Sama, P., Duan, X., Meekhof, C., St\u00fchmer, J., Cashman, T.J., Tekin, B., and Sch\u00f6nberger, J.L. (2020). Hololens 2 research mode as a tool for computer vision research. arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1080\/01431160412331269698","article-title":"Random forest classifier for remote sensing classification","volume":"26","author":"Pal","year":"2005","journal-title":"Int. J. Remote Sens."}],"container-title":["Biosensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2079-6374\/15\/2\/70\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,8]],"date-time":"2025-10-08T10:34:43Z","timestamp":1759919683000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2079-6374\/15\/2\/70"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,23]]},"references-count":26,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,2]]}},"alternative-id":["bios15020070"],"URL":"https:\/\/doi.org\/10.3390\/bios15020070","relation":{},"ISSN":["2079-6374"],"issn-type":[{"value":"2079-6374","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,23]]}}}