{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T04:35:55Z","timestamp":1781584555370,"version":"3.54.5"},"reference-count":63,"publisher":"American Association for the Advancement of Science (AAAS)","issue":"85","content-domain":{"domain":["www.science.org"],"crossmark-restriction":true},"short-container-title":["Sci. Robot."],"published-print":{"date-parts":[[2023,12,20]]},"abstract":"<jats:p>Extra robotic arms (XRAs) are gaining interest in neuroscience and robotics, offering potential tools for daily activities. However, this compelling opportunity poses new challenges for sensorimotor control strategies and human-machine interfaces (HMIs). A key unsolved challenge is allowing users to proficiently control XRAs without hindering their existing functions. To address this, we propose a pipeline to identify suitable HMIs given a defined task to accomplish with the XRA. Following such a scheme, we assessed a multimodal motor HMI based on gaze detection and diaphragmatic respiration in a purposely designed modular neurorobotic platform integrating virtual reality and a bilateral upper limb exoskeleton. Our results show that the proposed HMI does not interfere with speaking or visual exploration and that it can be used to control an extra virtual arm independently from the biological ones or in coordination with them. Participants showed significant improvements in performance with daily training and retention of learning, with no further improvements when artificial haptic feedback was provided. As a final proof of concept, na\u00efve and experienced participants used a simplified version of the HMI to control a wearable XRA. Our analysis indicates how the presented HMI can be effectively used to control XRAs. The observation that experienced users achieved a success rate 22.2% higher than that of na\u00efve users, combined with the result that na\u00efve users showed average success rates of 74% when they first engaged with the system, endorses the viability of both the virtual reality\u2013based testing and training and the proposed pipeline.<\/jats:p>","DOI":"10.1126\/scirobotics.adh1438","type":"journal-article","created":{"date-parts":[[2023,12,13]],"date-time":"2023-12-13T18:55:19Z","timestamp":1702493719000},"update-policy":"https:\/\/doi.org\/10.34133\/aaas_crossmark","source":"Crossref","is-referenced-by-count":29,"title":["Human motor augmentation with an extra robotic arm without functional interference"],"prefix":"10.1126","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4271-1359","authenticated-orcid":true,"given":"Giulia","family":"Dominijanni","sequence":"first","affiliation":[{"name":"Neuro-X Institute, Ecole Polytechnique F\u00e9d\u00e9rale de Lausanne, Lausanne, Switzerland."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0078-993X","authenticated-orcid":true,"given":"Daniel Leal","family":"Pinheiro","sequence":"additional","affiliation":[{"name":"Neuro-X Institute, Ecole Polytechnique F\u00e9d\u00e9rale de Lausanne, Lausanne, Switzerland."},{"name":"Neuroengineering and Neurocognition Laboratory, Escola Paulista de Medicina, Department of Neurology and Neurosurgery, Division of Neuroscience, Universidade Federal de S\u00e3o Paulo, S\u00e3o Paulo, Brazil."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-8648-4091","authenticated-orcid":true,"given":"Leonardo","family":"Pollina","sequence":"additional","affiliation":[{"name":"Neuro-X Institute, Ecole Polytechnique F\u00e9d\u00e9rale de Lausanne, Lausanne, Switzerland."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0771-8408","authenticated-orcid":true,"given":"Bastien","family":"Orset","sequence":"additional","affiliation":[{"name":"Neuro-X Institute, Ecole Polytechnique F\u00e9d\u00e9rale de Lausanne, Lausanne, Switzerland."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Martina","family":"Gini","sequence":"additional","affiliation":[{"name":"BioRobotics Institute, Health Interdisciplinary Center, and Department of Excellence in AI and Robotics, Scuola Superiore Sant\u2019Anna, Pisa, Italy."},{"name":"Neuroelectronic Interfaces, Faculty of Electrical Engineering and IT, Rheinisch-Westf\u00e4lische Technische Hochschule (RWTH) Aachen, Aachen 52074, Germany."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6429-3848","authenticated-orcid":true,"given":"Eugenio","family":"Anselmino","sequence":"additional","affiliation":[{"name":"BioRobotics Institute, Health Interdisciplinary Center, and Department of Excellence in AI and Robotics, Scuola Superiore Sant\u2019Anna, Pisa, Italy."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Camilla","family":"Pierella","sequence":"additional","affiliation":[{"name":"Department of Neurosciences, Rehabilitation, Ophthalmology, Genetics, and Maternal and Children\u2019s Sciences (DINOGMI), University of Genoa, Genoa, Italy."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6026-381X","authenticated-orcid":true,"given":"J\u00e9r\u00e9my","family":"Olivier","sequence":"additional","affiliation":[{"name":"Institute for Industrial Sciences and Technologies, Haute Ecole du Paysage, d\u2019Ing\u00e9nierie et d\u2019Architecture (HEPIA), HES-SO University of Applied Sciences and Arts Western Switzerland, Geneva, Switzerland."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2251-7142","authenticated-orcid":true,"given":"Solaiman","family":"Shokur","sequence":"additional","affiliation":[{"name":"Neuro-X Institute, Ecole Polytechnique F\u00e9d\u00e9rale de Lausanne, Lausanne, Switzerland."},{"name":"BioRobotics Institute, Health Interdisciplinary Center, and Department of Excellence in AI and Robotics, Scuola Superiore Sant\u2019Anna, Pisa, Italy."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4396-8217","authenticated-orcid":true,"given":"Silvestro","family":"Micera","sequence":"additional","affiliation":[{"name":"Neuro-X Institute, Ecole Polytechnique F\u00e9d\u00e9rale de Lausanne, Lausanne, Switzerland."},{"name":"BioRobotics Institute, Health Interdisciplinary Center, and Department of Excellence in AI and Robotics, Scuola Superiore Sant\u2019Anna, Pisa, Italy."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"221","reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1038\/scientificamerican0960-62"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-021-00398-9"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1088\/2516-1091\/ac2294"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-022-28725-7"},{"key":"e_1_3_2_6_2","doi-asserted-by":"crossref","unstructured":"I. Hussain L. Meli C. Pacchierotti G. Salvietti D. Prattichizzo Vibrotactile haptic feedback for intuitive control of robotic extra fingers in 2015 IEEE World Haptics Conference (WHC) (IEEE 2015) pp. 394\u2013399.","DOI":"10.1109\/WHC.2015.7177744"},{"key":"e_1_3_2_7_2","doi-asserted-by":"crossref","unstructured":"T. Lisini Baldi N. D\u2019Aurizio C. Gaudeni S. Gurgone D. Borzelli A. D\u2019Avella D. Prattichizzo Exploiting intrinsic kinematic null space for supernumerary robotic limbs control in 2023 IEEE International Conference on Robotics and Automation (ICRA) (IEEE 2023) pp. 11957\u201311963.","DOI":"10.1109\/ICRA48891.2023.10160964"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.abd7935"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-021-00376-6"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2016.2530793"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNSRE.2016.2529684"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1080\/01691864.2019.1566090"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0134501"},{"key":"e_1_3_2_14_2","doi-asserted-by":"crossref","unstructured":"M. Y. Saraiji T. Sasaki K. Minamizawa M. Inami MetaArms: Body remapping using feet-controlled artificial arms in Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology (ACM 2018) pp. 140\u2013142.","DOI":"10.1145\/3242587.3242665"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1177\/02783649211017642"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-021-88862-9"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-022-13981-w"},{"key":"e_1_3_2_18_2","first-page":"1493","article-title":"Human performance of three hands in unimanual, bimanual and trimanual tasks","author":"Huang Y.","year":"2022","unstructured":"Y. Huang, J. Eden, E. Ivanova, E. Burdet, Human performance of three hands in unimanual, bimanual and trimanual tasks. Annu Int. Conf. IEEE Eng. Med. Biol. Soc. EMBC, 1493\u20131497 (2022).","journal-title":"Annu Int. Conf. IEEE Eng. Med. Biol. Soc. EMBC"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1177\/02783649231180366"},{"key":"e_1_3_2_20_2","doi-asserted-by":"crossref","unstructured":"F. Parietti H. H. Asada Independent voluntary control of extra robotic limbs in 2017 IEEE International Conference on Robotics and Automation (IEEE 2017) pp. 5954\u20135961.","DOI":"10.1109\/ICRA.2017.7989702"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.aat1228"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2020.2970948"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1088\/1741-2552\/aadf3d"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1088\/1741-2552\/ac47db"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1088\/1741-2552\/abcdbf"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.3389\/fnbot.2023.1154427"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1046\/j.1460-9568.2000.00298.x"},{"key":"e_1_3_2_28_2","first-page":"385","article-title":"Predicting targets of human reaching motions with an arm rehabilitation exoskeleton","volume":"51","author":"Novak V.","year":"2015","unstructured":"V. Novak, R. Riener, Predicting targets of human reaching motions with an arm rehabilitation exoskeleton. Biomed. Sci. Instrum. 51, 385\u2013392 (2015).","journal-title":"Biomed. Sci. Instrum."},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCC.2012.2226444"},{"key":"e_1_3_2_30_2","doi-asserted-by":"crossref","unstructured":"Z. Fan C. Lin C. Fu A gaze signal based control method for supernumerary robotic limbs in 2020 3rd International Conference on Control and Robots (IEEE 2020) pp. 107\u2013111.","DOI":"10.1109\/ICCR51572.2020.9344272"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1113\/jphysiol.1987.sp016445"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.3390\/s20185446"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1088\/0967-3334\/34\/1\/35"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1186\/1743-0003-10-4"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-019-13967-9"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2022.119685"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1152\/jn.00617.2019"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.resp.2014.08.003"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1115\/1.1526508"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNSRE.2015.2500586"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1080\/17434440.2017.1332989"},{"key":"e_1_3_2_42_2","unstructured":"M. Bergamasco F. Salsedo B. Lenzo An exoskeleton structure for physical interaction with a human being (2013); https:\/\/patents.google.com\/patent\/WO2013186705A3\/en."},{"key":"e_1_3_2_43_2","doi-asserted-by":"crossref","unstructured":"J. K. Kruschke Doing Bayesian Data Analysis: A Tutorial with R JAGS and Stan (Academic Press ed. 2 2014).","DOI":"10.1016\/B978-0-12-405888-0.00008-8"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1097\/SIH.0b013e3181e3f329"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1037\/0022-0663.100.1.223"},{"key":"e_1_3_2_46_2","doi-asserted-by":"crossref","unstructured":"S. Kobori Y. Abe Pupillary responses during learning of inverted tracking tasks in World Congress on Medical Physics and Biomedical Engineering IFMBE Proceedings (Springer 2009) pp. 211\u2013214.","DOI":"10.1007\/978-3-642-03889-1_57"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/8466.003.0067"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.0800-05.2005"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.abf3368"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-022-06040-x"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-control-071020-104336"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41551-020-00630-8"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.3758\/s13423-012-0333-8"},{"key":"e_1_3_2_54_2","first-page":"392","article-title":"Visually triggered eye-arm movements in man","volume":"42","author":"Herman R.","year":"1981","unstructured":"R. Herman, R. Herman, R. Maulucci, Visually triggered eye-arm movements in man. Exp. Brain Res. 42, 392\u2013398 (1981).","journal-title":"Exp. Brain Res."},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.21-17-06917.2001"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1152\/jn.00117.2004"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-019-0093-5"},{"key":"e_1_3_2_58_2","doi-asserted-by":"crossref","unstructured":"J. L. Pons Wearable Robots: Biomechatronic Exoskeletons (John Wiley & Sons 2008).","DOI":"10.1002\/9780470987667"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1109\/TOH.2020.3017548"},{"key":"e_1_3_2_60_2","doi-asserted-by":"crossref","unstructured":"F. Clemente C. Cipriani A novel device for multi-modal sensory feedback in hand prosthetics: Design and preliminary prototype in 2014 IEEE Haptics Symposium (IEEE 2014) pp. 569\u2013573 (2014).","DOI":"10.1109\/HAPTICS.2014.6775518"},{"key":"e_1_3_2_61_2","doi-asserted-by":"crossref","unstructured":"H. Huang T. Li C. Bruschini C. Enz J. Justiz C. Antfolk V. M. Koch Multi-modal sensory feedback system for upper limb amputees in 2017 New Generation of CAS (NGCAS) (IEEE 2017) pp. 193\u2013196.","DOI":"10.1109\/NGCAS.2017.62"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1109\/TOH.2023.3272698"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0166-4115(08)62386-9"},{"key":"e_1_3_2_64_2","doi-asserted-by":"crossref","unstructured":"S. Khorram M. G. McInnis E. M. Provost Trainable time warping: Aligning time-series in the continuous-time domain in IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP) (IEEE 2019) pp. 3502\u20133506.","DOI":"10.1109\/ICASSP.2019.8682322"}],"container-title":["Science Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.science.org\/doi\/pdf\/10.1126\/scirobotics.adh1438","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,1,16]],"date-time":"2024-01-16T12:02:07Z","timestamp":1705406527000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.science.org\/doi\/10.1126\/scirobotics.adh1438"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,20]]},"references-count":63,"journal-issue":{"issue":"85","published-print":{"date-parts":[[2023,12,20]]}},"alternative-id":["10.1126\/scirobotics.adh1438"],"URL":"https:\/\/doi.org\/10.1126\/scirobotics.adh1438","relation":{},"ISSN":["2470-9476"],"issn-type":[{"value":"2470-9476","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,20]]},"assertion":[{"value":"2023-02-13","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-11-15","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-12-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"eadh1438"}}