{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,11]],"date-time":"2026-04-11T07:24:23Z","timestamp":1775892263092,"version":"3.50.1"},"reference-count":46,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,7,17]],"date-time":"2025-07-17T00:00:00Z","timestamp":1752710400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Robot. AI"],"abstract":"<jats:p>Bimanual teleoperation imposes cognitive and coordination demands on a single human operator tasked with simultaneously controlling two robotic arms. Although assigning each arm to a separate operator can distribute workload, it often leads to ambiguities in decision authority and degrades overall efficiency. To overcome these challenges, we propose a novel bimanual teleoperation large language model assistant (BTLA) framework, an intelligent co-pilot that augments a single operator\u2019s motor control capabilities. In particular, BTLA enables operators to directly control one robotic arm through conventional teleoperation while directing a second assistive arm via simple voice commands, and therefore commanding two robotic arms simultaneously. By integrating the GPT-3.5-turbo model, BTLA interprets contextual voice instructions and autonomously selects among six predefined manipulation skills, including real-time mirroring, trajectory following, and autonomous object grasping. Experimental evaluations in bimanual object manipulation tasks demonstrate that BTLA increased task coverage by 76.1<jats:inline-formula><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\" id=\"m1\"><mml:mrow><mml:mi>%<\/mml:mi><\/mml:mrow><\/mml:math><\/jats:inline-formula> and success rate by 240.8<jats:inline-formula><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\" id=\"m2\"><mml:mrow><mml:mi>%<\/mml:mi><\/mml:mrow><\/mml:math><\/jats:inline-formula> relative to solo teleoperation, and outperformed dyadic control with a 19.4<jats:inline-formula><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\" id=\"m3\"><mml:mrow><mml:mi>%<\/mml:mi><\/mml:mrow><\/mml:math><\/jats:inline-formula> gain in coverage and a 69.9<jats:inline-formula><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\" id=\"m4\"><mml:mrow><mml:mi>%<\/mml:mi><\/mml:mrow><\/mml:math><\/jats:inline-formula> gain in success. Furthermore, NASA Task Load Index (NASA-TLX) assessments revealed a 38\u201352<jats:inline-formula><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\" id=\"m5\"><mml:mrow><mml:mi>%<\/mml:mi><\/mml:mrow><\/mml:math><\/jats:inline-formula> reduction in operator mental workload, and 85<jats:inline-formula><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\" id=\"m6\"><mml:mrow><mml:mi>%<\/mml:mi><\/mml:mrow><\/mml:math><\/jats:inline-formula> of participants rated the voice-based interaction as \u201cnatural\u201d and \u201chighly effective.\u201d<\/jats:p>","DOI":"10.3389\/frobt.2025.1621033","type":"journal-article","created":{"date-parts":[[2025,7,17]],"date-time":"2025-07-17T04:10:22Z","timestamp":1752725422000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Large language model-driven natural language interaction control framework for single-operator bimanual teleoperation"],"prefix":"10.3389","volume":"12","author":[{"given":"Haolin","family":"Fei","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tao","family":"Xue","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yiyang","family":"He","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sheng","family":"Lin","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Guanglong","family":"Du","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yao","family":"Guo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ziwei","family":"Wang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2025,7,17]]},"reference":[{"key":"B1","first-page":"9422","article-title":"Dynamic grasping with reachability and motion awareness","author":"Akinola","year":"2021"},{"key":"B2","doi-asserted-by":"publisher","first-page":"352","DOI":"10.1109\/tmrb.2022.3170786","article-title":"Deep reinforcement learning-based control framework for multilateral telesurgery","volume":"4","author":"Bacha","year":"2022","journal-title":"IEEE Trans. Med. Robotics Bionics"},{"key":"B3","doi-asserted-by":"publisher","first-page":"72","DOI":"10.1109\/tmrb.2022.3145673","article-title":"Anthropomorphic dual-arm coordinated control for a single-port surgical robot based on dual-step optimization","volume":"4","author":"Bai","year":"2022","journal-title":"IEEE Trans. Med. Robotics Bionics"},{"key":"B4","first-page":"902","article-title":"Dual-arm coordinated manipulation for object twisting with human intelligence","author":"Bai","year":"2021"},{"key":"B5","first-page":"2294","article-title":"Human-guided grasp measures improve grasp robustness on physical robot","author":"Balasubramanian","year":"2010"},{"key":"B6","first-page":"6248","article-title":"Online recognition of bimanual coordination provides important context for movement data in bimanual teleoperated robots","author":"Boehm","year":"2021"},{"key":"B7","doi-asserted-by":"publisher","first-page":"641","DOI":"10.1109\/tra.2002.802199","article-title":"Design of bilateral teleoperation controllers for haptic exploration and telemanipulation of soft environments","volume":"18","author":"Cavusoglu","year":"2002","journal-title":"IEEE Trans. Robotics Automation"},{"key":"B8","first-page":"1290","article-title":"Two is not always better than one: effects of teleoperation and haptic coupling","author":"Che","year":"2016"},{"key":"B9","first-page":"6844","article-title":"Foot gestures to control the grasping of a surgical robot","author":"Cheng","year":"2023"},{"key":"B10","first-page":"5187","article-title":"On the role of wearable haptics for force feedback in teleimpedance control for dual-arm robotic teleoperation","author":"Clark","year":"2019"},{"key":"B11","volume-title":"Human demonstrations are generalizable knowledge for robots","author":"Cui","year":"2024"},{"key":"B12","doi-asserted-by":"publisher","first-page":"4379","DOI":"10.3390\/app12094379","article-title":"Augmented reality-based interface for bimanual robot teleoperation","volume":"12","author":"Garc\u00eda","year":"2022","journal-title":"Appl. Sci."},{"key":"B13","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/srep03824","article-title":"Two is better than one: physical interactions improve motor performance in humans","volume":"4","author":"Gowrishankar","year":"2014","journal-title":"Sci. Rep."},{"key":"B14","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/thms.2021.3107519","article-title":"Eye-tracking for performance evaluation and workload estimation in space telerobotic training","volume":"52","author":"Guo","year":"2022","journal-title":"IEEE Trans. Human-Machine Syst."},{"key":"B15","doi-asserted-by":"publisher","first-page":"139","DOI":"10.1016\/s0166-4115(08)62386-9","article-title":"Development of nasa-tlx (task load index): results of empirical and theoretical research","volume":"52","author":"Hart","year":"1988","journal-title":"Adv. Psychol."},{"key":"B16","volume-title":"Look before you leap: unveiling the power of GPT-4v in robotic vision-language planning","author":"Hu","year":"2023"},{"key":"B17","doi-asserted-by":"publisher","first-page":"4553","DOI":"10.1109\/lra.2023.3285478","article-title":"Towards human-robot collaborative surgery: trajectory and strategy learning in bimanual peg transfer","volume":"8","author":"Hu","year":"2023","journal-title":"IEEE Robotics Automation Lett."},{"key":"B18","doi-asserted-by":"publisher","first-page":"13162","DOI":"10.1109\/tie.2021.3139228","article-title":"Cooperative manipulation of deformable objects by single-leader\u2013dual-follower teleoperation","volume":"69","author":"Huang","year":"2022","journal-title":"IEEE Trans. Industrial Electron."},{"key":"B19","doi-asserted-by":"publisher","first-page":"2282","DOI":"10.1007\/s10439-021-02766-3","article-title":"A three-limb teleoperated robotic system with foot control for flexible endoscopic surgery","volume":"49","author":"Huang","year":"2021","journal-title":"Ann. Biomed. Eng."},{"key":"B20","doi-asserted-by":"publisher","first-page":"8341","DOI":"10.3390\/s21248341","article-title":"A novel training and collaboration integrated framework for human\u2013agent teleoperation","volume":"21","author":"Huang","year":"2021","journal-title":"Sensors"},{"key":"B21","doi-asserted-by":"publisher","first-page":"8928","DOI":"10.3390\/app11198928","article-title":"Leader\u2013follower role allocation for physical collaboration in human dyads","volume":"11","author":"Kropiv\u0161ek Leskovar","year":"2021","journal-title":"Appl. Sci."},{"key":"B22","first-page":"1","article-title":"Shared-autonomy control for intuitive bimanual tele-manipulation","author":"Laghi","year":"2018"},{"key":"B23","doi-asserted-by":"publisher","first-page":"104402","DOI":"10.1016\/j.robot.2023.104402","article-title":"A telerobotic system enabling online switching among various architectures and controllers","volume":"166","author":"Li","year":"","journal-title":"Robotics Aut. Syst."},{"key":"B24","doi-asserted-by":"publisher","first-page":"86","DOI":"10.1109\/toh.2023.3239602","article-title":"A haptic shared autonomy with partial orientation regulation for dof deficiency in remote side","volume":"16","author":"Li","year":"","journal-title":"IEEE Trans. Haptics"},{"key":"B25","doi-asserted-by":"publisher","first-page":"118","DOI":"10.1109\/toh.2023.3253856","article-title":"The classification and new trends of shared control strategies in telerobotic systems: a survey","volume":"16","author":"Li","year":"","journal-title":"IEEE Trans. Haptics"},{"key":"B26","volume-title":"Growing from exploration: a self-exploring framework for robots based on foundation models","author":"Li","year":"2024"},{"key":"B27","doi-asserted-by":"publisher","first-page":"032004","DOI":"10.1088\/2516-1091\/ac8193","article-title":"A review on interaction control for contact robots through intent detection","volume":"4","author":"Li","year":"2022","journal-title":"Prog. Biomed. Eng."},{"key":"B28","volume-title":"Advancing object goal navigation through llm-enhanced object affinities transfer","author":"Lin","year":"2024"},{"key":"B29","doi-asserted-by":"publisher","first-page":"103973","DOI":"10.1016\/j.robot.2021.103973","article-title":"Teleoperation methods and enhancement techniques for mobile robots: a comprehensive survey","volume":"150","author":"Moniruzzaman","year":"2022","journal-title":"Robotics Aut. Syst."},{"key":"B30","doi-asserted-by":"publisher","first-page":"880","DOI":"10.1109\/tro.2016.2572698","article-title":"A model for human\u2013human collaborative object manipulation and its application to human\u2013robot interaction","volume":"32","author":"Noohi","year":"2016","journal-title":"IEEE Trans. Robotics"},{"key":"B31","doi-asserted-by":"publisher","first-page":"eaaw0955","DOI":"10.1126\/scirobotics.aaw0955","article-title":"Shared control\u2013based bimanual robot manipulation","volume":"4","author":"Rakita","year":"2019","journal-title":"Sci. Robotics"},{"key":"B32","first-page":"1234","article-title":"Haptic bimanual system for teleoperation of time-delayed tasks","author":"Sena","year":"2021"},{"key":"B33","doi-asserted-by":"publisher","first-page":"2174","DOI":"10.3390\/electronics9122174","article-title":"Research of hrv as a measure of mental workload in human and dual-arm robot interaction","volume":"9","author":"Shao","year":"2020","journal-title":"Electronics"},{"key":"B34","volume-title":"Yell at your robot: improving on-the-fly from language corrections","author":"Shi","year":"2024"},{"key":"B35","first-page":"11523","article-title":"ProgPrompt: generating situated robot task plans using large language models","author":"Singh","year":"2023"},{"key":"B36","doi-asserted-by":"publisher","first-page":"390","DOI":"10.1108\/ir-12-2015-0213","article-title":"Haptic industrial robot control with variable time delayed bilateral teleoperation","volume":"43","author":"Soyguder","year":"2016","journal-title":"Industrial Robot Int. J."},{"key":"B37","doi-asserted-by":"publisher","first-page":"1022","DOI":"10.1109\/tro.2020.2973099","article-title":"Single master bimanual teleoperation system with efficient regulation","volume":"36","author":"Sun","year":"2020","journal-title":"IEEE Trans. Robotics"},{"key":"B38","doi-asserted-by":"publisher","first-page":"2859","DOI":"10.1080\/00207721.2020.1869346","article-title":"Adaptive interval type-2 fuzzy control for multi-legged underwater robot with input saturation and full-state constraints","volume":"54","author":"Sun","year":"2023","journal-title":"Int. J. Syst. Sci."},{"key":"B39","doi-asserted-by":"publisher","first-page":"416","DOI":"10.1109\/tcds.2023.3272730","article-title":"Learning to assist bimanual teleoperation using interval type-2 polynomial fuzzy inference","volume":"16","author":"Wang","year":"2024","journal-title":"IEEE Trans. Cognitive Dev. Syst."},{"key":"B40","doi-asserted-by":"publisher","first-page":"2785","DOI":"10.1109\/tfuzz.2020.3007438","article-title":"Event-triggered prescribed-time fuzzy control for space teleoperation systems subject to multiple constraints and uncertainties","volume":"29","author":"Wang","year":"2021","journal-title":"IEEE Trans. Fuzzy Syst."},{"key":"B41","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.robot.2018.10.007","article-title":"Coordinated control of a dual-arm robot for surgical instrument sorting tasks","volume":"112","author":"Wu","year":"2019","journal-title":"Robotics Aut. Syst."},{"key":"B42","volume-title":"Distilling and retrieving generalizable knowledge for robot manipulation via language corrections","author":"Zha","year":"2023"},{"key":"B43","doi-asserted-by":"publisher","first-page":"75","DOI":"10.1016\/j.actaastro.2017.01.041","article-title":"Coordinated stabilization for space robot after capturing a noncooperative target with large inertia","volume":"134","author":"Zhang","year":"2017","journal-title":"Acta Astronaut."},{"key":"B44","first-page":"15195","article-title":"A user-centered shared control scheme with learning from demonstration for robotic surgery","author":"Zheng","year":"2024"},{"key":"B45","volume-title":"Planagent: a multi-modal large language agent for closed-loop vehicle motion planning","author":"Zheng","year":"2024"},{"key":"B46","first-page":"495","article-title":"A bilateral dual-arm teleoperation robot system with a unified control architecture","author":"Zhou","year":"2021"}],"container-title":["Frontiers in Robotics and AI"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2025.1621033\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,17]],"date-time":"2025-07-17T04:10:23Z","timestamp":1752725423000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2025.1621033\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,17]]},"references-count":46,"alternative-id":["10.3389\/frobt.2025.1621033"],"URL":"https:\/\/doi.org\/10.3389\/frobt.2025.1621033","relation":{},"ISSN":["2296-9144"],"issn-type":[{"value":"2296-9144","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,17]]},"article-number":"1621033"}}