{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,1]],"date-time":"2026-06-01T23:39:54Z","timestamp":1780357194043,"version":"3.54.1"},"reference-count":55,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T00:00:00Z","timestamp":1774569600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100014013","name":"UK Research and Innovation","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100014013","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Robot. AI"],"abstract":"<jats:sec>\n                    <jats:title>Introduction<\/jats:title>\n                    <jats:p>In this study, we address intent-driven task planning for complex multi-action manipulation sequences in heterogeneous multi-robot cells. Given a perception back-end that outputs a structured object-level scene description and a human operator\u2019s natural-language intent, we generate a precedence-consistent object-level robot-action sequence, which can then be executed by passing each such action to a lower-level motion planning module.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>The pipeline integrates i. perception-to-text scene encoding, ii. an ensemble of large language models (LLMs) that generate candidate action sequences based on the operator\u2019s intent, iii. an LLM-based verifier that enforces formatting and precedence constraints, and iv. a deterministic consistency filter that rejects hallucinated objects. The pipeline is evaluated on an example task in which two robot arms work collaboratively to dismantle an electric-vehicle (EV) battery for recycling applications. A variety of components must be grasped and removed in specific sequences, determined either by human instructions or by task-order feasibility decisions made by the autonomous system.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>On 200 real scenes with 600 operator prompts across five component classes, we used metrics of full-sequence correctness and next-task correctness to evaluate and compare five LLM-based planners (including ablation analyses of pipeline components). We also evaluated the LLM-based human interface in terms of time to execution and NASA TLX using human participant experiments. On 200 real scenes and 600 prompts, full-sequence correctness improves from 0.761 (single LLM) to 0.824 (6-LLM + verifier + deterministic filter), and next-object correctness improves from 0.866 to 0.894.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Discussion<\/jats:title>\n                    <jats:p>Results in our case study indicate that our ensemble-with-verification approach reliably maps operator intent to safe multi-robot plans while maintaining low user effort.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/frobt.2026.1727433","type":"journal-article","created":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T10:59:19Z","timestamp":1774609159000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Intent-driven LLM ensemble planning for flexible multi-robot manipulation"],"prefix":"10.3389","volume":"13","author":[{"given":"Cansu","family":"Erdogan","sequence":"first","affiliation":[{"name":"Extreme Robotics Lab, School of Metallurgy and Materials, University of Birmingham","place":["Birmingham, United Kingdom"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cesar Alan","family":"Contreras","sequence":"additional","affiliation":[{"name":"Extreme Robotics Lab, School of Metallurgy and Materials, University of Birmingham","place":["Birmingham, United Kingdom"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alireza","family":"Rastegarpanah","sequence":"additional","affiliation":[{"name":"Extreme Robotics Lab, School of Metallurgy and Materials, University of Birmingham","place":["Birmingham, United Kingdom"]},{"name":"Department of Applied Artificial Intelligence and Robotics, School of Computer Science, Aston University","place":["Birmingham, United Kingdom"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Manolis","family":"Chiou","sequence":"additional","affiliation":[{"name":"School of Electronic Engineering and Computer Science, Queen Mary University of London","place":["London, United Kingdom"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rustam","family":"Stolkin","sequence":"additional","affiliation":[{"name":"Extreme Robotics Lab, School of Metallurgy and Materials, University of Birmingham","place":["Birmingham, United Kingdom"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,3,27]]},"reference":[{"key":"B1","article-title":"Supporting the implementation of robotics and remote systems in the nuclear back-end","year":"2025"},{"key":"B2","doi-asserted-by":"publisher","first-page":"483","DOI":"10.1016\/j.jmsy.2024.09.010","article-title":"Robotic disassembly for end-of-life products focusing on task and motion planning: a comprehensive survey","volume":"77","author":"Asif","year":"2024","journal-title":"J. Manuf. Syst."},{"key":"B3","doi-asserted-by":"crossref","DOI":"10.1109\/HRI61500.2025.10973985","article-title":"Toward zero-shot user intent recognition in shared autonomy","volume-title":"Proceedings of the 2025 ACM\/IEEE international conference on human-robot interaction (HRI)","author":"Belsare","year":"2025"},{"key":"B4","doi-asserted-by":"publisher","first-page":"109345","DOI":"10.1016\/j.cie.2023.109345","article-title":"A novel framework to improve motion planning of robotic systems through semantic knowledge-based reasoning","volume":"182","author":"Bernardo","year":"2023","journal-title":"Comput. and Industrial Eng."},{"key":"B5","doi-asserted-by":"publisher","first-page":"232","DOI":"10.1002\/rob.22048","article-title":"Vega\u2014a small, low cost, ground robot for nuclear decommissioning","volume":"39","author":"Bird","year":"2021","journal-title":"J. Field Robotics"},{"key":"B6","doi-asserted-by":"publisher","first-page":"167","DOI":"10.1007\/s10846-024-02185-1","article-title":"Intent-based task-oriented shared control for intuitive telemanipulation","volume":"110","author":"Bowman","year":"2024","journal-title":"J. Intelligent and Robotic Syst."},{"key":"B55","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2307.15818","article-title":"RT-2: vision-language-action models transfer web knowledge to robotic control","author":"Brohan","year":"2023"},{"key":"B7","doi-asserted-by":"crossref","DOI":"10.1145\/1015330.1015432","article-title":"Ensemble selection from libraries of models","volume-title":"Proceedings of the twenty-first international conference on machine learning","author":"Caruana","year":"2004"},{"key":"B8","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3472206","article-title":"Mixed-initiative variable autonomy for remotely operated mobile robots","volume":"10","author":"Chiou","year":"2021","journal-title":"ACM Trans. Human-Robot Interact. (THRI)"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2210.11416","article-title":"Scaling instruction-finetuned language models","author":"Chung","year":"2022","journal-title":"arXiv preprint arXiv:2210.11416"},{"key":"B10","doi-asserted-by":"crossref","DOI":"10.1007\/s10846-026-02362-4","article-title":"Probabilistic human intent prediction for mobile manipulation: an evaluation with human-inspired constraints","author":"Contreras","year":""},{"key":"B11","doi-asserted-by":"publisher","first-page":"1540476","DOI":"10.3389\/frobt.2025.1540476","article-title":"A mini-review on mobile manipulators with variable autonomy","volume":"12","author":"Contreras","year":"","journal-title":"Front. Robotics AI"},{"key":"B12","doi-asserted-by":"crossref","first-page":"1624","DOI":"10.1109\/ICSMC.2001.973517","article-title":"Experiments in adjustable autonomy","volume-title":"2001 IEEE international conference on systems, man and cybernetics. e-Systems and e-Man for cybernetics in cyberspace (cat. No. 01CH37236)","author":"Crandall","year":"2001"},{"key":"B13","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2303.03378","article-title":"PaLM-E: an embodied multimodal language model","author":"Driess","year":"2023"},{"key":"B14","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2025.findings-naacl.344","article-title":"M-ifeval: multilingual instruction-following evaluation","volume-title":"Findings of the association for computational linguistics: NAACL 2025","author":"Dussolle","year":"2025"},{"key":"B15","doi-asserted-by":"publisher","first-page":"75","DOI":"10.3390\/robotics13050075","article-title":"Multi-robot task planning for efficient battery disassembly in electric vehicles","volume":"13","author":"Erdogan","year":"2024","journal-title":"Robotics"},{"key":"B16","doi-asserted-by":"publisher","first-page":"102770","DOI":"10.1016\/j.omega.2022.102770","article-title":"Scheduling of multi-robot job shop systems in dynamic environments: mixed-integer linear programming and constraint programming approaches","volume":"115","author":"Fatemi-Anaraki","year":"2023","journal-title":"Omega"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2101.03961","article-title":"Switch transformers: scaling to trillion parameter models with simple and efficient sparsity","author":"Fedus","year":"2021","journal-title":"arXiv preprint arXiv:2101.03961"},{"key":"B18","doi-asserted-by":"crossref","DOI":"10.1109\/ICRA55743.2025.11127315","article-title":"Towards generalizable vision-language robotic manipulation: a benchmark and llm-guided 3d policy","author":"Garcia","year":"2025"},{"key":"B19","doi-asserted-by":"publisher","first-page":"1727","DOI":"10.1177\/1541931215591373","article-title":"How high is high? a meta-analysis of nasa-tlx global workload scores","volume":"59","author":"Grier","year":"2015","journal-title":"Proc. Human Factors Ergonomics Society Annual Meeting"},{"key":"B21","unstructured":"What is yolov8? the ultimate guide\n          \n          \n            \n              Jacob Solawetz\n              F.\n            \n          \n          \n          2025"},{"key":"B22","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2024.acl-long.257","article-title":"Followbench: a multi-level fine-grained constraints following benchmark for large language models","volume-title":"Proceedings of the 62nd annual meeting of the association for computational linguistics (ACL)","author":"Jiang","year":"2024"},{"key":"B23","doi-asserted-by":"publisher","first-page":"371","DOI":"10.1145\/3610977.3634966","article-title":"Understanding large-language model (Llm)-powered human-robot interaction","author":"Kim","year":"2024","journal-title":"arXiv"},{"key":"B24","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1109\/THMS.2015.2480801","article-title":"Human interaction with robot swarms: a survey","volume":"46","author":"Kolling","year":"2016","journal-title":"IEEE Trans. Human-Machine Syst."},{"key":"B25","doi-asserted-by":"publisher","first-page":"1347538","DOI":"10.3389\/frobt.2024.1347538","article-title":"Exploring a gpt-based large language model for variable autonomy in a vr-based human-robot teaming simulation","volume":"11","author":"Lakhnati","year":"2024","journal-title":"Front. Robotics AI"},{"key":"B26","article-title":"Simple and scalable predictive uncertainty estimation using deep ensembles","volume-title":"Advances in neural information processing systems (NeurIPS)","author":"Lakshminarayanan","year":"2017"},{"key":"B27","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR52733.2024.01710","article-title":"Manipllm: embodied multimodal large language model for object-centric robotic manipulation","volume-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (CVPR)","author":"Li","year":"2024"},{"key":"B28","doi-asserted-by":"crossref","DOI":"10.65109\/SWQK9871","article-title":"Goal inference improves objective and perceived performance in human-robot collaboration","volume-title":"Proceedings of the 2016 international conference on autonomous agents and multiagent systems (AAMAS)","author":"Liu","year":"2016"},{"key":"B29","doi-asserted-by":"publisher","first-page":"6904","DOI":"10.1109\/lra.2024.3415931","article-title":"Enhancing the llm-based robot manipulation through human-robot collaboration","volume":"9","author":"Liu","year":"2024","journal-title":"IEEE Robotics Automation Lett."},{"key":"B30","doi-asserted-by":"crossref","DOI":"10.1109\/ICRA57147.2024.10610797","article-title":"Robollm: robotic vision tasks grounded on multimodal large language models","author":"Long","year":"2024"},{"key":"B31","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/RAHA.2016.7931866","article-title":"Towards advanced robotic manipulation for nuclear decommissioning: a pilot study on tele-operation and autonomy","volume-title":"2016 international conference on robotics and automation for humanitarian applications (RAHA)","author":"Marturi","year":"2016"},{"key":"B32","first-page":"61","article-title":"Towards advanced robotic manipulations for nuclear decommissioning","volume-title":"Robots operating in hazardous environments","author":"Marturi","year":"2017"},{"key":"B33","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3645090","article-title":"Who\u2019s in charge here? a survey on trustworthy ai in variable autonomy robotic systems","volume":"56","author":"Methnani","year":"2024","journal-title":"ACM Computing Surveys"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.4108\/airo.5602","article-title":"Micro robots in healthcare: features, applications, and future directions: a feature review","volume":"3","author":"Moshayedi","year":"","journal-title":"EAI Endorsed Trans. AI Robotics"},{"key":"B35","doi-asserted-by":"publisher","first-page":"36148","DOI":"10.1109\/ACCESS.2024.3355278","article-title":"Design and development of FOODIEBOT robot: from simulation to design","volume":"12","author":"Moshayedi","year":"","journal-title":"IEEE Access"},{"key":"B36","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1007\/s43674-025-00083-z","article-title":"Comparative performance analysis of a novel fusion-based algorithm for AGV navigation","volume":"5","author":"Moshayedi","year":"2025","journal-title":"Adv. Comput. Intell."},{"key":"B37","doi-asserted-by":"publisher","first-page":"467","DOI":"10.1007\/s11423-019-09657-4","article-title":"Nasa-tlx for predictability and measurability of instructional design models: case study in design methods","volume":"67","author":"Nikulin","year":"2019","journal-title":"Educ. Technol. Res. Dev."},{"key":"B38","article-title":"Training language models to follow instructions with human feedback","volume-title":"Advances in neural information processing systems (NeurIPS)","author":"Ouyang","year":"2022"},{"key":"B39","doi-asserted-by":"crossref","first-page":"3482","DOI":"10.1109\/IROS58592.2024.10802695","article-title":"Robust and safe task-driven planning and navigation for heterogeneous multi-robot teams with uncertain dynamics","volume-title":"2024 IEEE\/RSJ international conference on intelligent robots and systems (IROS)","author":"Pan","year":"2024"},{"key":"B40","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1109\/SMC52423.2021.9658942","article-title":"A bayesian-based approach to human operator intent recognition in remote Mobile robot navigation","volume-title":"2021 IEEE international conference on systems, man, and cybernetics (SMC)","author":"Panagopoulos","year":"2021"},{"key":"B41","article-title":"Mixture-of-depths: dynamically allocating compute in transformer-based language models","author":"Raposo","year":"2024"},{"key":"B42","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3636432","article-title":"Variable autonomy through responsible robotics: design guidelines and research agenda","volume":"13","author":"Reinmund","year":"2024","journal-title":"ACM Trans. Human-Robot Interact."},{"key":"B43","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1613\/jair.1037","article-title":"Towards adjustable autonomy for the real world","volume":"17","author":"Scerri","year":"2002","journal-title":"J. Artif. Intell. Res."},{"key":"B44","article-title":"Multi-robot vision-based task and motion planning for ev battery disassembly and sorting","author":"Shaarawy","year":"2025"},{"key":"B45","article-title":"Status, barriers and cost-benefits of robotic and remote systems applications in nuclear decommissioning and radioactive waste management","volume-title":"White paper","author":"Stolkin","year":"2023"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2024.3520414","article-title":"Technoeconomic assessment of electric vehicle battery disassembly\u2014challenges and opportunities from a robotics perspective","author":"Stolkin","year":"2024","journal-title":"IEEE Access"},{"key":"B46","doi-asserted-by":"publisher","first-page":"72","DOI":"10.1109\/MRA.2012.2205651","article-title":"The open motion planning library","volume":"19","author":"\u015eucan","year":"2012","journal-title":"IEEE Robotics and Automation Mag."},{"key":"B47","doi-asserted-by":"crossref","first-page":"166","DOI":"10.1109\/SSRR.2016.7784294","article-title":"Towards robotic decommissioning of legacy nuclear plant: results of human-factors experiments with tele-robotic manipulation, and a discussion of challenges and approaches for decommissioning","volume-title":"2016 IEEE international symposium on safety, security, and rescue robotics (SSRR)","author":"Talha","year":"2016"},{"key":"B48","doi-asserted-by":"publisher","first-page":"2597","DOI":"10.1109\/LRA.2022.3142919","article-title":"Efficient industrial solution for robotic task sequencing problem with mutual collision avoidance and cycle time optimization","volume":"7","author":"Touzani","year":"2022","journal-title":"IEEE Robotics Automation Lett."},{"key":"B49","doi-asserted-by":"publisher","first-page":"8333","DOI":"10.1016\/j.ifacol.2023.10.1023","article-title":"A supervised machine learning approach to operator intent recognition for teleoperated mobile robot navigation","volume":"56","author":"Tsagkournis","year":"2023","journal-title":"IFAC-PapersOnLine"},{"key":"B50","doi-asserted-by":"publisher","first-page":"112","DOI":"10.3390\/robotics10040112","article-title":"A suite of robotic solutions for nuclear waste decommissioning","volume":"10","author":"Vitanov","year":"2021","journal-title":"Robotics"},{"key":"B51","doi-asserted-by":"publisher","first-page":"104604","DOI":"10.1016\/j.robot.2023.104604","article-title":"Task scheduling for heterogeneous agents pickup and delivery using recurrent open shop scheduling models","volume":"172","author":"Wang","year":"2024","journal-title":"Robotics Aut. Syst."},{"key":"B52","article-title":"Finetuned language models are zero-shot learners","volume-title":"International conference on learning representations (ICLR)","author":"Wei","year":"2022"},{"key":"B53","doi-asserted-by":"crossref","DOI":"10.1109\/ICRA55743.2025.11127646","article-title":"Adaptive task allocation in multi-human multi-robot teams under team heterogeneity and dynamic information uncertainty","volume-title":"Proceedings of the IEEE international conference on robotics and automation (ICRA)","author":"Yuan","year":"2025"},{"key":"B54","article-title":"Instruction-following evaluation for large language models","author":"Zhou","year":"2023"}],"container-title":["Frontiers in Robotics and AI"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2026.1727433\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T10:59:26Z","timestamp":1774609166000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2026.1727433\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,27]]},"references-count":55,"alternative-id":["10.3389\/frobt.2026.1727433"],"URL":"https:\/\/doi.org\/10.3389\/frobt.2026.1727433","relation":{},"ISSN":["2296-9144"],"issn-type":[{"value":"2296-9144","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,27]]},"article-number":"1727433"}}