{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,1]],"date-time":"2026-06-01T03:01:27Z","timestamp":1780282887040,"version":"3.54.0"},"reference-count":43,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2026,4,11]],"date-time":"2026-04-11T00:00:00Z","timestamp":1775865600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,6,1]],"date-time":"2026-06-01T00:00:00Z","timestamp":1780272000000},"content-version":"vor","delay-in-days":51,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100008047","name":"Carnegie Mellon University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100008047","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Intell Robot Syst"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    As robotic systems become increasingly integrated into complex real-world environments, there is a growing need for approaches that enable robots to understand and act on natural language instructions without extensive pre-programmed knowledge. This paper presents PLATO, a system that addresses this challenge by leveraging specialized large language model agents to process language inputs, understand the environment, predict tool affordances, and generate executable actions. Unlike traditional systems that depend on hard-coded environmental information, PLATO employs a modular agent-based architecture that operates without any initial knowledge of the environment. These agents identify objects and their locations, generate a high-level plan, translate it into low-level actions, and verify successful execution. The system is tested particularly on challenging tool-use tasks, which involve handling diverse objects and require long-horizon planning. PLATO\u2019s design allows it to adapt to dynamic and unstructured settings, enhancing its flexibility and robustness. By evaluating the system across various complex scenarios, we demonstrate its capability to tackle a diverse range of tasks and offer a novel solution to integrate LLMs with robotic platforms, advancing the state-of-the-art in autonomous robotic task execution. For videos and access to our code base, please see our project website:\n                    <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" xlink:href=\"https:\/\/sites.google.com\/view\/plato-anonymous\" ext-link-type=\"uri\">https:\/\/sites.google.com\/view\/plato-anonymous<\/jats:ext-link>\n                    .\n                  <\/jats:p>","DOI":"10.1007\/s10846-026-02392-y","type":"journal-article","created":{"date-parts":[[2026,4,11]],"date-time":"2026-04-11T13:35:20Z","timestamp":1775914520000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["PLATO: Planning with LLMs and Affordances for Tool Manipulation"],"prefix":"10.1007","volume":"112","author":[{"given":"Arvind","family":"Car","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sai","family":"Yarlagadda","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alison","family":"Bartsch","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Abraham","family":"George","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Amir","family":"Barati\u00a0Farimani","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,4,11]]},"reference":[{"key":"2392_CR1","doi-asserted-by":"crossref","unstructured":"Wong, C., Yang, E., Yan, X.-T., Gu, D.: Autonomous robots for harsh environments: a holistic overview of current solutions and ongoing challenges. Syst. Sci. Contr. Eng. 6(1), 213\u2013219 (2018)","DOI":"10.1080\/21642583.2018.1477634"},{"issue":"3","key":"2392_CR2","doi-asserted-by":"publisher","first-page":"122","DOI":"10.3390\/robotics2030122","volume":"2","author":"P Kormushev","year":"2013","unstructured":"Kormushev, P., Calinon, S., Caldwell, D.G.: Reinforcement learning in robotics: Applications and real-world challenges. Robotics 2(3), 122\u2013148 (2013)","journal-title":"Robotics"},{"key":"2392_CR3","doi-asserted-by":"crossref","unstructured":"Ciocarlie, M., Hsiao, K., Jones, E.G., Chitta, S., Rusu, R.B., \u015eucan, I.A.: Towards reliable grasping and manipulation in household environments. In: Experimental Robotics: The 12th International Symposium on Experimental Robotics, pp. 241\u2013252 (2014). Springer","DOI":"10.1007\/978-3-642-28572-1_17"},{"key":"2392_CR4","doi-asserted-by":"crossref","unstructured":"Edsinger, A., Kemp, C.C.: Manipulation in human environments. In: 2006 6th IEEE-RAS International Conference on Humanoid Robots, pp. 102\u2013109 (2006). IEEE","DOI":"10.1109\/ICHR.2006.321370"},{"issue":"6446","key":"2392_CR5","doi-asserted-by":"publisher","first-page":"8414","DOI":"10.1126\/science.aat8414","volume":"364","author":"A Billard","year":"2019","unstructured":"Billard, A., Kragic, D.: Trends and challenges in robot manipulation. Science 364(6446), 8414 (2019)","journal-title":"Science"},{"key":"2392_CR6","doi-asserted-by":"publisher","first-page":"102360","DOI":"10.1016\/j.rcim.2022.102360","volume":"77","author":"Z Liu","year":"2022","unstructured":"Liu, Z., Liu, Q., Xu, W., Wang, L., Zhou, Z.: Robot learning towards smart robotic manufacturing: A review. Robot. Comput.-Integr. Manuf. 77, 102360 (2022)","journal-title":"Robot. Comput.-Integr. Manuf."},{"key":"2392_CR7","doi-asserted-by":"publisher","first-page":"789107","DOI":"10.3389\/frobt.2021.789107","volume":"8","author":"Z Wang","year":"2022","unstructured":"Wang, Z., Hirai, S., Kawamura, S.: Challenges and opportunities in robotic food handling: A review. Front. Robot. AI 8, 789107 (2022)","journal-title":"Front. Robot. AI"},{"issue":"3","key":"2392_CR8","doi-asserted-by":"publisher","first-page":"1269","DOI":"10.33736\/ijbs.4301.2021","volume":"22","author":"D Fernandez","year":"2021","unstructured":"Fernandez, D., Aman, A.: The challenges of implementing robotic process automation in global business services. Int. J. Bus. Soc. 22(3), 1269\u20131282 (2021)","journal-title":"Int. J. Bus. Soc."},{"key":"2392_CR9","doi-asserted-by":"crossref","unstructured":"Bartsch, A., Car, A., Avra, C., Farimani, A.B.: Sculptdiff: Learning robotic clay sculpting from humans with goal conditioned diffusion policy (2024). arXiv:2403.10401","DOI":"10.1109\/IROS58592.2024.10803054"},{"key":"2392_CR10","doi-asserted-by":"crossref","unstructured":"Bartsch, A., Avra, C., Farimani, A.B.: Sculptbot: Pre-trained models for 3d deformable object manipulation. In: 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 12548\u201312555 (2024). IEEE","DOI":"10.1109\/ICRA57147.2024.10610899"},{"key":"2392_CR11","unstructured":"George, A., Gano, S., Katragadda, P., Farimani, A.B.: Visuo-tactile pretraining for cable plugging (2024). arXiv:2403.11898"},{"key":"2392_CR12","doi-asserted-by":"crossref","unstructured":"George, A., Bartsch, A., Farimani, A.B.: Minimizing human assistance: Augmenting a single demonstration for deep reinforcement learning. In: 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 5027\u20135033 (2023). IEEE","DOI":"10.1109\/ICRA48891.2023.10161119"},{"key":"2392_CR13","unstructured":"George, A., Farimani, A.B.: One act play: Single demonstration behavior cloning with action chunking transformers (2023). arXiv:2309.10175"},{"key":"2392_CR14","unstructured":"Ahn, M., Brohan, A., Brown, N., Chebotar, Y., Cortes, O., David, B., Finn, C., Fu, C., Gopalakrishnan, K., Hausman, K., et al.: Do as i can, not as i say: Grounding language in robotic affordances (2022). arXiv:2204.01691"},{"key":"2392_CR15","doi-asserted-by":"crossref","unstructured":"Liang, J., Huang, W., Xia, F., Xu, P., Hausman, K., Ichter, B., Florence, P., Zeng, A.: Code as policies: Language model programs for embodied control. In: 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 9493\u20139500 (2023). IEEE","DOI":"10.1109\/ICRA48891.2023.10160591"},{"key":"2392_CR16","unstructured":"Dikshit, A., Bartsch, A., George, A., Farimani, A.B.: Robochop: Autonomous framework for fruit and vegetable chopping leveraging foundational models (2023). arXiv:2307.13159"},{"key":"2392_CR17","doi-asserted-by":"crossref","unstructured":"Tang, L., Jia, Y.-B., Xue, Y.: Robotic manipulation of hand tools: The case of screwdriving. In: 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 13883\u201313890 (2024). IEEE","DOI":"10.1109\/ICRA57147.2024.10610831"},{"issue":"2\u20133","key":"2392_CR18","doi-asserted-by":"publisher","first-page":"202","DOI":"10.1177\/0278364919872545","volume":"39","author":"K Fang","year":"2020","unstructured":"Fang, K., Zhu, Y., Garg, A., Kurenkov, A., Mehta, V., Fei-Fei, L., Savarese, S.: Learning task-oriented grasping for tool manipulation from simulated self-supervision. Int. J. Robot. Res. 39(2\u20133), 202\u2013216 (2020)","journal-title":"Int. J. Robot. Res."},{"key":"2392_CR19","doi-asserted-by":"crossref","unstructured":"Qin, Z., Fang, K., Zhu, Y., Fei-Fei, L., Savarese, S.: Keto: Learning keypoint representations for tool manipulation. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 7278\u20137285 (2020). IEEE","DOI":"10.1109\/ICRA40945.2020.9196971"},{"key":"2392_CR20","doi-asserted-by":"crossref","unstructured":"Ding, Y., Zhang, X., Paxton, C., Zhang, S.: Task and motion planning with large language models for object rearrangement. In: 2023 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2086\u20132092 (2023). IEEE","DOI":"10.1109\/IROS55552.2023.10342169"},{"issue":"8","key":"2392_CR21","doi-asserted-by":"publisher","first-page":"1087","DOI":"10.1007\/s10514-023-10139-z","volume":"47","author":"J Wu","year":"2023","unstructured":"Wu, J., Antonova, R., Kan, A., Lepert, M., Zeng, A., Song, S., Bohg, J., Rusinkiewicz, S., Funkhouser, T.: Tidybot: Personalized robot assistance with large language models. Auton. Robot. 47(8), 1087\u20131102 (2023)","journal-title":"Auton. Robot."},{"key":"2392_CR22","unstructured":"Xu, M., Huang, P., Yu, W., Liu, S., Zhang, X., Niu, Y., Zhang, T., Xia, F., Tan, J., Zhao, D.: Creative robot tool use with large language models (2023). arXiv:2310.13065"},{"key":"2392_CR23","doi-asserted-by":"crossref","unstructured":"Bartsch, A., Farimani, A.B.: Llm-craft: Robotic crafting of elasto-plastic objects with large language models (2024). arXiv:2406.08648","DOI":"10.1109\/LRA.2025.3597835"},{"key":"2392_CR24","doi-asserted-by":"crossref","unstructured":"Chen, Y., Arkin, J., Dawson, C., Zhang, Y., Roy, N., Fan, C.: Autotamp: Autoregressive task and motion planning with llms as translators and checkers. In: 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 6695\u20136702 (2024). IEEE","DOI":"10.1109\/ICRA57147.2024.10611163"},{"key":"2392_CR25","unstructured":"Zhu, F., Hu, S., Leng, L., Bartsch, A., George, A., Farimani, A.B.: Pour me a drink: Robotic precision pouring carbonated beverages into transparent containers (2023). arXiv:2309.08892"},{"key":"2392_CR26","doi-asserted-by":"crossref","unstructured":"Dorbala, V.S., Mullen Jr, J.F., Manocha, D.: Can an embodied agent find your\" cat-shaped mug\"? llm-guided exploration for zero-shot object navigation (2023). arXiv:2303.03480","DOI":"10.1109\/LRA.2023.3346800"},{"key":"2392_CR27","doi-asserted-by":"crossref","unstructured":"Singh, I., Blukis, V., Mousavian, A., Goyal, A., Xu, D., Tremblay, J., Fox, D., Thomason, J., Garg, A.: Progprompt: Generating situated robot task plans using large language models. In: 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 11523\u201311530 (2023). IEEE","DOI":"10.1109\/ICRA48891.2023.10161317"},{"key":"2392_CR28","unstructured":"Rana, K., Haviland, J., Garg, S., Abou-Chakra, J., Reid, I., Suenderhauf, N.: Sayplan: Grounding large language models using 3d scene graphs for scalable task planning (2023). arXiv:2307.06135"},{"key":"2392_CR29","unstructured":"Sharan, S., Zhao, R., Wang, Z., Chinchali, S.P., et al.: Plan diffuser: Grounding llm planners with diffusion models for robotic manipulation. In: Bridging the Gap Between Cognitive Science and Robot Learning in the Real World: Progresses and New Directions (2024)"},{"key":"2392_CR30","doi-asserted-by":"crossref","unstructured":"Liu, F., Fang, K., Abbeel, P., Levine, S.: Moka: Open-world robotic manipulation through mark-based visual prompting (2024). arXiv:2403.03174","DOI":"10.15607\/RSS.2024.XX.062"},{"key":"2392_CR31","doi-asserted-by":"crossref","unstructured":"Shentu, Y., Wu, P., Rajeswaran, A., Abbeel, P.: From llms to actions: latent codes as bridges in hierarchical robot control. In: 2024 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 8539\u20138546 (2024). IEEE","DOI":"10.1109\/IROS58592.2024.10801683"},{"key":"2392_CR32","unstructured":"Huang, W., Wang, C., Li, Y., Zhang, R., Fei-Fei, L.: Rekep: Spatio-temporal reasoning of relational keypoint constraints for robotic manipulation (2024). arXiv:2409.01652"},{"key":"2392_CR33","doi-asserted-by":"crossref","unstructured":"Huang, H., Lin, F., Hu, Y., Wang, S., Gao, Y.: Copa: General robotic manipulation through spatial constraints of parts with foundation models. In: 2024 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 9488\u20139495 (2024). IEEE","DOI":"10.1109\/IROS58592.2024.10801352"},{"key":"2392_CR34","unstructured":"Seita, D., Wang, Y., Shetty, S.J., Li, E.Y., Erickson, Z., Held, D.: Toolflownet: Robotic manipulation with tools via predicting tool flow from point clouds. In: Conference on Robot Learning, pp. 1038\u20131049 (2023). PMLR"},{"key":"2392_CR35","doi-asserted-by":"crossref","unstructured":"Shirai, Y., Jha, D.K., Raghunathan, A.U., Hong, D.: Tactile tool manipulation. In: 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 12597\u201312603 (2023). IEEE","DOI":"10.1109\/ICRA48891.2023.10160480"},{"key":"2392_CR36","unstructured":"Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al.: Gpt-4 technical report (2023). arXiv:2303.08774"},{"key":"2392_CR37","unstructured":"Zheng, Z., Feng, Q., Li, H., Knoll, A., Feng, J.: Evaluating uncertainty-based failure detection for closed-loop llm planners (2024). arXiv preprint arXiv:2406.00430"},{"key":"2392_CR38","doi-asserted-by":"crossref","unstructured":"Luo, F.: Vision-language models for robot success detection. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, pp. 23750\u201323752 (2024)","DOI":"10.1609\/aaai.v38i21.30552"},{"key":"2392_CR39","doi-asserted-by":"crossref","unstructured":"Liu, S., Zeng, Z., Ren, T., Li, F., Zhang, H., Yang, J., Li, C., Yang, J., Su, H., Zhu, J., et al.: Grounding dino: Marrying dino with grounded pre-training for open-set object detection (2023). arXiv:2303.05499","DOI":"10.1007\/978-3-031-72970-6_3"},{"key":"2392_CR40","doi-asserted-by":"crossref","unstructured":"Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.-Y., et al.: Segment anything. In: Proceedings of the IEEE\/CVF International Conference on Computer Vision, pp. 4015\u20134026 (2023)","DOI":"10.1109\/ICCV51070.2023.00371"},{"key":"2392_CR41","doi-asserted-by":"crossref","unstructured":"Holomjova, V., Starkey, A.J., Yun, B., Mei\u00dfner, P.: One-shot learning for task-oriented grasping. IEEE Robot. Auto. Lett. (2023)","DOI":"10.1109\/LRA.2023.3326001"},{"key":"2392_CR42","unstructured":"Hadjivelichkov, D., Zwane, S., Agapito, L., Deisenroth, M.P., Kanoulas, D.: One-shot transfer of affordance regions? affcorrs! In: Conference on Robot Learning, pp. 550\u2013560 (2023). PMLR"},{"issue":"1","key":"2392_CR43","doi-asserted-by":"publisher","first-page":"87","DOI":"10.2307\/2340521","volume":"85","author":"RA Fisher","year":"1922","unstructured":"Fisher, R.A.: On the interpretation of $$\\chi $$ 2 from contingency tables, and the calculation of p. J. Roy. Stat. Soc. 85(1), 87\u201394 (1922)","journal-title":"J. Roy. Stat. Soc."}],"container-title":["Journal of Intelligent &amp; Robotic Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10846-026-02392-y","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10846-026-02392-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10846-026-02392-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,1]],"date-time":"2026-06-01T02:31:51Z","timestamp":1780281111000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10846-026-02392-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,11]]},"references-count":43,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2026,6]]}},"alternative-id":["2392"],"URL":"https:\/\/doi.org\/10.1007\/s10846-026-02392-y","relation":{},"ISSN":["1573-0409"],"issn-type":[{"value":"1573-0409","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,11]]},"assertion":[{"value":"24 November 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 March 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 April 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}},{"value":"Not applicable","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}}],"article-number":"58"}}