{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,29]],"date-time":"2025-10-29T12:40:41Z","timestamp":1761741641098,"version":"build-2065373602"},"reference-count":36,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,10,29]],"date-time":"2025-10-29T00:00:00Z","timestamp":1761696000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Robot. AI"],"abstract":"<jats:p>This paper addresses the challenge of enabling robots to autonomously prepare meals by bridging natural language recipe instructions and robotic action execution. We propose a novel methodology leveraging Actionable Knowledge Graphs to map recipe instructions into six core categories of robotic manipulation tasks, termed Action Cores cutting, pouring, mixing, preparing, pick and place, and cook and cool. Each AC is subdivided into Action Groups which represent a specific motion parameterization required for task execution. Using the Recipe1M + dataset (Mar\u00edn et al., IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43, 187\u2013203), encompassing over one million recipes, we systematically analysed action verbs and matched them to ACs by using direct matching and cosine similarity, achieving a coverage of 76.5%. For the unmatched verbs, we employ a neuro-symbolic approach, matching verbs to existing AGs or generating new action cores utilizing a Large Language Model Our findings highlight the versatility of AKGs in adapting general plans to specific robotic tasks, validated through an experimental application in a meal preparation scenario. This work sets a foundation for adaptive robotic systems capable of performing a wide array of complex culinary tasks with minimal human intervention.<\/jats:p>","DOI":"10.3389\/frobt.2025.1682031","type":"journal-article","created":{"date-parts":[[2025,10,29]],"date-time":"2025-10-29T07:07:56Z","timestamp":1761721676000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Everything robots need to know about cooking actions: creating actionable knowledge graphs to support robotic meal preparation"],"prefix":"10.3389","volume":"12","author":[{"given":"Michaela","family":"K\u00fcmpel","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Manuel","family":"Scheibl","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jan-Philipp","family":"T\u00f6berg","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Vanessa","family":"Hassouna","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Philipp","family":"Cimiano","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Britta","family":"Wrede","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michael","family":"Beetz","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2025,10,29]]},"reference":[{"key":"B1","first-page":"287","article-title":"Do as I can, not as I say: grounding language in robotic affordances","volume-title":"Proc. Of the 6th conference on robot learning (CoRL)(PMLR), vol. 205 of Proceedings of machine learning research","author":"Ahn","year":"2023"},{"key":"B2","article-title":"The Claude 3 model family: opus, sonnet, haiku","year":"2024","journal-title":"Tech. Rep. Claude-3 Model Card."},{"key":"B3","doi-asserted-by":"crossref","first-page":"86","DOI":"10.3115\/980845.980860","article-title":"The berkeley FrameNet project","volume-title":"Proc. Of the 36th annual meeting on association for computational linguistics (ACL)","author":"Baker","year":"1998"},{"key":"B4","article-title":"Transforming web knowledge into actionable knowledge graphs for robot manipulation tasks","volume-title":"ESWC 2024 workshops and tutorials Joint proceeding","author":"Beetz","year":"2024"},{"key":"B5","doi-asserted-by":"crossref","first-page":"481","DOI":"10.1007\/978-3-319-00065-7_33","article-title":"Interpreting and executing recipes with a cooking robot","volume-title":"Experimental robotics","author":"Bollini","year":"2013"},{"key":"B6","doi-asserted-by":"crossref","first-page":"12482","DOI":"10.18653\/v1\/2024.findings-emnlp.728","article-title":"PizzaCommonSense: a dataset for commonsense reasoning about intermediate steps in cooking recipes","volume-title":"Findings of the association for computational linguistics: EMNLP 2024","author":"Diallo","year":"2024"},{"key":"B7","first-page":"266","article-title":"Verb physics: relative physical knowledge of actions and objects","volume-title":"Proc. Of the 55th annual meeting of the association for computational linguistics (ACL)","author":"Forbes","year":"2017"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2408.00118","article-title":"Gemma 2: improving open Language Models at a practical size","author":"Riviere","year":"2024","journal-title":"Tech. Rep."},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2407.21783","article-title":"The Llama 3 herd of models","author":"Grattafiori","year":"2024","journal-title":"Tech. Rep."},{"key":"B10","doi-asserted-by":"publisher","first-page":"81","DOI":"10.5121\/ijaia.2018.9207","article-title":"Unsupervised robotic sorting: towards autonomous decision making robots","volume":"9","author":"Gu\u00e9rin","year":"2018","journal-title":"Int. J. Artif. Intell. Appl. (IJAIA)"},{"key":"B11","article-title":"Steps towards generalized manipulation action plans - tackling mixing task","volume-title":"ESWC 2024 workshops and tutorials Joint proceedings","author":"Hassouna","year":"2024"},{"key":"B12","article-title":"More than eleven thousand words: towards using Language Models for robotic sorting of unseen objects into arbitrary categories","volume-title":"Proc. Of the CoRL workshop on language and robot learning (LangRob)","author":"H\u00f8eg","year":"2022"},{"key":"B13","doi-asserted-by":"publisher","first-page":"1318","DOI":"10.1080\/01691864.2024.2407136","article-title":"Real-world cooking robot system from recipes based on food state recognition using foundation models and PDDL","volume":"38","author":"Kanazawa","year":"2024","journal-title":"Adv. Robot."},{"key":"B14","doi-asserted-by":"crossref","first-page":"982","DOI":"10.18653\/v1\/D15-1114","article-title":"Mise en Place: Unsupervised Interpretation of Instructional Recipes","volume-title":"Proc. Of the 2015 conference on empirical methods in natural language processing (EMNLP)","author":"Kiddon","year":"2015"},{"key":"B15","volume-title":"Actionable knowledge graphs - how daily activity applications can benefit from embodied web knowledge","author":"K\u00fcmpel","year":"2024"},{"key":"B16","article-title":"Enhancing cognitive robotics with actionable knowledge graphs: a framework for context-aware reasoning","volume-title":"1st German robotics conference (GRC)","author":"K\u00fcmpel","year":"2025"},{"key":"B17","article-title":"Creating and accessing knowledge graphs for action parameterisation","volume-title":"Ekaw 2024: EKAW 2024 workshops, tutorials, posters and demos, 24th international conference on knowledge engineering and knowledge management (EKAW 2024)","author":"K\u00fcmpel","year":"2024"},{"key":"B18","article-title":"Towards a knowledge engineering methodology for flexible robot manipulation in everyday tasks","volume-title":"ESWC 2024 workshops and tutorials Joint proceedings","author":"K\u00fcmpel","year":"2024"},{"key":"B19","volume-title":"RoBERTa: a robustly optimized BERT pretraining approach","author":"Liu","year":"2019"},{"key":"B20","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1109\/TPAMI.2019.2927476","article-title":"Recipe1M+: a dataset for learning cross-modal embeddings for cooking recipes and food images","volume":"43","author":"Mar\u00edn","year":"2021","journal-title":"IEEE Trans. Pattern Analysis Mach. Intell."},{"key":"B21","first-page":"17679","article-title":"Cook2LTL: translating cooking recipes to LTL formulae using Large Language Models","author":"Mavrogiannis","year":"2024"},{"key":"B22","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1145\/219717.219748","article-title":"WordNet: a lexical database for English","volume":"38","author":"Miller","year":"1995","journal-title":"Commun. ACM"},{"key":"B23","first-page":"22","article-title":"A benchmark for recipe understanding in artificial agents","volume-title":"Proc. Of the 2024 Joint international conference on computational linguistics, language Resources and evaluation (LREC-COLING)","author":"Nevens","year":"2024"},{"key":"B24","first-page":"243","article-title":"Everything robots always wanted to know about housework (but were afraid to ask)","author":"Nyga","year":"2012"},{"key":"B25","article-title":"GPT-4 technical report","volume-title":"Tech. Rep.","year":"2023"},{"key":"B26","article-title":"From Language to action with object-level planning","volume-title":"Proc. Of the 2nd CoRL workshop on learning effective abstractions for planning","author":"Paulius","year":"2024"},{"key":"B27","doi-asserted-by":"publisher","first-page":"1820","DOI":"10.1109\/TASE.2021.3129077","article-title":"Towards foodservice robotics: a taxonomy of actions of foodservice workers and a critical review of supportive Technology","volume":"19","author":"Pereira","year":"2022","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"B28","article-title":"SayPlan: grounding Large Language Models using 3D scene graphs for scalable task planning","author":"Rana","year":"2023"},{"key":"B29","first-page":"12704","article-title":"From cooking recipes to robot task trees - improving planning correctness and task efficiency by leveraging LLMs with a knowledge network","author":"Sakib","year":"2024"},{"key":"B30","doi-asserted-by":"publisher","first-page":"11492","DOI":"10.1109\/LRA.2022.3191068","article-title":"Approximate task tree retrieval in a knowledge network for robotic cooking","volume":"7","author":"Sakib","year":"2022","journal-title":"IEEE Robot. Autom. Lett."},{"key":"B31","first-page":"277","article-title":"Sous chef? Quickly teaching food preparation tasks to an autonomous robot Team","author":"Scheutz","year":"2025"},{"key":"B32","volume-title":"VerbNet: a broad-coverage, comprehensive verb lexicon","author":"Schuler","year":"2005"},{"key":"B33","first-page":"1229","article-title":"Practical task and motion planning for robotic food preparation","author":"Siburian","year":"2025"},{"key":"B34","doi-asserted-by":"publisher","first-page":"1328934","DOI":"10.3389\/frobt.2024.1328934","article-title":"Commonsense knowledge in cognitive robotics: a systematic literature review","volume":"11","author":"T\u00f6berg","year":"2024","journal-title":"Front. Robot. AI"},{"key":"B35","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1142\/S1793351X25410041","article-title":"Generation of robot manipulation plans using generative Large Language Models","volume":"19","author":"T\u00f6berg","year":"2025","journal-title":"Int. J. Semantic Comput."},{"key":"B36","first-page":"25","article-title":"Concurrence of word concepts in cooking recipe search","volume-title":"Proc. Of the 9th workshop on multimedia for cooking and eating activities in conjunction with the 2017 international Joint conference on artificial intelligence (CEA) (melbourne Australia: ACM)","author":"Yasukawa","year":"2017"}],"container-title":["Frontiers in Robotics and AI"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2025.1682031\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,29]],"date-time":"2025-10-29T07:08:01Z","timestamp":1761721681000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2025.1682031\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,29]]},"references-count":36,"alternative-id":["10.3389\/frobt.2025.1682031"],"URL":"https:\/\/doi.org\/10.3389\/frobt.2025.1682031","relation":{},"ISSN":["2296-9144"],"issn-type":[{"value":"2296-9144","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,29]]},"article-number":"1682031"}}