{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,22]],"date-time":"2026-10-22T16:53:21Z","timestamp":1792688001126,"version":"4.5.0"},"reference-count":80,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T00:00:00Z","timestamp":1790035200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Robot. AI"],"abstract":"<jats:p>\n                    We present a framework for assistive robot manipulation that addresses two fundamental challenges: efficient adaptation of large-scale models for scene affordance understanding and effective learning of robot actions by grounding the visual affordance. To tackle the first challenge, we adopt a parameter-efficient prompt tuning method, prepending learnable text prompts to a frozen vision model to predict affordances, while considering spatial and semantic relationships in multi-task scenarios. For the second challenge, we propose a flow matching method, representing a robot visuomotor policy as a conditional process of flowing random waypoints to desired robot actions. We introduce a real-world dataset with 10 tasks to evaluate our approach. Experiments show our prompt tuning method achieves competitive or superior performance to other finetuning protocols across data scales, while satisfying parameter efficiency. Furthermore, flow matching yields more stable training and faster inference compared to diffusion policy; specifically, 1-step flow matching achieves comparable accuracy to 16-step DDIM while reducing inference time by roughly 85%. Our framework seamlessly unifies high-level parameter-efficient affordance representation learning with low-level flow matching policies, illustrating how explicit affordances serve as effective spatial grounding for flow-based policies.\n                    <jats:ext-link>https:\/\/github.com\/HRI-EU\/flow_matching<\/jats:ext-link>\n                    .\n                  <\/jats:p>","DOI":"10.3389\/frobt.2026.1906861","type":"journal-article","created":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T10:26:31Z","timestamp":1790072791000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Affordance-based robot manipulation with flow matching"],"prefix":"10.3389","volume":"13","author":[{"given":"Fan","family":"Zhang","sequence":"first","affiliation":[{"name":"Honda Research Institute EU","place":["Offenbach, Germany"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael","family":"Gienger","sequence":"additional","affiliation":[{"name":"Honda Research Institute EU","place":["Offenbach, Germany"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,9,22]]},"reference":[{"key":"B1","author":"Albergo","year":"2022","journal-title":"Building Normalizing Flows with Stochastic InterpolantsarXiv Preprint arXiv:2209.15571"},{"key":"B2","first-page":"13778","article-title":"Affordances from human videos as a versatile representation for robotics","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Bahl","year":"2023"},{"key":"B3","article-title":"Track2act: predicting point tracks from internet videos enables diverse zero-shot robot manipulation","author":"Bharadhwaj","year":"2024","journal-title":"arXiv E-Prints"},{"key":"B4","volume-title":"A vision-language-action Flow Model for General Robot Control","author":"Black","year":"2024"},{"key":"B5","doi-asserted-by":"crossref","first-page":"51536","DOI":"10.52202\/075280-2244","article-title":"Information maximizing curriculum: a curriculum-based approach for learning versatile skills","volume":"36","author":"Blessing","year":"2023","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"B6","first-page":"5144","volume-title":"Riemannian Flow Matching Policy for Robot Motion Learning","author":"Braun","year":"2024"},{"key":"B7","volume-title":"Diffusion Policy: Visuomotor Policy Learning via Action Diffusion","author":"Chi","year":"2023"},{"key":"B8","article-title":"Learning robotic manipulation policies from point clouds with conditional flow matching","author":"Chisari","year":"2024"},{"key":"B9","doi-asserted-by":"crossref","first-page":"5882","DOI":"10.1109\/ICRA.2018.8460902","article-title":"Affordancenet: an end-to-end deep learning approach for object affordance detection","volume-title":"2018 IEEE International Conference on Robotics and Automation (ICRA)","author":"Do","year":"2018"},{"key":"B10","article-title":"Scaling rectified flow transformers for high-resolution image synthesis","volume-title":"Forty-First International Conference on Machine Learning","author":"Esser","year":"2024"},{"key":"B11","author":"Florence","year":"2018","journal-title":"Dense Object Nets: Learning Dense Visual Object Descriptors by and for Robotic ManipulationarXiv Preprint arXiv:180608756"},{"key":"B12","first-page":"158","article-title":"Implicit behavioral cloning","volume-title":"Conference on Robot Learning","author":"Florence","year":"2022"},{"key":"B13","volume-title":"One Step Diffusion via Shortcut Models","author":"Frans","year":"2024"},{"key":"B14","first-page":"13861","article-title":"Question aware vision transformer for multimodal reasoning","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Ganz","year":"2024"},{"key":"B15","doi-asserted-by":"publisher","first-page":"581","DOI":"10.1007\/s11263-023-01891-x","article-title":"Clip-adapter: better vision-language models with feature adapters","volume":"132","author":"Gao","year":"2024","journal-title":"Int. J. Comput. Vis."},{"key":"B16","doi-asserted-by":"crossref","DOI":"10.4324\/9781315740218","volume-title":"The Ecological Approach to Visual Perception: Classic Edition","author":"Gibson","year":"2014"},{"key":"B17","first-page":"694","article-title":"Rvt: robotic view transformer for 3d object manipulation","volume-title":"Conference on Robot Learning","author":"Goyal","year":"2023"},{"key":"B18","author":"Gupta","year":"2019","journal-title":"Relay Policy Learning: Solving long-horizon Tasks via Imitation and Reinforcement LearningarXiv Preprint arXiv:1910.11956"},{"key":"B19","first-page":"16399","article-title":"Towards general purpose vision systems: an end-to-end task-agnostic vision-language architecture","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Gupta","year":"2022"},{"key":"B20","first-page":"770","article-title":"Deep residual learning for image recognition","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"He","year":"2016"},{"key":"B21","author":"He","year":"2021","journal-title":"Towards a Unified View of parameter-efficient Transfer LearningarXiv Preprint arXiv:2110.04366"},{"key":"B22","first-page":"16000","article-title":"Masked autoencoders are scalable vision learners","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"He","year":"2022"},{"key":"B23","article-title":"Generative adversarial imitation learning","volume":"29","author":"Ho","year":"2016","journal-title":"Adv. Neural Information Processing Systems"},{"key":"B24","first-page":"6840","article-title":"Denoising diffusion probabilistic models","volume":"33","author":"Ho","year":"2020","journal-title":"Adv. Neural Information Processing Systems"},{"key":"B25","author":"Hu","year":"2021","journal-title":"Lora: Low-Rank Adaptation of Large Language ModelsarXiv Preprint arXiv:2106.09685"},{"key":"B26","author":"Hu","year":"2024","journal-title":"Adaflow: Imitation Learning with variance-adaptive Flow-based PoliciesarXiv Preprint arXiv:2402.04292"},{"key":"B27","volume-title":"Voxposer: Composable 3d Value Maps for Robotic Manipulation with Language Models","author":"Huang","year":"2023"},{"key":"B28","author":"Ingelhag","year":"2024","journal-title":"A Robotic Skill Learning System Built upon Diffusion Policies and Foundation ModelsarXiv Preprint arXiv:2403.16730"},{"key":"B29","first-page":"709","article-title":"Visual prompt tuning","volume-title":"European Conference on Computer Vision","author":"Jia","year":"2022"},{"key":"B30","article-title":"Vima: general robot manipulation with multimodal prompts","author":"Jiang","year":"2022","journal-title":"arXiv"},{"key":"B31","first-page":"1","article-title":"End-to-end training of deep visuomotor policies","volume":"17","author":"Levine","year":"2016","journal-title":"J. Mach. Learn. Res."},{"key":"B32","author":"Li","year":"2025","journal-title":"Hamster: Hierarchical Action Models for open-world Robot ManipulationarXiv Preprint arXiv:2502.05485"},{"key":"B33","doi-asserted-by":"crossref","first-page":"9493","DOI":"10.1109\/ICRA48891.2023.10160591","article-title":"Code as policies: language model programs for embodied control","volume-title":"2023 IEEE International Conference on Robotics and Automation (ICRA)","author":"Liang","year":"2023"},{"key":"B34","volume-title":"Flow Matching for Generative Modeling","author":"Lipman","year":"2022"},{"key":"B35","volume-title":"Rectified Flow: A Marginal Preserving Approach to Optimal Transport","author":"Liu","year":"2022"},{"key":"B36","author":"Liu","year":"2021","journal-title":"P-tuning v2: Prompt Tuning Can Be Comparable to fine-tuning Universally Across Scales and TasksarXiv Preprint arXiv:2110.07602"},{"key":"B37","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3560815","article-title":"Pre-train, prompt, and predict: a systematic survey of prompting methods in natural language processing","volume":"55","author":"Liu","year":"","journal-title":"ACM Comput. Surv."},{"key":"B38","article-title":"Tail: task-Specific adapters for imitation learning with large pretrained models","author":"Liu","year":"","journal-title":"arXiv Preprint arXiv:2310.05905"},{"key":"B39","author":"Liu","year":"2024","journal-title":"Moka: Open-Vocabulary Robotic Manipulation Through Mark-based Visual PromptingarXiv Preprint arXiv:2403.03174"},{"key":"B40","author":"Lu","year":"2024","journal-title":"Simplifying, Stabilizing and Scaling continuous-time Consistency ModelsarXiv Preprint arXiv:2410.11081"},{"key":"B41","doi-asserted-by":"crossref","first-page":"5775","DOI":"10.52202\/068431-0418","article-title":"Dpm-solver: a fast ode solver for diffusion probabilistic model sampling in around 10 steps","volume":"35","author":"Lu","year":"2022","journal-title":"Adv. Neural Information Processing Systems"},{"key":"B42","article-title":"Eureka: human-level reward design via coding large language models","author":"Ma","year":"2023","journal-title":"arXiv Preprint arXiv:2310, 12931"},{"key":"B43","doi-asserted-by":"publisher","first-page":"859","DOI":"10.3758\/s13415-018-00679-8","article-title":"No matter how: top-down effects of verbal and semantic category knowledge on early visual perception","volume":"19","author":"Maier","year":"2019","journal-title":"Cognitive, Affect. and Behav. Neurosci."},{"key":"B44","article-title":"What matters in learning from offline human demonstrations for robot manipulation","author":"Mandlekar","year":"2021","journal-title":"arXiv Preprint arXiv:2108.03298"},{"key":"B45","first-page":"132","article-title":"Kpam: keypoint affordances for category-level robotic manipulation","volume-title":"The International Symposium of Robotics Research","author":"Manuelli","year":"2019"},{"key":"B46","doi-asserted-by":"publisher","first-page":"1678567","DOI":"10.3389\/frobt.2025.1678567","article-title":"Imitation learning for legged robot locomotion: a survey","volume":"12","author":"Mirza","year":"2025","journal-title":"Front. Robotics AI"},{"key":"B47","article-title":"Step-by-step diffusion: an elementary tutorial","author":"Nakkiran","year":"2024","journal-title":"arXiv Preprint arXiv:2406.08929"},{"key":"B48","author":"Nasiriany","year":"2024","journal-title":"Rt-affordance: Affordances are Versatile Intermediate Representations for Robot ManipulationarXiv Preprint arXiv:241102704"},{"key":"B49","doi-asserted-by":"crossref","DOI":"10.1109\/IROS60139.2025.11246537","article-title":"Flowmp: learning motion fields for robot planning with conditional flow matching","author":"Nguyen","year":"2025"},{"key":"B50","article-title":"Open x-embodiment: robotic learning datasets and rt-x models","author":"Padalkar","year":"2023","journal-title":"arXiv Preprint arXiv:2310.08864"},{"key":"B51","doi-asserted-by":"crossref","DOI":"10.1609\/aaai.v32i1.11671","article-title":"Film: visual reasoning with a general conditioning layer","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Perez","year":"2018"},{"key":"B52","doi-asserted-by":"publisher","first-page":"355","DOI":"10.1561\/2200000073","article-title":"Computational optimal transport: with applications to data science","volume":"11","author":"Peyr\u00e9","year":"2019","journal-title":"Found. Trends\u00ae Mach. Learn."},{"key":"B53","author":"Prasad","year":"2024","journal-title":"Consistency Policy: Accelerated Visuomotor Policies via Consistency DistillationarXiv Preprint arXiv:240507503"},{"key":"B54","first-page":"7587","article-title":"Affordancellm: grounding affordance from vision language models","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Qian","year":"2024"},{"key":"B55","first-page":"8748","article-title":"Learning transferable visual models from natural language supervision","volume-title":"International Conference on Machine Learning","author":"Radford","year":"2021"},{"key":"B56","doi-asserted-by":"crossref","DOI":"10.15607\/RSS.2023.XIX.028","volume-title":"Goal-Conditioned Imitation Learning Using Score-based Diffusion Policies","author":"Reuss","year":"2023"},{"key":"B57","volume-title":"Diffusion Predictive Control with Constraints","author":"R\u00f6mer","year":"2024"},{"key":"B58","first-page":"234","article-title":"U-net: convolutional networks for biomedical image segmentation","volume-title":"Medical Image Computing and computer-assisted intervention\u2013MICCAI 2015: 18Th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18","author":"Ronneberger","year":"2015"},{"key":"B59","doi-asserted-by":"crossref","first-page":"528","DOI":"10.1109\/Humanoids58906.2024.10769838","article-title":"Flow matching imitation learning for multi-support manipulation","volume-title":"2024 IEEE-RAS 23rd International Conference on Humanoid Robots (Humanoids)","author":"Rouxel","year":"2024"},{"key":"B60","volume-title":"Lossless Adaptation of Pretrained Vision Models for Robotic Manipulation","author":"Sharma","year":"2023"},{"key":"B61","first-page":"19840","article-title":"Visual prompt tuning for generative transfer learning","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Sohn","year":"2023"},{"key":"B62","volume-title":"Denoising Diffusion Implicit Models","author":"Song","year":""},{"key":"B63","author":"Song","year":"","journal-title":"Score-Based Generative Modeling Through Stochastic Differential EquationsarXiv Preprint arXiv:201113456"},{"key":"B64","author":"Sridhar","year":"2023","journal-title":"Memory-Consistent Neural Networks for Imitation LearningarXiv Preprint arXiv:2310.06171"},{"key":"B65","author":"Stone","year":"2023","journal-title":"Open-World Object Manipulation Using Pre-trained vision-language ModelsarXiv Preprint arXiv:230300905"},{"key":"B66","first-page":"9130","volume-title":"To Help or Not to Help: Llm-Based Attentive Support for human-robot Group Interactions","author":"Tanneberg","year":"2024"},{"key":"B67","article-title":"Visualizing data using t-sne","volume":"9","author":"Van der Maaten","year":"2008","journal-title":"J. Machine Learning Research"},{"key":"B68","volume-title":"Topics in Optimal Transportation","author":"Villani","year":"2021"},{"key":"B69","author":"Wang","year":"2023","journal-title":"Prompt a Robot to Walk with Large Language ModelsarXiv Preprint arXiv:2309.09969"},{"key":"B70","article-title":"Masked visual pre-training for motor control","author":"Xiao","year":"2022","journal-title":"arXiv Preprint arXiv:2203.06173"},{"key":"B71","doi-asserted-by":"crossref","DOI":"10.1109\/LRA.2024.3477169","article-title":"Conditional neural expert processes for learning movement primitives from demonstration","author":"Yildirim","year":"2024","journal-title":"IEEE Robotics Automation Lett."},{"key":"B72","article-title":"Robopoint: a vision-language model for spatial affordance prediction for robotics","author":"Yuan","year":"2024","journal-title":"arXiv Preprint arXiv:2406.10721"},{"key":"B73","first-page":"726","article-title":"Transporter networks: rearranging the visual world for robotic manipulation","volume-title":"Conference on Robot Learning","author":"Zeng","year":"2021"},{"key":"B74","doi-asserted-by":"publisher","first-page":"3143","DOI":"10.3390\/s25103143","article-title":"Optimization of smooth trajectories for two-wheel differential robots under kinematic constraints using clothoid curves","volume":"25","author":"Zeng","year":"2025","journal-title":"Sensors"},{"key":"B75","doi-asserted-by":"publisher","first-page":"eabm6010","DOI":"10.1126\/scirobotics.abm6010","article-title":"Learning garment manipulation policies toward robot-assisted dressing","volume":"7","author":"Zhang","year":"2022","journal-title":"Sci. Robotics"},{"key":"B76","doi-asserted-by":"crossref","first-page":"49842","DOI":"10.52202\/075280-2170","article-title":"Unipc: a unified predictor-corrector framework for fast sampling of diffusion models","volume":"36","author":"Zhao","year":"2023","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"B77","article-title":"Guided flows for generative modeling and decision making","author":"Zheng","year":"2023","journal-title":"arXiv Preprint arXiv:2311.13443"},{"key":"B78","doi-asserted-by":"publisher","first-page":"10270","DOI":"10.1609\/aaai.v38i9.28893","article-title":"Dancemvp: self-supervised learning for multi-task primitive-based dance performance assessment via transformer text prompting","volume":"38","author":"Zhong","year":"2024","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"B79","first-page":"1","article-title":"Contrastive self-supervised learning for automated multi-modal dance performance assessment","volume-title":"ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","author":"Zhong","year":"2023"},{"key":"B80","first-page":"9516","article-title":"Visual prompt multi-modal tracking","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Zhu","year":"2023"}],"container-title":["Frontiers in Robotics and AI"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2026.1906861\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T10:26:43Z","timestamp":1790072803000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2026.1906861\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,22]]},"references-count":80,"alternative-id":["10.3389\/frobt.2026.1906861"],"URL":"https:\/\/doi.org\/10.3389\/frobt.2026.1906861","relation":{},"ISSN":["2296-9144"],"issn-type":[{"value":"2296-9144","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,22]]},"article-number":"1906861"}}