{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,14]],"date-time":"2026-03-14T20:59:34Z","timestamp":1773521974321,"version":"3.50.1"},"reference-count":34,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,4,29]],"date-time":"2024-04-29T00:00:00Z","timestamp":1714348800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurorobot."],"abstract":"<jats:sec><jats:title>Introduction<\/jats:title><jats:p>Robotic assembly tasks require precise manipulation and coordination, often necessitating advanced learning techniques to achieve efficient and effective performance. While residual reinforcement learning with a base policy has shown promise in this domain, existing base policy approaches often rely on hand-designed full-state features and policies or extensive demonstrations, limiting their applicability in semi-structured environments.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>In this study, we propose an innovative Object-Embodiment-Centric Imitation and Residual Reinforcement Learning (OEC-IRRL) approach that leverages an object-embodiment-centric (OEC) task representation to integrate vision models with imitation and residual learning. By utilizing a single demonstration and minimizing interactions with the environment, our method aims to enhance learning efficiency and effectiveness. The proposed method involves three key steps: creating an object-embodiment-centric task representation, employing imitation learning for a base policy using via-point movement primitives for generalization to different settings, and utilizing residual RL for uncertainty-aware policy refinement during the assembly phase.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>Through a series of comprehensive experiments, we investigate the impact of the OEC task representation on base and residual policy learning and demonstrate the effectiveness of the method in semi-structured environments. Our results indicate that the approach, requiring only a single demonstration and less than 1.2 h of interaction, improves success rates by 46% and reduces assembly time by 25%.<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussion<\/jats:title><jats:p>This research presents a promising avenue for robotic assembly tasks, providing a viable solution without the need for specialized expertise or custom fixtures.<\/jats:p><\/jats:sec>","DOI":"10.3389\/fnbot.2024.1355170","type":"journal-article","created":{"date-parts":[[2024,4,29]],"date-time":"2024-04-29T04:37:08Z","timestamp":1714365428000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Extended residual learning with one-shot imitation learning for robotic assembly in semi-structured environment"],"prefix":"10.3389","volume":"18","author":[{"given":"Chuang","family":"Wang","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chupeng","family":"Su","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Baozheng","family":"Sun","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gang","family":"Chen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Longhan","family":"Xie","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2024,4,29]]},"reference":[{"key":"B1","article-title":"Do as I can, not as I say: grounding language in robotic affordances","author":"Ahn","year":"2022","journal-title":"arXiv preprint arXiv:2204.01691"},{"key":"B2","article-title":"Residual reinforcement learning from demonstrations","author":"Alakuijala","year":"2021","journal-title":"arXiv preprint arXiv:2106.08050"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.3389\/fnbot.2023.1239581","article-title":"Neurorobotic reinforcement learning for domains with parametrical uncertainty","author":"Amaya","year":"2023","journal-title":"Front. Neurorob"},{"key":"B4","doi-asserted-by":"publisher","first-page":"5709","DOI":"10.1109\/LRA.2020.3010739","article-title":"Learning force control for contact-rich manipulation tasks with rigid position-controlled robots","volume":"5","author":"Beltran-Hernandez","year":"2020","journal-title":"IEEE Robot. Autom. Lett"},{"key":"B5","doi-asserted-by":"crossref","first-page":"405","DOI":"10.1109\/Humanoids53995.2022.10000148","article-title":"\u201cAdapting object-centric probabilistic movement primitives with residual reinforcement learning,\u201d","volume-title":"2022 IEEE-RAS 21st International Conference on Humanoid Robots (Humanoids)","author":"Carvalho","year":"2022"},{"key":"B6","doi-asserted-by":"publisher","first-page":"4488","DOI":"10.1109\/LRA.2022.3150024","article-title":"Residual learning from demonstration: adapting DMPS for contact-rich manipulation","volume":"7","author":"Davchev","year":"2022","journal-title":"IEEE Robot. Autom. Lett"},{"key":"B7","doi-asserted-by":"publisher","first-page":"3656","DOI":"10.1109\/LRA.2022.3146916","article-title":"Meta-residual policy learning: zero-trial robot skill adaptation via knowledge fusion","volume":"7","author":"Hao","year":"2022","journal-title":"IEEE Robot. Autom. Lett"},{"key":"B8","article-title":"Robotic manipulation datasets for offline compositional reinforcement learning","author":"Hussing","year":"2023","journal-title":"arXiv preprint arXiv:2307.07091"},{"key":"B9","doi-asserted-by":"publisher","first-page":"1280773","DOI":"10.3389\/fnbot.2023.1280773","article-title":"Vision-force-fused curriculum learning for robotic contact-rich assembly tasks","volume":"17","author":"Jin","year":"2023","journal-title":"Front. Neurorob"},{"key":"B10","doi-asserted-by":"crossref","first-page":"6023","DOI":"10.1109\/ICRA.2019.8794127","article-title":"\u201cResidual reinforcement learning for robot control,\u201d","volume-title":"2019 International Conference on Robotics and Automation (ICRA)","author":"Johannink","year":"2019"},{"key":"B11","doi-asserted-by":"publisher","first-page":"149","DOI":"10.1016\/j.rcim.2018.09.008","article-title":"Assembly process monitoring algorithm using force data and deformation data","volume":"56","author":"Lee","year":"2019","journal-title":"Robot. Comput. Integr. Manufact"},{"key":"B12","doi-asserted-by":"crossref","first-page":"7505","DOI":"10.1109\/ICRA40945.2020.9197125","article-title":"\u201cGuided uncertainty-aware policy optimization: combining learning and model-based strategies for sample-efficient policy learning,\u201d","volume-title":"2020 IEEE International Conference on Robotics and Automation (ICRA)","author":"Lee","year":"2020"},{"key":"B13","doi-asserted-by":"crossref","first-page":"6343","DOI":"10.1109\/ICRA48506.2021.9560986","article-title":"\u201cIkea furniture assembly environment for long-horizon complex manipulation tasks,\u201d","volume-title":"2021 IEEE International Conference on Robotics and Automation (ICRA)","author":"Lee","year":"2021"},{"key":"B14","article-title":"\u201cRearrangement planning for general part assembly,\u201d","author":"Li","year":"2023","journal-title":"7th Annual Conference on Robot Learning"},{"key":"B15","article-title":"Robust multi-modal policies for industrial assembly via reinforcement learning and demonstrations: a large-scale study","author":"Luo","year":"2021","journal-title":"arXiv preprint arXiv:2103.11512"},{"key":"B16","doi-asserted-by":"publisher","first-page":"4492","DOI":"10.1109\/TII.2020.3020065","article-title":"Efficient insertion control for precision assembly based on demonstration learning and reinforcement learning","volume":"17","author":"Ma","year":"2020","journal-title":"IEEE Trans. Ind. Inform"},{"key":"B17","article-title":"Mimicgen: A data generation system for scalable robot learning using human demonstrations","author":"Mandlekar","year":"2023","journal-title":"arXiv preprint arXiv:2310.17596"},{"key":"B18","doi-asserted-by":"publisher","first-page":"105164","DOI":"10.1016\/j.engappai.2022.105164","article-title":"Pose estimation and robotic insertion tasks based on yolo and layout features","volume":"114","author":"Mou","year":"2022","journal-title":"Eng. Applic. Artif. Intell"},{"key":"B19","doi-asserted-by":"crossref","first-page":"6292","DOI":"10.1109\/ICRA.2018.8463162","article-title":"\u201cOvercoming exploration in reinforcement learning with demonstrations,\u201d","volume-title":"2018 IEEE International Conference on Robotics and Automation (ICRA)","author":"Nair","year":"2018"},{"key":"B20","doi-asserted-by":"crossref","first-page":"2383","DOI":"10.1109\/IROS51168.2021.9636176","article-title":"\u201cResidual feedback learning for contact-rich manipulation tasks with uncertainty,\u201d","volume-title":"2021 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)","author":"Ranjbar","year":"2021"},{"key":"B21","doi-asserted-by":"crossref","first-page":"5548","DOI":"10.1109\/IROS45743.2020.9341714","article-title":"\u201cDeep reinforcement learning for industrial insertion tasks with visual inputs and natural rewards,\u201d","volume-title":"2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)","author":"Schoettler","year":"2020"},{"key":"B22","doi-asserted-by":"publisher","first-page":"185","DOI":"10.1016\/j.robot.2019.06.009","article-title":"An introductory review of active compliant control","volume":"119","author":"Schumacher","year":"2019","journal-title":"Robot. Auton. Syst"},{"key":"B23","article-title":"Waypoint-based imitation learning for robotic manipulation","author":"Shi","year":"2023","journal-title":"arXiv preprint arXiv:2307.14326"},{"key":"B24","first-page":"765","article-title":"\u201cProactive action visual residual reinforcement learning for contact-rich tasks using a torque-controlled robot,\u201d","volume-title":"2021 IEEE International Conference on Robotics and Automation (ICRA)","author":"Shi","year":""},{"key":"B25","first-page":"1062","article-title":"\u201cCombining learning from demonstration with learning by exploration to facilitate contact-rich tasks,\u201d","volume-title":"2021 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)","author":"Shi","year":""},{"key":"B26","article-title":"Residual policy learning","author":"Silver","year":"2018","journal-title":"arXiv preprint arXiv:1812.06298"},{"key":"B27","doi-asserted-by":"publisher","first-page":"552","DOI":"10.1080\/01691864.2015.1130172","article-title":"Guidance algorithm for complex-shape peg-in-hole strategy based on geometrical information and force control","volume":"30","author":"Song","year":"2016","journal-title":"Adv. Robot"},{"key":"B28","doi-asserted-by":"crossref","first-page":"421","DOI":"10.1109\/ICRA.2016.7487162","article-title":"\u201cA framework for fine robotic assembly,\u201d","volume-title":"2016 IEEE International Conference on Robotics and Automation (ICRA)","author":"Su\u00e1rez-Ruiz","year":"2016"},{"key":"B29","article-title":"Mimicplay: long-horizon imitation learning by watching human play","author":"Wang","year":"2023","journal-title":"arXiv preprint arXiv:2302.12422"},{"key":"B30","doi-asserted-by":"publisher","first-page":"777363","DOI":"10.3389\/frobt.2021.777363","article-title":"An adaptive imitation learning framework for robotic complex contact-rich insertion tasks","volume":"8","author":"Wang","year":"2022","journal-title":"Front. Robot. AI"},{"key":"B31","article-title":"Efficient skill acquisition for complex manipulation tasks in obstructed environments","author":"Yamada","year":"2023","journal-title":"arXiv preprint arXiv:2303.03365"},{"key":"B32","doi-asserted-by":"publisher","first-page":"1320251","DOI":"10.3389\/fnbot.2023.1320251","article-title":"Peg-in-hole assembly skill imitation learning method based on promps under task geometric representation","volume":"17","author":"Zang","year":"2023","journal-title":"Front. Neurorob"},{"key":"B33","doi-asserted-by":"publisher","DOI":"10.1109\/TIE.2023.3299051","article-title":"\u201cA learning-based two-stage method for submillimeter insertion tasks with only visual inputs,\u201d","author":"Zhao","year":"2023","journal-title":"IEEE Transactions on Industrial Electronics"},{"key":"B34","doi-asserted-by":"crossref","first-page":"4301","DOI":"10.1109\/IROS40897.2019.8968586","article-title":"\u201cLearning via-point movement primitives with inter-and extrapolation capabilities,\u201d","volume-title":"2019 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)","author":"Zhou","year":"2019"}],"container-title":["Frontiers in Neurorobotics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2024.1355170\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,29]],"date-time":"2024-04-29T04:37:18Z","timestamp":1714365438000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2024.1355170\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,29]]},"references-count":34,"alternative-id":["10.3389\/fnbot.2024.1355170"],"URL":"https:\/\/doi.org\/10.3389\/fnbot.2024.1355170","relation":{},"ISSN":["1662-5218"],"issn-type":[{"value":"1662-5218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,29]]},"article-number":"1355170"}}