{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,20]],"date-time":"2026-07-20T11:56:24Z","timestamp":1784548584504,"version":"3.55.0"},"reference-count":45,"publisher":"Wiley","issue":"1","license":[{"start":{"date-parts":[[2021,9,2]],"date-time":"2021-09-02T00:00:00Z","timestamp":1630540800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["advanced.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Advanced Intelligent Systems"],"published-print":{"date-parts":[[2022,1]]},"abstract":"<jats:sec><jats:label\/><jats:p>Adapting to uncertainties is essential yet challenging for robots while conducting assembly tasks in real\u2010world scenarios. Reinforcement learning (RL) methods provide a promising solution for these cases. However, training robots with RL can be a data\u2010extensive, time\u2010consuming, and potentially unsafe process. In contrast, classical control strategies can have near\u2010optimal performance without training and be certifiably safe. However, this is achieved at the cost of assuming that the environment is known up to small uncertainties. Herein, an architecture aiming at getting the best out of the two worlds, by combining RL and classical strategies so that each one deals with the right portion of the assembly problem, is proposed. A time\u2010varying weighted sum combines a recurrent RL method with a nominal strategy. The output serves as the reference for a task space impedance controller. The proposed approach can learn to insert an object in a frame within a few minutes of real\u2010world training. A success rate of 94% in the presence of considerable uncertainties is observed. Furthermore, the approach is robust to changes in the experimental setup and task, even when no retrain is performed. For example, the same policy achieves a success rate of 85% when the object properties change.<\/jats:p><\/jats:sec>","DOI":"10.1002\/aisy.202100095","type":"journal-article","created":{"date-parts":[[2021,9,2]],"date-time":"2021-09-02T09:31:27Z","timestamp":1630575087000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":27,"title":["Learning Assembly Tasks in a Few Minutes by Combining Impedance Control and Residual Recurrent Reinforcement Learning"],"prefix":"10.1002","volume":"4","author":[{"given":"Padmaja","family":"Kulkarni","sequence":"first","affiliation":[{"name":"Department of Cognitive Robotics Delft University of Technology  Delft 2628 CD The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jens","family":"Kober","sequence":"additional","affiliation":[{"name":"Department of Cognitive Robotics Delft University of Technology  Delft 2628 CD The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Robert","family":"Babu\u0161ka","sequence":"additional","affiliation":[{"name":"Department of Cognitive Robotics Delft University of Technology  Delft 2628 CD The Netherlands"},{"name":"Czech Institute of Informatics Robotics and Cybernetics Czech Technical University in Prague  Prague 160 00 Czech Republic"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1067-1134","authenticated-orcid":false,"given":"Cosimo","family":"Della Santina","sequence":"additional","affiliation":[{"name":"Department of Cognitive Robotics Delft University of Technology  Delft 2628 CD The Netherlands"},{"name":"Institute of Robotics and Mechatronics German Aerospace Center (DLR)  Oberpfaffenhofen 82234 Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2021,9,2]]},"reference":[{"key":"e_1_2_9_2_1","doi-asserted-by":"crossref","unstructured":"P.Zou Q.Zhu J.Wu J.Jin inProc. \u2013 2019 Chinese Automation Congress CAC 2019 IEEE Piscataway NJ2019 pp.3269\u20133273.","DOI":"10.1109\/CAC48633.2019.8997006"},{"key":"e_1_2_9_3_1","unstructured":"A.Bicchi V.Kumar inProc. 2000 ICRA. Millennium Conf. IEEE Int. Conf. on Robotics and Automation. Symp. Proc.(Cat. No.00CH37065) IEEE Piscataway NJ2020 p.1."},{"key":"e_1_2_9_4_1","doi-asserted-by":"crossref","first-page":"6446","DOI":"10.1126\/science.aat8414","volume":"364","author":"Billard A.","year":"2019","journal-title":"Science"},{"key":"e_1_2_9_5_1","unstructured":"L.Manuelli W.Gao P.Florence R.Tedrake kPAM: KeyPoint Affordances for Category-Level Robotic Manipulation arXiv 2019."},{"key":"e_1_2_9_6_1","doi-asserted-by":"crossref","unstructured":"H.Bruyninckyx S.Dutre J.De Schutter inProc. \u2013 IEEE Int. Conf. on Robotics and Automation vol.2 IEEE Piscataway NJ1995 pp.1919\u20131924.","DOI":"10.1109\/ROBOT.1995.525545"},{"key":"e_1_2_9_7_1","doi-asserted-by":"crossref","unstructured":"F.Dietrich D.Buchholz F.Wobbe F.Sowinski A.Raatz W.Schumacher F. M.Wahl inIEEE\/RSJ 2010 Int. Conf. on Intelligent Robots and Systems IROS 2010 \u2013 Conf. Proc. IEEE Piscataway NJ2010 pp.2313\u20132318.","DOI":"10.1109\/IROS.2010.5652704"},{"key":"e_1_2_9_8_1","doi-asserted-by":"crossref","first-page":"1238","DOI":"10.1177\/0278364913495721","volume":"32","author":"Kober J.","year":"2013","journal-title":"Int. J. Robotics Res."},{"key":"e_1_2_9_9_1","doi-asserted-by":"crossref","first-page":"398","DOI":"10.1109\/TASE.2021.3064065","volume":"18","author":"Hofer S.","year":"2021","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"e_1_2_9_10_1","doi-asserted-by":"crossref","unstructured":"G.Schoettler A.Nair J.Luo S.Bahl J. A.Ojea E.Solowjow S.Levine inIEEE Int. Conf. on Intelligent Robots and Systems IEEE Piscataway NJ2019 pp.5548\u20135555.","DOI":"10.1109\/IROS45743.2020.9341714"},{"key":"e_1_2_9_11_1","doi-asserted-by":"crossref","first-page":"236","DOI":"10.1016\/j.engappai.2018.11.006","volume":"78","author":"Pane Y. P.","year":"2019","journal-title":"Eng. Appl. Artif. Intell."},{"key":"e_1_2_9_12_1","doi-asserted-by":"crossref","unstructured":"S.Dong D. K.Jha D.Romeres S.Kim D.Nikovski A.Rodriguez Tactile-RL for Insertion: Generalization to Objects of Unknown Geometry arxiv 2021.","DOI":"10.1109\/ICRA48506.2021.9561646"},{"key":"e_1_2_9_13_1","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1177\/0278364919887447","volume":"39","author":"Andrychowicz O. M.","year":"2020","journal-title":"Int. J. Robotics Res."},{"key":"e_1_2_9_14_1","doi-asserted-by":"crossref","unstructured":"S.Gu E.Holly T.Lillicrap S.Levine inProc. \u2013 IEEE Int. Conf. on Robotics and Automation IEEE Piscataway NJ2016 pp.3389\u20133396.","DOI":"10.1109\/ICRA.2017.7989385"},{"key":"e_1_2_9_15_1","first-page":"5049","volume":"2017","author":"Andrychowicz M.","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_2_9_16_1","first-page":"1","volume":"22","author":"Kroemer O.","year":"2021","journal-title":"J. Mach. Learn. Res."},{"key":"e_1_2_9_17_1","doi-asserted-by":"crossref","first-page":"698","DOI":"10.1177\/0278364920987859","volume":"40","author":"Ibarz J.","year":"2021","journal-title":"Int. J. Robot. Res."},{"key":"e_1_2_9_18_1","first-page":"1","volume":"10","author":"Beltran-Hernandez C. C.","year":"2020","journal-title":"Appl. Sci. (Switzerland)"},{"key":"e_1_2_9_19_1","unstructured":"T.Inoue G.De Magistris A.Munawar T.Yokoya R.Tachibana inIEEE Int. Conf. on Intelligent Robots and Systems Vol.2017 IEEE Piscataway NJ2017 pp.819\u2013825."},{"key":"e_1_2_9_20_1","doi-asserted-by":"crossref","unstructured":"Y.Fan J.Luo M.Tomizuka inProc. \u2013 IEEE Int. Conf. on Robotics and Automation IEEE Piscataway NJ2018 p.811.","DOI":"10.1109\/ICRA.2019.8793659"},{"key":"e_1_2_9_21_1","doi-asserted-by":"crossref","unstructured":"J.Luo E.Solowjow C.Wen J. A.Ojea A. M.Agogino inIEEE Int. Conf. on Intelligent Robots and Systems IEEE Piscataway NJ2018 pp.2062\u20132069.","DOI":"10.1109\/IROS.2018.8594353"},{"key":"e_1_2_9_22_1","doi-asserted-by":"crossref","unstructured":"G.Thomas M.Chien A.Tamar J. A.Ojea P.Abbeel inProc. \u2013 IEEE Int. Conf. on Robotics and Automation IEEE Piscataway NJ2018 pp.3524\u20133531.","DOI":"10.1109\/ICRA.2018.8460696"},{"key":"e_1_2_9_23_1","doi-asserted-by":"crossref","unstructured":"Y.Wu M.Mozifian F.Shkurti Shaping Rewards for Reinforcement Learning with Imperfect Demonstrations using Generative Models arXiv 2020.","DOI":"10.1109\/ICRA48506.2021.9561333"},{"key":"e_1_2_9_24_1","doi-asserted-by":"crossref","first-page":"17","DOI":"10.3390\/robotics7020017","volume":"7","author":"Zhu Z.","year":"2018","journal-title":"Robotics"},{"key":"e_1_2_9_25_1","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1007\/s10514-018-9725-6","volume":"43","author":"Kyrarini M.","year":"2019","journal-title":"Autonomous Robots"},{"key":"e_1_2_9_26_1","first-page":"1","volume":"20","author":"Ding G.","year":"2020","journal-title":"Sensors (Switzerland)"},{"key":"e_1_2_9_27_1","unstructured":"K.Kronander E.Burdet A.Billard inWorkshop on Human-Robot Interaction for Industrial Manufacturing Robotics Science and Systems2014."},{"key":"e_1_2_9_28_1","doi-asserted-by":"crossref","unstructured":"L.Rozo M.Guo A. G.Kupcsik M.Todescato P.Schillinger M.Giftthaler M.Ochs M.Spies N.Waniek P.Kesper M.B\u00fcerger inIEEE Int. Conf. on Intelligent Robots and Systems IEEE Piscataway NJ2020 pp.9072\u20139079.","DOI":"10.1109\/IROS45743.2020.9341570"},{"key":"e_1_2_9_29_1","unstructured":"J. F.Broenink M. L. J.Tiernego inProc. 8th European Simulation Symp. Simulation in Industry Society for Computer Simulation International Genoa Italy1996 pp.504\u2013508."},{"key":"e_1_2_9_30_1","unstructured":"N.Hogan inProc. of the American Control Conf. Vol.1 IEEE Piscataway NJ1984 pp.304\u2013313."},{"key":"e_1_2_9_31_1","doi-asserted-by":"crossref","unstructured":"T.Johannink S.Bahl A.Nair J.Luo A.Kumar M.Loskyll J. A.Ojea E.Solowjow S.Levine inProc. \u2013 IEEE Int. Conf. on Robotics and Automation IEEE Piscataway NJ2018 p.6023.","DOI":"10.1109\/ICRA.2019.8794127"},{"key":"e_1_2_9_32_1","first-page":"1587","volume-title":"Proceedings of the 35th International Conference on Machine Learning","author":"Fujimoto S.","year":"2018"},{"key":"e_1_2_9_33_1","doi-asserted-by":"crossref","unstructured":"D.Wierstra A.Foerster J.Peters J.Schmidhuber inLecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) Vol.4668 LNCS Springer Verlag Berlin2007 pp.697\u2013706.","DOI":"10.1007\/978-3-540-74690-4_71"},{"key":"e_1_2_9_34_1","unstructured":"N.Heess J. J.Hunt T. P.Lillicrap D.Silver Memory-based control with recurrent neural networks arXiv2015."},{"key":"e_1_2_9_35_1","doi-asserted-by":"crossref","unstructured":"H.Dang P. K.Allen inProc. \u2013 IEEE Int. Conf. on Robotics and Automation IEEE Piscataway NJ2012 pp.2392\u20132397.","DOI":"10.1109\/ICRA.2012.6224754"},{"key":"e_1_2_9_36_1","unstructured":"J.Xu Z.Hou Z.Liu H.Qiao Compare Contact Model-based Control and Contact Model-free Learning: A Survey of Robotic Peg-in-hole Assembly Strategies arXiv2019."},{"key":"e_1_2_9_37_1","doi-asserted-by":"crossref","first-page":"479","DOI":"10.1163\/156855305323383767","volume":"19","author":"Lefebvre T.","year":"2005","journal-title":"Adv. Robotics"},{"key":"e_1_2_9_38_1","volume-title":"Cartesian Impedance Control of Redundant and Flexible-Joint Robots","author":"Ott C.","year":"2008"},{"key":"e_1_2_9_39_1","unstructured":"Libfranka https:\/\/github.com\/frankaemika\/libfranka\/(accessed: May 2020)."},{"key":"e_1_2_9_40_1","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1016\/j.artint.2015.04.001","volume":"247","author":"Pajarinen J.","year":"2017","journal-title":"Artif. Intell."},{"key":"e_1_2_9_41_1","doi-asserted-by":"crossref","unstructured":"S. P.Singh T.Jaakkola M. I.Jordan InMachine Learning Proc. 1994 Elsevier Amsterdam1994 pp.284\u2013292.","DOI":"10.1016\/B978-1-55860-335-6.50042-8"},{"key":"e_1_2_9_42_1","doi-asserted-by":"crossref","first-page":"761","DOI":"10.1080\/08839510802170538","volume":"22","author":"Dung L. T.","year":"2008","journal-title":"Appl. Artif. Intell."},{"key":"e_1_2_9_43_1","volume-title":"AAAI Fall Symposium on Sequential Decision Making for Intelligent Agents (AAAI\u2010SDMIA15)","author":"Hausknecht M.","year":"2015"},{"key":"e_1_2_9_44_1","unstructured":"Franka Imeka Robot https:\/\/www.franka.de\/(accessed: May 2020)."},{"key":"e_1_2_9_45_1","unstructured":"S.Guadarrama A.Korattikara O.Ramirez P.Castro E.Holly S.Fishman K.Wang E.Gonina N.Wu E.Kokiopoulou L.Sbaiz J.Smith G.Bart\u00f3k J.Berent C.Harris V.Vanhoucke E.Brevdo TF-Agents: A library for Reinforcement Learning in TensorFlow 2018 https:\/\/github.com\/tensorflow\/agents(accessed: April 2021)."},{"key":"e_1_2_9_46_1","doi-asserted-by":"crossref","first-page":"1479","DOI":"10.1109\/TRO.2016.2602374","volume":"32","author":"Varava A.","year":"2016","journal-title":"IEEE Trans. Robotics"}],"container-title":["Advanced Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/aisy.202100095","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/aisy.202100095","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/advanced.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/aisy.202100095","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,7]],"date-time":"2025-10-07T23:39:50Z","timestamp":1759880390000},"score":1,"resource":{"primary":{"URL":"https:\/\/advanced.onlinelibrary.wiley.com\/doi\/10.1002\/aisy.202100095"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,2]]},"references-count":45,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,1]]}},"alternative-id":["10.1002\/aisy.202100095"],"URL":"https:\/\/doi.org\/10.1002\/aisy.202100095","archive":["Portico"],"relation":{},"ISSN":["2640-4567","2640-4567"],"issn-type":[{"value":"2640-4567","type":"print"},{"value":"2640-4567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,2]]},"assertion":[{"value":"2021-05-15","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-09-02","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"2100095"}}