{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T16:54:09Z","timestamp":1790787249171,"version":"4.1.0"},"reference-count":86,"publisher":"Springer Science and Business Media LLC","issue":"8","license":[{"start":{"date-parts":[[2023,10,17]],"date-time":"2023-10-17T00:00:00Z","timestamp":1697500800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,10,17]],"date-time":"2023-10-17T00:00:00Z","timestamp":1697500800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001710","name":"Branco Weiss Fellowship - Society in Science","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100001710","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Auton Robot"],"published-print":{"date-parts":[[2023,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We rethink the traditional reinforcement learning approach, which is based on optimizing over feedback policies, and propose a new framework that optimizes over feedforward inputs instead. This not only mitigates the risk of destabilizing the system during training but also reduces the bulk of the learning to a supervised learning task. As a result, efficient and well-understood supervised learning techniques can be applied and are tuned using a validation data set. The labels are generated with a variant of iterative learning control, which also includes prior knowledge about the underlying dynamics. Our framework is applied for intercepting and returning ping-pong balls that are played to a four-degrees-of-freedom robotic arm in real-world experiments. The robot arm is driven by pneumatic artificial muscles, which makes the control and learning tasks challenging. We highlight the potential of our framework by comparing it to a reinforcement learning approach that optimizes over feedback policies. We find that our framework achieves a higher success rate for the returns (<jats:inline-formula><jats:alternatives><jats:tex-math>$$100\\%$$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                  <mml:mrow>\n                    <mml:mn>100<\/mml:mn>\n                    <mml:mo>%<\/mml:mo>\n                  <\/mml:mrow>\n                <\/mml:math><\/jats:alternatives><\/jats:inline-formula> vs. <jats:inline-formula><jats:alternatives><jats:tex-math>$$96\\%$$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                  <mml:mrow>\n                    <mml:mn>96<\/mml:mn>\n                    <mml:mo>%<\/mml:mo>\n                  <\/mml:mrow>\n                <\/mml:math><\/jats:alternatives><\/jats:inline-formula>, on 107 consecutive trials, see <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/youtu.be\/kR9jowEH7PY\">https:\/\/youtu.be\/kR9jowEH7PY<\/jats:ext-link>) while requiring only about one tenth of the samples during training. We also find that our approach is able to deal with a variant of different incoming trajectories.\n<\/jats:p>","DOI":"10.1007\/s10514-023-10140-6","type":"journal-article","created":{"date-parts":[[2023,10,17]],"date-time":"2023-10-17T08:02:08Z","timestamp":1697529728000},"page":"1387-1403","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Reinforcement learning with model-based feedforward inputs for robotic table tennis"],"prefix":"10.1007","volume":"47","author":[{"given":"Hao","family":"Ma","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dieter","family":"B\u00fcchler","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bernhard","family":"Sch\u00f6lkopf","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael","family":"Muehlebach","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,10,17]]},"reference":[{"key":"10140_CR1","unstructured":"Achterhold, J., Tobuschat, P., Ma, H., et\u00a0al. (2023). Black-box vs. gray-box: A case study on learning table tennis ball trajectory prediction with spin and impacts. In: Proceedings of the learning for dynamics and control conference, pp. 878\u2013890."},{"issue":"6","key":"10140_CR2","doi-asserted-by":"publisher","first-page":"1099","DOI":"10.1109\/TSMCC.2007.905759","volume":"37","author":"HS Ahn","year":"2007","unstructured":"Ahn, H. S., Chen, Y., & Moore, K. L. (2007). Iterative learning control: Brief survey and categorization. IEEE Transactions on Systems, Man, and Cybernetics, Part C, 37(6), 1099\u20131121.","journal-title":"IEEE Transactions on Systems, Man, and Cybernetics, Part C"},{"key":"10140_CR3","unstructured":"Alet, F., Lozano-Perez, T., & Kaelbling, L. P. (2018). Modular meta-learning. In: Proceedings of the conference on robot learning, pp. 856\u2013868"},{"key":"10140_CR4","unstructured":"Arimoto, S., Kawamura, S., Miyazaki, F. (1986). Convergence, stability and robustness of learning control schemes for robot manipulators. In: Proceedings of the international symposium on robot manipulators on recent trends in robotics: Modeling, control and education, pp. 307\u2013316"},{"issue":"4","key":"10140_CR5","doi-asserted-by":"publisher","first-page":"1835","DOI":"10.1109\/TMECH.2016.2558292","volume":"21","author":"DX Ba","year":"2016","unstructured":"Ba, D. X., Dinh, T. Q., & Ahn, K. K. (2016). An integrated intelligent nonlinear control method for a pneumatic artificial muscle. IEEE\/ASME Transactions on Mechatronics, 21(4), 1835\u20131845.","journal-title":"IEEE\/ASME Transactions on Mechatronics"},{"key":"10140_CR6","unstructured":"Bagnell, J., Chestnutt, J., Bradley, D., et\u00a0al. (2006). Boosting structured prediction for imitation learning. In: Proceedings of the advances in neural information processing systems, pp. 1\u20138"},{"issue":"66","key":"10140_CR7","doi-asserted-by":"publisher","first-page":"2752","DOI":"10.21105\/joss.02752","volume":"6","author":"V Berenz","year":"2021","unstructured":"Berenz, V., Naveau, M., Widmaier, F., et al. (2021). The o80 c++ templated toolbox: Designing customized python APIs for synchronizing realtime processes. Journal of Open Source Software, 6(66), 2752\u20132755.","journal-title":"Journal of Open Source Software"},{"key":"10140_CR8","unstructured":"Bertsekas, D. (2012). Dynamic programming and optimal control: Volume I. Athena Scientific"},{"key":"10140_CR9","volume-title":"Reinforcement learning and optimal control","author":"D Bertsekas","year":"2019","unstructured":"Bertsekas, D. (2019). Reinforcement learning and optimal control. Athena Scientific."},{"key":"10140_CR10","doi-asserted-by":"crossref","unstructured":"Boedecker, J., Springenberg, J. T., W\u00fclfing, J., et\u00a0al. (2014). Approximate real-time optimal control based on sparse gaussian process models. In IEEE symposium on adaptive dynamic programming and reinforcement learning pp. 1\u20138.","DOI":"10.1109\/ADPRL.2014.7010608"},{"issue":"3","key":"10140_CR11","doi-asserted-by":"publisher","first-page":"96","DOI":"10.1109\/MCS.2006.1636313","volume":"26","author":"D Bristow","year":"2006","unstructured":"Bristow, D., Tharayil, M., & Alleyne, A. (2006). A survey of iterative learning control. IEEE Control Systems Magazine, 26(3), 96\u2013114.","journal-title":"IEEE Control Systems Magazine"},{"key":"10140_CR12","doi-asserted-by":"crossref","unstructured":"Brunner, G., Richter, O., Wang, Y., et\u00a0al. (2018). Teaching a machine to read maps with deep reinforcement learning. In: Proceedings of the AAAI conference on artificial intelligence, pp. 2763\u20132770","DOI":"10.1609\/aaai.v32i1.11645"},{"key":"10140_CR13","unstructured":"B\u00fcchler, D., Guist, S., Calandra, R., et\u00a0al. (2020). Learning to play table tennis from scratch using muscular robots. pp. 1\u201311. arXiv:2006.05935v1"},{"key":"10140_CR14","doi-asserted-by":"publisher","first-page":"104230","DOI":"10.1016\/j.robot.2022.104230","volume":"159","author":"D B\u00fcchler","year":"2023","unstructured":"B\u00fcchler, D., Calandra, R., & Peters, J. (2023). Learning to control highly accelerated ballistic movements on muscular robots. Robotics and Autonomous Systems, 159, 104230\u2013104241.","journal-title":"Robotics and Autonomous Systems"},{"key":"10140_CR15","doi-asserted-by":"crossref","unstructured":"Cao, Q., Lin, L., Shi, Y., et\u00a0al. (2017). Attention-aware face hallucination via deep reinforcement learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 690\u2013698","DOI":"10.1109\/CVPR.2017.180"},{"key":"10140_CR16","doi-asserted-by":"crossref","unstructured":"Chen, Z., Liang, X., & Zheng, M. (2021). Deep iterative learning control for quadrotor\u2019s trajectory tracking. In: Proceedings of the American control conference, pp. 1408\u20131413","DOI":"10.23919\/ACC50511.2021.9483044"},{"key":"10140_CR17","doi-asserted-by":"crossref","unstructured":"Cheng, R., Orosz, G., Murray, R. M., et\u00a0al. (2019). End-to-end safe reinforcement learning through barrier functions for safety-critical continuous control tasks. pp 1\u201311. arxiv:1903.08792v1","DOI":"10.1609\/aaai.v33i01.33013387"},{"key":"10140_CR18","doi-asserted-by":"crossref","unstructured":"Deisenroth, M. P., Englert, P., Peters, J., et\u00a0al. (2014). Multi-task policy search for robotics. In: Proceedings of the IEEE international conference on robotics and automation, pp. 3876\u20133881","DOI":"10.1109\/ICRA.2014.6907421"},{"key":"10140_CR19","doi-asserted-by":"crossref","unstructured":"Devrim\u00a0Kaba, M., Gokhan\u00a0Uzunbas, M., Nam\u00a0Lim, S. (2017). A reinforcement learning approach to the view planning problem. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 6933\u20136941","DOI":"10.1109\/CVPR.2017.541"},{"key":"10140_CR20","doi-asserted-by":"crossref","unstructured":"Dittrich, A., Schneider, J., Guist, S., et\u00a0al. (2022). AIMY: An open-source table tennis ball launcher for versatile and high-fidelity trajectory generation. pp 1\u201314. arXiv:2210.06048v2","DOI":"10.1109\/ICRA48891.2023.10160336"},{"key":"10140_CR21","unstructured":"Dosovitskiy, A., Ros, G., Codevilla, F., et\u00a0al. (2017). CARLA: An open urban driving simulator. pp 1\u201316. arXiv:1711.03938v1"},{"issue":"4","key":"10140_CR22","doi-asserted-by":"publisher","first-page":"756","DOI":"10.1109\/TAC.1978.1101812","volume":"23","author":"J Doyle","year":"1978","unstructured":"Doyle, J. (1978). Guaranteed margins for LQG regulators. IEEE Transactions on Automatic Control, 23(4), 756\u2013757.","journal-title":"IEEE Transactions on Automatic Control"},{"key":"10140_CR23","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1016\/j.asoc.2015.11.030","volume":"40","author":"F Fathinezhad","year":"2016","unstructured":"Fathinezhad, F., Derhami, V., & Rezaeian, M. (2016). Supervised fuzzy reinforcement learning for robot navigation. Applied Soft Computing, 40, 33\u201341.","journal-title":"Applied Soft Computing"},{"key":"10140_CR24","unstructured":"Finn, C., Yu, T., Fu, J., et\u00a0al. (2017). Generalizing skills with semi-supervised reinforcement learning, pp. 1\u201311. arXiv:1612.00429v2"},{"key":"10140_CR25","unstructured":"Firoiu, V., Whitney, W. F., & Tenenbaum, J. B. (2017). Beating the world\u2019s best at super smash bros with deep reinforcement learning, pp. 1\u20137. arXiv:1702.06230v3"},{"key":"10140_CR26","doi-asserted-by":"crossref","unstructured":"Furrer, F., Burri, M., Achtelik, M., et\u00a0al. (2016). RotorS\u2014A modular Gazebo MAV simulator framework. In: Proceedings of the robot operating system: The complete reference, pp. 595\u2013625","DOI":"10.1007\/978-3-319-26054-9_23"},{"issue":"8","key":"10140_CR27","doi-asserted-by":"publisher","first-page":"1135","DOI":"10.1016\/j.mechatronics.2012.09.010","volume":"22","author":"S Ganguly","year":"2012","unstructured":"Ganguly, S., Garg, A., Pasricha, A., et al. (2012). Control of pneumatic artificial muscle system through experimental modelling. Mechatronics, 22(8), 1135\u20131147.","journal-title":"Mechatronics"},{"key":"10140_CR28","first-page":"1437","volume":"16","author":"J Garcia","year":"2015","unstructured":"Garcia, J., & Fernandez, F. (2015). A comprehensive survey on safe reinforcement learning. Journal of Machine Learning Research, 16, 1437\u20131480.","journal-title":"Journal of Machine Learning Research"},{"key":"10140_CR29","volume-title":"Optimal control with engineering applications","author":"HP Geering","year":"2007","unstructured":"Geering, H. P. (2007). Optimal control with engineering applications. Springer."},{"key":"10140_CR30","doi-asserted-by":"crossref","unstructured":"Glover, J., & Kaelbling, L. P. (2014). Tracking the spin on a ping pong ball with the quaternion bingham filter. In: Proceedings of the IEEE international conference on robotics and automation, pp. 4133\u20134140.","DOI":"10.1109\/ICRA.2014.6907460"},{"issue":"4","key":"10140_CR31","doi-asserted-by":"publisher","first-page":"90","DOI":"10.3390\/robotics8040090","volume":"8","author":"S Gomez-Gonzalez","year":"2019","unstructured":"Gomez-Gonzalez, S., Nemmour, Y., Sch\u00f6lkopf, B., et al. (2019). Reliable real-time ball tracking for robot table tennis. Robotics, 8(4), 90\u2013102.","journal-title":"Robotics"},{"key":"10140_CR32","unstructured":"Grzes, M. (2017). Reward shaping in episodic reinforcement learning. In: Proceedings of the international conference on autonomous agents and multiagent systems, pp. 565\u2013573."},{"key":"10140_CR33","unstructured":"Heess, N., Tb, D., Sriram, S., et\u00a0al. (2017). Emergence of locomotion behaviours in rich environments, pp. 1\u201314. arXiv:1707.02286v2"},{"key":"10140_CR34","doi-asserted-by":"crossref","unstructured":"Hofer, M., & D\u2019Andrea, R. (2018). Design, modeling and control of a soft robotic arm. In: Proceedings of the IEEE\/RSJ international conference on intelligent robots and systems, pp. 1456\u20131463","DOI":"10.1109\/IROS.2018.8594221"},{"key":"10140_CR35","doi-asserted-by":"crossref","unstructured":"Hofer, M., Spannagl, L., & D\u2019Andrea, R. (2019). Iterative learning control for fast and accurate position tracking with an articulated soft robotic arm. In: Proceedings of the IEEE\/RSJ international conference on intelligent robots and systems, pp. 6602\u20136607","DOI":"10.1109\/IROS40897.2019.8967636"},{"issue":"1","key":"10140_CR36","doi-asserted-by":"publisher","first-page":"368","DOI":"10.1109\/TIE.2018.2826450","volume":"66","author":"Y Jian","year":"2019","unstructured":"Jian, Y., Huang, D., Liu, J., et al. (2019). High-precision tracking of piezoelectric actuator using iterative learning control and direct inverse compensation of hysteresis. IEEE Transactions on Industrial Electronics, 66(1), 368\u2013377.","journal-title":"IEEE Transactions on Industrial Electronics"},{"key":"10140_CR37","unstructured":"Kalashnikov, D., Irpan, A., Pastor, P., et\u00a0al. (2018). QT-Opt: Scalable deep reinforcement learning for vision-based robotic manipulation, pp. 1\u201323. arXiv:1806.10293v3"},{"key":"10140_CR38","unstructured":"Kansky, K., Silver, T., M\u00e9ly, D. A., et\u00a0al. (2017). Schema networks: Zero-shot transfer with a generative causal model of intuitive physics. In: Proceedings of the international conference on machine learning, pp. 1809\u20131818"},{"key":"10140_CR39","doi-asserted-by":"crossref","unstructured":"Kaushik, R., Anne, T., Mouret, J. B. (2020). Fast online adaptation in robotics through meta-learning embeddings of simulated priors. In: Proceedings of IEEE\/RSJ international conference on intelligent robots and systems, pp. 5269\u20135276.","DOI":"10.1109\/IROS45743.2020.9341462"},{"issue":"11","key":"10140_CR40","doi-asserted-by":"publisher","first-page":"1238","DOI":"10.1177\/0278364913495721","volume":"32","author":"JJ Kober","year":"2013","unstructured":"Kober, J. J., Andrew, B., & Peters, J. (2013). Reinforcement learning in robotics: A survey. The International Journal of Robotics Research, 32(11), 1238\u20131274.","journal-title":"The International Journal of Robotics Research"},{"key":"10140_CR41","doi-asserted-by":"crossref","unstructured":"Kogiso, K., Sawano, K., Itto, T., et\u00a0al. (2012). Identification procedure for mckibben pneumatic artificial muscle systems. In Proceedings of the IEEE\/RSJ international conference on intelligent robots and systems, pp. 3714\u20133721","DOI":"10.1109\/IROS.2012.6385778"},{"key":"10140_CR42","unstructured":"Konyushkova, K., Zolna, K., Aytar, Y., et\u00a0al. (2020). Semi-supervised reward learning for offline reinforcement learning. pp. 1\u201312. arXiv:2012.06899v1"},{"key":"10140_CR43","doi-asserted-by":"publisher","first-page":"415","DOI":"10.1016\/j.artint.2014.11.005","volume":"247","author":"A Kupcsik","year":"2017","unstructured":"Kupcsik, A., Deisenroth, M. P., Peters, J., et al. (2017). Model-based contextual policy search for data-efficient generalization of robot skills. Artificial Intelligence, 247, 415\u2013439.","journal-title":"Artificial Intelligence"},{"key":"10140_CR44","doi-asserted-by":"crossref","unstructured":"Lake, B. M., Ullman, T. D., Tenenbaum, J. B., et al. (2016). Building machines that learn and think like people. pp. 1\u201358. arXiv:1604.00289v3","DOI":"10.1017\/S0140525X16001837"},{"key":"10140_CR45","unstructured":"Laskin, M., Lee, K., Stooke, A., et\u00a0al. (2020). Reinforcement Learning with Augmented Data. In: Proceedings of the advances in neural information processing systems, pp. 19884\u201319895"},{"key":"10140_CR46","unstructured":"Laud, A. D. (2004). Theory and application of reward shaping in reinforcement learning. PhD thesis, University of Illinois at Urbana-Champaign"},{"key":"10140_CR47","unstructured":"Levine, S., & Koltun, V. (2014). Learning complex neural network policies with trajectory optimization. In: Proceedings of the international conference on machine learning, pp. 829\u2013837"},{"key":"10140_CR48","doi-asserted-by":"crossref","unstructured":"Liu, F., Li, S., Zhang, L., et\u00a0al. (2017). 3DCNN-DQN-RNN: A deep reinforcement learning framework for semantic parsing of large-scale 3D point clouds. In: Proceedings of the IEEE international conference on computer vision, pp. 5678\u20135687","DOI":"10.1109\/ICCV.2017.605"},{"key":"10140_CR49","unstructured":"Luo, F. M., Xu, T., Lai, H., et\u00a0al. (2022). A survey on model-based reinforcement learning, pp. 1\u201328. arXiv:2206.09328v1"},{"issue":"6","key":"10140_CR50","doi-asserted-by":"publisher","first-page":"1447","DOI":"10.1007\/s10514-017-9629-x","volume":"41","author":"J Luo","year":"2017","unstructured":"Luo, J., & Hauser, K. (2017). Robust trajectory optimization under frictional contact with iterative learning. Autonomous Robots, 41(6), 1447\u20131461.","journal-title":"Autonomous Robots"},{"key":"10140_CR51","doi-asserted-by":"crossref","unstructured":"Ma, H., B\u00fcchler, D., Sch\u00f6lkopf, B., et\u00a0al. (2022). A learning-based iterative control framework for controlling a robot arm with pneumatic artificial muscles. In: Proceedings of robotics: Science and systems XVIII, pp. 1\u201310","DOI":"10.15607\/RSS.2022.XVIII.029"},{"key":"10140_CR52","unstructured":"Matas, J., James, S., & Davison, A. J. (2018). Sim-to-real reinforcement learning for deformable object manipulation, pp. 1\u201310. arXiv:1806.07851v2"},{"key":"10140_CR53","doi-asserted-by":"crossref","unstructured":"McCord, C., Queralta, J. P., Gia, T. N., et\u00a0al. (2019). Distributed progressive formation control for multi-agent systems: 2D and 3D deployment of UAVs in ROS\/Gazebo with RotorS. In European conference on mobile robots, pp. 1\u20136.","DOI":"10.1109\/ECMR.2019.8870934"},{"key":"10140_CR54","unstructured":"Moldovan, T. M., Abbeel, P. (2012). Safe exploration in markov decision processes, pp. 1\u201310. arXiv:1205.4810v3"},{"key":"10140_CR55","doi-asserted-by":"crossref","unstructured":"Mueller, F. L., Schoellig, A. P., D\u2019Andrea, R. (2012). Iterative Learning of feed-forward corrections for high-performance tracking. In: Proceedings of the IEEE\/RSJ international conference on intelligent robots and systems, pp. 3276\u20133281","DOI":"10.1109\/IROS.2012.6385647"},{"issue":"5","key":"10140_CR56","first-page":"933","volume":"45","author":"VL Nickel","year":"1963","unstructured":"Nickel, V. L., Perry, J., & Garrett, A. L. (1963). Development of useful function in the severely paralyzed hand. The Bone and Joint Journal, 45(5), 933\u2013952.","journal-title":"The Bone and Joint Journal"},{"key":"10140_CR57","unstructured":"Oh, J., Chockalingam, V., Satinder, et\u00a0al. (2016). Control of memory, active perception, and action in minecraft. In: Proceedings of the international conference on machine learning, pp. 2790\u20132799"},{"key":"10140_CR58","unstructured":"Paulus, R., Xiong, C., & Socher, R. (2017). A deep reinforced model for abstractive summarization, pp. 1\u201312. arXiv:1705.04304v3"},{"key":"10140_CR59","doi-asserted-by":"crossref","unstructured":"Piazzi, A., & Visioli, A. (1997). An interval algorithm for minimum-jerk trajectory planning of robot manipulators. In: Proceedings of the IEEE conference on decision and control, pp. 1924\u20131927.","DOI":"10.1109\/CDC.1997.657874"},{"key":"10140_CR60","unstructured":"Piche, A., Pardinas, R., Vazquez, D., et\u00a0al. (2022). Implicit offline reinforcement learning via supervised learning, pp. 1\u201314. arXiv:2210.12272v1"},{"issue":"2","key":"10140_CR61","doi-asserted-by":"publisher","first-page":"153","DOI":"10.1007\/s10846-017-0468-y","volume":"86","author":"AS Polydoros","year":"2017","unstructured":"Polydoros, A. S., & Nalpantidis, L. (2017). Survey of model-based reinforcement learning: applications on robotics. Journal of Intelligent & Robotic Systems, 86(2), 153\u2013173.","journal-title":"Journal of Intelligent & Robotic Systems"},{"key":"10140_CR62","doi-asserted-by":"crossref","unstructured":"Quintanilla, R., Wen, J. T. (2007). Iterative learning control for nonsmooth dynamical systems. In: Proceedings of the IEEE conference on decision and control, pp. 245\u2013251","DOI":"10.1109\/CDC.2007.4434885"},{"key":"10140_CR63","doi-asserted-by":"crossref","unstructured":"Ramya Ramakrishnan, E. K., Debadeepta, D., Eric, H., et al. (2020). Blind spot detection for safe sim-to-real transfer. Journal of Artificial Intelligence Research,67(1), 1\u201324.","DOI":"10.1613\/jair.1.11436"},{"issue":"1","key":"10140_CR64","doi-asserted-by":"publisher","first-page":"297","DOI":"10.1146\/annurev-control-100819-063206","volume":"3","author":"H Ravichandar","year":"2020","unstructured":"Ravichandar, H., Polydoros, A. S., Chernova, S., et al. (2020). Recent advances in robot learning from demonstration. Review of Control, Robotics, and Autonomous Systems, 3(1), 297\u2013330.","journal-title":"Review of Control, Robotics, and Autonomous Systems"},{"key":"10140_CR65","unstructured":"Schneider, J. (1996). Exploiting model uncertainty estimates for safe dynamic control learning. In: Proceedings of the advances in neural information processing systems, pp. 1047\u20131053"},{"issue":"1","key":"10140_CR66","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1007\/s10514-012-9283-2","volume":"33","author":"AP Schoellig","year":"2012","unstructured":"Schoellig, A. P., Mueller, F. L., & D\u2019Andrea, R. (2012). Optimization-based iterative learning for precise quadrocopter trajectory tracking. Autonomous Robots, 33(1), 103\u2013127.","journal-title":"Autonomous Robots"},{"issue":"6","key":"10140_CR67","doi-asserted-by":"publisher","first-page":"2225","DOI":"10.1002\/oca.2656","volume":"41","author":"C Sferrazza","year":"2020","unstructured":"Sferrazza, C., Muehlebach, M., & D\u2019Andrea, R. (2020). Learning-based parametrized model predictive control for trajectory tracking. Optimal Control Applications and Methods, 41(6), 2225\u20132249.","journal-title":"Optimal Control Applications and Methods"},{"key":"10140_CR68","doi-asserted-by":"crossref","unstructured":"Shah, S., Dey, D., Lovett, C., et\u00a0al. (2017). AirSim: High-fidelity visual and physical simulation for autonomous vehicles, pp 1\u201314. arXiv:1705.05065v2","DOI":"10.1007\/978-3-319-67361-5_40"},{"issue":"7587","key":"10140_CR69","doi-asserted-by":"publisher","first-page":"484","DOI":"10.1038\/nature16961","volume":"529","author":"D Silver","year":"2016","unstructured":"Silver, D., Huang, A., Maddison, C. J., et al. (2016). Mastering the game of go with deep neural networks and tree search. Nature, 529(7587), 484\u2013489.","journal-title":"Nature"},{"issue":"7676","key":"10140_CR70","doi-asserted-by":"publisher","first-page":"354","DOI":"10.1038\/nature24270","volume":"550","author":"D Silver","year":"2017","unstructured":"Silver, D., Schrittwieser, J., Simonyan, K., et al. (2017). Mastering the game of go without human knowledge. Nature, 550(7676), 354\u2013359.","journal-title":"Nature"},{"key":"10140_CR71","doi-asserted-by":"crossref","unstructured":"Tessler, C., Givony, S., Zahavy, T., et\u00a0al. (2017). A deep hierarchical approach to lifelong learning in minecraft. In: Proceedings of the AAAI conference on artificial intelligence, pp. 1553\u20131561","DOI":"10.1609\/aaai.v31i1.10744"},{"key":"10140_CR72","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4615-5529-2","volume-title":"Learning to learn","author":"S Thrun","year":"1998","unstructured":"Thrun, S., & Pratt, L. (1998). Learning to learn. US: Springer."},{"key":"10140_CR73","doi-asserted-by":"crossref","unstructured":"Todorov, E., Erez, T., & Tassa, Y. (2012). MuJoCo: A physics engine for model-based control. In: Proceedings of the IEEE\/RSJ international conference on intelligent robots and systems, pp. 5026\u20135033.","DOI":"10.1109\/IROS.2012.6386109"},{"key":"10140_CR74","doi-asserted-by":"crossref","unstructured":"Tondu, B., & Zagal, S. (2006). McKibben artificial muscle can be in accordance with the Hill skeletal muscle model. In: Proceedings of the RAS-EMBS international conference on biomedical robotics and biomechatronics, pp. 714\u2013720","DOI":"10.1109\/BIOROB.2006.1639174"},{"issue":"8","key":"10140_CR75","doi-asserted-by":"publisher","first-page":"966","DOI":"10.1016\/j.mechatronics.2014.05.007","volume":"24","author":"J Van Rooijen","year":"2014","unstructured":"Van Rooijen, J., Grondman, I., & Babu\u0161ka, R. (2014). Learning rate free reinforcement learning for real-time motion control using a value-gradient based policy. Mechatronics, 24(8), 966\u2013974.","journal-title":"Mechatronics"},{"key":"10140_CR76","unstructured":"Vanschoren, J. (2018). Meta-learning: A survey, pp. 1\u201329. arXiv:1810.03548v1"},{"key":"10140_CR77","unstructured":"Wang, T., Bao, X., Clavera, I., et\u00a0al. (2019b). Benchmarking model-based reinforcement learning, pp. 1\u201325. arxiv:1907.02057v1"},{"key":"10140_CR78","unstructured":"Wang, H., Zheng, S., Xiong, C., et\u00a0al. (2019a). On the generalization gap in reparameterizable reinforcement Learning. In: Proceedings of the international conference on machine learning, pp. 6648\u20136658"},{"key":"10140_CR79","first-page":"253","volume":"15","author":"A Wilson","year":"2014","unstructured":"Wilson, A., Fern, A., & Tadepalli, P. (2014). Using trajectory data to improve Bayesian optimization for reinforcement learning. Journal of Machine Learning Research, 15, 253\u2013282.","journal-title":"Journal of Machine Learning Research"},{"key":"10140_CR80","unstructured":"Yogatama, D., Blunsom, P., Dyer, C., et\u00a0al. (2016). Learning to compose words into sentences with reinforcement learning, pp. 1\u201310. arXiv:1611.09100v1"},{"key":"10140_CR81","doi-asserted-by":"crossref","unstructured":"Zhang, X., & Lapata, M. (2017). Sentence simplification with deep reinforcement learning, pp. 1\u201311. arxiv:1703.10931v2","DOI":"10.18653\/v1\/D17-1062"},{"key":"10140_CR82","doi-asserted-by":"crossref","unstructured":"Zhao, W., Queralta, J. P., Westerlund, T. (2020). Sim-to-real transfer in deep reinforcement learning for robotics: A survey. In: Proceedings of the IEEE symposium series on computational intelligence, pp. 737\u2013744.","DOI":"10.1109\/SSCI47803.2020.9308468"},{"key":"10140_CR83","doi-asserted-by":"crossref","unstructured":"Zhao, Y., Xiong, R., & Zhang, Y. (2017). Model based motion state estimation and trajectory prediction of spinning ball for ping-pong robots using expectation-maximization algorithm. Journal of Intelligent & Robotic Systems,87(3), 407\u2013423.","DOI":"10.1007\/s10846-017-0515-8"},{"issue":"5","key":"10140_CR84","doi-asserted-by":"publisher","first-page":"2921","DOI":"10.1109\/TIE.2014.2364800","volume":"62","author":"YM Zhao","year":"2015","unstructured":"Zhao, Y. M., Lin, Y., Xi, F., et al. (2015). Calibration-based iterative learning control for path tracking of industrial robots. IEEE Transactions on Industrial Electronics, 62(5), 2921\u20132929.","journal-title":"IEEE Transactions on Industrial Electronics"},{"issue":"1","key":"10140_CR85","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1109\/JPROC.2020.3004555","volume":"109","author":"F Zhuang","year":"2021","unstructured":"Zhuang, F., Qi, Z., Duan, K., et al. (2021). A comprehensive survey on transfer learning. Proceedings of the IEEE, 109(1), 43\u201376.","journal-title":"Proceedings of the IEEE"},{"key":"10140_CR86","doi-asserted-by":"crossref","unstructured":"Zughaibi, J., Hofer, M., D\u2019Andrea, R. (2021). A fast and reliable pick-and-place application with a spherical soft robotic arm. In: Proceedings of the IEEE international conference on soft robotics, pp. 599\u2013606.","DOI":"10.1109\/RoboSoft51838.2021.9479227"}],"container-title":["Autonomous Robots"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10514-023-10140-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10514-023-10140-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10514-023-10140-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,28]],"date-time":"2023-11-28T18:12:32Z","timestamp":1701195152000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10514-023-10140-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,17]]},"references-count":86,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2023,12]]}},"alternative-id":["10140"],"URL":"https:\/\/doi.org\/10.1007\/s10514-023-10140-6","relation":{},"ISSN":["0929-5593","1573-7527"],"issn-type":[{"value":"0929-5593","type":"print"},{"value":"1573-7527","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,10,17]]},"assertion":[{"value":"31 January 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 September 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 October 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no competing interests to declare that are relevant to the content of this article.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing Interests"}},{"value":"Important parts of our source code are available here: . The remaining parts can be reproduced from the article.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Code availability"}}]}}