{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,30]],"date-time":"2026-10-30T16:31:38Z","timestamp":1793377898506,"version":"4.5.0"},"reference-count":111,"publisher":"SAGE Publications","issue":"5","license":[{"start":{"date-parts":[[2024,10,23]],"date-time":"2024-10-23T00:00:00Z","timestamp":1729641600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"name":"The AI Institute","id":[{"id":"https:\/\/ror.org\/053cgvy07","id-type":"ROR","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["CMMI-1944722"],"award-info":[{"award-number":["CMMI-1944722"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100007631","name":"Canadian Institute for Advanced Research","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100007631","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of Robotics Research"],"published-print":{"date-parts":[[2025,4]]},"abstract":"<jats:p>This paper presents a comprehensive study on using deep reinforcement learning (RL) to create dynamic locomotion controllers for bipedal robots. Going beyond focusing on a single locomotion skill, we develop a general control solution that can be used for a range of dynamic bipedal skills, from periodic walking and running to aperiodic jumping and standing. Our RL-based controller incorporates a novel dual-history architecture, utilizing both a long-term and short-term input\/output (I\/O) history of the robot. This control architecture, when trained through the proposed end-to-end RL approach, consistently outperforms other methods across a diverse range of skills in both simulation and the real world. The study also delves into the adaptivity and robustness introduced by the proposed RL system in developing locomotion controllers. We demonstrate that the proposed architecture can adapt to both time-invariant dynamics shifts and time-variant changes, such as contact events, by effectively using the robot\u2019s I\/O history. Additionally, we identify task randomization as another key source of robustness, fostering better task generalization and compliance to disturbances. The resulting control policies can be successfully deployed on Cassie, a torque-controlled human-sized bipedal robot. This work pushes the limits of agility for bipedal robots through extensive real-world experiments. We demonstrate a diverse range of locomotion skills, including: robust standing, versatile walking, fast running with a demonstration of a 400-meter dash, and a diverse set of jumping skills, such as standing long jumps and high jumps.<\/jats:p>","DOI":"10.1177\/02783649241285161","type":"journal-article","created":{"date-parts":[[2024,10,24]],"date-time":"2024-10-24T02:31:48Z","timestamp":1729737108000},"page":"840-888","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":140,"title":["Reinforcement learning for versatile, dynamic, and robust bipedal locomotion control"],"prefix":"10.1177","volume":"44","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8040-1098","authenticated-orcid":false,"given":"Zhongyu","family":"Li","sequence":"first","affiliation":[{"name":"Department of Mechanical Engineering, University of California Berkeley, Berkeley, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xue Bin","family":"Peng","sequence":"additional","affiliation":[{"name":"School of Computing Science, Simon Fraser University, Burnaby, BC, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pieter","family":"Abbeel","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering and Computer Sciences, University of California Berkeley, Berkeley, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sergey","family":"Levine","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering and Computer Sciences, University of California Berkeley, Berkeley, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Glen","family":"Berseth","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Operations Research, Universit\u00e9 de Montr\u00e9al, Montreal, QC, Canada"},{"name":"Mila \u2013 Quebec AI Institute, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Koushil","family":"Sreenath","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, University of California Berkeley, Berkeley, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2024,10,23]]},"reference":[{"key":"e_1_3_6_2_1","volume-title":"Model-based design for legged robots: predictive control and reinforcement learning","author":"Agrawal A","year":"2022","unstructured":"Agrawal A (2022) Model-based design for legged robots: predictive control and reinforcement learning. PhD Thesis. Berkeley: UC Berkeley."},{"key":"e_1_3_6_3_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-control-062922-090153"},{"key":"e_1_3_6_4_1","doi-asserted-by":"publisher","DOI":"10.3389\/frobt.2022.854212"},{"key":"e_1_3_6_5_1","doi-asserted-by":"crossref","unstructured":"Boroujeni MG Daneshman E Righetti L et al. (2021) A unified framework for walking and running of bipedal robots. In: 2021 20th International Conference on Advanced Robotics (ICAR) Ljubljana Slovenia 06\u201310 December 2021 pp. 396\u2013403 IEEE.","DOI":"10.1109\/ICAR53236.2021.9659392"},{"key":"e_1_3_6_6_1","doi-asserted-by":"crossref","unstructured":"Bouyarmane K Kheddar A (2011) Using a multi-objective controller to synthesize simulated humanoid robot motion with changing contact configurations. In: 2011 IEEE\/RSJ International Conference on Intelligent Robots and Systems San Francisco CA USA 25\u201330 September 2011 pp. 4414\u20134419 IEEE.","DOI":"10.1109\/IROS.2011.6094483"},{"key":"e_1_3_6_7_1","doi-asserted-by":"crossref","unstructured":"Caron S Kheddar A Tempier O (2019) Stair climbing stabilization of the hrp-4 humanoid robot using whole-body admittance control. In: 2019 International Conference on Robotics and Automation (ICRA) Montreal QC Canada 20\u201324 May 2019 pp. 277\u2013283 IEEE.","DOI":"10.1109\/ICRA.2019.8794348"},{"key":"e_1_3_6_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2022.3151771"},{"key":"e_1_3_6_9_1","doi-asserted-by":"crossref","unstructured":"Chen S Zhang B Mueller MW et al. (2023) Learning torque control for quadrupedal locomotion. In: 2023 IEEE-RAS 22nd International Conference on Humanoid Robots (Humanoids) Austin TX USA 12\u201314 December 2023 pp. 1\u20138.","DOI":"10.1109\/Humanoids57100.2023.10375154"},{"key":"e_1_3_6_10_1","doi-asserted-by":"crossref","unstructured":"Cheng X Kumar A Pathak D (2023) Legs as manipulator: pushing quadrupedal agility beyond locomotion. In: 2023 IEEE International Conference on Robotics and Automation (ICRA) 29 May\u201302 June 2023 London UK pp. 5106\u20135112.","DOI":"10.1109\/ICRA48891.2023.10161470"},{"key":"e_1_3_6_11_1","doi-asserted-by":"crossref","unstructured":"Chignoli M Kim D Stanger-Jones E et al. (2021) The mit humanoid robot: Design motion planning and control for acrobatic behaviors. In: 2020 IEEE-RAS 20th International Conference on Humanoid Robots (Humanoids) Munich Germany 19\u201321 July 2021 pp. 1\u20138 IEEE.","DOI":"10.1109\/HUMANOIDS47582.2021.9555782"},{"key":"e_1_3_6_12_1","doi-asserted-by":"crossref","unstructured":"Crowley D Dao J Duan H et al. (2023) Optimizing bipedal locomotion for the 100m dash with comparison to human running. In: 2023 IEEE International Conference on Robotics and Automation (ICRA) London UK 29 May\u201302 June 2023 12205\u201312211 IEEE.","DOI":"10.1109\/ICRA48891.2023.10160436"},{"key":"e_1_3_6_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2016.2582731"},{"key":"e_1_3_6_14_1","doi-asserted-by":"crossref","unstructured":"Dai H Valenzuela A Tedrake R (2014) Whole-body motion planning with centroidal dynamics and full kinematics. In: 2014 IEEE-RAS International Conference on Humanoid Robots Madrid Spain 18-20 November 2014 pp. 295\u2013302 IEEE.","DOI":"10.1109\/HUMANOIDS.2014.7041375"},{"key":"e_1_3_6_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2021.3061381"},{"key":"e_1_3_6_16_1","doi-asserted-by":"crossref","unstructured":"Dao J Green K Duan H et al. (2022) Sim-to-real learning for bipedal locomotion under unsensed dynamic loads. In: 2022 International Conference on Robotics and Automation (ICRA) Philadelphia PA USA 23\u201327 May 2022 pp. 10449\u201310455 IEEE.","DOI":"10.1109\/ICRA46639.2022.9811783"},{"key":"e_1_3_6_17_1","doi-asserted-by":"crossref","unstructured":"Deits R Tedrake R (2014) Footstep planning on uneven terrain with mixed-integer convex optimization. In: 2014 IEEE-RAS International Conference on Humanoid Robots Madrid Spain 18\u201320 November 2014 pp. 279\u2013286 IEEE.","DOI":"10.1109\/HUMANOIDS.2014.7041373"},{"key":"e_1_3_6_18_1","unstructured":"Deits R Kuindersma S Kelly MP et al. (2022) Robot movement and online trajectory optimization. US Patent App. 17\/358 628."},{"key":"e_1_3_6_19_1","unstructured":"DRL (2023) cassie-mujoco-sim. https:\/\/github.com\/osudrl\/cassie-mujoco-sim."},{"key":"e_1_3_6_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2021.3056064"},{"key":"e_1_3_6_21_1","doi-asserted-by":"crossref","unstructured":"Escontrela A Peng XB Yu W et al. (2022) Adversarial motion priors make good substitutes for complex reward functions. In: 2022 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS) Kyoto Japan 23\u201327 October 2022 pp. 25\u201332 IEEE.","DOI":"10.1109\/IROS47612.2022.9981973"},{"key":"e_1_3_6_22_1","unstructured":"Feng G Zhang H Li Z et al. (2023) Genloco: generalized locomotion controllers for quadrupedal robots. In: Conference on Robot Learning Auckland New Zealand 14 December 2022 pp. 1893\u20131903 PMLR."},{"key":"e_1_3_6_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2020.2964787"},{"key":"e_1_3_6_24_1","doi-asserted-by":"crossref","unstructured":"Fevre M Wensing PM Schmiedeler JP (2020) Rapid bipedal gait optimization in casadi. In: 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS) Las Vegas NV USA 24 October 2020\u201324 January 2021 pp. 3672\u20133678 IEEE.","DOI":"10.1109\/IROS45743.2020.9341586"},{"key":"e_1_3_6_25_1","unstructured":"Fu Z Kumar A Malik J et al. (2021) Minimizing energy consumption leads to the emergence of gaits in legged robots. In: 5th Annual Conference on Robot Learning London UK 08 November 2021."},{"key":"e_1_3_6_26_1","unstructured":"Fu Z Cheng X Pathak D (2023) Deep whole-body control: learning a unified policy for manipulation and locomotion. In: Conference on Robot Learning Auckland New Zealand 14 December 2022 pp. 138\u2013149 PMLR."},{"key":"e_1_3_6_27_1","doi-asserted-by":"publisher","DOI":"10.1115\/1.4055770"},{"key":"e_1_3_6_28_1","doi-asserted-by":"crossref","unstructured":"Gong Y Hartley R Da X et al. (2019) Feedback control of a cassie bipedal robot: walking standing and riding a segway. In: 2019 American Control Conference (ACC) Philadelphia PA USA 10\u201312 July 2019 pp. 4559\u20134566 IEEE.","DOI":"10.23919\/ACC.2019.8814833"},{"key":"e_1_3_6_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2009.2026502"},{"key":"e_1_3_6_30_1","doi-asserted-by":"crossref","unstructured":"Haarnoja T Ha S Zhou A et al. (2019) Learning to walk via deep reinforcement learning. In: Robotics: Science and Systems (RSS) Freiburg im Breisgau 22\u201326 June 2019.","DOI":"10.15607\/RSS.2019.XV.011"},{"key":"e_1_3_6_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2017.2783371"},{"key":"e_1_3_6_32_1","doi-asserted-by":"crossref","unstructured":"Hereid A Harib O Hartley R et al. (2019) Rapid trajectory optimization using c-frost with illustration on a cassie-series dynamic walking biped. In: 2019 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS) Macau China 03\u201308 November 2019. pp. 4722\u20134729.","DOI":"10.1109\/IROS40897.2019.8968056"},{"key":"e_1_3_6_33_1","doi-asserted-by":"crossref","unstructured":"Huang X Li Z Xiang Y et al. (2023) Creating a dynamic quadrupedal robotic goalkeeper with reinforcement learning. In: 2023 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS) Detroit MI USA 1\u20135 October 2023 pp. 2715\u20132722.","DOI":"10.1109\/IROS55552.2023.10341936"},{"key":"e_1_3_6_34_1","unstructured":"Huang X Chi Y Wang R et al. (2024) Diffuseloco: real-time legged locomotion control with diffusion from offline datasets. arXiv preprint arXiv:2404.19264."},{"key":"e_1_3_6_35_1","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.aau5872"},{"key":"e_1_3_6_36_1","doi-asserted-by":"crossref","unstructured":"Ibanez A Bidaud P Padois V (2014) Emergence of humanoid walking behaviors from mixed-integer model predictive control. In: 2014 IEEE\/RSJ International Conference on Intelligent Robots and Systems Chicago IL USA 14\u201318 September 2014 pp. 4014\u20134021 IEEE.","DOI":"10.1109\/IROS.2014.6943127"},{"key":"e_1_3_6_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2022.3151396"},{"key":"e_1_3_6_38_1","doi-asserted-by":"crossref","unstructured":"Kajita S Kanehiro F Kaneko K et al. (2001) The 3d linear inverted pendulum mode: a simple modeling for a biped walking pattern generation. In: Proceedings 2001 IEEE\/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No. 01CH37180) volume 1 Maui HI USA 29 October\u201303 November 2001 pp. 239\u2013246 IEEE.","DOI":"10.1109\/IROS.2001.973365"},{"key":"e_1_3_6_39_1","unstructured":"Kalashnikov D Varley J Chebotar Y et al. (2021) Mt-opt: continuous multi-task robotic reinforcement learning at scale. arXiv preprint arXiv:2104.08212."},{"key":"e_1_3_6_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2023.3304561"},{"key":"e_1_3_6_41_1","doi-asserted-by":"crossref","unstructured":"Kojima K Kojio Y Ishikawa T et al. (2019) A robot design method for weight saving aimed at dynamic motions: design of humanoid jaxon3-p and realization of jump motions. In: 2019 IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids) Toronto ON Canada 15\u201317 October 2019 pp. 586\u2013593 IEEE.","DOI":"10.1109\/Humanoids43949.2019.9035049"},{"key":"e_1_3_6_42_1","doi-asserted-by":"crossref","unstructured":"Kuindersma S Permenter F Tedrake R (2014) An efficiently solvable quadratic program for stabilizing dynamic locomotion. In: 2014 IEEE International Conference on Robotics and Automation (ICRA) Hong Kong China 31 May\u201307 June 2014 pp. 2589\u20132594 IEEE.","DOI":"10.1109\/ICRA.2014.6907230"},{"key":"e_1_3_6_43_1","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2021.XVII.011"},{"key":"e_1_3_6_44_1","doi-asserted-by":"crossref","unstructured":"Kumar A Li Z Zeng J et al. (2022) Adapting rapid motor adaptation for bipedal robots. In: 2022 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS) Kyoto Japan 23\u201327 October 2022 pp. 1161\u20131168.","DOI":"10.1109\/IROS47612.2022.9981091"},{"key":"e_1_3_6_45_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-85729-664-1"},{"key":"e_1_3_6_46_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-95459-8_48"},{"key":"e_1_3_6_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2024.3351554"},{"key":"e_1_3_6_48_1","unstructured":"Lee Y Sun SH Somasundaram S et al. (2018) Composing complex skills by learning transition policies. In: International Conference on Learning Representations New Orleans Louisiana USA 6\u20139 May 2019."},{"key":"e_1_3_6_49_1","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.abc5986"},{"key":"e_1_3_6_50_1","doi-asserted-by":"crossref","unstructured":"Li Z Cummings C Sreenath K (2020) Animated cassie: a dynamic relatable robotic character. In: 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS) Las Vegas NV USA 24 October 2020 - 24 January 2021 pp. 3739\u20133746.","DOI":"10.1109\/IROS45743.2020.9340894"},{"key":"e_1_3_6_51_1","doi-asserted-by":"crossref","unstructured":"Li Z Cheng X Peng XB et al. (2021) Reinforcement learning for robust parameterized locomotion control of bipedal robots. In: 2021 IEEE International Conference on Robotics and Automation (ICRA) Xi'an China 30 May\u201305 June 2021 pp. 2811\u20132817 IEEE.","DOI":"10.1109\/ICRA48506.2021.9560769"},{"key":"e_1_3_6_52_1","doi-asserted-by":"crossref","unstructured":"Li Y Li J Fu W et al. (2023a) Learning agile bipedal motions on a quadrupedal robot. arXiv preprint arXiv:2311.05818.","DOI":"10.1109\/ICRA57147.2024.10611442"},{"key":"e_1_3_6_53_1","doi-asserted-by":"crossref","unstructured":"Li Z Peng XB Abbeel P et al. (2023b) Robust and versatile bipedal jumping control through reinforcement learning. In: Robotics: Science and Systems XIX Daegu Republic of Korea 10\u201314 July 2023.","DOI":"10.15607\/RSS.2023.XIX.052"},{"key":"e_1_3_6_54_1","doi-asserted-by":"crossref","unstructured":"Lim D Kim MJ Cha J et al. (2023) Proprioceptive external torque learning for floating base robot and its applications to humanoid locomotion. In: 2023 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS) Detroit MI USA 01\u201305 October 2023.","DOI":"10.1109\/IROS55552.2023.10342530"},{"key":"e_1_3_6_55_1","unstructured":"Liu S Xu M Huang P et al. (2023) Continual vision-based reinforcement learning with group symmetries. In: Conference on Robot Learning Atlanta USA 06 November 2023 pp 222\u2013240 PMLR."},{"key":"e_1_3_6_56_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-1768-8_11"},{"key":"e_1_3_6_57_1","doi-asserted-by":"crossref","unstructured":"Ma WL Kolathaya S Ambrose ER et al. (2017) Bipedal robotic running with durus-2d: bridging the gap between theory and experiment. In: Proceedings of the 20th international conference on hybrid systems: computation and control Pittsburgh PA USA April 18-20 2017 pp. 265\u2013274.","DOI":"10.1145\/3049797.3049823"},{"key":"e_1_3_6_58_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2016.2547024"},{"key":"e_1_3_6_59_1","unstructured":"Margolis GB Agrawal P (2023) Walk these ways: tuning robot control for generalization with multiplicity of behavior. In: Conference on Robot Learning Auckland New Zealand 14 December 2022 pp. 22\u201331 PMLR."},{"key":"e_1_3_6_60_1","doi-asserted-by":"crossref","unstructured":"Margolis G Yang G Paigwar K et al. (2022) Rapid locomotion via reinforcement learning. In: Robotics: Science and Systems New York City NY USA June 27\u20131 July 2022.","DOI":"10.15607\/RSS.2022.XVIII.022"},{"key":"e_1_3_6_61_1","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2022.3228390"},{"key":"e_1_3_6_62_1","unstructured":"Meuleau N Peshkin L Kim KE et al. (1999) Learning finite-state controllers for partially observable environments. In: Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence Stockholm Sweden July 30\u20131 August 1999 pp. 427\u2013436."},{"key":"e_1_3_6_63_1","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.abk2822"},{"key":"e_1_3_6_64_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-94-007-6046-2_51"},{"key":"e_1_3_6_65_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10514-013-9341-4"},{"key":"e_1_3_6_66_1","doi-asserted-by":"crossref","unstructured":"Peng XB Andrychowicz M Zaremba W et al. (2018) Sim-to-real transfer of robotic control with dynamics randomization. In: 2018 IEEE international Conference on Robotics and Automation (ICRA) Brisbane QLD Australia 21\u201325 May 2018 pp. 3803\u20133810.","DOI":"10.1109\/ICRA.2018.8460528"},{"key":"e_1_3_6_67_1","unstructured":"Peng XB Coumans E Zhang T et al. (2020) Learning agile robotic locomotion skills by imitating animals. In: Robotics: Science and Systems (RSS) held virtually 12\u201316 July 2020."},{"key":"e_1_3_6_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459670"},{"key":"e_1_3_6_69_1","doi-asserted-by":"publisher","DOI":"10.1177\/0278364913506757"},{"key":"e_1_3_6_70_1","doi-asserted-by":"publisher","DOI":"10.1177\/0278364912452762"},{"key":"e_1_3_6_71_1","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2023.3271136"},{"key":"e_1_3_6_72_1","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.adi9579"},{"key":"e_1_3_6_73_1","doi-asserted-by":"publisher","DOI":"10.1177\/027836498400300207"},{"key":"e_1_3_6_74_1","doi-asserted-by":"crossref","unstructured":"Reher J Ames AD (2021) Control lyapunov functions for compliant hybrid zero dynamic walking. arXiv preprint arXiv:2107.04241.","DOI":"10.1109\/ICRA48506.2021.9560906"},{"key":"e_1_3_6_75_1","doi-asserted-by":"crossref","unstructured":"Rodriguez D Behnke S (2021) Deepwalk: omnidirectional bipedal gait by deep reinforcement learning. In: 2021 IEEE international conference on robotics and automation (ICRA) Xi'an China 30 May\u201305 June 2021 pp. 3033\u20133039.","DOI":"10.1109\/ICRA48506.2021.9561717"},{"key":"e_1_3_6_76_1","doi-asserted-by":"crossref","unstructured":"Rummel J Blum Y Maus HM et al. (2010) Stable and robust walking with compliant legs. In: 2010 IEEE International Conference on Robotics and Automation Anchorage AK USA 3\u20137 May 2010 pp. 5250\u20135255 IEEE.","DOI":"10.1109\/ROBOT.2010.5509500"},{"key":"e_1_3_6_77_1","unstructured":"Schulman J Wolski F Dhariwal P et al. (2017) Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347."},{"key":"e_1_3_6_78_1","doi-asserted-by":"crossref","unstructured":"Sentis L Khatib O (2006) A whole-body control framework for humanoids operating in human environments. In: Proceedings 2006 IEEE International Conference on Robotics and Automation 2006. ICRA 2006 Orlando FL USA 15\u201319 May 2006 pp. 2641\u20132648 IEEE.","DOI":"10.1109\/ROBOT.2006.1642100"},{"key":"e_1_3_6_79_1","unstructured":"SFU (2018) Sfu motion capture database. https:\/\/mocap.cs.sfu.ca\/."},{"key":"e_1_3_6_80_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2021.3136645"},{"key":"e_1_3_6_81_1","doi-asserted-by":"crossref","unstructured":"Siekmann J Valluri S Dao J et al. (2020) Learning memory-based control for human-scale bipedal locomotion. In: Robotics Science and Systems Corvalis Oregon USA 12\u201316 July 2020.","DOI":"10.15607\/RSS.2020.XVI.031"},{"key":"e_1_3_6_82_1","doi-asserted-by":"crossref","unstructured":"Siekmann J Godse Y Fern A et al. (2021a) Sim-to-real learning of all common bipedal gaits via periodic reward composition. In: 2021 IEEE International Conference on Robotics and Automation (ICRA) Xi'an China 30 May\u201305 June 2021 pp. 7309\u20137315 IEEE.","DOI":"10.1109\/ICRA48506.2021.9561814"},{"key":"e_1_3_6_83_1","doi-asserted-by":"crossref","unstructured":"Siekmann J Green K Warila J et al. (2021b) Blind bipedal stair traversal via sim-to-real reinforcement learning. In: Robotics: Science and Systems Held Virtually July 12\u201316 2021.","DOI":"10.15607\/RSS.2021.XVII.061"},{"key":"e_1_3_6_84_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2023.3301175"},{"key":"e_1_3_6_85_1","doi-asserted-by":"crossref","unstructured":"Smith L Kew JC Peng XB et al. (2022) Legged robots that keep on learning: fine-tuning locomotion policies in the real world. In: 2022 International Conference on Robotics and Automation (ICRA) Philadelphia PA USA 23\u201327 May 2022 pp. 1593\u20131599 IEEE.","DOI":"10.1109\/ICRA46639.2022.9812166"},{"issue":"3","key":"e_1_3_6_86_1","first-page":"4","article-title":"Demonstrating a walk in the park: learning to walk in 20 minutes with model-free reinforcement learning","volume":"2","author":"Smith L","year":"2023","unstructured":"Smith L, Kostrikov I, Levine S (2023) Demonstrating a walk in the park: learning to walk in 20 minutes with model-free reinforcement learning. Robotics: Science and Systems (RSS) Demo 2(3): 4.","journal-title":"Robotics: Science and Systems (RSS) Demo"},{"key":"e_1_3_6_87_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-27645-3_12"},{"key":"e_1_3_6_88_1","doi-asserted-by":"publisher","DOI":"10.1177\/0278364910379882"},{"key":"e_1_3_6_89_1","doi-asserted-by":"publisher","DOI":"10.1177\/0278364912473344"},{"key":"e_1_3_6_90_1","doi-asserted-by":"crossref","unstructured":"Takenaka T Matsumoto T Yoshiike T (2009a) Real time motion generation and control for biped robot-1 st report: walking gait pattern generation. In: 2009 IEEE\/RSJ International Conference on Intelligent Robots and Systems St. Louis MO USA 10\u201315 October 2009 pp. 1084\u20131091 IEEE.","DOI":"10.1109\/IROS.2009.5354662"},{"key":"e_1_3_6_91_1","doi-asserted-by":"crossref","unstructured":"Takenaka T Matsumoto T Yoshiike T et al. (2009b) Real time motion generation and control for biped robot-2 nd report: running gait pattern generation. In: 2009 IEEE\/RSJ International Conference on Intelligent Robots and Systems St. Louis MO USA 10\u201315 October 2009 pp. 1092\u20131099 IEEE.","DOI":"10.1109\/IROS.2009.5354654"},{"key":"e_1_3_6_92_1","doi-asserted-by":"crossref","unstructured":"Todorov E Erez T Tassa Y (2012) Mujoco: a physics engine for model-based control. In: 2012 IEEE\/RSJ international conference on intelligent robots and systems Vilamoura-Algarve Portugal 07\u201312 October 2012 pp. 5026\u20135033.","DOI":"10.1109\/IROS.2012.6386109"},{"key":"e_1_3_6_93_1","doi-asserted-by":"crossref","unstructured":"van Marum B Shrestha A Duan H et al. (2024) Revisiting reward design and evaluation for robust humanoid standing and walking. arXiv preprint arXiv:2404.19173.","DOI":"10.1109\/IROS58592.2024.10802680"},{"key":"e_1_3_6_94_1","doi-asserted-by":"publisher","DOI":"10.1142\/S0219843604000083"},{"key":"e_1_3_6_95_1","doi-asserted-by":"publisher","DOI":"10.1142\/S0219843615500395"},{"key":"e_1_3_6_96_1","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2023.3324580"},{"key":"e_1_3_6_97_1","unstructured":"Westenbroek T Castaneda F Agrawal A et al. (2022) Lyapunov design for robust and efficient robotic reinforcement learning. In: 6th Annual Conference on Robot Learning Auckland New Zealand 14 December 2022."},{"key":"e_1_3_6_98_1","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2002.806653"},{"key":"e_1_3_6_99_1","unstructured":"Wu J Xue Y Qi C (2023a) Learning multiple gaits within latent space for quadruped robots. arXiv preprint arXiv:2308.03014."},{"key":"e_1_3_6_100_1","unstructured":"Wu P Escontrela A Hafner D et al. (2023b) Daydreamer: world models for physical robot learning. In: Conference on Robot Learning Auckland New Zealand 14 December 2022 pp. 2226\u20132240 PMLR."},{"key":"e_1_3_6_101_1","doi-asserted-by":"crossref","unstructured":"Xie Z Berseth G Clary P et al. (2018) Feedback control for cassie with deep reinforcement learning. In: 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS) Madrid Spain 1\u20135 October 2018 1241\u20131246 IEEE.","DOI":"10.1109\/IROS.2018.8593722"},{"key":"e_1_3_6_102_1","first-page":"317","article-title":"Learning locomotion skills for cassie: iterative design and sim-to-real","volume":"100","author":"Xie Z","year":"2020","unstructured":"Xie Z, Clary P, Dao J, et al. (2020) Learning locomotion skills for cassie: iterative design and sim-to-real. Proceedings of the Conference on Robot Learning 100: 317\u2013329.","journal-title":"Proceedings of the Conference on Robot Learning"},{"key":"e_1_3_6_103_1","doi-asserted-by":"crossref","unstructured":"Xie Z Da X Van de Panne M et al. (2021) Dynamics randomization revisited: a case study for quadrupedal locomotion. In: 2021 IEEE International Conference on Robotics and Automation (ICRA) Xi\u2019an China 30 May\u201305 June 2021 pp. 4955\u20134961 IEEE.","DOI":"10.1109\/ICRA48506.2021.9560837"},{"key":"e_1_3_6_104_1","doi-asserted-by":"crossref","unstructured":"Xiong X Ames AD (2018) Bipedal hopping: reduced-order model embedding via optimization-based control. In: 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS) Madrid Spain 1\u20135 October 2018 pp. 3821\u20133828 IEEE.","DOI":"10.1109\/IROS.2018.8593547"},{"key":"e_1_3_6_105_1","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2022.3150219"},{"key":"e_1_3_6_106_1","doi-asserted-by":"crossref","unstructured":"Yang W Posa M (2021) Impact invariant control with applications to bipedal locomotion. In: 2021 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS) Prague Czech Republic 2021 pp. 5151\u20135158 IEEE.","DOI":"10.1109\/IROS51168.2021.9636094"},{"key":"e_1_3_6_107_1","unstructured":"Yang W Posa M (2023) Impact-invariant control: maximizing control authority during impacts. arXiv preprint arXiv:2303.00817."},{"key":"e_1_3_6_108_1","doi-asserted-by":"crossref","unstructured":"Yang L Li Z Zeng J et al. (2022) Bayesian optimization meets hybrid zero dynamics: safe parameter learning for bipedal locomotion control. In: 2022 International Conference on Robotics and Automation (ICRA) Philadelphia PA USA 23\u201327 May 2022 pp. 10456\u201310462 IEEE.","DOI":"10.1109\/ICRA46639.2022.9812154"},{"key":"e_1_3_6_109_1","doi-asserted-by":"crossref","unstructured":"Yu W Kumar VC Turk G et al. (2019) Sim-to-real transfer for biped locomotion. In: 2019 IEEE\/RSJ international conference on intelligent robots and systems (IROS) Macau China 03\u201308 November 2019. pp. 3503\u20133510.","DOI":"10.1109\/IROS40897.2019.8968053"},{"key":"e_1_3_6_110_1","doi-asserted-by":"crossref","unstructured":"Yu F Batke R Dao J et al. (2022) Dynamic bipedal turning through sim-to-real reinforcement learning. In: 2022 IEEE-RAS 21st International Conference on Humanoid Robots (Humanoids) Ginowan Japan 28\u201330 November 2022 pp. 903\u2013910 IEEE.","DOI":"10.1109\/Humanoids53995.2022.10000225"},{"key":"e_1_3_6_111_1","doi-asserted-by":"crossref","unstructured":"Yunt K Glocker C (2006) Trajectory optimization of mechanical hybrid systems using sumt. In: 9th IEEE International Workshop on Advanced Motion Control 2006 Istanbul Turkey 27\u201329 March 2006 pp. 665\u2013671 IEEE.","DOI":"10.1109\/AMC.2006.1631739"},{"key":"e_1_3_6_112_1","doi-asserted-by":"crossref","unstructured":"Zhu Y Pan Z Hauser K (2021) Contact-implicit trajectory optimization with learned deformable contacts using bilevel optimization. In: 2021 IEEE International Conference on Robotics and Automation (ICRA) Xi'an China 30 May\u201305 June 2021 pp. 9921\u20139927.","DOI":"10.1109\/ICRA48506.2021.9561521"}],"container-title":["The International Journal of Robotics Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/02783649241285161","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/02783649241285161","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/02783649241285161","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/02783649241285161","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T10:17:35Z","timestamp":1777457855000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/02783649241285161"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,23]]},"references-count":111,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2025,4]]}},"alternative-id":["10.1177\/02783649241285161"],"URL":"https:\/\/doi.org\/10.1177\/02783649241285161","relation":{},"ISSN":["0278-3649","1741-3176"],"issn-type":[{"value":"0278-3649","type":"print"},{"value":"1741-3176","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,23]]}}}