{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,21]],"date-time":"2026-04-21T05:09:39Z","timestamp":1776748179298,"version":"3.51.2"},"reference-count":39,"publisher":"American Association for the Advancement of Science (AAAS)","issue":"74","content-domain":{"domain":["www.science.org"],"crossmark-restriction":true},"short-container-title":["Sci. Robot."],"published-print":{"date-parts":[[2023,1,25]]},"abstract":"<jats:p>Simulation-based reinforcement learning approaches are leading the next innovations in legged robot control. However, the resulting control policies are still not applicable on soft and deformable terrains, especially at high speed. The primary reason is that reinforcement learning approaches, in general, are not effective beyond the data distribution: The agent cannot perform well in environments that it has not experienced. To this end, we introduce a versatile and computationally efficient granular media model for reinforcement learning. Our model can be parameterized to represent diverse types of terrain from very soft beach sand to hard asphalt. In addition, we introduce an adaptive control architecture that can implicitly identify the terrain properties as the robot feels the terrain. The identified parameters are then used to boost the locomotion performance of the legged robot. We applied our techniques to the Raibo robot, a dynamic quadrupedal robot developed in-house. The trained networks demonstrated high-speed locomotion capabilities on deformable terrains: The robot was able to run on soft beach sand at 3.03 meters per second although the feet were completely buried in the sand during the stance phase. We also demonstrate its ability to generalize to different terrains by presenting running experiments on vinyl tile flooring, athletic track, grass, and a soft air mattress.<\/jats:p>","DOI":"10.1126\/scirobotics.ade2256","type":"journal-article","created":{"date-parts":[[2023,1,25]],"date-time":"2023-01-25T18:58:16Z","timestamp":1674673096000},"update-policy":"https:\/\/doi.org\/10.34133\/aaas_crossmark","source":"Crossref","is-referenced-by-count":146,"title":["Learning quadrupedal locomotion on deformable terrain"],"prefix":"10.1126","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3116-1072","authenticated-orcid":false,"given":"Suyoung","family":"Choi","sequence":"first","affiliation":[{"name":"Robotics &amp; Artificial Intelligence Lab, KAIST, Daejeon, Korea."}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6478-6426","authenticated-orcid":false,"given":"Gwanghyeon","family":"Ji","sequence":"additional","affiliation":[{"name":"Robotics &amp; Artificial Intelligence Lab, KAIST, Daejeon, Korea."}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7619-1884","authenticated-orcid":false,"given":"Jeongsoo","family":"Park","sequence":"additional","affiliation":[{"name":"Robotics &amp; Artificial Intelligence Lab, KAIST, Daejeon, Korea."}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1422-449X","authenticated-orcid":false,"given":"Hyeongjun","family":"Kim","sequence":"additional","affiliation":[{"name":"Robotics &amp; Artificial Intelligence Lab, KAIST, Daejeon, Korea."}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6558-3660","authenticated-orcid":false,"given":"Juhyeok","family":"Mun","sequence":"additional","affiliation":[{"name":"Robotics &amp; Artificial Intelligence Lab, KAIST, Daejeon, Korea."}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6598-2875","authenticated-orcid":false,"given":"Jeong Hyun","family":"Lee","sequence":"additional","affiliation":[{"name":"Robotics &amp; Artificial Intelligence Lab, KAIST, Daejeon, Korea."}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3444-8079","authenticated-orcid":false,"given":"Jemin","family":"Hwangbo","sequence":"additional","affiliation":[{"name":"Robotics &amp; Artificial Intelligence Lab, KAIST, Daejeon, Korea."}]}],"member":"221","reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"crossref","unstructured":"H. Kolvenbach P.\u00a0Arm E.\u00a0Hampp A.\u00a0Dietsche V.\u00a0Bickel B.\u00a0Sun C.\u00a0Meyer M.\u00a0Hutter Traversing steep and granular martian analog slopes with a dynamic quadrupedal robot. arXiv:2106.01974 (2021).","DOI":"10.55417\/fr.2022030"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1242\/jeb.161109"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1098\/rsbl.2009.1041"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.1255718"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.abc5986"},{"key":"e_1_3_2_7_2","doi-asserted-by":"crossref","unstructured":"A. Kumar Z.\u00a0Fu D.\u00a0Pathak J.\u00a0Malik RMA: Rapid motor adaptation for legged robots in Proceedings of the Robotics: Science and Systems (2021).","DOI":"10.15607\/RSS.2021.XVII.011"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2022.3151396"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1177\/0278364917694244"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0809095106"},{"key":"e_1_3_2_11_2","doi-asserted-by":"crossref","unstructured":"C.\u00a0M.\u00a0Hubicki J.\u00a0J.\u00a0Aguilar D.\u00a0I.\u00a0Goldman A.\u00a0D.\u00a0Ames Tractable terrain-aware motion planning on granular media: An impulsive jumping study in Proceedings of the 2016 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IEEE 2016) pp.\u00a03887\u20133892.","DOI":"10.1109\/IROS.2016.7759572"},{"key":"e_1_3_2_12_2","doi-asserted-by":"crossref","unstructured":"V. Vasilopoulos I.\u00a0S.\u00a0Paraskevas E.\u00a0G.\u00a0Papadopoulos Compliant terrain legged locomotion using a viscoplastic approach in Proceedings of the 2014 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IEEE 2014) pp.\u00a04849\u20134854.","DOI":"10.1109\/IROS.2014.6943251"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2020.2977260"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCST.2020.3009636"},{"key":"e_1_3_2_15_2","doi-asserted-by":"crossref","unstructured":"J. Siekmann K.\u00a0Green J.\u00a0Warila A.\u00a0Fern J.\u00a0Hurst Blind bipedal stair traversal via sim-to-real reinforcement learning in Proceedings of the Robotics: Science and Systems (2021).","DOI":"10.15607\/RSS.2021.XVII.061"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.aau5872"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.abk2822"},{"key":"e_1_3_2_18_2","doi-asserted-by":"crossref","unstructured":"T. Haarnoja S.\u00a0Ha A.\u00a0Zhou J.\u00a0Tan G.\u00a0Tucker S.\u00a0Levine Learning to walk via deep reinforcement learning in Proceedings of the Robotics: Science and Systems (2019).","DOI":"10.15607\/RSS.2019.XV.011"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2020.2974685"},{"key":"e_1_3_2_20_2","doi-asserted-by":"crossref","unstructured":"X.\u00a0B.\u00a0Peng M.\u00a0Andrychowicz W.\u00a0Zaremba P.\u00a0Abbeel Sim-to-real transfer of robotic control with dynamics randomization in Proceedings of the 2018 IEEE International Conference on Robotics and Automation (IEEE 2018) pp.\u00a03803\u20133810.","DOI":"10.1109\/ICRA.2018.8460528"},{"key":"e_1_3_2_21_2","doi-asserted-by":"crossref","unstructured":"R. Kaushik T.\u00a0Anne J.\u00a0Mouret Fast online adaptation in robotics through meta-learning embeddings of simulated priors in Proceedings of the 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IEEE 2020) pp.\u00a05269\u20135276.","DOI":"10.1109\/IROS45743.2020.9341462"},{"key":"e_1_3_2_22_2","unstructured":"E. Coumans Y.\u00a0Bai Pybullet a python module for physics simulation for games robotics and machine learning (2016);pybullet.org."},{"key":"e_1_3_2_23_2","doi-asserted-by":"crossref","unstructured":"E. Todorov T.\u00a0Erez Y.\u00a0Tassa MuJoCo: A physics engine for model-based control in Proceedings of the 2012 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IEEE 2012) pp.\u00a05026\u20135033.","DOI":"10.1109\/IROS.2012.6386109"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2018.2792536"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1680\/geot.1979.29.1.47"},{"key":"e_1_3_2_26_2","unstructured":"T. P\u00f6schel T.\u00a0Schwager Computational Granular Dynamics: Models and Algorithms (Springer-Verlag 2005)."},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.1229163"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1038\/nphys3568"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1126\/sciadv.abe0631"},{"key":"e_1_3_2_30_2","unstructured":"M. Sundararajan A.\u00a0Taly Q.\u00a0Yan Axiomatic attribution for deep networks arXiv:1703.01365 (2017)."},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1038\/380340a0"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.87.052208"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"e_1_3_2_34_2","unstructured":"J. Schulman F.\u00a0Wolski P.\u00a0Dhariwal A.\u00a0Radford O.\u00a0Klimov Proximal policy optimization algorithms arXiv:1707.06347 (2017)."},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1177\/0278364919887447"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1996.8.3.643"},{"key":"e_1_3_2_37_2","doi-asserted-by":"crossref","unstructured":"T. Fields G.\u00a0Hsieh J.\u00a0Chenou Mitigating drift in time series data with noise augmentation in Proceedings of the 2019 International Conference on Computational Science and Computational Intelligence (IEEE 2019) pp.\u00a0227\u2013230.","DOI":"10.1109\/CSCI49370.2019.00046"},{"key":"e_1_3_2_38_2","doi-asserted-by":"crossref","unstructured":"B. Katz J.\u00a0D.\u00a0Carlo S.\u00a0Kim Mini cheetah: A platform for pushing the limits of dynamic quadruped control in Proceedings of the 2019 IEEE International Conference on Robotics and Automation (IEEE 2019) pp.\u00a06295\u20136301.","DOI":"10.1109\/ICRA.2019.8793865"},{"key":"e_1_3_2_39_2","unstructured":"S. Bai J.\u00a0Z.\u00a0Kolter V.\u00a0Koltun An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv:1803.01271 [cs.LG] (4 Mar 2018)."},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1038\/nphys583"}],"container-title":["Science Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.science.org\/doi\/pdf\/10.1126\/scirobotics.ade2256","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,1,16]],"date-time":"2024-01-16T11:25:37Z","timestamp":1705404337000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.science.org\/doi\/10.1126\/scirobotics.ade2256"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,25]]},"references-count":39,"journal-issue":{"issue":"74","published-print":{"date-parts":[[2023,1,25]]}},"alternative-id":["10.1126\/scirobotics.ade2256"],"URL":"https:\/\/doi.org\/10.1126\/scirobotics.ade2256","relation":{},"ISSN":["2470-9476"],"issn-type":[{"value":"2470-9476","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,25]]},"assertion":[{"value":"2022-08-02","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-12-21","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-01-25","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"eade2256"}}