{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,28]],"date-time":"2026-02-28T09:57:18Z","timestamp":1772272638040,"version":"3.50.1"},"reference-count":50,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,1,23]],"date-time":"2023-01-23T00:00:00Z","timestamp":1674432000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurorobot."],"abstract":"<jats:p>Generating multimodal locomotion in underactuated bipedal robots requires control solutions that can facilitate motion patterns for drastically different dynamical modes, which is an extremely challenging problem in locomotion-learning tasks. Also, in such multimodal locomotion, utilizing body morphology is important because it leads to energy-efficient locomotion. This study provides a framework that reproduces multimodal bipedal locomotion using passive dynamics through deep reinforcement learning (DRL). An underactuated bipedal model was developed based on a passive walker, and a controller was designed using DRL. By carefully planning the weight parameter settings of the DRL reward function during the learning process based on a curriculum learning method, the bipedal model successfully learned to walk, run, and perform gait transitions by adjusting only one command input. These results indicate that DRL can be applied to generate various gaits with the effective use of passive dynamics.<\/jats:p>","DOI":"10.3389\/fnbot.2022.1054239","type":"journal-article","created":{"date-parts":[[2023,1,23]],"date-time":"2023-01-23T06:01:10Z","timestamp":1674453670000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["Multimodal bipedal locomotion generation with passive dynamics via deep reinforcement learning"],"prefix":"10.3389","volume":"16","author":[{"given":"Shunsuke","family":"Koseki","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kyo","family":"Kutsuzawa","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dai","family":"Owaki","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mitsuhiro","family":"Hayashibe","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2023,1,23]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"262","DOI":"10.2307\/3619558","article-title":"Walking and running","volume":"80","author":"Alexander","year":"1996","journal-title":"Math. Gazette"},{"key":"B2","doi-asserted-by":"crossref","first-page":"1067","DOI":"10.70252\/NRSU8209","article-title":"Is the relationship between stride length, frequency, and velocity influenced by running on a treadmill or overground?","volume":"10","author":"Bailey","year":"2017","journal-title":"Int. J. Exerc. Sci"},{"key":"B3","doi-asserted-by":"publisher","first-page":"441","DOI":"10.1142\/97898144159580057","article-title":"\u201cDesign and control of ranger: an energy-efficient, dynamic walking robot,\u201d","author":"Bhounsule","year":"2012","journal-title":"Proceedings of CLAWAR2012"},{"key":"B4","doi-asserted-by":"publisher","first-page":"1217","DOI":"10.1016\/0021-9290(89)90224-8","article-title":"The spring-mass model for running and hopping","volume":"22","author":"Blickhan","year":"1989","journal-title":"J. Biomech"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2010.03848","article-title":"Guided curriculum learning for walking over complex terrain","author":"Brendan","year":"2020","journal-title":"arXiv:2010.03848 [cs.RO]"},{"key":"B6","doi-asserted-by":"publisher","first-page":"3426","DOI":"10.1152\/jn.00081.2006","article-title":"Motor patterns in human walking and running","volume":"95","author":"Cappellini","year":"2006","journal-title":"J. Neurophysiol"},{"key":"B7","doi-asserted-by":"crossref","first-page":"5136","DOI":"10.1109\/IROS51168.2021.9636467","article-title":"\u201cRobust feedback motion policy design using reinforcement learning on a 3D digit bipedal robot,\u201d","volume-title":"2021 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)","author":"Castillo","year":"2021"},{"key":"B8","doi-asserted-by":"publisher","first-page":"4051","DOI":"10.1242\/jeb.02344","article-title":"The landing-take-off asymmetry in human running","volume":"209","author":"Cavagna","year":"2006","journal-title":"J. Exp. Biol"},{"key":"B9","doi-asserted-by":"publisher","first-page":"657","DOI":"10.1111\/j.1469-7793.2000.00657.x","article-title":"The role of gravity in human walking: pendular energy exchange, external work and optimal speed","author":"Cavagna","year":"2000","journal-title":"J. Physiol"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.1242\/jeb.232645","article-title":"The phase shift between potential and kinetic energy in human walking","author":"Cavagna","year":"2020","journal-title":"J. Exp. Biol"},{"key":"B11","doi-asserted-by":"publisher","first-page":"639","DOI":"10.1113\/jphysiol.1976.sp011613","article-title":"The sources of external work in level walking and running","volume":"262","author":"Cavagna","year":"1976","journal-title":"J. Physiol"},{"key":"B12","doi-asserted-by":"crossref","first-page":"1983","DOI":"10.1109\/ROBOT.2005.1570404","article-title":"\u201cA bipedal walking robot with efficient and human-like gait,\u201d","author":"Collins","year":"2005","journal-title":"Proceedings of the 2005 IEEE International Conference on Robotics and Automation"},{"key":"B13","doi-asserted-by":"publisher","first-page":"100","DOI":"10.1126\/science.288.5463.100","article-title":"How animals move: an integrative view","volume":"288","author":"Dickinson","year":"2000","journal-title":"Science"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1037\/0096-1523.21.1.183","article-title":"Why change gaits? dynamics of the walk-run transition","author":"Diedrich","year":"1995","journal-title":"J. Exp. Psychol"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.3389\/fnhum.2014.00371","article-title":"The contribution of a central pattern generator in a reflex-based neuromuscular model","author":"Dzeladini","year":"2014","journal-title":"Front. Hum. Neurosci"},{"key":"B16","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1016\/0021-9290(95)00029-1","article-title":"Leg stiffness and stride frequency in human running","volume":"29","author":"Farley","year":"1996","journal-title":"J. Biomech"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2111.01674","article-title":"Minimizing energy consumption leads to the emergence of gaits in legged robots","author":"Fu","year":"2021","journal-title":"arXiv:2111.01674 [cs.RO]"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2019.XV.011","article-title":"Learning to walk via deep reinforcement learning","author":"Haarnoja","year":"","journal-title":"arXiv preprint"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1812.05905","article-title":"Soft actor-critic algorithms and applications","author":"Haarnoja","year":"","journal-title":"arXiv preprint"},{"key":"B20","doi-asserted-by":"publisher","first-page":"407","DOI":"10.1016\/j.clinbiomech.2003.12.001","article-title":"Roll-over shapes of human locomotor systems: effects of walking speed","volume":"19","author":"Hansen","year":"2004","journal-title":"Clin. Biomech"},{"key":"B21","doi-asserted-by":"publisher","first-page":"2092","DOI":"10.1109\/ROBOT.1991.131936","article-title":"\u201cBiped gait transitions,\u201d","author":"Hodgins","year":"1991","journal-title":"Proceedings of 1991 IEEE International Conference on Robotics and Automation"},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.aau5872","article-title":"Learning agile and dynamic motor skills for legged robots","author":"Hwangbo","year":"2019","journal-title":"Sci. Rob"},{"key":"B23","doi-asserted-by":"publisher","first-page":"20662","DOI":"10.1109\/ACCESS.2021.3055241","article-title":"Finite state machine-based motion-free learning of biped walking","volume":"9","author":"Kang","year":"2021","journal-title":"IEEE Access"},{"key":"B24","doi-asserted-by":"crossref","first-page":"1769","DOI":"10.1109\/ICRA.2016.7487321","article-title":"\u201cUnified bipedal gait for walking and running by dynamics-based virtual holonomic constraint in PDAC,\u201d","author":"Kobayashi","year":"2016","journal-title":"2016 IEEE International Conference on Robotics and Automation (ICRA)"},{"key":"B25","doi-asserted-by":"publisher","first-page":"617","DOI":"10.1016\/j.humov.2007.04.003","article-title":"The six determinants of gait and the inverted pendulum analogy: a dynamic walking perspective","volume":"26","author":"Kuo","year":"2007","journal-title":"Hum. Mov. Sci"},{"key":"B26","doi-asserted-by":"crossref","first-page":"1350","DOI":"10.1109\/ROBOT.2003.1241779","article-title":"\u201cGait transitions for walking and running of biped robots,\u201d","author":"Kwon","year":"2003","journal-title":"2003 IEEE International Conference On Robotics and Automation (Cat. No. 03CH37422), Vol. 1"},{"key":"B27","doi-asserted-by":"crossref","first-page":"2811","DOI":"10.1109\/ICRA48506.2021.9560769","article-title":"\u201cReinforcement learning for robust parameterized locomotion control of bipedal robots,\u201d","volume-title":"2021 IEEE International Conference on Robotics and Automation (ICRA)","author":"Li","year":"2021"},{"key":"B28","doi-asserted-by":"publisher","first-page":"62","DOI":"10.1177\/027836499000900206","article-title":"Passive dynamic walking","volume":"9","author":"McGeer","year":"1990","journal-title":"Int. J. Robotics Res"},{"key":"B29","doi-asserted-by":"publisher","first-page":"467","DOI":"10.1006\/jtbi.1997.0407","article-title":"A theory of metabolic costs for bipedal gaits","volume":"186","author":"Minetti","year":"1997","journal-title":"J. Theor. Biol"},{"key":"B30","doi-asserted-by":"crossref","first-page":"3189","DOI":"10.1109\/ROBOT.2004.1308745","article-title":"\u201cIntegrated motion control for walking, jumping and running on a small bipedal entertainment robot,\u201d","author":"Nagasaka","year":"2004","journal-title":"IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA'04. 2004, Vol. 4"},{"key":"B31","doi-asserted-by":"publisher","first-page":"1121","DOI":"10.1177\/0954411911424210","article-title":"Mass-spring-damper modelling of the human body to study running and hopping-an overview","volume":"225","author":"Nikooyan","year":"2011","journal-title":"Proc. Instit. Mech. Eng. H J. Eng. Med"},{"key":"B32","doi-asserted-by":"publisher","DOI":"10.3389\/fnbot.2018.00043","article-title":"Generation of human-like movement from symbolized information","author":"Okajima","year":"2018","journal-title":"Front. Neurorobot"},{"key":"B33","doi-asserted-by":"publisher","first-page":"156","DOI":"10.1109\/TRO.2010.2098610","article-title":"A 2-d passive-dynamic-running biped with elastic elements","volume":"27","author":"Owaki","year":"2011","journal-title":"IEEE Trans. Rob"},{"key":"B34","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1109\/ROBOT.2008.4543231","article-title":"\u201cOn the embodiment that enables passive dynamic bipedal running,\u201d","author":"Owaki","year":"2008","journal-title":"2008 IEEE International Conference on Robotics and Automation"},{"key":"B35","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1007\/978-3-642-00616-6_5","article-title":"\u201cMorphological computation-connecting brain, body, and environment,\u201d","author":"Pfeifer","year":"2009","journal-title":"Creating Brain-Like Intelligence"},{"key":"B36","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/6979.001.0001","author":"Pfeifer","year":"2001","journal-title":"Understanding Intelligence"},{"key":"B37","doi-asserted-by":"publisher","first-page":"170","DOI":"10.1016\/j.jtbi.2005.04.004","article-title":"A collisional model of the energetic cost of support work qualitatively explains leg sequencing in walking and galloping, pseudo-elastic leg behavior in running and the walk-to-run transition","volume":"237","author":"Ruina","year":"2005","journal-title":"J. Theor. Biol"},{"key":"B38","first-page":"3783","article-title":"\u201cA neural primitive model with sensorimotor coordination for dynamic quadruped locomotion with malfunction compensation,\u201d","author":"Saputra","year":"2020","journal-title":"2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)"},{"key":"B39","doi-asserted-by":"publisher","first-page":"1230","DOI":"10.1109\/LRA.2021.3136645","article-title":"Learning free gait transition for quadruped robots via phase-guided controller","volume":"7","author":"Shao","year":"2021","journal-title":"IEEE Rob. Autom. Lett"},{"key":"B40","author":"Sharbafi","year":"2017","journal-title":"Bioinspired Legged Locomotion"},{"key":"B41","doi-asserted-by":"crossref","first-page":"7309","DOI":"10.1109\/ICRA48506.2021.9561814","article-title":"\u201cSim-to-real learning of all common bipedal gaits via periodic reward composition,\u201d","author":"Siekmann","year":"2021","journal-title":"2021 IEEE International Conference on Robotics and Automation (ICRA)"},{"key":"B42","doi-asserted-by":"publisher","first-page":"1124","DOI":"10.1109\/LRA.2017.2661801","article-title":"The energetic benefit of robotic gait selection\u2013a case study on the robot ramone","volume":"2","author":"Smit-Anseeuw","year":"2017","journal-title":"IEEE Rob. Autom. Lett"},{"key":"B43","doi-asserted-by":"publisher","first-page":"324","DOI":"10.1177\/0278364912473344","article-title":"Embedding active force control within the compliant hybrid zero dynamics to achieve stable, fast running on mabel","volume":"32","author":"Sreenath","year":"2013","journal-title":"Int. J. Rob. Res"},{"key":"B44","doi-asserted-by":"publisher","first-page":"72","DOI":"10.1038\/nature04113","article-title":"Computer optimization of a minimal biped model discovers walking and running","volume":"439","author":"Srinivasan","year":"2006","journal-title":"Nature"},{"key":"B45","doi-asserted-by":"crossref","first-page":"1571","DOI":"10.1109\/ROBOT.2009.5152404","article-title":"\u201cFast running experiments involving a humanoid robot,\u201d","author":"Tajima","year":"2009","journal-title":"2009 IEEE International Conference on Robotics and Automation"},{"key":"B46","first-page":"5026","article-title":"\u201cMujoco: a physics engine for model-based control,\u201d","author":"Todorov","year":"2012","journal-title":"2012 IEEE\/RSJ International Conference on Intelligent Robots and Systems"},{"key":"B47","doi-asserted-by":"publisher","first-page":"1241","DOI":"10.1109\/IROS.2018.8593722","author":"Xie","year":"2018"},{"key":"B48","doi-asserted-by":"publisher","first-page":"213","DOI":"10.1111\/cgf.14115","article-title":"Allsteps: curriculum-driven learning of stepping stone skills","volume":"39","author":"Xie","year":"2020","journal-title":"Comput. Graph. Forum"},{"key":"B49","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3197517.3201397","article-title":"Learning symmetric and low-energy locomotion","volume":"37","author":"Yu","year":"2018","journal-title":"ACM Trans. Graph"},{"key":"B50","doi-asserted-by":"publisher","first-page":"639","DOI":"10.1016\/S0021-9290(01)00243-3","article-title":"A model analysis of internal loads, energetics, and effects of wobbling mass during the whole-body vibration","volume":"35","author":"Yue","year":"2002","journal-title":"J. Biomech"}],"container-title":["Frontiers in Neurorobotics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2022.1054239\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,12]],"date-time":"2024-10-12T19:15:26Z","timestamp":1728760526000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2022.1054239\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,23]]},"references-count":50,"alternative-id":["10.3389\/fnbot.2022.1054239"],"URL":"https:\/\/doi.org\/10.3389\/fnbot.2022.1054239","relation":{},"ISSN":["1662-5218"],"issn-type":[{"value":"1662-5218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,23]]},"article-number":"1054239"}}