{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T04:50:13Z","timestamp":1791521413952,"version":"4.3.4"},"reference-count":45,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T00:00:00Z","timestamp":1791331200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T00:00:00Z","timestamp":1791331200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Science and Technology Council","award":["113-2221-E-003-010"],"award-info":[{"award-number":["113-2221-E-003-010"]}]},{"name":"National Science and Technology Council","award":["112-2221-E-003-013"],"award-info":[{"award-number":["112-2221-E-003-013"]}]},{"award":["113-2221-E-003-010"],"award-info":[{"award-number":["113-2221-E-003-010"]}],"id":[{"id":"https:\/\/ror.org\/02kv4zf79","id-type":"ROR","asserted-by":"publisher"}]},{"award":["112-2221-E-003-013"],"award-info":[{"award-number":["112-2221-E-003-013"]}],"id":[{"id":"https:\/\/ror.org\/02kv4zf79","id-type":"ROR","asserted-by":"publisher"}]},{"name":"National Taiwan Normal University","id":[{"id":"https:\/\/ror.org\/059dkdx38","id-type":"ROR","asserted-by":"crossref"}]},{"name":"Gogoro Corporation"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This paper presents a simulation-to-reality (sim-to-real) transfer process, allowing a full-sized humanoid robot to autonomously balance a two-wheeled scooter and track operator-supplied heading commands through a deep reinforcement learning (DRL) policy, including while carrying a human passenger for the first time. The learned policy governs the coupled balance and steering dynamics only: the heading reference and the target speed are supplied by a human operator, the throttle is set externally, and the initial launch over the first 2 to 3 m and the final stop are performed manually for safety. The system therefore performs autonomous balance and steering-command tracking during motion, rather than autonomous scooter driving. Unlike four-wheeled vehicles where the inherent stability simplifies control, scooter operation demands continuous and precise dynamic balancing coupled with real-time steering control, creating a challenging full-body control task. In this study, we demonstrate a successful integration of DRL techniques to bridge the sim-to-real gap, achieving stable closed-loop control of a humanoid robot balancing and steering a scooter under significant model uncertainty. A multi-scenario real-world evaluation shows the system maintaining control in the demanding low-speed regime of 0.8 to 3.0 m\u00a0s\u22121, where balance control is most challenging and where speed varied across the range during trials rather than being held at a set point. Success rates are 0.96 on straight-line driving, 0.92 and 0.72 on light and sharp turns, 0.88 on the traversal of a 60 mm speed bump, and 0.73 while carrying a 70 kg passenger. They fall where sustained precise tracking is required, to 0.28 on a 4.5 m roundabout and 0.33 on part of the standardized Taiwanese scooter license test.<\/jats:p>","DOI":"10.3390\/s26196324","type":"journal-article","created":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T10:13:02Z","timestamp":1791367982000},"page":"6324","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Scooter-Driving Humanoid Robot: Sim-to-Real Transfer Through Deep Reinforcement Learning"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-6640-3273","authenticated-orcid":false,"given":"Ugo Richard","family":"Roux","sequence":"first","affiliation":[{"name":"Educational Robotics Center, Department of Electrical Engineering, National Taiwan Normal University, Taipei 106, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Saeed","family":"Saeedvand","sequence":"additional","affiliation":[{"name":"Educational Robotics Center, Department of Electrical Engineering, National Taiwan Normal University, Taipei 106, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3360-1892","authenticated-orcid":false,"given":"Jacky","family":"Baltes","sequence":"additional","affiliation":[{"name":"Educational Robotics Center, Department of Electrical Engineering, National Taiwan Normal University, Taipei 106, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,10,7]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Gong, Y., Hartley, R., Da, X., Hereid, A., Harib, O., Huang, J.K., and Grizzle, J. (2019, January 10\u201312). Feedback Control of a Cassie Bipedal Robot: Walking, Standing, and Riding a Segway. Proceedings of the 2019 American Control Conference (ACC), Philadelphia, PA, USA.","DOI":"10.23919\/ACC.2019.8814833"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Gong, Y., Sun, G., Nair, A., Bidwai, A., CS, R., Grezmak, J., Sartoretti, G., and Daltorio, K.A. (2023). Legged Robots for Object Manipulation: A Review. arXiv.","DOI":"10.3389\/fmech.2023.1142421"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1087","DOI":"10.1007\/s10514-023-10139-z","article-title":"Tidybot: Personalized robot assistance with large language models","volume":"47","author":"Wu","year":"2023","journal-title":"Auton. Robot."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1109\/MRA.2019.2943395","article-title":"Humanoid Robots in Aircraft Manufacturing: The Airbus Use Cases","volume":"26","author":"Kheddar","year":"2019","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"107601","DOI":"10.1016\/j.asoc.2021.107601","article-title":"Hierarchical deep reinforcement learning to drag heavy objects by adult-sized humanoid robot","volume":"110","author":"Saeedvand","year":"2021","journal-title":"Appl. Soft Comput."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Mandala, H., Saeedvand, S., and Baltes, J. (2020, January 19\u201321). Synchronous Dual-Arm Manipulation by Adult-Sized Humanoid Robot. Proceedings of the 2020 International Conference on Advanced Robotics and Intelligent Systems (ARIS), Taipei, Taiwan.","DOI":"10.1109\/ARIS50834.2020.9205783"},{"key":"ref_7","unstructured":"Christmann, G., Yu-Ren, L., da Silva Guerra, R., and Baltes, J. (2022). Can a Robot Shoot an Olympic Recurve Bow? A preliminary study. arXiv."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"e6","DOI":"10.1017\/S0269888923000061","article-title":"An analysis and review of robot magic shows","volume":"38","author":"Yang","year":"2023","journal-title":"Knowl. Eng. Rev."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"106941","DOI":"10.1016\/j.engappai.2023.106941","article-title":"A deep reinforcement learning algorithm to control a two-wheeled scooter with a humanoid robot","volume":"126","author":"Baltes","year":"2023","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_10","unstructured":"Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal Policy Optimization Algorithms. arXiv."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Peng, X.B., Andrychowicz, M., Zaremba, W., and Abbeel, P. (2018). Sim-to-Real Transfer of Robotic Control with Dynamics Randomization. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), IEEE.","DOI":"10.1109\/ICRA.2018.8460528"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Liao, Q., Zhang, B., Huang, X., Huang, X., Li, Z., and Sreenath, K. (2025, January 19\u201323). Berkeley Humanoid: A Research Platform for Learning-Based Control. Proceedings of the 2025 IEEE International Conference on Robotics and Automation (ICRA), Atlanta, GA, USA.","DOI":"10.1109\/ICRA55743.2025.11127524"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1002\/rob.21731","article-title":"Autonomous car driving by a humanoid robot","volume":"35","author":"Paolillo","year":"2018","journal-title":"J. Field Robot."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"407","DOI":"10.1007\/s10846-019-01130-x","article-title":"Ground Vehicle Driving by Full Sized Humanoid","volume":"99","author":"Sohn","year":"2020","journal-title":"J. Intell. Robot. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1109\/MRA.2020.2987805","article-title":"Toward Autonomous Driving by Musculoskeletal Humanoids: A Study of Developed Hardware and Learning-Based Software","volume":"27","author":"Kawaharazuka","year":"2020","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_16","unstructured":"Bjorck, J., Casta\u00f1eda, F., Cherniadev, N., Da, X., Ding, R., Fan, L., Fang, Y., Fox, D., Hu, F., and Huang, S. (2025). Gr00t n1: An open foundation model for generalist humanoid robots. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Black, K., Brown, N., Driess, D., Esmail, A., Equi, M., Finn, C., Fusai, N., Groom, L., Hausman, K., and Ichter, B. (2024). \u03c00: A Vision-Language-Action Flow Model for General Robot Control. arXiv.","DOI":"10.15607\/RSS.2025.XXI.010"},{"key":"ref_18","unstructured":"Zhao, T.Z., Tompson, J., Driess, D., Florence, P., Ghasemipour, K., Finn, C., and Wahid, A. (2024). ALOHA Unleashed: A Simple Recipe for Robot Dexterity. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"84837","DOI":"10.1109\/ACCESS.2020.2992792","article-title":"Bicycle robot balance control based on a robust intelligent controller","volume":"8","author":"Chiu","year":"2020","journal-title":"IEEE Access"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Suebsomran, A. (2012, January 27\u201331). Balancing control of bicycle robot. Proceedings of the 2012 IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems (CYBER), Bangkok, Thailand.","DOI":"10.1109\/CYBER.2012.6392529"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"102386","DOI":"10.1016\/j.mechatronics.2020.102386","article-title":"Development and control of a bicycle robot based on steering and pendulum balancing","volume":"69","author":"Seekhao","year":"2020","journal-title":"Mechatronics"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Klemm, V., Morra, A., Salzmann, C., Tschopp, F., Bodie, K., Gulich, L., K\u00fcng, N., Mannhart, D., Pfister, C., and Vierneisel, M. (2019, January 20\u201324). Ascento: A Two-Wheeled Jumping Robot. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8793792"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"607","DOI":"10.1007\/s42235-023-00468-1","article-title":"Development of wheel-legged biped robots: A review","volume":"21","author":"Liu","year":"2024","journal-title":"J. Bionic Eng."},{"key":"ref_24","unstructured":"Tanaka, Y., and Murakami, T. (2004, January 28). Self sustaining bicycle robot with steering controller. Proceedings of the 8th IEEE International Workshop on Advanced Motion Control, Kawasaki, Japan."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"188","DOI":"10.20485\/jsaeijae.13.4_188","article-title":"Robust Control Strategy for Robotic Motorcycle Without Falling Down at Low-Speed Driving","volume":"13","author":"Tsuchiya","year":"2022","journal-title":"Int. J. Automot. Eng."},{"key":"ref_26","unstructured":"Yi, J., Song, D., Levandowski, A., and Jayasuriya, S. (2006, January 15\u201319). Trajectory tracking and balance stabilization control of autonomous motorcycles. Proceedings of the Proceedings 2006 IEEE International Conference on Robotics and Automation, Orlando, FL, USA."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"JAMDSM0030","DOI":"10.1299\/jamdsm.2020jamdsm0030","article-title":"Robust control system design for self-standable motorcycle","volume":"14","author":"Hara","year":"2020","journal-title":"J. Adv. Mech. Des. Syst. Manuf."},{"key":"ref_28","unstructured":"RAI Institute (2025, November 03). Designing Wheeled Robotic Systems for Ultra Mobility and Performance. Available online: https:\/\/rai-inst.com\/resources\/blog\/designing-wheeled-robotic-systems\/."},{"key":"ref_29","unstructured":"Yamaguchi, M.D.G. (2025, January 13). Primer V2: The Bicycle Riding Robot. Available online: https:\/\/ai2001.ifdef.jp\/primer_V2\/primer_V2.html."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"918","DOI":"10.1080\/01691864.2018.1509017","article-title":"Balancing control of a bicycle-riding humanoid robot with center of gravity estimation","volume":"32","author":"Huang","year":"2018","journal-title":"Adv. Robot."},{"key":"ref_31","unstructured":"Yamaha Motor Co., Ltd. (2026, September 08). MOTOBOT: A Motorcycle-Riding Humanoid Robot. Available online: https:\/\/global.yamaha-motor.com\/design_technology\/technology\/motobot\/."},{"key":"ref_32","unstructured":"AgiBot (2026, September 08). AGIBOT X2: A Generalist Humanoid Robot Advancing Motion, Interaction, and Task Intelligence. Video Published 10 March 2025. Available online: https:\/\/www.youtube.com\/watch?v=iyCjevFGLiA."},{"key":"ref_33","unstructured":"Roux, U. (2026, September 29). Scooter-Driving Humanoid Robot. First Public Demonstration. Video Published 21 August 2024. Available online: https:\/\/youtu.be\/II6gaoqaYfU."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Radosavovic, I., Xiao, T., Zhang, B., Darrell, T., Malik, J., and Sreenath, K. (2023). Real-World Humanoid Locomotion with Reinforcement Learning. arXiv.","DOI":"10.1126\/scirobotics.adi9579"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Castillo, G.A., Weng, B., Zhang, W., and Hereid, A. (2021). Robust Feedback Motion Policy Design Using Reinforcement Learning on a 3D Digit Bipedal Robot. arXiv.","DOI":"10.1109\/IROS51168.2021.9636467"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Crowley, D., Dao, J., Duan, H., Green, K., Hurst, J., and Fern, A. (June, January 29). Optimizing Bipedal Locomotion for The 100m Dash with Comparison to Human Running. Proceedings of the 2023 IEEE International Conference on Robotics and Automation (ICRA), London, UK.","DOI":"10.1109\/ICRA48891.2023.10160436"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"3154","DOI":"10.1109\/TRO.2023.3271136","article-title":"Vertical Jump of a Humanoid Robot with CoP-Guided Angular Momentum Control and Impact Absorption","volume":"39","author":"Qi","year":"2023","journal-title":"IEEE Trans. Robot."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Dao, J., Duan, H., and Fern, A. (2023). Sim-to-Real Learning for Humanoid Box Loco-Manipulation. arXiv.","DOI":"10.1109\/ICRA57147.2024.10610977"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., and Abbeel, P. (2017). Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World. arXiv.","DOI":"10.1109\/IROS.2017.8202133"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Vadlamudi, S., Lakshmi, K.V., Yaramala, A., Uppalapati, C., and Kolluri, P.K. (2024). Self balancing motorcycle using reinforcement learning. Proceedings of the 2024 International Conference on Emerging Systems and Intelligent Computing (ESIC), IEEE.","DOI":"10.1109\/ESIC60604.2024.10481556"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Chebotar, Y., Handa, A., Makoviychuk, V., Macklin, M., Issac, J., Ratliff, N.D., and Fox, D. (2018). Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience. arXiv.","DOI":"10.1109\/ICRA.2019.8793789"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Yu, W., Tan, J., Liu, C.K., and Turk, G. (2017). Preparing for the Unknown: Learning a Universal Policy with Online System Identification. arXiv.","DOI":"10.15607\/RSS.2017.XIII.048"},{"key":"ref_43","unstructured":"Morariu, M. (2025, January 16). LBFGS++: A C++ Library for Limited-Memory BFGS Optimization. Available online: https:\/\/lbfgspp.statr.me\/doc\/index.html."},{"key":"ref_44","unstructured":"ROBOTIS Co., Ltd. (2025, January 15). THORMANG3 Robot: Introduction and Documentation. Available online: https:\/\/emanual.robotis.com\/docs\/en\/platform\/thormang3\/introduction\/."},{"key":"ref_45","unstructured":"Gogoro Inc. (2025, January 15). Gogoro VIVA Smart Scooter: Overview and Specifications. Available online: https:\/\/www.gogoro.com\/tw\/smartscooter\/specs\/."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/19\/6324\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T04:19:54Z","timestamp":1791519594000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/19\/6324"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,7]]},"references-count":45,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2026,10]]}},"alternative-id":["s26196324"],"URL":"https:\/\/doi.org\/10.3390\/s26196324","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,7]]}}}