{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T06:56:32Z","timestamp":1790751392255,"version":"4.1.0"},"reference-count":34,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T00:00:00Z","timestamp":1790726400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurorobot."],"abstract":"<jats:p>\n                    Humanoid motion tracking provides a scalable interface for transferring kinematic human motion to whole-body robot control, but highly dynamic and contact-rich references remain difficult because underactuation and discontinuous contacts can restrict the closed-loop state distributions explored during training. External wrenches can reduce dynamic mismatch during imitation, but retaining such assistance in the final controller can compromise physical realism. This paper proposes residual-wrench-assisted reinforcement learning (RWARL), a two-stage framework that uses base residual wrenches as a temporary optimization scaffold. RWARL first solves a relaxed tracking problem with residual-wrench assistance and then anneals the residual-wrench strength while adapting early-termination bounds and segment sampling to observed training difficulty. To keep the policy adaptable as the training distribution shifts from assisted to unassisted dynamics, RWARL incorporates plasticity-preserving mechanisms during relaxation. Experiments on highly dynamic and contact-rich references show that RWARL generally improves global tracking accuracy, with the largest gains near difficult transitions where standard training tends to produce conservative behavior. Compared with the baseline, the final unassisted RWARL policies reduce global body-position tracking error by 8.6%\u201337.8% on the highly agile and contact-rich motions examined in this work and achieve the largest successful-frame-rate gain of 41.8 percentage points on\n                    <jats:italic>jump twist<\/jats:italic>\n                    . A closed-loop predecessor-state diagnostic further links these gains to more favorable predecessor-state distributions around critical phases. We deploy the final unassisted controllers on Unitree G1 hardware, demonstrating that the learned policies remain executable after residual assistance is removed. Together, these findings suggest that RWARL uses residual wrenches to reshape learning around difficult, low-reachability transitions, allowing the final unassisted policy to inherit better tracking behavior in phases that standard training tends to underexplore.\n                  <\/jats:p>","DOI":"10.3389\/fnbot.2026.1920711","type":"journal-article","created":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T06:05:03Z","timestamp":1790748303000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Assisted exploration of difficult transitions in humanoid motion tracking"],"prefix":"10.3389","volume":"20","author":[{"given":"Chaojie","family":"Fu","sequence":"first","affiliation":[{"name":"The Center for X-Mechanics, Zhejiang University","place":["Hangzhou, Zhejiang, China"]},{"name":"Department of Engineering Mechanics, The Institute of Applied Mechanics, Zhejiang University","place":["Hangzhou, Zhejiang, China"]},{"name":"Zhejiang University (ZJU)-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University","place":["Hangzhou, Zhejiang, China"]},{"name":"The State Key Laboratory of Fluid Power and Mechatronic System, Zhejiang University","place":["Hangzhou, Zhejiang, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chengkai","family":"Su","sequence":"additional","affiliation":[{"name":"The Center for X-Mechanics, Zhejiang University","place":["Hangzhou, Zhejiang, China"]},{"name":"Department of Engineering Mechanics, The Institute of Applied Mechanics, Zhejiang University","place":["Hangzhou, Zhejiang, China"]},{"name":"Zhejiang University (ZJU)-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University","place":["Hangzhou, Zhejiang, China"]},{"name":"The State Key Laboratory of Fluid Power and Mechatronic System, Zhejiang University","place":["Hangzhou, Zhejiang, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lei","family":"Jiang","sequence":"additional","affiliation":[{"name":"The Center for X-Mechanics, Zhejiang University","place":["Hangzhou, Zhejiang, China"]},{"name":"Department of Engineering Mechanics, The Institute of Applied Mechanics, Zhejiang University","place":["Hangzhou, Zhejiang, China"]},{"name":"Zhejiang University (ZJU)-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University","place":["Hangzhou, Zhejiang, China"]},{"name":"The State Key Laboratory of Fluid Power and Mechatronic System, Zhejiang University","place":["Hangzhou, Zhejiang, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chengwei","family":"Zhuo","sequence":"additional","affiliation":[{"name":"The Center for X-Mechanics, Zhejiang University","place":["Hangzhou, Zhejiang, China"]},{"name":"Department of Engineering Mechanics, The Institute of Applied Mechanics, Zhejiang University","place":["Hangzhou, Zhejiang, China"]},{"name":"Zhejiang University (ZJU)-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University","place":["Hangzhou, Zhejiang, China"]},{"name":"The State Key Laboratory of Fluid Power and Mechatronic System, Zhejiang University","place":["Hangzhou, Zhejiang, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kaixin","family":"Lan","sequence":"additional","affiliation":[{"name":"The Center for X-Mechanics, Zhejiang University","place":["Hangzhou, Zhejiang, China"]},{"name":"Department of Engineering Mechanics, The Institute of Applied Mechanics, Zhejiang University","place":["Hangzhou, Zhejiang, China"]},{"name":"Zhejiang University (ZJU)-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University","place":["Hangzhou, Zhejiang, China"]},{"name":"The State Key Laboratory of Fluid Power and Mechatronic System, Zhejiang University","place":["Hangzhou, Zhejiang, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shaowen","family":"Cheng","sequence":"additional","affiliation":[{"name":"Zhejiang University (ZJU)-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University","place":["Hangzhou, Zhejiang, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hongtao","family":"Wang","sequence":"additional","affiliation":[{"name":"The Center for X-Mechanics, Zhejiang University","place":["Hangzhou, Zhejiang, China"]},{"name":"Department of Engineering Mechanics, The Institute of Applied Mechanics, Zhejiang University","place":["Hangzhou, Zhejiang, China"]},{"name":"Zhejiang University (ZJU)-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University","place":["Hangzhou, Zhejiang, China"]},{"name":"The State Key Laboratory of Fluid Power and Mechatronic System, Zhejiang University","place":["Hangzhou, Zhejiang, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,9,30]]},"reference":[{"key":"B1","first-page":"3884","article-title":"\u201cOn warm-starting neural network training,\u201d","volume-title":"Advances in neural information processing systems, Vol. 33","author":"Ash","year":"2020"},{"key":"B2","article-title":"Learning motion skills with adaptive assistive curriculum force in humanoid robots","author":"Cao","year":"2025","journal-title":"arXiv"},{"key":"B3","article-title":"GMT: general motion tracking for humanoid whole-body control","author":"Chen","year":"2025","journal-title":"arXiv"},{"key":"B4","doi-asserted-by":"publisher","first-page":"768","DOI":"10.1038\/s41586-024-07711-7","article-title":"Loss of plasticity in deep continual learning","volume":"632","author":"Dohare","year":"2024","journal-title":"Nature"},{"key":"B5","article-title":"KungfuBot2: learning versatile motion skills for humanoid whole-body control","author":"Han","year":"2025","journal-title":"arXiv"},{"key":"B6","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3386569.3392480","article-title":"Robust motion in-betweening","volume":"39","author":"Harvey","year":"2020","journal-title":"ACM Trans. Graph"},{"key":"B7","doi-asserted-by":"crossref","DOI":"10.15607\/RSS.2025.XXI.066","article-title":"\u201cASAP: aligning simulation and real-world physics for learning agile humanoid whole-body skills,\u201d","volume-title":"Robotics: science and systems XXI. robotics: science and systems foundation","author":"He","year":"2025"},{"key":"B8","doi-asserted-by":"crossref","DOI":"10.15607\/RSS.2025.XXI.064","article-title":"\u201cLearning humanoid standing-up control across diverse postures,\u201d","volume-title":"Robotics: science and systems XXI, RSS2025. robotics: science and systems foundation","author":"Huang","year":"2025"},{"key":"B9","article-title":"\u201cA study of plasticity loss in on-policy deep reinforcement learning,\u201d","volume-title":"Advances in neural information processing systems 38: annual conference on neural information processing systems 2024, NeurIPS 2024","author":"Juliani","year":"2024"},{"key":"B10","article-title":"A Kung Fu athlete bot that can do it all day: highly dynamic, balance-challenging motion dataset and autonomous fall-resilient tracking","author":"Lei","year":"2026","journal-title":"arXiv"},{"key":"B11","article-title":"OmniClone: engineering a robust, all-rounder whole-body humanoid teleoperation system","author":"Li","year":"","journal-title":"arXiv"},{"key":"B12","article-title":"OmniTrack: general motion tracking via physics-consistent reference","author":"Li","year":"","journal-title":"arXiv"},{"key":"B13","article-title":"BeyondMimic: from motion tracking to versatile humanoid control via guided diffusion","author":"Liao","year":"2025","journal-title":"arXiv"},{"key":"B14","doi-asserted-by":"publisher","first-page":"10861","DOI":"10.1109\/ICCV51070.2023.01000","article-title":"\u201cPerpetual humanoid control for real-time simulated avatars,\u201d","author":"Luo","year":"2023","journal-title":"2023 IEEE\/CVF international conference on computer vision (ICCV)"},{"key":"B15","article-title":"\u201cDynamics-regulated kinematic policy for egocentric pose estimation,\u201d","volume-title":"Proceedings of the 35th international conference on neural information processing systems, Nips '21","author":"Luo","year":"2021"},{"key":"B16","article-title":"From universal humanoid control to automatic physically valid character creation","author":"Luo","year":"2022","journal-title":"arXiv"},{"key":"B17","article-title":"SONIC: supersizing motion tracking for natural humanoid whole-body control","author":"Luo","year":"2026","journal-title":"arXiv"},{"key":"B18","first-page":"23190","article-title":"\u201cUnderstanding plasticity in neural networks,\u201d","volume-title":"Proceedings of the 40th international conference on machine learning, Vol. 202 of Proceedings of machine learning research","author":"Lyle","year":"2023"},{"key":"B19","doi-asserted-by":"publisher","first-page":"251","DOI":"10.1111\/cgf.142630","article-title":"Learning and exploring motor skills with spacetime bounds","volume":"40","author":"Ma","year":"2021","journal-title":"Comput. Graph. Forum"},{"key":"B20","doi-asserted-by":"publisher","first-page":"5442","DOI":"10.1109\/ICCV.2019.00554","article-title":"\u201cAMASS: archive of motion capture as surface shapes,\u201d","author":"Mahmood","year":"2019","journal-title":"Proceedings of the IEEE\/CVF international conference on computer vision"},{"key":"B21","doi-asserted-by":"publisher","first-page":"1678567","DOI":"10.3389\/frobt.2025.1678567","article-title":"Imitation learning for legged robot locomotion: a survey","volume":"12","author":"Mirza","year":"2025","journal-title":"Front. Robot. AI"},{"key":"B22","article-title":"SPIDER: scalable physics-informed dexterous retargeting","author":"Pan","year":"2025","journal-title":"arXiv"},{"key":"B23","article-title":"Agility meets stability: versatile humanoid control with heterogeneous data","author":"Pan","year":"2026","journal-title":"arXiv"},{"key":"B24","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3197517.3201311","article-title":"Deepmimic: example-guided deep reinforcement learning of physics-based character skills","volume":"37","author":"Peng","year":"2018","journal-title":"ACM Trans. Graph"},{"key":"B25","article-title":"Zest: zero-shot embodied skill transfer for athletic robot control","author":"Sleiman","year":"2026","journal-title":"arXiv"},{"key":"B26","article-title":"MOSAIC: bridging the sim-to-real gap in generalist humanoid motion tracking and teleoperation with rapid residual adaptation","author":"Sun","year":"2026","journal-title":"arXiv"},{"key":"B27","article-title":"From experts to a generalist: toward general whole-body control for humanoid robots","author":"Wang","year":"2025","journal-title":"arXiv"},{"key":"B28","article-title":"OmniXtreme: breaking the generality barrier in high-dynamic humanoid control","author":"Wang","year":"2026","journal-title":"arXiv"},{"key":"B29","doi-asserted-by":"publisher","first-page":"62406","DOI":"10.52202\/085713-2089","article-title":"Kungfubot: physics-based humanoid whole-body control for learning highly-dynamic skills","volume":"38","author":"Xie","year":"2026","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"B30","article-title":"UniTracker: learning universal whole-body motion tracker for humanoid robots","author":"Yin","year":"2025","journal-title":"arXiv"},{"key":"B31","doi-asserted-by":"publisher","first-page":"5907","DOI":"10.1109\/LRA.2026.3675955","article-title":"EFGCL: learning dynamic motion through spotting-inspired external force guided curriculum learning","volume":"11","author":"Yoneda","year":"2026","journal-title":"IEEE Robot. Autom. Lett"},{"key":"B32","first-page":"21763","article-title":"Residual force control for agile human behavior imitation and extended motion synthesis","volume":"33","author":"Yuan","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"B33","article-title":"Track any motions under any disturbances","author":"Zhang","year":"2025","journal-title":"arXiv"},{"key":"B34","article-title":"ResMimic: from general motion tracking to humanoid whole-body loco-manipulation via residual learning","author":"Zhao","year":"2025","journal-title":"arXiv"}],"container-title":["Frontiers in Neurorobotics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2026.1920711\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T06:05:07Z","timestamp":1790748307000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2026.1920711\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,30]]},"references-count":34,"alternative-id":["10.3389\/fnbot.2026.1920711"],"URL":"https:\/\/doi.org\/10.3389\/fnbot.2026.1920711","relation":{},"ISSN":["1662-5218"],"issn-type":[{"value":"1662-5218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,30]]},"article-number":"1920711"}}