{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,3]],"date-time":"2026-09-03T11:17:42Z","timestamp":1788434262495,"version":"build-2803163510"},"reference-count":34,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2026,8,28]],"date-time":"2026-08-28T00:00:00Z","timestamp":1787875200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Biomimetics"],"abstract":"<jats:p>Robotic piano playing is a challenging benchmark for biomimetic dexterous manipulation, requiring precise timing, coordinated multi-finger motion, and stable key contact. This study proposes a robotic piano-playing framework based on a two-stage reinforcement learning curriculum. Musical Instrument Digital Interface (MIDI) data are converted into target-key and fingering grids to provide future musical goals for policy learning in a parallel MJLab simulation environment. A Soft Actor\u2013Critic (SAC) agent takes a 2106-dimensional observation vector, including joint states, previous actions, musical phase, future key targets, fingering assignments, and piano-key states, and outputs a 21-dimensional continuous action vector for wrist, finger, and global hand-positioning control. Stage 1 weakens physical regularization to facilitate key-pressing acquisition, whereas Stage 2 strengthens power, velocity, acceleration, collision, posture, and finger-speed constraints to improve the regularity of policy outputs and readiness for real-world deployment. Simulation experiments on 30 s right-hand excerpts from F\u00fcr Elise, Canon, and Beethoven\u2019s Symphony No. 5 achieve frame-wise key-state F1 scores above 0.99 on the first two excerpts and approximately 0.945 on Beethoven. Real-world deployment uses open-loop playback of policy-generated high-level trajectories with low-level joint-position feedback and achieves F1 scores of 0.95, 0.91, and 0.83, respectively, while reproducing representative piano techniques such as chords, octaves, mixed black-and-white-key patterns, overlapping finger actions, and rapid sequential movements. These physical results demonstrate the feasibility of the proposed sim-to-real pipeline for complete 30 s executions; they are not intended as a statistical repeatability study. The results further show that biomimetic robotic hands can learn complex piano-playing skills from MIDI-based task objectives without relying on human motion demonstration trajectories.<\/jats:p>","DOI":"10.3390\/biomimetics11090610","type":"journal-article","created":{"date-parts":[[2026,8,31]],"date-time":"2026-08-31T09:21:19Z","timestamp":1788168079000},"page":"610","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Biomimetic Dexterous Hand Control for Robotic Piano Playing Using a Two-Stage Reinforcement Learning Curriculum"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-3467-4626","authenticated-orcid":false,"given":"Lei","family":"Jiang","sequence":"first","affiliation":[{"name":"Center for X-Mechanics, Zhejiang University, Hangzhou 310012, China"},{"name":"State Key Laboratory of Fluid Power and Mechatronic System, Zhejiang University, Hangzhou 310012, China"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"conceptualization","vocabulary":"credit"},{"role":"methodology","vocabulary":"credit"},{"role":"software","vocabulary":"credit"}]},{"given":"Jinyi","family":"Chen","sequence":"additional","affiliation":[{"name":"Center for X-Mechanics, Zhejiang University, Hangzhou 310012, China"},{"name":"State Key Laboratory of Fluid Power and Mechatronic System, Zhejiang University, Hangzhou 310012, China"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"methodology","vocabulary":"credit"},{"role":"software","vocabulary":"credit"},{"role":"validation","vocabulary":"credit"}]},{"given":"Kaixin","family":"Lan","sequence":"additional","affiliation":[{"name":"Center for X-Mechanics, Zhejiang University, Hangzhou 310012, China"},{"name":"State Key Laboratory of Fluid Power and Mechatronic System, Zhejiang University, Hangzhou 310012, China"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"formal-analysis","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"}]},{"given":"Xianwei","family":"Liu","sequence":"additional","affiliation":[{"name":"Center for X-Mechanics, Zhejiang University, Hangzhou 310012, China"},{"name":"State Key Laboratory of Fluid Power and Mechatronic System, Zhejiang University, Hangzhou 310012, China"},{"name":"ZJU-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University, Hangzhou 311200, China"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"data-curation","vocabulary":"credit"},{"role":"resources","vocabulary":"credit"}]},{"given":"Yongbin","family":"Jin","sequence":"additional","affiliation":[{"name":"Center for X-Mechanics, Zhejiang University, Hangzhou 310012, China"},{"name":"State Key Laboratory of Fluid Power and Mechatronic System, Zhejiang University, Hangzhou 310012, China"},{"name":"ZJU-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University, Hangzhou 311200, China"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"resources","vocabulary":"credit"},{"role":"data-curation","vocabulary":"credit"},{"role":"writing-original-draft","vocabulary":"credit"}]},{"given":"Hongtao","family":"Wang","sequence":"additional","affiliation":[{"name":"Center for X-Mechanics, Zhejiang University, Hangzhou 310012, China"},{"name":"State Key Laboratory of Fluid Power and Mechatronic System, Zhejiang University, Hangzhou 310012, China"},{"name":"ZJU-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University, Hangzhou 311200, China"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"software","vocabulary":"credit"},{"role":"data-curation","vocabulary":"credit"},{"role":"validation","vocabulary":"credit"},{"role":"writing-original-draft","vocabulary":"credit"}]}],"member":"1968","published-online":{"date-parts":[[2026,8,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Jurini\u0107, A., Pranji\u0107, M., Huang, A., Burkhart, T.A., Tan, D., and Namburi, P. (2025). The biomechanics of piano playing: A systematic review of kinematic, kinetic, and electromyographic literature. Front. Psychol., 16.","DOI":"10.3389\/fpsyg.2025.1690422"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2849","DOI":"10.1152\/jn.00378.2011","article-title":"Hand kinematics of piano playing","volume":"106","author":"Furuya","year":"2011","journal-title":"J. Neurophysiol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"12224","DOI":"10.1038\/s41598-019-48718-9","article-title":"Neuromuscular and biomechanical functions subserving finger dexterity in musicians","volume":"9","author":"Kimoto","year":"2019","journal-title":"Sci. Rep."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Li, Y., Wang, P., Li, R., Tao, M., Liu, Z., and Qiao, H. (2022). A survey of multifingered robotic manipulation: Biological results, structural evolvements, and learning methods. Front. Neurorobotics, 16.","DOI":"10.3389\/fnbot.2022.843267"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1016\/j.humov.2018.04.012","article-title":"Distinct neural control of intrinsic and extrinsic muscles of the hand during single finger pressing","volume":"59","author":"Dupan","year":"2018","journal-title":"Hum. Mov. Sci."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Sugano, S., and Kato, I. (1987). WABOT-2: Autonomous Robot with Dexterous Finger-Arm Coordination Control in Keyboard Performance. Proceedings of the 1987 IEEE International Conference on Robotics and Automation, IEEE.","DOI":"10.1109\/ROBOT.1987.1088025"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1145\/2818994","article-title":"A survey of robotic musicianship","volume":"59","author":"Bretan","year":"2016","journal-title":"Commun. ACM"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Zhang, A., Malhotra, M., and Matsuoka, Y. (2011). Musical piano performance by the ACT Hand. Proceedings of the 2011 IEEE International Conference on Robotics and Automation (ICRA), IEEE.","DOI":"10.1109\/ICRA.2011.5980342"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1177\/0278364919887447","article-title":"Learning dexterous in-hand manipulation","volume":"39","author":"Andrychowicz","year":"2020","journal-title":"Int. J. Robot. Res."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Rajeswaran, A., Kumar, V., Gupta, A., Vezzani, G., Schulman, J., Todorov, E., and Levine, S. (2018). Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations. Proceedings of Robotics: Science and Systems XIV (RSS), Robotics Science and Systems Foundation.","DOI":"10.15607\/RSS.2018.XIV.049"},{"key":"ref_11","unstructured":"Akkaya, I., Andrychowicz, M., Chociej, M., Litwin, M., McGrew, B., Petron, A., Paino, A., Plappert, M., Powell, G., and Ribas, R. (2019). Solving Rubik\u2019s Cube with a robot hand. arXiv."},{"key":"ref_12","first-page":"2975","article-title":"RoboPianist: Dexterous Piano Playing with Deep Reinforcement Learning","volume":"Volume 229","author":"Zakka","year":"2023","journal-title":"Proceedings of the 7th Conference on Robot Learning (CoRL)"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Xu, H., Luo, Y., Wang, S., Darrell, T., and Calandra, R. (2022). Towards learning to play piano with dexterous hands and touch. Proceedings of the 2022 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS 2022), IEEE.","DOI":"10.1109\/IROS47612.2022.9981221"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Handa, A., Allshire, A., Makoviychuk, V., Petrenko, A., Singh, R., Liu, J., Makoviichuk, D., Van Wyk, K., Zhurkevich, A., and Sundaralingam, B. (2023). DeXtreme: Transfer of agile in-hand manipulation from simulation to reality. 2023 IEEE International Conference on Robotics and Automation (ICRA), IEEE.","DOI":"10.1109\/ICRA48891.2023.10160216"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Shaw, K., Agarwal, A., and Pathak, D. (2023). LEAP Hand: Low-Cost, Efficient, and Anthropomorphic Hand for Robot Learning. Proceedings of Robotics: Science and Systems (RSS), Robotics Science and Systems Foundation.","DOI":"10.15607\/RSS.2023.XIX.089"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2804","DOI":"10.1109\/TPAMI.2023.3339515","article-title":"Bi-DexHands: Towards human-level bimanual dexterous manipulation with reinforcement learning","volume":"46","author":"Chen","year":"2024","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Qin, Y., Wu, Y.-H., Liu, S., Jiang, H., Yang, R., Fu, Y., and Wang, X. (2022). DexMV: Imitation Learning for Dexterous Manipulation from Human Videos. Computer Vision\u2014ECCV 2022, Springer.","DOI":"10.1007\/978-3-031-19842-7_33"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Peng, X.B., Andrychowicz, M., Zaremba, W., and Abbeel, P. (2018). Sim-to-Real Transfer of Robotic Control with Dynamics Randomization. 2018 IEEE International Conference on Robotics and Automation (ICRA), IEEE.","DOI":"10.1109\/ICRA.2018.8460528"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., and Abbeel, P. (2017). Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World. 2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE.","DOI":"10.1109\/IROS.2017.8202133"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"153171","DOI":"10.1109\/ACCESS.2021.3126658","article-title":"Crossing the reality gap: A survey on sim-to-real transferability of robot controllers in reinforcement learning","volume":"9","author":"Salvato","year":"2021","journal-title":"IEEE Access"},{"key":"ref_21","unstructured":"Zeulner, Y.-S., Selvaraj, S., and Calandra, R. (2025). Learning to Play Piano in the Real World. arXiv."},{"key":"ref_22","unstructured":"Xie, A., Qi, H., and Sadigh, D. (2026). HandelBot: Real-World Piano Playing via Fast Adaptation of Dexterous Robot Policies. arXiv."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Wang, R., Xu, P., Shi, H., Schumann, E., and Liu, C.K. (2024). F\u00fcrElise: Capturing and Physically Synthesizing Hand Motion of Piano Performance. Proceedings of the SIGGRAPH Asia 2024 Conference Papers, ACM.","DOI":"10.1145\/3680528.3687703"},{"key":"ref_24","first-page":"1194","article-title":"PianoMime: Learning a Generalist, Dexterous Piano Player from Internet Demonstrations","volume":"Volume 270","author":"Qian","year":"2025","journal-title":"Proceedings of the 8th Conference on Robot Learning (CoRL)"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Bengio, Y., Louradour, J., Collobert, R., and Weston, J. (2009). Curriculum learning. Proceedings of the 26th Annual International Conference on Machine Learning (ICML), ACM.","DOI":"10.1145\/1553374.1553380"},{"key":"ref_26","first-page":"482","article-title":"Reverse curriculum generation for reinforcement learning","volume":"Volume 78","author":"Florensa","year":"2017","journal-title":"Proceedings of the 1st Annual Conference on Robot Learning (CoRL)"},{"key":"ref_27","first-page":"1","article-title":"Curriculum learning for reinforcement learning domains: A framework and survey","volume":"21","author":"Narvekar","year":"2020","journal-title":"J. Mach. Learn. Res."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Todorov, E., Erez, T., and Tassa, Y. (2012). MuJoCo: A Physics Engine for Model-Based Control. Proceedings of the 2012 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS 2012), IEEE.","DOI":"10.1109\/IROS.2012.6386109"},{"key":"ref_29","unstructured":"Zakka, K., Liao, Q., Yi, B., Le Lay, L., Sreenath, K., and Abbeel, P. (2026). mjlab: A lightweight framework for GPU-accelerated robot learning. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Jiang, L., Lan, K., Liu, X., Fu, C., Jin, Y., and Wang, H. (2026). Data-Driven Inverse Design Enables a Dexterous Hand with Human-Comparable Dynamic Performance in Structured Tasks. Biomimetics, 11.","DOI":"10.3390\/biomimetics11060434"},{"key":"ref_31","first-page":"1861","article-title":"Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor","volume":"Volume 80","author":"Haarnoja","year":"2018","journal-title":"Proceedings of the 35th International Conference on Machine Learning (ICML)"},{"key":"ref_32","unstructured":"Haarnoja, T., Zhou, A., Hartikainen, K., Tucker, G., Ha, S., Tan, J., Kumar, V., Zhu, H., Gupta, A., and Abbeel, P. (2018). Soft Actor-Critic algorithms and applications. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Liang, X., and Li, J. (2026). ESO-Enhanced Actor\u2013Critic Reinforcement Learning-Optimised Trajectory Tracking Control for 3-DOF Marine Vessels. Mathematics, 14.","DOI":"10.3390\/math14050867"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"2944","DOI":"10.1049\/cth2.12758","article-title":"USLC: Universal self-learning control via physical performance policy-optimization neural network","volume":"18","author":"Zhang","year":"2024","journal-title":"IET Control Theory Appl."}],"container-title":["Biomimetics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-7673\/11\/9\/610\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,3]],"date-time":"2026-09-03T10:41:41Z","timestamp":1788432101000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-7673\/11\/9\/610"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,28]]},"references-count":34,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2026,9]]}},"alternative-id":["biomimetics11090610"],"URL":"https:\/\/doi.org\/10.3390\/biomimetics11090610","relation":{},"ISSN":["2313-7673"],"issn-type":[{"value":"2313-7673","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,28]]}}}