{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,30]],"date-time":"2025-07-30T16:31:33Z","timestamp":1753893093441,"version":"3.41.2"},"reference-count":40,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,11,18]],"date-time":"2024-11-18T00:00:00Z","timestamp":1731888000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100005416","name":"Norges Forskningsr\u00e5d","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100005416","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Robot. AI"],"abstract":"<jats:p>This paper investigates the potential of the intrinsically motivated reinforcement learning (IMRL) approach for robotic drumming. For this purpose, we implemented an IMRL-based algorithm for a drumming robot called <jats:italic>ZRob<\/jats:italic>, an underactuated two-DoF robotic arm with flexible grippers. Two ZRob robots were instructed to play rhythmic patterns derived from MIDI files. The RL algorithm is based on the deep deterministic policy gradient (DDPG) method, but instead of relying solely on extrinsic rewards, the robots are trained using a combination of both extrinsic and intrinsic reward signals. The results of the training experiments show that the utilization of intrinsic reward can lead to meaningful novel rhythmic patterns, while using only extrinsic reward would lead to predictable patterns identical to the MIDI inputs. Additionally, the observed drumming patterns are influenced not only by the learning algorithm but also by the robots\u2019 physical dynamics and the drum\u2019s constraints. This work suggests new insights into the potential of embodied intelligence for musical performance.<\/jats:p>","DOI":"10.3389\/frobt.2024.1450097","type":"journal-article","created":{"date-parts":[[2024,11,18]],"date-time":"2024-11-18T11:14:39Z","timestamp":1731928479000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Embodied intelligence for drumming; a reinforcement learning approach to drumming robots"],"prefix":"10.3389","volume":"11","author":[{"given":"Seyed Mojtaba","family":"Karbasi","sequence":"first","affiliation":[]},{"given":"Alexander Refsum","family":"Jensenius","sequence":"additional","affiliation":[]},{"given":"Rolf Inge","family":"God\u00f8y","sequence":"additional","affiliation":[]},{"given":"Jim","family":"Torresen","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2024,11,18]]},"reference":[{"key":"B1","first-page":"112","article-title":"Intrinsically motivated learning of hierarchical collections of skills","author":"Barto","year":"2004"},{"key":"B2","article-title":"A physically intuitive haptic drumstick","volume-title":"International conference on mathematics and computing","author":"Berdahl","year":"2007"},{"volume-title":"A robotic prosthesis for an amputee drummer","year":"2016","author":"Bretan","key":"B3"},{"key":"B4","first-page":"116","article-title":"DRMMR: an augmented percussion implement","volume-title":"Proceedings of the international conference on new interfaces for musical expression","author":"Brown","year":"2019"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.30535\/mto.24.3.8","article-title":"Motor constraints shaping musical experience","volume":"24","author":"God\u00f8y","year":"2018","journal-title":"Music Theory Online"},{"key":"B6","doi-asserted-by":"publisher","first-page":"207","DOI":"10.1016\/j.robot.2016.08.020","article-title":"A generative physical model approach for enhancing the stroke palette for robotic drummers","volume":"86","author":"Gopinath","year":"2016","journal-title":"Robotics Aut. Syst."},{"key":"B7","doi-asserted-by":"publisher","first-page":"5721","DOI":"10.1038\/s41467-021-25874-z","article-title":"Embodied intelligence via learning and evolution","volume":"12","author":"Gupta","year":"2021","journal-title":"Nat. Commun."},{"key":"B8","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1007\/s10514-011-9237-0","article-title":"Interactive improvisation with a robotic marimba player","volume":"31","author":"Hoffman","year":"2011","journal-title":"Aut. Robots"},{"key":"B9","doi-asserted-by":"publisher","first-page":"eaau3098","DOI":"10.1126\/scirobotics.aau3098","article-title":"An anthropomorphic soft skeleton hand exploiting conditional models for piano playing","volume":"3","author":"Hughes","year":"2018","journal-title":"Sci. Robotics"},{"key":"B10","article-title":"A comparison of solenoid-based strategies for robotic drumming","volume-title":"International conference on mathematics and computing","author":"Kapur","year":"2007"},{"key":"B11","first-page":"54","article-title":"A learning method for stiffness control of a drum robot for rebounding double strokes","author":"Karbasi","year":""},{"key":"B12","article-title":"A generative model for creating musical rhythms with deep reinforcement learning","volume-title":"Proceedings of the 2nd joint conference on AI music creativity (AIMC)","author":"Karbasi","year":""},{"article-title":"A robotic drummer with a flexible joint: the effect of passive impedance on drumming","year":"2022","author":"Karbasi","key":"B13"},{"key":"B14","first-page":"262","article-title":"Exploring emerging drumming patterns in a chaotic dynamical system using ZRob","volume-title":"Proceedings of the international conference on new interfaces for musical expression","author":"Karbasi","year":"2023"},{"key":"B15","first-page":"3892","article-title":"Drum stroke variation using variable stiffness actuators","author":"Kim","year":"2014"},{"key":"B16","doi-asserted-by":"publisher","first-page":"63","DOI":"10.1016\/j.tics.2018.10.006","article-title":"Predictive processes and the peculiar case of music","volume":"23","author":"Koelsch","year":"2019","journal-title":"Trends cognitive Sci."},{"volume-title":"Continuous control with deep reinforcement learning","year":"2015","author":"Lillicrap","key":"B17"},{"key":"B18","first-page":"305","article-title":"Robot learning to play drums with an open-ended internal model","author":"Liu","year":"2018"},{"volume-title":"librosa\/librosa: 0.10.2.post1","year":"2024","author":"McFee","key":"B19"},{"key":"B20","article-title":"Better drumming through calibration: techniques for pre-performance robotic percussion optimization","volume-title":"Proceedings of the International Conference on New Interfaces for Musical Expression","author":"Murphy","year":"2012"},{"key":"B21","first-page":"1","article-title":"Control of single-stroke movement of a drum-playing robot by reinforcement learning using a realistic artificial muscle-driven robot","author":"Okui","year":"2022"},{"key":"B22","doi-asserted-by":"crossref","DOI":"10.31234\/osf.io\/3p8f6","volume-title":"Computational theories of curiosity-driven learning","author":"Oudeyer","year":"2018"},{"key":"B23","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1007\/978-3-642-32375-1_13","article-title":"Intrinsically motivated learning of real-world sensorimotor skills with developmental constraints","volume-title":"Intrinsically motivated learning in natural and artificial systems","author":"Oudeyer","year":"2013"},{"key":"B24","doi-asserted-by":"publisher","first-page":"265","DOI":"10.1109\/tevc.2006.890271","article-title":"Intrinsic motivation systems for autonomous mental development","volume":"11","author":"Oudeyer","year":"2007","journal-title":"IEEE Trans. Evol. Comput."},{"volume-title":"From machine learning to robotics: challenges and opportunities for embodied intelligence","year":"2021","author":"Roy","key":"B25"},{"key":"B26","doi-asserted-by":"publisher","first-page":"896267","DOI":"10.3389\/frobt.2022.896267","article-title":"Towards autonomous robotic biopsy\u2014design, modeling and control of a robot for needle insertion of a commercial full core biopsy instrument","volume":"9","author":"Sajadi","year":"2022","journal-title":"Front. Robotics AI"},{"key":"B27","doi-asserted-by":"publisher","first-page":"173","DOI":"10.1080\/09540090600768658","article-title":"Developmental robotics, optimal artificial curiosity, creativity, music, and the fine arts","volume":"18","author":"Schmidhuber","year":"2006","journal-title":"Connect. Sci."},{"key":"B28","doi-asserted-by":"publisher","first-page":"230","DOI":"10.1109\/tamd.2010.2056368","article-title":"Formal theory of creativity, fun, and intrinsic motivation (1990\u20132010)","volume":"2","author":"Schmidhuber","year":"2010","journal-title":"IEEE Trans. Aut. Ment. Dev."},{"key":"B29","doi-asserted-by":"crossref","DOI":"10.15607\/RSS.2020.XVI.053","volume-title":"Emergent real-world robotic skills via unsupervised off-policy reinforcement learning","author":"Sharma","year":"2020"},{"volume-title":"Dynamics-aware unsupervised discovery of skills","year":"2019","author":"Sharma","key":"B30"},{"volume-title":"Robot modeling and control","year":"2020","author":"Spong","key":"B31"},{"key":"B32","doi-asserted-by":"publisher","first-page":"022036","DOI":"10.1088\/1742-6596\/1168\/2\/022036","article-title":"Design and implementation of a drum robot based on ros","volume":"1168","author":"Su","year":"2019","journal-title":"J. Phys. Conf. Ser."},{"volume-title":"Reinforcement learning: an introduction","year":"2018","author":"Sutton","key":"B33"},{"key":"B34","unstructured":"Human-informed robotic percussion renderings: acquisition, analysis, and rendering of percussion performances using stochastic models and robotics\n          \n          \n            \n              Van Rooyen\n              R. M.\n            \n          \n          Canada\n          University of Victoria\n          \n          2018"},{"key":"B35","doi-asserted-by":"publisher","first-page":"631752","DOI":"10.3389\/frobt.2021.631752","article-title":"Creative ai and musicking robots","volume":"8","author":"Vear","year":"2021","journal-title":"Front. Robotics AI"},{"key":"B36","first-page":"300","article-title":"Data-driven simulation framework for expressive piano playing by anthropomorphic hand with variable passive properties","author":"Wang","year":"2022"},{"key":"B37","doi-asserted-by":"publisher","first-page":"066009","DOI":"10.1088\/1748-3190\/acfa51","article-title":"Coordinating upper limbs for octave playing on the piano via neuro-musculoskeletal modeling","volume":"18","author":"Wang","year":"2023","journal-title":"Bioinspiration and Biomimetics"},{"key":"B38","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-38930-7","volume-title":"Robotic musicianship: embodied artificial creativity and mechatronic musical expression","author":"Weinberg","year":"2020"},{"key":"B39","first-page":"559","article-title":"Developing robot drumming skill with listening-playing loop","author":"Wu","year":"2017"},{"key":"B40","first-page":"10317","article-title":"Drumming arm: an upper-limb prosthetic system to restore grip control for a transradial amputee drummer","author":"Yang","year":"2021"}],"container-title":["Frontiers in Robotics and AI"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2024.1450097\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,18]],"date-time":"2024-11-18T11:14:46Z","timestamp":1731928486000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2024.1450097\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,18]]},"references-count":40,"alternative-id":["10.3389\/frobt.2024.1450097"],"URL":"https:\/\/doi.org\/10.3389\/frobt.2024.1450097","relation":{},"ISSN":["2296-9144"],"issn-type":[{"type":"electronic","value":"2296-9144"}],"subject":[],"published":{"date-parts":[[2024,11,18]]},"article-number":"1450097"}}