{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T16:32:30Z","timestamp":1774456350280,"version":"3.50.1"},"reference-count":36,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,7,12]],"date-time":"2023-07-12T00:00:00Z","timestamp":1689120000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001691","name":"Japan Society for the Promotion of Science","doi-asserted-by":"publisher","award":["JP18H05466"],"award-info":[{"award-number":["JP18H05466"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Robot. AI"],"abstract":"<jats:p>A high degree of freedom (DOF) benefits manipulators by presenting various postures when reaching a target. Using a tendon-driven system with an underactuated structure can provide flexibility and weight reduction to such manipulators. The design and control of such a composite system are challenging owing to its complicated architecture and modeling difficulties. In our previous study, we developed a tendon-driven, high-DOF underactuated manipulator inspired from an ostrich neck referred to as the Robostrich arm. This study particularly focused on the control problems and simulation development of such a tendon-driven high-DOF underactuated manipulator. We proposed a curriculum-based reinforcement-learning approach. Inspired by human learning, progressing from simple to complex tasks, the Robostrich arm can obtain manipulation abilities by step-by-step reinforcement learning ranging from simple position control tasks to practical application tasks. In addition, an approach was developed to simulate tendon-driven manipulation with a complicated structure. The results show that the Robostrich arm can continuously reach various targets and simultaneously maintain its tip at the desired orientation while mounted on a mobile platform in the presence of perturbation. These results show that our system can achieve flexible manipulation ability even if vibrations are presented by locomotion.<\/jats:p>","DOI":"10.3389\/frobt.2023.1066518","type":"journal-article","created":{"date-parts":[[2023,7,13]],"date-time":"2023-07-13T00:12:35Z","timestamp":1689207155000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["Curriculum-reinforcement learning on simulation platform of tendon-driven high-degree of freedom underactuated manipulator"],"prefix":"10.3389","volume":"10","author":[{"given":"Keung","family":"Or","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kehua","family":"Wu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kazashi","family":"Nakano","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Masahiro","family":"Ikeda","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mitsuhito","family":"Ando","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yasuo","family":"Kuniyoshi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ryuma","family":"Niiyama","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2023,7,12]]},"reference":[{"key":"B1","first-page":"823","article-title":"Design and development of a redundant modular multipurpose agricultural manipulator","author":"Baur","year":"2012"},{"key":"B2","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1145\/1553374.1553380","article-title":"Curriculum learning","volume-title":"Proceedings of the 26th annual international conference on machine learning","author":"Bengio","year":"2009"},{"key":"B3","first-page":"793","article-title":"Design of tendon-driven robotic fingers: Modeling and control issues","author":"Borghesan","year":"2010"},{"key":"B4","volume-title":"Open gym","author":"Brockman","year":"2016"},{"key":"B5","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1109\/100.388263","article-title":"A hyper-redundant manipulator","volume":"1","author":"Chirikjian","year":"1994","journal-title":"IEEE Robotics Automation Mag."},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1610.03518","article-title":"Transfer from simulation to real world through learning deep inverse dynamics model","author":"Christiano","year":"2016"},{"key":"B7","doi-asserted-by":"publisher","first-page":"1302","DOI":"10.1177\/027836499801701203","article-title":"Interaction control of a redundant mobile manipulator","volume":"17","author":"Chung","year":"1998","journal-title":"Int. J. Robotics Res."},{"key":"B8","doi-asserted-by":"publisher","first-page":"721877","DOI":"10.1371\/journal.pone.0072187","article-title":"Inter-vertebral flexibility of the ostrich neck: Implications for estimating sauropod neck flexibility","volume":"8","author":"Cobley","year":"2013","journal-title":"PLOS ONE"},{"key":"B9","article-title":"Curriculum learning for robot manipulation using deep reinforcement learning","author":"Diego Mendoza","year":"2017"},{"key":"B10","doi-asserted-by":"publisher","first-page":"701","DOI":"10.1002\/jmor.10542","article-title":"Flexibility along the neck of the ostrich (struthio camelus) and consequences for the reconstruction of dinosaurs with extreme neck length","volume":"268","author":"Dzemski","year":"2007","journal-title":"J. Morphol."},{"key":"B11","article-title":"Addressing function approximation error in actor-critic methods","author":"Fujimoto","year":"2018"},{"key":"B12","first-page":"133","article-title":"Trajectory planning based on multi-objective optimization for redundant robots","author":"Gong","year":"2016"},{"key":"B13","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.1801.01290","article-title":"Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor","author":"Haarnoja","year":"2018"},{"key":"B14","first-page":"319","article-title":"Modeling and simulation of 6 dof robotic arm based on gazebo","author":"Huang","year":"2020"},{"key":"B15","article-title":"Follow the object: Curriculum learning for manipulation tasks with imagined goals","author":"Kilinc","year":"2020"},{"key":"B16","doi-asserted-by":"publisher","first-page":"5002","DOI":"10.1109\/LRA.2020.3005131","article-title":"A tendon-driven robot gripper with passively switchable underactuated surface and its physics simulation based parameter optimization","volume":"5","author":"Ko","year":"2020","journal-title":"IEEE Robotics Automation Lett."},{"key":"B17","doi-asserted-by":"publisher","first-page":"328","DOI":"10.20965\/jrm.2022.p0328","article-title":"Robostrich arm: Wire-driven high-dof underactuated manipulator","volume":"34","author":"Misu","year":"2022","journal-title":"J. Robotics Mechatronics"},{"key":"B18","doi-asserted-by":"publisher","first-page":"212","DOI":"10.20965\/jrm.2022.p0212","article-title":"Ostrich-inspired soft robotics: A flexible bipedal manipulator for aggressive physical interaction","volume":"34","author":"Mochiyama","year":"2022","journal-title":"J. Robotics Mechatronics"},{"key":"B19","first-page":"141","article-title":"Model-free reinforcement learning with ensemble for a soft continuum robot arm","author":"Morimoto","year":"2021"},{"key":"B20","article-title":"Deep curriculum learning for polar image classification","author":"Mousavi","year":"2022"},{"key":"B21","doi-asserted-by":"publisher","first-page":"3062","DOI":"10.1109\/LRA.2023.3265301","article-title":"Robostrich manipulator: A novel mechanical design and control based on the anatomy and behavior of an ostrich neck","volume":"8","author":"Nakano","year":"2023","journal-title":"IEEE Robotics Automation Lett."},{"key":"B22","doi-asserted-by":"crossref","DOI":"10.1299\/jsmermd.2022.2A1-S09","article-title":"A study of sagittal motions with a flexible manipulator based on the anatomy of the ostrich neck","volume-title":"The Robotics and mechatronics conference 2022","author":"Nakano","year":"2022"},{"key":"B23","doi-asserted-by":"crossref","first-page":"288","DOI":"10.1007\/978-3-319-78963-7_37","article-title":"Cable-driven parallel robot simulation using gazebo and ros","volume-title":"Romansy 22 \u2013 robot design, dynamics and control","author":"Okoli","year":"2019"},{"key":"B24","doi-asserted-by":"publisher","first-page":"9772","DOI":"10.1016\/j.ifacol.2020.12.2658","article-title":"End-effector stabilization of a 10-dof mobile manipulator using nonlinear model predictive control","volume":"53","author":"Osman","year":"2020","journal-title":"IFAC-PapersOnLine"},{"key":"B25","first-page":"600","article-title":"Simulation of pick and place robotic arm using coppeliasim","author":"Rooban","year":"2022"},{"key":"B26","first-page":"497","article-title":"Continuous control of a soft continuum arm using deep reinforcement learning","author":"Satheeshbabu","year":"2020"},{"key":"B27","doi-asserted-by":"publisher","first-page":"10227","DOI":"10.3390\/app112110227","article-title":"Decentralized multi-agent control of a manipulator in continuous task learning","volume":"11","author":"Shahid","year":"2021","journal-title":"Appl. Sci."},{"key":"B28","doi-asserted-by":"publisher","first-page":"873","DOI":"10.1017\/S0263574708005274","article-title":"Adaptive control of redundant robot manipulators with sub-task objectives","volume":"27","author":"Tatlicioglu","year":"2009","journal-title":"Robotica"},{"key":"B29","doi-asserted-by":"crossref","DOI":"10.1109\/IROS.2017.8202133","article-title":"Domain randomization for transferring deep neural networks from simulation to the real world","author":"Tobin","year":"2017"},{"key":"B30","first-page":"5026","article-title":"Moco: A physics engine for model-based control","author":"Todorov","year":"2012"},{"key":"B31","doi-asserted-by":"publisher","first-page":"1352","DOI":"10.1093\/icb\/41.6.1352","article-title":"Control of the cranio-cervical system during feeding in birds","volume":"41","author":"van der Leeuw","year":"2001","journal-title":"Am. Zoologist"},{"key":"B32","first-page":"376","article-title":"Simulation research of a six degrees of freedom manipulator kinematics based on Matlab toolbox","author":"Xiao","year":"2017"},{"key":"B33","doi-asserted-by":"crossref","first-page":"6095","DOI":"10.18653\/v1\/2020.acl-main.542","article-title":"Curriculum learning for natural language understanding","volume-title":"Proceedings of the 58th annual meeting of the association for computational linguistics","author":"Xu","year":"2020"},{"key":"B34","first-page":"4184","article-title":"Obstacle avoidance of 7-dof redundant manipulators","author":"Xu","year":"2019"},{"key":"B35","article-title":"Sim-to-real transfer of accurate grasping with eye-in-hand observations and continuous control","author":"Yan","year":"2017"},{"key":"B36","first-page":"44","article-title":"Robotic arm motion planning based on curriculum reinforcement learning","author":"Zhou","year":"2021"}],"container-title":["Frontiers in Robotics and AI"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2023.1066518\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,7,13]],"date-time":"2023-07-13T00:12:42Z","timestamp":1689207162000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2023.1066518\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,12]]},"references-count":36,"alternative-id":["10.3389\/frobt.2023.1066518"],"URL":"https:\/\/doi.org\/10.3389\/frobt.2023.1066518","relation":{},"ISSN":["2296-9144"],"issn-type":[{"value":"2296-9144","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,12]]},"article-number":"1066518"}}