{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,29]],"date-time":"2026-01-29T22:45:03Z","timestamp":1769726703658,"version":"3.49.0"},"reference-count":25,"publisher":"Emerald","issue":"5","license":[{"start":{"date-parts":[[2023,7,5]],"date-time":"2023-07-05T00:00:00Z","timestamp":1688515200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.emerald.com\/insight\/site-policies"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IR"],"published-print":{"date-parts":[[2023,8,9]]},"abstract":"<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Purpose<\/jats:title>\n<jats:p>The purpose of this paper is using a model-free reinforcement learning (RL) algorithm to optimize manipulability which can overcome difficulties of dilemmas of matrix inversion, complicated formula transformation and expensive calculation time.<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Design\/methodology\/approach<\/jats:title>\n<jats:p>Manipulability optimization is an effective way to solve the singularity problem arising in manipulator control. Some control schemes are proposed to optimize the manipulability during trajectory tracking, but they involve the dilemmas of matrix inversion, complicated formula transformation and expensive calculation time.<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Findings<\/jats:title>\n<jats:p>The redundant manipulator trained by RL can adjust its configuration in real-time to optimize the manipulability in an inverse-free manner while tracking the desired trajectory. Computer simulations and physics experiments demonstrate that compared with the existing methods, the average manipulability is increased by 58.9%, and the calculation time is reduced to 17.9%. Therefore, the proposed method effectively optimizes the manipulability, and the calculation time is significantly shortened.<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Originality\/value<\/jats:title>\n<jats:p>To the best of the authors\u2019 knowledge, this is the first method to optimize manipulability using RL during trajectory tracking. The authors compare their approach to existing singularity avoidance and manipulability maximization techniques, and prove that their method has better optimization effects and less computing time.<\/jats:p>\n<\/jats:sec>","DOI":"10.1108\/ir-01-2023-0002","type":"journal-article","created":{"date-parts":[[2023,7,4]],"date-time":"2023-07-04T07:38:48Z","timestamp":1688456328000},"page":"830-840","source":"Crossref","is-referenced-by-count":3,"title":["Manipulability optimization of redundant manipulators using reinforcement learning"],"prefix":"10.1108","volume":"50","author":[{"given":"Haoqiang","family":"Yang","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xinliang","family":"Li","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Deshan","family":"Meng","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xueqian","family":"Wang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bin","family":"Liang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"140","published-online":{"date-parts":[[2023,7,5]]},"reference":[{"key":"key2023080809532238500_ref001","first-page":"1","article-title":"Dexterous manipulators for nuclear inspection and maintenance \u2013 case study","year":"2010"},{"issue":"6","key":"key2023080809532238500_ref002","doi-asserted-by":"crossref","first-page":"1261","DOI":"10.1109\/TRO.2015.2489500","article-title":"Continuum robots for medical applications: a survey","volume":"31","year":"2015","journal-title":"IEEE Transactions on Robotics"},{"issue":"5","key":"key2023080809532238500_ref003","doi-asserted-by":"crossref","first-page":"2165","DOI":"10.1109\/TMECH.2017.2732827","article-title":"Grasp mode and compliance control of an underactuated origami gripper using adjustable stiffness joints","volume":"22","year":"2017","journal-title":"IEEE\/ASME Transactions on Mechatronics"},{"key":"key2023080809532238500_ref004","first-page":"1587","article-title":"Addressing function approximation error in actor-critic methods","year":"2018"},{"issue":"3","key":"key2023080809532238500_ref005","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1109\/TRA.2002.1019457","article-title":"Manipulability, force, and compliance analysis for planar continuum manipulators","volume":"18","year":"2002","journal-title":"IEEE Transactions on Robotics and Automation"},{"key":"key2023080809532238500_ref006","first-page":"2017","article-title":"Manipulability optimization for trajectory generation","year":"2006"},{"issue":"6","key":"key2023080809532238500_ref007","doi-asserted-by":"crossref","first-page":"4710","DOI":"10.1109\/TIE.2017.2674624","article-title":"Manipulability optimization of redundant manipulators using dynamic neural networks","volume":"64","year":"2017","journal-title":"IEEE Transactions on Industrial Electronics"},{"issue":"8","key":"key2023080809532238500_ref008","first-page":"7209","article-title":"Perturbed manipulability optimization in a distributed network of redundant robots","volume":"68","year":"2020","journal-title":"IEEE Transactions on Industrial Electronics"},{"key":"key2023080809532238500_ref009","first-page":"2197","article-title":"Autonomous steering of concentric tube robots for enhanced force\/velocity manipulability","volume-title":"2019 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE","year":"2019"},{"key":"key2023080809532238500_ref010","first-page":"4920","article-title":"Force\/velocity manipulability analysis for 3d continuum robots","volume-title":"2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE","year":"2018"},{"key":"key2023080809532238500_ref011","first-page":"193","article-title":"Implicit active constraints for concentric tube robots based on analysis of the safe and dexterous workspace","volume-title":"2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE","year":"2017"},{"key":"key2023080809532238500_ref012","article-title":"Continuous control with deep reinforcement learning","year":"2015"},{"issue":"2","key":"key2023080809532238500_ref013","first-page":"930","article-title":"A hybrid active and passive cable-driven segmented redundant manipulator: design, kinematics, and planning","volume":"26","year":"2020","journal-title":"IEEE\/ASME Transactions on Mechatronics"},{"key":"key2023080809532238500_ref014","first-page":"6628","article-title":"A cable-driven redundant spatial manipulator with improved stiffness and load capacity","volume-title":"2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE","year":"2018"},{"key":"key2023080809532238500_ref015","first-page":"8258","article-title":"Fast manipulability maximization using continuous-time trajectory optimization","volume-title":"2019 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE","year":"2019"},{"key":"key2023080809532238500_ref016","first-page":"1","article-title":"Robot tendrils: long, thin continuum robots for inspection in space operations","volume-title":"2017 IEEE Aerospace Conference, IEEE","year":"2017"},{"key":"key2023080809532238500_ref017","first-page":"278","article-title":"Policy invariance under reward transformations: theory and application to reward shaping","volume":"99","year":"1999","journal-title":"Icml"},{"key":"key2023080809532238500_ref018","first-page":"1312","article-title":"Universal value function approximators","year":"2015"},{"key":"key2023080809532238500_ref019","first-page":"1323","article-title":"Manipulability optimization control of a serial redundant robot for robot-assisted minimally invasive surgery","volume-title":"2019 International Conference on Robotics and Automation (ICRA), IEEE","year":"2019"},{"issue":"3","key":"key2023080809532238500_ref020","doi-asserted-by":"crossref","first-page":"935","DOI":"10.1109\/TMECH.2019.2909758","article-title":"Path tracking of a cable-driven snake robot with a two-level motion planning method","volume":"24","year":"2019","journal-title":"IEEE\/ASME Transactions on Mechatronics"},{"key":"key2023080809532238500_ref021","first-page":"5026","article-title":"Mujoco: a physics engine for model-based control","year":"2012"},{"issue":"4","key":"key2023080809532238500_ref022","doi-asserted-by":"crossref","first-page":"1693","DOI":"10.1109\/TMECH.2018.2842141","article-title":"Kinematics, dynamics, and control of a cable-driven hyper-redundant manipulator","volume":"23","year":"2018","journal-title":"IEEE\/ASME Transactions on Mechatronics"},{"issue":"2","key":"key2023080809532238500_ref023","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1177\/027836498500400201","article-title":"Manipulability of robotic mechanisms","volume":"4","year":"1985","journal-title":"The International Journal of Robotics Research"},{"issue":"12","key":"key2023080809532238500_ref024","doi-asserted-by":"crossref","first-page":"5116","DOI":"10.1109\/TNNLS.2020.2963998","article-title":"Rnn for perturbed manipulability optimization of manipulators based on a distributed scheme: a game-theoretic perspective","volume":"31","year":"2020","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"issue":"1","key":"key2023080809532238500_ref025","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1007\/s11071-016-2681-9","article-title":"Qp-based refined manipulability-maximizing scheme for coordinated motion planning and control of physically constrained wheeled mobile redundant manipulators","volume":"85","year":"2016","journal-title":"Nonlinear Dynamics"}],"container-title":["Industrial Robot: the international journal of robotics research and application"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/IR-01-2023-0002\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/IR-01-2023-0002\/full\/html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,24]],"date-time":"2025-07-24T21:38:17Z","timestamp":1753393097000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.emerald.com\/ir\/article\/50\/5\/830-840\/175014"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,5]]},"references-count":25,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2023,7,5]]},"published-print":{"date-parts":[[2023,8,9]]}},"alternative-id":["10.1108\/IR-01-2023-0002"],"URL":"https:\/\/doi.org\/10.1108\/ir-01-2023-0002","relation":{},"ISSN":["0143-991X","0143-991X"],"issn-type":[{"value":"0143-991X","type":"print"},{"value":"0143-991X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,5]]}}}