{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,15]],"date-time":"2026-05-15T02:52:00Z","timestamp":1778813520976,"version":"3.51.4"},"reference-count":36,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,3,28]],"date-time":"2024-03-28T00:00:00Z","timestamp":1711584000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurorobot."],"abstract":"<jats:sec><jats:title>Introduction<\/jats:title><jats:p>Redundant robots offer greater flexibility compared to non-redundant ones but are susceptible to increased collision risks when the end-effector approaches the robot's own links. Redundant degrees of freedom (DoFs) present an opportunity for collision avoidance; however, selecting an appropriate inverse kinematics (IK) solution remains challenging due to the infinite possible solutions.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>This study proposes a reinforcement learning (RL) enhanced pseudo-inverse approach to address self-collision avoidance in redundant robots. The RL agent is integrated into the redundancy resolution process of a pseudo-inverse method to determine a suitable IK solution for avoiding self-collisions during task execution. Additionally, an improved replay buffer is implemented to enhance the performance of the RL algorithm.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>Simulations and experiments validate the effectiveness of the proposed method in reducing the risk of self-collision in redundant robots.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p>The RL enhanced pseudo-inverse approach presented in this study demonstrates promising results in mitigating self-collision risks in redundant robots, highlighting its potential for enhancing safety and performance in robotic systems.<\/jats:p><\/jats:sec>","DOI":"10.3389\/fnbot.2024.1375309","type":"journal-article","created":{"date-parts":[[2024,3,28]],"date-time":"2024-03-28T04:28:14Z","timestamp":1711600094000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["A reinforcement learning enhanced pseudo-inverse approach to self-collision avoidance of redundant robots"],"prefix":"10.3389","volume":"18","author":[{"given":"Tinghe","family":"Hong","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Weibing","family":"Li","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kai","family":"Huang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2024,3,28]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1016\/j.mechmachtheory.2015.07.013","article-title":"Dynamic singularity avoidance for parallel manipulators using a task-priority based control scheme","volume":"96","author":"Agarwal","year":"2016","journal-title":"Mechan. Mach. Theory"},{"key":"B2","first-page":"981","article-title":"\u201cRedundancy resolution as action bias in policy search for robotic manipulation,\u201d","volume-title":"Conference on Robot Learning","author":"Al-Hafez","year":"2021"},{"key":"B3","article-title":"\u201cHindsight experience replay,\u201d","author":"Andrychowicz","year":"2017","journal-title":"Advance Neural Information Processing System"},{"key":"B4","doi-asserted-by":"publisher","first-page":"7863","DOI":"10.1109\/TNNLS.2021.3088947","article-title":"Complex robotic manipulation via graph-based hindsight goal generation","volume":"33","author":"Bing","year":"","journal-title":"IEEE Trans. Neural Netw. Learn. Syst"},{"key":"B5","doi-asserted-by":"publisher","first-page":"92","DOI":"10.1109\/MRA.2022.3204237","article-title":"Simulation to real: learning energy-efficient slithering gaits for a snake-like robot","volume":"29","author":"Bing","year":"","journal-title":"IEEE Robot. Autom. Magaz"},{"key":"B6","first-page":"1","article-title":"\u201cMeta-reinforcement learning in nonstationary and nonparametric environments,\u201d","author":"Bing","year":"","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"key":"B7","doi-asserted-by":"publisher","first-page":"3476","DOI":"10.1109\/TPAMI.2022.3185549","article-title":"Meta-reinforcement learning in non-stationary and dynamic environments","volume":"45","author":"Bing","year":"","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B8","doi-asserted-by":"publisher","first-page":"2759","DOI":"10.1109\/TIE.2022.3172754","article-title":"Solving robotic manipulation with sparse reward reinforcement learning via graph-based diversity and proximity","volume":"70","author":"Bing","year":"","journal-title":"IEEE Trans. Industr. Electron"},{"key":"B9","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511804441","volume-title":"Convex Optimization","author":"Boyd","year":"2004"},{"key":"B10","doi-asserted-by":"publisher","first-page":"1384","DOI":"10.1109\/TMECH.2022.3224314","article-title":"Efficient grasp detection network with gaussian-based grasp representation for robotic manipulation","volume":"28","author":"Cao","year":"","journal-title":"IEEE\/ASME Trans. Mechatr"},{"key":"B11","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TIM.2022.3179469","article-title":"Neurograsp: Multimodal neural network with euler region regression for neuromorphic vision-based grasp pose estimation","volume":"71","author":"Cao","year":"2022","journal-title":"IEEE Trans. Instrum. Meas"},{"key":"B12","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TAI.2023.3326795","article-title":"Ghostvit: Expediting vision transformers via cheap operations","volume":"2023","author":"Cao","year":"","journal-title":"IEEE Trans. Artif. Intellig"},{"key":"B13","doi-asserted-by":"publisher","first-page":"806","DOI":"10.1109\/3477.735390","article-title":"Window-shaped obstacle avoidance for a redundant manipulator. IEEE Trans. Syst. Man. Cybern","volume":"28","author":"Cheng","year":"1998","journal-title":"B Cybern"},{"key":"B14","doi-asserted-by":"publisher","first-page":"944","DOI":"10.1109\/TMECH.2014.2326304","article-title":"Closed-loop inverse kinematics for redundant robots: Comparative assessment and two enhancements","volume":"20","author":"Colom\u00e9","year":"2015","journal-title":"IEEE\/ASME Trans. Mechatron"},{"key":"B15","doi-asserted-by":"publisher","first-page":"215","DOI":"10.1115\/1.4011045","article-title":"A kinematic notation for lower-pair mechanisms based on matrices","volume":"22","author":"Denavit","year":"1955","journal-title":"J. Appl, Mech"},{"key":"B16","first-page":"77","article-title":"Iw-pso approach to the inverse kinematics problem solution of a 7-dof serial robot manipulator","volume":"36","author":"Dereli","year":"2018","journal-title":"Sigma J. Eng. Natural Sci"},{"key":"B17","doi-asserted-by":"publisher","first-page":"949","DOI":"10.1007\/s10462-019-09683-x","article-title":"A meta-heuristic proposal for inverse kinematics solution of 7-dof serial robotic manipulator: quantum behaved particle swarm algorithm","volume":"53","author":"Dereli","year":"2020","journal-title":"Artif. Intellig. Rev"},{"key":"B18","doi-asserted-by":"publisher","first-page":"28","DOI":"10.1016\/j.rcim.2011.07.004","article-title":"Obstacle avoidance of redundant manipulators using neural networks based reinforcement learning","volume":"2","author":"Duguleana","year":"2012","journal-title":"Robot. Comp. Integrat. Manufact"},{"key":"B19","unstructured":"\u201cAddressing function approximation error in actor-critic methods,\u201d15821591\n            FujimotoS.\n            HoofH.\n            MegerD.\n          \n            DyJ. G.\n            KrauseA.\n          StockholmPMLRProceedings of the 35th International Conference on Machine Learning2018"},{"key":"B20","doi-asserted-by":"publisher","first-page":"153","DOI":"10.1109\/ROBOT.1990.125964","article-title":"Joint trajectory generation for redundant robots in an environment with obstacles","volume":"35","author":"Guo","year":"1990","journal-title":"IEEE Trans. Biomed. Eng"},{"key":"B21","first-page":"1861","article-title":"\u201cSoft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,\u201d","volume-title":"Proceedings of the 35th International Conference on Machine Learning","author":"Haarnoja","year":"2018"},{"key":"B22","doi-asserted-by":"publisher","first-page":"1043","DOI":"10.1109\/LRA.2021.3056060","article-title":"Neo: a novel expeditious optimisation algorithm for reactive motion control of manipulators","volume":"6","author":"Haviland","year":"2021","journal-title":"IEEE Robot. Automat. Lett"},{"key":"B23","first-page":"1026","article-title":"\u201cDelving deep into rectifiers: Surpassing human-level performance on imagenet classification,\u201d","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"He","year":"2015"},{"key":"B24","unstructured":"Pyrep: Bringing v-rep to deep robot learning\n            JamesS.\n            FreeseM.\n            DavisonA.\n          arXiv2019"},{"key":"B25","doi-asserted-by":"publisher","first-page":"553","DOI":"10.1007\/s00366-015-0432-z","article-title":"A neuro-genetic-simulated annealing approach to the inverse kinematics solution of robots: a simulation based study","volume":"32","author":"K\u00f6ker","year":"2016","journal-title":"Eng. Comput"},{"key":"B26","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1109\/TII.2021.3125447","article-title":"A general framework of motion planning for redundant robot manipulator based on deep reinforcement learning","volume":"8","author":"Li","year":"2022","journal-title":"IEEE Trans. Indust. Informat"},{"key":"B27","first-page":"1324","article-title":"\u201cCombining reinforcement learning and differential inverse kinematics for collision-free motion of multilink manipulators,\u201d","volume-title":"Biological and Artificial Computation: From Neuroscience to Technology, International Work-Conference on Artificial and Natural Neural Networks","author":"Martin","year":"1997"},{"key":"B28","doi-asserted-by":"publisher","first-page":"225","DOI":"10.18576\/amis\/100122","article-title":"Solution of inverse kinematics problem using genetic algorithms","volume":"10","author":"Momani","year":"2016","journal-title":"Appl. Mathem. Informat. Sci"},{"key":"B29","doi-asserted-by":"publisher","first-page":"163","DOI":"10.1115\/1.3143764","article-title":"Inverse kinematic solutions with singularity robustness for robot manipulator control","volume":"108","author":"Nakamura","year":"1986","journal-title":"J. Dyn. Syst. Meas. Control"},{"key":"B30","first-page":"1398","article-title":"\u201cKinematic control equations for simple manipulators,\u201d","volume-title":"IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes","author":"Paul","year":"1978"},{"key":"B31","doi-asserted-by":"publisher","first-page":"1602","DOI":"10.1016\/j.proeng.2013.09.242","article-title":"Inverse kinematics using particle swarm optimization, a statistical analysis","volume":"64","author":"Rokbani","year":"2013","journal-title":"Procedia Eng"},{"key":"B32","doi-asserted-by":"publisher","first-page":"338","DOI":"10.5220\/0010621103380349","article-title":"\u201cSingularity avoidance of task-redundant robots in pointing tasks: on nullspace projection and cardan angles as orientation coordinates,\u201d","author":"Schappler","year":"2021","journal-title":"Proceedings of the 18th International Conference on Informatics in Control, Automation and Robotics"},{"key":"B33","unstructured":"SuttonR. S.\n            BartoA. G.\n          Reinforcement Learning: An Introduction2018"},{"key":"B34","doi-asserted-by":"publisher","first-page":"189","DOI":"10.1115\/1.3258708","article-title":"Solving the kinematics of the most general six- and five-degree-of-freedom manipulators by continuation methods","volume":"107","author":"Tsai","year":"1985","journal-title":"J. Mech. Design"},{"key":"B35","first-page":"1359","article-title":"\u201cA computational technique for inverse kinematics,\u201d","author":"Wolovich","year":"1984"},{"key":"B36","doi-asserted-by":"publisher","first-page":"829","DOI":"10.1016\/j.mechmachtheory.2017.12.005","article-title":"A novel closed-form solution for the inverse kinematics of redundant manipulators through workspace analysis","volume":"121","author":"Zaplana","year":"2018","journal-title":"Mech. Mach. Theory"}],"container-title":["Frontiers in Neurorobotics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2024.1375309\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,28]],"date-time":"2024-03-28T04:28:35Z","timestamp":1711600115000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2024.1375309\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,28]]},"references-count":36,"alternative-id":["10.3389\/fnbot.2024.1375309"],"URL":"https:\/\/doi.org\/10.3389\/fnbot.2024.1375309","relation":{},"ISSN":["1662-5218"],"issn-type":[{"value":"1662-5218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,28]]},"article-number":"1375309"}}