{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,7]],"date-time":"2026-03-07T19:12:32Z","timestamp":1772910752710,"version":"3.50.1"},"reference-count":30,"publisher":"World Scientific Pub Co Pte Ltd","issue":"06","funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["92367301"],"award-info":[{"award-number":["92367301"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["92267201"],"award-info":[{"award-number":["92267201"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"State Key Laboratory of Robotics of China","award":["2024-Z19"],"award-info":[{"award-number":["2024-Z19"]}]},{"name":"State Key Laboratory of Robotics of China","award":["2024-Z22"],"award-info":[{"award-number":["2024-Z22"]}]},{"name":"Fundamental Research Project of SIA","award":["2024JC1K13"],"award-info":[{"award-number":["2024JC1K13"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Un. Sys."],"published-print":{"date-parts":[[2025,11]]},"abstract":"<jats:p>In recent years, driven by the rapid advancement of industrial automation and intelligent manufacturing, the demand for manipulators to perform tasks in complex environments has significantly increased. Traditional motion planning methods face challenges of computational complexity and lack of adaptability when handling robots with redundant degrees of freedom. To address these issues, this paper proposes a novel obstacle avoidance motion planning method for manipulators based on deep reinforcement learning (DRL), which is especially important when unmanned systems operate in a GPS-free environment. The primary contributions are twofold: (i) we propose an innovative algorithm, termed PPO-HER, which incorporates hindsight experience replay (HER) into proximal policy optimization (PPO) during training, effectively leveraging failed experiences to enhance policy learning efficiency; (ii) an adaptive action exploration strategy is designed to accelerate training convergence and avoid local optima. Experimental results demonstrate that the PPO-HER algorithm and the adaptive action exploration strategy perform exceptionally well across three environments of varying difficulty, significantly improving network training speed and success rates, shortening planned paths, and enhancing autonomous obstacle avoidance capabilities. These findings provide new ideas for target location and tracking of unmanned systems, and lay the foundation for future learning and data-driven approaches in complex environments.<\/jats:p>","DOI":"10.1142\/s2301385025410079","type":"journal-article","created":{"date-parts":[[2025,4,5]],"date-time":"2025-04-05T01:35:58Z","timestamp":1743816958000},"page":"1545-1558","source":"Crossref","is-referenced-by-count":1,"title":["Obstacle-Avoidance Motion Planning Method of Manipulator Based on Deep Reinforcement Learning"],"prefix":"10.1142","volume":"13","author":[{"given":"Yiyang","family":"Liu","sequence":"first","affiliation":[{"name":"State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, P.\u00a0R.\u00a0China"},{"name":"Key Laboratory of Networked Control Systems, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, P.\u00a0R.\u00a0China"},{"name":"Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang 110169, P.\u00a0R.\u00a0China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-2497-3994","authenticated-orcid":false,"given":"Defan","family":"Wu","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, P.\u00a0R.\u00a0China"},{"name":"Key Laboratory of Networked Control Systems, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, P.\u00a0R.\u00a0China"},{"name":"Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang 110169, P.\u00a0R.\u00a0China"},{"name":"University of Chinese Academy of Sciences, Beijing 100049, P.\u00a0R.\u00a0China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1094-9335","authenticated-orcid":false,"given":"Yukai","family":"Fu","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, P.\u00a0R.\u00a0China"},{"name":"Key Laboratory of Networked Control Systems, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, P.\u00a0R.\u00a0China"},{"name":"Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang 110169, P.\u00a0R.\u00a0China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hongfei","family":"Bai","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, P.\u00a0R.\u00a0China"},{"name":"Key Laboratory of Networked Control Systems, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, P.\u00a0R.\u00a0China"},{"name":"Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang 110169, P.\u00a0R.\u00a0China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chao","family":"Deng","sequence":"additional","affiliation":[{"name":"Institute of Advanced Technology, Nanjing University of Posts and Telecommunications Nanjing 210023, P.\u00a0R.\u00a0China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"219","published-online":{"date-parts":[[2025,5,7]]},"reference":[{"key":"S2301385025410079BIB001","doi-asserted-by":"publisher","DOI":"10.1016\/j.arcontrol.2007.01.002"},{"key":"S2301385025410079BIB002","first-page":"1","author":"Kostyukov V.","year":"2024","journal-title":"Unmanned Syst."},{"key":"S2301385025410079BIB003","doi-asserted-by":"publisher","DOI":"10.1007\/s10846-023-01822-5"},{"issue":"2","key":"S2301385025410079BIB004","first-page":"102","volume":"19","author":"Wang S.","year":"1997","journal-title":"Robot"},{"issue":"1","key":"S2301385025410079BIB005","first-page":"55","volume":"44","author":"Li Y.","year":"2022","journal-title":"Robot"},{"key":"S2301385025410079BIB006","doi-asserted-by":"publisher","DOI":"10.1109\/TMECH.2024.3377002"},{"key":"S2301385025410079BIB007","doi-asserted-by":"publisher","DOI":"10.1109\/TMECH.2022.3160605"},{"key":"S2301385025410079BIB008","doi-asserted-by":"publisher","DOI":"10.1109\/TII.2021.3060019"},{"key":"S2301385025410079BIB009","doi-asserted-by":"publisher","DOI":"10.3390\/computation12060116"},{"key":"S2301385025410079BIB010","doi-asserted-by":"publisher","DOI":"10.1109\/TII.2019.2922694"},{"key":"S2301385025410079BIB012","doi-asserted-by":"publisher","DOI":"10.1109\/70.508439"},{"key":"S2301385025410079BIB013","doi-asserted-by":"publisher","DOI":"10.1142\/S2301385024500225"},{"key":"S2301385025410079BIB014","doi-asserted-by":"publisher","DOI":"10.1177\/0278364911406761"},{"key":"S2301385025410079BIB015","doi-asserted-by":"publisher","DOI":"10.1142\/S2301385023410054"},{"key":"S2301385025410079BIB016","doi-asserted-by":"publisher","DOI":"10.1142\/S230138502441005X"},{"key":"S2301385025410079BIB017","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.1998.712192"},{"key":"S2301385025410079BIB018","doi-asserted-by":"publisher","DOI":"10.1142\/S2301385024420068"},{"key":"S2301385025410079BIB019","doi-asserted-by":"publisher","DOI":"10.1109\/TII.2021.3125447"},{"key":"S2301385025410079BIB020","doi-asserted-by":"publisher","DOI":"10.1109\/TCDS.2018.2868921"},{"key":"S2301385025410079BIB021","doi-asserted-by":"publisher","DOI":"10.1007\/s13246-019-00828-4"},{"key":"S2301385025410079BIB022","doi-asserted-by":"publisher","DOI":"10.3390\/s20123515"},{"key":"S2301385025410079BIB023","doi-asserted-by":"publisher","DOI":"10.1016\/j.ast.2019.105657"},{"key":"S2301385025410079BIB024","doi-asserted-by":"publisher","DOI":"10.1109\/IAEAC47372.2019.8997839"},{"key":"S2301385025410079BIB026","doi-asserted-by":"publisher","DOI":"10.1016\/j.oceaneng.2023.115208"},{"key":"S2301385025410079BIB027","doi-asserted-by":"publisher","DOI":"10.1016\/j.oceaneng.2023.115783"},{"key":"S2301385025410079BIB029","doi-asserted-by":"publisher","DOI":"10.3390\/app14114936"},{"key":"S2301385025410079BIB030","doi-asserted-by":"publisher","DOI":"10.1016\/j.icte.2022.06.004"},{"key":"S2301385025410079BIB031","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRev.36.823"},{"key":"S2301385025410079BIB032","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-17319-6_8"},{"key":"S2301385025410079BIB034","doi-asserted-by":"publisher","DOI":"10.23919\/ECC.2018.8550363"}],"container-title":["Unmanned Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.worldscientific.com\/doi\/pdf\/10.1142\/S2301385025410079","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,29]],"date-time":"2025-10-29T06:57:22Z","timestamp":1761721042000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.worldscientific.com\/doi\/10.1142\/S2301385025410079"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,7]]},"references-count":30,"journal-issue":{"issue":"06","published-print":{"date-parts":[[2025,11]]}},"alternative-id":["10.1142\/S2301385025410079"],"URL":"https:\/\/doi.org\/10.1142\/s2301385025410079","relation":{},"ISSN":["2301-3850","2301-3869"],"issn-type":[{"value":"2301-3850","type":"print"},{"value":"2301-3869","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,7]]}}}