{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T05:51:45Z","timestamp":1791525105351,"version":"4.3.4"},"reference-count":24,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T00:00:00Z","timestamp":1791504000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:p>Collaborative robots operating in modern manufacturing environments require trajectory planning strategies that simultaneously support efficiency, adaptability, and operational safety. This study proposes a Multi-Level Deep Deterministic Policy Gradient (ML-DDPG) framework for safety-constrained trajectory optimization. The framework combines a strategic waypoint level, a continuous DDPG motion-control policy, and a reactive feasibility projection that corrects unsafe actions before execution. The trajectory-planning problem is formulated as a Markov Decision Process with reward terms for goal progress, obstacle clearance, smoothness, energy, execution time, and safety. Descriptive experiments in a custom PyTorch-based three-dimensional environment compare ML-DDPG with conventional planners and the DDPG, TD3, and SAC baselines. In the retained single training run, ML-DDPG produced observed values of 98.7% success, 1.08\u202fmm trajectory error, a 0.98 smoothness index, and no recorded collision. These values are single-run observations and are not means over independent seeds; variability and statistical significance cannot be estimated from the available record. The exported traces show faster reward growth within the 200-episode budget, and the single-run ablation record associates the largest observed gains with the combined multi-level, reward, and projection components. One UR5e demonstration yielded 0.018\u202fm mean tracking error, 0.042\u202fm maximum tracking error, and no collision in that trial. A separate ABB welding sequence is included only as qualitative feasibility evidence because synchronized trajectory, force, and weld-quality logs were unavailable. The findings motivate further evaluation, while multi-seed training, repeated hardware trials, complete reporting of evaluation procedures, and cross-platform validation remain necessary before broader generalization.<\/jats:p>","DOI":"10.3389\/frai.2026.1914042","type":"journal-article","created":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T05:30:24Z","timestamp":1791523824000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Multi-Level Deep Deterministic Policy Gradient reinforcement learning for safety-constrained trajectory optimization of collaborative industrial robots"],"prefix":"10.3389","volume":"9","author":[{"given":"Akhanseri","family":"Ikramov","sequence":"first","affiliation":[{"name":"Laboratory of Mechanics of Robots and Manipulators, Joldasbekov Institute of Mechanics and Engineering","place":["Almaty, Kazakhstan"]},{"name":"Al-Farabi Kazakh National University","place":["Almaty, Kazakhstan"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sayat","family":"Ibrayev","sequence":"additional","affiliation":[{"name":"Laboratory of Mechanics of Robots and Manipulators, Joldasbekov Institute of Mechanics and Engineering","place":["Almaty, Kazakhstan"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,10,9]]},"reference":[{"key":"ref1","first-page":"22","author":"Achiam","year":"2017"},{"key":"ref2","doi-asserted-by":"publisher","first-page":"341","DOI":"10.1023\/a:1025696116075","article-title":"Recent advances in hierarchical reinforcement learning","volume":"13","author":"Barto","year":"2003","journal-title":"Discrete Event Dyn. Syst."},{"key":"ref3","author":"Dalal","year":"2018"},{"key":"ref4","doi-asserted-by":"publisher","first-page":"103012","DOI":"10.1016\/j.rcim.2025.103012","article-title":"Towards safe motion planning for industrial human-robot interaction: a co-evolution approach based on human digital twin and mixed reality","volume":"95","author":"Feng","year":"2025","journal-title":"Robot. Comput. Integr. Manuf."},{"key":"ref5","first-page":"1587","author":"Fujimoto","year":"2018"},{"key":"ref6","doi-asserted-by":"publisher","first-page":"16334","DOI":"10.1109\/tase.2025.3576586","article-title":"Reinforcement learning-based admittance control for physical human-robot interaction with output constraints","volume":"22","author":"Gao","year":"2025","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"ref7","first-page":"1861","author":"Haarnoja","year":"2018"},{"key":"ref8","doi-asserted-by":"publisher","first-page":"100","DOI":"10.1109\/TSSC.1968.300136","article-title":"A formal basis for the heuristic determination of minimum cost paths","volume":"4","author":"Hart","year":"1968","journal-title":"IEEE Trans. Syst. Sci. Cybern."},{"key":"ref9","doi-asserted-by":"publisher","first-page":"846","DOI":"10.1177\/0278364911406761","article-title":"Sampling-based algorithms for optimal motion planning","volume":"30","author":"Karaman","year":"2011","journal-title":"Int. J. Robot. Res."},{"key":"ref10","doi-asserted-by":"publisher","first-page":"102632","DOI":"10.1016\/j.rcim.2023.102632","article-title":"Digital twin for autonomous collaborative robot by using synthetic data and reinforcement learning","volume":"85","author":"Kim","year":"2024","journal-title":"Robot. Comput. Integr. Manuf."},{"key":"ref11","author":"Lillicrap","year":"2016"},{"key":"ref12","doi-asserted-by":"publisher","first-page":"e70195","DOI":"10.1111\/nyas.70195","article-title":"Adaptive obstacle avoidance for robotic arms using hierarchical reinforcement learning and path optimization","volume":"1556","author":"Ma","year":"2026","journal-title":"Ann. N. Y. Acad. Sci."},{"key":"ref13","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref14","doi-asserted-by":"publisher","first-page":"891","DOI":"10.32604\/cmes.2025.068615","article-title":"Three-dimensional trajectory planning for robotic manipulators using model predictive control and point cloud optimization","volume":"145","author":"Momynkulov","year":"2025","journal-title":"Comput. Model. Eng. Sci."},{"key":"ref15","author":"Navazani","year":"2026"},{"key":"ref16","first-page":"3803","author":"Peng","year":"2018"},{"key":"ref17","author":"Schulman","year":"2017"},{"key":"ref18","volume-title":"Reinforcement Learning: An Introduction","author":"Sutton","year":"2018"},{"key":"ref19","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1016\/s0004-3702(99)00052-1","article-title":"Between MDPs and semi-MDPs: a framework for temporal abstraction in reinforcement learning","volume":"112","author":"Sutton","year":"1999","journal-title":"Artif. Intell."},{"key":"ref20","doi-asserted-by":"publisher","first-page":"153","DOI":"10.1146\/annurev-control-030323-022510","article-title":"Deep reinforcement learning for robotics: a survey of real-world successes","volume":"8","author":"Tang","year":"2025","journal-title":"Annu. Rev. Control Robot. Auton. Syst."},{"key":"ref21","first-page":"23","author":"Tobin","year":"2017"},{"key":"ref22","doi-asserted-by":"publisher","first-page":"8850","DOI":"10.1109\/LRA.2025.3589162","article-title":"Towards autonomous reinforcement learning for real-world robotic manipulation with large language models","volume":"10","author":"Turcato","year":"2025","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref23","doi-asserted-by":"publisher","first-page":"102227","DOI":"10.1016\/j.rcim.2021.102227","article-title":"A reinforcement learning method for human-robot collaboration in assembly tasks","volume":"73","author":"Zhang","year":"2022","journal-title":"Robot. Comput. Integr. Manuf."},{"key":"ref24","doi-asserted-by":"publisher","first-page":"8286","DOI":"10.1038\/s41598-025-93175-2","article-title":"Deep reinforcement learning trajectory planning for robotic manipulator based on simulation-efficient training","volume":"15","author":"Zhao","year":"2025","journal-title":"Sci. Rep."}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2026.1914042\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T05:30:25Z","timestamp":1791523825000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2026.1914042\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,9]]},"references-count":24,"alternative-id":["10.3389\/frai.2026.1914042"],"URL":"https:\/\/doi.org\/10.3389\/frai.2026.1914042","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,9]]},"article-number":"1914042"}}