{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T20:19:13Z","timestamp":1777407553283,"version":"3.51.4"},"reference-count":54,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,1,23]],"date-time":"2023-01-23T00:00:00Z","timestamp":1674432000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neuroinform."],"abstract":"<jats:p>Aiming at the poor robustness and adaptability of traditional control methods for different situations, the deep deterministic policy gradient (DDPG) algorithm is improved by designing a hybrid function that includes different rewards superimposed on each other. In addition, the experience replay mechanism of DDPG is also improved by combining priority sampling and uniform sampling to accelerate the DDPG\u2019s convergence. Finally, it is verified in the simulation environment that the improved DDPG algorithm can achieve accurate control of the robot arm motion. The experimental results show that the improved DDPG algorithm can converge in a shorter time, and the average success rate in the robotic arm end-reaching task is as high as 91.27%. Compared with the original DDPG algorithm, it has more robust environmental adaptability.<\/jats:p>","DOI":"10.3389\/fninf.2023.1096053","type":"journal-article","created":{"date-parts":[[2023,1,23]],"date-time":"2023-01-23T04:56:56Z","timestamp":1674449816000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":24,"title":["An enhanced deep deterministic policy gradient algorithm for intelligent control of robotic arms"],"prefix":"10.3389","volume":"17","author":[{"given":"Ruyi","family":"Dong","sequence":"first","affiliation":[]},{"given":"Junjie","family":"Du","sequence":"additional","affiliation":[]},{"given":"Yanan","family":"Liu","sequence":"additional","affiliation":[]},{"given":"Ali Asghar","family":"Heidari","sequence":"additional","affiliation":[]},{"given":"Huiling","family":"Chen","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2023,1,23]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2021.107529","article-title":"Boosted kernel search: Framework, analysis and case studies on the economic emission dispatch problem.","volume":"233","author":"Dong","year":"2021","journal-title":"Knowl. Based Syst."},{"key":"B2","article-title":"Guided cost learning: Deep inverse optimal control via policy optimization","author":"Finn","year":"2016","journal-title":"Proceedings of the 33rd international conference on machine learning"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2020.106736","article-title":"Regularly updated deterministic policy gradient algorithm.","volume":"214","author":"Han","year":"2021","journal-title":"Knowl. Based Syst."},{"key":"B4","doi-asserted-by":"publisher","first-page":"1057","DOI":"10.1051\/jnwpu\/20213951057","article-title":"Robotic arm reinforcement learning control method based on autonomous visual perception.","volume":"39","author":"Hu","year":"2021","journal-title":"J. Northwest. Polytechnical Univ."},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.3390\/app9020348","article-title":"Pick and place operations in logistics using a mobile manipulator controlled with deep reinforcement learning.","volume":"9","author":"Iriondo","year":"2019","journal-title":"Appl. Sci."},{"key":"B6","doi-asserted-by":"publisher","first-page":"44903","DOI":"10.1109\/ACCESS.2019.2908653","article-title":"Hybrid multiverse optimization algorithm with gravitational search algorithm for multithreshold color image segmentation.","volume":"7","author":"Jia","year":"2019","journal-title":"IEEE Access"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.1061\/(ASCE)AS.1943-5525.0001335","article-title":"Coordinated control based on reinforcement learning for dual-arm continuum manipulators in space capture missions.","volume":"34","author":"Jiang","year":"2021","journal-title":"J. Aerosp. Eng."},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.1016\/j.compchemeng.2021.107527","article-title":"Twin actor twin delayed deep deterministic policy gradient (TATD3) learning for batch process control.","volume":"155","author":"Joshi","year":"2021","journal-title":"Comput. Chem. Eng."},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2022.105498","article-title":"CADxReport: Chest x-ray report generation using co-attention mechanism and reinforcement learning.","volume":"145","author":"Kaur","year":"2022","journal-title":"Comput. Biol. Med."},{"key":"B10","doi-asserted-by":"crossref","DOI":"10.1016\/j.aei.2022.101710","article-title":"Digital twin-driven deep reinforcement learning for adaptive task allocation in robotic construction.","volume":"53","author":"Lee","year":"2022","journal-title":"Adv. Eng. Inform."},{"key":"B11","doi-asserted-by":"crossref","first-page":"2450","DOI":"10.1109\/TSMC.2021.3050183","article-title":"On time-synchronized stability and control.","volume":"52","author":"Li","year":"2021","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"B12","doi-asserted-by":"crossref","DOI":"10.1016\/j.apenergy.2020.116386","article-title":"Efficient experience replay based deep deterministic policy gradient for AGC dispatch in integrated energy system.","volume":"285","author":"Li","year":"2021","journal-title":"Appl. Energy"},{"key":"B13","doi-asserted-by":"publisher","first-page":"10731","DOI":"10.3934\/mbe.2022502","article-title":"Efficient algorithms for scheduling equal-length jobs with processing set restrictions on uniform parallel batch machines.","volume":"19","author":"Li","year":"2022","journal-title":"Math. Bios. Eng."},{"key":"B14","doi-asserted-by":"publisher","first-page":"9697","DOI":"10.3934\/mbe.2022450","article-title":"Scheduling uniform machines with restricted assignment.","volume":"19","author":"Li","year":"2022","journal-title":"Math. Biosci. Eng."},{"key":"B15","article-title":"Robust multi-agent reinforcement learning via minimax deep deterministic policy gradient","author":"Li","year":"2019","journal-title":"Proceedings of the AAAI conference on artificial intelligence"},{"key":"B16","doi-asserted-by":"crossref","DOI":"10.1016\/j.cie.2021.107621","article-title":"Deep deterministic policy gradient algorithm for crowd-evacuation path planning.","volume":"161","author":"Li","year":"2021","journal-title":"Comput. Ind. Eng."},{"key":"B17","doi-asserted-by":"crossref","first-page":"11258","DOI":"10.1109\/ACCESS.2019.2891673","article-title":"Modified grasshopper algorithm-based multilevel thresholding for color image segmentation.","volume":"7","author":"Liang","year":"2019","journal-title":"IEEE Access"},{"key":"B18","doi-asserted-by":"crossref","first-page":"4180","DOI":"10.1109\/TPWRS.2020.2999536","article-title":"Agent-based modeling in electricity market using deep deterministic policy gradient algorithm.","volume":"35","author":"Liang","year":"2020","journal-title":"IEEE Trans. Power Syst."},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.3390\/mi13030458","article-title":"Target tracking strategy using deep deterministic policy gradient.","volume":"95","author":"Liang","year":"2020","journal-title":"Appl. Soft Comput."},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1509.02971","article-title":"Continuous control with deep reinforcement learning","author":"Lillicrap","year":"2015","journal-title":"arXiv"},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.21595\/jve.2016.16714","article-title":"Tip position control method of single flexible manipulators based on LQR with the Mamdani model.","volume":"18","author":"Lu","year":"2016","journal-title":"J. Vibroeng."},{"key":"B22","doi-asserted-by":"publisher","first-page":"881","DOI":"10.1002\/acs.2313","article-title":"Survey of industrial optimized adaptive control.","volume":"26","author":"Mart\u00edn-S\u00e1nchez","year":"2012","journal-title":"Int. J. Adapt. Control Signal Process."},{"key":"B23","doi-asserted-by":"publisher","first-page":"3814","DOI":"10.1109\/TNNLS.2021.3054611","article-title":"Motion planning and adaptive neural tracking control of an uncertain two-link rigid-flexible manipulator with vibration amplitude constraint.","volume":"33","author":"Meng","year":"2021","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.3390\/app10165574","article-title":"Deep reinforcement learning with interactive feedback in a human\u2013robot environment.","volume":"10","author":"Moreira","year":"2020","journal-title":"Appl. Sci."},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.1109\/FSKD.2016.7603175","article-title":"Chaotic glowworm swarm optimization algorithm based on Gauss mutation","author":"Pan","year":"2016","journal-title":"Proceedings of the 2016 12th international conference on natural computation, fuzzy systems and knowledge discovery (ICNC-FSKD)"},{"key":"B26","doi-asserted-by":"publisher","first-page":"213","DOI":"10.1080\/10803548.2017.1350392","article-title":"A survey on industrial applications of fuzzy control.","volume":"62","author":"Precup","year":"2011","journal-title":"Comput. Ind."},{"key":"B27","doi-asserted-by":"publisher","first-page":"8577","DOI":"10.1109\/JIOT.2019.2921159","article-title":"Deep deterministic policy gradient (DDPG)-based energy harvesting wireless communications.","volume":"6","author":"Qiu","year":"2019","journal-title":"IEEE Internet Things J."},{"key":"B28","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2022.105860","article-title":"Subcutaneous insulin administration by deep reinforcement learning for blood glucose level control of type-2 diabetic patients.","volume":"148","author":"Raheb","year":"2022","journal-title":"Comput. Biol. Med."},{"key":"B29","first-page":"714","article-title":"Design of fixed time observer and adaptive sliding film control method for robotic arm.","volume":"39","author":"Ren","year":"2020","journal-title":"Mech. Sci. Technol."},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.3390\/app11177917","article-title":"Vision-based robotic arm control algorithm using deep reinforcement learning for autonomous objects grasping.","volume":"11","author":"Sekkat","year":"2021","journal-title":"Appl. Sci."},{"key":"B31","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1016\/j.neucom.2020.03.063","article-title":"Adaptive neuro-fuzzy PID controller based on twin delayed deep deterministic policy gradient algorithm.","volume":"402","author":"Shi","year":"2020","journal-title":"Neurocomputing"},{"key":"B32","doi-asserted-by":"publisher","first-page":"190","DOI":"10.1016\/j.neucom.2019.11.022","article-title":"Optimizing zinc electrowinning processes with current switching via Deep Deterministic Policy Gradient learning.","volume":"380","author":"Shi","year":"2020","journal-title":"Neurocomputing"},{"key":"B33","doi-asserted-by":"publisher","first-page":"467","DOI":"10.1007\/s11071-013-0983-8","article-title":"A particle swarm optimization approach for fuzzy sliding mode control for tracking the robot manipulator.","volume":"74","author":"Soltanpour","year":"2013","journal-title":"Nonlinear Dyn."},{"key":"B34","doi-asserted-by":"publisher","first-page":"1782","DOI":"10.1109\/9.173151","article-title":"On the robust control of robot manipulators.","volume":"37","author":"Spong","year":"1992","journal-title":"IEEE Trans. Automat. Control"},{"key":"B35","doi-asserted-by":"publisher","first-page":"4209","DOI":"10.1016\/j.egyr.2022.03.078","article-title":"Optimization planning method of distributed generation based on steady-state security region of distribution network.","volume":"8","author":"Sun","year":"2022","journal-title":"Energy Rep."},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.3390\/sym14040825","article-title":"Analysis of position, pose and force decoupling characteristics of a 4-UPS\/1-RPS parallel grinding robot.","volume":"14","author":"Wang","year":"","journal-title":"Symmetry"},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.3390\/app12147219","article-title":"An algorithm for painting large objects based on a nine-axis UR5 robotic manipulator.","volume":"12","author":"Wang","year":"","journal-title":"Appl. Sci."},{"key":"B38","doi-asserted-by":"publisher","first-page":"2199","DOI":"10.1109\/TPEL.2022.3146644","article-title":"Model predictive current control of nine-phase open-end winding PMSMs with an online virtual vector synthesis strategy.","volume":"70","author":"Wang","year":"","journal-title":"IEEE Trans. Ind. Electron."},{"key":"B39","doi-asserted-by":"crossref","first-page":"8895","DOI":"10.1109\/TPEL.2022.3146644","article-title":"Low-complexity model-predictive control for a nine-phase open-end winding PMSM with dead-time compensation.","volume":"37","author":"Wang","year":"","journal-title":"IEEE Trans. Power Electron."},{"key":"B40","doi-asserted-by":"publisher","first-page":"1726","DOI":"10.1631\/FITEE.1900533","article-title":"Deep reinforcement learning: A survey.","volume":"21","author":"Wang","year":"2020","journal-title":"Front. Inform. Technol. Electron. Eng."},{"key":"B41","doi-asserted-by":"publisher","first-page":"2588","DOI":"10.1109\/TIE.2021.3070514","article-title":"Deep deterministic policy gradient-drl enabled multiphysics-constrained fast charging of lithium-ion battery.","volume":"69","author":"Wei","year":"2021","journal-title":"IEEE Trans. Ind. Electron."},{"key":"B42","doi-asserted-by":"publisher","first-page":"12786","DOI":"10.1109\/TVT.2020.3025627","article-title":"Battery-involved energy management for hybrid electric bus based on expert-assistance deep deterministic policy gradient algorithm.","volume":"69","author":"Wu","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"B43","doi-asserted-by":"publisher","first-page":"1584","DOI":"10.1109\/TNNLS.2020.3042943","article-title":"Semicentralized deep deterministic policy gradient in cooperative StarCraft games.","volume":"33","author":"Xie","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"B44","doi-asserted-by":"publisher","DOI":"10.1109\/JESTPE.2022.3185777","article-title":"Fast terminal sliding mode current control with adaptive extended state disturbance observer for PMSM system","author":"Xu","year":"2022","journal-title":"Proceedings of the IEEE journal of emerging and selected topics in power electronics"},{"key":"B45","doi-asserted-by":"crossref","first-page":"2816","DOI":"10.1109\/TASE.2021.3100709","article-title":"Hybrid car-following strategy based on deep deterministic policy gradient and cooperative adaptive cruise control","volume":"19","author":"Yan","year":"2021","journal-title":"IEEE Trans. Autom. Sci. Eng"},{"key":"B46","doi-asserted-by":"publisher","first-page":"135","DOI":"10.1109\/JAS.2021.1004255","article-title":"Collaborative pushing and grasping of tightly stacked objects via deep reinforcement learning.","volume":"9","author":"Yang","year":"2021","journal-title":"IEEE CAA J. Autom. Sinica"},{"key":"B47","doi-asserted-by":"publisher","first-page":"99816","DOI":"10.1109\/ACCESS.2021.3093340","article-title":"Ball motion control in the table tennis robot system using time-series deep reinforcement learning.","volume":"9","author":"Yang","year":"2021","journal-title":"IEEE Access"},{"key":"B48","doi-asserted-by":"publisher","DOI":"10.1016\/j.compchemeng.2020.107133","article-title":"Reinforcement learning based optimal control of batch processes using Monte-Carlo deep deterministic policy gradient with phase segmentation.","volume":"144","author":"Yoo","year":"2021","journal-title":"Comput. Chem. Eng."},{"key":"B49","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2021.104718","article-title":"Erythropoiesis stimulating agent recommendation model using recurrent neural networks for patient with kidney failure with replacement therapy.","volume":"137","author":"Yun","year":"2021","journal-title":"Comput. Biol. Med."},{"key":"B50","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1511.03791","article-title":"Towards vision-based deep reinforcement learning for robotic motion control.","author":"Zhang","year":"2015","journal-title":"ArXiv"},{"key":"B51","first-page":"1636","article-title":"A novel droop control method to achieve maximum power output of photovoltaic for parallel inverter system","volume":"8","author":"Zhang","year":"2021","journal-title":"CSEE J. Power Energy Syst"},{"key":"B52","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2019.112976","article-title":"Gaussian mutational chaotic fruit fly-built optimization and feature selection.","volume":"141","author":"Zhang","year":"2020","journal-title":"Exp. Syst. Appl."},{"key":"B53","doi-asserted-by":"publisher","first-page":"604","DOI":"10.1109\/TCYB.2019.2939174","article-title":"Asynchronous episodic deep deterministic policy gradient: Toward continuous control in computationally complex environments.","volume":"51","author":"Zhang","year":"2019","journal-title":"IEEE Trans. Cybern."},{"key":"B54","doi-asserted-by":"publisher","first-page":"4535","DOI":"10.1109\/LRA.2022.3151164","article-title":"Liquid-metal magnetic soft robot with reprogrammable magnetization and stiffness.","volume":"7","author":"Zhao","year":"2022","journal-title":"IEEE Robot. Autom. Lett."}],"container-title":["Frontiers in Neuroinformatics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fninf.2023.1096053\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,23]],"date-time":"2023-01-23T04:57:08Z","timestamp":1674449828000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fninf.2023.1096053\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,23]]},"references-count":54,"alternative-id":["10.3389\/fninf.2023.1096053"],"URL":"https:\/\/doi.org\/10.3389\/fninf.2023.1096053","relation":{},"ISSN":["1662-5196"],"issn-type":[{"value":"1662-5196","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,23]]},"article-number":"1096053"}}