{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,5]],"date-time":"2026-05-05T18:15:07Z","timestamp":1778004907245,"version":"3.51.4"},"reference-count":15,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2020,6,8]],"date-time":"2020-06-08T00:00:00Z","timestamp":1591574400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>The cable-driven soft arm is mostly made of soft material; it is difficult to control because of the material characteristics, so the traditional robot arm modeling and control methods cannot be directly applied to the soft robot arm. In this paper, we combine the data-driven modeling method with the reinforcement learning control method to realize the position control task of robotic soft arm, the method of control strategy based on deep Q learning. In order to solve slow convergence and unstable effect in the process of simulation and migration when deep reinforcement learning is applied to the actual robot control task, a control strategy learning method is designed, which is based on the experimental data, to establish a simulation environment for control strategy training, and then applied to the real environment. Finally, it is proved by experiment that the method can effectively complete the control of the soft robot arm, which has better robustness than the traditional method.<\/jats:p>","DOI":"10.3390\/info11060310","type":"journal-article","created":{"date-parts":[[2020,6,9]],"date-time":"2020-06-09T04:19:39Z","timestamp":1591676379000},"page":"310","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":32,"title":["Position Control of Cable-Driven Robotic Soft Arm Based on Deep Reinforcement Learning"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5153-6524","authenticated-orcid":false,"given":"Qiuxuan","family":"Wu","sequence":"first","affiliation":[{"name":"School of Automation, Hangzhou Dianzi University, Hangzhou 310018, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yueqin","family":"Gu","sequence":"additional","affiliation":[{"name":"School of Automation, Hangzhou Dianzi University, Hangzhou 310018, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yancheng","family":"Li","sequence":"additional","affiliation":[{"name":"School of Automation, Hangzhou Dianzi University, Hangzhou 310018, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Botao","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Automation, Hangzhou Dianzi University, Hangzhou 310018, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sergey A.","family":"Chepinskiy","sequence":"additional","affiliation":[{"name":"Faculty of Control Systems and Robotics, ITMO University, St. Petersburg 197101, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jian","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Automation, Hangzhou Dianzi University, Hangzhou 310018, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Anton A.","family":"Zhilenkov","sequence":"additional","affiliation":[{"name":"Department of Marine Electronics, State Marine Technical University, St. Petersburg 198262, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6026-6706","authenticated-orcid":false,"given":"Aleksandr Y.","family":"Krasnov","sequence":"additional","affiliation":[{"name":"Faculty of Control Systems and Robotics, ITMO University, St. Petersburg 197101, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5702-3260","authenticated-orcid":false,"given":"Sergei","family":"Chernyi","sequence":"additional","affiliation":[{"name":"Department of Integrated Information Security, Admiral Makarov State University of Maritime and Inland Shipping, St. Petersburg 198035, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,6,8]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"467","DOI":"10.1038\/nature14543","article-title":"Design, fabrication and control of soft robots","volume":"521","author":"Rus","year":"2015","journal-title":"Nature"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2229","DOI":"10.1016\/j.procs.2020.03.275","article-title":"Numerical Methods for Solving the Problem of Calibrating a Projective Stereo Pair Camera, Optimized for Implementation on FPGA","volume":"167","author":"Anton","year":"2020","journal-title":"Procedia Comput. Sci."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Phung, A.S., Malzahn, J., Hoffmann, F., and Bertram, T. (2011, January 7\u201311). Data Based Kinematic Model of a Multi-Flexible-Link Robot Arm for Varying Payloads. Proceedings of the 2011 IEEE International Conference on Robotics and Biomimetics, Karon Beach, Phuket, Thailand.","DOI":"10.1109\/ROBIO.2011.6181460"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Satheeshbabu, S., Uppalapati, N.K., Chowdhary, G., and Krishnan, G. (2019, January 20\u201324). Open Loop Position Control of Soft Continuum Arm Using Deep Reinforcement Learning. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8793653"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"You, X., Zhang, Y., Chen, X., Liu, X., Wang, Z., Jiang, H., and Chen, X. (2017, January 24\u201328). Model-Free Control for Soft Manipulators Based on Reinforcement Learning. Proceedings of the 2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada.","DOI":"10.1109\/IROS.2017.8206123"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Ivanov, A., and Zhilenkov, A. (February, January 29). The Use of IMU MEMS-Sensors for Designing of Motion Capture System for Control of Robotic Objects. Proceedings of the 2018 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (EIConRus), Moscow\/St. Petersburg, Russia.","DOI":"10.1109\/EIConRus.2018.8317231"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"405","DOI":"10.1177\/0278364918770733","article-title":"The limits and potentials of deep learning for robotics","volume":"37","author":"Snderhauf","year":"2008","journal-title":"Int. J. Robot. Res."},{"key":"ref_8","first-page":"124","article-title":"Model-based reinforcement learning for closed-loop dynamic control of soft robotic manipulators","volume":"35","author":"Wang","year":"2017","journal-title":"IEEE Trans. Robot."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Van Hasselt, H., Guez, A., and Silver, D. (2016, January 12\u201317). Deep Reinforcement Learning with Double Q-Learning. Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"ref_10","unstructured":"Schaul, T., Quan, J., Antonoglou, I., and Silver, D. (2015). Prioritized experience replay. arXiv."},{"key":"ref_11","unstructured":"Wang, Z., Schaul, T., Hessel, M., Van Hasselt, H., Lanctot, M., and De Freitas, N. (2015). Dueling network architectures for deep reinforcement learning. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zhilenkov, A., Chernyi, S., Sokolov, S., and Nyrkov, A. (2020). Intelligent autonomous navigation system for UAV in randomly changing environmental conditions. J. Intell. Fuzzy Syst., 1\u20137.","DOI":"10.3233\/JIFS-179741"},{"key":"ref_13","unstructured":"Zhang, F., Leitner, J., Milford, M., Upcroft, B., and Corke, P. (2015). Towards vision-based deep reinforcement learning for robotic motion control. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2317","DOI":"10.1109\/LRA.2020.2970945","article-title":"Learning to Walk a Tripod Mobile Robot Using Nonlinear Soft Vibration Actuators With Entropy Adaptive Reinforcement Learning","volume":"5","author":"Kim","year":"2020","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1016\/j.engappai.2019.08.016","article-title":"A fiber-reinforced human-like soft robotic manipulator based on sEMG force estimation","volume":"86","author":"Feng","year":"2019","journal-title":"Eng. Appl. Artif. Intell."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/11\/6\/310\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:36:51Z","timestamp":1760175411000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/11\/6\/310"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,6,8]]},"references-count":15,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2020,6]]}},"alternative-id":["info11060310"],"URL":"https:\/\/doi.org\/10.3390\/info11060310","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,6,8]]}}}