{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,3,3]],"date-time":"2025-03-03T05:24:31Z","timestamp":1740979471758,"version":"3.38.0"},"reference-count":50,"publisher":"SAGE Publications","issue":"2","license":[{"start":{"date-parts":[[2022,9,28]],"date-time":"2022-09-28T00:00:00Z","timestamp":1664323200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["11872277"],"award-info":[{"award-number":["11872277"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["11932015"],"award-info":[{"award-number":["11932015"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["12072237"],"award-info":[{"award-number":["12072237"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["91748205"],"award-info":[{"award-number":["91748205"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004543","name":"China Scholarship Council","doi-asserted-by":"publisher","award":["201906260056"],"award-info":[{"award-number":["201906260056"]}],"id":[{"id":"10.13039\/501100004543","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Proceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Engineering"],"published-print":{"date-parts":[[2023,2]]},"abstract":"<jats:p> In this article, we propose a cascade control framework to attenuate the residual vibration of the underactuated manipulator. The control framework is divided into two phases. In the first phase, a path generator trained by the reinforcement learning produces the leading signal for the tracking controller. In the second phase, the leading signal stabilizes the underactuated manipulator, and the adaptive proportional derivative controller is implemented to reduce the vibration. In the process, a novel path planning method is proposed to improve exploration efficiency, and a negative reward is introduced to avoid unsafe strategies and simulation instability. The effectiveness of the proposed control scheme is verified in the simulations of the double pendulum crane and the two-link flexible manipulator. <\/jats:p>","DOI":"10.1177\/09596518221125533","type":"journal-article","created":{"date-parts":[[2022,9,28]],"date-time":"2022-09-28T07:11:59Z","timestamp":1664349119000},"page":"231-243","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":0,"title":["Cascade control of underactuated manipulator based on reinforcement learning framework"],"prefix":"10.1177","volume":"237","author":[{"given":"Naijing","family":"Jiang","sequence":"first","affiliation":[{"name":"School of Aerospace Engineering and Applied Mechanics, Tongji University, Shanghai, China"},{"name":"Shanghai Microport Medbot, Shanghai, China"}]},{"given":"Dingxu","family":"Guo","sequence":"additional","affiliation":[{"name":"School of Aerospace Engineering and Applied Mechanics, Tongji University, Shanghai, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6741-1299","authenticated-orcid":false,"given":"Shu","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Aerospace Engineering and Applied Mechanics, Tongji University, Shanghai, China"}]},{"given":"Dan","family":"Zhang","sequence":"additional","affiliation":[{"name":"Lassonde School of Engineering, York University, Toronto, ON, Canada"}]},{"given":"Jian","family":"Xu","sequence":"additional","affiliation":[{"name":"School of Aerospace Engineering and Applied Mechanics, Tongji University, Shanghai, China"}]}],"member":"179","published-online":{"date-parts":[[2022,9,28]]},"reference":[{"key":"bibr1-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1109\/TIE.2019.2926050"},{"key":"bibr2-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2018.2803167"},{"key":"bibr3-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2017.2743103"},{"key":"bibr4-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1109\/TMECH.2011.2174373"},{"key":"bibr5-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2017.2651941"},{"key":"bibr6-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2019.2931483"},{"key":"bibr7-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1115\/1.2894142"},{"key":"bibr8-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1016\/j.ymssp.2018.01.029"},{"key":"bibr9-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1109\/TFUZZ.2019.2892339"},{"key":"bibr10-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1016\/j.ymssp.2018.09.003"},{"key":"bibr11-09596518221125533","doi-asserted-by":"publisher","DOI":"10.2514\/2.4033"},{"key":"bibr12-09596518221125533","first-page":"489","volume-title":"2016 IEEE international conference on real-time computing and robotics (RCAR)","volume":"3","author":"Fan C"},{"key":"bibr13-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1007\/s11044-019-09695-z"},{"key":"bibr14-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1017\/S0263574710000767"},{"key":"bibr15-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2874184"},{"key":"bibr16-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1023\/A:1021783020548"},{"key":"bibr17-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1016\/j.oceaneng.2017.02.035"},{"key":"bibr18-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1109\/TAES.2017.2668071"},{"key":"bibr19-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1016\/j.rcim.2017.10.001"},{"key":"bibr20-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2018.2853806"},{"key":"bibr21-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1177\/0959651820937085"},{"key":"bibr22-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1177\/0959651820956738"},{"key":"bibr23-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2021.3097660"},{"key":"bibr24-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1177\/0278364920979367"},{"key":"bibr25-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2016.2516565"},{"key":"bibr26-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2020.2963998"},{"volume-title":"Reinforcement learning: an introduction","year":"2018","author":"Sutton RS","key":"bibr27-09596518221125533"},{"key":"bibr28-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2017.2743240"},{"volume-title":"Learning from delayed rewards","year":"1989","author":"Watkins CJCH","key":"bibr29-09596518221125533"},{"key":"bibr30-09596518221125533","first-page":"1057","volume":"2000","author":"Sutton RS","year":"2000","journal-title":"Adv Neural Inf Process Syst"},{"key":"bibr31-09596518221125533","unstructured":"Silver D, Lever G, Heess N, et al. Deterministic policy gradient algorithms. In: International conference on machine learning, 2014, vol. 32, pp.605\u2013619, http:\/\/proceedings.mlr.press\/v32\/silver14.pdf"},{"key":"bibr32-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1038\/nature14236"},{"key":"bibr33-09596518221125533","first-page":"1","author":"Lillicrap TP","year":"2015","journal-title":"Arxiv"},{"key":"bibr34-09596518221125533","unstructured":"Schulman J, Wolski F, Dhariwal P, et al. Proximal policy optimization algorithms. Arxiv, 2017, pp.1\u201312, https:\/\/arxiv.org\/abs\/1707.06347"},{"key":"bibr35-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1016\/j.ymssp.2020.107502"},{"key":"bibr36-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2020.2979600"},{"volume-title":"IEEE CCTA","author":"Norouzi A","key":"bibr37-09596518221125533"},{"volume-title":"IEEE CCECE","author":"Norouzi A","key":"bibr38-09596518221125533"},{"key":"bibr39-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1115\/1.4037265"},{"key":"bibr40-09596518221125533","first-page":"6213","volume":"53","author":"Norouzi A","year":"2020","journal-title":"IFAC Proc"},{"key":"bibr41-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2017.2673865"},{"key":"bibr42-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1016\/j.ymssp.2018.09.001"},{"issue":"1","key":"bibr43-09596518221125533","doi-asserted-by":"crossref","first-page":"136","DOI":"10.1109\/JAS.2017.7510871","volume":"7","author":"Pradhan SK","year":"2020","journal-title":"IEEE\/CAA J Autom Sin"},{"key":"bibr44-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1109\/TASE.2012.2189004"},{"volume-title":"Robot dynamics and control","year":"2008","author":"Spong MW","key":"bibr45-09596518221125533"},{"key":"bibr46-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1109\/TRA.2002.807548"},{"key":"bibr47-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1177\/027836499501400201"},{"key":"bibr48-09596518221125533","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.123."},{"key":"bibr49-09596518221125533","unstructured":"Glorot X, Bengio Y. Understanding the difficulty of training deep feedforward neural networks. In: International conference on artificial intelligence and statistics, 2010, vol. 9, pp.249\u2013256, https:\/\/proceedings.mlr.press\/v9\/glorot10a\/glorot10a.pdf"},{"key":"bibr50-09596518221125533","first-page":"1","volume-title":"Arxiv","author":"Schulman J","year":"2015"}],"container-title":["Proceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/09596518221125533","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/09596518221125533","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/09596518221125533","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,2]],"date-time":"2025-03-02T06:29:35Z","timestamp":1740896975000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/09596518221125533"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,28]]},"references-count":50,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2023,2]]}},"alternative-id":["10.1177\/09596518221125533"],"URL":"https:\/\/doi.org\/10.1177\/09596518221125533","relation":{},"ISSN":["0959-6518","2041-3041"],"issn-type":[{"type":"print","value":"0959-6518"},{"type":"electronic","value":"2041-3041"}],"subject":[],"published":{"date-parts":[[2022,9,28]]}}}