{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T20:02:58Z","timestamp":1776110578074,"version":"3.50.1"},"reference-count":35,"publisher":"Cambridge University Press (CUP)","issue":"4","license":[{"start":{"date-parts":[[2024,2,12]],"date-time":"2024-02-12T00:00:00Z","timestamp":1707696000000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/www.cambridge.org\/core\/terms"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Robotica"],"published-print":{"date-parts":[[2024,4]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Snake robots can move flexibly due to their special bodies and gaits. However, it is difficult to plan their motion in multi-obstacle environments due to their complex models. To solve this problem, this work investigates a reinforcement learning-based motion planning method. To plan feasible paths, together with a modified deep Q-learning algorithm, a Floyd-moving average algorithm is proposed to ensure smoothness and adaptability of paths for snake robots\u2019 passing. An improved path integral algorithm is used to work out gait parameters to control snake robots to move along the planned paths. To speed up the training of parameters, a strategy combining serial training, parallel training, and experience replaying modules is designed. Moreover, we have designed a motion planning framework consists of path planning, path smoothing, and motion planning. Various simulations are conducted to validate the effectiveness of the proposed algorithms.<\/jats:p>","DOI":"10.1017\/s0263574723001613","type":"journal-article","created":{"date-parts":[[2024,2,12]],"date-time":"2024-02-12T08:49:22Z","timestamp":1707727762000},"page":"947-961","source":"Crossref","is-referenced-by-count":16,"title":["Reinforcement learning-based motion control for snake robots in complex environments"],"prefix":"10.1017","volume":"42","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0090-9006","authenticated-orcid":false,"given":"Dong","family":"Zhang","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0057-3138","authenticated-orcid":false,"given":"Renjie","family":"Ju","sequence":"additional","affiliation":[]},{"given":"Zhengcai","family":"Cao","sequence":"additional","affiliation":[]}],"member":"56","published-online":{"date-parts":[[2024,2,12]]},"reference":[{"key":"S0263574723001613_ref4","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2015.2400655"},{"key":"S0263574723001613_ref12","doi-asserted-by":"publisher","DOI":"10.1016\/S0921-8890(02)00312-3"},{"key":"S0263574723001613_ref30","doi-asserted-by":"publisher","DOI":"10.1109\/RAAD.2014.7002234"},{"key":"S0263574723001613_ref32","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1016\/j.procs.2019.11.279","article-title":"Optimal coordination path selecting method for conduction transformation based on floyd algorithm","volume":"162","author":"Zhao","year":"2019","journal-title":"Procedia Comput Sci"},{"key":"S0263574723001613_ref31","first-page":"1","article-title":"Target directed locomotion of a snake like robot based on path integral reinforcement learning","volume":"32","author":"Fang","year":"2019","journal-title":"Patt Recog Art Intell"},{"key":"S0263574723001613_ref3","volume-title":"Biologically Inspired Robots: Snake-Like Locomotors and Minipulators","author":"Hirose","year":"1993"},{"key":"S0263574723001613_ref26","doi-asserted-by":"publisher","DOI":"10.1016\/j.ifacol.2020.12.2619"},{"key":"S0263574723001613_ref35","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TIM.2023.3328080","article-title":"Semi-direct multimap slam system for real-time sparse 3-d map reconstruction","volume":"72","author":"Xie","year":"2023","journal-title":"IEEE Trans Instru Measure"},{"key":"S0263574723001613_ref14","doi-asserted-by":"crossref","unstructured":"[14] Alexander, H. C. , Nak-seung, P. H. , Erik, I. V. and Patricio, A. V. , \u201cOptimal Trajectory Planning and Feedback Control of Lateral Undulation in Snake-Like Robots,\u201d In: Annual American Control Conference (ACC), Milwaukee, WI, USA (IEEE, 2018) pp. 2114\u20132120 .","DOI":"10.23919\/ACC.2018.8431753"},{"key":"S0263574723001613_ref28","first-page":"3137","article-title":"A generalized path integral control approach to reinforcement learning","volume":"11","author":"Theodorou","year":"2010","journal-title":"J Mach Learn Res"},{"key":"S0263574723001613_ref13","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2013.6631317"},{"key":"S0263574723001613_ref10","doi-asserted-by":"publisher","DOI":"10.1109\/TIE.2023.3288202"},{"key":"S0263574723001613_ref2","doi-asserted-by":"publisher","DOI":"10.1115\/1.4025750"},{"key":"S0263574723001613_ref24","doi-asserted-by":"publisher","DOI":"10.1109\/IROS45743.2020.9341520"},{"key":"S0263574723001613_ref6","doi-asserted-by":"publisher","DOI":"10.1109\/REMAR.2018.8449833"},{"key":"S0263574723001613_ref8","first-page":"8","volume-title":"Snake Robots: Modelling, Mechatronics, and Control: Advances in Industrial Control","author":"Liljeb\u00e4ck","year":"2012"},{"key":"S0263574723001613_ref19","doi-asserted-by":"publisher","DOI":"10.1109\/TIE.2021.3075851"},{"key":"S0263574723001613_ref25","doi-asserted-by":"publisher","DOI":"10.1109\/ICAI52203.2021.9445200"},{"key":"S0263574723001613_ref15","doi-asserted-by":"publisher","DOI":"10.1109\/ROBIO.2013.6739514"},{"key":"S0263574723001613_ref27","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2017.2768820"},{"key":"S0263574723001613_ref1","doi-asserted-by":"publisher","DOI":"10.1016\/j.robot.2011.08.010"},{"key":"S0263574723001613_ref18","doi-asserted-by":"crossref","unstructured":"[18] Matsuno, F. and Sato, H. , \u201cTrajectory Tracking Control of Snake Robots Based on Dynamic Model,\u201d In: IEEE International Conference on Robotics and Automation, Barcelona, Spain (IEEE, 2005) pp. 3029\u20133034.","DOI":"10.1109\/ROBOT.2005.1570575"},{"key":"S0263574723001613_ref7","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2010.2056211"},{"key":"S0263574723001613_ref33","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2015.7140022"},{"key":"S0263574723001613_ref34","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2022.3229978"},{"key":"S0263574723001613_ref20","doi-asserted-by":"publisher","DOI":"10.1109\/TCYB.2021.3055519"},{"key":"S0263574723001613_ref29","doi-asserted-by":"publisher","DOI":"10.1109\/TCYB.2020.2983923"},{"key":"S0263574723001613_ref11","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2022.3194689"},{"key":"S0263574723001613_ref17","doi-asserted-by":"publisher","DOI":"10.3182\/20130626-3-AU-2035.00035"},{"key":"S0263574723001613_ref23","doi-asserted-by":"publisher","DOI":"10.1109\/RCAR52367.2021.9517436"},{"key":"S0263574723001613_ref22","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2103.04511"},{"key":"S0263574723001613_ref21","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2010.2088830"},{"key":"S0263574723001613_ref16","doi-asserted-by":"publisher","DOI":"10.1109\/TCST.2020.3043446"},{"key":"S0263574723001613_ref5","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2018.8594466"},{"key":"S0263574723001613_ref9","doi-asserted-by":"publisher","DOI":"10.1109\/TSMC.2023.3256383"}],"container-title":["Robotica"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S0263574723001613","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,11]],"date-time":"2024-11-11T02:01:08Z","timestamp":1731290468000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S0263574723001613\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,12]]},"references-count":35,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2024,4]]}},"alternative-id":["S0263574723001613"],"URL":"https:\/\/doi.org\/10.1017\/s0263574723001613","relation":{},"ISSN":["0263-5747","1469-8668"],"issn-type":[{"value":"0263-5747","type":"print"},{"value":"1469-8668","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,12]]}}}