{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T15:04:59Z","timestamp":1764687899039,"version":"3.38.0"},"reference-count":26,"publisher":"SAGE Publications","issue":"10","license":[{"start":{"date-parts":[[2020,10,8]],"date-time":"2020-10-08T00:00:00Z","timestamp":1602115200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["51779058 and 51979048"],"award-info":[{"award-number":["51779058 and 51979048"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Proceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Engineering"],"published-print":{"date-parts":[[2021,11]]},"abstract":"<jats:p> The environmental adaptability of autonomous underwater vehicles is always a problem for its path planning. Although reinforcement learning can improve the environmental adaptability, the slow convergence of reinforcement learning is caused by multi-behavior coupling, so it is difficult for autonomous underwater vehicle to avoid moving obstacles. This article proposes a multi-behavior critic reinforcement learning algorithm applied to autonomous underwater vehicle path planning to overcome problems associated with oscillating amplitudes and low learning efficiency in the early stages of training which are common in traditional actor\u2013critic algorithms. Behavior critic reinforcement learning assesses the actions of the actor from perspectives such as energy saving and security, combining these aspects into a whole evaluation of the actor. In this article, the policy gradient method is selected as the actor part, and the value function method is selected as the critic part. The strategy gradient and the value function methods for actor and critic, respectively, are approximated by a backpropagation neural network, the parameters of which are updated using the gradient descent method. The simulation results show that the method has the ability of optimizing learning in the environment and can improve learning efficiency, which meets the needs of real time and adaptability for autonomous underwater vehicle dynamic obstacle avoidance. <\/jats:p>","DOI":"10.1177\/0959651820937085","type":"journal-article","created":{"date-parts":[[2020,10,8]],"date-time":"2020-10-08T08:53:24Z","timestamp":1602147204000},"page":"1787-1796","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":16,"title":["Autonomous underwater vehicle path planning based on actor-multi-critic reinforcement learning"],"prefix":"10.1177","volume":"235","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3885-4821","authenticated-orcid":false,"given":"Zhuo","family":"Wang","sequence":"first","affiliation":[{"name":"Science and Technology on Underwater Vehicle Laboratory, Harbin Engineering University, Harbin, China"},{"name":"Peng Cheng Laboratory, Shenzhen, China"}]},{"given":"Shiwei","family":"Zhang","sequence":"additional","affiliation":[{"name":"Science and Technology on Underwater Vehicle Laboratory, Harbin Engineering University, Harbin, China"}]},{"given":"Xiaoning","family":"Feng","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Harbin Engineering University, Harbin, China"}]},{"given":"Yancheng","family":"Sui","sequence":"additional","affiliation":[{"name":"Science and Technology on Underwater Vehicle Laboratory, Harbin Engineering University, Harbin, China"}]}],"member":"179","published-online":{"date-parts":[[2020,10,8]]},"reference":[{"key":"bibr1-0959651820937085","doi-asserted-by":"publisher","DOI":"10.1007\/s10489-018-1241-z"},{"first-page":"1928","volume-title":"Proceedings of the 2016 International conference on machine learning","author":"Mnih V","key":"bibr2-0959651820937085"},{"first-page":"57","volume-title":"Proceedings of the 2017 IEEE\/RSJ international conference on intelligent robots and systems (IROS)","author":"Tai L","key":"bibr3-0959651820937085"},{"key":"bibr4-0959651820937085","doi-asserted-by":"publisher","DOI":"10.1016\/j.oceaneng.2019.106299"},{"key":"bibr5-0959651820937085","doi-asserted-by":"publisher","DOI":"10.1177\/0036850419879024"},{"first-page":"3458","volume-title":"Proceedings of the 2014 IEEE international conference on systems, man, and cybernetics (SMC)","author":"Cruz DL","key":"bibr6-0959651820937085"},{"first-page":"123","volume-title":"Proceedings of the 2015 IEEE student conference on research and development (SCOReD)","author":"Yusof Y","key":"bibr7-0959651820937085"},{"first-page":"3397","volume-title":"Proceedings of the 2017 36th Chinese control conference (CCC)","author":"Yijing Z","key":"bibr8-0959651820937085"},{"key":"bibr9-0959651820937085","doi-asserted-by":"publisher","DOI":"10.1016\/j.robot.2019.02.013"},{"key":"bibr10-0959651820937085","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2929120"},{"key":"bibr11-0959651820937085","doi-asserted-by":"publisher","DOI":"10.3390\/app9020323"},{"first-page":"837","volume-title":"Proceedings of the 2017 IEEE international conference on industrial technology (ICIT)","author":"Sharma A","key":"bibr12-0959651820937085"},{"first-page":"1","volume-title":"Proceedings of the IEEE underwater technology (UT)","author":"Noguchi Y","key":"bibr13-0959651820937085"},{"key":"bibr14-0959651820937085","first-page":"679","volume":"6","author":"Bellman R","year":"1957","journal-title":"J Math Mech"},{"key":"bibr15-0959651820937085","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.42.10.767"},{"key":"bibr16-0959651820937085","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2009.07.008"},{"first-page":"1","volume-title":"Proceedings of the 2016 IEEE international workshop on acoustic signal enhancement (IWAENC)","author":"Xu L","key":"bibr17-0959651820937085"},{"key":"bibr18-0959651820937085","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2019.05.010"},{"key":"bibr19-0959651820937085","doi-asserted-by":"publisher","DOI":"10.1016\/0893-6080(89)90020-8"},{"key":"bibr20-0959651820937085","first-page":"315","volume":"15","author":"Glorot X","year":"2011","journal-title":"J Mach Learn Res"},{"key":"bibr21-0959651820937085","first-page":"1","volume":"10","author":"Ramachandran P","year":"2017","journal-title":"Comput Sci"},{"key":"bibr22-0959651820937085","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2013.2296048"},{"key":"bibr23-0959651820937085","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2014.2334366"},{"key":"bibr24-0959651820937085","doi-asserted-by":"publisher","DOI":"10.1109\/TVT.2019.2927893"},{"key":"bibr25-0959651820937085","doi-asserted-by":"publisher","DOI":"10.1016\/j.jfranklin.2019.05.034"},{"key":"bibr26-0959651820937085","doi-asserted-by":"publisher","DOI":"10.1016\/j.oceaneng.2019.106341"}],"container-title":["Proceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0959651820937085","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/0959651820937085","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0959651820937085","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,3]],"date-time":"2025-03-03T06:15:53Z","timestamp":1740982553000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/0959651820937085"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,8]]},"references-count":26,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2021,11]]}},"alternative-id":["10.1177\/0959651820937085"],"URL":"https:\/\/doi.org\/10.1177\/0959651820937085","relation":{},"ISSN":["0959-6518","2041-3041"],"issn-type":[{"type":"print","value":"0959-6518"},{"type":"electronic","value":"2041-3041"}],"subject":[],"published":{"date-parts":[[2020,10,8]]}}}