{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,17]],"date-time":"2026-08-17T23:08:05Z","timestamp":1787008085531,"version":"build-2736575974"},"reference-count":31,"publisher":"IOP Publishing","issue":"6","license":[{"start":{"date-parts":[[2025,10,15]],"date-time":"2025-10-15T00:00:00Z","timestamp":1760486400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/publishingsupport.iopscience.iop.org\/iop-standard\/v1"},{"start":{"date-parts":[[2025,10,15]],"date-time":"2025-10-15T00:00:00Z","timestamp":1760486400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Bioinspir. Biomim."],"published-print":{"date-parts":[[2025,11,28]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    Achieving robust and energy-efficient navigation in unknown fluid environments remains a key challenge for bioinspired underwater robots. In this study, we develop a reinforcement learning-based control framework that enables a fish-like swimmer to autonomously acquire effective navigation strategies within a high-fidelity computational fluid dynamics environment. By shaping the reward function to favor energy efficiency, the agent spontaneously discovers different locomotion patterns, ranging from continuous bursting to burst-and-coast gaits, all without prior knowledge of fluid mechanics. Although the agent is trained in a quiescent fluid environment, the learned swimming policies are generalized well in various navigation tasks and remain robust under complex flow perturbations, including uniform currents and unsteady vortex wakes. In all test scenarios, the agent achieves a 100\n                    <jats:inline-formula>\n                      <jats:tex-math>\n                        \n                      <\/jats:tex-math>\n                      <mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\" overflow=\"scroll\">\n                        <mml:mrow>\n                          <mml:mi mathvariant=\"normal\">%<\/mml:mi>\n                        <\/mml:mrow>\n                      <\/mml:math>\n                    <\/jats:inline-formula>\n                    navigation success rate. These findings highlight the potential of integrating physics-based simulation with learning-based control strategy to advance the design of adaptive, efficient, and resilient aquatic robots inspired by biological swimmers.\n                  <\/jats:p>","DOI":"10.1088\/1748-3190\/ae0dd1","type":"journal-article","created":{"date-parts":[[2025,9,30]],"date-time":"2025-09-30T22:52:18Z","timestamp":1759272738000},"page":"066007","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Reinforcement learning for robust navigation of fish-like agents in various fluid environments"],"prefix":"10.1088","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0009-0002-6058-5437","authenticated-orcid":true,"given":"Jin","family":"Zhang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-9291-2761","authenticated-orcid":true,"given":"Xiaolong","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6387-2314","authenticated-orcid":true,"given":"Bochao","family":"Cao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"266","published-online":{"date-parts":[[2025,10,15]]},"reference":[{"key":"bbae0dd1bib1","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1146\/annurev.fluid.32.1.33","type":"journal-article","article-title":"Hydrodynamics of fishlike swimming","volume":"32","author":"Triantafyllou","year":"2000","journal-title":"Annu. Rev. Fluid Mech."},{"key":"bbae0dd1bib2","doi-asserted-by":"publisher","first-page":"eaar3449","DOI":"10.1126\/scirobotics.aar3449","type":"journal-article","article-title":"Exploration of underwater life with an acoustically controlled soft robotic fish","volume":"3","author":"Katzschmann","year":"2018","journal-title":"Sci. Robot."},{"key":"bbae0dd1bib3","doi-asserted-by":"publisher","first-page":"66","DOI":"10.1038\/s41586-020-03153-z","type":"journal-article","article-title":"Self-powered soft robot in the mariana trench","volume":"591","author":"Li","year":"2021","journal-title":"Nature"},{"key":"bbae0dd1bib4","doi-asserted-by":"publisher","DOI":"10.1088\/1748-3190\/11\/3\/031001","type":"journal-article","article-title":"Fish-inspired robots: design, sensing, actuation and autonomy\u2014a review of research","volume":"11","author":"Raj","year":"2016","journal-title":"Bioinspir. Biomim."},{"key":"bbae0dd1bib5","doi-asserted-by":"publisher","first-page":"773","DOI":"10.3390\/jmse10060773","type":"journal-article","article-title":"Recent progress in modeling and control of bio-inspired fish robots","volume":"10","author":"Sun","year":"2022","journal-title":"J. Mar. Sci. Eng."},{"key":"bbae0dd1bib6","doi-asserted-by":"publisher","DOI":"10.1155\/2014\/631296","type":"journal-article","article-title":"Design and control of an embedded vision guided robotic fish with multiple control surfaces","volume":"2014","author":"Yu","year":"2014","journal-title":"Sci. World J."},{"key":"bbae0dd1bib7","doi-asserted-by":"publisher","first-page":"1962","DOI":"10.1109\/TNNLS.2015.2459913","type":"journal-article","article-title":"CPG network optimization for a biomimetic robotic fish via PSO","volume":"27","author":"Yu","year":"2016","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"bbae0dd1bib8","doi-asserted-by":"publisher","DOI":"10.1098\/rspa.2012.0671","type":"journal-article","article-title":"Flow-relative control of an underwater robot","volume":"469","author":"Salum\u00e4e","year":"2013","journal-title":"Proc. R. Soc. A"},{"key":"bbae0dd1bib9","first-page":"pp 4602","type":"conference-proceedings","article-title":"Flow-aided path following of an underwater robot","author":"Jung","year":"2013"},{"key":"bbae0dd1bib10","doi-asserted-by":"publisher","first-page":"477","DOI":"10.1146\/annurev-fluid-010719-060214","type":"journal-article","article-title":"Machine learning for fluid mechanics","volume":"52","author":"Brunton","year":"2020","journal-title":"Annu. Rev. Fluid Mech."},{"key":"bbae0dd1bib11","doi-asserted-by":"publisher","DOI":"10.1063\/5.0143913","type":"journal-article","article-title":"Recent advances in applying deep reinforcement learning for flow control: perspectives and future directions","volume":"35","author":"Vignon","year":"2023","journal-title":"Phys. Fluids"},{"key":"bbae0dd1bib12","doi-asserted-by":"publisher","first-page":"B622","DOI":"10.1137\/130943078","type":"journal-article","article-title":"Reinforcement learning and wavelet adapted vortex methods for simulations of self-propelled swimmers","volume":"36","author":"Gazzola","year":"2014","journal-title":"SIAM J. Sci. Comput."},{"key":"bbae0dd1bib13","doi-asserted-by":"publisher","first-page":"5849","DOI":"10.1073\/pnas.1800923115","type":"journal-article","article-title":"Efficient collective swimming by harnessing vortices through deep reinforcement learning","volume":"115","author":"Verma","year":"2018","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"bbae0dd1bib14","doi-asserted-by":"publisher","DOI":"10.1088\/1748-3190\/ac9fb5","type":"journal-article","article-title":"Learning to school in dense configurations with multi-agent deep reinforcement learning","volume":"18","author":"Zhu","year":"2022","journal-title":"Bioinspir. Biomim."},{"key":"bbae0dd1bib15","doi-asserted-by":"publisher","DOI":"10.1088\/1748-3190\/ac165e","type":"journal-article","article-title":"Fish can save energy via proprioceptive sensing","volume":"16","author":"Li","year":"2021","journal-title":"Bioinspir. Biomim."},{"key":"bbae0dd1bib16","doi-asserted-by":"publisher","first-page":"1691","DOI":"10.1038\/s41598-021-81124-8","type":"journal-article","article-title":"A numerical study of fish adaption behaviors in complex environments with a deep reinforcement learning and immersed boundary\u2013Lattice Boltzmann method","volume":"11","author":"Zhu","year":"2021","journal-title":"Sci. Rep."},{"key":"bbae0dd1bib17","doi-asserted-by":"publisher","first-page":"7143","DOI":"10.1038\/s41467-021-27015-y","type":"journal-article","article-title":"Learning efficient navigation in vortical flow fields","volume":"12","author":"Gunnarson","year":"2021","journal-title":"Nat. Commun."},{"key":"bbae0dd1bib18","doi-asserted-by":"publisher","DOI":"10.1088\/1748-3190\/ad6544","type":"journal-article","article-title":"Learning obstacle avoidance and predation in complex reef environments with deep reinforcement learning","volume":"19","author":"Hou","year":"2024","journal-title":"Bioinspir. Biomim."},{"key":"bbae0dd1bib19","doi-asserted-by":"publisher","first-page":"A27","DOI":"10.1017\/jfm.2025.45","type":"journal-article","article-title":"Intermittent swimmers optimize energy expenditure with flick-to-flick motor control","volume":"1006","author":"Zhu","year":"2025","journal-title":"J. Fluid Mech."},{"key":"bbae0dd1bib20","doi-asserted-by":"publisher","DOI":"10.1088\/1748-3190\/ada59c","type":"journal-article","article-title":"Simulating fish autonomous swimming behaviours using deep reinforcement learning based on Kolmogorov\u2013Arnold networks","volume":"20","author":"Li","year":"2025","journal-title":"Bioinspir. Biomim."},{"key":"bbae0dd1bib21","doi-asserted-by":"publisher","first-page":"446","DOI":"10.1016\/j.jcp.2013.04.033","type":"journal-article","article-title":"A unified mathematical framework and an adaptive numerical method for fluid\u2013structure interaction with rigid, deforming and elastic bodies","volume":"250","author":"Bhalla","year":"2013","journal-title":"J. Comput. Phys."},{"key":"bbae0dd1bib22","doi-asserted-by":"publisher","first-page":"1059","DOI":"10.1242\/jeb.00209","type":"journal-article","article-title":"The K\u00e1rm\u00e1n gait: novel body kinematics of rainbow trout swimming in a vortex street","volume":"206","author":"Liao","year":"2003","journal-title":"J. Exp. Biol."},{"key":"bbae0dd1bib23","doi-asserted-by":"publisher","first-page":"4077","DOI":"10.1242\/jeb.02487","type":"journal-article","article-title":"The role of the lateral line and vision on body kinematics and hydrodynamic preference of rainbow trout in turbulent flow","volume":"209","author":"Liao","year":"2006","journal-title":"J. Exp. Biol."},{"key":"bbae0dd1bib24","doi-asserted-by":"publisher","first-page":"8636","DOI":"10.1016\/j.jcp.2008.06.017","type":"journal-article","article-title":"An immersed boundary method for smoothed particle hydrodynamics of self-propelled swimmers","volume":"227","author":"Hieber","year":"2008","journal-title":"J. Comput. Phys."},{"key":"bbae0dd1bib25","doi-asserted-by":"publisher","first-page":"1891","DOI":"10.1002\/fld.2670","type":"journal-article","article-title":"Simulation of self-propelled anguilliform swimming by local domain-free discretization method","volume":"69","author":"Zhou","year":"2012","journal-title":"Int. J. Numer. Methods Fluids"},{"key":"bbae0dd1bib26","doi-asserted-by":"publisher","first-page":"A46","DOI":"10.1017\/jfm.2024.262","type":"journal-article","article-title":"Tailbeat perturbations improve swimming efficiency in self-propelled flapping foils","volume":"984","author":"Chao","year":"2024","journal-title":"J. Fluid Mech."},{"key":"bbae0dd1bib27","first-page":"pp 16895","type":"conference-proceedings","article-title":"Hydrodynamic interactions in schooling fish: prioritizing real fish kinematics over travelling-wavy undulation","author":"Chao","year":"2024"},{"key":"bbae0dd1bib28","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1146\/annurev-fluid-120720-022129","type":"journal-article","article-title":"Immersed boundary methods: historical perspective and future outlook","volume":"55","author":"Verzicco","year":"2023","journal-title":"Annu. Rev. Fluid Mech."},{"key":"bbae0dd1bib29","article-title":"Proximal policy optimization algorithms","author":"Schulman","year":"2017","type":"preprint"},{"key":"bbae0dd1bib30","doi-asserted-by":"publisher","first-page":"153","DOI":"10.1146\/annurev-control-030323-022510","type":"journal-article","article-title":"Deep reinforcement learning for robotics: a survey of real-world successes","volume":"8","author":"Tang","year":"2025","journal-title":"Annu. Rev. Control Robot. Auton. Syst."},{"key":"bbae0dd1bib31","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1016\/j.cub.2017.12.002","type":"journal-article","article-title":"Structure of the zebrafish locomotor repertoire revealed with unsupervised behavioral clustering","volume":"28","author":"Marques","year":"2018","journal-title":"Curr. Biol."}],"container-title":["Bioinspiration &amp; Biomimetics"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1748-3190\/ae0dd1","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1748-3190\/ae0dd1\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1748-3190\/ae0dd1","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1748-3190\/ae0dd1\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1748-3190\/ae0dd1\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1748-3190\/ae0dd1\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1748-3190\/ae0dd1\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1748-3190\/ae0dd1\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,15]],"date-time":"2025-10-15T07:50:49Z","timestamp":1760514649000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1748-3190\/ae0dd1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,15]]},"references-count":31,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2025,10,15]]},"published-print":{"date-parts":[[2025,11,28]]}},"URL":"https:\/\/doi.org\/10.1088\/1748-3190\/ae0dd1","relation":{},"ISSN":["1748-3182","1748-3190"],"issn-type":[{"value":"1748-3182","type":"print"},{"value":"1748-3190","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,15]]},"assertion":[{"value":"Reinforcement learning for robust navigation of fish-like agents in various fluid environments","name":"article_title","label":"Article Title"},{"value":"Bioinspiration & Biomimetics","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2025 IOP Publishing Ltd. All rights, including for text and data mining, AI training, and similar technologies, are reserved.","name":"copyright_information","label":"Copyright Information"},{"value":"2025-08-01","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2025-09-30","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2025-10-15","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}