{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,20]],"date-time":"2026-05-20T16:30:27Z","timestamp":1779294627646,"version":"3.51.4"},"reference-count":45,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2024,1,16]],"date-time":"2024-01-16T00:00:00Z","timestamp":1705363200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Autonomous mobile robots have become integral to daily life, providing crucial services across diverse domains. This paper focuses on path following, a fundamental technology and critical element in achieving autonomous mobility. Existing methods predominantly address tracking through steering control, neglecting velocity control or relying on path-specific reference velocities, thereby constraining their generality. In this paper, we propose a novel approach that integrates the conventional pure pursuit algorithm with deep reinforcement learning for a nonholonomic mobile robot. Our methodology employs pure pursuit for steering control and utilizes the soft actor-critic algorithm to train a velocity control strategy within randomly generated path environments. Through simulation and experimental validation, our approach exhibits notable advancements in path convergence and adaptive velocity adjustments to accommodate paths with varying curvatures. Furthermore, this method holds the potential for broader applicability to vehicles adhering to nonholonomic constraints beyond the specific model examined in this paper. In summary, our study contributes to the progression of autonomous mobility by harmonizing conventional algorithms with cutting-edge deep reinforcement learning techniques, enhancing the robustness of path following.<\/jats:p>","DOI":"10.3390\/s24020561","type":"journal-article","created":{"date-parts":[[2024,1,16]],"date-time":"2024-01-16T04:03:30Z","timestamp":1705377810000},"page":"561","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":28,"title":["Path Following for Autonomous Mobile Robots with Deep Reinforcement Learning"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5194-0225","authenticated-orcid":false,"given":"Yu","family":"Cao","sequence":"first","affiliation":[{"name":"Program of Intelligence and Control, Cluster of Electronics and Mechanical Engineering, School of Science and Technology, Gunma University, 1-5-1 Tenjin-cho, Kiryu 376-8515, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kan","family":"Ni","sequence":"additional","affiliation":[{"name":"Program of Intelligence and Control, Cluster of Electronics and Mechanical Engineering, School of Science and Technology, Gunma University, 1-5-1 Tenjin-cho, Kiryu 376-8515, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4460-8694","authenticated-orcid":false,"given":"Takahiro","family":"Kawaguchi","sequence":"additional","affiliation":[{"name":"Program of Intelligence and Control, Cluster of Electronics and Mechanical Engineering, School of Science and Technology, Gunma University, 1-5-1 Tenjin-cho, Kiryu 376-8515, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Seiji","family":"Hashimoto","sequence":"additional","affiliation":[{"name":"Program of Intelligence and Control, Cluster of Electronics and Mechanical Engineering, School of Science and Technology, Gunma University, 1-5-1 Tenjin-cho, Kiryu 376-8515, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,1,16]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"39830","DOI":"10.1109\/ACCESS.2020.2975643","article-title":"A review on challenges of autonomous mobile robot and sensor fusion methods","volume":"8","author":"Alatise","year":"2020","journal-title":"IEEE Access"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Li, G., Lin, R., Li, M., Sun, R., and Piao, S. (2019). A master-slave separate parallel intelligent mobile robot used for autonomous pallet transportation. Appl. Sci., 9.","DOI":"10.3390\/app9030368"},{"key":"ref_3","first-page":"307","article-title":"Security challenges for user-oriented RFID applications within the Internet of thing","volume":"11","author":"Hancke","year":"2010","journal-title":"J. Internet Technol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"787","DOI":"10.1080\/00207179.2013.858829","article-title":"Navigation of autonomous vehicles for oil spill cleaning in dynamic and uncertain environments","volume":"87","author":"Jin","year":"2014","journal-title":"Int. J. Control"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1080\/01691864.2019.1694068","article-title":"System for augmented human\u2013robot interaction through mixed reality and robot training by non-experts in customer service environments","volume":"34","author":"Isobe","year":"2020","journal-title":"Adv. Robot."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"271","DOI":"10.1108\/01439911211217107","article-title":"An autonomous restaurant service robot with high positioning accuracy","volume":"39","author":"Yu","year":"2012","journal-title":"Ind. Robot. Int. J."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1016\/j.jterra.2020.06.006","article-title":"A review of autonomous agricultural vehicles (The experience of Hokkaido University)","volume":"91","author":"Roshanianfard","year":"2020","journal-title":"J. Terramech."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"134","DOI":"10.5772\/61391","article-title":"CF-pursuit: A pursuit method with a clothoid fitting and a fuzzy controller for autonomous vehicles","volume":"12","author":"Shan","year":"2015","journal-title":"Int. J. Adv. Robot. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"44","DOI":"10.5772\/51314","article-title":"Design of a control system for an autonomous vehicle based on adaptive-pid","volume":"9","author":"Zhao","year":"2012","journal-title":"Int. J. Adv. Robot. Syst."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"997","DOI":"10.1243\/09544070JAUTO1366","article-title":"Parallel auto-parking of a model vehicle using a self-organizing fuzzy controller","volume":"224","author":"Huang","year":"2010","journal-title":"Proc. Inst. Mech. Eng. Part J. Automob. Eng."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Faulwasser, T., Kern, B., and Findeisen, R. (2009, January 15\u201318). Model predictive path-following for constrained nonlinear systems. Proceedings of the 48h IEEE Conference on Decision and Control (CDC) Held Jointly with 2009 28th Chinese Control Conference, Shanghai, China.","DOI":"10.1109\/CDC.2009.5399744"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1109\/TIV.2016.2578706","article-title":"A Survey of Motion Planning and Control Techniques for Self-Driving Urban Vehicles","volume":"1","author":"Paden","year":"2016","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_13","unstructured":"Wallace, R.S., Stentz, A., Thorpe, C.E., Moravec, H.P., Whittaker, W., and Kanade, T. (1985, January 18\u201323). First Results in Robot Road-Following. Proceedings of the 9th International Joint Conference on Artificial Intelligence, San Francisco, CA, USA."},{"key":"ref_14","unstructured":"Amidi, O., and Thorpe, C.E. (1991, January 1). Integrated mobile robot control. Proceedings of the Mobile Robots V. SPIE, Boston, MA, USA."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Buehler, M., Iagnemma, K., and Singh, S. (2007). The 2005 DARPA Grand Challenge: The Great Robot Race, Springer.","DOI":"10.1007\/978-3-540-73429-1"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Buehler, M., Iagnemma, K., and Singh, S. (2009). The DARPA Urban Challenge: Autonomous Vehicles in City Traffic, Springer.","DOI":"10.1007\/978-3-642-03991-1"},{"key":"ref_17","unstructured":"Coulter, R.C. (1992). Implementation of the Pure Pursuit Path Tracking Algorithm, Carnegie-Mellon UNIV Robotics INST. Technical Report."},{"key":"ref_18","unstructured":"Li, Y. (2017). Deep reinforcement learning: An overview. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"484","DOI":"10.1038\/nature16961","article-title":"Mastering the game of Go with deep neural networks and tree search","volume":"529","author":"Silver","year":"2016","journal-title":"Nature"},{"key":"ref_20","first-page":"1334","article-title":"End-to-end training of deep visuomotor policies","volume":"17","author":"Levine","year":"2016","journal-title":"J. Mach. Learn. Res."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"6962","DOI":"10.1109\/TSMC.2020.2966631","article-title":"Multi-kernel online reinforcement learning for path tracking control of intelligent vehicles","volume":"51","author":"Liu","year":"2020","journal-title":"IEEE Trans. Syst. Man, Cybern. Syst."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Chen, L., Chen, Y., Yao, X., Shan, Y., and Chen, L. (2019, January 9\u201312). An adaptive path tracking controller based on reinforcement learning with urban driving application. Proceedings of the 2019 IEEE Intelligent Vehicles Symposium (IV), Paris, France.","DOI":"10.1109\/IVS.2019.8814130"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"10581","DOI":"10.1109\/TVT.2020.3014628","article-title":"A reinforcement learning-based adaptive path tracking approach for autonomous driving","volume":"69","author":"Shan","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"541","DOI":"10.1177\/0954407020954591","article-title":"Deep reinforcement learning based path tracking controller for autonomous vehicle","volume":"235","author":"Chen","year":"2021","journal-title":"Inst. Mech. Eng. Part J. Automob. Eng."},{"key":"ref_25","unstructured":"Gonz\u00e1lez, R., Rodr\u00edguez, F., and Guzm\u00e1n, J.L. (2014). Modeling, Localization, and Motion Control, Springer."},{"key":"ref_26","unstructured":"Corke, P.I., Jachimczyk, W., and Pillat, R. (2011). Robotics, Vision and Control: Fundamental Algorithms in MATLAB, Springer."},{"key":"ref_27","unstructured":"Kanayama, Y., Kimura, Y., Miyazaki, F., and Noguchi, T. (1990, January 13\u201318). A stable tracking control method for an autonomous mobile robot. Proceedings of the IEEE International Conference on Robotics and Automation, Cincinnati, OH, USA."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Martinsen, A.B., and Lekkas, A.M. (2018, January 22\u201325). Curved path following with deep reinforcement learning: Results from three vessel models. Proceedings of the OCEANS 2018 MTS\/IEEE Charleston, Charleston, SC, USA.","DOI":"10.1109\/OCEANS.2018.8604829"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Boyd, S.P., and Vandenberghe, L. (2004). Convex Optimization, Cambridge University Press.","DOI":"10.1017\/CBO9780511804441"},{"key":"ref_30","first-page":"35","article-title":"A survey of nonlinear conjugate gradient methods","volume":"2","author":"Hager","year":"2006","journal-title":"Pac. J. Optim."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1016\/S0927-0507(05)80172-0","article-title":"Markov decision processes","volume":"2","author":"Puterman","year":"1990","journal-title":"Handbooks in Operations Research and Management Science"},{"key":"ref_32","unstructured":"Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. (2018, January 10\u201315). Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. Proceedings of the 35th International Conference on Machine Learning. PMLR, Stockholmsm\u00e4ssan, Stockholm, Sweden."},{"key":"ref_33","unstructured":"Haarnoja, T., Zhou, A., Hartikainen, K., Tucker, G., Ha, S., Tan, J., Kumar, V., Zhu, H., Gupta, A., and Abbeel, P. (2018). Soft actor-critic algorithms and applications. arXiv."},{"key":"ref_34","unstructured":"Fujimoto, S., Hoof, H., and Meger, D. (2018, January 10\u201315). Addressing function approximation error in actor-critic methods. Proceedings of the 35th International Conference on Machine Learning, PMLR, Stockholmsm\u00e4ssan, Stockholm, Sweden."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Cao, Y., Ni, K., Jiang, X., Kuroiwa, T., Zhang, H., Kawaguchi, T., Hashimoto, S., and Jiang, W. (2023). Path following for Autonomous Ground Vehicle Using DDPG Algorithm: A Reinforcement Learning Approach. Appl. Sci., 13.","DOI":"10.3390\/app13116847"},{"key":"ref_36","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_37","unstructured":"Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., and Zaremba, W. (2016). Openai gym. arXiv."},{"key":"ref_38","unstructured":"Wang, H., Kearney, J., and Atkinson, K. (July, January 27). Arc-length parameterized spline curves for real-time simulation. Proceedings of the 5th International Conference on Curves and Surfaces, Saint-Malo, France."},{"key":"ref_39","unstructured":"(2023, November 18). Gmapping Package. Available online: http:\/\/wiki.ros.org\/gmapping."},{"key":"ref_40","unstructured":"(2023, November 18). Adaptive Monte Carlo Localization Package. Available online: http:\/\/wiki.ros.org\/amcl."},{"key":"ref_41","unstructured":"Ramstedt, S., Bouteiller, Y., Beltrame, G., Pal, C., and Binas, J. (2020). Reinforcement learning with random delays. arXiv."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1007\/s10514-020-09951-8","article-title":"Deep reinforcement learning for quadrotor path following with adaptive velocity","volume":"45","author":"Morcego","year":"2021","journal-title":"Auton. Robot."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"3861","DOI":"10.1109\/TAC.2016.2638961","article-title":"Control barrier function based quadratic programs for safety critical systems","volume":"62","author":"Ames","year":"2016","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Molnar, T.G., and Ames, A.D. (2023). Safety-Critical Control with Bounded Inputs via Reduced Order Models. arXiv.","DOI":"10.23919\/ACC55779.2023.10155871"},{"key":"ref_45","unstructured":"Janwani, N.C., Da\u015f, E., Touma, T., Wei, S.X., Molnar, T.G., and Burdick, J.W. (2023). A learning-based framework for safe human-robot collaboration with multiple backup control barrier functions. arXiv."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/2\/561\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T13:47:52Z","timestamp":1760104072000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/2\/561"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,16]]},"references-count":45,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2024,1]]}},"alternative-id":["s24020561"],"URL":"https:\/\/doi.org\/10.3390\/s24020561","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,1,16]]}}}