{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,3]],"date-time":"2026-06-03T13:08:09Z","timestamp":1780492089439,"version":"3.54.1"},"reference-count":40,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2020,3,11]],"date-time":"2020-03-11T00:00:00Z","timestamp":1583884800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100002322","name":"Coordena\u00e7\u00e3o de Aperfei\u00e7oamento de Pessoal de N\u00edvel Superior","doi-asserted-by":"publisher","award":["88887.123914\/2015-00"],"award-info":[{"award-number":["88887.123914\/2015-00"]}],"id":[{"id":"10.13039\/501100002322","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Path planning for sailboat robots is a challenging task particularly due to the kinematics and dynamics modelling of such kinds of wind propelled boats. The problem is divided into two layers. The first one is global were a general trajectory composed of waypoints is planned, which can be done automatically based on some variables such as weather conditions or defined by hand using some human\u2013robot interface (a ground-station). In the second local layer, at execution time, the global route should be followed by making the sailboat proceed between each pair of consecutive waypoints. Our proposal in this paper is an algorithm for the global, path generation layer, which has been developed for the N-Boat (The Sailboat Robot project), in order to compute feasible sailing routes between a start and a target point while avoiding dangerous situations such as obstacles and borders. A reinforcement learning approach (Q-Learning) is used based on a reward matrix and a set of actions that changes according to wind directions to account for the dead zone, which is the region against the wind where the sailboat can not gain velocity. Our algorithm generates straight and zigzag paths accounting for wind direction. The path generated also guarantees the sailboat safety and robustness, enabling it to sail for long periods of time, depending only on the start and target points defined for this global planning. The result is the development of a complete path planner algorithm that, together with the local planner solved in previous work, can be used to allow the final developments of an N-Boat making it a fully autonomous sailboat.<\/jats:p>","DOI":"10.3390\/s20061550","type":"journal-article","created":{"date-parts":[[2020,3,12]],"date-time":"2020-03-12T04:13:57Z","timestamp":1583986437000},"page":"1550","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":61,"title":["High-Level Path Planning for an Autonomous Sailboat Robot Using Q-Learning"],"prefix":"10.3390","volume":"20","author":[{"given":"Andouglas Gon\u00e7alves da Silva","family":"Silva Junior","sequence":"first","affiliation":[{"name":"Universidade Federal do Rio Grande do Norte, DCA-CT-UFRN, Campus Universitario, Lagoa Nova, Natal, RN 59078-970, Brazil"},{"name":"Instituto Federal do Rio Grande do Norte, Av. Sen. Salgado Filho, 1559 - Tirol, Natal - RN 59015-000, Brazil"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Davi Henrique dos","family":"Santos","sequence":"additional","affiliation":[{"name":"Universidade Federal do Rio Grande do Norte, DCA-CT-UFRN, Campus Universitario, Lagoa Nova, Natal, RN 59078-970, Brazil"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alvaro Pinto Fernandes de","family":"Negreiros","sequence":"additional","affiliation":[{"name":"Universidade Federal do Rio Grande do Norte, DCA-CT-UFRN, Campus Universitario, Lagoa Nova, Natal, RN 59078-970, Brazil"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jo\u00e3o Moreno Vilas Boas de Souza","family":"Silva","sequence":"additional","affiliation":[{"name":"Instituto Federal do Rio Grande do Norte, Av. Sen. Salgado Filho, 1559 - Tirol, Natal - RN 59015-000, Brazil"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7735-5630","authenticated-orcid":false,"given":"Luiz Marcos Garcia","family":"Gon\u00e7alves","sequence":"additional","affiliation":[{"name":"Universidade Federal do Rio Grande do Norte, DCA-CT-UFRN, Campus Universitario, Lagoa Nova, Natal, RN 59078-970, Brazil"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,3,11]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1042","DOI":"10.1109\/TLA.2018.8362135","article-title":"Green Robotics: Concepts, challenges, and strategies","volume":"16","author":"Aroca","year":"2018","journal-title":"IEEE Lat. Am. Trans."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Jorge, V.A., Granada, R., Maidana, R.G., Jurak, D.A., Heck, G., Negreiros, A.P., Dos Santos, D.H., Gon\u00e7alves, L.M., and Amory, A.M. (2019). A survey on unmanned surface vehicles for disaster robotics: Main challenges and directions. Sensors, 19.","DOI":"10.3390\/s19030702"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Silva Junior, A.G.D., Lima Sa, S.T.D., Santos, D.H.D., Negreiros, A.P.F.D., Souza e Silva, J.M.V.B.D., Alvarez Jacobo, J.E., and Garcia Goncalves, L.M. (2016). Towards a Real-Time Embedded System for Water Monitoring Installed in a Robotic Sailboat. Sensors, 16.","DOI":"10.3390\/s16081226"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Santos, D., Silva Junior, A., Negreiros, A., Vilas Boas, J., Alvarez, J., Araujo, A., Aroca, R., and Gon\u00e7alves, L. (2016). Design and implementation of a control system for a sailboat robot. Robotics, 5.","DOI":"10.3390\/robotics5010005"},{"key":"ref_5","first-page":"1","article-title":"A gain-scheduling control strategy and short-term path optimization with genetic algorithm for autonomous navigation of a sailboat robot","volume":"16","author":"Goncalves","year":"2019","journal-title":"Int. J. Adv. Rob. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"106299","DOI":"10.1016\/j.oceaneng.2019.106299","article-title":"A knowledge-free path planning approach for smart ships based on reinforcement learning","volume":"189","author":"Chen","year":"2019","journal-title":"Ocean Eng."},{"key":"ref_7","unstructured":"Watkins, C.J.C.H. (1989). Learning from Delayed Rewards. [Ph.D. Thesis, King\u2019s College]."},{"key":"ref_8","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, The MIT Press. [2nd ed.]."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1109\/5254.867915","article-title":"Tracing patterns and attention: Humanoid robot cognition","volume":"15","author":"Garcia","year":"2000","journal-title":"IEEE Intell. Syst. Appl."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Barrios-Aranibar, D., and Goncalves, L.M.G. (2007, January 20\u201324). Learning Coordination in Multi-Agent Systems Using Influence Value Reinforcement Learning. Proceedings of the Seventh International Conference on Intelligent Systems Design and Applications (ISDA 2007), Rio de Janeiro, Brazil.","DOI":"10.1109\/ISDA.2007.4389653"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Sariff, N., and Buniyamin, N. (2006, January 27\u201328). An Overview of Autonomous Mobile Robot Path Planning Algorithms. Proceedings of the 4th Student Conference on Research and Development, Selangor, Malaysia.","DOI":"10.1109\/SCORED.2006.4339335"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Klan\u010dar, G., Zde\u0161ar, A., Bla\u017ei\u010d, S., and \u0160krjanc, I. (2017). Chapter 4 - Path Planning. Wheeled Mobile Robotics, Butterworth-Heinemann.","DOI":"10.1016\/B978-0-12-804204-5.00004-4"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/978-3-319-14705-5_1","article-title":"Path Planning and Trajectory Planning Algorithms: A General Overview","volume":"29","author":"Gasparetto","year":"2015","journal-title":"Mech. Mach. Sci."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"416","DOI":"10.1016\/j.trc.2015.09.011","article-title":"Real-time motion planning methods for autonomous on-road driving: State-of-the-art and future research directions","volume":"60","author":"Katrakazas","year":"2015","journal-title":"Transp. Res. C Emerg. Technol."},{"key":"ref_15","unstructured":"Choset, H., Lynch, K.M., Hutchinson, S., Kantor, G.A., Burgard, W., Kavraki, L.E., and Thrun, S. (2005). Principles of Robot Motion: Theory, Algorithms, and Implementations. Intelligent Robotics and Autonomous Agents Series, MIT Press."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1145\/1721654.1721679","article-title":"Toward Robotic Cars","volume":"53","author":"Thrun","year":"2010","journal-title":"Commun. ACM"},{"key":"ref_17","unstructured":"Oliveira, M., Santos, V., and Sappa, A.D. (2012, January 7\u201312). Short term path planning using a multiple hypothesis evaluation approach for an autonomous driving competition. Proceedings of the Workshop on Planning, Perception and Navigation for Intelligent Vehicles (IROS2012\/PPNIV\u201912), Vilamoura, Portugal."},{"key":"ref_18","first-page":"679","article-title":"A Markovian decision process","volume":"6","author":"Bellman","year":"1957","journal-title":"J. Math. Mech."},{"key":"ref_19","unstructured":"Melo, F.S. (2020, March 08). Convergence of Q-Learning: A Simple Proof. Available online: http:\/\/users.isr.ist.utl.pt\/~mtjspaan\/readingGroup\/ProofQlearning.pdf."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"604","DOI":"10.1016\/j.robot.2007.10.004","article-title":"Autonomous sailboat navigation for short course racing","volume":"56","author":"Stelzer","year":"2008","journal-title":"Rob. Autom. Syst."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2592","DOI":"10.1017\/S0263574715000235","article-title":"Occupancy-elevation grid: An alternative approach for robotic mapping and navigation","volume":"34","author":"Souza","year":"2016","journal-title":"Robotica"},{"key":"ref_22","unstructured":"Baker, R., Kambourian, L., Hajarian, S., Augenstein, T., Harnett, S., Lee, G.M., Sudarshan, M., Richter, C., Trouillot, C., and Williamson, P. (September, January 31). Design and Development of a Self-Stabilizing, Autonomous Sailboat with Zero-Net Stored-Energy Use. Proceedings of the World Robotic Sailing championship and International Robotic Sailing Conference, Mariehamn, Finland."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Saoud, H., Hua, M.D., Plumet, F., and Amar, F.B. (2015, January 2\u20134). Routing and course control of an autonomous sailboat. Proceedings of the 2015 European Conference on Mobile Robots (ECMR), Lincoln, UK.","DOI":"10.1109\/ECMR.2015.7324218"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Langbein, J., Stelzer, R., and Fr\u00fchwirth, T. (2011, January 16\u201320). A rule-based approach to long-term routing for autonomous sailboats. Proceedings of the Robotic sailing, L\u00fcbeck, Germany.","DOI":"10.1007\/978-3-642-22836-0_14"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Cabrera-G\u00e1mez, J., Isern-Gonz\u00e1lez, J., Hern\u00e1ndez-Sosa, D., Dom\u00ednguez-Brito, A.C., and Fern\u00e1ndez-Perdomo, E. (2012, January 17\u201321). Optimization-based weather routing for sailboats. Proceedings of the Robotic Sailing 2012, Cardiff, UK.","DOI":"10.1007\/978-3-642-33084-1_3"},{"key":"ref_26","unstructured":"Du, M., Kang, M., Hou, C., and Xu, J. (September, January 28). Study of Long-term Route Planning for Autonomous Sailboat. Proceedings of the International Robotic Sailing Conference 2018, Southampton, UK."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"394","DOI":"10.1016\/j.ijleo.2018.09.104","article-title":"Monitoring trajectory optimization for unmanned surface vessel in sailboat race","volume":"176","author":"Wang","year":"2019","journal-title":"Optik"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1016\/j.rcim.2010.06.019","article-title":"Reinforcement based mobile robot navigation in dynamic environment","volume":"27","author":"Jaradat","year":"2011","journal-title":"Rob. Comput. Integr. Manuf."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"4","DOI":"10.5772\/10528","article-title":"Mobile robot navigation based on Q-Learning technique","volume":"8","author":"Khriji","year":"2011","journal-title":"Int. J. Adv. Rob. Syst."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1007\/s13748-018-00168-6","article-title":"Path planning of a mobile robot in a free-space environment using Q-Learning","volume":"8","author":"Jiang","year":"2019","journal-title":"Prog. Artif. Intell."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Cui, J.H., Wei, R.X., Liu, Z.C., and Zhou, K. (2018). UAV motion strategies in uncertain dynamic environments: A path planning method based on Q-Learning strategy. Appl. Sci., 8.","DOI":"10.3390\/app8112169"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"391","DOI":"10.1007\/s10846-013-9901-z","article-title":"Geometric reinforcement learning for path planning of UAVs","volume":"77","author":"Zhang","year":"2015","journal-title":"J. Intell. Rob. Syst."},{"key":"ref_33","unstructured":"Bradski, G. (2020, March 08). The OpenCV Library. Available online: https:\/\/www.drdobbs.com\/open-source\/the-opencv-library\/184404319."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Paravisi, M., H Santos, D., Jorge, V., Heck, G., Gon\u00e7alves, L.M., and Amory, A. (2019). Unmanned Surface Vehicle Simulator with Realistic Environmental Disturbances. Sensors, 19.","DOI":"10.3390\/s19051068"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Stateczny, A., Kazimierski, W., Burdziakowski, P., Motyl, W., and Wisniewska, M. (2019). Shore Construction Detection by Automotive Radar for the Needs of Autonomous Surface Vehicle Navigation. ISPRS Int. J. Geo-Inf., 8.","DOI":"10.3390\/ijgi8020080"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"30","DOI":"10.2478\/pomr-2019-0004","article-title":"Universal Autonomous Control and Management System for Multipurpose Unmanned Surface Vessel","volume":"26","author":"Burdziakowski","year":"2019","journal-title":"Pol. Marit. Res."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1","DOI":"10.5565\/rev\/elcvia.1084","article-title":"Depth Data Error Modeling of the ZED 3D Vision Sensor from Stereolabs","volume":"17","author":"Ortiz","year":"2018","journal-title":"ELCVIA: Electron. Lett. Comput. Vision Image Anal."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Cabrera, E.V., Ortiz, L.E., Silva, B.M.F.d., Clua, E.W.G., and Gon\u00e7alves, L.M.G. (2018). A Versatile Method for Depth Data Error Estimation in RGB-D Sensors. Sensors, 18.","DOI":"10.3390\/s18093122"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1016\/j.neucom.2012.10.033","article-title":"Visual attention guided features selection with foveated images","volume":"120","author":"Gomes","year":"2013","journal-title":"Neurocomputing"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Oliveira, F., Souza, A., Fernandes, M., Gomes, R., and Goncalves, L. (2018). Efficient 3D objects recognition using multifoveated point clouds. Sensors, 18.","DOI":"10.3390\/s18072302"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/6\/1550\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:05:59Z","timestamp":1760173559000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/6\/1550"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,3,11]]},"references-count":40,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2020,3]]}},"alternative-id":["s20061550"],"URL":"https:\/\/doi.org\/10.3390\/s20061550","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,3,11]]}}}