{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,4]],"date-time":"2026-05-04T11:21:18Z","timestamp":1777893678756,"version":"3.51.4"},"reference-count":33,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,7,6]],"date-time":"2023-07-06T00:00:00Z","timestamp":1688601600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100017130","name":"Ji Hua Laboratory","doi-asserted-by":"publisher","award":["X190021TB190"],"award-info":[{"award-number":["X190021TB190"]}],"id":[{"id":"10.13039\/100017130","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003399","name":"Science and Technology Commission of Shanghai Municipality","doi-asserted-by":"publisher","award":["19511132000"],"award-info":[{"award-number":["19511132000"]}],"id":[{"id":"10.13039\/501100003399","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurorobot."],"abstract":"<jats:p>This paper proposes a self-learning Monte Carlo tree search algorithm (SL-MCTS), which has the ability to continuously improve its problem-solving ability in single-player scenarios. SL-MCTS combines the MCTS algorithm with a two-branch neural network (PV-Network). The MCTS architecture can balance the search for exploration and exploitation. PV-Network replaces the rollout process of MCTS and predicts the promising search direction and the value of nodes, which increases the MCTS convergence speed and search efficiency. The paper proposes an effective method to assess the trajectory of the current model during the self-learning process by comparing the performance of the current model with that of its best-performing historical model. Additionally, this method can encourage SL-MCTS to generate optimal solutions during the self-learning process. We evaluate the performance of SL-MCTS on the robot path planning scenario. The experimental results show that the performance of SL-MCTS is far superior to the traditional MCTS and single-player MCTS algorithms in terms of path quality and time consumption, especially its time consumption is half less than that of the traditional MCTS algorithms. SL-MCTS also performs comparably to other iterative-based search algorithms designed specifically for path planning tasks.<\/jats:p>","DOI":"10.3389\/fnbot.2023.1039644","type":"journal-article","created":{"date-parts":[[2023,7,6]],"date-time":"2023-07-06T13:53:31Z","timestamp":1688651611000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":16,"title":["A self-learning Monte Carlo tree search algorithm for robot path planning"],"prefix":"10.3389","volume":"17","author":[{"given":"Wei","family":"Li","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yi","family":"Liu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yan","family":"Ma","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kang","family":"Xu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jiang","family":"Qiu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhongxue","family":"Gan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2023,7,6]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"270","DOI":"10.1016\/j.comcom.2019.10.014","article-title":"Path planning techniques for unmanned aerial vehicles: a review, solutions, and challenges","volume":"149","author":"Aggarwal","year":"2020","journal-title":"Comput. Commun"},{"key":"B2","doi-asserted-by":"publisher","first-page":"235","DOI":"10.1023\/A:1013689704352","article-title":"Finite-time analysis of the multiarmed bandit problem","volume":"47","author":"Auer","year":"2002","journal-title":"Mach. Learn"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.3389\/fnbot.2021.770361","article-title":"An improved pso-gwo algorithm with chaos and adaptive inertial weight for robot path planning","author":"Cheng","year":"2021","journal-title":"Front. Neurorobot"},{"key":"B4","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1007\/978-3-540-87608-3_11","article-title":"\u201cWhole-history rating: a Bayesian rating system for players of time-varying strength,\u201d","volume-title":"International Conference on Computers and Games","author":"Coulom","year":"2008"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2021.116224","article-title":"An analysis of single-player Monte Carlo tree search performance in sokoban","author":"Crippa","year":"2022","journal-title":"Expert Syst. Appl"},{"key":"B6","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3389\/fnbot.2019.00015","article-title":"Mobile robot path planning based on ant colony algorithm with a* heuristic method","volume":"13","author":"Dai","year":"2019","journal-title":"Front. Neurorobot"},{"key":"B7","doi-asserted-by":"publisher","first-page":"11213","DOI":"10.1109\/LRA.2022.3199674","article-title":"Monte-carlo robot path planning","volume":"7","author":"Dam","year":"2022","journal-title":"IEEE Robot. Autom. Lett"},{"key":"B8","doi-asserted-by":"publisher","first-page":"28","DOI":"10.1109\/MCI.2006.329691","article-title":"Ant colony optimization","volume":"1","author":"Dorigo","year":"2006","journal-title":"IEEE Comput. Intell. Mag"},{"key":"B9","doi-asserted-by":"publisher","first-page":"106","DOI":"10.1145\/2093548.2093574","article-title":"The grand challenge of computer go: Monte Carlo tree search and extensions","volume":"55","author":"Gelly","year":"2012","journal-title":"Commun. ACM"},{"key":"B10","article-title":"\u201cExploration exploitation in go: UCT for Monte-Carlo go,\u201d","volume-title":"NIPS: Neural Information Processing Systems Conference On-line trading of Exploration and Exploitation Workshop","author":"Gelly","year":"2006"},{"key":"B11","doi-asserted-by":"crossref","first-page":"209","DOI":"10.1007\/978-3-030-70281-6_12","article-title":"\u201cParticle swarm optimization in global path planning for swarm of robots,\u201d","volume-title":"Applying Particle Swarm Optimization","author":"Halder","year":"2021"},{"key":"B12","doi-asserted-by":"publisher","first-page":"4610","DOI":"10.1007\/s10489-021-02586-x","article-title":"Monte Carlo tree search for dynamic bike repositioning in bike-sharing systems","volume":"52","author":"Huang","year":"2022","journal-title":"Appl. Intell"},{"key":"B13","first-page":"282","article-title":"\u201cBandit based monte-carlo planning,\u201d","volume-title":"European Conference on Machine Learning","author":"Kocsis","year":"2006"},{"key":"B14","doi-asserted-by":"publisher","first-page":"1221","DOI":"10.1080\/09540091.2022.2052265","article-title":"An improved Monte Carlo tree search approach to workflow scheduling","volume":"34","author":"Kung","year":"2022","journal-title":"Conn. Sci"},{"key":"B15","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.ins.2015.11.004","article-title":"An effective initialization method for genetic algorithm-based robot path planning using a directed acyclic graph","volume":"332","author":"Lee","year":"2016","journal-title":"Inf. Sci"},{"key":"B16","doi-asserted-by":"publisher","first-page":"11272","DOI":"10.1609\/aaai.v35i13.17344","article-title":"Lifelong multi-agent path finding in large-scale warehouses","volume":"35","author":"Li","year":"2021","journal-title":"Proc. AAAI Conf. Artif. Intell"},{"key":"B17","unstructured":"LiuY.\n          Code of SL-MCTS2023"},{"key":"B18","doi-asserted-by":"publisher","first-page":"1555","DOI":"10.1007\/s00521-019-04172-2","article-title":"Research on path planning of mobile robot based on improved ant colony algorithm","volume":"32","author":"Luo","year":"2020","journal-title":"Neural Comput. Appl"},{"key":"B19","doi-asserted-by":"publisher","first-page":"221","DOI":"10.3390\/e22020221","article-title":"On a generalization of the jensen-shannon divergence and the jensen-shannon centroid","volume":"22","author":"Nielsen","year":"2020","journal-title":"Entropy"},{"key":"B20","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/978-3-642-12384-9_26","article-title":"\u201cAn UCT approach for anytime agent-based planning,\u201d","volume-title":"Advances in Practical Applications of Agents and Multiagent Systems: 8th International Conference on Practical Applications of Agents and Multiagent Systems (PAAMS 2010)","author":"Pellier","year":"2010"},{"key":"B21","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1007\/978-3-642-29178-4_26","article-title":"\u201cMonte-carlo tree search for the physical travelling salesman problem,\u201d","volume-title":"European Conference on the Applications of Evolutionary Computation","author":"Perez","year":"2012"},{"key":"B22","doi-asserted-by":"publisher","first-page":"12721","DOI":"10.1007\/s00521-021-05918-7","article-title":"Collective intelligence evolution using ant colony optimization and neural networks","volume":"33","author":"Qi","year":"2021","journal-title":"Neural Comput. Appl"},{"key":"B23","doi-asserted-by":"publisher","first-page":"101","DOI":"10.15302\/J-SSCAE-2018.04.017","article-title":"Theory of collective intelligence evolution and its applications in intelligent robots","volume":"20","author":"Qi","year":"2018","journal-title":"Strateg. Study Chin. Acad. Eng"},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.1016\/j.lfs.2022.120872","article-title":"Monocarboxylate transporters (MCTS) in skeletal muscle and hypothalamus of less or more physically active mice exposed to aerobic training","author":"Scariot","year":"2022","journal-title":"Life Sci. 307"},{"key":"B25","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1016\/j.knosys.2011.08.008","article-title":"Single-player monte-carlo tree search for samegame","volume":"34","author":"Schadd","year":"2012","journal-title":"Knowl. Based Syst"},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1707.06347","article-title":"Proximal policy optimization algorithms","author":"Schulman","year":"2017","journal-title":"arXiv"},{"key":"B27","doi-asserted-by":"publisher","first-page":"484","DOI":"10.1038\/nature16961","article-title":"Mastering the game of go with deep neural networks and tree search","volume":"529","author":"Silver","year":"2016","journal-title":"Nature"},{"key":"B28","doi-asserted-by":"publisher","first-page":"354","DOI":"10.1038\/nature24270","article-title":"Mastering the game of go without human knowledge","volume":"550","author":"Silver","year":"2017","journal-title":"Nature"},{"key":"B29","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1007\/978-3-540-87608-3_4","article-title":"\u201cAn analysis of UCT in multi-player games,\u201d","volume-title":"Computers and Games","author":"Sturtevant","year":"2008"},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.3389\/fnbot.2021.642733","article-title":"Mobile robot path planning based on time taboo ant colony optimization in dynamic environment","author":"Xiong","year":"2021","journal-title":"Front. Neurorobot"},{"key":"B31","unstructured":"\u201cPath planning using neural a* search,\u201d\n            YonetaniR.\n            TaniaiT.\n            BarekatainM.\n            NishimuraM.\n            KanezakiA.\n          Sydney, NSWInternational Conference on Machine Learning2021"},{"key":"B32","doi-asserted-by":"publisher","first-page":"22547","DOI":"10.1109\/JIOT.2022.3182798","article-title":"A novel hybrid particle swarm optimization algorithm for path planning of UAVS","volume":"9","author":"Yu","year":"2022","journal-title":"IEEE Internet Things J"},{"key":"B33","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1016\/j.neucom.2019.05.075","article-title":"A novel learning-based global path planning algorithm for planetary rovers","volume":"361","author":"Zhang","year":"2019","journal-title":"Neurocomputing"}],"container-title":["Frontiers in Neurorobotics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1039644\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,7,6]],"date-time":"2023-07-06T13:53:43Z","timestamp":1688651623000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1039644\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,6]]},"references-count":33,"alternative-id":["10.3389\/fnbot.2023.1039644"],"URL":"https:\/\/doi.org\/10.3389\/fnbot.2023.1039644","relation":{},"ISSN":["1662-5218"],"issn-type":[{"value":"1662-5218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,6]]},"article-number":"1039644"}}