{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,14]],"date-time":"2026-07-14T02:16:44Z","timestamp":1783995404787,"version":"3.55.0"},"reference-count":106,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2021,11,25]],"date-time":"2021-11-25T00:00:00Z","timestamp":1637798400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,11,25]],"date-time":"2021-11-25T00:00:00Z","timestamp":1637798400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"University of Eastern Finland (UEF) including Kuopio University Hospital"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Intell Manuf"],"published-print":{"date-parts":[[2022,2]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Principles of typical motion planning algorithms are investigated and analyzed in this paper. These algorithms include traditional planning algorithms, classical machine learning algorithms, optimal value reinforcement learning, and policy gradient reinforcement learning. Traditional planning algorithms investigated include<jats:italic>graph search algorithms<\/jats:italic>,<jats:italic>sampling-based algorithms<\/jats:italic>,<jats:italic>interpolating curve algorithms<\/jats:italic>, and<jats:italic>reaction-based algorithms<\/jats:italic>. Classical machine learning algorithms include<jats:italic>multiclass support vector machine<\/jats:italic>,<jats:italic>long short-term memory<\/jats:italic>,<jats:italic>Monte-Carlo tree search<\/jats:italic>and<jats:italic>convolutional neural network<\/jats:italic>. Optimal value reinforcement learning algorithms include<jats:italic>Q learning<\/jats:italic>,<jats:italic>deep Q-learning network<\/jats:italic>,<jats:italic>double deep Q-learning network<\/jats:italic>,<jats:italic>dueling deep Q-learning network<\/jats:italic>. Policy gradient algorithms include<jats:italic>policy gradient method<\/jats:italic>,<jats:italic>actor-critic algorithm<\/jats:italic>,<jats:italic>asynchronous advantage actor-critic<\/jats:italic>,<jats:italic>advantage actor-critic<\/jats:italic>,<jats:italic>deterministic policy gradient<\/jats:italic>,<jats:italic>deep deterministic policy gradient<\/jats:italic>,<jats:italic>trust region policy optimization<\/jats:italic>and<jats:italic>proximal policy optimization<\/jats:italic>. New general criteria are also introduced to evaluate the performance and application of motion planning algorithms by analytical comparisons. The convergence speed and stability of optimal value and policy gradient algorithms are specially analyzed. Future directions are presented analytically according to principles and analytical comparisons of motion planning algorithms. This paper provides researchers with a clear and comprehensive understanding about advantages, disadvantages, relationships, and future of motion planning algorithms in robots, and paves ways for better motion planning algorithms in academia, engineering, and manufacturing.<\/jats:p>","DOI":"10.1007\/s10845-021-01867-z","type":"journal-article","created":{"date-parts":[[2021,11,25]],"date-time":"2021-11-25T16:05:09Z","timestamp":1637856309000},"page":"387-424","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":239,"title":["A review of motion planning algorithms for intelligent robots"],"prefix":"10.1007","volume":"33","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8297-5949","authenticated-orcid":false,"given":"Chengmin","family":"Zhou","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bingding","family":"Huang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9554-2827","authenticated-orcid":false,"given":"Pasi","family":"Fr\u00e4nti","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,11,25]]},"reference":[{"key":"1867_CR1","unstructured":"Arkin, R. C., Riseman, E. M., & Hansen, A. (1887). AuRA: an architecture for vision-based robot navigation. Proceedings of the DARPA Image Understanding Workshop, Los Angeles, CA, February 1987, pp. 417\u2013413."},{"key":"1867_CR2","unstructured":"Babaeizadeh, M., Frosio, I., Tyree, S., Clemons, J., Kautz J. (2016). Reinforcement learning through asynchronous advantage Actor-Critic on a GPU. arXiv, arXiv:1611.06256 [cs.LG]."},{"key":"1867_CR3","doi-asserted-by":"publisher","first-page":"3057","DOI":"10.3390\/app9153057","volume":"9","author":"H Bae","year":"2019","unstructured":"Bae, H., Kim, G., Kim, J., Qian, D., & Lee, S. (2019). Multi-robot path planning method using reinforcement learning. Applied Science., 9, 3057.","journal-title":"Applied Science."},{"key":"1867_CR4","doi-asserted-by":"crossref","unstructured":"Bai, H., Cai, S., Ye, N., Hsu, D., & Lee, W. S. (2015). Intention-aware online POMDP planning for autonomous driving in a crowd. 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, pp. 454\u2013460.","DOI":"10.1109\/ICRA.2015.7139219"},{"issue":"4","key":"1867_CR5","first-page":"1","volume":"17","author":"GD Bautista","year":"2015","unstructured":"Bautista, G. D., Perez, J., Milan\u00e9s, V., & Nashashibi, F. (2015). A review of motion planning techniques for automated vehicles. IEEE Transactions on Intelligent Transportation Systems, 17(4), 1\u201311.","journal-title":"IEEE Transactions on Intelligent Transportation Systems"},{"key":"1867_CR6","unstructured":"Bautista, D. G., Perez, J., Lattarulo, R. A., Milanes, V., Nashashibi, F. (2014). Continuous curvature planning with obstacle avoidance capabilities in urban scenarios. IEEE International Conference on Intelligent Transportation Systems, pp. 1430\u20131435."},{"key":"1867_CR7","unstructured":"Bellemare, M. G., Dabney, W., R\u00e9mi, M. (2017). A distributional perspective on reinforcement learning. arXiv:1707.06887 [cs.LG]."},{"key":"1867_CR9","unstructured":"Bilbeisi, K. M., & Kesse, M. (2017). Tesla: A successful entrepreneurship strategy. Morrow, GA: Clayton State University."},{"issue":"5","key":"1867_CR10","doi-asserted-by":"publisher","first-page":"1179","DOI":"10.1109\/21.44033","volume":"19","author":"J Borenstein","year":"1989","unstructured":"Borenstein, J., & Koren, Y. (1989). Real-time obstacle avoidance for fast mobile robots. IEEE Transactions on Systems, Man, and Cybernetics, 19(5), 1179\u20131187.","journal-title":"IEEE Transactions on Systems, Man, and Cybernetics"},{"issue":"3","key":"1867_CR11","doi-asserted-by":"publisher","first-page":"278","DOI":"10.1109\/70.88137","volume":"7","author":"J Borenstein","year":"1991","unstructured":"Borenstein, J., & Koren, Y. (1991). The vector field histogram-fast obstacle avoidance for mobile robots. IEEE Transactions on Robotics and Automation, 7(3), 278\u2013288.","journal-title":"IEEE Transactions on Robotics and Automation"},{"key":"1867_CR12","volume-title":"Neurocomputing: Algorithms, architectures and applications","author":"JS Bridle","year":"1990","unstructured":"Bridle, J. S. (1990). Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition. In F. Fogleman Soulie & J. Herault (Eds.), Neurocomputing: Algorithms, architectures and applications. Springer-Verlag."},{"issue":"1","key":"1867_CR13","first-page":"1","volume":"1","author":"RA Brooks","year":"1986","unstructured":"Brooks, R. A. (1986). A robust layered control system for a mobile robot. IEEE Transactions on Robotics and Automation, 1(1), 1\u201310.","journal-title":"IEEE Transactions on Robotics and Automation"},{"issue":"9","key":"1867_CR14","first-page":"2444","volume":"47","author":"M Cai","year":"2017","unstructured":"Cai, M., Lin, Y., Han, B., Liu, C., & Zhang, W. (2017). On a simple and efficient approach to probability distribution function aggregation. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 47(9), 2444\u20132453.","journal-title":"IEEE Transactions on Systems, Man, and Cybernetics: Systems"},{"key":"1867_CR15","unstructured":"Chan, \u041a. C., Lenard, C. T., Mills, T. M. (2012). An introduction to Markov chains. In 49th Annual Conference of Mathematical Association of Victoria, Melbourne, pp 40\u201347."},{"key":"1867_CR16","doi-asserted-by":"publisher","first-page":"297","DOI":"10.1007\/s10846-019-01073-3","volume":"98","author":"Y Chao","year":"2020","unstructured":"Chao, Y., Xiang, X., & Wang, C. (2020). Towards real-time path planning through deep reinforcement learning for UAV in dynamic environment. Journal of Intelligent and Robotic Systems, 98, 297\u2013309.","journal-title":"Journal of Intelligent and Robotic Systems"},{"key":"1867_CR17","doi-asserted-by":"crossref","unstructured":"Chen, Y., Liu, M., Everett, M., & How, J. P. (2016). Decentralized non-communicating multiagent collision avoidance with deep reinforcement learning, arXiv:1609.07845v2 [cs.MA].","DOI":"10.1109\/ICRA.2017.7989037"},{"key":"1867_CR18","doi-asserted-by":"crossref","unstructured":"Chen, C., Liu, Y., Kreiss, S., & Alahi, A. (2019). Crowd-robot interaction: Crowd-aware robot navigation with attention-based deep reinforcement learning. International Conference on Robotics and Automation (ICRA), pp. 6015\u20136022.","DOI":"10.1109\/ICRA.2019.8794134"},{"key":"1867_CR19","doi-asserted-by":"crossref","unstructured":"Chen, C., Hu, S., Nikdel, P., Mori, G., & Savva, M. (2020). Relational graph learning for crowd navigation. arXiv:1909.13165v3 [cs.RO].","DOI":"10.1109\/IROS45743.2020.9340705"},{"key":"1867_CR20","doi-asserted-by":"crossref","unstructured":"Codevilla, F., M\u00fcller, M., L\u00f3pez, A., Koltun, V. & Dosovitskiy, A. (2018). End-to-end driving via conditional imitation learning. IEEE International Conference on Robotics and Automation (ICRA), 2018, pp. 4693\u20134700.","DOI":"10.1109\/ICRA.2018.8460487"},{"key":"1867_CR21","volume-title":"Introduction to algorithms","author":"TH Cormen","year":"2009","unstructured":"Cormen, T. H., Leiserson, C. E., Rivest, R. L., & Stein, C. (2009). Introduction to algorithms. The MIT Press."},{"key":"1867_CR22","unstructured":"Coulom, R. (2006). Efficient selectivity and backup operators in Monte-Carlo tree search. Computers and Games, 5th International Conference, CG 2006, Turin, Italy, May 29\u201331, 2006."},{"issue":"1","key":"1867_CR23","first-page":"533","volume":"39","author":"K Daniel","year":"2014","unstructured":"Daniel, K., Nash, A., Koenig, S., & Felner, A. (2014). Theta*: Any-angle path planning on grids. Journal of Artificial Intelligence Research, 39(1), 533\u2013579.","journal-title":"Journal of Artificial Intelligence Research"},{"key":"1867_CR24","doi-asserted-by":"publisher","first-page":"269","DOI":"10.1007\/BF01386390","volume":"1","author":"EW Dijkstra","year":"1959","unstructured":"Dijkstra, E. W. (1959). A note on two problems in connexion with graphs. Numerische Mathematik, 1, 269\u2013271.","journal-title":"Numerische Mathematik"},{"key":"1867_CR83","doi-asserted-by":"publisher","first-page":"1146","DOI":"10.1016\/j.procs.2017.05.431","volume":"109","author":"A Dos Santos Mignon","year":"2017","unstructured":"Dos Santos Mignon, A., & De Azevedo Da Rocha, R. L. (2017). An adaptive implementation of \u03f5-greedy in reinforcement learning. In Procedia Computer Science, 109, 1146\u20131151.","journal-title":"In Procedia Computer Science"},{"key":"1867_CR25","volume-title":"Springer handbook of robotics. Springer handbooks","author":"H Durrant-Whyte","year":"2008","unstructured":"Durrant-Whyte, H., & Henderson, T. C. (2008). Multi-sensor data fusion. In B. Siciliano & O. Khatib (Eds.), Springer handbook of robotics. Springer handbooks. Springer."},{"key":"1867_CR26","doi-asserted-by":"crossref","unstructured":"Everett, M., Chen, Y., How, J. P. (2018). Motion planning among dynamic, decision-making robots with deep reinforcement learning. 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, pp. 3052\u20133059.","DOI":"10.1109\/IROS.2018.8593871"},{"key":"1867_CR27","doi-asserted-by":"crossref","unstructured":"Evgeniou, T., & Pontil, M. (1999) Support vector machines: Theory and applications. In\u00a0Advanced Course on Artificial Intelligence, Springer, Berlin, Heidelberg, pp. 249\u2013257.","DOI":"10.1007\/3-540-44673-7_12"},{"key":"1867_CR28","unstructured":"Fan, H., Zhu, F., Liu, C., Zhang, L., Zhuang, L., Li, D., Zhu, W., Hu, J., Li, H., & Kong, Q. (2008). Baidu apollo em motion planner. arXiv:1807.08048."},{"issue":"1","key":"1867_CR29","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1007\/BF02519035","volume":"2","author":"RT Farouki","year":"1994","unstructured":"Farouki, R. T., & Sakkalis, T. (1994). Pythagorean-hodograph space curves. Advances in Computational Mathematics, 2(1), 41\u201366.","journal-title":"Advances in Computational Mathematics"},{"issue":"11\u201312","key":"1867_CR30","doi-asserted-by":"publisher","first-page":"939","DOI":"10.1002\/rob.20265","volume":"25","author":"D Ferguson","year":"2008","unstructured":"Ferguson, D., Howard, T. M., & Likhachev, M. (2008). Motion planning in urban environments. Journal of Field Robotics, 25(11\u201312), 939\u2013960.","journal-title":"Journal of Field Robotics"},{"issue":"2","key":"1867_CR31","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1002\/rob.20109","volume":"23","author":"D Ferguson","year":"2006","unstructured":"Ferguson, D., & Stentz, A. (2006). Using interpolation to improve path planning: The field D* algorithm. Journal of Field Robotics, 23(2), 79\u2013101.","journal-title":"Journal of Field Robotics"},{"issue":"7","key":"1867_CR32","doi-asserted-by":"publisher","first-page":"760","DOI":"10.1177\/027836499801700706","volume":"17","author":"P Fiorini","year":"1998","unstructured":"Fiorini, P., & Shiller, Z. (1998). Motion planning in dynamic environments using velocity obstacles. International Journal of Robotics Research, 17(7), 760\u2013772.","journal-title":"International Journal of Robotics Research"},{"key":"1867_CR33","unstructured":"Fortunato, M., Azar, M. G., Piot, B., et al. (2017). Noisy networks for exploration, arXiv:1706.10295 [cs.LG]."},{"issue":"1","key":"1867_CR34","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1109\/100.580977","volume":"4","author":"D Fox","year":"1997","unstructured":"Fox, D., Burgard, W., & Thrun, S. (1997). The dynamic window approach to collision avoidance. IEEE Robotics and Automation Magazine, 4(1), 23\u201333.","journal-title":"IEEE Robotics and Automation Magazine"},{"key":"1867_CR35","first-page":"541","volume":"2012","author":"J Funke","year":"2012","unstructured":"Funke, J., Theodosis, P., Hindiyeh, R., et al. (2012). Up to the limits: Autonomous Audi TTS. IEEE Intelligent Vehicles Symposium (IV). Alcala De Henares, 2012, 541\u2013547.","journal-title":"Alcala De Henares"},{"key":"1867_CR36","unstructured":"G., Samuel, G. (2017). Google sibling waymo launches fully autonomous ride-hailing service. The Guardian 7."},{"key":"1867_CR37","unstructured":"Gao, W., Hus, D., Lee, W. S., Shen, S., Subramanian, K. (2017). Intention-net: integrating planning and deep learning for goal-directed autonomous navigation. arXiv, arXiv:1710.05627 [cs.AI]."},{"key":"1867_CR38","unstructured":"Gilhyun, R. (2018). Applying asynchronous deep classification networks and gaming reinforcement learning-based motion planners to mobile robots. IEEE Robotics and Automation Society, pp. 6268\u20136275."},{"key":"1867_CR39","doi-asserted-by":"crossref","unstructured":"Guy, S. J., Chhugani, J., Kim, C., Satish, N., Lin, M., Manocha, D., & Dubey P. (2009). Clearpath: Highly parallel collision avoidance for multi-agent simulation. Proceedings of the 2009 ACM SIGGRAPH\/Eurographics Symposium on Computer Animation, pp. 177\u2013187.","DOI":"10.1145\/1599470.1599494"},{"issue":"2","key":"1867_CR40","doi-asserted-by":"publisher","first-page":"100","DOI":"10.1109\/TSSC.1968.300136","volume":"4","author":"PE Hart","year":"1968","unstructured":"Hart, P. E., Nilsson, N. J., & Raphael, B. (1968). A formal basis for the heuristic determination of minimum cost paths. IEEE Transactions on Systems Science and Cybernetics, 4(2), 100\u2013107.","journal-title":"IEEE Transactions on Systems Science and Cybernetics"},{"key":"1867_CR41","doi-asserted-by":"crossref","unstructured":"Hasselt, H. V., Guez, A. & Silver, D. (2016). Deep reinforcement learning with double q-learning. In\u00a0Proceedings of the AAAI conference on artificial intelligence, vol. 30, no. 1.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"1867_CR42","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., Sun, J. (2016). Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, pp. 770\u2013778.","DOI":"10.1109\/CVPR.2016.90"},{"key":"1867_CR43","doi-asserted-by":"crossref","unstructured":"Hessel, M., Modayil, J., Van, H. H., et al. (2017). Rainbow: Combining improvements in deep reinforcement learning. arXiv:1710.02298 [cs.AI].","DOI":"10.1609\/aaai.v32i1.11796"},{"issue":"8","key":"1867_CR44","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","volume":"9","author":"S Hochreiter","year":"1997","unstructured":"Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735\u20131780.","journal-title":"Neural Computation"},{"key":"1867_CR45","doi-asserted-by":"crossref","unstructured":"Indrajaya, M. A., Affandi, A., & Pratomo, I. (2015). Design of geographic information system for tracking and routing using dijkstra algorithm for public transportation. In\u00a02015 1st International Conference on Wireless and Telematics (ICWT), pp. 1\u20134.","DOI":"10.1109\/ICWT.2015.7449240"},{"key":"1867_CR46","volume-title":"Advances in computer communication and computational sciences","author":"M Inoue","year":"2019","unstructured":"Inoue, M., Yamashita, T., & Nishida, T. (2019). Robot path planning by LSTM network under changing environment. In S. Bhatia, S. Tiwari, K. Mishra, & M. Trivedi (Eds.), Advances in computer communication and computational sciences. Springer."},{"key":"1867_CR47","doi-asserted-by":"crossref","unstructured":"Isele, D., Cosgun, A., Subramanian, K., Ffjimura, K. (2017). Navigating occluded intersections with autonomous vehicles using deep reinforcement learning. arXiv, arXiv:1705.01196 [cs.AI].","DOI":"10.1109\/ICRA.2018.8461233"},{"key":"1867_CR48","unstructured":"Jeon, J. H., Cowlagi, R. V., Peter, S. C., et al. (2013). Optimal motion planning with the half-car dynamical model for autonomous high-speed driving. American Control Conference (ACC), pp. 188\u2013193."},{"issue":"2","key":"1867_CR49","doi-asserted-by":"publisher","first-page":"2834","DOI":"10.1109\/JSYST.2020.3027716","volume":"15","author":"C Ji","year":"2020","unstructured":"Ji, C., Lu, X., & Zhang, W. (2020). A biobjective optimization model for expert opinions aggregation and its application in group decision making. IEEE Systems Journal, 15(2), 2834\u20132844.","journal-title":"IEEE Systems Journal"},{"key":"1867_CR50","doi-asserted-by":"crossref","unstructured":"Jorgensen, T., Tama, A. (2019). Harnessing reinforcement learning for neural motion planning. arXiv, arXiv:1906.00214 [cs.RO].","DOI":"10.15607\/RSS.2019.XV.026"},{"key":"1867_CR51","first-page":"267","volume":"2","author":"S Kakade","year":"2002","unstructured":"Kakade, S., & Langford, J. (2002). Approximately optimal approximate reinforcement learning. ICML, 2, 267\u2013274.","journal-title":"ICML"},{"key":"1867_CR52","doi-asserted-by":"crossref","unstructured":"Kalos, M. H., & Whitlock, P. A. (2008). Monte Carlo methods. Wiley-VCH. ISBN 978-3-527-40760-6.","DOI":"10.1002\/9783527626212"},{"issue":"4","key":"1867_CR53","doi-asserted-by":"publisher","first-page":"566","DOI":"10.1109\/70.508439","volume":"12","author":"LE Kavraki","year":"2002","unstructured":"Kavraki, L. E., Svestka, P., Latombe, J. C., & Overmars, M. H. (2002). Probabilistic roadmaps for path planning in high-dimensional configuration spaces. IEEE Transactions on Robotics and Automation, 12(4), 566\u2013580.","journal-title":"IEEE Transactions on Robotics and Automation"},{"issue":"1","key":"1867_CR54","doi-asserted-by":"publisher","first-page":"90","DOI":"10.1177\/027836498600500106","volume":"5","author":"O Khatib","year":"1986","unstructured":"Khatib, O. (1986). Real-time obstacle avoidance for manipulators and mobile robots. International Journal of Robotics Research, 5(1), 90\u201398.","journal-title":"International Journal of Robotics Research"},{"issue":"01","key":"1867_CR55","doi-asserted-by":"publisher","first-page":"8017","DOI":"10.1609\/aaai.v33i01.33018017","volume":"33","author":"B Kim","year":"2019","unstructured":"Kim, B., Kaelbling, L. P., & Lozano-Perez, T. (2019). Adversarial actor-critic method for task and motion planning problems using planning experience. AAAI Conference on Artificial Intelligence (AAAI), 33(01), 8017\u20138024.","journal-title":"AAAI Conference on Artificial Intelligence (AAAI)"},{"issue":"13","key":"1867_CR56","doi-asserted-by":"publisher","first-page":"1604","DOI":"10.1016\/j.patrec.2011.06.023","volume":"32","author":"T Kinnunen","year":"2011","unstructured":"Kinnunen, T., Sidoroff, I., Tuononen, M., & Fr\u00e4nti, P. (2011). Comparison of clustering methods: A case study of text-independent speaker modeling. Pattern Recognition Letters, 32(13), 1604\u20131617.","journal-title":"Pattern Recognition Letters"},{"key":"1867_CR57","unstructured":"Konda, V. R., Tsitsiklis, J. N. (2001). Actor-critic algorithms. Society for Industrial and Applied Mathematics, vol 42."},{"key":"1867_CR58","unstructured":"Krogh, B. (1984). A generalized potential field approach to obstacle avoidance control. Proceedings of SME Conference on Robotics Research: The Next Five Years and Beyond, Bethlehem, PA."},{"issue":"5","key":"1867_CR59","doi-asserted-by":"publisher","first-page":"378","DOI":"10.1177\/02783640122067453","volume":"20","author":"SM LaValle","year":"1999","unstructured":"LaValle, S. M., & Kuffner, J. J. (1999). Randomized kinodynamic planning. The International Journal of Robotics Research, 20(5), 378\u2013400.","journal-title":"The International Journal of Robotics Research"},{"issue":"11","key":"1867_CR60","doi-asserted-by":"publisher","first-page":"2278","DOI":"10.1109\/5.726791","volume":"86","author":"Y Lecun","year":"1998","unstructured":"Lecun, Y., Bottou, L., Bengio, Y., & Patrick, H. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278\u20132324.","journal-title":"Proceedings of the IEEE"},{"key":"1867_CR61","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1155\/2018\/5781591","volume":"2018","author":"X Lei","year":"2018","unstructured":"Lei, X., Zhang, Z., & Dong, P. (2018). Dynamic path planning of unknown environment based on deep reinforcement learning. Journal of Robotics, 2018, 1\u201310.","journal-title":"Journal of Robotics"},{"issue":"14","key":"1867_CR62","doi-asserted-by":"publisher","first-page":"1613","DOI":"10.1016\/j.artint.2007.11.009","volume":"172","author":"M Likhachev","year":"2008","unstructured":"Likhachev, M., Ferguson, D., Gordon, G., Stentz, A., & Thrun, S. (2008). Anytime search in dynamic graphs. Artificial Intelligence, 172(14), 1613\u20131643.","journal-title":"Artificial Intelligence"},{"key":"1867_CR63","unstructured":"Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., Wierstra, D. (2019). Continuous control with deep reinforcement learning. arXiv:1509.02971 [cs.LG]."},{"key":"1867_CR64","doi-asserted-by":"crossref","unstructured":"Long, P., Fan, T., Liao, X., Liu, W., Zhang, H., & Pan, J. (2018). Towards optimally decentralized multi-robot collision avoidance via deep reinforcement learning. , arXiv:1709.10082v3 [cs.RO].","DOI":"10.1109\/ICRA.2018.8461113"},{"issue":"3","key":"1867_CR65","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3234692","volume":"4","author":"IR Mariescu","year":"2018","unstructured":"Mariescu, I. R., & Franti, P. (2018). Cell Net: Inferring road networks from GPS trajectories. ACM Transactions on Spatial Algorithms and Systems, 4(3), 1\u201322.","journal-title":"ACM Transactions on Spatial Algorithms and Systems"},{"issue":"3","key":"1867_CR66","doi-asserted-by":"publisher","first-page":"372","DOI":"10.1109\/TSMCA.2007.893483","volume":"37","author":"AA Masoud","year":"2007","unstructured":"Masoud, A. A. (2007). Decentralized self-organizing potential field-based control for individually motivated mobile agents in a cluttered environment: A vector-harmonic potential field approach. IEEE Transactions on Systems, Man, and Cybernetics-Part a: Systems and Humans, 37(3), 372\u2013390.","journal-title":"IEEE Transactions on Systems, Man, and Cybernetics-Part a: Systems and Humans"},{"key":"1867_CR67","doi-asserted-by":"crossref","unstructured":"Meyes, R., Tercan, H., Roggendorf, S., Thomas, T., Christian, B., Markus, O., Christian, B., Sabina, J., Tobias, M. (2017). Motion planning for industrial robots using reinforcement learning. In 50th CIRP Conference on Manufacturing Systems, 63:107\u2013112.","DOI":"10.1016\/j.procir.2017.03.095"},{"key":"1867_CR68","first-page":"63","volume":"2","author":"A Meystel","year":"1990","unstructured":"Meystel, A. (1990). Knowledge based nested hierarchical control. Advances in Automation and Robotics, 2, 63\u2013152.","journal-title":"Advances in Automation and Robotics"},{"key":"1867_CR69","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-30301-5_36","volume-title":"Motion planning and obstacle avoidance","author":"J Minguez","year":"2008","unstructured":"Minguez, J., Lamiraux, F., & Laumond, J. P. (2008). Motion planning and obstacle avoidance. Springer International Publishing."},{"key":"1867_CR71","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., et al. (2013). Playing atari with deep reinforcement learning. arXiv, arXiv:1312.5602 [cs.LG]."},{"key":"1867_CR70","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1038\/nature14236","volume":"518","author":"V Mnih","year":"2015","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., et al. (2015). Human-level control through deep reinforcement learning. Nature, 518, 529\u2013533.","journal-title":"Nature"},{"key":"1867_CR72","unstructured":"Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., Kavukcuoglu, K. (2016). Asynchronous methods for deep reinforcement learning. arXiv, arXiv:1602.01783 [cs.LG]."},{"issue":"9","key":"1867_CR73","doi-asserted-by":"publisher","first-page":"569","DOI":"10.1002\/rob.20258","volume":"25","author":"M Montemerlo","year":"2008","unstructured":"Montemerlo, M., Becker, J., Bhat, S., et al. (2008). Junior: The stanford entry in the urban challenge. Journal of Field Robotics, 25(9), 569\u2013597.","journal-title":"Journal of Field Robotics"},{"key":"1867_CR74","unstructured":"Munos, R., Stepleton, T., Harutyunyan, A., Bellemare, M. G. (2016). Safe and efficient off-policy reinforcement learning. arXiv, arXiv:1606.02647 [cs.LG]."},{"key":"1867_CR75","volume-title":"Introduction to AI robotics","author":"RR Murphy","year":"2000","unstructured":"Murphy, R. R. (2000). Introduction to AI robotics. MIT press."},{"key":"1867_CR76","unstructured":"Oh, J., Guo, Y., Singh, S. & Lee, H. (2018). Self-imitation learning. arXiv 2018, arXiv:1806.05635v1 [cs.LG]."},{"key":"1867_CR77","doi-asserted-by":"publisher","first-page":"347","DOI":"10.1016\/j.procs.2018.01.054","volume":"123","author":"AI Panov","year":"2018","unstructured":"Panov, A. I., Yakovlev, K. S., & Suvorov, R. (2018). Grid path planning with deep reinforcement learning: Preliminary results. Procedia Computer Science, 123, 347\u2013353.","journal-title":"Procedia Computer Science"},{"key":"1867_CR78","doi-asserted-by":"crossref","unstructured":"Paxton, C., Raman, V., Hager, G. D., & Kobilarov, M. (2017). Combining neural networks and tree search for task and motion planning in challenging environments. 2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, pp. 6059\u20136066.","DOI":"10.1109\/IROS.2017.8206505"},{"key":"1867_CR79","doi-asserted-by":"crossref","unstructured":"Qureshi, A. H., Simeonov, A., Bency, M. J., Yip, M. C. (2018). Motion planning networks, arXiv:1806.05767 [cs.RO].","DOI":"10.1109\/ICRA.2019.8793889"},{"issue":"2","key":"1867_CR80","doi-asserted-by":"publisher","first-page":"367","DOI":"10.2140\/pjm.1990.145.367","volume":"145","author":"JA Reeds","year":"1990","unstructured":"Reeds, J. A., & Shepp, L. A. (1990). Optimal paths for a car that goes both forward and backward. Pacific Journal of Math, 145(2), 367\u2013393.","journal-title":"Pacific Journal of Math"},{"key":"1867_CR81","volume-title":"On-line Q-learning using connectionist systems","author":"GA Rummery","year":"1994","unstructured":"Rummery, G. A., & Niranjan, M. (1994). On-line Q-learning using connectionist systems. University of Cambridge."},{"key":"1867_CR82","doi-asserted-by":"crossref","unstructured":"Salemi, B., Moll, M., & Shen, W. M. (2006). SUPERBOT: A deployable multi\u2013functional and modular self\u2013reconfigurable robotic system. Proceeding of IEEE\/RSJ International Conference on Intelligent and Robots System, pp. 3636\u20133641.","DOI":"10.1109\/IROS.2006.281719"},{"key":"1867_CR84","unstructured":"Schaul, T., Quan, J., Antonoglou, I., Silver, D. (2016). Prioritized experience replay. International Conference on Learning Representations (ICLR)."},{"key":"1867_CR85","unstructured":"Schulman, J., Levine, S., Moritz, P., Jordan, M. I., & Abbeel, P. (2017). Trust region policy optimization. arXiv:1502.05477v5 [cs.LG]."},{"key":"1867_CR86","unstructured":"Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O. (2017). Proximal policy optimization algorithms. arXiv:1707.06347v2 [cs.LG]."},{"issue":"7587","key":"1867_CR87","doi-asserted-by":"publisher","first-page":"484","DOI":"10.1038\/nature16961","volume":"529","author":"D Silver","year":"2016","unstructured":"Silver, D., Huang, A., Maddison, C. J., et al. (2016). Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587), 484\u2013489.","journal-title":"Nature"},{"key":"1867_CR88","unstructured":"Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., Riedmiller, M. (2014). Deterministic policy gradient algorithms. In Proceedings of the 31st International Conference on International Conference on Machine Learning, vol. 32. pp 387\u2013395."},{"key":"1867_CR89","doi-asserted-by":"crossref","unstructured":"Smart, W. D., Kaelbling, L. P. (2002). Effective reinforcement learning for mobile robots. IEEE International Conference on Robotics and Automation, vol 4.","DOI":"10.1109\/ROBOT.2002.1014237"},{"key":"1867_CR90","doi-asserted-by":"crossref","unstructured":"Stentz, A. (1994). Optimal and efficient path planning for partially-known environments. Robotics and Automation, pp. 203\u2013220.","DOI":"10.1007\/978-1-4615-6325-9_11"},{"key":"1867_CR91","doi-asserted-by":"crossref","unstructured":"Sui, Z., Pu, Z., Yi, J., Tian, X. (2018). Path Planning of multiagent constrained formation through deep reinforcement Learning. 2018 International Joint Conference on Neural Networks (IJCNN).","DOI":"10.1109\/IJCNN.2018.8489066"},{"key":"1867_CR92","volume-title":"Reinforcement learning: An introduction","author":"RS Sutton","year":"1998","unstructured":"Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. MIT Press."},{"key":"1867_CR93","unstructured":"Sutton, R., Mcallester, D. A., Singh, S., Mansour, Y. (1999). Policy gradient methods for reinforcement learning with function approximation. In Proceedings of the 12th International Conference on Neural Information Processing Systems, MIT Press, Cambridge, MA, USA, pp 1057\u20131063."},{"key":"1867_CR94","doi-asserted-by":"crossref","unstructured":"Sutton, R. S., Maei, H. R., Precup, D., et al. (2009). Fast gradient-descent methods for temporal-difference learning with linear function approximation. In 26th International Conference on Machine Learning, Montreal, Canada.","DOI":"10.1145\/1553374.1553501"},{"key":"1867_CR95","unstructured":"Tobaruela, J. A. (2012). Reactive and path-planning methods for mobile robot navigation. PhD dissertation, Universitat de les Illes Balears. http:\/\/hdl.handle.net\/11201\/151880."},{"issue":"1","key":"1867_CR96","first-page":"59","volume":"3","author":"JN Tsitsiklis","year":"2003","unstructured":"Tsitsiklis, J. N. (2003). On the convergence of optimistic policy iteration. Journal of Machine Learning Research, 3(1), 59\u201372.","journal-title":"Journal of Machine Learning Research"},{"key":"1867_CR8","first-page":"1928","volume":"2008","author":"J Van den Berg","year":"2008","unstructured":"Van den Berg, J., Lin, M., & Manocha, D. (2008). Reciprocal velocity obstacles for real-time multi-agent navigation. IEEE International Conference on Robotics and Automation, 2008, 1928\u20131935.","journal-title":"IEEE International Conference on Robotics and Automation"},{"key":"1867_CR97","doi-asserted-by":"crossref","unstructured":"Van Den Berg, J., Guy, S. J., Lin, M., Manocha, D. (2011). Reciprocal n-body collision avoidance. Robotics research, 2011, pp. 3\u201319.","DOI":"10.1007\/978-3-642-19457-3_1"},{"issue":"8","key":"1867_CR98","doi-asserted-by":"publisher","first-page":"1334","DOI":"10.1109\/TPAMI.2005.165","volume":"27","author":"L Wang","year":"2005","unstructured":"Wang, L. (2005). On the euclidean distance of image. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(8), 1334\u20131339.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"1867_CR99","doi-asserted-by":"publisher","first-page":"2885","DOI":"10.1109\/ACCESS.2019.2962058","volume":"8","author":"F Wang","year":"2020","unstructured":"Wang, F., Qian, Z., Yan, Z., Yuan, C., & Zhang, W. (2020). A novel resilient robot: Kinematic analysis and experimentation. IEEE Access, 8, 2885\u20132892.","journal-title":"IEEE Access"},{"key":"1867_CR100","unstructured":"Wang, Z., Freitas, N., Lanctot, M. (2015). Dueling network architectures for deep reinforcement learning. arXiv:1511.06581 [cs.LG]."},{"key":"1867_CR101","unstructured":"Weston, J., Watkins, C. (1998). Multi-class support vector machines. Technical report, Department of computer science, Royal Holloway, university of London, May 20."},{"key":"1867_CR102","doi-asserted-by":"publisher","first-page":"229","DOI":"10.1007\/BF00992696","volume":"8","author":"RJ Williams","year":"1992","unstructured":"Williams, R. J. (1992). Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8, 229\u2013256.","journal-title":"Machine Learning"},{"key":"1867_CR103","unstructured":"Xu, W., Wei, J., Dolan, J. M., Zhao, H., Zha, H. (2012). A real-time motion planner with trajectory optimization for autonomous vehicles. IEEE International Conference on Robotics and Automation, pp. 2061\u20132067."},{"issue":"4","key":"1867_CR104","doi-asserted-by":"publisher","first-page":"424","DOI":"10.1080\/17517575.2019.1597169","volume":"13","author":"WJ Zhang","year":"2019","unstructured":"Zhang, W. J., Wang, J. W., & Lin, Y. (2019). Integrated design and operation management for enterprise systems. Enterprise Information Systems, 13(4), 424\u2013429.","journal-title":"Enterprise Information Systems"},{"key":"1867_CR105","unstructured":"Zhang, J. (2019) Gradient descent based optimization algorithms for deep learning models training, arXiv:1903.03614v1 [cs.LG]."},{"key":"1867_CR106","doi-asserted-by":"crossref","unstructured":"Ziegler, J., Stiller, C. (2009). Spatiotemporal state lattices for fast trajectory planning in dynamic on-road driving scenarios. IEEE\/RSJ International Conference on Intelligent Robots and Systems, pp. 1879\u20131884.","DOI":"10.1109\/IROS.2009.5354448"}],"container-title":["Journal of Intelligent Manufacturing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10845-021-01867-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10845-021-01867-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10845-021-01867-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,13]],"date-time":"2023-11-13T03:42:07Z","timestamp":1699846927000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10845-021-01867-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,25]]},"references-count":106,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2022,2]]}},"alternative-id":["1867"],"URL":"https:\/\/doi.org\/10.1007\/s10845-021-01867-z","relation":{},"ISSN":["0956-5515","1572-8145"],"issn-type":[{"value":"0956-5515","type":"print"},{"value":"1572-8145","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,11,25]]},"assertion":[{"value":"11 February 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 October 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 November 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interests"}}]}}