{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,27]],"date-time":"2026-06-27T13:13:23Z","timestamp":1782566003830,"version":"3.54.5"},"reference-count":35,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2021,1,27]],"date-time":"2021-01-27T00:00:00Z","timestamp":1611705600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Priority research &amp; design program of Chongqing technology innovation and application demonstration","award":["No. cstc2017zdcy-zdyfX0036"],"award-info":[{"award-number":["No. cstc2017zdcy-zdyfX0036"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This paper presents a novel decentralized multi-robot collision avoidance method with deep reinforcement learning, which is not only suitable for the large-scale grid map workspace multi-robot system, but also directly processes Lidar signals instead of communicating between the robots. According to the particularity of the workspace, we handcrafted a reward function, which considers both the collision avoidance among the robots and as little as possible change of direction of the robots during driving. Using Double Deep Q-Network (DDQN), the policy was trained in the simulation grid map workspace. By designing experiments, we demonstrated that the learned policy can guide the robot well to effectively travel from the initial position to the goal position in the grid map workspace and to avoid collisions with others while driving.<\/jats:p>","DOI":"10.3390\/s21030841","type":"journal-article","created":{"date-parts":[[2021,1,27]],"date-time":"2021-01-27T12:20:26Z","timestamp":1611750026000},"page":"841","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["Non-Communication Decentralized Multi-Robot Collision Avoidance in Grid Map Workspace with Double Deep Q-Network"],"prefix":"10.3390","volume":"21","author":[{"given":"Lin","family":"Chen","sequence":"first","affiliation":[{"name":"Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Sciences, Chongqing 400700, China"},{"name":"School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing 100049, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yongting","family":"Zhao","sequence":"additional","affiliation":[{"name":"Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Sciences, Chongqing 400700, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Huanjun","family":"Zhao","sequence":"additional","affiliation":[{"name":"Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Sciences, Chongqing 400700, China"},{"name":"School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing 100049, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bin","family":"Zheng","sequence":"additional","affiliation":[{"name":"Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Sciences, Chongqing 400700, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,1,27]]},"reference":[{"key":"ref_1","unstructured":"Chen, Y.F., Liu, M., Everett, M., and How, J.P. (June, January 29). Decentralized non-communicating multiagent collision avoidance with deep reinforcement learning. Proceedings of the International Conference on Robotics and Automation, Singapore."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Chen, Y.F., Everett, M., Liu, M., and How, J.P. (2017). Socially aware motion planning with deep reinforcement learning. arXiv.","DOI":"10.1109\/IROS.2017.8202312"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"3221","DOI":"10.1109\/LRA.2020.2974695","article-title":"Multi-Agent Motion Planning for Dense and Dynamic Environments via Deep Reinforcement Learning","volume":"5","author":"Semnani","year":"2020","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Long, P., Fan, T., Liao, X., Liu, W., Zhang, H., and Pan, J. (2018, January 21\u201325). Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.","DOI":"10.1109\/ICRA.2018.8461113"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Van den Berg, J., Lin, M., and Manocha, D. (2008, January 19\u201323). Reciprocal velocity obstacles for real-time multi-agent navigation. Proceedings of the 2008 IEEE International Conference on Robotics and Automation (ICRA), Pasadena, CA, USA.","DOI":"10.1109\/ROBOT.2008.4543489"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Alonso-Mora, J., Breitenmoser, A., Rufli, M., Beardsley, P., and Siegwart, R. (2013). Optimal reciprocal collision avoidance for multiple non-holonomic robots. Distributed Autonomous Robotic Systems, Springer.","DOI":"10.1007\/978-3-642-32723-0_15"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Chen, G., Yao, S., Ma, J., Pan, L., Chen, Y., Xu, P., Ji, J., and Chen, X. (2020). Distributed Non-Communicating Multi-Robot Collision Avoidance via Map-Based Deep Reinforcement Learning. Sensors, 20.","DOI":"10.3390\/s20174836"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Everett, M., Chen, Y.F., and How, J.P. (2018, January 1\u20135). Motion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learning. Proceedings of the 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8593871"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1163","DOI":"10.1109\/TRO.2016.2593448","article-title":"Optimal Multirobot Path Planning on Graphs: Complete Algorithms and Effective Heuristics","volume":"32","author":"Yu","year":"2016","journal-title":"IEEE Trans. Robot."},{"key":"ref_10","unstructured":"Semnani, S.H., de Ruiter, A., and Liu, H. (2019). Force-based algorithm for motion planning of large agent teams. arXiv."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"266","DOI":"10.1177\/0278364919859627","article-title":"The Critical Radius in Sampling-Based Motion Planning","volume":"39","author":"Kiril","year":"2020","journal-title":"Int. J. Robot. Res."},{"key":"ref_12","unstructured":"Luna, R.J., and Bekris, K.E. (2011, January 16\u201322). Push and swap: Fast cooperative path-finding with completeness guarantees. Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence, Barcelona, Spain."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1750","DOI":"10.1177\/0278364916672311","article-title":"On the hardness of unlabeled multi-robot motion planning","volume":"35","author":"Solovey","year":"2016","journal-title":"Int. J. Robot. Res."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Mellinger, D., Kushleyev, A., and Kumar, V. (2012, January 14\u201318). Mixed-integer quadratic program trajectory generation for heterogeneous quadrotor teams. Proceedings of the 2012 IEEE International Conference on Robotics and Automation (ICRA), Saint Paul, MN, USA.","DOI":"10.1109\/ICRA.2012.6225009"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Hoy, M., Matveev, A.S., and Savkin, A.V. (2015). Algorithms for collision-free navigation of mobile robots in complex cluttered environments: A survey. Robotica.","DOI":"10.1016\/B978-0-12-803730-0.00003-2"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"582","DOI":"10.1016\/j.dt.2019.04.011","article-title":"A review: On path planning strategies for navigation of mobile robot","volume":"15","author":"Patle","year":"2019","journal-title":"Def. Technol."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1062","DOI":"10.1177\/0278364917741532","article-title":"Hold or take optimal plan (hoop): A quadratic programming approach to multi-robot trajectory generation","volume":"37","author":"Tang","year":"2018","journal-title":"Int. J. Robot. Res."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Augugliaro, F., Schoellig, A.P., and D\u2019Andrea, R. (2012, January 7\u201312). Generation of collision-free trajectories for a quadrocopter fleet: A sequential convex programming approach. Proceedings of the 2012 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Vilamoura, Portugal.","DOI":"10.1109\/IROS.2012.6385823"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Preiss, J.A., H\u00f6nig, W., Ayanian, N., and Sukhatme, G.S. (2017, January 24\u201328). Downwash-aware trajectory planning for large quadrotor teams. Proceedings of the 2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada.","DOI":"10.1109\/IROS.2017.8202165"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Van den Berg, J., Guy, S.J., Lin, M., and Manocha, D. (2011). Reciprocal n-body collision avoidance. Robotics Research, Springer.","DOI":"10.1007\/978-3-642-19457-3_1"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Van den Berg, J., Snape, J., Guy, S.J., and Manocha, D. (2011, January 13). Reciprocal collision avoidance with acceleration-velocity obstacles. Proceedings of the 2011 IEEE International Conference on Robotics and Automation, Shanghai, China.","DOI":"10.1109\/ICRA.2011.5980408"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"347","DOI":"10.1109\/TIE.2013.2245612","article-title":"A Decentralized Cooperative Control Scheme with Obstacle Avoidance for a Team of Mobile Robots","volume":"61","author":"Rezaee","year":"2014","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1047","DOI":"10.1109\/LRA.2017.2656241","article-title":"Fast, On-line Collision Avoidance for Dynamic Vehicles Using Buffered Voronoi Cells","volume":"2","author":"Zhou","year":"2017","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_24","unstructured":"Standley, T. (2010, January 11\u201315). Finding Optimal Solutions to Cooperative Pathfinding Problems. Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2010, Atlanta, GA, USA."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1613\/jair.4171","article-title":"Enhanced Partial Expansion A*","volume":"50","author":"Goldenberg","year":"2014","journal-title":"J. Artif. Intell. Res."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"470","DOI":"10.1016\/j.artint.2012.11.006","article-title":"The increasing cost tree search for optimal multi-agent pathfinding","volume":"195","author":"Sharon","year":"2013","journal-title":"Artif. Intell."},{"key":"ref_27","unstructured":"Barer, M., Sharon, G., Stern, R., and Felner, A. (2014, January 18\u201322). Suboptimal variants of the conflict-based search algorithm for the multi-agent pathfinding problem. Proceedings of the ECAI 201\u201421st European Conference on Artificial Intelligence, Prague, Czech Republic."},{"key":"ref_28","unstructured":"Boyarski, E., Felner, A., Stern, R., Sharon, G., Betzalel, O., Tolpin, D., and Shimony, E. (2015, January 11\u201313). ICBS: Improved Conflict-Based Search Algorithm for Multi-Agent Pathfinding. Proceedings of the Eighth Annual Symposium on Combinatorial Search, Ein Gedi, Israel."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Snape, J., van den Berg, J., Guy, S.J., and Manocha, D. (2009, January 11\u201315). Independent Navigation of Multiple Mobile Robots with Hybrid Reciprocal Velocity Obstacles. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems, St. Louis, MO, USA.","DOI":"10.1109\/IROS.2009.5354821"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"696","DOI":"10.1109\/TRO.2011.2120810","article-title":"The hybrid reciprocal velocity obstacle","volume":"27","author":"Snape","year":"2011","journal-title":"IEEE Trans. Robot."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1274","DOI":"10.1177\/0278364908097581","article-title":"Planning time-minimal safe paths amidst unpredictably moving obstacles","volume":"27","author":"Overmars","year":"2008","journal-title":"Int. J. Robot. Res."},{"key":"ref_32","unstructured":"Nair, V., and Hinton, G.E. (2010, January 21\u201324). Rectified linear units improve restricted boltzmann machines. Proceedings of the International Conference on Machine Learning, Haifa, Israel."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Van Hasselt, H., Guez, A., and Silver, D. (2015). Deep Reinforcement Learning with Double Q-learning. arXiv.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"ref_34","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3\u20136). ImageNet Classification with Deep Convolutional Neural Networks. Proceedings of the International Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Fan, T., Long, P., Liu, W., and Pan, J. (2020). Distributed multi-robot collision avoidance via deep reinforcement learning for navigation in complex scenarios. Int. J. Robot. Res.","DOI":"10.1177\/0278364920916531"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/3\/841\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:16:04Z","timestamp":1760159764000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/3\/841"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,1,27]]},"references-count":35,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2021,2]]}},"alternative-id":["s21030841"],"URL":"https:\/\/doi.org\/10.3390\/s21030841","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,1,27]]}}}