{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,10]],"date-time":"2026-04-10T16:27:44Z","timestamp":1775838464147,"version":"3.50.1"},"reference-count":27,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,10,12]],"date-time":"2023-10-12T00:00:00Z","timestamp":1697068800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurorobot."],"abstract":"<jats:p>Navigating safely and efficiently in dense crowds remains a challenging problem for mobile robots. The interaction mechanisms involved in collision avoidance require robots to exhibit active and foresighted behaviors while understanding the crowd dynamics. Deep reinforcement learning methods have shown superior performance compared to model-based approaches. However, existing methods lack an intuitive and quantitative safety evaluation for agents, and they may potentially trap agents in local optima during training, hindering their ability to learn optimal strategies. In addition, sparse reward problems further compound these limitations. To address these challenges, we propose SafeCrowdNav, a comprehensive crowd navigation algorithm that emphasizes obstacle avoidance in complex environments. Our approach incorporates a safety evaluation function to quantitatively assess the current safety score and an intrinsic exploration reward to balance exploration and exploitation based on scene constraints. By combining prioritized experience replay and hindsight experience replay techniques, our model effectively learns the optimal navigation policy in crowded environments. Experimental outcomes reveal that our approach enables robots to improve crowd comprehension during navigation, resulting in reduced collision probabilities and shorter navigation times compared to state-of-the-art algorithms. Our code is available at <jats:ext-link>https:\/\/github.com\/Janet-xujing-1216\/SafeCrowdNav<\/jats:ext-link>.<\/jats:p>","DOI":"10.3389\/fnbot.2023.1276519","type":"journal-article","created":{"date-parts":[[2023,10,12]],"date-time":"2023-10-12T09:25:20Z","timestamp":1697102720000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["SafeCrowdNav: safety evaluation of robot crowd navigation in complex scenes"],"prefix":"10.3389","volume":"17","author":[{"given":"Jing","family":"Xu","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wanruo","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jialun","family":"Cai","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hong","family":"Liu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2023,10,12]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"961","DOI":"10.1109\/CVPR.2016.110","article-title":"Social lstm: Human trajectory prediction in crowded spaces","author":"Alahi","year":"2016","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B2","first-page":"30","article-title":"Hindsight experience replay","author":"Andrychowicz","year":"2017","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B3","article-title":"Never give up: Learning directed exploration strategies","author":"Badia","year":"2020","journal-title":"arXiv preprint arXiv:2002.06038"},{"key":"B4","doi-asserted-by":"publisher","first-page":"765","DOI":"10.1587\/transinf.2022DLP0057","article-title":"Spsd: Semantics and deep reinforcement learning based motion planning for supermarket robot","volume":"106","author":"Cai","year":"2023","journal-title":"IEICE Trans. Inf. Syst"},{"key":"B5","doi-asserted-by":"crossref","first-page":"10007","DOI":"10.1109\/IROS45743.2020.9340705","article-title":"Relational graph learning for crowd navigation","volume-title":"2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)","author":"Chen","year":"2020"},{"key":"B6","doi-asserted-by":"crossref","first-page":"6015","DOI":"10.1109\/ICRA.2019.8794134","article-title":"Crowd-robot interaction: Crowd-aware robot navigation with attention-based deep reinforcement learning","volume-title":"2019 International Conference on Robotics and Automation (ICRA)","author":"Chen","year":"2019"},{"key":"B7","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1109\/ICRA.2017.7989037","article-title":"Decentralized non-communicating multiagent collision avoidance with deep reinforcement learning","volume-title":"2017 IEEE International Conference on Robotics and Automation (ICRA)","author":"Chen","year":"2017"},{"key":"B8","doi-asserted-by":"publisher","first-page":"104","DOI":"10.1016\/j.eswa.2016.06.021","article-title":"Neural networks based reinforcement learning for mobile robots obstacle avoidance","volume":"62","author":"Duguleana","year":"2016","journal-title":"Exp. Syst. Applic"},{"key":"B9","doi-asserted-by":"publisher","first-page":"10357","DOI":"10.1109\/ACCESS.2021.3050338","article-title":"Collision avoidance in pedestrian-rich environments with deep reinforcement learning","volume":"9","author":"Everett","year":"2021","journal-title":"IEEE Access"},{"key":"B10","doi-asserted-by":"publisher","first-page":"4282","DOI":"10.1103\/PhysRevE.51.4282","article-title":"Social force model for pedestrian dynamics","volume":"51","author":"Helbing","year":"1995","journal-title":"Phys. Rev. E"},{"key":"B11","doi-asserted-by":"publisher","first-page":"2673","DOI":"10.3390\/s21082673","article-title":"Optimization-based online initialization and calibration of monocular visual-inertial odometry considering spatial-temporal constraints","volume":"21","author":"Huang","year":"2021","journal-title":"Sensors"},{"key":"B12","doi-asserted-by":"crossref","first-page":"6456","DOI":"10.1109\/IROS51168.2021.9636226","article-title":"Arena-rosnav: Towards deployment of deep-reinforcement-learning-based obstacle avoidance into conventional autonomous navigation systems","volume-title":"2021 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)","author":"K\u00e4stner","year":"2021"},{"key":"B13","doi-asserted-by":"publisher","first-page":"7386","DOI":"10.1109\/TITS.2021.3069362","article-title":"Human trajectory forecasting in crowds: A deep learning perspective","volume":"23","author":"Kothari","year":"2021","journal-title":"IEEE Trans. Intell. Transp. Syst"},{"key":"B14","doi-asserted-by":"crossref","first-page":"346","DOI":"10.1109\/ROBIO54168.2021.9739519","article-title":"Human-aware robot navigation via reinforcement learning with hindsight experience replay and curriculum learning","volume-title":"2021 IEEE International Conference on Robotics and Biomimetics (ROBIO)","author":"Li","year":"2021"},{"key":"B15","doi-asserted-by":"crossref","first-page":"9470","DOI":"10.1109\/ICRA46639.2022.9811641","article-title":"Integrating point and line features for visual-inertial initialization","volume-title":"2022 International Conference on Robotics and Automation (ICRA)","author":"Liu","year":"2022"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA48891.2023.10160660","article-title":"Intention aware robot crowd navigation with attention-based interaction graph","author":"Liu","year":"2023","journal-title":"IEEE International Conference on Robotics and Automation (ICRA)"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA48891.2023.10160876","article-title":"Improving robot navigation in crowded environments using intrinsic rewards","author":"Martinez-Baselga","year":"2023","journal-title":"arXiv preprint arXiv:2302.06554"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.15406\/iratj.2017.02.00023","article-title":"Mobile robot navigation and obstacle avoidance techniques: a review","volume":"2","author":"Pandey","year":"2017","journal-title":"Int. Rob. Auto. J"},{"key":"B19","first-page":"2778","article-title":"Curiosity-driven exploration by self-supervised prediction","volume-title":"International Conference on Machine Learning","author":"Pathak","year":"2017"},{"key":"B20","article-title":"Benchmarking safe exploration in deep reinforcement learning","author":"Ray","year":"2019","journal-title":"arXiv preprint arXiv:1910.01708"},{"key":"B21","doi-asserted-by":"publisher","first-page":"4352","DOI":"10.1109\/LRA.2020.2996593","article-title":"Frozone: Freezing-free, pedestrian-friendly navigation in human crowds","volume":"5","author":"Sathyamoorthy","year":"2020","journal-title":"IEEE Robot. Autom. Lett"},{"key":"B22","article-title":"Prioritized experience replay","author":"Schaul","year":"2015","journal-title":"arXiv preprint arXiv:1511.05952"},{"key":"B23","doi-asserted-by":"crossref","first-page":"797","DOI":"10.1109\/IROS.2010.5654369","article-title":"Unfreezing the robot: Navigation in dense, interacting crowds","volume-title":"2010 IEEE\/RSJ International Conference on Intelligent Robots and Systems","author":"Trautman","year":"2010"},{"key":"B24","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/978-3-642-19457-3_1","article-title":"Reciprocal n-body collision avoidance","volume-title":"Robotics Research: The 14th International Symposium ISRR","author":"Van Den Berg","year":"2011"},{"key":"B25","doi-asserted-by":"crossref","first-page":"1928","DOI":"10.1109\/ROBOT.2008.4543489","article-title":"Reciprocal velocity obstacles for real-time multi-agent navigation","volume-title":"2008 IEEE International Conference on Robotics and Automation","author":"Van den Berg","year":"2008"},{"key":"B26","doi-asserted-by":"crossref","first-page":"9011","DOI":"10.1109\/IROS47612.2022.9982107","article-title":"Adaptive environment modeling based reinforcement learning for collision avoidance in complex scenes","volume-title":"2022 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)","author":"Wang","year":"2022"},{"key":"B27","doi-asserted-by":"publisher","first-page":"15600","DOI":"10.1007\/s10489-022-03191-2","article-title":"Robot navigation in a crowd by integrating deep reinforcement learning and online planning","volume":"52","author":"Zhou","year":"2022","journal-title":"Appl. Intell"}],"container-title":["Frontiers in Neurorobotics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1276519\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,12]],"date-time":"2023-10-12T09:25:30Z","timestamp":1697102730000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1276519\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,12]]},"references-count":27,"alternative-id":["10.3389\/fnbot.2023.1276519"],"URL":"https:\/\/doi.org\/10.3389\/fnbot.2023.1276519","relation":{},"ISSN":["1662-5218"],"issn-type":[{"value":"1662-5218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,10,12]]},"article-number":"1276519"}}