{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,23]],"date-time":"2026-07-23T16:06:49Z","timestamp":1784822809031,"version":"3.55.0"},"reference-count":23,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2021,1,18]],"date-time":"2021-01-18T00:00:00Z","timestamp":1610928000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61672190"],"award-info":[{"award-number":["61672190"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>Existing crowd evacuation guidance systems require the manual design of models and input parameters, incurring a significant workload and a potential for errors. This paper proposed an end-to-end intelligent evacuation guidance method based on deep reinforcement learning, and designed an interactive simulation environment based on the social force model. The agent could automatically learn a scene model and path planning strategy with only scene images as input, and directly output dynamic signage information. Aiming to solve the \u201cdimension disaster\u201d phenomenon of the deep Q network (DQN) algorithm in crowd evacuation, this paper proposed a combined action-space DQN (CA-DQN) algorithm that grouped Q network output layer nodes according to action dimensions, which significantly reduced the network complexity and improved system practicality in complex scenes. In this paper, the evacuation guidance system is defined as a reinforcement learning agent and implemented by the CA-DQN method, which provides a novel approach for the evacuation guidance problem. The experiments demonstrate that the proposed method is superior to the static guidance method, and on par with the manually designed model method.<\/jats:p>","DOI":"10.3390\/a14010026","type":"journal-article","created":{"date-parts":[[2021,1,19]],"date-time":"2021-01-19T04:55:31Z","timestamp":1611032131000},"page":"26","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":26,"title":["Crowd Evacuation Guidance Based on Combined Action Reinforcement Learning"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4800-8442","authenticated-orcid":false,"given":"Yiran","family":"Xue","sequence":"first","affiliation":[{"name":"School of Computer Science and Technology, Harbin Institute of Technology, Harbin 150001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rui","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Harbin Institute of Technology, Harbin 150001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiafeng","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Harbin Institute of Technology, Harbin 150001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xianglong","family":"Tang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Harbin Institute of Technology, Harbin 150001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,1,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1016\/j.autcon.2013.10.022","article-title":"Influences of intelligent evacuation guidance system on crowd evacuation in building fire","volume":"41","author":"Ran","year":"2014","journal-title":"Autom. Constr."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Desmet, A., and Gelenbe, E. (2014, January 24\u201328). Capacity based evacuation with dynamic exit signs. Proceedings of the 2014 IEEE International Conference on Pervasive Computing and Communication Workshops (PERCOM WORKSHOPS), Budapest, Hungary.","DOI":"10.1109\/PerComW.2014.6815227"},{"key":"ref_3","first-page":"523","article-title":"Simulation of crowd movement and design and implementation of evacuation optimization method","volume":"26","author":"Zhao","year":"2014","journal-title":"J. Syst. Simul."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"627","DOI":"10.1109\/TITS.2012.2228260","article-title":"Emergency management of urban rail transportation based on parallel systems","volume":"14","author":"Dong","year":"2012","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"101081","DOI":"10.1016\/j.gmod.2020.101081","article-title":"A review on crowd simulation and modeling","volume":"111","author":"Yang","year":"2020","journal-title":"Graph. Model."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"799","DOI":"10.1007\/s11390-014-1469-y","article-title":"Crowd simulation and its applications: Recent advances","volume":"29","author":"Xu","year":"2014","journal-title":"J. Comput. Sci. Technol."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"487","DOI":"10.1016\/S0378-4371(99)00018-7","article-title":"Jamming transition in pedestrian counter flow","volume":"267","author":"Muramatsu","year":"1999","journal-title":"Phys. A Stat. Mech. Appl."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Bisagno, N., Conci, N., and Zhang, B. (September, January 29). Data-driven crowd simulation. Proceedings of the 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Lecce, Italy.","DOI":"10.1109\/AVSS.2017.8078494"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1515\/amcs-2017-0013","article-title":"Abnormal prediction of dense crowd videos by a purpose\u2013driven lattice boltzmann model","volume":"27","author":"Xue","year":"2017","journal-title":"Int. J. Appl. Math. Comput. Sci."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"4282","DOI":"10.1103\/PhysRevE.51.4282","article-title":"Social force model for pedestrian dynamics","volume":"51","author":"Helbing","year":"1995","journal-title":"Phys. Rev. E"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"360","DOI":"10.1016\/j.asoc.2018.04.015","article-title":"A path planning approach for crowd evacuation in buildings based on improved artificial bee colony algorithm","volume":"68","author":"Liu","year":"2018","journal-title":"Appl. Soft Comput."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"947","DOI":"10.1016\/j.firesaf.2017.04.011","article-title":"Real-time evacuation route selection methodology for complex buildings","volume":"91","author":"Cuesta","year":"2017","journal-title":"Fire Saf. J."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1081","DOI":"10.1109\/JAS.2019.1911672","article-title":"Guided crowd evacuation: Approaches and challenges","volume":"6","author":"Zhou","year":"2019","journal-title":"IEEE\/CAA J. Autom. Sin."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1002\/fam.2665","article-title":"The influence of emergency signage on building evacuation behavior: An experimental study","volume":"43","author":"Fu","year":"2019","journal-title":"Fire Mater."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"723","DOI":"10.1177\/0037549717741350","article-title":"A social force evacuation model considering the effect of emergency signs","volume":"94","author":"Yuan","year":"2018","journal-title":"Simulation"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"908","DOI":"10.1016\/j.firesaf.2017.03.022","article-title":"Evaluating the effectiveness of an improved active dynamic signage system using full scale evacuation trials","volume":"91","author":"Galea","year":"2017","journal-title":"Fire Saf. J."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1016\/j.autcon.2015.05.004","article-title":"An automated direction setting algorithm for a smart exit sign","volume":"59","author":"Cho","year":"2015","journal-title":"Autom. Constr."},{"key":"ref_19","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"484","DOI":"10.1038\/nature16961","article-title":"Mastering the game of Go with deep neural networks and tree search","volume":"529","author":"Silver","year":"2016","journal-title":"Nature"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"350","DOI":"10.1038\/s41586-019-1724-z","article-title":"Grandmaster level in StarCraft II using multi-agent reinforcement learning","volume":"575","author":"Vinyals","year":"2019","journal-title":"Nature"},{"key":"ref_22","unstructured":"Hasselt, H.v., Guez, A., and Silver, D. (2016, January 12\u201317). Deep reinforcement learning with double Q-Learning. Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA."},{"key":"ref_23","unstructured":"Schaul, T., Quan, J., Antonoglou, I., and Silver, D. (2015). Prioritized experience replay. arXiv."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/14\/1\/26\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:12:34Z","timestamp":1760159554000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/14\/1\/26"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,1,18]]},"references-count":23,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2021,1]]}},"alternative-id":["a14010026"],"URL":"https:\/\/doi.org\/10.3390\/a14010026","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,1,18]]}}}