{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T02:55:53Z","timestamp":1781837753279,"version":"3.54.5"},"reference-count":35,"publisher":"MDPI AG","issue":"22","license":[{"start":{"date-parts":[[2022,11,11]],"date-time":"2022-11-11T00:00:00Z","timestamp":1668124800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"the National Key Research and Development Program of China","award":["2018YFB1601100"],"award-info":[{"award-number":["2018YFB1601100"]}]},{"name":"the National Key Research and Development Program of China","award":["CIT&TCD201904013"],"award-info":[{"award-number":["CIT&TCD201904013"]}]},{"name":"the National Key Research and Development Program of China","award":["CIT&TCD201804006"],"award-info":[{"award-number":["CIT&TCD201804006"]}]},{"name":"the National Key Research and Development Program of China","award":["KM202010009007"],"award-info":[{"award-number":["KM202010009007"]}]},{"name":"the Youth Top Talent Training Program of Beijing","award":["2018YFB1601100"],"award-info":[{"award-number":["2018YFB1601100"]}]},{"name":"the Youth Top Talent Training Program of Beijing","award":["CIT&TCD201904013"],"award-info":[{"award-number":["CIT&TCD201904013"]}]},{"name":"the Youth Top Talent Training Program of Beijing","award":["CIT&TCD201804006"],"award-info":[{"award-number":["CIT&TCD201804006"]}]},{"name":"the Youth Top Talent Training Program of Beijing","award":["KM202010009007"],"award-info":[{"award-number":["KM202010009007"]}]},{"name":"the General Program of Science and Technology Plan of Beijing Municipal Education Commission","award":["2018YFB1601100"],"award-info":[{"award-number":["2018YFB1601100"]}]},{"name":"the General Program of Science and Technology Plan of Beijing Municipal Education Commission","award":["CIT&TCD201904013"],"award-info":[{"award-number":["CIT&TCD201904013"]}]},{"name":"the General Program of Science and Technology Plan of Beijing Municipal Education Commission","award":["CIT&TCD201804006"],"award-info":[{"award-number":["CIT&TCD201804006"]}]},{"name":"the General Program of Science and Technology Plan of Beijing Municipal Education Commission","award":["KM202010009007"],"award-info":[{"award-number":["KM202010009007"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Deep reinforcement learning provides a new approach to solving complex signal optimization problems at intersections. Earlier studies were limited to traditional traffic detection techniques, and the obtained traffic information was not accurate. With the advanced in technology, we can obtain highly accurate information on the traffic states by advanced detector technology. This provides an accurate source of data for deep reinforcement learning. There are many intersections in the urban network. To successfully apply deep reinforcement learning in a situation closer to reality, we need to consider the problem of extending the knowledge gained from the training to new scenarios. This study used advanced sensor technology as a data source to explore the variation pattern of state space under different traffic scenarios. It analyzes the relationship between the traffic demand and the actual traffic states. The model learned more from a more comprehensive state space of traffic. This model was successful applied to new traffic scenarios without additional training. Compared our proposed model with the popular SAC signal control model, the result shows that the average delay of the DQN model is 5.13 s and the SAC model is 6.52 s. Therefore, our model exhibits better control performance.<\/jats:p>","DOI":"10.3390\/s22228732","type":"journal-article","created":{"date-parts":[[2022,11,14]],"date-time":"2022-11-14T04:30:52Z","timestamp":1668400252000},"page":"8732","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":25,"title":["Deep Reinforcement Learning for Traffic Signal Control Model and Adaptation Study"],"prefix":"10.3390","volume":"22","author":[{"given":"Jiyuan","family":"Tan","sequence":"first","affiliation":[{"name":"School of Electrical and Control Engineering, North China University of Technology, Beijing 100144, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6776-3788","authenticated-orcid":false,"given":"Qian","family":"Yuan","sequence":"additional","affiliation":[{"name":"School of Electrical and Control Engineering, North China University of Technology, Beijing 100144, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Weiwei","family":"Guo","sequence":"additional","affiliation":[{"name":"School of Electrical and Control Engineering, North China University of Technology, Beijing 100144, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Na","family":"Xie","sequence":"additional","affiliation":[{"name":"School of Management Science and Engineering, Central University of Finance and Economics, Beijing 100081, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fuyu","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Electrical and Control Engineering, North China University of Technology, Beijing 100144, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jing","family":"Wei","sequence":"additional","affiliation":[{"name":"School of Electrical and Control Engineering, North China University of Technology, Beijing 100144, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xinwei","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Electrical and Control Engineering, North China University of Technology, Beijing 100144, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,11,11]]},"reference":[{"key":"ref_1","first-page":"58","article-title":"Development and Tendency of Intelligent Transportation Systems in China","volume":"1","author":"Liu","year":"2015","journal-title":"Autom. Panor."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"8667","DOI":"10.1109\/TVT.2017.2702388","article-title":"Distributed Cooperative Reinforcement Learning-Based Traffic Signal Control That Integrates V2X Networks\u2019 Dynamic Clustering","volume":"66","author":"Liu","year":"2017","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_3","first-page":"84","article-title":"Single intersection signal control based on multi-sensor information fusion","volume":"24","author":"Gao","year":"2006","journal-title":"China Sci. Technol. Inf."},{"key":"ref_4","unstructured":"Si, W. (2020). Intelligent Traffic Signal Control System Design and Development Practice Based on ITS System Framework. [Master\u2019s Thesis, Zhejiang University]."},{"key":"ref_5","unstructured":"Zhou, J. (2020). Induced Signal Control Evaluation and Parameter Optimization Based on Video-Detected Traffic Flow Data. [Master\u2019s Thesis, Wuhan University of Technology]."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1080\/00207217.2014.954634","article-title":"Design of real-time video watermarking based on Integer DCT for H.264 encoder","volume":"102","author":"Joshi","year":"2015","journal-title":"Int. J. Electron."},{"key":"ref_7","unstructured":"Xu, Y. (2021). Study on the Application of Simulation and Evaluation Methods for Urban Traffic Signal Control. [Master\u2019s Thesis, Nanjing University of Technology]."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"21482","DOI":"10.1109\/ACCESS.2018.2825250","article-title":"Construction of large-scale low-cost delivery infrastructure using vehicular networks","volume":"6","author":"Liu","year":"2018","journal-title":"IEEE Access"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"885","DOI":"10.3390\/electronics9060885","article-title":"A Review of Research on Intersection Control Based on Connected Vehicles and Data-Driven Intelligent Approaches","volume":"9","author":"Du","year":"2020","journal-title":"Electronics"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"He, Y., Yao, D., Zhang, Y., Pei, X., and Li, L. (2016, January 10\u201312). Cellular automaton model for bidirectional traffic under condition of intelligent vehicle infrastructure cooperative systems. Proceedings of the 2016 IEEE International Conference on Vehicular Electronics and Safety (ICVES), Beijing, China.","DOI":"10.1109\/ICVES.2016.7548172"},{"key":"ref_11","first-page":"1","article-title":"Deep reinforcement learning for intelligent transportation systems: A survey","volume":"10","author":"Haydari","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1007\/978-3-319-25808-9_4","article-title":"An experimental review of reinforcement learning algorithms for adaptive traffic signal control","volume":"10","author":"Mannion","year":"2016","journal-title":"Auton. Road Transp. Support Syst."},{"key":"ref_13","first-page":"127","article-title":"A novel approach for traffic signal control: A recommendation perspective","volume":"9","author":"Zhao","year":"2017","journal-title":"IEEE Intell. Transp. Syst."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1616","DOI":"10.1109\/TITS.2020.2973736","article-title":"An end-to-end recommendation system for urban traffic controls and management under a parallel learning framework","volume":"22","author":"Jin","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1086","DOI":"10.1109\/TITS.2019.2901791","article-title":"Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Control","volume":"21","author":"Chu","year":"2019","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"108304","DOI":"10.1016\/j.knosys.2022.108304","article-title":"Distributed agent-based deep reinforcement learning for large scale traffic signal control","volume":"241","author":"Wu","year":"2022","journal-title":"Knowl.-Based Syst."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"103059","DOI":"10.1016\/j.trc.2021.103059","article-title":"Network-wide traffic signal control optimization using a multi-agent deep reinforcement learning","volume":"125","author":"Li","year":"2021","journal-title":"Transp. Res. Part Emerg. Technol."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"145228","DOI":"10.1109\/ACCESS.2021.3123273","article-title":"Traffic Signal Control Under Mixed Traffic With Connected and Automated Vehicles: A Transfer-Based Deep Reinforcement Learning Approach","volume":"9","author":"Song","year":"2021","journal-title":"IEEE Access"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1054","DOI":"10.1109\/TNN.1998.712192","article-title":"Reinforcement learning: An introduction","volume":"9","author":"Sutton","year":"2002","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_20","first-page":"529","article-title":"Evaluating reinforcement learning state representations for adaptive traffic signal control","volume":"518","author":"Minh","year":"2015","journal-title":"Nature"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Wei, H., Zheng, G., Yao, H., and Li, Z. (2018, January 19\u201323). IntelliLight: A Reinforcement Learning Approach for Intelligent Traffic Light Control. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, London, UK.","DOI":"10.1145\/3219819.3220096"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1109\/JAS.2016.7508798","article-title":"Traffic signal timing via deep reinforcement learning","volume":"3","author":"Li","year":"2016","journal-title":"IEEE\/CAA J. Autom. Sin."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1243","DOI":"10.1109\/TVT.2018.2890726","article-title":"A Deep Reinforcement Learning Network for Traffic Light Cycle Control","volume":"2","author":"Liang","year":"2019","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Kim, D., and Jeong, O. (2020). Cooperative Traffic Signal Control with Traffic Flow Prediction in Multi-Intersection. Sensors, 20.","DOI":"10.3390\/s20010137"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"431","DOI":"10.1016\/j.future.2020.03.065","article-title":"Deep reinforcement learning for traffic signal control under disturbances: A case study on Sunway city, Malaysia","volume":"109","author":"Rasheed","year":"2020","journal-title":"Future Gener. Comput.-Syst.-Int. J. Esci."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Yen, C.C., Ghosal, D., Zhang, M., and Chuah, C.N. (2020, January 20\u201323). A Deep On-Policy Learning Agent for Traffic Signal Control of Multiple Intersections. Proceedings of the IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), Rhodes, Greece.","DOI":"10.1109\/ITSC45102.2020.9294471"},{"key":"ref_27","unstructured":"Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Rizzo, S.G., Vantini, G., and Chawla, S. (2019, January 4\u20138). Time critic policy gradient methods for traffic signal control in complex and congested scenarios. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery Data Mining, Anchorage, AK, USA.","DOI":"10.1145\/3292500.3330988"},{"key":"ref_29","first-page":"1861","article-title":"Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor","volume":"10","author":"Haarnoja","year":"2018","journal-title":"Proc. Int. Conf. Mach. Learn."},{"key":"ref_30","first-page":"2","article-title":"A Comparison of Deep Reinforcement Learning Models for Isolated Traffic Signal Control","volume":"3","author":"Mao","year":"2022","journal-title":"IEEE Intell. Transp. Syst. Mag."},{"key":"ref_31","first-page":"652","article-title":"Fairness control of traffic light via deep reinforcement learning","volume":"10","author":"Li","year":"2020","journal-title":"Proc. IEEE 16th Int. Conf. Automat. Sci. Eng. (CASE)"},{"key":"ref_32","unstructured":"Casas, N. (2017). Deep Deterministic Policy Gradient for Urban Traffic Light Control. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Yu, B., Guo, J., and Zhao, Q. (2020, January 19\u201323). Smarter and Safer Traffic Signal Controlling via Deep Reinforcement Learning. Proceedings of the CIKM \u201920: The 29th ACM International Conference on Information and Knowledge Management, Online.","DOI":"10.1145\/3340531.3417450"},{"key":"ref_34","first-page":"1","article-title":"Network-wide traffic signal control based on the discovery of critical nodes and deep reinforcement learning","volume":"10","author":"Xu","year":"2018","journal-title":"J. Intell. Transp. Syst."},{"key":"ref_35","unstructured":"Lin, Y., Dai, X., Li, L., and Wang, F.Y. (2018). An efficient deep reinforcement learning model for urban traffic control. arXiv."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/22\/8732\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:16:34Z","timestamp":1760145394000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/22\/8732"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,11]]},"references-count":35,"journal-issue":{"issue":"22","published-online":{"date-parts":[[2022,11]]}},"alternative-id":["s22228732"],"URL":"https:\/\/doi.org\/10.3390\/s22228732","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,11]]}}}