{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T01:30:18Z","timestamp":1760059818319,"version":"build-2065373602"},"reference-count":38,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2025,7,14]],"date-time":"2025-07-14T00:00:00Z","timestamp":1752451200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Vietnamese Government, Ministry of Science and Technology","award":["KC.01.02\/21-30"],"award-info":[{"award-number":["KC.01.02\/21-30"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MAKE"],"abstract":"<jats:p>This study investigates the application of the Dueling Double Deep Q-Network (3DQN) algorithm to optimize traffic signal control at a major urban intersection in Danang City, Vietnam. The objective is to enhance signal timing efficiency in response to mixed traffic flow and real-world traffic dynamics. A simulation environment was developed using the Simulation of Urban Mobility (SUMO) software version 1.11, incorporating both a fixed-time signal controller and two 3DQN models trained with 1 million (1M-Step) and 5 million (5M-Step) iterations. The models were evaluated using randomized traffic demand scenarios ranging from 50% to 150% of baseline traffic volumes. The results demonstrate that the 3DQN models outperform the fixed-time controller, significantly reducing vehicle delays, with the 5M-Step model achieving average waiting times of under five minutes. To further assess the model\u2019s responsiveness to real-time conditions, traffic flow data were collected using YOLOv8 for object detection and SORT for vehicle tracking from live camera feeds, and integrated into the SUMO-3DQN simulation. The findings highlight the robustness and adaptability of the 3DQN approach, particularly under peak traffic conditions, underscoring its potential for deployment in intelligent urban traffic management systems.<\/jats:p>","DOI":"10.3390\/make7030065","type":"journal-article","created":{"date-parts":[[2025,7,14]],"date-time":"2025-07-14T10:55:41Z","timestamp":1752490541000},"page":"65","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Application of Dueling Double Deep Q-Network for Dynamic Traffic Signal Optimization: A Case Study in Danang City, Vietnam"],"prefix":"10.3390","volume":"7","author":[{"given":"Tho Cao","family":"Phan","sequence":"first","affiliation":[{"name":"Department of Civil Engineering, The University of Danang\u2014University of Technology and Education, Danang 550000, Vietnam"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Viet Dinh","family":"Le","sequence":"additional","affiliation":[{"name":"School of Architecture, Civil and Environmental Engineering, Kumoh National Institute of Technology, Gumi 39177, Gyeongbuk, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6822-0753","authenticated-orcid":false,"given":"Teron","family":"Nguyen","sequence":"additional","affiliation":[{"name":"Samwoh Innovation Centre, Samwoh Smart Hub, 12 Kranji Way, Singapore 739454, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,7,14]]},"reference":[{"key":"ref_1","unstructured":"General Statistics Office of Vietnam (2019). Population and Housing Census 2019: Summary Results, Ministry of Planning and Investment."},{"key":"ref_2","unstructured":"Webster, F.V. (1958). Traffic Signal Settings. Road Research Technical Paper, HMSO."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1057\/jors.1963.61","article-title":"Settings for Fixed-Cycle Traffic Signals","volume":"14","author":"Miller","year":"1963","journal-title":"Oper. Res. Soc."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1016\/S0191-2615(03)00003-1","article-title":"Comparison of Delay Estimates at Under-Saturated and Over-Saturated Pre-Timed Signalized Intersections","volume":"38","author":"Dion","year":"2004","journal-title":"Transp. Res. Part B"},{"key":"ref_5","first-page":"209","article-title":"Adaptive Look-Ahead Optimization of Traffic Signals","volume":"4","author":"Porche","year":"1999","journal-title":"J. Intell. Transp. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Cools, S.-B., Gershenson, C., and D\u2019Hooghe, B. (2013). Self-Organizing Traffic Lights: A Realistic Simulation. Advances in Applied Self-Organizing Systems, Springer.","DOI":"10.1007\/978-1-4471-5113-5_3"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"032097","DOI":"10.1088\/1742-6596\/1650\/3\/032097","article-title":"Environmental Adaptive Urban Traffic Signal Control Based on Reinforcement Learning Algorithm","volume":"1650","author":"Feng","year":"2020","journal-title":"J. Phys. Conf. Ser."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"488","DOI":"10.1139\/cjce-2022-0273","article-title":"Adaptive Traffic Signal Control Using Deep Q-Learning: Case Study on Optimal Implementations","volume":"50","author":"Pan","year":"2023","journal-title":"Can. J. Civ. Eng."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"13851","DOI":"10.1007\/s11042-023-16112-3","article-title":"Adaptive Traffic Light Control Using Deep Reinforcement Learning Technique","volume":"83","author":"Kumar","year":"2024","journal-title":"Multimed. Tools Appl."},{"key":"ref_10","first-page":"3","article-title":"Swarm Intelligence Inspired Adaptive Traffic Control for Traffic Networks","volume":"Volume 221","author":"Tian","year":"2018","journal-title":"Advances in Swarm Intelligence"},{"key":"ref_11","first-page":"101","article-title":"Traffic Signal Optimization Using Genetic Algorithms","volume":"Volume 496","author":"Mohamed","year":"2024","journal-title":"Advances in Intelligent Systems and Computing"},{"key":"ref_12","first-page":"26","article-title":"Traffic Light Control with Reinforcement Learning","volume":"43","author":"Pan","year":"2024","journal-title":"Adv. Educ. Technol."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1049\/iet-its.2009.0096","article-title":"Urban Traffic Signal Control Using Reinforcement Learning Agents","volume":"4","author":"Balaji","year":"2010","journal-title":"IET Intell. Transp. Syst."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1007\/s10462-012-9383-6","article-title":"Application of Reinforcement Learning to Routing in Distributed Wireless Networks: A Review","volume":"43","author":"Ng","year":"2015","journal-title":"Artif. Intell. Rev."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"437","DOI":"10.1111\/mafi.12382","article-title":"Recent Advances in Reinforcement Learning in Finance","volume":"33","author":"Mnih","year":"2023","journal-title":"Math. Financ."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-Level Control Through Deep Reinforcement Learning","volume":"518","author":"Kavukcuoglu","year":"2015","journal-title":"Nature"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Park, B., Lee, J., Kim, T., and Har, D.S. (2022). Kick-Motion Training with DQN in AI Soccer Environment. arXiv.","DOI":"10.1109\/ICAIIC57133.2023.10067011"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1109\/JAS.2016.7508798","article-title":"Traffic Signal Timing via Deep Reinforcement Learning","volume":"3","author":"Li","year":"2016","journal-title":"IEEE\/CAA J. Autom. Sin."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1086","DOI":"10.1109\/TITS.2019.2901791","article-title":"Multi-Agent Deep Reinforcement Learning for Large-Scale Traffic Signal Control","volume":"21","author":"Chu","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_20","first-page":"954","article-title":"Deep Reinforcement Learning for Traffic Signal Control","volume":"74","author":"Skuba","year":"2023","journal-title":"Transp. Res. Proc."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"275","DOI":"10.1016\/j.procs.2023.03.036","article-title":"Deep Reinforcement Learning Based Traffic Signal Control: A Comparative Analysis","volume":"220","author":"Wu","year":"2023","journal-title":"Procedia Comput. Sci."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"11650","DOI":"10.36948\/ijfmr.2024.v06i01.11650","article-title":"A Review of Deep Reinforcement Learning for Traffic Signal Control","volume":"6","author":"Prajapati","year":"2024","journal-title":"Int. J. Multidiscip. Res."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Zheng, Q., Xu, H., Chen, J., and Zhang, K. (2025). Multi-Dimensional Double Deep Dynamic Q-Network with Aligned Q-Fusion for Dual-Ring Barrier Traffic Signal Control. Appl. Sci., 15.","DOI":"10.3390\/app15031118"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Chu, X., and Cao, X. (2024, January 22\u201324). Adaptive traffic signal control for road networks based on dueling double deep q-network. Proceedings of the International Conference on Frontiers of Traffic and Transportation Engineering (FTTE 2024), Lanzhou, China.","DOI":"10.1117\/12.3059190"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1007\/s40747-025-01841-9","article-title":"Traffic signal optimization control method based on attention mechanism updated weights double deep Q network","volume":"11","author":"Zhang","year":"2025","journal-title":"Complex Intell. Syst."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Zhang, R., Leteurtre, R., Striner, B., Alanazi, A., Alghafis, A., and Tonguz, O. (2019). Partially Detected Intelligent Traffic Signal Control: Environmental Adaptation. arXiv.","DOI":"10.1109\/ICMLA.2019.00314"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Wei, H., Zheng, G., Yao, H., and Li, Z. (2018, January 19\u201323). Intellilight: A Reinforcement Learning Approach for Intelligent Traffic Light Control. Proceedings of the KDD \u201918: The 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, London, UK.","DOI":"10.1145\/3219819.3220096"},{"key":"ref_28","first-page":"192","article-title":"Deep Reinforcement Q-Learning for Intelligent Traffic Signal Control with Partial Detection","volume":"21","author":"Ducrocq","year":"2023","journal-title":"Int. J. Intell. Transp. Syst. Res."},{"key":"ref_29","unstructured":"Cardoso, R.F.D. (2023). Intelligent Traffic Light Management for Automated Guided Vehicle Systems Using Deep Reinforcement Learning. [Master\u2019s Thesis, Universidade do Porto]."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"2797","DOI":"10.1109\/TITS.2020.2975120","article-title":"Estimation of Urban Traffic State with Probe Vehicles","volume":"22","author":"Nguyen","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"404","DOI":"10.1109\/TITS.2019.2958859","article-title":"Using Reinforcement Learning with Partial Vehicle Detection for Intelligent Traffic Signal Control","volume":"22","author":"Zhang","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_32","unstructured":"Ministry of Transport of Vietnam (2020). Journey-Tracking Devices and Regulations on Their Installation, Management, and Data Usage for Commercial Transport Vehicles. Decree 10\/2020\/ND-CP and Circular 12\/2020\/TT-BGTVT, Ministry of Transport."},{"key":"ref_33","unstructured":"People\u2019s Committee of Danang City (2024). Regulation, Tasks, and Competition Plan for the Planning and Architectural Design of the Intersection Cluster at Lethanhnghi\u2014Cachmangthang 8\u2014Thang Long\u2014Hoaxuan Bridge Approach Road, Da Nang City Government."},{"key":"ref_34","first-page":"269","article-title":"Traffic Simulation with SUMO Simulation of Urban Mobility","volume":"Volume 145","author":"Krajzewicz","year":"2010","journal-title":"Advances in Intelligent and Soft Computing"},{"key":"ref_35","first-page":"265","article-title":"A Study on Traffic Signal System Improvement Using SUMO (Simulation of Urban Mobility)","volume":"42","author":"Kim","year":"2024","journal-title":"J. Korean Soc. Geospat. Inf. Syst."},{"key":"ref_36","unstructured":"(2025, June 13). Explore Ultralytics YOLOv8. Available online: https:\/\/yolov8.com\/."},{"key":"ref_37","unstructured":"Bansal, A. (2025, June 13). Traffic Counting Program Using YOLOv8. Github. Available online: https:\/\/github.com\/anujeshify\/Traffic-Counting-Program-using-YOLOv8."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Bewley, A., Zongyuan, G., Ott, L., Ramos, F., and Upcroft, B. (2016, January 25\u201328). Simple online and realtime tracking. Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA.","DOI":"10.1109\/ICIP.2016.7533003"}],"container-title":["Machine Learning and Knowledge Extraction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-4990\/7\/3\/65\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:09:33Z","timestamp":1760033373000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-4990\/7\/3\/65"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,14]]},"references-count":38,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2025,9]]}},"alternative-id":["make7030065"],"URL":"https:\/\/doi.org\/10.3390\/make7030065","relation":{},"ISSN":["2504-4990"],"issn-type":[{"type":"electronic","value":"2504-4990"}],"subject":[],"published":{"date-parts":[[2025,7,14]]}}}