{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,20]],"date-time":"2026-10-20T12:13:11Z","timestamp":1792498391173,"version":"4.5.0"},"reference-count":45,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2019,12,24]],"date-time":"2019-12-24T00:00:00Z","timestamp":1577145600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["2019R1A2C1008412"],"award-info":[{"award-number":["2019R1A2C1008412"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100010418","name":"Institute for Information and Communications Technology Promotion","doi-asserted-by":"publisher","award":["2015-0-00932"],"award-info":[{"award-number":["2015-0-00932"]}],"id":[{"id":"10.13039\/501100010418","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Ministry of Science and ICT","award":["2015-0-00932"],"award-info":[{"award-number":["2015-0-00932"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>As traffic congestion in cities becomes serious, intelligent traffic signal control has been actively studied. Deep Q-Network (DQN), a representative deep reinforcement learning algorithm, is applied to various domains from fully-observable game environment to traffic signal control. Due to the effective performance of DQN, deep reinforcement learning has improved speeds and various DQN extensions have been introduced. However, most traffic signal control researches were performed at a single intersection, and because of the use of virtual simulators, there are limitations that do not take into account variables that affect actual traffic conditions. In this paper, we propose a cooperative traffic signal control with traffic flow prediction (TFP-CTSC) for a multi-intersection. A traffic flow prediction model predicts future traffic state and considers the variables that affect actual traffic conditions. In addition, for cooperative traffic signal control in multi-intersection, each intersection is modeled as an agent, and each agent is trained to take best action by receiving traffic states from the road environment. To deal with multi-intersection efficiently, agents share their traffic information with other adjacent intersections. In the experiment, TFP-CTSC is compared with existing traffic signal control algorithms in a 4 \u00d7 4 intersection environment. We verify our traffic flow prediction and cooperative method.<\/jats:p>","DOI":"10.3390\/s20010137","type":"journal-article","created":{"date-parts":[[2019,12,24]],"date-time":"2019-12-24T10:28:43Z","timestamp":1577183323000},"page":"137","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":68,"title":["Cooperative Traffic Signal Control with Traffic Flow Prediction in Multi-Intersection"],"prefix":"10.3390","volume":"20","author":[{"given":"Daeho","family":"Kim","sequence":"first","affiliation":[{"name":"Department of Software, Gachon University, Gyeonggi 13120, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Okran","family":"Jeong","sequence":"additional","affiliation":[{"name":"Department of Software, Gachon University, Gyeonggi 13120, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2019,12,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"690","DOI":"10.1139\/cjce-2017-0408","article-title":"Continuous residual reinforcement learning for traffic signal control optimization","volume":"45","author":"Aslani","year":"2018","journal-title":"Can. J. Civ. Eng."},{"key":"ref_2","unstructured":"Lafferriere, G. (2019). A Decentralized Network Consensus Control Approach for Urban Traffic Signal Optimization, PDXScholar."},{"key":"ref_3","first-page":"289","article-title":"Developing adaptive traffic signal control by actor\u2013critic and direct exploration methods","volume":"172","author":"Aslani","year":"2019","journal-title":"Proc. Inst. Civ. Eng.-Transp."},{"key":"ref_4","unstructured":"Lv, Y., Duan, Y., Kang, W., and Wang, F.-Y. (2015, January 15\u201318). Traffic flow prediction with big data: A deep learning approach. Proceedings of the IEEE Transactions on Intelligent Transportation Systems, Washington, DC, USA."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2191","DOI":"10.1109\/TITS.2014.2311123","article-title":"Deep architecture for traffic flow prediction: Deep belief networks with multitask learning","volume":"15","author":"Huang","year":"2014","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.trc.2017.02.024","article-title":"Deep learning for short-term traffic flow prediction","volume":"79","author":"Polson","year":"2017","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"894","DOI":"10.1016\/j.procs.2018.04.086","article-title":"Implementing an adaptive traffic signal control algorithm in an agent-based transport simulation","volume":"130","author":"Theresa","year":"2018","journal-title":"Procedia Comput. Sci."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"132","DOI":"10.3141\/1644-14","article-title":"Urban freeway traffic flow prediction: Application of seasonal autoregressive integrated moving average and exponential smoothing models","volume":"1644","author":"Williams","year":"1998","journal-title":"Transp. Res. Rec."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1459","DOI":"10.1109\/TVT.2013.2241460","article-title":"Adaptive traffic signal control with vehicular ad hoc networks","volume":"62","author":"Pandit","year":"2013","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"574","DOI":"10.1016\/j.engappai.2012.04.008","article-title":"Distributed and adaptive traffic signal control within a realistic traffic simulation","volume":"26","author":"Mckenney","year":"2013","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_11","unstructured":"Yang, K., Tan, I., and Menendez, M. (2017, January 17\u201319). A reinforcement learning based traffic signal control algorithm in a connected vehicle environment. Proceedings of the 17th Swiss Transport Research Conference (STRC 2017), Ascona, Switzerland."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"232","DOI":"10.1016\/S0377-2217(00)00123-5","article-title":"Reinforcement learning in neurofuzzy traffic signal control","volume":"131","author":"Bingham","year":"2011","journal-title":"Eur. J. Oper. Res."},{"key":"ref_13","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, L., Wierstra, D., and Riedmiller, M. (2013, December 19). Playing Atari with Deep Reinforcement Learning. Available online: https:\/\/arxiv.org\/abs\/1312.5602."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zeng, J., Hu, J., and Zhang, Y. (2018, January 26\u201330). Adaptive traffic signal control with deep recurrent Q-learning. Proceedings of the 2018 IEEE Intelligent Vehicles Symposium (IV), Changshu, China.","DOI":"10.1109\/IVS.2018.8500414"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Wei, H., Zheng, G., Yao, H., and Li, Z. (2018, January 19\u201323). Intellilight: A reinforcement learning approach for intelligent traffic light control. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, New York, NY, USA.","DOI":"10.1145\/3219819.3220096"},{"key":"ref_17","unstructured":"Wang, X., Ke, L., Qiao, Z., and Chai, X. (2019). Large-scale traffic signal control using a novel multi-agent reinforcement learning. arXiv Prepr."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"104855","DOI":"10.1016\/j.knosys.2019.07.026","article-title":"Cooperative traffic signal control using multi-step return and off-policy asynchronous advantage actor-critic graph algorithm","volume":"183","author":"Yang","year":"2019","journal-title":"Knowl.-Based Syst."},{"key":"ref_19","first-page":"656","article-title":"A two stage fuzzy logic adaptive traffic signal control for an isolated intersection based on real data using SUMO simulator","volume":"3","author":"Mahmood","year":"2019","journal-title":"Electron. Commun. Eng."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"121","DOI":"10.12815\/kits.2018.17.6.121","article-title":"Estimation of road surface condition during summer season using machine learning","volume":"17","author":"Lee","year":"2018","journal-title":"J. Korea Inst. Intell. Transp. Syst."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"9508","DOI":"10.1109\/TVT.2016.2585575","article-title":"Improving traffic flow prediction with weather information in connected cars: A deep learning approach","volume":"65","author":"Koesdwiady","year":"2016","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1111\/j.1467-8667.2006.00439.x","article-title":"Bi-level programming formulation and heuristic solution approach for dynamic traffic signal optimization","volume":"21","author":"Sun","year":"2006","journal-title":"Comput.-Aided Civ. Infrastruct. Eng."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Razzaq, M., and Shin, S. (2019). Fuzzy-logic dijkstra-based energy-efficient algorithm for data transmission in WSNs. Sensors, 19.","DOI":"10.3390\/s19051040"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Cheng, L., Li, Y., Wang, Y., Bi, Y., Feng, L., and Xue, M. (2019). A triple-filter NLOS localization algorithm based on fuzzy c-means for wireless sensor networks. Sensors, 19.","DOI":"10.3390\/s19051215"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"3009","DOI":"10.1109\/TITS.2017.2762085","article-title":"Optimal type-2 fuzzy system for arterial traffic signal control","volume":"19","author":"Bi","year":"2017","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1007\/s40815-016-0143-7","article-title":"Maximum green time settings for traffic-actuated signal control at isolated intersections using fuzzy logic","volume":"19","author":"Shiri","year":"2017","journal-title":"Int. J. Fuzzy Syst."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Yang, W., Zhang, L., He, Z., and Zhuang, L. (2012, January 16\u201319). Optimized two-stage fuzzy control for urban traffic signals at isolated intersection and paramics simulation. Proceedings of the 2012 15th International IEEE Conference on Intelligent Transportation Systems, Anchorage, AK, USA.","DOI":"10.1109\/ITSC.2012.6338691"},{"key":"ref_28","unstructured":"Bookseller, A.A., and Jagtap, R.R. (2013). Image processing based adaptive traffic control system. IOSR J. Electron. Commun. Eng., 33\u201337."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1016\/j.trc.2015.03.010","article-title":"A modified reinforcement learning algorithm for solving coordinated signalized networks","volume":"54","author":"Ozan","year":"2015","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1005","DOI":"10.1049\/iet-its.2018.5170","article-title":"Value-based deep reinforcement learning for adaptive isolated intersection signal control","volume":"12","author":"Wan","year":"2018","journal-title":"IET Intell. Transp. Syst."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Co\u015fkun, M., Baggag, A., and Chawla, S. (2018, January 17\u201320). Deep reinforcement learning for traffic light optimization. Proceedings of the 2018 IEEE International Conference on Data Mining Workshops (ICDMW), Singapore.","DOI":"10.1109\/ICDMW.2018.00088"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1109\/JAS.2016.7508798","article-title":"Traffic signal timing via deep reinforcement learning","volume":"3","author":"Li","year":"2016","journal-title":"IEEE\/CAA J. Autom. Sin."},{"key":"ref_33","unstructured":"Genders, W., and Razavi, S. (2016). Using a deep reinforcement learning agent for traffic signal control. arXiv Prepr."},{"key":"ref_34","unstructured":"Hasselt, H.V., Guez, A., and Silver, D. (2016, January 12\u201317). Deep reinforcement learning with double q-learning. Proceedings of the Thirtieth AAAI conference on artificial intelligence, Phoenix, AZ, USA."},{"key":"ref_35","unstructured":"Wang, Z., Schaul, T., Hessel, M., Hasselt, H., Lanctot, M., and Freitas, N. (2016). Dueling network architectures for deep reinforcement learning. arXiv Prepr."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Hessel, M., Modayil, J., van Hasselt, H., Schaul, T., Ostrovski, G., Dabney, W., Horgan, D., Piot, B., Azar, M., and Silver, D. (2017, January 2\u20137). Rainbow: Combining improvements in deep reinforcement learning. In Proceeding of the Thirty-Second AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.","DOI":"10.1609\/aaai.v32i1.11796"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"40797","DOI":"10.1109\/ACCESS.2019.2907618","article-title":"Cooperative deep q-learning with q-value transfer for multi-intersection signal control","volume":"7","author":"Ge","year":"2019","journal-title":"IEEE Access"},{"key":"ref_38","unstructured":"Schaul, T., Quan, J., Antonoglou, I., and Silver, D. (2016). Prioritized experience replay. arXiv Prepr."},{"key":"ref_39","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press."},{"key":"ref_40","unstructured":"Bellemare, M.G., Dabney, W., and Munos, R. (2017, January 6\u201311). A distributional perspective on reinforcement learning. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_41","unstructured":"Fortunato, M., Azar, M.G., Piot, B., Menick, J., Osband, I., Graves, A., Mnih, V., Munos, R., Hassabis, D., and Pietquin, O. (2017). Noisy networks for exploration. arXiv Prepr."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1109\/MRA.2017.2662222","article-title":"Autonomous surveillance robots: A decision-making framework for networked muiltiagent systems","volume":"24","author":"Witwicki","year":"2017","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_44","first-page":"128","article-title":"Recent development and applications of SUMO-simulation of urban mobility","volume":"5","author":"Krajzewicz","year":"2012","journal-title":"Int. J. Adv. Syst. Meas."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Wegener, A., Pi\u00f3rkowski, M., Raya, M., Hellbr\u00fcck, H., Fischer, S., and Hubaux, J.-P. (2008, January 14\u201317). TraCI: An interface for coupling road traffic and network simulators. Proceedings of the 11th Communications and Networking Simulation Symposium, Ottawa, ON, Canada.","DOI":"10.1145\/1400713.1400740"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/1\/137\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:45:17Z","timestamp":1760190317000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/1\/137"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,12,24]]},"references-count":45,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,1]]}},"alternative-id":["s20010137"],"URL":"https:\/\/doi.org\/10.3390\/s20010137","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,12,24]]}}}