{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,16]],"date-time":"2026-03-16T17:21:32Z","timestamp":1773681692810,"version":"3.50.1"},"reference-count":32,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2022,3,8]],"date-time":"2022-03-08T00:00:00Z","timestamp":1646697600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"U.S. Department of Transportation","award":["69A3551747105"],"award-info":[{"award-number":["69A3551747105"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computers"],"abstract":"<jats:p>Optimizing traffic signal control (TSC) at intersections continues to pose a challenging problem, particularly for large-scale traffic networks. It has been shown in past research that it is feasible to optimize the operations of individual TSC systems or a small collection of such systems. However, it has been computationally difficult to scale these solution approaches to large networks partly due to the curse of dimensionality that is encountered as the number of intersections increases. Fortunately, recent studies have recognized the potential of exploiting advancements in deep and reinforcement learning to address this problem, and some preliminary successes have been achieved in this regard. However, facilitating such intelligent solution approaches may require large amounts of infrastructure investments such as roadside units (RSUs) and drones, to ensure that connectivity is available across all intersections in the large network. This represents an investment that may be burdensome for the road agency. As such, this study builds on recent work to present a scalable TSC model that may reduce the number of enabling infrastructure that is required. This is achieved using graph attention networks (GATs) to serve as the neural network for deep reinforcement learning. GAT helps to maintain the graph topology of the traffic network while disregarding any irrelevant information. A case study is carried out to demonstrate the effectiveness of the proposed model, and the results show much promise. The overall research outcome suggests that by decomposing large networks using fog nodes, the proposed fog-based graphic RL (FG-RL) model can be easily applied to scale into larger traffic networks.<\/jats:p>","DOI":"10.3390\/computers11030038","type":"journal-article","created":{"date-parts":[[2022,3,8]],"date-time":"2022-03-08T12:35:37Z","timestamp":1646742937000},"page":"38","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Scalable Traffic Signal Controls Using Fog-Cloud Based Multiagent Reinforcement Learning"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8511-8010","authenticated-orcid":false,"given":"Paul (Young Joun)","family":"Ha","sequence":"first","affiliation":[{"name":"Center for Connected and Automated Transportation (CCAT), Lyles School of Civil Engineering, Purdue University, West Lafayette, IN 47907, USA"}]},{"given":"Sikai","family":"Chen","sequence":"additional","affiliation":[{"name":"Center for Connected and Automated Transportation (CCAT), Lyles School of Civil Engineering, Purdue University, West Lafayette, IN 47907, USA"},{"name":"Visiting Research Fellow, Robotics Institute, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA 15213, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8403-4715","authenticated-orcid":false,"given":"Runjia","family":"Du","sequence":"additional","affiliation":[{"name":"Center for Connected and Automated Transportation (CCAT), Lyles School of Civil Engineering, Purdue University, West Lafayette, IN 47907, USA"}]},{"given":"Samuel","family":"Labi","sequence":"additional","affiliation":[{"name":"Center for Connected and Automated Transportation (CCAT), Lyles School of Civil Engineering, Purdue University, West Lafayette, IN 47907, USA"}]}],"member":"1968","published-online":{"date-parts":[[2022,3,8]]},"reference":[{"key":"ref_1","unstructured":"FHWA (2021, July 03). Traffic Congestion and Reliability: Trends and Advanced Strategies for Congestion Mitigation, Available online: https:\/\/ops.fhwa.dot.gov\/congestion_report\/executive_summary.htm."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"485","DOI":"10.1109\/TSMCC.2011.2161577","article-title":"Computational intelligence in urban traffic signal control: A survey","volume":"42","author":"Zhao","year":"2011","journal-title":"IEEE Trans. Syst. Man Cybern. Part C"},{"key":"ref_3","unstructured":"Webster, F.V. (1958). Traffic Signal Settings, H.M.S.O. Road Research Technical Paper no. 39."},{"key":"ref_4","unstructured":"Koonce, P., and Rodegerdts, L. (2008). Traffic Signal Timing Manual, Federal Highway Administration. No. FHWA-HOP-08-024."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1016\/S0191-2615(03)00015-8","article-title":"Traffic signal timing optimisation based on genetic algorithm approach, including drivers\u2019 routing","volume":"38","author":"Ceylan","year":"2004","journal-title":"Transp. Res. Part B Methodol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1016\/j.trc.2019.01.026","article-title":"Urban traffic signal control with connected and automated vehicles: A survey","volume":"101","author":"Guo","year":"2019","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"174","DOI":"10.1109\/TCYB.2020.3015811","article-title":"Large-Scale Traffic Signal Control Using a Novel Multiagent Reinforcement Learning","volume":"51","author":"Wang","year":"2021","journal-title":"IEEE Trans. Cybern."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1145\/203330.203343","article-title":"Temporal difference learning and TD-Gammon","volume":"38","author":"Tesauro","year":"1995","journal-title":"Commun. ACM"},{"key":"ref_9","unstructured":"Lin, L.-J. (1992). Reinforcement Learning for Robots Using Neural Networks, Carnegie Mellon University."},{"key":"ref_10","unstructured":"Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv."},{"key":"ref_11","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv."},{"key":"ref_12","unstructured":"Wiering, M.A. (2000). Multi-agent reinforcement learning for traffic light control. Machine Learning: Proceedings of the Seventeenth International Conference (ICML\u20192000), Morgan Kaufmann Publishers Inc."},{"key":"ref_13","first-page":"412","article-title":"Reinforcement learning with function approximation for traffic signal control","volume":"12","author":"Prashanth","year":"2010","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1086","DOI":"10.1109\/TITS.2019.2901791","article-title":"Multi-agent deep reinforcement learning for large-scale traffic signal control","volume":"21","author":"Chu","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Vinitsky, E., Parvate, K., Kreidieh, A., Wu, C., and Bayen, A. (2018, January 4\u20137). Lagrangian Control through Deep-RL: Applications to Bottleneck Decongestion. Proceedings of the IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC, 2018, Maui, HI, USA.","DOI":"10.1109\/ITSC.2018.8569615"},{"key":"ref_16","unstructured":"Ha, P.Y.J., Chen, S., Dong, J., Du, R., Li, Y., and Labi, S. (2020). Leveraging the capabilities of connected and autonomous vehicles and multi-agent reinforcement learning to mitigate highway bottleneck congestion. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"83120","DOI":"10.1109\/ACCESS.2019.2925019","article-title":"A deep reinforcement learning approach to proactive content pushing and recommendation for mobile users","volume":"7","author":"Liu","year":"2019","journal-title":"IEEE Access"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"103192","DOI":"10.1016\/j.trc.2021.103192","article-title":"Space-weighted information fusion using deep reinforcement learning: The context of tactical control of lane-changing autonomous vehicles and connectivity range assessment","volume":"128","author":"Dong","year":"2021","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Du, R., Chen, S., Dong, J., Ha PY, J., and Labi, S. (2021, January 7\u201310). GAQ-EBkSP: A DRL-based Urban Traffic Dynamic Rerouting Framework using Fog-Cloud Architecture. Proceedings of the 2021 IEEE International Smart Cities Conference (ISC2), Manchester, UK.","DOI":"10.1109\/ISC253183.2021.9562832"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1109\/JAS.2016.7508798","article-title":"Traffic signal timing via deep reinforcement learning","volume":"3","author":"Li","year":"2016","journal-title":"IEEE\/CAA J. Autom. Sin."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"106694","DOI":"10.1016\/j.asoc.2020.106694","article-title":"Real-time deep reinforcement learning based vehicle navigation","volume":"96","author":"Songsang","year":"2020","journal-title":"Appl. Soft Comput."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1007\/BF00992698","article-title":"Q-learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Mach. Learn."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1095","DOI":"10.1073\/pnas.39.10.1095","article-title":"Stochastic games","volume":"39","author":"Shapley","year":"1953","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_24","unstructured":"Wang, Y., Xu, T., Niu, X., Tan, C., Chen, E., and Xiong, H. (2020). STMARL: A Spatio-Temporal Multi-Agent Reinforcement Learning Approach for Cooperative Traffic Light Control. IEEE Trans. Mob. Comput., Available online: https:\/\/ieeexplore.ieee.org\/document\/9240060."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Wei, H., Xu, N., Zhang, H., Zheng, G., Zang, X., Chen, C., Zhang, W., Zhu, Y., Xu, K., and Li, Z. (2019, January 3\u20137). Colight: Learning network-level cooperation for traffic signal control. Proceedings of the 28th ACM International Conference on Information and Knowledge Management, Beijing, China.","DOI":"10.1145\/3357384.3357902"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Devailly, F.X., Larocque, D., and Charlin, L. (2021). IG-RL: Inductive Graph Reinforcement Learning for Massive-Scale Traffic Signal Control. IEEE Trans Intell. Transp. Syst., Available online: https:\/\/ieeexplore.ieee.org\/document\/9405489.","DOI":"10.1109\/TITS.2021.3070835"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"838","DOI":"10.1111\/mice.12702","article-title":"Graph neural network and reinforcement learning for multi-agent cooperative control of connected autonomous vehicles","volume":"36","author":"Sikai","year":"2021","journal-title":"Comput.-Aided Civ. Infrastruct. Eng."},{"key":"ref_28","unstructured":"Goodfellow, I., Bengio, Y., Courville, A., and Bengio, Y. (2016). Deep Learning, MIT Press Cambridge."},{"key":"ref_29","unstructured":"Kipf, T.N., and Welling, M. (2017, January 24\u201326). Semi-supervised classification with graph convolutional networks. Proceedings of the 5th International Conference on Learning Representations, ICLR 2017\u2014Conference Track Proceedings, Toulon, France."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Schlichtkrull, M., Kipf, T.N., Bloem, P., van den Berg, R., Titov, I., and Welling, M. (2018, January 3\u20137). Modeling relational data with graph convolutional networks. Proceedings of the European Semantic Web Conference, Crete, Greece.","DOI":"10.1007\/978-3-319-93417-4_38"},{"key":"ref_31","unstructured":"Velickovi\u0107, P., Cucurull, G., Casanova, A., Romero, A., Li\u00f2, P., and Bengio, Y. (2017). Graph attention networks. arXiv."},{"key":"ref_32","first-page":"128","article-title":"Recent Development and Applications of SUMO\u2014Simulation of Urban MObility","volume":"5","author":"Krajzewicz","year":"2012","journal-title":"Int. J. Adv. Syst. Meas."}],"container-title":["Computers"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-431X\/11\/3\/38\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:34:08Z","timestamp":1760135648000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-431X\/11\/3\/38"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,8]]},"references-count":32,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2022,3]]}},"alternative-id":["computers11030038"],"URL":"https:\/\/doi.org\/10.3390\/computers11030038","relation":{},"ISSN":["2073-431X"],"issn-type":[{"value":"2073-431X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,8]]}}}