{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T15:55:22Z","timestamp":1784130922940,"version":"3.55.0"},"reference-count":61,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2022,10,3]],"date-time":"2022-10-03T00:00:00Z","timestamp":1664755200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000266","name":"EPSRC","doi-asserted-by":"publisher","award":["EP\/R018634\/1"],"award-info":[{"award-number":["EP\/R018634\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Adaptive traffic signal control (ATSC) is an effective method to reduce traffic congestion in modern urban areas. Many studies adopted various approaches to adjust traffic signal plans according to real-time traffic in response to demand fluctuations to improve urban network performance (e.g., minimise delay). Recently, learning-based methods such as reinforcement learning (RL) have achieved promising results in signal plan optimisation. However, adopting these self-learning techniques in future traffic environments in the presence of connected and automated vehicles (CAVs) remains largely an open challenge. This study develops a real-time RL-based adaptive traffic signal control that optimises a signal plan to minimise the total queue length while allowing the CAVs to adjust their speed based on a fixed timing strategy to decrease total stop delays. The highlight of this work is combining a speed guidance system with a reinforcement learning-based traffic signal control. Two different performance measures are implemented to minimise total queue length and total stop delays. Results indicate that the proposed method outperforms a fixed timing plan (with optimal speed advisory in a CAV environment) and traditional actuated control, in terms of average stop delay of vehicle and queue length, particularly under saturated and oversaturated conditions.<\/jats:p>","DOI":"10.3390\/s22197501","type":"journal-article","created":{"date-parts":[[2022,10,10]],"date-time":"2022-10-10T05:12:21Z","timestamp":1665378741000},"page":"7501","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":45,"title":["Real-Time Adaptive Traffic Signal Control in a Connected and Automated Vehicle Environment: Optimisation of Signal Planning with Reinforcement Learning under Vehicle Speed Guidance"],"prefix":"10.3390","volume":"22","author":[{"given":"Saeed","family":"Maadi","sequence":"first","affiliation":[{"name":"Urban Big Data Centre, Department of Urban Studies, University of Glasgow, Glasgow G12 8QQ, UK"},{"name":"School of Engineering, Damghan University, Damghan 36716-41167, Iran"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1828-4008","authenticated-orcid":false,"given":"Sebastian","family":"Stein","sequence":"additional","affiliation":[{"name":"School of Computing Science, University of Glasgow, Glasgow G12 8QQ, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jinhyun","family":"Hong","sequence":"additional","affiliation":[{"name":"Smart City Department, University of Seoul, Seoul 02504, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4228-7962","authenticated-orcid":false,"given":"Roderick","family":"Murray-Smith","sequence":"additional","affiliation":[{"name":"School of Computing Science, University of Glasgow, Glasgow G12 8QQ, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,10,3]]},"reference":[{"key":"ref_1","unstructured":"(2021, January 11). Institute of Economic Affairs (IEA) Discussion Paper, No.68, 2016. SEEING RED Traffic Controls and the Economy. Available online: https:\/\/iea.org.uk\/wpcontent\/uploads\/2016\/07\/IEA%20Seeing%20Red%20%20Traffic%20Controls%20and%20the%20Economy.pdf."},{"key":"ref_2","first-page":"75","article-title":"OPAC: A demand-responsive strategy for traffic signal control","volume":"906","author":"Gartner","year":"1983","journal-title":"Transp. Res. Rec. J. Transp. Res. Board"},{"key":"ref_3","first-page":"82","article-title":"Hierarchical framework for real-time traffic control","volume":"1360","author":"Head","year":"1992","journal-title":"Transp. Res. Rec."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"175","DOI":"10.3141\/1856-19","article-title":"ACS-Lite algorithmic architecture: Applying adaptive control system technology to closed-loop traffic signal control systems","volume":"1856","author":"Luyanda","year":"2003","journal-title":"Transp. Res. Record"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"415","DOI":"10.1016\/S0968-090X(00)00047-4","article-title":"A real-time traffic signal control system: Architecture, algorithms, and analysis","volume":"9","author":"Mirchandani","year":"2001","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Ran, Q., and Yang, J. (2012, January 23\u201325). A novel closed-loop feedback traffic signal control strategy at an isolated intersection. Proceedings of the 2012 IEEE International Conference on Information Science and Technology, ICIST, Wuhan, China.","DOI":"10.1109\/ICIST.2012.6221615"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1016\/j.tra.2015.04.003","article-title":"Preparing a nation for autonomous vehicles: Opportunities, barriers and policy recommendations","volume":"77","author":"Fagnant","year":"2015","journal-title":"Transp. Res. Part A Policy Pract."},{"key":"ref_8","unstructured":"Kockelman, K., and Boyles, S. (2021, March 15). Smart Transport for Cities & Nations: The Rise of Self-Driving & Connected Vehicles. Available online: https:\/\/bit.ly\/3gDpEaa."},{"key":"ref_9","first-page":"109","article-title":"Recent advances in connected and automated vehicles","volume":"6","author":"Elliott","year":"2019","journal-title":"J. Traffic Transp. Eng."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"460","DOI":"10.1016\/j.trc.2015.01.007","article-title":"A real-time adaptive signal control in a connected vehicle environment","volume":"55","author":"Feng","year":"2015","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"127708","DOI":"10.1016\/j.physa.2022.127708","article-title":"CTM-based traffic signal optimisation of mixed traffic flow with connected automated vehicles and human-driven vehicles","volume":"603","author":"Yao","year":"2022","journal-title":"Phys. A: Stat. Mech. Its Appl."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1455","DOI":"10.1109\/TITS.2019.2909390","article-title":"A Dynamic Predictive Traffic Signal Control Framework in a Cross-Sectional Vehicle Infrastructure Integration Environment","volume":"21","author":"Yao","year":"2020","journal-title":"Proc. IEEE Trans. Intell. Transp. Syst."},{"key":"ref_13","first-page":"310","article-title":"Signalized Corridor Timing Plan Change Assessment Using Connected Vehicle Data","volume":"12","author":"Li","year":"2022","journal-title":"J. Transp. Technol."},{"key":"ref_14","first-page":"184","article-title":"A dynamic optimization method for adaptive signal control in a connected vehicle environment, Journal of Intelligent Transportation","volume":"24","author":"Yao","year":"2020","journal-title":"Systems"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1016\/j.trc.2019.01.026","article-title":"Urban traffic signal control with connected and automated vehicles: A survey","volume":"101","author":"Guo","year":"2019","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1186\/s12544-020-00458-y","article-title":"Enhanced speed advice for connected vehicles in the proximity of signalized intersections","volume":"13","author":"Mintsis","year":"2021","journal-title":"Eur. Transp. Res. Rev."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"9945398","DOI":"10.1155\/2021\/9945398","article-title":"A Two-Level Model for Traffic Signal Timing and Trajectories Planning of Multiple CAVs in a Random Environment","volume":"2021","author":"Jiang","year":"2021","journal-title":"J. Adv. Transp."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1016\/j.trc.2015.11.001","article-title":"Optimal vehicle speed trajectory on a signalised arterial with consideration of queue","volume":"61","author":"He","year":"2015","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"271","DOI":"10.1016\/j.trc.2013.11.023","article-title":"Rolling horizon control framework for driver assistance systems. Part I: Mathematical formulation and non-cooperative systems","volume":"40","author":"Wang","year":"2014","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2786","DOI":"10.1109\/TITS.2015.2422778","article-title":"Energy-optimal speed control for electric vehicles on signalised arterials","volume":"16","author":"Wu","year":"2015","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Wang, P., Chan, C.Y., and de La, F.A. (2018, January 26\u201330). A reinforcement learning-based approach for automated lane change maneuvers. Proceedings of the 2018 IEEE Intelligent Vehicles Symposium (IV), Changshu\/Suzhou, China.","DOI":"10.1109\/IVS.2018.8500556"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"981","DOI":"10.1139\/l03-014","article-title":"Reinforcement learning: Introduction to theory and potential for transport applications","volume":"30","author":"Abdulhai","year":"2003","journal-title":"Can. J. Civ. Eng."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1016\/j.ijtst.2018.04.001","article-title":"Automated intersection delay estimation using the input-output principle and turning movement data","volume":"7","author":"Shatnawi","year":"2018","journal-title":"Int. J. Transp. Sci. Technol."},{"key":"ref_24","unstructured":"(2021, March 16). Traffic Analysis Toolbox Volume III: Guidelines for Applying Traffic Microsimulation Modeling Software, Available online: https:\/\/ops.fhwa.dot.gov\/trafficanalysistools\/tat_vol3\/vol3_guidelines.pdf."},{"key":"ref_25","first-page":"190","article-title":"The SCOOT online traffic signal optimisation technique (Glasgow)","volume":"23","author":"Hunt","year":"1982","journal-title":"Traffic Eng. Control."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1109\/T-VT.1980.23833","article-title":"The Sydney Coordinated Adaptive Traffic (SCAT) System Philosophy and Benefits","volume":"29","author":"Sims","year":"1980","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_27","unstructured":"Athmaraman, N., and Soundararajan, S. (2005, January 18\u201319). Adaptive predictive traffic timer control algorithm. Proceedings of the 2005 Mid-Continent Transportation Research Symposium, Ames, IA, USA. Available online: https:\/\/www.cs.uic.edu\/~nathmara\/AthmaramanTraffic.pdf."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1096123","DOI":"10.1155\/2018\/1096123","article-title":"A Review of the Self-Adaptive Traffic Signal Control System Based on Future Traffic Environment","volume":"2018","author":"Wang","year":"2018","journal-title":"J. Adv. Transp."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Ke-Zhao, B., Rui-Xiong, C., Mu-ren, L., Ling-Jiang, K., and Rong-sen, Z. (2009). Study of the grade roundabout crossing. Comput. Sci.","DOI":"10.7498\/aps.58.4500"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1140","DOI":"10.1109\/TITS.2013.2255286","article-title":"Multiagent Reinforcement Learning for Integrated Network of Adaptive Traffic Signal Controllers Application on Downtown Toronto","volume":"14","author":"Member","year":"2013","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Jing, P., Huang, H., and Chen, L. (2017). An Adaptive Traffic Signal Control in a Connected Vehicle Environment: A Systematic Review. Information, 8.","DOI":"10.3390\/info8030101"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Wu, W., Huang, L., and Du, R. (2020). Simultaneous Optimization of Vehicle Arrival Time and Signal Timings within a Connected Vehicle Environment. Sensors, 20.","DOI":"10.3390\/s20010191"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Katsaros, K., Kernchen, R., Dianati, M., and Rieck, D. (2011, January 4\u20138). Performance study of a Green Light Optimized Speed Advisory (GLOSA) application using an integrated cooperative ITS simulation platform. Proceedings of the IWCMC 2011 7th International Wireless Communications and Mobile Computing Conference, Istanbul, Turkey.","DOI":"10.1109\/IWCMC.2011.5982524"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"399","DOI":"10.1016\/j.physa.2018.01.005","article-title":"A speed guidance strategy for multiple signalised intersections based on car-following model","volume":"496","author":"Tang","year":"2018","journal-title":"Phys. A: Stat. Mech. Its Appl."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3141\/2619-01","article-title":"Adaptive coordination based on connected vehicle technology","volume":"2619","author":"Beak","year":"2017","journal-title":"Transp. Res. Rec."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1459","DOI":"10.1109\/TVT.2013.2241460","article-title":"Adaptive traffic signal control with vehicular Ad Hoc networks","volume":"62","author":"Pandit","year":"2013","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1016\/j.trb.2018.04.007","article-title":"Integrated optimisation of traffic signals and vehicle trajectories at isolated urban intersections","volume":"112","author":"Yu","year":"2018","journal-title":"Transp. Res. Part B Methodol."},{"key":"ref_38","first-page":"P04019","article-title":"Self-control of traffic lights and vehicle flows in urban road networks","volume":"2008","author":"Helbing","year":"2008","journal-title":"J. Stat. Mech. Theory Exp."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1261","DOI":"10.1109\/TITS.2011.2156792","article-title":"Local feedback-based mainstream traffic flow control on motorways using variable speed limits","volume":"12","author":"Carlson","year":"2011","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1016\/S0967-0661(01)00121-6","article-title":"A multivariable regulator approach to traffic-responsive network\u2014Wide signal control","volume":"10","author":"Diakaki","year":"2002","journal-title":"Control Eng. Pract."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Chin, Y.K., Lee, L.K., Yang, S.S., Tze, K., and Teo, K. (2011, January 26\u201328). Exploring Q-Learning Optimisation in Traffic Signal Timing Plan Management. Proceedings of the 2011 Third International Conference on Computational Intelligence, Communication Systems and Networks, Bali, Indonesia.","DOI":"10.1109\/CICSyN.2011.64"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1086","DOI":"10.1109\/TITS.2019.2901791","article-title":"Multi-agent deep reinforcement learning for large-scale traffic signal control","volume":"21","author":"Chu","year":"2019","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Kim, D., and Jeong, O. (2019). Cooperative Traffic Signal Control with Traffic Flow Prediction in Multi-Intersection. Sensors, 20.","DOI":"10.3390\/s20010137"},{"key":"ref_44","unstructured":"Yang, K., Tan, I., and Menendez, M. (2017, January 17\u201319). A reinforcement learning-based traffic signal control algorithm in a connected vehicle environment. Proceedings of the 17th Swiss Transport Research Conference (STRC 2017), Ascona, Switzerland."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"639","DOI":"10.1016\/j.aei.2018.08.002","article-title":"Advanced Engineering Informatics Traffic signal optimisation through discrete and continuous reinforcement learning with robustness analysis in downtown Tehran","volume":"38","author":"Aslani","year":"2018","journal-title":"Adv. Eng. Inform."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Prabuchandran, K.J., Hemanth Kumar, A.N., and Bhatnagar, S. (2015, January 6\u201310). Decentralised Learning for Traffic Signal Control. Proceedings of the Intelligent Transportation System Workshop, COMSNETS, Bangalore, India.","DOI":"10.1109\/COMSNETS.2015.7098712"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Chen, J., Xue, Z., and Fan, D. (2020). Deep Reinforcement Learning Based Left-Turn Connected and Automated Vehicle Control at Signalized Intersection in Vehicle-to-Infrastructure Environment. Information, 11.","DOI":"10.3390\/info11020077"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"8667","DOI":"10.1109\/TVT.2017.2702388","article-title":"Distributed Cooperative Reinforcement Learning-Based Traffic Signal Control That Integrates V2X Networks\u2019 Dynamic Clustering","volume":"66","author":"Liu","year":"2017","journal-title":"Proc. IEEE Trans. Veh. Technol."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1049\/iet-its.2017.0153","article-title":"Traffic light control using deep policy- gradient and value-function-based reinforcement learning","volume":"11","author":"Mousavi","year":"2017","journal-title":"IET Intell. Transp. Syst."},{"key":"ref_50","unstructured":"van der Pol, E., and Oliehoek, F.A. (2016, January 5\u201310). Coordinated deep reinforcement learners for traffic light control. Proceedings of the 30th Conference on Neural Information Processing Systems, Barcelona, Spain."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Nishi, T., Otaki, K., Hayakawa, K., and Yoshimura, T. (2018, January 4\u20137). Traffic Signal Control Based on Reinforcement Learning with Graph Convolutional Neural Nets. Proceedings of the 2018 21st International Conference on Intelligent Transportation Systems (ITSC), Maui, HI, USA.","DOI":"10.1109\/ITSC.2018.8569301"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"3874","DOI":"10.1109\/TVT.2015.2506629","article-title":"Dynamic traffic signal timing optimisation strategy incorporating various vehicle fuel consumption characteristics","volume":"65","author":"Zhao","year":"2015","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"761","DOI":"10.1016\/j.trpro.2016.06.063","article-title":"Investigation of Automated Vehicle Effects on Driver\u2019s Behavior and Traffic Performance","volume":"15","author":"Aria","year":"2016","journal-title":"Transp. Res. Procedia"},{"key":"ref_54","unstructured":"ATKINS (2016). Research on the Impacts of Connected and Autonomous Vehicles (CAVs) on Traffic Flow."},{"key":"ref_55","unstructured":"Stanek, D., Huang, E., and Milam, R.T. (2018, January 7\u201311). Measuring Autonomous Vehicle Impacts on Congested Networks Using Simulation. Proceedings of the Transportation Research Board 97th Annual Meeting, Washington, DC, USA."},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Asadi, F.E., Anwar, A.K., and Miles, J.C. (2019, January 4\u20138). Investigating the potential transportation impacts of connected and autonomous vehicles. Proceedings of the 2019 8th IEEE International Conference on Connected Vehicles and Expo, ICCVE, Graz, Austria.","DOI":"10.1109\/ICCVE45908.2019.8964994"},{"key":"ref_57","unstructured":"Zeidler, V., Buck, S., Kautzsch, L., Vortisch, P., and Weyland, C. (2019, January 13\u201317). Simulation of Autonomous Vehicles Based on Wiedemann\u2019s Car Following Model in PTV Vissim. Proceedings of the Transportation Research Board 98th Annual Meeting, Washington, DC, USA."},{"key":"ref_58","unstructured":"PTV Group (2021, March 01). PTV Vissim 10 User Manual. Ptv Ag. Available online: https:\/\/usermanual.wiki\/Document\/Vissim20102020Manual.1098038624.pdf."},{"key":"ref_59","unstructured":"CoEXist (2020, January 12). Working towards a Shared Road Network. Available online: https:\/\/www.h2020-coexist.eu\/."},{"key":"ref_60","unstructured":"Council, N.R. (2000). TRB. Highway Capacity Manual, National Research Council."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"551","DOI":"10.1080\/03052150500114289","article-title":"Function-transformation methods for multi-objective optimisation","volume":"37","author":"Marler","year":"2005","journal-title":"Eng. Optim."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/19\/7501\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:45:52Z","timestamp":1760143552000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/19\/7501"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,3]]},"references-count":61,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2022,10]]}},"alternative-id":["s22197501"],"URL":"https:\/\/doi.org\/10.3390\/s22197501","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,10,3]]}}}