{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T06:42:31Z","timestamp":1777704151111,"version":"3.51.4"},"reference-count":29,"publisher":"SAGE Publications","issue":"3","license":[{"start":{"date-parts":[[2019,12,17]],"date-time":"2019-12-17T00:00:00Z","timestamp":1576540800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Journal of Intelligent &amp; Fuzzy Systems"],"published-print":{"date-parts":[[2020,3,4]]},"abstract":"<jats:p>Aiming at the problem of mainline congestion and ramp queue spill in urban expressway, an optimized on-ramp control method based on reinforcement learning is put forward. Online reinforcement learning algorithm is used to optimize on-ramp control regulation by taking the metering rate as the action, the length of ramp queue, the throughput and the occupancy rate of the interweaving area as the state, and the volume of the road network as the reward function. By iterating the value function with actual behavior, the proposed method can avoid the establishment of an accurate traffic model and the reliance on prior knowledge. Meanwhile, the real-time update of the value function Q compensates for the defect of control hysteresis. Compared with the classical method in simulation scenarios of Nanjing Kazimen Expressway, the average delay of the proposed method is reduced by 16.83%, the total delay reduced by 15.83%, the average speed enhanced by 6.80%, and the total travel time decreased by 5.22%; the average queue length of the on-ramp decreased by 89%; the average occupancy rate of the weaving area is decreased by 2.42% at rush hours, and the average traffic volume increased by 109veh\/h.<\/jats:p>","DOI":"10.3233\/jifs-179556","type":"journal-article","created":{"date-parts":[[2019,12,20]],"date-time":"2019-12-20T11:48:33Z","timestamp":1576842513000},"page":"2703-2715","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":9,"title":["An optimized on-ramp metering method for urban expressway based on reinforcement learning"],"prefix":"10.1177","volume":"38","author":[{"given":"Gan","family":"Chai","sequence":"first","affiliation":[{"name":"Intelligent Transportation System Research Center, Southeast University, Nanjing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jinde","family":"Cao","sequence":"additional","affiliation":[{"name":"School of Mathematics, and Research Center for Complex Systems and Network Sciences, Southeast University, Nanjing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shaosheng","family":"Xu","sequence":"additional","affiliation":[{"name":"School of Automation, Southeast University, Nanjing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2019,12,17]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1049\/iet-its.2014.0207"},{"key":"e_1_3_2_3_2","unstructured":"National Research Council.: Highway Capacity Manual 2010 Transportation Research Board Washington D.C. USA 2010."},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0965-8564(01)00017-9"},{"issue":"12","key":"e_1_3_2_5_2","first-page":"2043","article-title":"Review of road traffic control strategies","volume":"91","author":"Papageorgiou M.","year":"2003","unstructured":"PapageorgiouM., DiakakiC., DinopoulouV., KotsialosA. and WangY.B., Review of road traffic control strategies, Proceedings of the IEEE 2003, 91(12) (2003), 2043\u20132067.","journal-title":"Proceedings of the IEEE 2003"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.trc.2006.08.001"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2002.806803"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2011.2164792"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12205-014-0613-0"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.3141\/2012-06"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.3141\/2470-02"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2007.908724"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.3141\/2178-02"},{"key":"e_1_3_2_14_2","first-page":"1723","article-title":"A Genetic-Fuzzy Control Application to Ramp Metering and Variable Speed Limit Control","author":"Ghods A.H.","year":"2007","unstructured":"GhodsA.H., KianA. and TabibiM.A., A Genetic-Fuzzy Control Application to Ramp Metering and Variable Speed Limit Control, IEEE Transactions On Intelligent Transportation Systems, 2007, 1723\u20131728.","journal-title":"IEEE Transactions On Intelligent Transportation Systems"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.arcontrol.2012.03.004"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1139\/l03-014"},{"key":"e_1_3_2_17_2","doi-asserted-by":"crossref","unstructured":"JiX. and HeZ. An optimal control method for expressways entering ramps metering based on Q-learning 2nd International Conference on Intelligent Computation Technology and Automation (ICTC\u201909) Changsha Hunan October 2009 (China) 739\u2013741.","DOI":"10.1109\/ICICTA.2009.184"},{"key":"e_1_3_2_18_2","doi-asserted-by":"crossref","unstructured":"DavarynejadM. HegyiA. VranckenJ. and BergJ.V.D. Motorway ramp-metering control with queuing consideration using Q-learning. 14th International IEEE Conference on Intelligent Transportation Systems (ITSC) Washington D.C. USA (2011) 1652\u20131658.","DOI":"10.1109\/ITSC.2011.6082976"},{"key":"e_1_3_2_19_2","doi-asserted-by":"crossref","unstructured":"RezaeeK. AbdulhaiB. and AbdelgawadH. Application of reinforcement learning with continuous state space to ramp metering in real-world conditions. 15th International IEEE Conference on Intelligent Transportation Systems (ITSC) Anchorage AK USA September (2012) 1590\u20131595.","DOI":"10.1109\/ITSC.2012.6338837"},{"key":"e_1_3_2_20_2","unstructured":"Greguri\u0107M. Ne\u010doskaD.K. and IvanjkoE. Application of Q-learning to ramp metering in cases of significant changes in traffic demand. Autonomic Road Transport Support systems: Early Career Researcher Conference La Valletta Malta 2015."},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1515\/cait-2015-0019"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.7307\/ptt.v28i4.1830"},{"key":"e_1_3_2_23_2","doi-asserted-by":"crossref","unstructured":"WangX.J. XiX.M. and GaoG.F. Reinforcement learning ramp metering without complete information Journal of Control Science and Engineering 2012 (2012) 2.","DOI":"10.1155\/2012\/208456"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1049\/iet-its.2009.0096"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.sbspro.2014.01.146"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.engappai.2013.06.016"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.3141\/2047-04"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.trpro.2017.03.050"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1061\/(ASCE)TE.1943-5436.0000234"},{"key":"e_1_3_2_30_2","volume-title":"Transportation Research Board 94th Annual Meeting","author":"Rezaee K.","year":"2015","unstructured":"RezaeeK., AbdulhaiB. and AbdelgawadH., Decentralized coordinated optimal ramp metering: Application to the Gardiner Expressway in Downtown Toronto. Transportation Research Board 94th Annual Meeting, Washington, D.C, USA 2015."}],"container-title":["Journal of Intelligent &amp; Fuzzy Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.3233\/JIFS-179556","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.3233\/JIFS-179556","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.3233\/JIFS-179556","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T09:40:33Z","timestamp":1777455633000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.3233\/JIFS-179556"}},"subtitle":[],"editor":[{"given":"Ahmed","family":"Farouk","sequence":"additional","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2019,12,17]]},"references-count":29,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2020,3,4]]}},"alternative-id":["10.3233\/JIFS-179556"],"URL":"https:\/\/doi.org\/10.3233\/jifs-179556","relation":{},"ISSN":["1064-1246","1875-8967"],"issn-type":[{"value":"1064-1246","type":"print"},{"value":"1875-8967","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,12,17]]}}}