{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T16:23:06Z","timestamp":1784132586544,"version":"3.55.0"},"reference-count":64,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2023,1,15]],"date-time":"2023-01-15T00:00:00Z","timestamp":1673740800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Oakland University"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The time that a vehicle merges in a lane reduction can significantly affect passengers\u2019 safety, comfort, and energy consumption, which can, in turn, affect the global adoption of autonomous electric vehicles. In this regard, this paper analyzes how connected and automated vehicles should cooperatively drive to reduce energy consumption and improve traffic flow. Specifically, a model-free deep reinforcement learning approach is used to find the optimal driving behavior in the scenario in which two platoons are merging into one. Several metrics are analyzed, including the time of the merge, energy consumption, and jerk, etc. Numerical simulation results show that the proposed framework can reduce the energy consumed by up to 76.7%, and the average jerk can be decreased by up to 50%, all by only changing the cooperative merge behavior. The present findings are essential since reducing the jerk can decrease the longitudinal acceleration oscillations, enhance comfort and drivability, and improve the general acceptance of autonomous vehicle platooning as a new technology.<\/jats:p>","DOI":"10.3390\/s23020990","type":"journal-article","created":{"date-parts":[[2023,1,16]],"date-time":"2023-01-16T05:30:07Z","timestamp":1673847007000},"page":"990","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["Comparative Study of Cooperative Platoon Merging Control Based on Reinforcement Learning"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9695-7680","authenticated-orcid":false,"given":"Ali","family":"Irshayyid","sequence":"first","affiliation":[{"name":"Department of Electrical and Computer Engineering, Oakland University, Rochester, MI 48309, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0934-8519","authenticated-orcid":false,"given":"Jun","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, Oakland University, Rochester, MI 48309, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,1,15]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Anderson, J.M., Kalra, N., Stanley, K.D., Sorensen, P., Samaras, C., and Oluwatola, T.A. (2016). Autonomous Vehicle Technology: A Guide for Policymakers, RAND Corporation.","DOI":"10.7249\/RR443-2"},{"key":"ref_2","unstructured":"Zabat, M., Stabile, N., Farascaroli, S., and Browand, F. (1995). The Aerodynamic Performance of Platoons: A Final Report, University of California."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"(2022, September 01). Vehicle Platooning: A Brief Survey and Categorization, Volume 3: 2011 ASME\/IEEE International Conference on Mechatronic and Embedded Systems and Applications, Parts A and B, International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, 2011. Available online: https:\/\/asmedigitalcollection.asme.org\/IDETC-CIE\/proceedings-pdf\/IDETC-CIE2011\/54808\/829\/2768062\/829_1.pdf.","DOI":"10.1115\/DETC2011-47861"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"203","DOI":"10.2991\/ijndc.k.200829.001","article-title":"Vehicle Platooning Systems: Review, Classification and Validation Strategies","volume":"8","author":"Fakhfakh","year":"2020","journal-title":"Int. J. Netw. Distrib. Comput."},{"key":"ref_5","unstructured":"(2022, November 16). Zipper Merge. Available online: https:\/\/www.dot.state.mn.us\/zippermerge\/."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1109\/MWC.2016.7553038","article-title":"The Grand Cooperative Driving Challenge 2016: Boosting the introduction of cooperative automated vehicles","volume":"23","author":"Englund","year":"2016","journal-title":"IEEE Wirel. Commun."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1109\/TIV.2015.2503342","article-title":"Lane change and merge maneuvers for connected and automated vehicles: A survey","volume":"1","author":"Bevly","year":"2016","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s11432-020-3107-7","article-title":"Distributed multilane merging for connected autonomous vehicle platooning","volume":"64","author":"Wu","year":"2021","journal-title":"Sci. China Inf. Sci."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"e139","DOI":"10.1002\/itl2.139","article-title":"PMCD: Platoon-Merging approach for cooperative driving","volume":"3","author":"Paranjothi","year":"2020","journal-title":"Internet Technol. Lett."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"102663","DOI":"10.1016\/j.trc.2020.102663","article-title":"Cooperative merging control via trajectory optimization in mixed vehicular traffic","volume":"116","author":"Karimi","year":"2020","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"3790","DOI":"10.1109\/TITS.2020.3040085","article-title":"A simulation study on effects of platooning gaps on drivers of conventional vehicles in highway merging situations","volume":"23","author":"Aramrattana","year":"2022","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Su, Z., and Chen, P. (2022, January 8\u201310). Optimal Platoon Merging and Catch-up Approach for Connected Electric Vehicles. Proceedings of the 2022 American Control Conference (ACC), Atlanta, GA, USA.","DOI":"10.23919\/ACC53348.2022.9867527"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Dos Santos, T.C., Bruno, D.R., Os\u00f3rio, F.S., and Wolf, D.F. (2019, January 9\u201312). Evaluation of lane-merging approaches for connected vehicles. Proceedings of the 2019 IEEE Intelligent Vehicles Symposium (IV), Paris, France.","DOI":"10.1109\/IVS.2019.8813802"},{"key":"ref_14","first-page":"1","article-title":"Stable-Baselines3: Reliable Reinforcement Learning Implementations","volume":"22","author":"Raffin","year":"2021","journal-title":"J. Mach. Learn. Res."},{"key":"ref_15","unstructured":"Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal Policy Optimization Algorithms. arXiv."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Wu, D., Wu, J., and Wang, R. (2019, January 18\u201321). An Energy-efficient and Trust-based Formation Algorithm for Cooperative Vehicle Platooning. Proceedings of the 2019 International Conference on Computing, Networking and Communications (ICNC), Honolulu, HI, USA.","DOI":"10.1109\/ICCNC.2019.8685651"},{"key":"ref_17","unstructured":"Wang, C., and Coifman, B. (2005, January 16). The study on the effect of lane change maneuvers on a simplified car-following theory. Proceedings of the 2005 IEEE Intelligent Transportation Systems, Vienna, Austria."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Goli, M., and Eskandarian, A. (2014, January 3\u20137). Evaluation of lateral trajectories with different controllers for multi-vehicle merging in platoon. Proceedings of the 2014 International Conference on Connected Vehicles and Expo (ICCVE), Vienna, Austria.","DOI":"10.1109\/ICCVE.2014.7297633"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Liu, H., Zhuang, W., Yin, G., Tang, Z., and Xu, L. (2018, January 9\u201311). Strategy for heterogeneous vehicular platoons merging in automated highway system. Proceedings of the 2018 Chinese Control and Decision Conference (CCDC), Shenyang, China.","DOI":"10.1109\/CCDC.2018.8407590"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Dasgupta, S., Raghuraman, V., Choudhury, A., Teja, T.N., and Dauwels, J. (December, January 27). Merging and splitting maneuver of platoons by means of a novel PID controller. Proceedings of the 2017 IEEE Symposium Series on Computational Intelligence (SSCI), Honolulu, HI, USA.","DOI":"10.1109\/SSCI.2017.8280871"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"429","DOI":"10.1109\/TITS.2006.884615","article-title":"The Impact of Cooperative Adaptive Cruise Control on Traffic-Flow Characteristics","volume":"7","author":"Visser","year":"2006","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1016\/j.ifacol.2019.09.057","article-title":"Vehicle Platooning and Cooperative Merging","volume":"52","author":"Schwab","year":"2019","journal-title":"IFAC-PapersOnLine"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1038\/nature24270","article-title":"Mastering the game of Go without human knowledge","volume":"550","author":"Silver","year":"2017","journal-title":"Nature"},{"key":"ref_24","unstructured":"Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., and Graepel, T. (2017). Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm. arXiv."},{"key":"ref_25","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Tang, Y., Pan, Z., Pedrycz, W., Ren, F., and Song, X. (2022). Viewpoint-based kernel fuzzy clustering with weight information granules. IEEE Trans. Emerg. Top. Comput. Intell.","DOI":"10.1109\/TETCI.2022.3201620"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1109\/TFUZZ.2018.2883033","article-title":"Deviation-sparse fuzzy c-means with neighbor information constraint","volume":"27","author":"Zhang","year":"2018","journal-title":"IEEE Trans. Fuzzy Syst."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"5229","DOI":"10.1109\/TNNLS.2021.3069728","article-title":"Reinforcement learning-based cooperative optimal output regulation via distributed adaptive internal model","volume":"33","author":"Gao","year":"2022","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_29","first-page":"1","article-title":"Algorithms for reinforcement learning","volume":"4","year":"2010","journal-title":"Synth. Lect. Artif. Intell. Mach. Learn."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Dong, H., Dong, H., Ding, Z., and Zhang, S. (2020). ; Chang. Deep Reinforcement Learning, Springer.","DOI":"10.1007\/978-981-15-4095-0"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"5811","DOI":"10.1109\/TVT.2022.3161585","article-title":"Deep reinforcement learning aided platoon control relying on V2X information","volume":"71","author":"Lei","year":"2022","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"70","DOI":"10.2352\/ISSN.2470-1173.2017.19.AVM-023","article-title":"Deep reinforcement learning framework for autonomous driving","volume":"2017","author":"Sallab","year":"2017","journal-title":"Electron. Imaging"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Chen, J., Meng, X., and Li, Z. (2022, January 8\u201310). Reinforcement Learning-based Event-Triggered Model Predictive Control for Autonomous Vehicle Path Following. Proceedings of the 2022 American Control Conference, Atlanta, GA, USA.","DOI":"10.23919\/ACC53348.2022.9867347"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Wang, P., Chan, C.Y., and de La Fortelle, A. (2018, January 26\u201330). A reinforcement learning based approach for automated lane change maneuvers. Proceedings of the 2018 IEEE Intelligent Vehicles Symposium (IV), Changshu, China.","DOI":"10.1109\/IVS.2018.8500556"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Wang, P., and Chan, C.Y. (2017, January 16\u201319). Formulation of deep reinforcement learning architecture toward autonomous driving for on-ramp merge. Proceedings of the 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), Yokohama, Japan.","DOI":"10.1109\/ITSC.2017.8317735"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1109\/TITS.2011.2106158","article-title":"A multiple-goal reinforcement learning method for complex vehicle overtaking maneuvers","volume":"12","author":"Ngai","year":"2011","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"99059","DOI":"10.1109\/ACCESS.2020.2998015","article-title":"An intelligent path planning scheme of autonomous vehicles platoon using deep reinforcement learning on network edge","volume":"8","author":"Chen","year":"2020","journal-title":"IEEE Access"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"13340","DOI":"10.1109\/TVT.2021.3122257","article-title":"A hybrid deep reinforcement learning for autonomous vehicles smart-platooning","volume":"70","author":"Prathiba","year":"2021","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"103744","DOI":"10.1016\/j.trc.2022.103744","article-title":"Reinforcement Learning based cooperative longitudinal control for reducing traffic oscillations and improving platoon stability","volume":"141","author":"Jiang","year":"2022","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Lownes, N.E., and Machemehl, R.B. (2006, January 3\u20136). VISSIM: A multi-parameter sensitivity analysis. Proceedings of the 2006 Winter Simulation Conference, Monterey, CA, USA.","DOI":"10.1109\/WSC.2006.323241"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Segata, M., Lo Cigno, R., Hardes, T., Heinovski, J., Schettler, M., Bloessl, B., Sommer, C., and Dressler, F. (2022). Multi-Technology Cooperative Driving: An Analysis Based on PLEXE. IEEE Trans. Mob. Comput. (TMC), to appear.","DOI":"10.1109\/TMC.2022.3154643"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Hidalgo, C., Lattarulo, R., Flores, C., and P\u00e9rez Rastelli, J. (2021). Platoon merging approach based on hybrid trajectory planning and CACC strategies. Sensors, 21.","DOI":"10.3390\/s21082626"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Farag, A., Hussein, A., Shehata, O.M., Garc\u00eda, F., Tadjine, H.H., and Matthes, E. (2019, January 9\u201312). Dynamics platooning model and protocols for self-driving vehicles. Proceedings of the 2019 IEEE Intelligent Vehicles Symposium (IV), Paris, France.","DOI":"10.1109\/IVS.2019.8813864"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1109\/TIV.2018.2886677","article-title":"Platooning maneuvers in vehicular networks: A distributed and consensus-based approach","volume":"4","author":"Santini","year":"2018","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Quang Tran, D., and Bae, S.H. (2020). Proximal policy optimization through a deep reinforcement learning framework for multiple autonomous vehicles at a non-signalized intersection. Appl. Sci., 10.","DOI":"10.3390\/app10165722"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Berahman, M., Rostami-Shahrbabaki, M., and Bogenberger, K. (2022). Multi-task vehicle platoon control: A deep deterministic policy gradient approach. Future Transp., 2.","DOI":"10.3390\/futuretransp2040057"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Goli, M., and Eskandarian, A. (2019, January 10\u201312). MPC-based lateral controller with look-ahead design for autonomous multi-vehicle merging into platoon. Proceedings of the 2019 American Control Conference (ACC), Philadelphia, PA, USA.","DOI":"10.23919\/ACC.2019.8814967"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Laumond, J.P. (1998). Feedback control of a nonholonomic car-like robot. Robot Motion Planning and Control, Springer.","DOI":"10.1007\/BFb0036069"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Chen, J., and Yi, Z. (2021, January 9\u201311). Comparison of Event-Triggered Model Predictive Control for Autonomous Vehicle Path Tracking. Proceedings of the 2021 IEEE Conference on Control Technology and Applications (CCTA), San Diego, CA, USA.","DOI":"10.1109\/CCTA48906.2021.9659192"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Rajamani, R. (2011). Vehicle Dynamics and Control, Springer Science & Business Media.","DOI":"10.1007\/978-1-4614-1433-9"},{"key":"ref_51","unstructured":"Choi, J.w., and Elkaim, G.H. (2008, January 22\u201324). B\u00e9zier curves for trajectory guidance. Proceedings of the World Congress on Engineering and Computer Science, WCECS, San Francisco, CA, USA."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"679","DOI":"10.1512\/iumj.1957.6.56038","article-title":"A Markovian Decision Process","volume":"6","author":"Bellman","year":"1957","journal-title":"Indiana Univ. Math. J."},{"key":"ref_53","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1007\/BF00992696","article-title":"Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning","volume":"8","author":"Williams","year":"1992","journal-title":"Mach. Learn."},{"key":"ref_55","first-page":"1928","article-title":"Asynchronous Methods for Deep Reinforcement Learning","volume":"Volume 48","author":"Balcan","year":"2016","journal-title":"Proceedings of the 33rd International Conference on Machine Learning"},{"key":"ref_56","first-page":"1889","article-title":"Trust Region Policy Optimization","volume":"Volume 37","author":"Bach","year":"2015","journal-title":"Proceedings of the 32nd International Conference on Machine Learning"},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Chen, J., Liang, M., and Ma, X. (2021, January 7\u20139). Probabilistic Analysis of Electric Vehicle Energy Consumption Using MPC Speed Control and Nonlinear Battery Model. Proceedings of the 2021 IEEE Green Technologies Conference, Denver, CO, USA.","DOI":"10.1109\/GreenTech48523.2021.00038"},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"1727","DOI":"10.1007\/s00221-021-06093-w","article-title":"Individual motion perception parameters and motion sickness frequency sensitivity in fore-aft motion","volume":"239","author":"Irmak","year":"2021","journal-title":"Exp. Brain Res."},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"2213","DOI":"10.1121\/1.401606","article-title":"Handbook of human vibration","volume":"90","author":"Griffin","year":"1991","journal-title":"J. Acoust. Soc. Am."},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"103881","DOI":"10.1016\/j.apergo.2022.103881","article-title":"Standards for passenger comfort in automated vehicles: Acceleration and jerk","volume":"106","author":"Irmak","year":"2023","journal-title":"Appl. Ergon."},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Huang, S., and Onta\u00f1\u00f3n, S. (2022, January 15\u201318). A Closer Look at Invalid Action Masking in Policy Gradient Algorithms. Proceedings of the Thirty-Fifth International Florida Artificial Intelligence Research Society Conference, FLAIRS 2022, Hutchinson Island, Jensen Beach, FL, USA.","DOI":"10.32473\/flairs.v35i.130584"},{"key":"ref_62","unstructured":"Huang, S., Dossa, R.F.J., Raffin, A., Kanervisto, A., and Wang, W. (, 2022). The 37 implementation details of proximal policy optimization. Proceedings of the ICLR Blog Track 2023, Virtual. Available online: https:\/\/elib.dlr.de\/191986\/."},{"key":"ref_63","unstructured":"Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., and Zaremba, W. (2016). Openai gym. arXiv."},{"key":"ref_64","unstructured":"Krajzewicz, D., Hertkorn, G., Feld, C., and Wagner, P. (2002, January 28\u201330). SUMO (Simulation of Urban MObility); An open-source traffic simulation. Proceedings of the 4th Middle East Symposium on Simulation and Modelling (MESM20002), Sharjah, United Arab Emirates."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/2\/990\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:06:28Z","timestamp":1760119588000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/2\/990"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,15]]},"references-count":64,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,1]]}},"alternative-id":["s23020990"],"URL":"https:\/\/doi.org\/10.3390\/s23020990","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,15]]}}}