{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,2]],"date-time":"2026-06-02T04:49:44Z","timestamp":1780375784514,"version":"3.54.1"},"reference-count":40,"publisher":"MDPI AG","issue":"24","license":[{"start":{"date-parts":[[2023,12,14]],"date-time":"2023-12-14T00:00:00Z","timestamp":1702512000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The complexity inherent in navigating intricate traffic environments poses substantial hurdles for intelligent driving technology. The continual progress in mapping and sensor technologies has equipped vehicles with the capability to intricately perceive their exact position and the intricate interplay among surrounding traffic elements. Building upon this foundation, this paper introduces a deep reinforcement learning method to solve the decision-making and trajectory planning problem of intelligent vehicles. The method employs a deep learning framework for feature extraction, utilizing a grid map generated from a blend of static environmental markers such as road centerlines and lane demarcations, in addition to dynamic environmental cues including vehicle positions across varied lanes, all harmonized within the Frenet coordinate system. The grid map serves as the input for the state space, and the input for the action space comprises a vector encompassing lane change timing, velocity, and vertical displacement at the lane change endpoint. To optimize the action strategy, a reinforcement learning approach is employed. The feasibility, stability, and efficiency of the proposed method are substantiated via experiments conducted in the CARLA simulator across diverse driving scenarios, and the proposed method can increase the average success rate of lane change by 6.8% and 13.1% compared with the traditional planning control algorithm and the simple reinforcement learning method.<\/jats:p>","DOI":"10.3390\/s23249819","type":"journal-article","created":{"date-parts":[[2023,12,18]],"date-time":"2023-12-18T11:28:07Z","timestamp":1702898887000},"page":"9819","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Intelligent Vehicle Decision-Making and Trajectory Planning Method Based on Deep Reinforcement Learning in the Frenet Space"],"prefix":"10.3390","volume":"23","author":[{"given":"Jiawei","family":"Wang","sequence":"first","affiliation":[{"name":"College of Automotive Engineering, Jilin University, Changchun 130022, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Liang","family":"Chu","sequence":"additional","affiliation":[{"name":"College of Automotive Engineering, Jilin University, Changchun 130022, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yao","family":"Zhang","sequence":"additional","affiliation":[{"name":"College of Automotive Engineering, Jilin University, Changchun 130022, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yabin","family":"Mao","sequence":"additional","affiliation":[{"name":"College of Automotive Engineering, Jilin University, Changchun 130022, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2068-908X","authenticated-orcid":false,"given":"Chong","family":"Guo","sequence":"additional","affiliation":[{"name":"College of Automotive Engineering, Jilin University, Changchun 130022, China"},{"name":"Changsha Automobile Innovation Research Institute, Changsha 410005, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,12,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2650","DOI":"10.1109\/TIV.2023.3260040","article-title":"Learning for Vehicle-to-Vehicle Cooperative Perception Under Lossy Communication","volume":"8","author":"Li","year":"2023","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2300177","DOI":"10.1002\/aisy.202300177","article-title":"Decision-Making and Planning Methods for Autonomous Vehicles Based on Multistate Estimations and Game Theory","volume":"1","author":"Yuan","year":"2023","journal-title":"Adv. Intell. Syst."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"104428","DOI":"10.1016\/j.autcon.2022.104428","article-title":"Sensing, perception, decision, planning and action of autonomous excavators","volume":"141","author":"Eraliev","year":"2022","journal-title":"Autom. Constr."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"14294","DOI":"10.1109\/TITS.2021.3126467","article-title":"Automated Ground Vehicle Path-Following: A Robust Energy-to-Peak Control Approach","volume":"23","author":"Zhou","year":"2022","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1826","DOI":"10.1109\/TITS.2019.2913998","article-title":"A Review of Motion Planning for Highway Autonomous Driving","volume":"21","author":"Claussmann","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"113816","DOI":"10.1016\/j.eswa.2020.113816","article-title":"Self-driving cars: A survey","volume":"165","author":"Claudine","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2162","DOI":"10.3390\/electronics11142162","article-title":"A Review on Autonomous Vehicles: Progress, Methods and Challenges","volume":"11","author":"Darsh","year":"2022","journal-title":"Electronics"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Ebrahimi Soorchaei, B., Razzaghpour, M., Valiente, R., Raftari, A., and Fallah, Y.P. (2022). High-Definition Map Representation Techniques for Automated Vehicles. Electronics, 11.","DOI":"10.20944\/preprints202209.0067.v1"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1109\/TIV.2019.2955366","article-title":"Motion Planning With Velocity Prediction and Composite Nonlinear Feedback Tracking Control for Lane-Change Strategy of Autonomous Vehicles","volume":"5","author":"Chen","year":"2020","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"3249","DOI":"10.1109\/JSYST.2021.3096932","article-title":"Decision-Making and Path Planning for Highway Autonomous Driving Based on Spatio-Temporal Lane-Change Gaps","volume":"16","author":"Feng","year":"2022","journal-title":"IEEE Syst. J."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"758","DOI":"10.1109\/TIV.2022.3161785","article-title":"Early Lane Change Prediction for Automated Driving Systems Using Multi-Task Attention-Based Convolutional Neural Networks","volume":"7","author":"Mozaffari","year":"2022","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1016\/j.trc.2019.07.002","article-title":"A data-driven lane-changing model based on deep learning","volume":"106","author":"Xie","year":"2019","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"6203","DOI":"10.1109\/TITS.2023.3248842","article-title":"Dual Transformer Based Prediction for Lane Change Intentions and Trajectories in Mixed Traffic Environment","volume":"24","author":"Gao","year":"2023","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1109\/MMUL.2021.3053774","article-title":"State Representation Learning With Adjacent State Consistency Loss for Deep Reinforcement Learning","volume":"28","author":"Zhao","year":"2021","journal-title":"IEEE Multimed."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"3169","DOI":"10.1109\/TCOMM.2022.3159836","article-title":"Self-Organizing mmWave MIMO Cell-Free Networks With Hybrid Beamforming: A Hierarchical DRL-Based Design","volume":"70","author":"Hossain","year":"2022","journal-title":"IEEE Trans. Commun."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1016\/j.neunet.2022.12.009","article-title":"Representation learning for continuous action spaces is beneficial for efficient policy learning","volume":"159","author":"Zhao","year":"2023","journal-title":"Neural Netw."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"13192","DOI":"10.1007\/s10489-022-04172-1","article-title":"Personalized lane change decision algorithm using deep reinforcement learning approach","volume":"53","author":"Li","year":"2023","journal-title":"Appl. Intell."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Chen, D., Jiang, L., Wang, Y., and Li, Z. (2020, January 1\u20133). Autonomous Driving using Safe Reinforcement Learning by Incorporating a Regret-based Human Lane-Changing Decision Model. Proceedings of the American Control Conference (ACC), Denver, CO, USA.","DOI":"10.23919\/ACC45564.2020.9147626"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"21848","DOI":"10.1109\/TITS.2022.3185255","article-title":"An Integrated Model for Autonomous Speed and Lane Change Decision-Making Based on Deep Reinforcement Learning","volume":"23","author":"Peng","year":"2022","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Shi, J., Zhang, T., Zhan, J., Chen, S., Xin, J., and Zheng, N. (2023, January 4\u20137). Efficient Lane-changing Behavior Planning via Reinforcement Learning with Imitation Learning Initialization. Proceedings of the IEEE Intelligent Vehicles Symposium (IV), Anchorage, AK, USA.","DOI":"10.1109\/IV55152.2023.10186577"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"3483","DOI":"10.1007\/s13042-023-01845-2","article-title":"An efficient planning method based on deep reinforcement learning with hybrid actions for autonomous driving on highway","volume":"14","author":"Zhang","year":"2023","journal-title":"Int. J. Mach. Learn. Cyber."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"102980","DOI":"10.1016\/j.trc.2021.102980","article-title":"Hybrid deep reinforcement learning based eco-driving for low-level connected and automated vehicles along signalized corridors","volume":"124","author":"Angah","year":"2021","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"19827","DOI":"10.1109\/TITS.2022.3161589","article-title":"SPAS: Smart Pothole-Avoidance Strategy for Autonomous Vehicles","volume":"23","author":"Raja","year":"2022","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Liu, L., Jin, S., Xue, Y., Wang, Z., Fang, C., Li, M., and Sun, Y. (2023). Qureshi. Delay-Aware Intelligent Asymmetrical Edge Control for Autonomous Vehicles with Dynamic Leading Velocity. Symmetry, 15.","DOI":"10.3390\/sym15051089"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Ghouri, U.H., Zafar, M.U., Bari, S., Khan, H., and Khan, M.U. (2019, January 6\u20137). Attitude Control of Quad-copter using Deterministic Policy Gradient Algorithms (DPGA). Proceedings of the 2019 2nd International Conference on Communication, Computing and Digital Systems (C-CODE), Islamabad, Pakistan.","DOI":"10.1109\/C-CODE.2019.8681003"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"104855","DOI":"10.1016\/j.knosys.2019.07.026","article-title":"Cooperative traffic signal control using Multi-step return and Off-policy Asynchronous Advantage Actor-Critic Graph algorithm","volume":"183","author":"Yang","year":"2019","journal-title":"Knowl.-Based Syst."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"480","DOI":"10.1109\/TIV.2022.3196396","article-title":"Verification and Validation Methods for Decision-Making and Planning of Automated Vehicles: A Review","volume":"7","author":"Ma","year":"2022","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Moghadam, M., Alizadeh, A., Tekin, E., and Elkaim, G. (2020). An End-to-end Deep Reinforcement Learning Approach for the Long-term Short-term Planning on the Frenet Space. arXiv.","DOI":"10.1109\/CASE49439.2021.9551598"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"15729","DOI":"10.1109\/TITS.2022.3145389","article-title":"Autonomous Driving on Curvy Roads Without Reliance on Frenet Frame: A Cartesian-Based Trajectory Planning Method","volume":"23","author":"Li","year":"2022","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_30","first-page":"9826","article-title":"Safe Planning and Control Under Uncertainty for Self-Driving","volume":"70","author":"Khaitan","year":"2021","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Werling, M., Ziegler, J., Kammel, S., and Thrun, S. (2010, January 3\u20138). Optimal trajectory generation for dynamic street scenarios in a Fren\u00e9t Frame. Proceedings of the IEEE International Conference on Robotics and Automation, Anchorage, AK, USA.","DOI":"10.1109\/ROBOT.2010.5509799"},{"key":"ref_32","unstructured":"Zhang, Y., Sun, H., Zhou, J., Pan, J., Hu, J., and Miao, J. (November, January 19). Optimal Vehicle Path Planning Using Quadratic Optimization for Baidu Apollo Open Platform. Proceedings of the IEEE Intelligent Vehicles Symposium (IV), Las Vegas, NV, USA."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"104120","DOI":"10.1016\/j.trc.2023.104120","article-title":"An automated driving systems data acquisition and analytics platform","volume":"151","author":"Xia","year":"2023","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"668","DOI":"10.20965\/jrm.2017.p0668","article-title":"Open Source Integrated Planner for Autonomous Navigation in Highly Dynamic Environments","volume":"29","author":"Darweesh","year":"2017","journal-title":"J. Robot. Mechatron."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"106125","DOI":"10.1016\/j.engappai.2023.106125","article-title":"A review of high-definition map creation methods for autonomous driving","volume":"122","author":"Bao","year":"2023","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_36","unstructured":"Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., and Koltun, V. (2017, January 13\u201315). CARLA: An Open Urban Driving Simulator. Proceedings of the 1st Annual Conference on Robot Learning ser. Proceedings of Machine Learning Research, Mountain View, CA, USA."},{"key":"ref_37","unstructured":"(2022, May 15). ASAM OpenDRIVE. Available online: https:\/\/www.asam.net\/standards\/detail\/opendrive\/."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"6926304","DOI":"10.1155\/2023\/6926304","article-title":"Dynamic Trajectory Planning for Automated Lane Changing Using the Quintic Polynomial Curve","volume":"2023","author":"Li","year":"2023","journal-title":"J. Adv. Transp."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"466","DOI":"10.1007\/s42154-023-00224-5","article-title":"A Trajectory Planning Method of Automatic Lane Change Based on Dynamic Safety Domain","volume":"6","author":"Wang","year":"2023","journal-title":"Automot. Innov."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1109\/TIV.2021.3117840","article-title":"An Enabling Trajectory Planning Scheme for Lane Change Collision Avoidance on Highways","volume":"8","author":"Zhang","year":"2023","journal-title":"IEEE Trans. Intell. Veh."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/24\/9819\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:38:42Z","timestamp":1760132322000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/24\/9819"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,14]]},"references-count":40,"journal-issue":{"issue":"24","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["s23249819"],"URL":"https:\/\/doi.org\/10.3390\/s23249819","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,14]]}}}