{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,21]],"date-time":"2026-07-21T14:29:52Z","timestamp":1784644192100,"version":"3.55.0"},"reference-count":31,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2020,9,22]],"date-time":"2020-09-22T00:00:00Z","timestamp":1600732800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["51675224"],"award-info":[{"award-number":["51675224"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["51775236"],"award-info":[{"award-number":["51775236"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"National Key Research&amp;Development Program of China","award":["2018YFB0105205"],"award-info":[{"award-number":["2018YFB0105205"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Changing lanes while driving requires coordinating the lateral and longitudinal controls of a vehicle, considering its running state and the surrounding environment. Although the existing rule-based automated lane-changing method is simple, it is unsuitable for unpredictable scenarios encountered in practice. Therefore, using a deep deterministic policy gradient (DDPG) algorithm, we propose an end-to-end method for automated lane changing based on lidar data. The distance state information of the lane boundary and the surrounding vehicles obtained by the agent in a simulation environment is denoted as the state space for an automated lane-change problem based on reinforcement learning. The steering wheel angle and longitudinal acceleration are used as the action space, and both the state and action spaces are continuous. In terms of the reward function, avoiding collision and setting different expected lane-changing distances that represent different driving styles are considered for security, and the angular velocity of the steering wheel and jerk are considered for comfort. The minimum speed limit for lane changing and the control of the agent for a quick lane change are considered for efficiency. For a one-way two-lane road, a visual simulation environment scene is constructed using Pyglet. By comparing the lane-changing process tracks of two driving styles in a simplified traffic flow scene, we study the influence of driving style on the lane-changing process and lane-changing time. Through the training and adjustment of the combined lateral and longitudinal control of autonomous vehicles with different driving styles in complex traffic scenes, the vehicles could complete a series of driving tasks while considering driving-style differences. The experimental results show that autonomous vehicles can reflect the differences in the driving styles at the time of lane change at the same speed. Under the combined lateral and longitudinal control, the autonomous vehicles exhibit good robustness to different speeds and traffic density in different road sections. Thus, autonomous vehicles trained using the proposed method can learn an automated lane-changing policy while considering safety, comfort, and efficiency.<\/jats:p>","DOI":"10.3390\/s20185443","type":"journal-article","created":{"date-parts":[[2020,9,22]],"date-time":"2020-09-22T09:40:56Z","timestamp":1600767656000},"page":"5443","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":28,"title":["End-to-End Automated Lane-Change Maneuvering Considering Driving Style Using a Deep Deterministic Policy Gradient Algorithm"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8083-6403","authenticated-orcid":false,"given":"Hongyu","family":"Hu","sequence":"first","affiliation":[{"name":"State Key Laboratory of Automotive Simulation and Control, Jilin University, Changchun 130022, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3926-1200","authenticated-orcid":false,"given":"Ziyang","family":"Lu","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Automotive Simulation and Control, Jilin University, Changchun 130022, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qi","family":"Wang","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Automotive Simulation and Control, Jilin University, Changchun 130022, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chengyuan","family":"Zheng","sequence":"additional","affiliation":[{"name":"Design School, Loughbrough University, London E20 3BS, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,9,22]]},"reference":[{"key":"ref_1","unstructured":"Singh, S. (2015). Critical Reasons for Crashes Investigated in the National Motor Vehicle Crash Causation Survey, National Center for Statistics and Analysis (NCSA)."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1016\/S0968-090X(96)00006-X","article-title":"A Microscopic Traffic Simulator for evaluation of dynamic traffic management systems","volume":"4","author":"Yang","year":"1996","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"403","DOI":"10.1016\/0191-2615(86)90012-3","article-title":"A model for the structure of lane-changing decisions","volume":"20","author":"Gipps","year":"1986","journal-title":"Transp. Res. Part B Methodol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"86","DOI":"10.3141\/1999-10","article-title":"General Lane-Changing Model MOBIL for Car-Following Models","volume":"1999","author":"Kesting","year":"2007","journal-title":"Transp. Res. Rec. J. Transp. Res. Board"},{"key":"ref_5","first-page":"17","article-title":"Autonomous Driving in Traffic: Boss and the Urban Challenge","volume":"30","author":"Urmson","year":"2009","journal-title":"AI Mag."},{"key":"ref_6","unstructured":"Bellman, R. (1957). Dynamic Programming and the Numerical Solution of Variational Problems, INFORMS."},{"key":"ref_7","unstructured":"Yu, A., Palefsky-Smith, R., and Bedi, R. (2020, July 17). Course Project Reports: Deep Reinforcement Learning for Simulated Autonomous Vehicle Control. Available online: http:\/\/cs231n.stanford.edu\/reports\/2016\/pdfs\/112_Report.pdf."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Wang, P., Chan, C.-Y., and de La Fortelle, A. (2018, January 26\u201330). A reinforcement learning based approach for automated lane change maneuvers. Proceedings of the 2018 IEEE Intelligent Vehicles Symposium (IV), Changshu, China.","DOI":"10.1109\/IVS.2018.8500556"},{"key":"ref_9","unstructured":"Watkins, C.J.C.H. (1989). Learning from Delayed Rewards. [Ph.D. Thesis, University of Cambridge]."},{"key":"ref_10","unstructured":"Kretchmar, R.M. (2002, January 14\u201318). Parallel reinforcement learning. Proceedings of the 6th World Conference on Systemics, Cybernetics, and Informatics, Orlando, FL, USA."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-Level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Van Hasselt, H., Guez, A., and Silver, D. (2016, January 12\u201317). Deep reinforcement learning with double q-learning. Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"429","DOI":"10.1038\/s41586-018-0102-6","article-title":"Vector-based navigation using grid-like representations in artificial agents","volume":"557","author":"Banino","year":"2018","journal-title":"Nature"},{"key":"ref_14","unstructured":"Finn, C., Abbeel, P., and Levine, S. (2017). Model-Agnostic meta-learning for fast adaptation of deep networks. arXiv."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1108\/17563781211255862","article-title":"A survey of inverse reinforcement learning techniques","volume":"5","author":"Gao","year":"2012","journal-title":"Int. J. Intell. Comput. Cybern."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"518","DOI":"10.1016\/j.ins.2019.09.066","article-title":"An ensemble method for inverse reinforcement learning","volume":"512","author":"Lin","year":"2020","journal-title":"Inf. Sci."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Sun, P., Yin, Y., Lin, L., and Wang, X. (2018, January 26\u201330). Human-like autonomous vehicle speed control by deep reinforcement learning with double Q-learning. Proceedings of the 2018 IEEE Intelligent Vehicles Symposium (IV), Changshu, China.","DOI":"10.1109\/IVS.2018.8500630"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Fares, A., and Gomaa, W. (2014, January 18\u201320). Freeway ramp-metering control based on reinforcement learning. Proceedings of the 11th IEEE International Conference on Control & Automation (ICCA), Taichung, Taiwan.","DOI":"10.1109\/ICCA.2014.6871097"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Yu, L., Shao, X., and Yan, X. (2017, January 5\u20138). Autonomous overtaking decision making of driverless bus based on deep Q-learning method. Proceedings of the 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO), Macau, China.","DOI":"10.1109\/ROBIO.2017.8324756"},{"key":"ref_20","unstructured":"Sallab, A.E., Abdou, M., Perot, E., and Yogamani, S. (2016). End-To-End deep reinforcement learning for lane keeping assist. arXiv."},{"key":"ref_21","unstructured":"Wang, P., and Chan, C.-Y. (2018). Autonomous ramp merge maneuver based on reinforcement learning with continuous action space. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Gu, W.-Y., Xu, X., and Yang, J. (2017, January 26\u201329). Path following with supervised deep reinforcement learning. Proceedings of the 2017 4th IAPR Asian Conference on Pattern Recognition (ACPR), Nanjing, China.","DOI":"10.1109\/ACPR.2017.30"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Liu, K., Wan, Q., and Li, Y. (2018, January 25\u201327). A deep reinforcement learning algorithm with expert demonstrations and supervised loss and its application in autonomous driving. Proceedings of the 2018 37th Chinese Control Conference (CCC), Wuhan, China.","DOI":"10.23919\/ChiCC.2018.8482790"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Wang, P., Li, H., and Chan, C.-Y. (2019, January 9\u201312). Continuous control for automated lane change behavior based on deep deterministic policy gradient algorithm. Proceedings of the 2019 IEEE Intelligent Vehicles Symposium (IV), Paris, France.","DOI":"10.1109\/IVS.2019.8813903"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Pan, X., You, Y., Wang, Z., and Lu, C. (2017). Virtual to real reinforcement learning for autonomous driving. arXiv.","DOI":"10.5244\/C.31.11"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Jaritz, M., De Charette, R., Toromanoff, M., Perot, E., and Nashashibi, F. (2018, January 21\u201325). End-To-End race driving with deep reinforcement learning. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.","DOI":"10.1109\/ICRA.2018.8460934"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"An, H., and Jung, J.-I. (2019). Decision-Making System for Lane Change Using Deep Reinforcement Learning in Connected and Automated Driving. Electronics, 8.","DOI":"10.3390\/electronics8050543"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"102662","DOI":"10.1016\/j.trc.2020.102662","article-title":"Safe, efficient, and comfortable velocity control based on reinforcement learning for autonomous driving","volume":"117","author":"Zhu","year":"2020","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"24611","DOI":"10.1109\/ACCESS.2019.2898689","article-title":"A Point Cloud-Based Robust Road Curb Detection and Tracking Method","volume":"7","author":"Wang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_30","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An introduction, MIT press."},{"key":"ref_31","unstructured":"Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/18\/5443\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:12:32Z","timestamp":1760177552000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/18\/5443"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,22]]},"references-count":31,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2020,9]]}},"alternative-id":["s20185443"],"URL":"https:\/\/doi.org\/10.3390\/s20185443","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,9,22]]}}}