{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,26]],"date-time":"2026-04-26T05:40:39Z","timestamp":1777182039022,"version":"3.51.4"},"reference-count":23,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2023,1,11]],"date-time":"2023-01-11T00:00:00Z","timestamp":1673395200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Natural Science Foundation of Chongqing, China","award":["cstc2021jcyj-msxmX0792"],"award-info":[{"award-number":["cstc2021jcyj-msxmX0792"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Compared with traditional rule-based algorithms, deep reinforcement learning methods in autonomous driving are able to reduce the response time of vehicles to the driving environment and fully exploit the advantages of autopilot. Nowadays, autonomous vehicles mainly drive on urban roads and are constrained by some map elements such as lane boundaries, lane driving rules, and lane center lines. In this paper, a deep reinforcement learning approach seriously considering map elements is proposed to deal with the autonomous driving issues of vehicles following and obstacle avoidance. When the deep reinforcement learning method is modeled, an obstacle representation method is proposed to represent the external obstacle information required by the ego vehicle input, aiming to address the problem that the number and state of external obstacles are not fixed.<\/jats:p>","DOI":"10.3390\/s23020844","type":"journal-article","created":{"date-parts":[[2023,1,11]],"date-time":"2023-01-11T05:26:31Z","timestamp":1673414791000},"page":"844","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":19,"title":["Research into Autonomous Vehicles Following and Obstacle Avoidance Based on Deep Reinforcement Learning Method under Map Constraints"],"prefix":"10.3390","volume":"23","author":[{"given":"Zheng","family":"Li","sequence":"first","affiliation":[{"name":"National Key Laboratory of Vehicular Transmission, Beijing Institute of Technology, Beijing 100081, China"}]},{"given":"Shihua","family":"Yuan","sequence":"additional","affiliation":[{"name":"National Key Laboratory of Vehicular Transmission, Beijing Institute of Technology, Beijing 100081, China"}]},{"given":"Xufeng","family":"Yin","sequence":"additional","affiliation":[{"name":"National Key Laboratory of Vehicular Transmission, Beijing Institute of Technology, Beijing 100081, China"}]},{"given":"Xueyuan","family":"Li","sequence":"additional","affiliation":[{"name":"National Key Laboratory of Vehicular Transmission, Beijing Institute of Technology, Beijing 100081, China"}]},{"given":"Shouxing","family":"Tang","sequence":"additional","affiliation":[{"name":"National Key Laboratory of Vehicular Transmission, Beijing Institute of Technology, Beijing 100081, China"}]}],"member":"1968","published-online":{"date-parts":[[2023,1,11]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"114","DOI":"10.1109\/TCST.2017.2658193","article-title":"Flexible Spacing Adaptive Cruise Control Using Stochastic Model Predictive Control","volume":"26","author":"Moser","year":"2017","journal-title":"IEEE Trans. Control. Syst. Technol."},{"key":"ref_2","unstructured":"Ioannou, P., Xu, Z., Eckert, S., Clemons, D., and Sieja, T. (1993, January 15\u201317). Intelligent cruise control theory and experiment. Proceedings of the 32nd IEEE Conference on Decision and Control, San Antonio, TX, USA."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Kitazono, S., and Ohmori, H. (2006, January 18\u201321). Semi-Autonomous Adaptive Cruise Control in Mixed Traffic. Proceedings of the 2006 SICE-ICASE International Joint Conference, Busan, Republic of Korea.","DOI":"10.1109\/SICE.2006.314886"},{"key":"ref_4","unstructured":"Choi, S.B., and Hedrick, J.K. (1995, January 21\u201323). Vehicle longitudinal control using an adaptive observer for automated highway systems. Proceedings of the 1995 American Control Conference\u2014ACC\u201995, Seattle, WA, USA."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"108276","DOI":"10.1109\/ACCESS.2022.3212741","article-title":"Reliable Path Planning Algorithm Based on Improved Artificial Potential Field Method","volume":"10","author":"Luo","year":"2022","journal-title":"IEEE Access"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Xie, Z., Wu, Y., Gao, J., Song, C., Chai, W., and Xi, J. (2021, January 9\u201312). Emergency obstacle avoidance system of driverless vehicle based on model predictive control. Proceedings of the 2021 International Conference on Advanced Mechatronic Systems (ICAMechS), Tokyo, Japan.","DOI":"10.1109\/ICAMechS54019.2021.9661515"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"4733","DOI":"10.1109\/TVT.2022.3152542","article-title":"Personalized Motion Planning and Tracking Control for Autonomous Vehicles Obstacle Avoidance","volume":"71","author":"Zhang","year":"2022","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Yang, S., and Lin, Y. (2021). Development of an Improved Rapidly Exploring Random Trees Algorithm for Static Obstacle Avoidance in Autonomous Vehicles. Sensors, 21.","DOI":"10.3390\/s21062244"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Zhang, X., Zhu, T., Du, L., Hu, Y., and Liu, H. (2022). Local Path Planning of Autonomous Vehicle Based on an Improved Heuristic Bi-RRT Algorithm in Dynamic Obstacle Avoidance Environment. Sensors, 22.","DOI":"10.3390\/s22207968"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"70","DOI":"10.2352\/ISSN.2470-1173.2017.19.AVM-023","article-title":"Deep Reinforcement Learning framework for Autonomous Driving","volume":"2017","author":"Sallab","year":"2017","journal-title":"Electron. Imaging"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Chen, J., Yuan, B., and Tomizuka, M. (2019, January 27\u201330). Model-free Deep Reinforcement Learning for Urban Autonomous Driving. Proceedings of the 2019 IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, New Zealand.","DOI":"10.1109\/ITSC.2019.8917306"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s11432-020-3071-8","article-title":"Trajectory prediction of cyclist based on dynamic Bayesian network and long short-term memory model at unsignalized intersections","volume":"64","author":"Gao","year":"2021","journal-title":"Sci. China Inf. Sci."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Nishitani, I., Yang, H., Guo, R., Keshavamurthy, S., and Oguchi, K. (2020\u201331, January 31). Deep Merging: Vehicle Merging Controller Based on Deep Reinforcement Learning with Embedding Network. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France.","DOI":"10.1109\/ICRA40945.2020.9197559"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1248","DOI":"10.1109\/TITS.2011.2157145","article-title":"Cooperative Adaptive Cruise Control: A Reinforcement Learning Approach","volume":"12","author":"Desjardins","year":"2011","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"12888","DOI":"10.1109\/TVT.2021.3121281","article-title":"Game Combined Multi-Agent Reinforcement Learning Approach for UAV Assisted Offloading","volume":"70","author":"Gao","year":"2021","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Li, Z. (2021, January 25\u201327). A Hierarchical Autonomous Driving Framework Combining Reinforcement Learning and Imitation Learning. Proceedings of the 2021 International Conference on Computer Engineering and Application (ICCEA), Kunming, China.","DOI":"10.1109\/ICCEA53728.2021.00084"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Jamshidi, F., Zhang, L., and Nezhadalinaei, F. (2021, January 19\u201320). Autonomous Driving Systems: Developing an Approach based on A and Double Q-Learning. Proceedings of the 2021 7th International Conference on Web Research (ICWR), Tehran, Iran.","DOI":"10.1109\/ICWR51868.2021.9443139"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"6748","DOI":"10.1109\/LRA.2020.3011912","article-title":"Deep Reinforcement Learning for Safe Local Planning of a Ground Vehicle in Unknown Rough Terrain","volume":"5","author":"Josef","year":"2020","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"41466","DOI":"10.1109\/ACCESS.2020.2976586","article-title":"Taming an Autonomous Surface Vehicle for Path Following and Collision Avoidance Using Deep Reinforcement Learning","volume":"8","author":"Meyer","year":"2020","journal-title":"IEEE Access"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"8329","DOI":"10.1109\/TVT.2020.2996187","article-title":"Enhancing the Fuel-Economy of V2I-Assisted Autonomous Driving: A Reinforcement Learning Approach","volume":"69","author":"Liu","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"8770","DOI":"10.1109\/TITS.2021.3086033","article-title":"Model-Reference Reinforcement Learning for Collision-Free Tracking Control of Autonomous Surface Vehicles","volume":"23","author":"Zhang","year":"2021","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"8707","DOI":"10.1109\/TVT.2021.3098321","article-title":"Interpretable Decision-Making for Autonomous Vehicles at Highway On-Ramps With Latent Space Reinforcement Learning","volume":"70","author":"Wang","year":"2021","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Li, Z., Zhou, J., Li, X., Du, X., Wang, L., and Wang, Y. (2020, January 27\u201328). Continuous Control for Moving Object Tracking of Unmanned Skid-Steered Vehicle Based on Reinforcement Learning. Proceedings of the 2020 3rd IEEE International Conference on Unmanned Systems (ICUS) and the organizing committee, Harbin, China.","DOI":"10.1109\/ICUS50048.2020.9274962"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/2\/844\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:03:05Z","timestamp":1760119385000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/2\/844"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,11]]},"references-count":23,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,1]]}},"alternative-id":["s23020844"],"URL":"https:\/\/doi.org\/10.3390\/s23020844","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,11]]}}}