{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,30]],"date-time":"2026-05-30T01:39:30Z","timestamp":1780105170032,"version":"3.54.0"},"reference-count":39,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2024,4,22]],"date-time":"2024-04-22T00:00:00Z","timestamp":1713744000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Science Foundation of China","doi-asserted-by":"publisher","award":["52202495"],"award-info":[{"award-number":["52202495"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Science Foundation of China","doi-asserted-by":"publisher","award":["52202494"],"award-info":[{"award-number":["52202494"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Adaptive cruise control (ACC) enables efficient, safe, and intelligent vehicle control by autonomously adjusting speed and ensuring a safe following distance from the vehicle in front. This paper proposes a novel adaptive cruise system, namely the Safety-First Reinforcement Learning Adaptive Cruise Control (SFRL-ACC). This system aims to leverage the model-free nature and high real-time inference efficiency of Deep Reinforcement Learning (DRL) to overcome the challenges of modeling difficulties and lower computational efficiency faced by current optimization control-based ACC methods while simultaneously maintaining safety advantages and optimizing ride comfort. Firstly, we transform the ACC problem into a safe DRL formulation Constrained Markov Decision Process (CMDP) by carefully designing state, action, reward, and cost functions. Subsequently, we propose the Projected Constrained Policy Optimization (PCPO)-based ACC Algorithm SFRL-ACC, which is specifically tailored to solve the CMDP problem. PCPO incorporates safety constraints that further restrict the trust region formed by the Kullback\u2013Leibler (KL) divergence, facilitating DRL policy updates that maximize performance while keeping safety costs within their limit bounds. Finally, we train an SFRL-ACC policy and compare its computation time, traffic efficiency, ride comfort, and safety with state-of-the-art MPC-based ACC control methods. The experimental results prove the superiority of the proposed method in the aforementioned performance aspects.<\/jats:p>","DOI":"10.3390\/s24082657","type":"journal-article","created":{"date-parts":[[2024,4,22]],"date-time":"2024-04-22T07:44:09Z","timestamp":1713771849000},"page":"2657","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Adaptive Cruise Control Based on Safe Deep Reinforcement Learning"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1597-1961","authenticated-orcid":false,"given":"Rui","family":"Zhao","sequence":"first","affiliation":[{"name":"College of Automotive Engineering, Jilin University, Changchun 130025, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kui","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering, Beijing Institute of Technology, Beijing 100081, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wenbo","family":"Che","sequence":"additional","affiliation":[{"name":"College of Automotive Engineering, Jilin University, Changchun 130025, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-7824-7751","authenticated-orcid":false,"given":"Yun","family":"Li","sequence":"additional","affiliation":[{"name":"Graduate School of Information and Science Technology, The University of Tokyo, Tokyo 113-8654, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-8309-7396","authenticated-orcid":false,"given":"Yuze","family":"Fan","sequence":"additional","affiliation":[{"name":"College of Automotive Engineering, Jilin University, Changchun 130025, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4195-5033","authenticated-orcid":false,"given":"Fei","family":"Gao","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Automotive Simulation and Control, Jilin University, Changhun 130025, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,4,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1109\/TIV.2020.2991953","article-title":"Provably-correct and comfortable adaptive cruise control","volume":"6","author":"Althoff","year":"2020","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1016\/j.trc.2019.04.010","article-title":"Cooperative adaptive cruise control for connected autonomous vehicles by factoring communication-related constraints","volume":"113","author":"Wang","year":"2020","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Rezaee, H., Zhang, K., Parisini, T., and Polycarpou, M.M. (2024). Cooperative Adaptive Cruise Control in the Presence of Communication and Radar Stochastic Data Loss. IEEE Trans. Intell. Transp. Syst.","DOI":"10.1109\/TITS.2023.3335310"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"111125","DOI":"10.1016\/j.ymssp.2024.111125","article-title":"Multi-objective dynamic coordinated Adaptive Cruise Control for intelligent electric vehicle with sensors fusion","volume":"209","author":"Wu","year":"2024","journal-title":"Mech. Syst. Signal Process."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"15217","DOI":"10.1016\/j.ifacol.2020.12.2304","article-title":"Cooperative adaptive cruise control of heterogeneous vehicle platoons","volume":"53","author":"Lefeber","year":"2020","journal-title":"IFAC-PapersOnLine"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Yang, F., Li, H., Lv, M., Hu, J., Zhou, Q., and Ghosh, B.K. (2024). Enhancing Safety in Nonlinear Systems: Design and Stability Analysis of Adaptive Cruise Control. arXiv.","DOI":"10.1109\/TIV.2024.3388425"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Das, L., and Won, M. (2020). D-ACC: Dynamic adaptive cruise control for highways with on-ramps based on deep qlearning. arXiv.","DOI":"10.1109\/ICRA48506.2021.9561021"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"58443","DOI":"10.1109\/ACCESS.2020.2983149","article-title":"A survey of autonomous driving: Common practices and emerging technologies","volume":"8","author":"Yurtsever","year":"2020","journal-title":"IEEE Access"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Rout, M.K., Sain, D., Swain, S.K., and Mishra, S.K. (2016, January 3\u20135). PID controller design for cruise control system using genetic algorithm. Proceedings of the 2016 International Conference on Electrical, Electronics, and Optimization Techniques (ICEEOT), Chennai, India.","DOI":"10.1109\/ICEEOT.2016.7755502"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"7084","DOI":"10.1080\/03772063.2021.2012282","article-title":"Design and implementation of an optimized PID controller for the adaptive cruise control system","volume":"69","author":"Chaturvedi","year":"2021","journal-title":"IETE J. Res."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Bauer, K.L., and Gauterin, F. (2016). A Two-Layer Approach for Predictive Optimal Cruise Control, SAE International. SAE 2016 World Congress and Exhibition.","DOI":"10.4271\/2016-01-0634"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"11390","DOI":"10.1109\/TVT.2018.2872654","article-title":"Adaptive tube-based nonlinear MPC for economic autonomous cruise control of plug-in hybrid electric vehicles","volume":"67","author":"Sakhdari","year":"2018","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1109\/TIV.2020.3012947","article-title":"Comparison of deep reinforcement learning and model predictive control for adaptive cruise control","volume":"6","author":"Lin","year":"2020","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1631\/jzus.A0900374","article-title":"Model predictive control for adaptive cruise control with multi-objectives: Comfort, fuel-economy, safety and car-following","volume":"11","author":"Luo","year":"2010","journal-title":"J. Zhejiang Univ. Sci. A"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Stanger, T., and del Re, L. (2013, January 17\u201319). A model predictive cooperative adaptive cruise control approach. Proceedings of the 2013 American Control Conference, Washington, DC, USA.","DOI":"10.1109\/ACC.2013.6580028"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Moser, D., Waschl, H., Kirchsteiger, H., Schmied, R., and Del Re, L. (2015, January 15\u201317). Cooperative adaptive cruise control applying stochastic linear model predictive control strategies. Proceedings of the 2015 European Control Conference (ECC), Linz, Austria.","DOI":"10.1109\/ECC.2015.7331057"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"882","DOI":"10.1016\/j.conengprac.2010.03.012","article-title":"Design and implementation of parameterized adaptive cruise control: An explicit model predictive control approach","volume":"18","author":"Naus","year":"2010","journal-title":"Control Eng. Pract."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"99","DOI":"10.20485\/jsaeijae.9.3_99","article-title":"Model predictive control approach to design practical adaptive cruise control for traffic jam","volume":"9","author":"Takahama","year":"2018","journal-title":"Int. J. Automot. Eng."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Munir, U., and Junzhi, Z. (2018, January 18\u201321). Weight changing model predictive controller for adaptive cruise control with multiple objectives. Proceedings of the 2018 IEEE International Conference on Mechatronics, Robotics and Automation (ICMRA), Hefei, China.","DOI":"10.1109\/ICMRA.2018.8490547"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Nie, Z., and Farzaneh, H. (2020). Adaptive cruise control for eco-driving based on model predictive control algorithm. Appl. Sci., 10.","DOI":"10.3390\/app10155271"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"688","DOI":"10.1109\/TIE.2023.3239878","article-title":"A new adaptive cruise control considering crash avoidance for intelligent vehicle","volume":"71","author":"Zhang","year":"2023","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"3526","DOI":"10.1109\/TITS.2018.2877389","article-title":"Ecological adaptive cruise control and energy management strategy for hybrid electric vehicles based on heuristic dynamic programming","volume":"20","author":"Li","year":"2018","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Bradford, E., and Imsl, L. (2018, January 12\u201315). Stochastic nonlinear model predictive control using Gaussian processes. Proceedings of the 2018 European Control Conference (ECC), Limassol, Cyprus.","DOI":"10.23919\/ECC.2018.8550249"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"3874","DOI":"10.1109\/TMECH.2022.3146727","article-title":"Extremum-seeking-based adaptive model-free control and its application to automated vehicle path tracking","volume":"27","author":"Wang","year":"2022","journal-title":"IEEE\/ASME Trans. Mechatron."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1109\/MSP.2017.2743240","article-title":"Deep reinforcement learning: A brief survey","volume":"34","author":"Arulkumaran","year":"2017","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_26","unstructured":"Li, Y. (2017). Deep reinforcement learning: An overview. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1561\/2200000071","article-title":"An introduction to deep reinforcement learning","volume":"11","author":"Henderson","year":"2018","journal-title":"Found. Trends\u00ae Mach. Learn."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Arulkumaran, K., Deisenroth, M.P., Brundage, M., and Bharath, A.A. (2017). A brief survey of deep reinforcement learning. arXiv.","DOI":"10.1109\/MSP.2017.2743240"},{"key":"ref_29","unstructured":"Wu, J., Huang, Z., Huang, C., Hu, Z., Hang, P., Xing, Y., and Lv, C. (2021). Human-in-the-loop deep reinforcement learning with application to autonomous driving. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"374","DOI":"10.1007\/s42154-020-00113-1","article-title":"Deep reinforcement learning enabled decision-making for autonomous driving at intersections","volume":"3","author":"Li","year":"2020","journal-title":"Automot. Innov."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1109\/TIV.2023.3318070","article-title":"Recent advancements in end-to-end autonomous driving using deep learning: A survey","volume":"9","author":"Chib","year":"2023","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"4909","DOI":"10.1109\/TITS.2021.3054625","article-title":"Deep reinforcement learning for autonomous driving: A survey","volume":"23","author":"Kiran","year":"2021","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Sallab, A.E., Abdou, M., Perot, E., and Yogamani, S. (2017). Deep reinforcement learning framework for autonomous driving. arXiv.","DOI":"10.2352\/ISSN.2470-1173.2017.19.AVM-023"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Chen, J., Yuan, B., and Tomizuka, M. (2019, January 27\u201330). Model-free deep reinforcement learning for urban autonomous driving. Proceedings of the 2019 IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, New Zealand.","DOI":"10.1109\/ITSC.2019.8917306"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Das, L., and Won, M. (June, January 30). D-ACC: Dynamic Adaptive Cruise Control for Highways with Ramps Based on Deep Q-Learning. Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi\u2019an, China.","DOI":"10.1109\/ICRA48506.2021.9561021"},{"key":"ref_36","unstructured":"Das, L.C., and Won, M. (2021, January 18\u201324). Saint-acc: Safety-aware intelligent adaptive cruise control for autonomous vehicles using deep reinforcement learning. Proceedings of the International Conference on Machine Learning, Virtual."},{"key":"ref_37","unstructured":"Kakade, S., and Langford, J. (2002, January 8\u201312). Approximately optimal approximate reinforcement learning. Proceedings of the Nineteenth International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_38","unstructured":"Schulman, J., Levine, S., Abbeel, P., Jordan, M., and Moritz, P. (2015, January 6). Trust region policy optimization. Proceedings of the International Conference on Machine Learning, Lille, France."},{"key":"ref_39","unstructured":"Adaptive Cruise Control Systems Performance Requirements and Test Procedures (Intelligent Transport Systems, USA). Standard No. ISO 15622:2018. Available online: https:\/\/www.iso.org\/standard\/71515.html."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/8\/2657\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:32:12Z","timestamp":1760106732000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/8\/2657"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,22]]},"references-count":39,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2024,4]]}},"alternative-id":["s24082657"],"URL":"https:\/\/doi.org\/10.3390\/s24082657","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,22]]}}}