{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T17:14:07Z","timestamp":1784135647871,"version":"3.55.0"},"reference-count":34,"publisher":"MDPI AG","issue":"24","license":[{"start":{"date-parts":[[2023,12,15]],"date-time":"2023-12-15T00:00:00Z","timestamp":1702598400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Research Foundation of Korea (NRF)","award":["NRF-2021R1A5A1032937"],"award-info":[{"award-number":["NRF-2021R1A5A1032937"]}]},{"DOI":"10.13039\/501100014188","name":"Korean government (MSIT)","doi-asserted-by":"publisher","award":["NRF-2021R1A5A1032937"],"award-info":[{"award-number":["NRF-2021R1A5A1032937"]}],"id":[{"id":"10.13039\/501100014188","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>As autonomous vehicles (AVs) are advancing to higher levels of autonomy and performance, the associated technologies are becoming increasingly diverse. Lane-keeping systems (LKS), corresponding to a key functionality of AVs, considerably enhance driver convenience. With drivers increasingly relying on autonomous driving technologies, the importance of safety features, such as fail-safe mechanisms in the event of sensor failures, has gained prominence. Therefore, this paper proposes a reinforcement learning (RL) control method for lane-keeping, which uses surrounding object information derived through LiDAR sensors instead of camera sensors for LKS. This approach uses surrounding vehicle and object information as observations for the RL framework to maintain the vehicle\u2019s current lane. The learning environment is established by integrating simulation tools, such as IPG CarMaker, which incorporates vehicle dynamics, and MATLAB Simulink for data analysis and RL model creation. To further validate the applicability of the LiDAR sensor data in real-world settings, Gaussian noise is introduced in the virtual simulation environment to mimic sensor noise in actual operational conditions.<\/jats:p>","DOI":"10.3390\/s23249843","type":"journal-article","created":{"date-parts":[[2023,12,18]],"date-time":"2023-12-18T11:28:07Z","timestamp":1702898887000},"page":"9843","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["A Deep Reinforcement Learning Strategy for Surrounding Vehicles-Based Lane-Keeping Control"],"prefix":"10.3390","volume":"23","author":[{"given":"Jihun","family":"Kim","sequence":"first","affiliation":[{"name":"Graduate School of Automotive Engineering, Kookmin University, Seoul 02707, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-8201-9246","authenticated-orcid":false,"given":"Sanghoon","family":"Park","sequence":"additional","affiliation":[{"name":"Graduate School of Automotive Engineering, Kookmin University, Seoul 02707, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2542-0234","authenticated-orcid":false,"given":"Jeesu","family":"Kim","sequence":"additional","affiliation":[{"name":"Departments of Cogno-Mechatronics Engineering and Optics and Mechatronics Engineering, Pusan National University, Busan 46241, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1025-3784","authenticated-orcid":false,"given":"Jinwoo","family":"Yoo","sequence":"additional","affiliation":[{"name":"Department of Automobile and IT Convergence, Kookmin University, Seoul 02707, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,12,15]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Biggi, G., and Stilgoe, J. (2021). Artificial intelligence in self-driving cars research and innovation: A scientometric and bibliometric analysis. SSRN Electron. J.","DOI":"10.2139\/ssrn.3829897"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"362","DOI":"10.1002\/rob.21918","article-title":"A survey of deep learning techniques for autonomous driving","volume":"37","author":"Grigorescu","year":"2020","journal-title":"J. Field Robot."},{"key":"ref_3","unstructured":"Behringer, R., Sundareswaran, S., Gregory, B., Elsley, R., Addison, B., Guthmiller, W., Daily, R., and Bevly, D. (2004, January 14\u201317). The DARPA grand challenge-development of an autonomous vehicle. Proceedings of the IEEE Intelligent Vehicles Symposium, Parma, Italy."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"208","DOI":"10.1016\/j.ijtst.2017.07.008","article-title":"Advancements, prospects, and impacts of automated driving systems","volume":"6","author":"Chan","year":"2017","journal-title":"Int. J. Transp. Sci. Technol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"101232","DOI":"10.1016\/j.techsoc.2020.101232","article-title":"Autonomous vehicles: US regulatory policy challenges","volume":"61","author":"Hemphill","year":"2020","journal-title":"Technol. Soc."},{"key":"ref_6","unstructured":"(2012). Road Vehicles\u2014Functional Safety\u2014Part 10: Guideline on ISO 26262 (Standard No. ISO 26262-10:2012)."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Chen, J., Zhang, S., and Zhou, S. (2023, January 4\u20136). Analysis of automatic emergency braking system performance insufficiency based on system theory process analysis. Proceedings of the 2023 IEEE International Conference on Industrial Technology (ICIT), Orlando, FL, USA.","DOI":"10.1109\/ICIT58465.2023.10143171"},{"key":"ref_8","unstructured":"(2013). Intelligent Transport Systems\u2013Lane Keeping Assistance Systems (LKAS)\u2013Performance Requirements and Test Procedures (Standard No. ISO DIS 11270)."},{"key":"ref_9","first-page":"174","article-title":"Development of Lidar-based MRM algorithm for LKS systems","volume":"20","author":"Son","year":"2021","journal-title":"Korean ITS J."},{"key":"ref_10","unstructured":"(2023, December 15). UNECE. Available online: http:\/\/www.unece.org\/trans\/main\/wp29\/faq.html."},{"key":"ref_11","unstructured":"(2020). GRVA-06-02-Rev.4 Proposal for a New UN Regulation on ALKS, GRVA."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"735","DOI":"10.1109\/THMS.2017.2674998","article-title":"Takeover requests in simulated partially autonomous vehicles considering human factors","volume":"47","author":"Kim","year":"2017","journal-title":"IEEE Trans. Hum.-Mach. Syst."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Magdici, S., and Matthias, A. (2016, January 1\u20134). Fail-safe motion planning of autonomous vehicles. Proceedings of the 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC), Rio de Janeiro, Brazil.","DOI":"10.1109\/ITSC.2016.7795594"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"23573","DOI":"10.1109\/TITS.2022.3201074","article-title":"Responses to take-over request in autonomous vehicles: Effects of environmental conditions and cues","volume":"23","author":"Heo","year":"2022","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_15","unstructured":"Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"103799","DOI":"10.1016\/j.engappai.2020.103799","article-title":"Trajectory based lateral control: A reinforcement learning case study","volume":"94","author":"Wasala","year":"2020","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Feh\u00e9r, \u00c1., Aradi, S., and B\u00e9csi, T. (2022). Online trajectory planning with reinforcement learning for pedestrian avoidance. Electronics, 11.","DOI":"10.3390\/electronics11152346"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"684","DOI":"10.1109\/TIV.2020.3003524","article-title":"Methods and models for simulating autonomous vehicle sensors","volume":"5","author":"Elmquist","year":"2020","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"3553","DOI":"10.1007\/s11042-021-11437-3","article-title":"Deep reinforcement learning based control for Autonomous Vehicles in CARLA","volume":"81","author":"Barea","year":"2022","journal-title":"Multimed. Tools Appl."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"123","DOI":"10.7467\/KSAE.2022.30.2.123","article-title":"Path-following correction control algorithm using vehicle state errors","volume":"30","author":"Lee","year":"2022","journal-title":"Trans. Korean Soc. Automot. Eng."},{"key":"ref_21","first-page":"35","article-title":"A review of some pure-pursuit based path tracking techniques for control of autonomous vehicle","volume":"135","author":"Samuel","year":"2016","journal-title":"Int. J. Comput. Appl."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"646","DOI":"10.1049\/itr2.12051","article-title":"Review and performance evaluation of path tracking controllers of autonomous vehicles","volume":"15","author":"Rokonuzzaman","year":"2021","journal-title":"IET Intell. Transp. Syst."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Isermann, R. (2011). Fault-Diagnosis Applications: Model-Based Condition Monitoring: Actuators, Drives, Machinery, Plants, Sensors, and Fault-tolerant Systems, Springer.","DOI":"10.1007\/978-3-642-12767-0"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Realpe, M., Vintimilla, B.X., and Vlacic, L. (2016, January 27\u201329). A fault tolerant perception system for autonomous vehicles. Proceedings of the 2016 35th Chinese Control Conference (CCC), Chengdu, China.","DOI":"10.1109\/ChiCC.2016.7554385"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1293","DOI":"10.1007\/s12555-017-0449-8","article-title":"Kinematics-based fault-tolerant techniques: Lane prediction for an autonomous lane keeping system","volume":"16","author":"Kang","year":"2018","journal-title":"Int. J. Control. Autom. Syst."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"712","DOI":"10.1109\/TITS.2019.2962338","article-title":"A survey of deep learning applications to autonomous vehicle control","volume":"22","author":"Kuutti","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Kuutti, S., Bowden, R., and Fallah, S. (2021). Weakly supervised reinforcement learning for autonomous highway driving via virtual safety cages. Sensors, 21.","DOI":"10.3390\/s21062032"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"128837","DOI":"10.1109\/ACCESS.2019.2939201","article-title":"A survey of deep learning-based object detection","volume":"7","author":"Jiao","year":"2019","journal-title":"IEEE Access"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Alaba, S.Y., and Ball, J.E. (2022). Ball. A survey on deep-learning-based lidar 3d object detection for autonomous driving. Sensors, 22.","DOI":"10.36227\/techrxiv.20442858.v2"},{"key":"ref_30","first-page":"36","article-title":"A comparison of ppo, td3 and sac reinforcement algorithms for quadruped walking gait generation","volume":"15","author":"Mock","year":"2023","journal-title":"J. Intell. Learn. Syst. Appl."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Riedmiller, M., Montemerlo, M., and Dahlkamp, H. (2007, January 11\u201313). Learning to drive a real car in 20 minutes. Proceedings of the 2007 Frontiers in the Convergence of Bioscience and Information Technologies, Jeju, Republic of Korea.","DOI":"10.1109\/FBIT.2007.37"},{"key":"ref_32","unstructured":"IPG Automotive GmbH (2019, March 13). Carmaker: Virtual Testing of Automobiles and Light-Duty Vehicles. Available online: https:\/\/ipg-automotive.com\/en\/products-solutions\/software\/carmaker\/#driver%20."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Cao, Y., Ni, K., Jiang, X., Kuroiwa, T., Zhang, H., Kawaguchi, T., Hashimoto, S., and Jiang, W. (2023). Path following for Autonomous Ground Vehicle Using DDPG Algorithm: A Reinforcement Learning Approach. Appl. Sci., 13.","DOI":"10.3390\/app13116847"},{"key":"ref_34","unstructured":"DDPG Agents\u2014MATLAB & Simulink (2023, November 11). MathWorks. Available online: https:\/\/kr.mathworks.com\/help\/reinforcement-learning\/ug\/ddpg-agents.html?lang=en."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/24\/9843\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:39:39Z","timestamp":1760132379000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/24\/9843"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,15]]},"references-count":34,"journal-issue":{"issue":"24","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["s23249843"],"URL":"https:\/\/doi.org\/10.3390\/s23249843","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,15]]}}}