{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T17:10:55Z","timestamp":1782321055115,"version":"3.54.5"},"reference-count":38,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2024,7,4]],"date-time":"2024-07-04T00:00:00Z","timestamp":1720051200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>Autonomous vehicles have gained extensive attention in recent years, both in academia and industry. For these self-driving vehicles, decision-making in urban environments poses significant challenges due to the unpredictable behavior of traffic participants and intricate road layouts. While existing decision-making approaches based on Deep Reinforcement Learning (DRL) show potential for tackling urban driving situations, they suffer from slow convergence, especially in complex scenarios with high mobility. In this paper, we present a new approach based on the Soft Actor-Critic (SAC) algorithm to control the autonomous vehicle to enter roundabouts smoothly and safely and ensure it reaches its destination without delay. For this, we introduce a destination vector concatenated with extracted features using Convolutional Neural Networks (CNN). To evaluate the performance of our model, we conducted extensive experiments in the CARLA simulator and compared it with the Deep Q-Network (DQN) and Proximal Policy Optimization (PPO) models. Qualitative results reveal that our model converges rapidly and achieves a high success rate in scenarios with high traffic compared to the DQN and PPO models.<\/jats:p>","DOI":"10.3390\/fi16070238","type":"journal-article","created":{"date-parts":[[2024,7,4]],"date-time":"2024-07-04T06:28:54Z","timestamp":1720074534000},"page":"238","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["Enhancing Autonomous Driving Navigation Using Soft Actor-Critic"],"prefix":"10.3390","volume":"16","author":[{"given":"Badr","family":"Elallid","sequence":"first","affiliation":[{"name":"School of Technology, Moulay Ismail University of Meknes, Meknes 50050, Morocco"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1804-6977","authenticated-orcid":false,"given":"Nabil","family":"Benamar","sequence":"additional","affiliation":[{"name":"School of Technology, Moulay Ismail University of Meknes, Meknes 50050, Morocco"},{"name":"School of Science and Engineering, Al Akhawayn University in Ifrane, P.O. Box 104, Hassan II Avenue, Ifrane 53000, Morocco"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Miloud","family":"Bagaa","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Quebec at Trois-Rivieres, Trois-Rivieres, QC G8Z 4M3, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4864-4609","authenticated-orcid":false,"given":"Yassine","family":"Hadjadj-Aoul","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Rennes, Inria, CNRS, IRISA, 35000 Rennes, France"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,7,4]]},"reference":[{"key":"ref_1","first-page":"629","article-title":"A survey on intelligent transportation systems","volume":"15","author":"Qureshi","year":"2013","journal-title":"Middle-East J. Sci. Res."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"102317","DOI":"10.1016\/j.inffus.2024.102317","article-title":"Comprehensive Systematic Review of Information Fusion Methods in Smart Cities and Urban Environments","volume":"107","author":"Fadhel","year":"2024","journal-title":"Inf. Fusion"},{"key":"ref_3","unstructured":"Be (2024, April 02). Road Traffic Injuries. Available online: https:\/\/www.who.int\/news-room\/fact-sheets\/detail\/road-traffic-injuries."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"137957","DOI":"10.1109\/ACCESS.2021.3118224","article-title":"A survey on motion prediction of pedestrians and vehicles for autonomous driving","volume":"9","author":"Gulzar","year":"2021","journal-title":"IEEE Access"},{"key":"ref_5","unstructured":"Elallid, B.B., El Alaoui, H., and Benamar, N. (2023, January 20\u201321). Deep Reinforcement Learning for Autonomous Vehicle Intersection Navigation. Proceedings of the 2023 International Conference on Innovation and Intelligence for Informatics, Computing, and Technologies (3ICT), Sakheer, Bahrain."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Sadaf, M., Iqbal, Z., Javed, A.R., Saba, I., Krichen, M., Majeed, S., and Raza, A. (2023). Connected and automated vehicles: Infrastructure, applications, security, critical challenges, and future aspects. Technologies, 11.","DOI":"10.3390\/technologies11050117"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"100023","DOI":"10.1016\/j.geits.2022.100023","article-title":"A review on cooperative perception and control supported infrastructure-vehicle system","volume":"1","author":"Yu","year":"2022","journal-title":"Green Energy Intell. Transp."},{"key":"ref_8","first-page":"100164","article-title":"Autonomous driving architectures: Insights of machine learning and deep learning algorithms","volume":"6","author":"Bachute","year":"2021","journal-title":"Mach. Learn. Appl."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"3781","DOI":"10.1109\/TIV.2023.3264658","article-title":"Multi-modal 3d object detection in autonomous driving: A survey and taxonomy","volume":"8","author":"Wang","year":"2023","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"740","DOI":"10.1109\/TITS.2020.3024655","article-title":"Survey of deep reinforcement learning for motion planning of autonomous vehicles","volume":"23","author":"Aradi","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"100348","DOI":"10.1109\/ACCESS.2022.3207759","article-title":"Review of decision-making and planning approaches in automated driving","volume":"10","author":"Garrido","year":"2022","journal-title":"IEEE Access"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"712","DOI":"10.1109\/TITS.2019.2962338","article-title":"A survey of deep learning applications to autonomous vehicle control","volume":"22","author":"Kuutti","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_13","first-page":"7366","article-title":"A Comprehensive Survey on the Application of Deep and Reinforcement Learning Approaches in Autonomous Driving","volume":"34","author":"Elallid","year":"2022","journal-title":"J. King Saud-Univ.-Comput. Inf. Sci."},{"key":"ref_14","first-page":"100986","article-title":"Triangulating the future: Developing scenarios of cyclist-automated vehicle interactions from literature, expert perspectives, and survey data","volume":"23","author":"Berge","year":"2024","journal-title":"Transp. Res. Interdiscip. Perspect."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1111\/cgf.13803","article-title":"A survey on visual traffic simulation: Models, evaluations, and applications in autonomous driving","volume":"Volume 39","author":"Chao","year":"2020","journal-title":"Computer Graphics Forum"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Yadav, P., Mishra, A., and Kim, S. (2023). A comprehensive survey on multi-agent reinforcement learning for connected and automated vehicles. Sensors, 23.","DOI":"10.3390\/s23104710"},{"key":"ref_17","unstructured":"Elallid, B.B., Abouaomar, A., Benamar, N., and Kobbane, A. (2023, January 4\u20138). Vehicles control: Collision avoidance using federated deep reinforcement learning. Proceedings of the GLOBECOM 2023-2023 IEEE Global Communications Conference, Kuala Lumpur, Malaysia."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"100104","DOI":"10.1016\/j.commtr.2023.100104","article-title":"Leveraging reinforcement learning for dynamic traffic control: A survey and challenges for field implementation","volume":"3","author":"Han","year":"2023","journal-title":"Commun. Transp. Res."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"3692","DOI":"10.1109\/TIV.2023.3274536","article-title":"Motion planning for autonomous driving: The state of the art and future perspectives","volume":"8","author":"Teng","year":"2023","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Huang, W., Braghin, F., and Wang, Z. (2019, January 4\u20136). Learning to drive via apprenticeship learning and deep reinforcement learning. Proceedings of the 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (Ictai), Portland, OR, USA.","DOI":"10.1109\/ICTAI.2019.00220"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"4423","DOI":"10.1109\/LRA.2018.2869644","article-title":"Reinforced imitation: Sample efficient deep reinforcement learning for mapless navigation by leveraging prior demonstrations","volume":"3","author":"Pfeiffer","year":"2018","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"730","DOI":"10.1109\/LRA.2021.3133591","article-title":"Goal-driven autonomous exploration through deep reinforcement learning","volume":"7","author":"Cimurs","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"104630","DOI":"10.1016\/j.robot.2024.104630","article-title":"Path planning algorithms in the autonomous driving system: A comprehensive review","volume":"174","author":"Reda","year":"2024","journal-title":"Robot. Auton. Syst."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Moon, S., Koo, S., Lim, Y., and Joo, H. (2024). Routing Control Optimization for Autonomous Vehicles in Mixed Traffic Flow Based on Deep Reinforcement Learning. Appl. Sci., 14.","DOI":"10.3390\/app14052214"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Elallid, B.B., Bagaa, M., Benamar, N., and Mrani, N. (2023, January 19\u201323). A reinforcement learning based approach for controlling autonomous vehicles in complex scenarios. Proceedings of the 2023 International Wireless Communications and Mobile Computing (IWCMC), Marrakesh, Morocco.","DOI":"10.1109\/IWCMC58020.2023.10182377"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Wang, L., Liu, J., Shao, H., Wang, W., Chen, R., Liu, Y., and Waslander, S.L. (2023). Efficient Reinforcement Learning for Autonomous Driving with Parameterized Skills and Priors. arXiv.","DOI":"10.15607\/RSS.2023.XIX.102"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"11623","DOI":"10.1109\/TITS.2023.3285442","article-title":"Deep multi-agent reinforcement learning for highway on-ramp merging in mixed traffic","volume":"24","author":"Chen","year":"2023","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"122158","DOI":"10.1016\/j.eswa.2023.122158","article-title":"A deep reinforcement learning-based approach for autonomous lane-changing velocity control in mixed flow of vehicle group level","volume":"238","author":"Wang","year":"2024","journal-title":"Expert Syst. Appl."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Wang, Z., Liu, X., and Wu, Z. (2023). Design of Unsignalized Roundabouts Driving Policy of Autonomous Vehicles Using Deep Reinforcement Learning. World Electr. Veh. J., 14.","DOI":"10.3390\/wevj14020052"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"32693","DOI":"10.1109\/ACCESS.2024.3370469","article-title":"Autonomous and Human-Driven Vehicles Interacting in a Roundabout: A Quantitative and Qualitative Evaluation","volume":"12","author":"Ferrarotti","year":"2024","journal-title":"IEEE Access"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Gan, J., Zhang, J., and Liu, Y. (2024). Research on behavioral decision at an unsignalized roundabout for automatic driving based on proximal policy optimization algorithm. Appl. Sci., 14.","DOI":"10.3390\/app14072889"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"107705","DOI":"10.1016\/j.engappai.2023.107705","article-title":"A hybrid deep learning framework for conflict prediction of diverse merge scenarios at roundabouts","volume":"130","author":"Li","year":"2024","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Qi, J., Zhou, Q., Lei, L., and Zheng, K. (2021). Federated reinforcement learning: Techniques, applications, and open challenges. arXiv.","DOI":"10.20517\/ir.2021.02"},{"key":"ref_34","unstructured":"Haarnoja, T., Tang, H., Abbeel, P., and Levine, S. (2017, January 6\u201311). Reinforcement learning with deep energy-based policies. Proceedings of the International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_35","unstructured":"Palanisamy, P. (2018). Hands-On Intelligent Agents with OpenAI Gym: Your Guide to Developing AI Agents Using Deep Reinforcement Learning, Packt Publishing."},{"key":"ref_36","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_37","unstructured":"Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., and Koltun, V. (2017, January 13\u201315). CARLA: An Open Urban Driving Simulator. Proceedings of the 1st Annual Conference on Robot Learning, Mountain View, CA, USA."},{"key":"ref_38","unstructured":"Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., and Zaremba, W. (2016). Openai gym. arXiv."}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/16\/7\/238\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:10:07Z","timestamp":1760109007000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/16\/7\/238"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,4]]},"references-count":38,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2024,7]]}},"alternative-id":["fi16070238"],"URL":"https:\/\/doi.org\/10.3390\/fi16070238","relation":{},"ISSN":["1999-5903"],"issn-type":[{"value":"1999-5903","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,4]]}}}