{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,2]],"date-time":"2026-01-02T07:33:21Z","timestamp":1767339201457,"version":"build-2065373602"},"reference-count":34,"publisher":"MDPI AG","issue":"13","license":[{"start":{"date-parts":[[2022,6,29]],"date-time":"2022-06-29T00:00:00Z","timestamp":1656460800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In the autonomous driving process, the decision-making system is mainly used to provide macro-control instructions based on the information captured by the sensing system. Learning-based algorithms have apparent advantages in information processing and understanding for an increasingly complex driving environment. To incorporate the interactive information between agents in the environment into the decision-making process, this paper proposes a generalized single-vehicle-based graph neural network reinforcement learning algorithm (SGRL algorithm). The SGRL algorithm introduces graph convolution into the traditional deep neural network (DQN) algorithm, adopts the training method for a single agent, designs a more explicit incentive reward function, and significantly improves the dimension of the action space. The SGRL algorithm is compared with the traditional DQN algorithm (NGRL) and the multi-agent training algorithm (MGRL) in the highway ramp scenario. Results show that the SGRL algorithm has outstanding advantages in network convergence, decision-making effect, and training efficiency.<\/jats:p>","DOI":"10.3390\/s22134935","type":"journal-article","created":{"date-parts":[[2022,6,29]],"date-time":"2022-06-29T22:43:28Z","timestamp":1656542608000},"page":"4935","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["Generalized Single-Vehicle-Based Graph Reinforcement Learning for Decision-Making in Autonomous Driving"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5119-0519","authenticated-orcid":false,"given":"Fan","family":"Yang","sequence":"first","affiliation":[{"name":"School of Mechanical Engineering, Beijing Institute of Technology, Beijing 100081, China"}]},{"given":"Xueyuan","family":"Li","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering, Beijing Institute of Technology, Beijing 100081, China"}]},{"given":"Qi","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering, Beijing Institute of Technology, Beijing 100081, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7056-4264","authenticated-orcid":false,"given":"Zirui","family":"Li","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering, Beijing Institute of Technology, Beijing 100081, China"},{"name":"Department of Transport and Planning, Faculty of Civil Engineering and Geosciences, Delft University of Technology, Stevinweg 1, 2628 CN Delft, The Netherlands"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7317-8059","authenticated-orcid":false,"given":"Xin","family":"Gao","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering, Beijing Institute of Technology, Beijing 100081, China"}]}],"member":"1968","published-online":{"date-parts":[[2022,6,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"294","DOI":"10.1109\/TIV.2019.2955905","article-title":"Combining Planning and Deep Reinforcement Learning in Tactical Decision Making for Autonomous Driving","volume":"5","author":"Hoel","year":"2020","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Liu, Q., Li, Z., Yuan, S., Zhu, Y., and Li, X. (2021). Review on Vehicle Detection Technology for Unmanned Ground Vehicles. Sensors, 21.","DOI":"10.3390\/s21041354"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"112953","DOI":"10.1016\/j.eswa.2019.112953","article-title":"A new safe lane-change trajectory model and collision avoidance control method for automatic driving vehicles","volume":"141","author":"Peng","year":"2020","journal-title":"Expert Syst. Appl."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Nageshrao, S., Tseng, H.E., and Filev, D. (2019, January 6\u20139). Autonomous highway driving using deep reinforcement learning. Proceedings of the 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), Bari, Italy.","DOI":"10.1109\/SMC.2019.8914621"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"4909","DOI":"10.1109\/TITS.2021.3054625","article-title":"Deep Reinforcement Learning for Autonomous Driving: A Survey","volume":"23","author":"Kiran","year":"2021","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Hoel, C.J., Wolff, K., and Laine, L. (2018, January 4\u20137). Automated speed and lane change decision making using deep reinforcement learning. Proceedings of the 2018 21st International Conference on Intelligent Transportation Systems (ITSC), Maui, HI, USA.","DOI":"10.1109\/ITSC.2018.8569568"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Gao, H., Shi, G., Xie, G., and Cheng, B. (2018). Car-following method based on inverse reinforcement learning for autonomous vehicle decision-making. Int. J. Adv. Robot. Syst., 15.","DOI":"10.1177\/1729881418817162"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"10704","DOI":"10.1109\/TIE.2022.3146549","article-title":"Personalized Driver Braking Behavior Modeling in the Car-Following Scenario: An Importance-Weight-Based Transfer Learning Approach","volume":"69","author":"Li","year":"2022","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"3281","DOI":"10.1109\/TITS.2019.2925510","article-title":"Transfer learning for driver model adaptation in lane-changing scenarios using manifold alignment","volume":"21","author":"Lu","year":"2019","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_10","first-page":"701","article-title":"Review of deep reinforcement learning and discussions on the development of computer Go","volume":"33","author":"Zhao","year":"2016","journal-title":"Control Theory Appl."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Wang, J., Zhang, Q., Zhao, D., and Chen, Y. (2019, January 14\u201319). Lane Change Decision-making through Deep Reinforcement Learning with Rule-based Constraints. Proceedings of the 2019 International Joint Conference on Neural Networks (IJCNN), Budapest, Hungary.","DOI":"10.1109\/IJCNN.2019.8852110"},{"key":"ref_12","unstructured":"Li, Y., Chen, S., Ha, P., Dong, J., Steinfeld, A., and Labi, S. (2020). Leveraging Vehicle Connectivity and Autonomy to Stabilize Flow in Mixed Traffic Conditions: Accounting for Human-driven Vehicle Driver Behavioral Heterogeneity and Perception-reaction Time Delay. arXiv."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Gong, C., Li, Z., Lu, C., Gong, J., and Hu, F. (2019, January 27\u201330). A comparative study on transferable driver behavior learning methods in the lane-changing scenario. Proceedings of the 2019 IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, New Zealand.","DOI":"10.1109\/ITSC.2019.8916986"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"70","DOI":"10.2352\/ISSN.2470-1173.2017.19.AVM-023","article-title":"Deep Reinforcement Learning framework for Autonomous Driving","volume":"2017","author":"Sallab","year":"2017","journal-title":"Electron. Imaging"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"3275","DOI":"10.1109\/TIE.2018.2840530","article-title":"Decision-Making Framework for Autonomous Driving at Road Intersections: Safeguarding Against Collision, Overly Conservative Behavior, and Violation Vehicles","volume":"66","author":"Noh","year":"2019","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Liu, Q., Li, X., Yuan, S., and Li, Z. (2021, January 19\u201322). Decision-Making Technology for Autonomous Vehicles: Learning-Based Methods, Applications and Future Outlook. Proceedings of the 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), Indianapolis, IN, USA.","DOI":"10.1109\/ITSC48978.2021.9564580"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1146\/annurev-control-060117-105157","article-title":"Planning and Decision-Making for Autonomous Vehicles","volume":"1","author":"Schwarting","year":"2018","journal-title":"Annu. Rev. Control. Robot. Auton. Syst."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"6814","DOI":"10.1109\/TVT.2018.2822762","article-title":"Humanlike Driving: Empirical Decision-Making System for Autonomous Vehicles","volume":"67","author":"Li","year":"2018","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_19","first-page":"3884","article-title":"A Reinforcement Learning Approach to Autonomous Decision Making of Intelligent Vehicles on Highways","volume":"50","author":"Xu","year":"2019","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1959013","DOI":"10.1142\/S0218001419590134","article-title":"Research on Management System of Automatic Driver Decision-Making Knowledge Base for Unmanned Vehicle","volume":"33","author":"Zhang","year":"2019","journal-title":"Int. J. Pattern Recognit. Artif. Intell."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1049\/iet-its.2019.0317","article-title":"Hierarchical reinforcement learning for self-driving decision-making without reliance on labelled driving data","volume":"14","author":"Duan","year":"2020","journal-title":"IET Intell. Transp. Syst."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Cheng, X., Jiang, R., and Chen, R. (2020, January 22\u201324). Simulation of decision-making method for vehicle longitudinal automatic driving based on deep Q neural network. Proceedings of the 2020 the 7th International Conference on Automation and Logistics (ICAL), Beijing, China.","DOI":"10.1145\/3412953.3412963"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Wang, P., Chan, C., and Fortelle, A.d.L. (2018, January 26\u201330). A Reinforcement Learning Based Approach for Automated Lane Change Maneuvers. Proceedings of the 2018 IEEE Intelligent Vehicles Symposium (IV), Changshu, China.","DOI":"10.1109\/IVS.2018.8500556"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"599","DOI":"10.1016\/j.trf.2019.02.013","article-title":"Learning to use automation: Behavioral changes in interaction with automated driving systems","volume":"62","author":"Forster","year":"2019","journal-title":"Transp. Res. Part F Traffic Psychol. Behav."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"932","DOI":"10.1080\/10447318.2018.1561792","article-title":"Human\u2013Vehicle Cooperation in Automated Driving: A Multidisciplinary Review and Appraisal","volume":"35","author":"Biondi","year":"2019","journal-title":"Int. J. Hum. Comput. Interact."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Li, Z., Gong, C., Lu, C., Gong, J., Lu, J., Xu, Y., and Hu, F. (2019, January 9\u201312). Transferable driver behavior learning via distribution adaption in the lane change scenario. Proceedings of the 2019 IEEE Intelligent Vehicles Symposium (IV), Paris, France.","DOI":"10.1109\/IVS.2019.8813781"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1016\/j.trc.2019.08.011","article-title":"Automated vehicle\u2019s behavior decision making using deep reinforcement learning and high-fidelity simulation environment","volume":"107","author":"Ye","year":"2019","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_28","first-page":"7739440","article-title":"An Automatic Driving Control Method Based on Deep Deterministic Policy Gradient","volume":"2022","author":"Zhang","year":"2022","journal-title":"Wireless Commun. Mob. Comput."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"735","DOI":"10.1109\/TITS.2019.2893683","article-title":"Distributed Multiagent Coordinated Learning for Autonomous Driving in Highways Based on Dynamic Coordination Graphs","volume":"21","author":"Yu","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Yuan, S., Zhao II, P., and Zhang III, Q. (2021, January 19\u201321). Research on automatic driving technology architecture based on cooperative vehicle-infrastructure system. Proceedings of the International Conference on Artificial Intelligence, Virtual Reality, and Visualization (AIVRV 2021), Sanya, China.","DOI":"10.1117\/12.2626758"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1339","DOI":"10.1109\/TMECH.2021.3073736","article-title":"Interactive Behavior Prediction for Heterogeneous Traffic Participants in the Urban Road: A Graph-Neural-Network-Based Multitask Learning Framework","volume":"26","author":"Li","year":"2021","journal-title":"IEEE\/ASME Trans. Mechatron."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Li, Z., Lu, C., Yi, Y., and Gong, J. (2021). A hierarchical framework for interactive behaviour prediction of heterogeneous traffic participants based on graph neural network. IEEE Trans. Intell. Transp. Syst., 1\u201313.","DOI":"10.1109\/TITS.2021.3113995"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"611","DOI":"10.1109\/TMECH.2021.3053248","article-title":"Toward Safe and Personalized Autonomous Driving: Decision-Making and Motion Control With DPF and CDT Techniques","volume":"26","author":"Huang","year":"2021","journal-title":"IEEE\/ASME Trans. Mechatron."},{"key":"ref_34","unstructured":"Dong, J., Chen, S., Ha, P., Li, Y., and Labi, S. (2020). A DRL-based Multiagent Cooperative Control Framework for CAV Networks: A Graphic Convolution Q Network. arXiv."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/13\/4935\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:40:42Z","timestamp":1760139642000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/13\/4935"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,29]]},"references-count":34,"journal-issue":{"issue":"13","published-online":{"date-parts":[[2022,7]]}},"alternative-id":["s22134935"],"URL":"https:\/\/doi.org\/10.3390\/s22134935","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2022,6,29]]}}}