{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,8]],"date-time":"2026-07-08T22:31:08Z","timestamp":1783549868844,"version":"3.55.0"},"reference-count":54,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2022,6,14]],"date-time":"2022-06-14T00:00:00Z","timestamp":1655164800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>With the rapid development of autonomous driving technology, both self-driven and human-driven vehicles will share roads in the future and complex information exchange among vehicles will be required. Therefore, autonomous vehicles need to behave as similar to human drivers as possible, to ensure that their behavior can be effectively understood by the drivers of other vehicles and be more in line with the cognition of humans on driving behavior. Therefore, this paper studies the evaluation function of human drivers, using the method of inverse reinforcement learning, aiming for the learned behavior to better imitate the behavior of human drivers. At the same time, this paper proposes a semi-Markov model, to extract the intentions of surrounding related vehicles and divides them into defensive and cooperative, leading the vehicle to adopt a reasonable response to different types of driving scenarios.<\/jats:p>","DOI":"10.3390\/s22124500","type":"journal-article","created":{"date-parts":[[2022,6,15]],"date-time":"2022-06-15T01:39:54Z","timestamp":1655257194000},"page":"4500","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Human-like Decision Making for Autonomous Vehicles at the Intersection Using Inverse Reinforcement Learning"],"prefix":"10.3390","volume":"22","author":[{"given":"Zheng","family":"Wu","sequence":"first","affiliation":[{"name":"Vehicle Engineering Department, Inner Mongolia Technical College of Mechanics and Electrics, Huhehot 010070, China"},{"name":"School of Mechanical and Engineering, Beijing Institute of Technology, Beijing 100081, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7208-849X","authenticated-orcid":false,"given":"Fangbing","family":"Qu","sequence":"additional","affiliation":[{"name":"School of Mechanical and Engineering, Beijing Institute of Technology, Beijing 100081, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lin","family":"Yang","sequence":"additional","affiliation":[{"name":"School of Mechanical and Engineering, Beijing Institute of Technology, Beijing 100081, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jianwei","family":"Gong","sequence":"additional","affiliation":[{"name":"School of Mechanical and Engineering, Beijing Institute of Technology, Beijing 100081, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,6,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"4178","DOI":"10.1109\/TVT.2021.3073407","article-title":"A human-like model to understand surrounding vehicles\u2019 lane changing intentions for autonomous driving","volume":"70","author":"Xia","year":"2021","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Codevilla, F., M\u00fcller, M., L\u00f3pez, A., Koltun, V., and Dosovitskiy, A. (2018, January 21\u201325). End-to-end driving via conditional imitation learning. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, QLD, Australia.","DOI":"10.1109\/ICRA.2018.8460487"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"525","DOI":"10.1177\/0018720816644364","article-title":"Human\u2013robot interaction: Status and challenges","volume":"58","author":"Sheridan","year":"2016","journal-title":"Hum. Factors"},{"key":"ref_4","unstructured":"Muir, H. (2022, May 12). Google Self-Driving Car Caught on Video Colliding with Bus. The Guardian, Available online: https:\/\/www.theguardian.com\/technology\/2016\/mar\/09\/google-self-driving-car-crash-video-accident-bus."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"6814","DOI":"10.1109\/TVT.2018.2822762","article-title":"Humanlike driving: Empirical decision-making system for autonomous vehicles","volume":"67","author":"Li","year":"2018","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1016\/j.iatssr.2016.11.002","article-title":"Human-like motion planning model for driving in signalized intersections","volume":"41","author":"Gu","year":"2017","journal-title":"IATSS Res."},{"key":"ref_7","unstructured":"Emuna, R., Borowsky, A., and Biess, A. (2020). Deep reinforcement learning for human-like driving policies in collision avoidance tasks of self-driving cars. arXiv."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2076","DOI":"10.1109\/TITS.2020.3036984","article-title":"Human-like decision making for autonomous driving: A noncooperative game theoretic approach","volume":"22","author":"Hang","year":"2020","journal-title":"IEEE Trans. Intell. Transp."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Hang, P., Zhang, Y., and Lv, C. (2022). Interacting with Human Drivers: Human-like Driving and Decision Making for Autonomous Vehicles. arXiv.","DOI":"10.1201\/9781003287087"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Guo, C., Kidono, K., Machida, T., Terashima, R., and Kojima, Y. (2017, January 11\u201314). Human-like behavior generation for intelligent vehicles in urban environment based on a hybrid potential map. Proceedings of the 2017 IEEE Intelligent Vehicles Symposium (IV), Los Angeles, CA, USA.","DOI":"10.1109\/IVS.2017.7995720"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Naumann, M., Lauer, M., and Stiller, C. (2018, January 4\u20137). Generating comfortable, safe and comprehensible trajectories for automated vehicles in mixed traffic. Proceedings of the 2018 21st International Conference on Intelligent Transportation Systems (ITSC), Maui, HI, USA.","DOI":"10.1109\/ITSC.2018.8569658"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1913","DOI":"10.1049\/iet-its.2020.0297","article-title":"2-dimensional human-like driver model for autonomous vehicles in mixed traffic","volume":"14","author":"Sharath","year":"2020","journal-title":"IET Intell. Transp. Syst."},{"key":"ref_13","unstructured":"Al-Shihabi, T., and Mourant, R.R. (June, January 28). A framework for modeling human-like driving behaviors for autonomous vehicles in driving simulators. Proceedings of the fifth International Conference on Autonomous Agents, Montreal, QC, Canada."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"3135","DOI":"10.1109\/TITS.2020.2982002","article-title":"A universal control scheme of human-like steering in multiple driving scenarios","volume":"22","author":"Cheng","year":"2020","journal-title":"IEEE Trans. Intell. Transp."},{"key":"ref_15","unstructured":"Wei, C., Paschalidis, E., Merat, N., Solernou, A., Hajiseyedjavadi, F., and Romano, R. (2021). Human-like Decision Making and Motion Control for Smooth and Natural Car Following. IEEE Trans. Intell. Veh."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Ulbrich, S., and Maurer, M. (2013, January 6\u20139). Probabilistic online POMDP decision making for lane changes in fully automated driving. Proceedings of the 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013), The Hague, The Netherlands.","DOI":"10.1109\/ITSC.2013.6728533"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Cheng, W., Wang, G., Wu, S., Xiong, G., and Gong, J. (2017, January 14\u201318). A human-like longitudinal decision model of intelligent vehicle at signalized intersections. Proceedings of the 2017 IEEE International Conference on Real-time Computing and Robotics (RCAR), Okinawa, Japan.","DOI":"10.1109\/RCAR.2017.8311897"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Lin, X., Zhang, J., Shang, J., Wang, Y., Yu, H., and Zhang, X. (2019, January 27\u201330). Decision making through occluded intersections for autonomous driving. Proceedings of the 2019 IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, New Zealand.","DOI":"10.1109\/ITSC.2019.8917348"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Oh, K., Oh, S., Lee, J., and Yi, K. (2021, January 12\u201315). Development of a Human-Like Learning Frame for Data-Driven Adaptive Control Algorithm of Automated Driving. Proceedings of the 2021 21st International Conference on Control, Automation and Systems (ICCAS), Jeju, Korea.","DOI":"10.23919\/ICCAS52745.2021.9649954"},{"key":"ref_20","unstructured":"Wu, J., Huang, Z., Huang, C., Hu, Z., Hang, P., Xing, Y., and Lv, C. (2021). Human-in-the-loop deep reinforcement learning with application to autonomous driving. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"7341","DOI":"10.1109\/TITS.2020.3001131","article-title":"Learning from naturalistic driving data for human-like autonomous highway driving","volume":"22","author":"Xu","year":"2020","journal-title":"IEEE Trans. Intell. Transp."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Sun, T., Gao, Z., Gao, F., Zhang, T., Chen, S., and Zhao, K. (2021). A brain-inspired decision-making linear neural network and its application in automatic drive. Sensors, 21.","DOI":"10.3390\/s21030794"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"362","DOI":"10.1002\/rob.21918","article-title":"A survey of deep learning techniques for autonomous driving","volume":"37","author":"Grigorescu","year":"2020","journal-title":"J. Field Robot."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2226","DOI":"10.1177\/0954407020983579","article-title":"Modeling human-like longitudinal driver model for intelligent vehicles based on reinforcement learning","volume":"235","author":"Xie","year":"2021","journal-title":"Proc. Inst. Mech. Eng. Part D J. Autom. Eng."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"348","DOI":"10.1016\/j.trc.2018.10.024","article-title":"Human-like autonomous car-following model with deep reinforcement learning","volume":"97","author":"Zhu","year":"2018","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Sun, P., Yin, Y., Lin, L., and Wang, X. (2018, January 26\u201330). Human-like autonomous vehicle speed control by deep reinforcement learning with double Q-learning. Proceedings of the 2018 IEEE Intelligent Vehicles Symposium (IV), Changshu, China.","DOI":"10.1109\/IVS.2018.8500630"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"9315","DOI":"10.1109\/TVT.2020.2980197","article-title":"Extracting human-like driving behaviors from expert driver data using deep learning","volume":"69","author":"Sama","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_28","unstructured":"Hecker, S., Dai, D., and Van Gool, L. (2019). Learning accurate, comfortable and human-like driving. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Lu, C., Gong, J., Lv, C., Chen, X., Cao, D., and Chen, Y. (2019). A personalized behavior learning system for human-like longitudinal speed control of autonomous vehicles. Sensors, 19.","DOI":"10.3390\/s19173672"},{"key":"ref_30","unstructured":"Chen, J., Li, S.E., and Tomizuka, M. (2020). Interpretable end-to-end urban autonomous driving with latent deep reinforcement learning. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1143","DOI":"10.1109\/LRA.2020.2966414","article-title":"Learning robust control policies for end-to-end autonomous driving from data-driven simulation","volume":"5","author":"Amini","year":"2020","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"11781","DOI":"10.1109\/JSEN.2020.3003121","article-title":"Multi-modal sensor fusion-based deep neural network for end-to-end autonomous driving with scene understanding","volume":"21","author":"Huang","year":"2020","journal-title":"IEEE Sens. J."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"4740","DOI":"10.1109\/TVT.2020.2979493","article-title":"Stabilization approaches for reinforcement learning-based end-to-end autonomous driving","volume":"69","author":"Chen","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_34","unstructured":"Tampuu, A., Matiisen, T., Semikin, M., Fishman, D., and Muhammad, N. (2020). A survey of end-to-end driving: Architectures and training methods. arXiv."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Abbeel, P., and Ng, A.Y. (2004, January 4\u20138). Apprenticeship learning via inverse reinforcement learning. Proceedings of the twenty-First International Conference on Machine Learning, Banff, AB, Canada.","DOI":"10.1145\/1015330.1015430"},{"key":"ref_36","unstructured":"Ziebart, B.D., Maas, A.L., Bagnell, J.A., and Dey, A.K. (2008, January 13\u201317). Maximum entropy inverse reinforcement learning. Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence, Chicago, IL, USA."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Kuderer, M., Gulati, S., and Burgard, W. (2015, January 26\u201330). Learning driving styles for autonomous vehicles from demonstration. Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA.","DOI":"10.1109\/ICRA.2015.7139555"},{"key":"ref_38","unstructured":"Levine, S., and Koltun, V. (2012). Continuous inverse optimal control with locally optimal examples. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Naumann, M., Sun, L., Zhan, W., and Tomizuka, M. (August, January 31). Analyzing the suitability of cost functions for explaining and imitating human driving behavior based on inverse reinforcement learning. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France.","DOI":"10.1109\/ICRA40945.2020.9196795"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Huang, Z., Wu, J., and Lv, C. (2021). Driving behavior modeling using naturalistic human driving data with inverse reinforcement learning. IEEE Trans. Intell. Transp. Syst.","DOI":"10.1109\/TITS.2021.3088935"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Le Mero, L., Yi, D., Dianati, M., and Mouzakitis, A. (2022). A survey on imitation learning techniques for end-to-end autonomous vehicles. IEEE Trans. Intell. Transp.","DOI":"10.1109\/TITS.2022.3144867"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"McAllister, R., Gal, Y., Kendall, A., Van Der Wilk, M., Shah, A., Cipolla, R., and Weller, A. (2017, January 19\u201325). Concrete problems for autonomous vehicle safety: Advantages of Bayesian deep learning. Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI-17), Melbourne, Australia.","DOI":"10.24963\/ijcai.2017\/661"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"3275","DOI":"10.1109\/TIE.2018.2840530","article-title":"Decision-making framework for autonomous driving at road intersections: Safeguarding against collision, overly conservative behavior, and violation vehicles","volume":"66","author":"Noh","year":"2018","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"678","DOI":"10.1109\/TITS.2009.2026452","article-title":"Situation assessment of an autonomous emergency brake for arbitrary vehicle-to-vehicle collision scenarios","volume":"10","author":"Kaempchen","year":"2009","journal-title":"IEEE Trands. Intell. Transp."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Kumar, A.S.B., Modh, A., Babu, M., Gopalakrishnan, B., and Krishna, K.M. (2018, January 26\u201330). A novel lane merging framework with probabilistic risk based lane selection using time scaled collision cone. Proceedings of the 2018 IEEE Intelligent Vehicles Symposium (IV), Changshu, China.","DOI":"10.1109\/IVS.2018.8500652"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1109\/MITS.2016.2630585","article-title":"Traffic coordination at road intersections: Autonomous decision-making algorithms using model-based heuristics","volume":"9","author":"Falcone","year":"2017","journal-title":"IEEE Intell. Transp. Syst. Mag."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1613\/jair.639","article-title":"Hierarchical reinforcement learning with the MAXQ value function decomposition","volume":"13","author":"Dietterich","year":"2000","journal-title":"J. Artif. Intell. Res."},{"key":"ref_48","first-page":"1162","article-title":"Reinforcement learning methods for continuous-time Markov decision problems","volume":"7","author":"Bradtke","year":"1994","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Qiao, Z., Muelling, K., Dolan, J., Palanisamy, P., and Mudalige, P. (2018, January 4\u20137). Pomdp and hierarchical options mdp with continuous actions for autonomous driving at intersections. Proceedings of the 2018 21st International Conference on Intelligent Transportation Systems (ITSC), Maui, HI, USA.","DOI":"10.1109\/ITSC.2018.8569400"},{"key":"ref_50","unstructured":"Ng, A.Y., and Russell, S.J. (July, January 29). Algorithms for Inverse Reinforcement Learning. Proceedings of the International Conference on Machine Learning (ICML), San Francisco, CA, USA."},{"key":"ref_51","unstructured":"Moehle, N. (2021). Risk-Sensitive Model Predictive Control. arXiv."},{"key":"ref_52","unstructured":"Urp\u00ed, N.A., Curi, S., and Krause, A. (2021). Risk-averse offline reinforcement learning. arXiv."},{"key":"ref_53","unstructured":"Di Filippo, I. (2021). Can Risk Aversion Improve the Efficiency of Traffic networks? A Comparison between Risk Neutral and Risk Averse Routing Games. [Bachelor\u2019s Thesis, Luiss Guido Carli]."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Lu, C., Lv, C., Gong, J., Wang, W., Cao, D., and Wang, F. (2022). Instance-Level Knowledge Transfer for Data-Driven Driver Model Adaptation with Homogeneous Domains. IEEE Trans. Intell. Transp. Syst.","DOI":"10.1109\/TITS.2022.3161939"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/12\/4500\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:31:13Z","timestamp":1760139073000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/12\/4500"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,14]]},"references-count":54,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2022,6]]}},"alternative-id":["s22124500"],"URL":"https:\/\/doi.org\/10.3390\/s22124500","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,6,14]]}}}