{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,12]],"date-time":"2026-03-12T07:31:01Z","timestamp":1773300661339,"version":"3.50.1"},"reference-count":49,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2022,9,8]],"date-time":"2022-09-08T00:00:00Z","timestamp":1662595200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["J. Hum.-Robot Interact."],"published-print":{"date-parts":[[2022,12,31]]},"abstract":"<jats:p>A major challenge for autonomous vehicles is interacting with other traffic participants safely and smoothly. A promising approach to handle such traffic interactions is equipping autonomous vehicles with interaction-aware controllers (IACs). These controllers predict how surrounding human drivers will respond to the autonomous vehicle\u2019s actions, based on a driver model. However, the predictive validity of driver models used in IACs is rarely validated, which can limit the interactive capabilities of IACs outside the simple simulated environments in which they are demonstrated. In this article, we argue that besides evaluating the interactive capabilities of IACs, their underlying driver models should be validated on natural human driving behavior. We propose a workflow for this validation that includes scenario-based data extraction and a two-stage (tactical\/operational) evaluation procedure based on human factors literature. We demonstrate this workflow in a case study on an inverse-reinforcement-learning-based driver model replicated from an existing IAC. This model only showed the correct tactical behavior in 40% of the predictions. The model\u2019s operational behavior was inconsistent with observed human behavior. The case study illustrates that a principled evaluation workflow is useful and needed. We believe that our workflow will support the development of appropriate driver models for future automated vehicles.<\/jats:p>","DOI":"10.1145\/3538705","type":"journal-article","created":{"date-parts":[[2022,5,24]],"date-time":"2022-05-24T07:00:57Z","timestamp":1653375657000},"page":"1-21","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":26,"title":["A Human Factors Approach to Validating Driver Models for Interaction-aware Automated Vehicles"],"prefix":"10.1145","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5614-1262","authenticated-orcid":false,"given":"Olger","family":"Siebinga","sequence":"first","affiliation":[{"name":"Delft University of Technology, CD Delft the Netherlands"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6593-6948","authenticated-orcid":false,"given":"Arkady","family":"Zgonnikov","sequence":"additional","affiliation":[{"name":"Delft University of Technology, CD Delft the Netherlands"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7778-0090","authenticated-orcid":false,"given":"David","family":"Abbink","sequence":"additional","affiliation":[{"name":"Delft University of Technology, CD Delft the Netherlands"}]}],"member":"320","published-online":{"date-parts":[[2022,9,8]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/1015330.1015430"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ssci.2019.01.016"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.trc.2019.11.023"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.3141\/2606-14"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.23919\/ACC.2019.8815251"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/ITSC.2016.7795596"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/ITSC.2019.8917314"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1201\/b10836-8"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2020.3036984"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.trc.2016.09.003"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.3141\/1965-12"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2021.3088935"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/ITSC.2018.8569729"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/ITSC.2019.8916982"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.trb.2015.08.003"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1299\/jmtl.1.170"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/ITSC.2018.8569552"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1068\/p050437"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/IVS.2016.7535424"},{"key":"e_1_3_2_21_2","first-page":"41","article-title":"Continuous inverse optimal control with locally optimal examples","volume":"1","author":"Levine Sergey","year":"2012","unstructured":"Sergey Levine and Vladlen Koltun. 2012. Continuous inverse optimal control with locally optimal examples. Proceedings of the 29th International Conference on Machine Learning 1 (2012), 41\u201348.","journal-title":"Proceedings of the 29th International Conference on Machine Learning"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/IVS.2015.7225835"},{"key":"e_1_3_2_23_2","unstructured":"Gustav Markkula and Mehmet Dogar. 2022. How accurate models of human behavior are needed for human-robot interaction? For automated driving? IEEE Robotics and Automation Magazine."},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/CONTROL.2016.7737643"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4613-2173-6_19"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/icsmc.2005.1571179"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSMC.2009.5346803"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA40945.2020.9196795"},{"key":"e_1_3_2_29_2","first-page":"663","article-title":"Algorithms for inverse reinforcement learning","volume":"0","author":"Ng Andrew","year":"2000","unstructured":"Andrew Ng and Stuart Russell. 2000. Algorithms for inverse reinforcement learning. Proceedings of the 17th International Conference on Machine Learning 0 (2000), 663\u2013670. Retrieved from http:\/\/www-cs.stanford.edu\/people\/ang\/papers\/icml00-irl.pdf.","journal-title":"Proceedings of the 17th International Conference on Machine Learning"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.trc.2010.05.006"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1111\/1753-6405.12588"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1201\/b10836-10"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/IROS40897.2019.8968205"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/ITSC.2013.6728509"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10514-018-9746-1"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.trc.2014.09.008"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1820676116"},{"key":"e_1_3_2_38_2","unstructured":"Wilko Schwarting Alyssa Pierson Javier Alonso-Mora Sertac Karaman and Daniela Rus. 2019. Social behavior for autonomous vehicles - Supporting Information. Retrieved 19 July 2021 from https:\/\/www.pnas.org\/content\/suppl\/2019\/11\/22\/1820676116.DCSupplemental."},{"key":"e_1_3_2_39_2","unstructured":"O. Siebinga. 2021. IRL Model Validation - TraViA extension code. Retrieved 21 March 2022 from https:\/\/github.com\/tud-hri\/irlmodelvalidation."},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.21105\/joss.03607"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/ITSC48978.2021.9564791"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/IROS45743.2020.9341371"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/CDC.2018.8619275"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.62.1805"},{"key":"e_1_3_2_45_2","unstructured":"U.S. Department of Transportation Federal Highway Administration. 2016. Next Generation Simulation (NGSIM) Vehicle Trajectories and Supporting Data. [Dataset]. Retrieved 19 July 2021 from https:\/\/data.transportation.gov\/Automobiles\/Next-Generation-Simulation-NGSIM-Vehicle-Trajector\/8ect-6jqj."},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIV.2017.2730588"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.trc.2018.01.016"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.23919\/ACC.2018.8431530"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.trc.2018.06.009"},{"key":"e_1_3_2_50_2","first-page":"1433","volume-title":"Proceedings of the National Conference on Artificial Intelligence","author":"Ziebart Brian D.","year":"2008","unstructured":"Brian D. Ziebart, Andrew Maas, J. Andrew Bagnell, and Anind K. Dey. 2008. Maximum entropy inverse reinforcement learning. In Proceedings of the National Conference on Artificial Intelligence. 1433\u20131438."}],"container-title":["ACM Transactions on Human-Robot Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3538705","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3538705","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T18:09:38Z","timestamp":1750183778000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3538705"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,8]]},"references-count":49,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2022,12,31]]}},"alternative-id":["10.1145\/3538705"],"URL":"https:\/\/doi.org\/10.1145\/3538705","relation":{},"ISSN":["2573-9522","2573-9522"],"issn-type":[{"value":"2573-9522","type":"print"},{"value":"2573-9522","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,8]]},"assertion":[{"value":"2021-12-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-04-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-09-08","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}