{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,14]],"date-time":"2026-04-14T06:41:04Z","timestamp":1776148864106,"version":"3.50.1"},"reference-count":63,"publisher":"Association for Computing Machinery (ACM)","issue":"FSE","funder":[{"DOI":"10.13039\/501100001809","name":"the Joint Funds of the National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["U24A20238"],"award-info":[{"award-number":["U24A20238"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"the National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62402367"],"award-info":[{"award-number":["62402367"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"the National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62232008,62032010"],"award-info":[{"award-number":["62232008,62032010"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Softw. Eng."],"published-print":{"date-parts":[[2025,6,19]]},"abstract":"<jats:p>The safety and reliability of Automated Driving Systems (ADS) are paramount, necessitating rigorous testing methodologies to uncover potential failures before deployment. Traditional testing approaches often prioritize either natural scenario sampling or safety-critical scenario generation, resulting in overly simplistic or unrealistic hazardous tests. In practice, the demand for natural scenarios (e.g., when evaluating the ADS's reliability in real-world conditions), critical scenarios (e.g., when evaluating safety in critical situations), or somewhere in between (e.g., when testing the ADS in regions with less civilized drivers) varies depending on the testing objectives. To address this issue, we propose the On-demand Scenario Generation (OSG) Framework, which generates diverse scenarios with varying risk levels. Achieving the goal of OSG is challenging due to the complexity of quantifying the criticalness and naturalness stemming from intricate vehicle-environment interactions, as well as the need to maintain scenario diversity across various risk levels. OSG learns from real-world traffic datasets and employs a Risk Intensity Regulator to quantitatively control the risk level. It also leverages an improved heuristic search method to ensure scenario diversity. We evaluate OSG on the Carla simulators using various ADSs. We verify OSG's ability to generate scenarios with different risk levels and demonstrate its necessity by comparing accident types across risk levels. With the help of OSG, we are now able to systematically and objectively compare the performance of different ADSs based on different risk levels.<\/jats:p>","DOI":"10.1145\/3715722","type":"journal-article","created":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T15:16:02Z","timestamp":1750346162000},"page":"86-105","source":"Crossref","is-referenced-by-count":5,"title":["On-Demand Scenario Generation for Testing Automated Driving Systems"],"prefix":"10.1145","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0628-8642","authenticated-orcid":false,"given":"Songyang","family":"Yan","sequence":"first","affiliation":[{"name":"Xi'an Jiaotong University, Xi'an, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8380-1019","authenticated-orcid":false,"given":"Xiaodong","family":"Zhang","sequence":"additional","affiliation":[{"name":"Xidian University, Xi'an, China"},{"name":"Xidian University, Xi'an, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0420-1347","authenticated-orcid":false,"given":"Kunkun","family":"Hao","sequence":"additional","affiliation":[{"name":"Synkrotron, Xi'an, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5223-0113","authenticated-orcid":false,"given":"Haojie","family":"Xin","sequence":"additional","affiliation":[{"name":"Xi'an Jiaotong University, Xi'an, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-3973-7606","authenticated-orcid":false,"given":"Yonggang","family":"Luo","sequence":"additional","affiliation":[{"name":"Chongqing Changan Automobile, Chongqing, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5046-2663","authenticated-orcid":false,"given":"Jucheng","family":"Yang","sequence":"additional","affiliation":[{"name":"Chongqing Changan Automobile, Chongqing, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9327-0987","authenticated-orcid":false,"given":"Ming","family":"Fan","sequence":"additional","affiliation":[{"name":"Xi'an Jiaotong University, Xi'an, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3311-291X","authenticated-orcid":false,"given":"Chao","family":"Yang","sequence":"additional","affiliation":[{"name":"Xidian University, Xi'an, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3545-1392","authenticated-orcid":false,"given":"Jun","family":"Sun","sequence":"additional","affiliation":[{"name":"Singapore Management University, Singapore, Singapore"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-5437-0253","authenticated-orcid":false,"given":"Zijiang","family":"Yang","sequence":"additional","affiliation":[{"name":"University of Science and Technology of China, Hefei, China"},{"name":"Synkrotron, Xi'an, China"}]}],"member":"320","published-online":{"date-parts":[[2025,6,19]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"2021 IEEE International Conference on Image Processing (ICIP). 3592\u20133596","author":"Abdelfattah Mazen","year":"2021","unstructured":"Mazen Abdelfattah, Kaiwen Yuan, Z Jane Wang, and Rabab Ward. 2021. Towards universal physical attacks on cascaded camera-lidar 3d object detection models. In 2021 IEEE International Conference on Image Processing (ICIP). 3592\u20133596. https:\/\/doi.org\/10.1109\/ICIP42928.2021.9506016 10.1109\/ICIP42928.2021.9506016"},{"key":"e_1_2_1_2_1","volume-title":"International Conference on Artificial Intelligence and Statistics. 595\u2013603","author":"Arief Mansur","year":"2021","unstructured":"Mansur Arief, Zhiyuan Huang, Guru Koushik Senthil Kumar, Yuanlu Bai, Shengyi He, Wenhao Ding, Henry Lam, and Ding Zhao. 2021. Deep probabilistic accelerated evaluation: A robust certifiable rare-event simulation methodology for black-box safety-critical systems. In International Conference on Artificial Intelligence and Statistics. 595\u2013603."},{"key":"e_1_2_1_3_1","volume-title":"Baidu Apollo team","year":"2024","unstructured":"Baidu. 2024. Baidu Apollo team (2024), Apollo: Open Source Autonomous Driving. https:\/\/github.com\/ApolloAuto\/apollo Accessed: 2024-03-25"},{"key":"e_1_2_1_4_1","unstructured":"Baidu. 2024. Issues of Apollo. https:\/\/github.com\/ApolloAuto\/apollo\/issues Accessed: 2024-04-08"},{"key":"e_1_2_1_5_1","unstructured":"Baidu. 2024. Release Note of Apollo 9.0. https:\/\/github.com\/ApolloAuto\/apollo\/releases Accessed: 2024-04-08"},{"key":"e_1_2_1_6_1","volume-title":"2020 IEEE Intelligent Vehicles Symposium (IV). 1929\u20131934","author":"Bock Julian","year":"2020","unstructured":"Julian Bock, Robert Krajewski, Tobias Moers, Steffen Runde, Lennart Vater, and Lutz Eckstein. 2020. The inD Dataset: A Drone Dataset of Naturalistic Road User Trajectories at German Intersections. In 2020 IEEE Intelligent Vehicles Symposium (IV). 1929\u20131934. https:\/\/doi.org\/10.1109\/IV47402.2020.9304839 10.1109\/IV47402.2020.9304839"},{"key":"e_1_2_1_7_1","volume-title":"Proceedings of the 4th Asia-Pacific conference on simulated evolution and learning. 2, 692\u2013696","author":"Brits Riaan","unstructured":"Riaan Brits, Andries P Engelbrecht, and Frans van den Bergh. 2002. A niching particle swarm optimizer. In Proceedings of the 4th Asia-Pacific conference on simulated evolution and learning. 2, 692\u2013696."},{"key":"e_1_2_1_8_1","volume-title":"2020 IEEE International Conference on Robotics and Automation (ICRA). 4023\u20134029","author":"Cai Panpan","year":"2020","unstructured":"Panpan Cai, Yiyuan Lee, Yuanfu Luo, and David Hsu. 2020. Summit: A simulator for urban driving in massive mixed traffic. In 2020 IEEE International Conference on Robotics and Automation (ICRA). 4023\u20134029."},{"key":"e_1_2_1_9_1","volume-title":"Zhuoqing Morley Mao, and Sara Rampazzi","author":"Cao Yulong","year":"2022","unstructured":"Yulong Cao, S. Hrushikesh Bhupathiraju, Pirouz Naghavi, Takeshi Sugawara, Zhuoqing Morley Mao, and Sara Rampazzi. 2022. You Can\u2019t See Me: Physical Removal Attacks on LiDAR-based Autonomous Vehicles Driving Frameworks. ArXiv, abs\/2210.09482 (2022), https:\/\/api.semanticscholar.org\/CorpusID:252967801"},{"key":"e_1_2_1_10_1","volume-title":"2021 IEEE Symposium on Security and Privacy (SP), 176\u2013194","author":"Cao Yulong","year":"2021","unstructured":"Yulong Cao, Ningfei Wang, Chaowei Xiao, Dawei Yang, Jin Fang, Ruigang Yang, Qi Alfred Chen, Mingyan Liu, and Bo Li. 2021. Invisible for both Camera and LiDAR: Security of Multi-Sensor Fusion based Perception in Autonomous Driving Under Physical-World Attacks. 2021 IEEE Symposium on Security and Privacy (SP), 176\u2013194. https:\/\/doi.org\/10.1109\/SP40001.2021.00076 10.1109\/SP40001.2021.00076"},{"key":"e_1_2_1_11_1","unstructured":"CARLA. 2024. CARLA Autonomous Driving Leaderboard. https:\/\/leaderboard.carla.org\/ Accessed: 2024-03-25"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSMC.2024.3392930"},{"key":"e_1_2_1_13_1","volume-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition. 7230\u20137240","author":"Chen Yun","year":"2021","unstructured":"Yun Chen, Frieda Rong, Shivam Duggal, Shenlong Wang, Xinchen Yan, Sivabalan Manivasagam, Shangjie Xue, Ersin Yumer, and Raquel Urtasun. 2021. Geosim: Realistic video simulation via geometry-aware composition for self-driving. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition. 7230\u20137240."},{"key":"e_1_2_1_14_1","volume-title":"Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis. 488\u2013500","author":"Cheng Mingfei","year":"2023","unstructured":"Mingfei Cheng, Yuan Zhou, and Xiaofei Xie. 2023. Behavexplor: Behavior diversity guided testing for autonomous driving systems. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis. 488\u2013500. https:\/\/doi.org\/10.1145\/3597926.3598072 10.1145\/3597926.3598072"},{"key":"e_1_2_1_15_1","volume-title":"Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, https:\/\/doi.org\/10","author":"Deng Yao","year":"2022","unstructured":"Yao Deng, James Xi Zheng, Mengshi Zhang, Guannan Lou, and Tianyi Zhang. 2022. Scenario-based test reduction and prioritization for multi-module autonomous driving systems. Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, https:\/\/doi.org\/10.1145\/3540250.3549152 10.1145\/3540250.3549152"},{"key":"e_1_2_1_16_1","doi-asserted-by":"crossref","first-page":"1551","DOI":"10.1109\/LRA.2021.3058873","article-title":"Multimodal safety-critical scenarios generation for decision-making algorithms evaluation","volume":"6","author":"Ding Wenhao","year":"2021","unstructured":"Wenhao Ding, Baiming Chen, Bo Li, Kim Ji Eun, and Ding Zhao. 2021. Multimodal safety-critical scenarios generation for decision-making algorithms evaluation. IEEE Robotics and Automation Letters, 6, 2 (2021), 1551\u20131558.","journal-title":"IEEE Robotics and Automation Letters"},{"key":"e_1_2_1_17_1","volume-title":"Proceedings of the 1st Annual Conference on Robot Learning. 1\u201316","author":"Dosovitskiy Alexey","year":"2017","unstructured":"Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. 2017. CARLA: An Open Urban Driving Simulator. In Proceedings of the 1st Annual Conference on Robot Learning. 1\u201316."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2020.2969927"},{"key":"e_1_2_1_19_1","volume-title":"2021 IEEE Intelligent Vehicles Symposium (IV). 157\u2013164","author":"Ghodsi Zahra","year":"2021","unstructured":"Zahra Ghodsi, Siva Kumar Sastry Hari, Iuri Frosio, Timothy Tsai, Alejandro Troccoli, Stephen W Keckler, Siddharth Garg, and Anima Anandkumar. 2021. Generating and characterizing scenarios for safety testing of autonomous vehicles. In 2021 IEEE Intelligent Vehicles Symposium (IV). 157\u2013164. https:\/\/doi.org\/10.1109\/IV48863.2021.9576023 10.1109\/IV48863.2021.9576023"},{"key":"e_1_2_1_20_1","unstructured":"Guardstrikelab. 2024. Carla Apollo Bridge. https:\/\/github.com\/guardstrikelab\/carla_apollo_bridge"},{"key":"e_1_2_1_21_1","volume-title":"Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis, https:\/\/doi.org\/10","author":"Guo An","year":"2022","unstructured":"An Guo, Yang Feng, and Zhenyu Chen. 2022. LiRTest: augmenting LiDAR point clouds for automated testing of autonomous driving systems. Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis, https:\/\/doi.org\/10.1145\/3533767.3534397 10.1145\/3533767.3534397"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIV.2023.3335862"},{"key":"e_1_2_1_23_1","volume-title":"Proceedings of the 44th international conference on software engineering. 811\u2013822","author":"Haq Fitash Ul","year":"2022","unstructured":"Fitash Ul Haq, Donghwan Shin, and Lionel Briand. 2022. Efficient online testing for dnn-enabled systems using surrogate-assisted and many-objective optimization. In Proceedings of the 44th international conference on software engineering. 811\u2013822. https:\/\/doi.org\/10.1145\/3510003.3510188 10.1145\/3510003.3510188"},{"key":"e_1_2_1_24_1","volume-title":"Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis, https:\/\/doi.org\/10","author":"Hildebrandt Carl","unstructured":"Carl Hildebrandt, Meriel von Stein, and Sebastian G. Elbaum. 2023. PhysCov: Physical Test Coverage for Autonomous Vehicles. Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis, https:\/\/doi.org\/10.1145\/3597926.3598069 10.1145\/3597926.3598069"},{"key":"e_1_2_1_25_1","volume-title":"Doppelg\u00e4nger Test Generation for Revealing Bugs in Autonomous Driving Software. 2023 IEEE\/ACM 45th International Conference on Software Engineering (ICSE), 2591\u20132603","author":"Huai Yuqi","year":"2023","unstructured":"Yuqi Huai, Yuntianyi Chen, Sumaya Almanee, Tu K Ngo, Xiang Liao, Ziwen Wan, Qi Alfred Chen, and Joshua Garcia. 2023. Doppelg\u00e4nger Test Generation for Revealing Bugs in Autonomous Driving Software. 2023 IEEE\/ACM 45th International Conference on Software Engineering (ICSE), 2591\u20132603. https:\/\/doi.org\/10.1109\/ICSE48619.2023.00216 10.1109\/ICSE48619.2023.00216"},{"key":"e_1_2_1_26_1","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision. 8240\u20138249","author":"Jaeger Bernhard","year":"2023","unstructured":"Bernhard Jaeger, Kashyap Chitta, and Andreas Geiger. 2023. Hidden biases of end-to-end driving models. In Proceedings of the IEEE\/CVF International Conference on Computer Vision. 8240\u20138249."},{"key":"e_1_2_1_27_1","volume-title":"Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, https:\/\/api.semanticscholar.org\/CorpusID:253265101","author":"Kim Seulbae","year":"2022","unstructured":"Seulbae Kim, Major Liu, Junghwan John Rhee, Yuseok Jeon, Yonghwi Kwon, and Chung Hwan Kim. 2022. DriveFuzz: Discovering Autonomous Driving Bugs through Driving Quality-Guided Fuzzing. Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, https:\/\/api.semanticscholar.org\/CorpusID:253265101"},{"key":"e_1_2_1_28_1","volume-title":"2018 IEEE Intelligent Vehicles Symposium (IV). 1\u20137. https:\/\/doi.org\/10","author":"Koren Mark","year":"2018","unstructured":"Mark Koren, Saud Alsaif, Ritchie Lee, and Mykel J Kochenderfer. 2018. Adaptive stress testing for autonomous vehicles. In 2018 IEEE Intelligent Vehicles Symposium (IV). 1\u20137. https:\/\/doi.org\/10.1109\/ICRA40945.2020.9197228 10.1109\/ICRA40945.2020.9197228"},{"key":"e_1_2_1_29_1","unstructured":"Edouard Leurent. 2018. An Environment for Autonomous Driving Decision-Making. https:\/\/github.com\/eleurent\/highway-env"},{"key":"e_1_2_1_30_1","volume-title":"Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis. 842\u2013853","author":"Li Changwen","year":"2023","unstructured":"Changwen Li, Joseph Sifakis, Qiang Wang, Rongjie Yan, and Jian Zhang. 2023. Simulation-Based Validation for Autonomous Driving Systems. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis. 842\u2013853. https:\/\/doi.org\/10.1145\/3597926.3598100 10.1145\/3597926.3598100"},{"key":"e_1_2_1_31_1","volume-title":"2020 IEEE 31st international symposium on software reliability engineering (ISSRE). 25\u201336","author":"Li Guanpeng","year":"2020","unstructured":"Guanpeng Li, Yiran Li, Saurabh Jha, Timothy Tsai, Michael Sullivan, Siva Kumar Sastry Hari, Zbigniew Kalbarczyk, and Ravishankar Iyer. 2020. Av-fuzzer: Finding safety violations in autonomous driving systems. In 2020 IEEE 31st international symposium on software reliability engineering (ISSRE). 25\u201336. https:\/\/doi.org\/10.1109\/ISSRE5003.2020.00012 10.1109\/ISSRE5003.2020.00012"},{"key":"e_1_2_1_32_1","volume-title":"Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 31\u201343","author":"Lou Guannan","year":"2022","unstructured":"Guannan Lou, Yao Deng, Xi Zheng, Mengshi Zhang, and Tianyi Zhang. 2022. Testing of autonomous driving systems: where are we and where should we go? In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 31\u201343. https:\/\/doi.org\/10.1145\/3540250.3549111 10.1145\/3540250.3549111"},{"key":"e_1_2_1_33_1","volume-title":"2021 36th IEEE\/ACM International Conference on Automated Software Engineering (ASE), 279\u2013291","author":"Luo Yixing","year":"2021","unstructured":"Yixing Luo, Xiaoyi Zhang, Paolo Arcaini, Zhi Jin, Haiyan Zhao, Fuyuki Ishikawa, Rongxin Wu, and Tao Xie. 2021. Targeting Requirements Violations of Autonomous Driving Systems by Dynamic Evolutionary Search. 2021 36th IEEE\/ACM International Conference on Automated Software Engineering (ASE), 279\u2013291. https:\/\/doi.org\/10.1109\/ASE51524.2021.9678883 10.1109\/ASE51524.2021.9678883"},{"key":"e_1_2_1_34_1","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 11167\u201311176","author":"Manivasagam Sivabalan","year":"2020","unstructured":"Sivabalan Manivasagam, Shenlong Wang, Kelvin Wong, Wenyuan Zeng, Mikita Sazanovich, Shuhan Tan, Bin Yang, Wei-Chiu Ma, and Raquel Urtasun. 2020. Lidarsim: Realistic lidar simulation by leveraging the real world. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 11167\u201311176."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0001-4575(00)00019-1"},{"key":"e_1_2_1_36_1","unstructured":"Wassim G Najm John D Smith and Mikio Yanagisawa. 2007. Pre-crash scenario typology for crash avoidance research. United States. Department of Transportation. National Highway Traffic Safety \u2026."},{"key":"e_1_2_1_37_1","volume-title":"2023 IEEE Intelligent Vehicles Symposium (IV). 1\u20138. https:\/\/doi.org\/10","author":"Niu Haoyi","year":"2023","unstructured":"Haoyi Niu, Kun Ren, Yizhou Xu, Ziyuan Yang, Yichen Lin, Yi Zhang, and Jianming Hu. 2023. (Re)2H2O: Autonomous Driving Scenario Generation via Reversely Regularized Hybrid Offline-and-Online Reinforcement Learning. In 2023 IEEE Intelligent Vehicles Symposium (IV). 1\u20138. https:\/\/doi.org\/10.1109\/IV55152.2023.10186559 10.1109\/IV55152.2023.10186559"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.5555\/3294771.3294994"},{"key":"e_1_2_1_39_1","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 17305\u201317315","author":"Rempe Davis","year":"2022","unstructured":"Davis Rempe, Jonah Philion, Leonidas J Guibas, Sanja Fidler, and Or Litany. 2022. Generating useful accident-prone driving scenarios via a learned traffic prior. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 17305\u201317315."},{"key":"e_1_2_1_40_1","volume-title":"Hadi Tabatabaee, Qiang Lu, Steve Lemke, M\u0101rti\u0146\u0161 Mo\u017eeiko, Eric Boise, Geehoon Uhm, Mark Gerow, and Shalin Mehta.","author":"Rong Guodong","year":"2020","unstructured":"Guodong Rong, Byung Hyun Shin, Hadi Tabatabaee, Qiang Lu, Steve Lemke, M\u0101rti\u0146\u0161 Mo\u017eeiko, Eric Boise, Geehoon Uhm, Mark Gerow, and Shalin Mehta. 2020. LGSVL Simulator: A High Fidelity Simulator for Autonomous Driving. arXiv preprint arXiv:2005.03778."},{"key":"e_1_2_1_41_1","volume-title":"2021 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS). 1229\u20131235","author":"Savkin Artem","year":"2021","unstructured":"Artem Savkin, Rachid Ellouze, Nassir Navab, and Federico Tombari. 2021. Unsupervised traffic scene generation with synthetic 3D scene graphs. In 2021 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS). 1229\u20131235. https:\/\/doi.org\/10.1109\/IROS51168.2021.9636318 10.1109\/IROS51168.2021.9636318"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.aap.2021.106454"},{"key":"e_1_2_1_43_1","volume-title":"Conference on Robot Learning. 726\u2013737","author":"Shao Hao","year":"2023","unstructured":"Hao Shao, Letian Wang, Ruobing Chen, Hongsheng Li, and Yu Liu. 2023. Safety-enhanced autonomous driving using interpretable sensor fusion transformer. In Conference on Robot Learning. 726\u2013737."},{"key":"e_1_2_1_44_1","volume-title":"2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS). 2103\u20132110","author":"Shiroshita Shinya","year":"2020","unstructured":"Shinya Shiroshita, Shirou Maruyama, Daisuke Nishiyama, Mario Ynocente Castro, Karim Hamzaoui, Guy Rosman, Jonathan DeCastro, Kuan-Hui Lee, and Adrien Gaidon. 2020. Behaviorally diverse traffic simulation via reinforcement learning. In 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS). 2103\u20132110. https:\/\/doi.org\/10.1109\/IROS45743.2020.9341493 10.1109\/IROS45743.2020.9341493"},{"key":"e_1_2_1_45_1","volume-title":"Merrill","author":"Siddiqui Faiz","year":"2023","unstructured":"Faiz Siddiqui and Jeremy B. Merrill. 2023. https:\/\/www.washingtonpost.com\/technology\/2023\/06\/10\/tesla-autopilot-crashes-elon-musk\/"},{"key":"e_1_2_1_46_1","volume-title":"USENIX Security Symposium. https:\/\/api.semanticscholar.org\/CorpusID:260777685","author":"Song Ruoyu","year":"2023","unstructured":"Ruoyu Song, Muslum Ozgur Ozmen, Hyungsub Kim, Raymond Muller, Z. Berkay Celik, and Antonio Bianchi. 2023. Discovering Adversarial Driving Maneuvers against Autonomous Vehicles. In USENIX Security Symposium. https:\/\/api.semanticscholar.org\/CorpusID:260777685"},{"key":"e_1_2_1_47_1","volume-title":"LawBreaker: An Approach for Specifying Traffic Laws and Fuzzing Autonomous Vehicles. In 37th IEEE\/ACM International Conference on Automated Software Engineering. 1\u201312","author":"Sun Yang","year":"2022","unstructured":"Yang Sun, Christopher M Poskitt, Jun Sun, Yuqi Chen, and Zijiang Yang. 2022. LawBreaker: An Approach for Specifying Traffic Laws and Fuzzing Autonomous Vehicles. In 37th IEEE\/ACM International Conference on Automated Software Engineering. 1\u201312."},{"key":"e_1_2_1_48_1","unstructured":"Trisha Thadani Rachel Lerman Imogen Piper Faiz Siddiqui and Irfan Uraizee. 2023. https:\/\/www.washingtonpost.com\/technology\/interactive\/2023\/tesla-autopilot-crash-analysis\/"},{"key":"e_1_2_1_49_1","volume-title":"Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 94\u2013106","author":"Tian Haoxiang","year":"2022","unstructured":"Haoxiang Tian, Yan Jiang, Guoquan Wu, Jiren Yan, Jun Wei, Wei Chen, Shuo Li, and Dan Ye. 2022. MOSAT: finding safety violations of autonomous driving systems using multi-objective genetic algorithm. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 94\u2013106. https:\/\/doi.org\/10.1145\/3540250.3549100 10.1145\/3540250.3549100"},{"key":"e_1_2_1_50_1","volume-title":"Proceedings of the 37th IEEE\/ACM International Conference on Automated Software Engineering, https:\/\/api.semanticscholar.org\/CorpusID:255441327","author":"Tian Haoxiang","year":"2022","unstructured":"Haoxiang Tian, Guoquan Wu, Jiren Yan, Yan Jiang, Jun Wei, W. Chen, Shuo Li, and Dan Ye. 2022. Generating Critical Test Scenarios for Autonomous Driving Systems via Influential Behavior Patterns. Proceedings of the 37th IEEE\/ACM International Conference on Automated Software Engineering, https:\/\/api.semanticscholar.org\/CorpusID:255441327"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.8127026"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","unstructured":"U.S. Department of Transportation Federal Highway Administration. 2016. Next Generation Simulation (NGSIM) Vehicle Trajectories and Supporting Data. [Dataset]. Provided by ITS DataHub through Data.transportation.gov. Accessed 2023-05-13 from http:\/\/doi.org\/10.21949\/1504477 10.21949\/1504477","DOI":"10.21949\/1504477"},{"key":"e_1_2_1_53_1","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 9909\u20139918","author":"Wang Jingkang","year":"2021","unstructured":"Jingkang Wang, Ava Pun, James Tu, Sivabalan Manivasagam, Abbas Sadat, Sergio Casas, Mengye Ren, and Raquel Urtasun. 2021. Advsim: Generating safety-critical scenarios for self-driving vehicles. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 9909\u20139918."},{"key":"e_1_2_1_54_1","volume-title":"Dance of the ADS: Orchestrating Failures through Historically-Informed Scenario Fuzzing. ArXiv, abs\/2407.04359","author":"Wang Tong","year":"2024","unstructured":"Tong Wang, Taotao Gu, Huan Deng, Hu Li, Xiaohui Kuang, and Gang Zhao. 2024. Dance of the ADS: Orchestrating Failures through Historically-Informed Scenario Fuzzing. ArXiv, abs\/2407.04359 (2024), https:\/\/api.semanticscholar.org\/CorpusID:271039375"},{"key":"e_1_2_1_55_1","volume-title":"2021 IEEE International Intelligent Transportation Systems Conference (ITSC). 466\u2013472","author":"Wang Xiao","year":"2021","unstructured":"Xiao Wang, Hanna Krasowski, and Matthias Althoff. 2021. CommonRoad-RL: A configurable reinforcement learning environment for motion planning of autonomous vehicles. In 2021 IEEE International Intelligent Transportation Systems Conference (ITSC). 466\u2013472. https:\/\/doi.org\/10.1109\/ITSC48978.2021.9564898 10.1109\/ITSC48978.2021.9564898"},{"key":"e_1_2_1_56_1","volume-title":"2024 IEEE\/ACM 46th International Conference on Software Engineering (ICSE), 1736\u20131748","author":"Woodlief Trey","unstructured":"Trey Woodlief, Felipe Toledo, Sebastian G. Elbaum, and Matthew B. Dwyer. 2024. S3C: Spatial Semantic Scene Coverage for Autonomous Vehicles. 2024 IEEE\/ACM 46th International Conference on Software Engineering (ICSE), 1736\u20131748. https:\/\/api.semanticscholar.org\/CorpusID:269133131"},{"key":"e_1_2_1_57_1","unstructured":"Haojie Xin Xiaodong Zhang Renzhi Tang Songyang Yan Qianrui Zhao Chunze Yang Wen Cui and Zijiang Yang. 2024. LitSim: A Conflict-aware Policy for Long-term Interactive Traffic Simulation. arxiv:2403.04299. arxiv:2403.04299"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","unstructured":"Songyang Yan Xiaodong Zhang Kunkun Hao Haojie Xin Yonggang Luo Jucheng Yang Ming Fan Chao Yang Jun Sun and Zijiang Yang. 2024. OSG DATA at FSE 25. https:\/\/doi.org\/10.5281\/zenodo.13621782 10.5281\/zenodo.13621782","DOI":"10.5281\/zenodo.13621782"},{"key":"e_1_2_1_59_1","unstructured":"Wei Zhan Liting Sun Di Wang Haojie Shi Aubrey Clausse Maximilian Naumann Julius Kummerle Hendrik Konigshof Christoph Stiller and Arnaud de La Fortelle. 2019. Interaction dataset: An international adversarial and cooperative motion dataset in interactive driving scenarios with semantic maps. arXiv preprint arXiv:1910.03088."},{"key":"e_1_2_1_60_1","volume-title":"Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis, https:\/\/doi.org\/10","author":"Zhang Xudong","year":"2023","unstructured":"Xudong Zhang and Yan Cai. 2023. Building Critical Testing Scenarios for Autonomous Driving from Real Accidents. Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis, https:\/\/doi.org\/10.1145\/3597926.3598070 10.1145\/3597926.3598070"},{"key":"e_1_2_1_61_1","volume-title":"Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis. 942\u2013953","author":"Zhang Xiaodong","year":"2023","unstructured":"Xiaodong Zhang, Wei Zhao, Yang Sun, Jun Sun, Yulong Shen, Xuewen Dong, and Zijiang Yang. 2023. Testing automated driving systems by breaking many laws efficiently. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis. 942\u2013953. https:\/\/doi.org\/10.1145\/3597926.3598108 10.1145\/3597926.3598108"},{"key":"e_1_2_1_62_1","volume-title":"Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis, https:\/\/api.semanticscholar.org\/CorpusID:246431152","author":"Zhong Ziyuan","year":"2021","unstructured":"Ziyuan Zhong, Zhisheng Hu, Shengjian Guo, Xinyang Zhang, Zhenyu Zhong, and Baishakhi Ray. 2021. Detecting multi-sensor fusion errors in advanced driver-assistance systems. Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis, https:\/\/api.semanticscholar.org\/CorpusID:246431152"},{"key":"e_1_2_1_63_1","volume-title":"Proceedings of the 19th ACM conference on embedded networked sensor systems. 329\u2013342","author":"Zhu Yi","year":"2021","unstructured":"Yi Zhu, Chenglin Miao, Foad Hajiaghajani, Mengdi Huai, Lu Su, and Chunming Qiao. 2021. Adversarial attacks against lidar semantic segmentation in autonomous driving. In Proceedings of the 19th ACM conference on embedded networked sensor systems. 329\u2013342. https:\/\/doi.org\/10.1145\/3485730.3485935 10.1145\/3485730.3485935"}],"container-title":["Proceedings of the ACM on Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3715722","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T15:19:45Z","timestamp":1750346385000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3715722"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,19]]},"references-count":63,"journal-issue":{"issue":"FSE","published-print":{"date-parts":[[2025,6,19]]}},"alternative-id":["10.1145\/3715722"],"URL":"https:\/\/doi.org\/10.1145\/3715722","relation":{},"ISSN":["2994-970X"],"issn-type":[{"value":"2994-970X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,19]]}}}