{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,8]],"date-time":"2026-02-08T07:15:52Z","timestamp":1770534952520,"version":"3.49.0"},"reference-count":61,"publisher":"Association for Computing Machinery (ACM)","issue":"ISSTA","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Softw. Eng."],"published-print":{"date-parts":[[2025,6,22]]},"abstract":"<jats:p>Testing Autonomous Driving Systems (ADSs) is crucial for ensuring their safety, reliability, and performance. Despite numerous testing methods available that can generate diverse and challenging scenarios to uncover potential vulnerabilities, these methods often treat ADS as a black-box, primarily focusing on identifying system-level failures like collisions or near-misses without pinpointing the specific modules responsible for these failures. This lack of root causes understanding for the failures hinders effective debugging and subsequent system repair. Furthermore, current approaches often fall short in generating violations that adequately test the individual modules of an ADS from a system-level perspective, such as perception, prediction, planning, and control. To bridge this gap, we introduce MoDitector, a root-cause-aware testing method for ADS that generates safety-critical scenarios specifically designed to expose weaknesses in targeted ADS modules. Unlike existing approaches, MoDitector not only produces scenarios that lead to violations but also pinpoints the specific module responsible for each failure. Specifically, our approach introduces Module-Specific Oracles to automatically detect module-level errors and identify the root-cause module responsible for system-level violations. To effectively generate module-specific failures, we propose a module-directed testing strategy that integrates Module-Specific Feedback and Adaptive Scenario Generation to guide the testing process. We evaluated MoDitector across four critical ADS modules and four representative testing scenarios. The results demonstrate that MoDitector can effectively and efficiently generate scenarios in which failures can be attributed to specific targeted modules. In total, MoDitector generated 216.7 expected scenarios, significantly outperforming the best baseline, which identified only 79.0 scenarios. Our approach represents a significant innovation in ADS testing by focusing on the identification and rectification of module-specific errors within the system, moving beyond conventional black-box failure detection.<\/jats:p>","DOI":"10.1145\/3728876","type":"journal-article","created":{"date-parts":[[2025,6,22]],"date-time":"2025-06-22T10:52:56Z","timestamp":1750589576000},"page":"137-158","source":"Crossref","is-referenced-by-count":3,"title":["MoDitector: Module-Directed Testing for Autonomous Driving Systems"],"prefix":"10.1145","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1219-0732","authenticated-orcid":false,"given":"Renzhi","family":"Wang","sequence":"first","affiliation":[{"name":"University of Alberta, Edmonton, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8982-1483","authenticated-orcid":false,"given":"Mingfei","family":"Cheng","sequence":"additional","affiliation":[{"name":"Singapore Management University, Singapore, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1288-6502","authenticated-orcid":false,"given":"Xiaofei","family":"Xie","sequence":"additional","affiliation":[{"name":"Singapore Management University, Singapore, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1583-7570","authenticated-orcid":false,"given":"Yuan","family":"Zhou","sequence":"additional","affiliation":[{"name":"Zhejiang Sci-Tech University, Singapore, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8621-2420","authenticated-orcid":false,"given":"Lei","family":"Ma","sequence":"additional","affiliation":[{"name":"University of Tokyo, Tokyo, Japan"},{"name":"University of Alberta, Edmonton, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,6,22]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"[n. d.]. GitHub - autowarefoundation\/autoware: Autoware - the world\u2019s leading open-source software project for autonomous driving \u2014 github.com. https:\/\/github.com\/autowarefoundation\/autoware [Accessed 08-Jul-2023]"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238192"},{"key":"e_1_2_1_3_1","unstructured":"Comma ai. 2023. Openpilot. https:\/\/comma.ai\/openpilot"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/0967-0661(93)91394-C"},{"key":"e_1_2_1_5_1","unstructured":"Baidu. 2023. Baidu-Apollo. https:\/\/www.apollo.auto"},{"key":"e_1_2_1_6_1","volume-title":"2020 IEEE International Black Sea Conference on Communications and Networking (BlackSeaCom). 1\u20136.","author":"Bothe Shruti","year":"2020","unstructured":"Shruti Bothe, Usama Masood, Hasan Farooq, and Ali Imran. 2020. Neuromorphic AI empowered root cause analysis of faults in emerging networks. In 2020 IEEE International Black Sea Conference on Communications and Networking (BlackSeaCom). 1\u20136."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2010.2048314"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/IV47402.2020.9304662"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICST46399.2020.00045"},{"key":"e_1_2_1_10_1","volume-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition. 15334\u201315342","author":"Cheng Bowen","year":"2021","unstructured":"Bowen Cheng, Ross Girshick, Piotr Doll\u00e1r, Alexander C Berg, and Alexander Kirillov. 2021. Boundary IoU: Improving object-centric image segmentation evaluation. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition. 15334\u201315342."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3597926.3598072"},{"key":"e_1_2_1_12_1","unstructured":"Mingfei Cheng Yuan Zhou and Xiaofei Xie. 2024. DriveTester: A Unified Platform for Simulation-Based Autonomous Driving Testing. arXiv preprint arXiv:2412.12656."},{"key":"e_1_2_1_13_1","unstructured":"Mingfei Cheng Yuan Zhou Xiaofei Xie Junjie Wang Guozhu Meng and Kairui Yang. 2024. Evaluating Decision Optimality of Autonomous Driving via Metamorphic Testing. arXiv preprint arXiv:2402.18393."},{"key":"e_1_2_1_14_1","volume-title":"29th Annual Network and Distributed System Security Symposium, NDSS","author":"Choi Hongjun","year":"2022","unstructured":"Hongjun Choi, Zhiyuan Cheng, and Xiangyu Zhang. 2022. RVPLAYER: Robotic Vehicle Forensics by Replay with What-if Reasoning. In 29th Annual Network and Distributed System Security Symposium, NDSS 2022."},{"key":"e_1_2_1_15_1","article-title":"A survey on safety-critical driving scenario generation\u2014A methodological perspective","author":"Ding Wenhao","year":"2023","unstructured":"Wenhao Ding, Chejian Xu, Mansur Arief, Haohong Lin, Bo Li, and Ding Zhao. 2023. A survey on safety-critical driving scenario generation\u2014A methodological perspective. IEEE Transactions on Intelligent Transportation Systems.","journal-title":"IEEE Transactions on Intelligent Transportation Systems."},{"key":"e_1_2_1_16_1","volume-title":"International Conference on Formal Modeling and Analysis of Timed Systems. 92\u2013106","author":"Donz\u00e9 Alexandre","year":"2010","unstructured":"Alexandre Donz\u00e9 and Oded Maler. 2010. Robust satisfaction of temporal logic over real-valued signals. In International Conference on Formal Modeling and Analysis of Timed Systems. 92\u2013106."},{"key":"e_1_2_1_17_1","volume-title":"Conference on robot learning. 1\u201316","author":"Dosovitskiy Alexey","year":"2017","unstructured":"Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. 2017. CARLA: An open urban driving simulator. In Conference on robot learning. 1\u201316."},{"key":"e_1_2_1_18_1","unstructured":"Electrek. 2023. Damning Footage Shows Tesla Vehicles on Autopilot Crashing into Police. https:\/\/electrek.co\/2023\/08\/09\/damning-footage-shows-tesla-vehicles-autopi"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2020.2969927"},{"key":"e_1_2_1_20_1","volume-title":"Dense reinforcement learning for safety validation of autonomous vehicles. Nature, 615, 7953","author":"Feng Shuo","year":"2023","unstructured":"Shuo Feng, Haowei Sun, Xintao Yan, Haojie Zhu, Zhengxia Zou, Shengyin Shen, and Henry X Liu. 2023. Dense reinforcement learning for safety validation of autonomous vehicles. Nature, 615, 7953 (2023), 620\u2013627."},{"key":"e_1_2_1_21_1","volume-title":"ROCAS: Root Cause Analysis of Autonomous Driving Accidents via Cyber-Physical Co-mutation. arXiv preprint arXiv:2409.07774.","author":"Feng Shiwei","year":"2024","unstructured":"Shiwei Feng, Yapeng Ye, Qingkai Shi, Zhiyuan Cheng, Xiangzhe Xu, Siyuan Cheng, Hongjun Choi, and Xiangyu Zhang. 2024. ROCAS: Root Cause Analysis of Autonomous Driving Accidents via Cyber-Physical Co-mutation. arXiv preprint arXiv:2409.07774."},{"key":"e_1_2_1_22_1","unstructured":"Weiwei Fu Heqing Huang Yifan Zhang Ke Zhang Jin Huang Wei-Bin Lee and Jianping Wang. 2024. ICSFuzz: Collision Detector Bug Discovery in Autonomous Driving Simulators. arXiv preprint arXiv:2408.05694."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.81"},{"key":"e_1_2_1_24_1","volume-title":"2021 IEEE International Conference on Robotics and Automation (ICRA). 8806\u20138813","author":"Gog Ionel","year":"2021","unstructured":"Ionel Gog, Sukrit Kalra, Peter Schafhalter, Matthew A Wright, Joseph E Gonzalez, and Ion Stoica. 2021. Pylot: A modular platform for exploring latency-accuracy tradeoffs in autonomous vehicles. In 2021 IEEE International Conference on Robotics and Automation (ICRA). 8806\u20138813."},{"key":"e_1_2_1_25_1","volume-title":"2021 IEEE\/ACM 14th International Workshop on Search-Based Software Testing (SBST). 44\u201347","author":"Han Seunghee","year":"2021","unstructured":"Seunghee Han, Jaeuk Kim, Geon Kim, Jaemin Cho, Jiin Kim, and Shin Yoo. 2021. Preliminary evaluation of path-aware crossover operators for search-based test data generation for autonomous driving. In 2021 IEEE\/ACM 14th International Workshop on Search-Based Software Testing (SBST). 44\u201347."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510188"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00155"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2023.3280810"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2022.106936"},{"key":"e_1_2_1_30_1","volume-title":"PID control","author":"Johnson Michael A","unstructured":"Michael A Johnson and Mohammad H Moradi. 2005. PID control. Springer."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3468601"},{"key":"e_1_2_1_32_1","volume-title":"29th USENIX Security Symposium (USENIX Security 20)","author":"Kim Taegyu","year":"2020","unstructured":"Taegyu Kim, Chung Hwan Kim, Altay Ozen, Fan Fei, Zhan Tu, Xiangyu Zhang, Xinyan Deng, Dave Jing Tian, and Dongyan Xu. 2020. From control model to program: Investigating robotic aerial vehicle accidents with $MAYDAY$. In 29th USENIX Security Symposium (USENIX Security 20). 913\u2013930."},{"key":"e_1_2_1_33_1","volume-title":"2018 21st International conference on intelligent transportation systems (ITSC). 2811\u20132818","author":"Kruber Friedrich","year":"2018","unstructured":"Friedrich Kruber, Jonas Wurst, and Michael Botsch. 2018. An unsupervised random forest clustering technique for automatic traffic scenario categorization. In 2018 21st International conference on intelligent transportation systems (ITSC). 2811\u20132818."},{"key":"e_1_2_1_34_1","unstructured":"lgsvl. 2022. LGSVL Sunsetting. https:\/\/github.com\/lgsvl\/simulator"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE5003.2020.00012"},{"key":"e_1_2_1_36_1","volume-title":"2023 38th IEEE\/ACM International Conference on Automated Software Engineering (ASE). 243\u2013254","author":"Li Zhuo","year":"2023","unstructured":"Zhuo Li, Xiongfei Wu, Derui Zhu, Mingfei Cheng, Siyuan Chen, Fuyuan Zhang, Xiaofei Xie, Lei Ma, and Jianjun Zhao. 2023. Generative model-based testing on decision-making policies. In 2023 38th IEEE\/ACM International Conference on Automated Software Engineering (ASE). 243\u2013254."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549111"},{"key":"e_1_2_1_38_1","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence. 38","author":"Ma Chen","year":"2024","unstructured":"Chen Ma, Ningfei Wang, Qi Alfred Chen, and Chao Shen. 2024. Slowtrack: Increasing the latency of camera-based perception in autonomous driving using adversarial examples. In Proceedings of the AAAI Conference on Artificial Intelligence. 38, 4062\u20134070."},{"key":"e_1_2_1_39_1","unstructured":"Wassim G Najm John D Smith and Mikio Yanagisawa. 2007. Pre-crash scenario typology for crash avoidance research. United States. National Highway Traffic Safety Administration."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3533767.3534388"},{"key":"e_1_2_1_41_1","volume-title":"2021 International Conference on Communications, Computing, Cybersecurity, and Informatics (CCCI). 1\u20136.","author":"Rana Ashish","year":"2021","unstructured":"Ashish Rana and Avleen Malhi. 2021. Building safer autonomous agents by leveraging risky driving behavior knowledge. In 2021 International Conference on Communications, Computing, Cybersecurity, and Informatics (CCCI). 1\u20136."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/ITSC45102.2020.9294422"},{"key":"e_1_2_1_43_1","doi-asserted-by":"crossref","unstructured":"Jieke Shi Zhou Yang Junda He Bowen Xu Dongsun Kim DongGyun Han and David Lo. 2024. Finding Safety Violations of AI-Enabled Control Systems through the Lens of Synthesized Proxy Programs. arXiv preprint arXiv:2410.04986.","DOI":"10.1145\/3715105"},{"key":"e_1_2_1_44_1","volume-title":"Proceedings of the 44th International Conference on Software Engineering: Software Engineering in Practice. 343\u2013352","author":"Song Jiayang","year":"2022","unstructured":"Jiayang Song, Deyun Lyu, Zhenya Zhang, Zhijie Wang, Tianyi Zhang, and Lei Ma. 2022. When cyber-physical systems meet AI: a benchmark, an evaluation, and a way forward. In Proceedings of the 44th International Conference on Software Engineering: Software Engineering in Practice. 343\u2013352."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3551349.3556968"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380353"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE51524.2021.9678735"},{"key":"e_1_2_1_48_1","unstructured":"Eric Thorn Shawn C Kimmel Michelle Chaka and Booz Allen Hamilton. 2018. A framework for automated driving system testable cases and scenarios. United States. Department of Transportation. National Highway Traffic Safety."},{"key":"e_1_2_1_49_1","volume-title":"24th International Technical Conference on the Enhanced Safety of Vehicles (ESV) National Highway Traffic Safety Administration.","author":"van der Made Robin","year":"2015","unstructured":"Robin van der Made, Martijn Tideman, Ulrich Lages, Roman Katz, and Martin Spencer. 2015. Automated generation of virtual driving scenarios from test drive data. In 24th International Technical Conference on the Enhanced Safety of Vehicles (ESV) National Highway Traffic Safety Administration."},{"key":"e_1_2_1_50_1","volume-title":"MORTAR: A Model-based Runtime Action Repair Framework for AI-enabled Cyber-Physical Systems. arXiv preprint arXiv:2408.03892.","author":"Wang Renzhi","year":"2024","unstructured":"Renzhi Wang, Zhehua Zhou, Jiayang Song, Xuan Xie, Xiaofei Xie, and Lei Ma. 2024. MORTAR: A Model-based Runtime Action Repair Framework for AI-enabled Cyber-Physical Systems. arXiv preprint arXiv:2408.03892."},{"key":"e_1_2_1_51_1","unstructured":"Zhijie Wang Yuheng Huang Lei Ma Haruki Yokoyama Susumu Tokumoto and Kazuki Munakata. 2022. An Exploratory Study of AI System Risk Assessment from the Lens of Data Distribution and Uncertainty. arXiv preprint arXiv:2212.06828."},{"key":"e_1_2_1_52_1","volume-title":"Mosaic: Model-based Safety Analysis Framework for AI-enabled Cyber-Physical Systems. arXiv preprint arXiv:2305.03882.","author":"Xie Xuan","year":"2023","unstructured":"Xuan Xie, Jiayang Song, Zhehua Zhou, Fuyuan Zhang, and Lei Ma. 2023. Mosaic: Model-based Safety Analysis Framework for AI-enabled Cyber-Physical Systems. arXiv preprint arXiv:2305.03882."},{"key":"e_1_2_1_53_1","unstructured":"Guangba Yu Gou Tan Haojia Huang Zhenyu Zhang Pengfei Chen Roberto Natella and Zibin Zheng. 2024. A Survey on Failure Analysis and Fault Injection in AI Systems. arXiv preprint arXiv:2407.00125."},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01473"},{"key":"e_1_2_1_55_1","volume-title":"33rd USENIX Security Symposium (USENIX Security 24)","author":"Zhang Qingzhao","year":"2024","unstructured":"Qingzhao Zhang, Shuowei Jin, Ruiyang Zhu, Jiachen Sun, Xumiao Zhang, Qi Alfred Chen, and Z Morley Mao. 2024. On data fabrication in collaborative vehicular perception: Attacks and countermeasures. In 33rd USENIX Security Symposium (USENIX Security 24). 6309\u20136326."},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2022.07.042"},{"key":"e_1_2_1_57_1","volume-title":"Proceedings of the AAAI conference on artificial intelligence. 34","author":"Zheng Zhaohui","year":"2020","unstructured":"Zhaohui Zheng, Ping Wang, Wei Liu, Jinze Li, Rongguang Ye, and Dongwei Ren. 2020. Distance-IoU loss: Faster and better learning for bounding box regression. In Proceedings of the AAAI conference on artificial intelligence. 34, 12993\u201313000."},{"key":"e_1_2_1_58_1","doi-asserted-by":"crossref","first-page":"1860","DOI":"10.1109\/TSE.2022.3195640","article-title":"Neural network guided evolutionary fuzzing for finding traffic violations of autonomous vehicles","volume":"49","author":"Zhong Ziyuan","year":"2022","unstructured":"Ziyuan Zhong, Gail Kaiser, and Baishakhi Ray. 2022. Neural network guided evolutionary fuzzing for finding traffic violations of autonomous vehicles. IEEE Transactions on Software Engineering, 49, 4 (2022), 1860\u20131875.","journal-title":"IEEE Transactions on Software Engineering"},{"key":"e_1_2_1_59_1","volume-title":"Yang Liu, and Baishakhi Ray.","author":"Zhong Ziyuan","year":"2021","unstructured":"Ziyuan Zhong, Yun Tang, Yuan Zhou, Vania de Oliveira Neves, Yang Liu, and Baishakhi Ray. 2021. A survey on scenario-based testing for automated driving systems in high-fidelity simulation. arXiv preprint arXiv:2112.00964."},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2023.3254142"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/3664605"}],"container-title":["Proceedings of the ACM on Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3728876","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,16]],"date-time":"2025-07-16T16:54:20Z","timestamp":1752684860000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3728876"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,22]]},"references-count":61,"journal-issue":{"issue":"ISSTA","published-print":{"date-parts":[[2025,6,22]]}},"alternative-id":["10.1145\/3728876"],"URL":"https:\/\/doi.org\/10.1145\/3728876","relation":{},"ISSN":["2994-970X"],"issn-type":[{"value":"2994-970X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,22]]}}}