{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T05:14:01Z","timestamp":1790054041604,"version":"4.0.1"},"reference-count":56,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2024,12,23]],"date-time":"2024-12-23T00:00:00Z","timestamp":1734912000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>This survey explores the evolution of test scenario generation for autonomous vehicles (AVs), distinguishing between non-adaptive and adaptive scenario approaches. Non-adaptive scenarios, where dynamic objects follow predetermined scripts, provide repeatable and reliable tests but fail to capture the complexity and unpredictability of real-world traffic interactions. In contrast, adaptive scenarios, which adapt in real time to environmental changes, offer a more realistic simulation of traffic conditions, enabling the assessment of an AV system\u2019s adaptability, safety, and robustness. The shift from non-adaptive to adaptive scenarios is increasingly emphasized in AV research, to better evaluate system performance in complex environments. However, generating adaptive scenario is more complex and faces challenges. These include the limited diversity in behaviors, low model interpretability, and high resource requirements. Future research should focus on enhancing the efficiency of adaptive scenario generation and developing comprehensive evaluation metrics to improve the realism and effectiveness of AV testing.<\/jats:p>","DOI":"10.3390\/fi16120480","type":"journal-article","created":{"date-parts":[[2024,12,23]],"date-time":"2024-12-23T09:13:38Z","timestamp":1734945218000},"page":"480","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":19,"title":["A Survey of Scenario Generation for Automated Vehicle Testing and Validation"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-7437-5299","authenticated-orcid":false,"given":"Ziyu","family":"Wang","sequence":"first","affiliation":[{"name":"Department of Data Science and Artificial Intelligence, School of Engineering, Computer and Mathematical Sciences, Auckland University of Technology, Auckland 1010, New Zealand"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5128-2445","authenticated-orcid":false,"given":"Jing","family":"Ma","sequence":"additional","affiliation":[{"name":"Department of Data Science and Artificial Intelligence, School of Engineering, Computer and Mathematical Sciences, Auckland University of Technology, Auckland 1010, New Zealand"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9159-3718","authenticated-orcid":false,"given":"Edmund M-K","family":"Lai","sequence":"additional","affiliation":[{"name":"Department of Data Science and Artificial Intelligence, School of Engineering, Computer and Mathematical Sciences, Auckland University of Technology, Auckland 1010, New Zealand"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,12,23]]},"reference":[{"key":"ref_1","unstructured":"National Highway Traffic Safety Administration (2016). 2015 Motor Vehicle Crashes: Overview. Traffic Saf. Facts Res. Note, 2016, 1\u20139."},{"key":"ref_2","unstructured":"European Commission (2018). 2018 Road Safety Statistics: What Is Behind the Figures?, European Commission."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"6971","DOI":"10.1109\/TITS.2023.3259322","article-title":"A survey on safety-critical driving scenario generation\u2014A methodological perspective","volume":"24","author":"Ding","year":"2023","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1109\/MITS.2016.2583491","article-title":"Autonomous vehicle safety: An interdisciplinary challenge","volume":"9","author":"Koopman","year":"2017","journal-title":"IEEE Intell. Transp. Syst. Mag."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"182","DOI":"10.1016\/j.tra.2016.09.010","article-title":"Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability?","volume":"94","author":"Kalra","year":"2016","journal-title":"Transp. Res. Part A Policy Pract."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"9712","DOI":"10.1109\/TITS.2023.3266639","article-title":"Functional testing scenario library generation framework for connected and automated vehicles","volume":"24","author":"Zhu","year":"2023","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_7","unstructured":"SAE International (2018). Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles. SAE Int., 4970, 1\u20135."},{"key":"ref_8","unstructured":"On-Road Automated Driving (Orad) Committee (2024, October 15). Taxonomy & Definitions for Operational Design Domain (ODD) for Driving Automation Systems. Available online: https:\/\/www.sae.org\/standards\/content\/j3259\/."},{"key":"ref_9","unstructured":"Koopman, P., and Fratrik, F. (2019, January 27\u201328). How many operational design domains, objects, and events?. Proceedings of the Safe AI 2019: AAAI Workshop on Artificial Intelligence Safety, Honolulu, HI, USA."},{"key":"ref_10","unstructured":"Czarnecki, K. (2018). Operational design domain for automated driving systems. Taxonomy of Basic Terms, Waterloo Intelligent Systems Engineering (WISE) Lab, University of Waterloo."},{"key":"ref_11","unstructured":"Thorn, E., Kimmel, S.C., Chaka, M., and Hamilton, B.A. (2018). A Framework for Automated Driving System Testable Cases and Scenarios, United States Department of Transportation, National Highway Traffic Safety Administration. Technical Report."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Sun, B., Li, Y., Zhao, S., Zhu, X., Ma, W., Ma, F., and Wu, L. (2022). Research on the Physics\u2013Intelligence Hybrid Theory Based Dynamic Scenario Library Generation for Automated Vehicles. Sensors, 22.","DOI":"10.3390\/s22218391"},{"key":"ref_13","unstructured":"PEGASUS Project (2024, June 12). Scenario Description and Knowledge-Based Scenario Generation. Available online: https:\/\/www.pegasusprojekt.de\/files\/tmpl\/Pegasus-Abschlussveranstaltung\/05_Scenario_Description_and_Knowledge-Based_Scenario_Generation.pdf."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Ma, Y., Jiang, W., Zhang, L., Chen, J., Wang, H., Lv, C., Wang, X., and Xiong, L. (2023). Evolving testing scenario generation method and intelligence evaluation framework for automated vehicles. arXiv.","DOI":"10.1016\/j.trc.2024.104620"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1781","DOI":"10.14778\/3229863.3229867","article-title":"Automating large-scale data quality verification","volume":"11","author":"Schelter","year":"2018","journal-title":"Proc. VLDB Endow."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2702","DOI":"10.1007\/s12205-019-1287-4","article-title":"Generating traffic safety test scenarios for automated vehicles using a big data technique","volume":"23","author":"So","year":"2019","journal-title":"KSCE J. Civ. Eng."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Sagiroglu, S., and Sinanc, D. (2013, January 20\u201324). Big data: A review. Proceedings of the 2013 International Conference on Collaboration Technologies and Systems (CTS), San Diego, CA, USA.","DOI":"10.1109\/CTS.2013.6567202"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Rana, A., and Malhi, A. (2021, January 3\u20135). Building safer autonomous agents by leveraging risky driving behavior knowledge. Proceedings of the 2021 International Conference on Communications, Computing, Cybersecurity, and Informatics (CCCI), Virtual Event.","DOI":"10.1109\/CCCI52664.2021.9583209"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1016\/S0927-0507(05)80172-0","article-title":"Markov decision processes","volume":"Volume 2","author":"Puterman","year":"1990","journal-title":"Handbooks in Operations Research and Management Science"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1109\/MSP.2017.2743240","article-title":"Deep reinforcement learning: A brief survey","volume":"34","author":"Arulkumaran","year":"2017","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Arief, M., Glynn, P., and Zhao, D. (2018, January 4\u20137). An accelerated approach to safely and efficiently test pre-production autonomous vehicles on public streets. Proceedings of the 2018 21st International Conference on Intelligent Transportation Systems (ITSC), Maui, HI, USA.","DOI":"10.1109\/ITSC.2018.8569371"},{"key":"ref_22","unstructured":"Zhao, D., Peng, H., Bao, S., Nobukawa, K., LeBlanc, D.J., and Pan, C.S. (2015, January 17\u201321). Accelerated evaluation of automated vehicles using extracted naturalistic driving data. Proceedings of the 24th International Symposium of Vehicles on Road and Tracks, Gothenburg, Sweden."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"595","DOI":"10.1109\/TITS.2016.2582208","article-title":"Accelerated evaluation of automated vehicles safety in lane-change scenarios based on importance sampling techniques","volume":"18","author":"Zhao","year":"2016","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1367","DOI":"10.1287\/mnsc.35.11.1367","article-title":"Importance sampling for stochastic simulations","volume":"35","author":"Glynn","year":"1989","journal-title":"Manag. Sci."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Tan, S., Wong, K., Wang, S., Manivasagam, S., Ren, M., and Urtasun, R. (2021, January 19\u201325). Scenegen: Learning to generate realistic traffic scenes. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00095"},{"key":"ref_26","first-page":"802","article-title":"Convolutional LSTM network: A machine learning approach for precipitation nowcasting","volume":"28","author":"Shi","year":"2015","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_28","unstructured":"Yegnanarayana, B. (2009). Artificial Neural Networks, PHI Learning Pvt. Ltd."},{"key":"ref_29","unstructured":"O\u2019Shea, K. (2015). An Introduction to Convolutional Neural Networks. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Liu, H., Zhang, L., Hari, S.K.S., and Zhao, J. (2024, January 13\u201317). Safety-Critical Scenario Generation via Reinforcement Learning Based Editing. Proceedings of the 2024 IEEE International Conference on Robotics and Automation (ICRA), Yokohama, Japan.","DOI":"10.1109\/ICRA57147.2024.10611555"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1613\/jair.301","article-title":"Reinforcement learning: A survey","volume":"4","author":"Kaelbling","year":"1996","journal-title":"J. Artif. Intell. Res."},{"key":"ref_32","unstructured":"Doersch, C. (2016). Tutorial on variational autoencoders. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1109\/TNNLS.2020.2978386","article-title":"A comprehensive survey on graph neural networks","volume":"32","author":"Wu","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_34","unstructured":"Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal policy optimization algorithms. arXiv."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1573","DOI":"10.1109\/TITS.2020.2972211","article-title":"Testing scenario library generation for connected and automated vehicles, part I: Methodology","volume":"22","author":"Feng","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"5635","DOI":"10.1109\/TITS.2020.2988309","article-title":"Testing scenario library generation for connected and automated vehicles, part II: Case studies","volume":"22","author":"Feng","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1213","DOI":"10.1109\/TITS.2020.3023668","article-title":"Testing scenario library generation for connected and automated vehicles: An adaptive framework","volume":"23","author":"Feng","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1109\/MPRV.2008.80","article-title":"Openstreetmap: User-generated street maps","volume":"7","author":"Haklay","year":"2008","journal-title":"IEEE Pervasive Comput."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1287\/mnsc.28.1.1","article-title":"State of the art\u2014A survey of partially observable Markov decision processes: Theory, models, and algorithms","volume":"28","author":"Monahan","year":"1982","journal-title":"Manag. Sci."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"712","DOI":"10.1109\/TITS.2019.2962338","article-title":"A survey of deep learning applications to autonomous vehicle control","volume":"22","author":"Kuutti","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"330","DOI":"10.1007\/s10683-020-09656-w","article-title":"Does level-k behavior imply level-k thinking?","volume":"24","author":"Jin","year":"2021","journal-title":"Exp. Econ."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1721","DOI":"10.1111\/j.1468-0262.2007.00810.x","article-title":"Level-k auctions: Can a nonequilibrium model of strategic thinking explain the winner\u2019s curse and overbidding in private-value auctions?","volume":"75","author":"Crawford","year":"2007","journal-title":"Econometrica"},{"key":"ref_43","unstructured":"Fujimoto, S., Hoof, H., and Meger, D. (2018, January 10\u201315). Addressing function approximation error in actor-critic methods. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"8655514","DOI":"10.1155\/2022\/8655514","article-title":"Heterogeneous driver modeling and corner scenarios sampling for automated vehicles testing","volume":"2022","author":"Ge","year":"2022","journal-title":"J. Adv. Transp."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Wei, Z., Huang, H., Zhang, G., Zhou, R., Luo, X., Li, S., and Zhou, H. (2024). Interactive Critical Scenario Generation for Autonomous Vehicles Testing Based on In-depth Crash Data Using Reinforcement Learning. IEEE Trans. Intell. Veh.","DOI":"10.1109\/TIV.2024.3415961"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"587","DOI":"10.1177\/03611981211018697","article-title":"Corner case generation and analysis for safety assessment of autonomous vehicles","volume":"2675","author":"Sun","year":"2021","journal-title":"Transp. Res. Rec."},{"key":"ref_47","unstructured":"Roderick, M., MacGlashan, J., and Tellex, S. (2017). Implementing the deep q-network. arXiv."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1002\/wics.101","article-title":"Principal component analysis","volume":"2","author":"Abdi","year":"2010","journal-title":"Wiley Interdiscip. Rev. Comput. Stat."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1109\/3477.764879","article-title":"Genetic K-means algorithm","volume":"29","author":"Krishna","year":"1999","journal-title":"IEEE Trans. Syst. Man Cybern. Part B"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Khan, K., Rehman, S.U., Aziz, K., Fong, S., and Sarasvady, S. (2014, January 17\u201319). DBSCAN: Past, present and future. Proceedings of the the Fifth International Conference on the Applications of Digital Information and Web Technologies (ICADIWT 2014), Bangalore, India.","DOI":"10.1109\/ICADIWT.2014.6814687"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"10333","DOI":"10.1109\/TITS.2021.3091477","article-title":"Adversarial evaluation of autonomous vehicles in lane-change scenarios","volume":"23","author":"Chen","year":"2021","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1002\/wics.56","article-title":"Importance sampling: A review","volume":"2","author":"Tokdar","year":"2010","journal-title":"Wiley Interdiscip. Rev. Comput. Stat."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"748","DOI":"10.1038\/s41467-021-21007-8","article-title":"Intelligent driving intelligence test for autonomous vehicles with naturalistic and adversarial environment","volume":"12","author":"Feng","year":"2021","journal-title":"Nat. Commun."},{"key":"ref_54","first-page":"21","article-title":"Imitation learning: A survey of learning methods","volume":"50","author":"Hussein","year":"2017","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"107279","DOI":"10.1016\/j.aap.2023.107279","article-title":"A dynamic test scenario generation method for autonomous vehicles based on conditional generative adversarial imitation learning","volume":"194","author":"Jia","year":"2024","journal-title":"Accid. Anal. Prev."},{"key":"ref_56","unstructured":"Ho, J., and Ermon, S. (2016). Generative adversarial imitation learning. Advances in Neural Information Processing Systems 29, Curran Associates Inc."}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/16\/12\/480\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:58:30Z","timestamp":1760115510000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/16\/12\/480"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,23]]},"references-count":56,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["fi16120480"],"URL":"https:\/\/doi.org\/10.3390\/fi16120480","relation":{},"ISSN":["1999-5903"],"issn-type":[{"value":"1999-5903","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,23]]}}}