{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,17]],"date-time":"2026-03-17T20:23:50Z","timestamp":1773779030501,"version":"3.50.1"},"reference-count":42,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2024,12,3]],"date-time":"2024-12-03T00:00:00Z","timestamp":1733184000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>The validation of autonomous vehicles remains a vexing challenge for the automotive industry\u2019s goal of fully autonomous driving. The systematic hierarchization of the test scenarios would provide valuable insights for the development, testing, and verification of autonomous vehicles, enabling nuanced performance evaluations based on scenario complexity. In this paper, an information entropy-based quantification method is proposed to evaluate the complexity of autonomous vehicle validation scenarios. The proposed method addresses the dynamic uncertainties within driving scenarios in a comprehensive way which includes the unpredictability of dynamic agents such as autonomous vehicles, human-driven vehicles, and pedestrians. The numerical complexity calculation of the approach and the ranking of the scenarios are presented through sample scenarios. To automate processes and assist with the calculations, a novel software tool with a user-friendly interface is developed. The performance of the approach is also evaluated through six example driving scenarios, then through extensive simulation using an open-source microscopic traffic simulator. The performance evaluation results confirm the numerical classification and demonstrate the method\u2019s adaptability to diverse scenarios with a comparison of complexity calculation ranking to the ratio of collision, near collision, and normal operation tests observed during simulation testing. The proposed quantification method contributes to the improvement of autonomous vehicle validation procedures by addressing the multifaceted nature of scenario complexities. Beyond advancing the field of validation, the approach also aligns with the broad and active drive of the industry for the widespread deployment of fully autonomous driving.<\/jats:p>","DOI":"10.3390\/info15120772","type":"journal-article","created":{"date-parts":[[2024,12,4]],"date-time":"2024-12-04T06:38:25Z","timestamp":1733294305000},"page":"772","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Complexity Evaluation of Test Scenarios for Autonomous Vehicle Safety Validation Using Information Theory"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-4822-2601","authenticated-orcid":false,"given":"Maja","family":"Issler","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering and Computer Science, Embry-Riddle Aeronautical University, Daytona Beach, FL 32119, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0454-5529","authenticated-orcid":false,"given":"Quentin","family":"Goss","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering and Computer Science, Embry-Riddle Aeronautical University, Daytona Beach, FL 32119, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5450-3522","authenticated-orcid":false,"given":"Mustafa \u0130lhan","family":"Akba\u015f","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering and Computer Science, Embry-Riddle Aeronautical University, Daytona Beach, FL 32119, USA"}]}],"member":"1968","published-online":{"date-parts":[[2024,12,3]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Chai, Z., Nie, T., and Becker, J. (2021). Autonomous Driving Changes the Future, Dover.","DOI":"10.1007\/978-981-15-6728-5"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"101488","DOI":"10.1016\/j.techsoc.2020.101488","article-title":"Talking about automated vehicles: What do levels of automation do?","volume":"64","author":"Hopkins","year":"2021","journal-title":"Technol. Soc."},{"key":"ref_3","unstructured":"(2021). Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles (Standard No. J3016_202104)."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"15","DOI":"10.4271\/2016-01-0128","article-title":"Challenges in autonomous vehicle testing and validation","volume":"4","author":"Koopman","year":"2016","journal-title":"SAE Int. J. Transp. Saf."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Stark, C., Medrano-Berumen, C., and Akba\u015f, M.\u0130. (2020, January 28\u201329). Generation of Autonomous Vehicle Validation Scenarios Using Crash Data. Proceedings of the 2020 SoutheastCon, Raleigh, NC, USA.","DOI":"10.1109\/SoutheastCon44009.2020.9249662"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"8867757","DOI":"10.1155\/2020\/8867757","article-title":"Safety of autonomous vehicles","volume":"2020","author":"Wang","year":"2020","journal-title":"J. Adv. Transp."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"103051","DOI":"10.1016\/j.trc.2021.103051","article-title":"Dynamic driving environment complexity quantification method and its verification","volume":"127","author":"Yu","year":"2021","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Liu, Q., Yuan, C., Zeng, Y., Chai, Y., and Shu, H. (2022, January 28\u201330). Assessment of driving task complexity of automated vehicles based on information entropy. Proceedings of the CAA International Conference on Vehicular Control and Intelligence (CVCI), Nanjing, China.","DOI":"10.1109\/CVCI56766.2022.9964585"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"87456","DOI":"10.1109\/ACCESS.2020.2993730","article-title":"Survey on scenario-based safety assessment of automated vehicles","volume":"8","author":"Riedmaier","year":"2020","journal-title":"IEEE Access"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"531","DOI":"10.1007\/s42154-023-00251-2","article-title":"Genetic Algorithm-Based SOTIF Scenario Construction for Complex Traffic Flow","volume":"6","author":"Zhao","year":"2023","journal-title":"Automot. Innov."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"438","DOI":"10.1007\/s42154-023-00231-6","article-title":"Deep Reinforcement Learning Based Decision-Making Strategy of Autonomous Vehicle in Highway Uncertain Driving Environments","volume":"6","author":"Deng","year":"2023","journal-title":"Automot. Innov."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Thompson, J.M., Goss, Q., and Akba\u015f, M.\u0130. (2023, January 17\u201319). A Strategy for Boundary Adherence and Exploration in Black-Box Testing of Autonomous Vehicles. Proceedings of the 2023 IEEE International Conference on Mobility, Operations, Services and Technologies (MOST), Detroit, MI, USA.","DOI":"10.1109\/MOST57249.2023.00028"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Akbas, M.I., Menase, M., Verma, S., and Razdan, R. (2024, January 1\u20133). PolyFlows: Modular Test Framework Design for Autonomous Vehicles. Proceedings of the 2024 IEEE International Conference on Mobility, Operations, Services and Technologies (MOST), Dallas, TX, USA.","DOI":"10.1109\/MOST60774.2024.00014"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1007\/s42154-023-00272-x","article-title":"Driver Steering Behaviour Modelling Based on Neuromuscular Dynamics and Multi-Task Time-Series Transformer","volume":"7","author":"Xing","year":"2024","journal-title":"Automot. Innov."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Ma, Y., Pan, X., Xiong, L., Xing, X., Bulut, S., and Chen, J. (2020). Definition and quantification of the complexity experienced by autonomous vehicles in the environment and driving Task. CICTP 2020, ASCE Publishing.","DOI":"10.1061\/9780784483053.087"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Zhang, L., Ma, Y., Xing, X., Xiong, L., and Chen, J. (2021, January 19\u201322). Research on the Complexity Quantification Method of Driving Scenarios Based on Information Entropy. Proceedings of the IEEE International Intelligent Transportation Systems Conference (ITSC), Indianapolis, IN, USA.","DOI":"10.1109\/ITSC48978.2021.9565025"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1007\/s42154-019-00084-y","article-title":"Active collision avoidance system design based on model predictive control with varying sampling time","volume":"3","author":"Xue","year":"2020","journal-title":"Automot. Innov."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Li, Y., and Shi, H. (2022). Advanced Driver Assistance Systems and Autonomous Vehicles: From Fundamentals to Applications, Springer Nature.","DOI":"10.1007\/978-981-19-5053-7"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"6372","DOI":"10.1109\/TITS.2023.3248121","article-title":"Online adaptive generation of critical boundary scenarios for evaluation of autonomous vehicles","volume":"24","author":"Zhou","year":"2023","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_20","unstructured":"Gray, R.M. (2023). Entropy and Information Theory, First Edition, Corrected, Springer."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1007\/s42154-021-00138-0","article-title":"A novel fault detection, identification and prediction approach for autonomous vehicle controllers using SVM","volume":"4","author":"Biddle","year":"2021","journal-title":"Automot. Innov."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"119","DOI":"10.4271\/12-04-01-0010","article-title":"Identification of Test Scenarios for Autonomous Vehicles Using Fatal Accident Data","volume":"4","author":"Aydin","year":"2021","journal-title":"SAE Intl. J. CAV"},{"key":"ref_23","first-page":"83","article-title":"Validation on safety of the intended functionality of automated vehicles: Concept development","volume":"6","author":"Hu","year":"2022","journal-title":"SAE Int. J. Connect. Autom. Veh."},{"key":"ref_24","first-page":"127221A","article-title":"A method for evaluating the complexity of test scenarios for autonomous vehicles","volume":"Volume 12722","author":"Yuan","year":"2023","journal-title":"Proceedings of the Volume 12722, Third International Conference on Mechanical, Electronics, and Electrical and Automation Control (METMS)"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Cheng, Y., Liu, Z., Gao, L., Zhao, Y., and Gao, T. (2022). Traffic risk environment impact analysis and complexity assessment of autonomous vehicles based on the potential field method. Int. J. Environ. Res. Public Health, 19.","DOI":"10.3390\/ijerph191610337"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Goss, Q., and Akba\u015f, M.\u0130. (2023, January 17\u201319). Integration of Formal Specification and Traffic Simulation for Scenario-Based Validation. Proceedings of the IEEE International Conference on Mobility, Operations, Services and Technologies (MOST), Detroit, MI, USA.","DOI":"10.1109\/MOST57249.2023.00030"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Goss, Q., Pate, W.C., and Akba\u015f, M.\u0130. (2024, January 1\u20133). An Integrated Scenario-Based Testing and Explanation Framework for Autonomous Vehicles. Proceedings of the 2024 IEEE International Conference on Mobility, Operations, Services and Technologies (MOST), Dallas, TX, USA.","DOI":"10.1109\/MOST60774.2024.00015"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Sell, R., Malayjerdi, M., Pikner, H., Razdan, R., Malayjerdi, E., and Bellone, M. (2024, January 10\u201313). Open-Source Level 4 Autonomous Shuttle for Last\u2014Mile Mobility. Proceedings of the 2024 IEEE 29th International Conference on Emerging Technologies and Factory Automation (ETFA), Padova, Italy.","DOI":"10.1109\/ETFA61755.2024.10710975"},{"key":"ref_29","unstructured":"(2024, November 30). ASAM Open Scenario V 1.2.0. ASAM Standard. Available online: https:\/\/www.asam.net\/standards\/detail\/openscenario."},{"key":"ref_30","unstructured":"(2024, November 30). ASAM Open Scenario V 2.0.0. ASAM Standard. Available online: https:\/\/www.asam.net\/standards\/detail\/openscenario\/v200\/."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Vin, E., Kashiwa, S., Rhea, M., Fremont, D.J., Kim, E., Dreossi, T., Ghosh, S., Yue, X., Sangiovanni-Vincentelli, A.L., and Seshia, S.A. (2023). 3D Environment Modeling for Falsification and Beyond with Scenic 3.0. arXiv.","DOI":"10.1007\/978-3-031-37706-8_13"},{"key":"ref_32","unstructured":"Foretellix Inc (2021). Measurable Scenario Description Language (M-SDL), Foretellix Inc.. M-SDL 21.05.0.1 Release."},{"key":"ref_33","first-page":"3391","article-title":"Specification-Based Autonomous Driving System Testing","volume":"49","author":"Zhou","year":"2023","journal-title":"IEEE Trans. Softw. Eng."},{"key":"ref_34","unstructured":"Wegener, A., Piorkowski, M., Raya, M., Hellbr\u00fcck, H., Fischer, S., and Hubaux, J.P. (May, January 29). TraCI: An Interface for Coupling Road Traffic and Network Simulators. Proceedings of the Communications and Networking Simulation Symposium (CNS), New York, NY, USA."},{"key":"ref_35","unstructured":"Foretellix (2024, November 30). The Foretify Platform. Available online: https:\/\/www.foretellix.com\/foretify-platform\/."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Dreossi, T., Fremont, D.J., Ghosh, S., Kim, E., Ravanbakhsh, H., Vazquez-Chanlatte, M., and Seshia, S.A. (2019, January 15\u201318). VERIFAI: A toolkit for the Formal Design and Analysis of Artificial Intelligence-Based Systems. Proceedings of the International Conference on Computer Aided Verification, New York City, NY, USA.","DOI":"10.1007\/978-3-030-25540-4_25"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Rong, G., Shin, B.H., Tabatabaee, H., Lu, Q., Lemke, S., Mo\u017eeiko, M., Boise, E., Uhm, G., Gerow, M., and Mehta, S. (2020). LGSVL Simulator: A High Fidelity Simulator for Autonomous Driving. arXiv.","DOI":"10.1109\/ITSC45102.2020.9294422"},{"key":"ref_38","unstructured":"Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., and Koltun, V. (2017). CARLA: An Open Urban Driving Simulator. arXiv."},{"key":"ref_39","unstructured":"Sussman, J.S. (2008). Perspectives on Intelligent Transportation Systems (ITS), Springer."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Wei, Z., Zhou, H., and Zhou, R. (2024). Risk and Complexity Assessment of Autonomous Vehicle Testing Scenarios. Appl. Sci., 14.","DOI":"10.3390\/app14219866"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"3135","DOI":"10.1109\/TITS.2019.2926042","article-title":"Is it safe to drive? An overview of factors, metrics, and datasets for driveability assessment in autonomous driving","volume":"21","author":"Guo","year":"2019","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Lopez, P.A., Behrisch, M., Bieker-Walz, L., Erdmann, J., Fl\u00f6tter\u00f6d, Y.P., Hilbrich, R., L\u00fccken, L., Rummel, J., Wagner, P., and Wie\u00dfner, E. (2018, January 4\u20137). Microscopic Traffic Simulation using SUMO. Proceedings of the IEEE International Intelligent Transportation Systems Conference(ITSC), Maui, HI, USA.","DOI":"10.1109\/ITSC.2018.8569938"}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/15\/12\/772\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:46:04Z","timestamp":1760114764000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/15\/12\/772"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,3]]},"references-count":42,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["info15120772"],"URL":"https:\/\/doi.org\/10.3390\/info15120772","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,3]]}}}