{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,12]],"date-time":"2026-06-12T19:03:39Z","timestamp":1781291019308,"version":"3.54.1"},"reference-count":39,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2023,10,6]],"date-time":"2023-10-06T00:00:00Z","timestamp":1696550400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Korea Agency for Infrastructure Technology Advancement (KAIA)","award":["RS-2021-KA160637"],"award-info":[{"award-number":["RS-2021-KA160637"]}]},{"name":"Ministry of Land, Infrastructure, and Transport","award":["RS-2021-KA160637"],"award-info":[{"award-number":["RS-2021-KA160637"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>With advances in the technology applied to automated driving systems (ADSs), active efforts have been made to evaluate the safety of ADS in various complex situations using simulations. In accordance with these efforts, numerous institutions have developed single-scenario pools that reflect a variety of road and traffic characteristics and ADS performances. However, a single scenario has limitations in comprehensively evaluating the performance of complex ADS. Therefore, this study proposed a methodology that combines and transforms single scenarios into multiple scenarios. This aided in continuously evaluating the ADS performance over entire road segments and implemented this methodology in the simulations.<\/jats:p>","DOI":"10.3390\/s23198271","type":"journal-article","created":{"date-parts":[[2023,10,6]],"date-time":"2023-10-06T08:29:12Z","timestamp":1696580952000},"page":"8271","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Method of Evaluating Multiple Scenarios in a Single Simulation Run for Automated Vehicle Assessment"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2532-1452","authenticated-orcid":false,"given":"Inyoung","family":"Kim","sequence":"first","affiliation":[{"name":"Department of D.N.A. Convergence, Ajou University, Suwon 16499, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Donghyo","family":"Kang","sequence":"additional","affiliation":[{"name":"Department of D.N.A. Convergence, Ajou University, Suwon 16499, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9128-3795","authenticated-orcid":false,"given":"Harim","family":"Jeong","sequence":"additional","affiliation":[{"name":"Department of Transportation System Engineering, Ajou University, Suwon 16499, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6633-997X","authenticated-orcid":false,"given":"Soomok","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of AI Mobility Engineering, Ajou University, Suwon 16499, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5618-7933","authenticated-orcid":false,"given":"Ilsoo","family":"Yun","sequence":"additional","affiliation":[{"name":"Department of Transportation System Engineering, Ajou University, Suwon 16499, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,10,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Menzel, T., Bagschik, G., Isensee, L., Schomburg, A., and Maurer, M. (2019, January 9\u201312). From functional to logical scenarios: Detailing a keyword-based scenario description for execution in a simulation environment. Proceedings of the 2019 IEEE Intelligent Vehicles Symposium, Paris, France.","DOI":"10.1109\/IVS.2019.8814099"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Xinxin, Z., Fei, L., and Xiangbin, W. (November, January 19). Csg: Critical scenario generation from real traffic accidents. Proceedings of the 2020 IEEE Intelligent Vehicles Symposium, Las Vegas, NV, USA.","DOI":"10.1109\/IV47402.2020.9304609"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"105664","DOI":"10.1016\/j.aap.2020.105664","article-title":"Safety assessment of highly automated driving systems in test tracks: A new framework","volume":"144","author":"Feng","year":"2020","journal-title":"Accid. Anal. Prev."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"14088","DOI":"10.1109\/TITS.2021.3136353","article-title":"Scenario-based test automation for highly automated vehicles: A review and paving the way for systematic safety assurance","volume":"23","author":"Sun","year":"2021","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Yang, Y., Wang, Y., Yin, C., and Ji, Q. (2021, January 29\u201331). Simulation Testing Scenario Generation for Comfort Evaluation of Automated Vehicles. Proceedings of the 2021 5th CAA International Conference on Vehicular Control and Intelligence, Tianjin, China.","DOI":"10.1109\/CVCI54083.2021.9661128"},{"key":"ref_6","unstructured":"(2023, March 28). PEGASUS Project. Scenario Description and Knowledge-Based Scenario Generation; pp. 1\u20133. Available online: https:\/\/www.pegasusprojekt.de\/files\/tmpl\/Pegasus-Abschlussveranstaltung\/05_Scenario_Description_and_Knowledge-Based_Scenario_Generation.pdf."},{"key":"ref_7","unstructured":"Winner, H., Lemmer, K., Form, T., and Mazzega, J. (2019). Road Vehicle Automation 5, Springer."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"87456","DOI":"10.1109\/ACCESS.2020.2993730","article-title":"Survey on scenario-based safety assessment of automated vehicles","volume":"8","author":"Riedmaier","year":"2020","journal-title":"IEEE Access"},{"key":"ref_9","unstructured":"(2023, March 28). ASAM Standards. Available online: https:\/\/www.asam.net\/standards\/."},{"key":"ref_10","unstructured":"Otto, T., and Auerswald, R. (2020). Automatisiertes Fahren 2019: Von der Fahrerassistenz zum Autonomen Fahren 5. Internationale ATZ-Fachtagung, Springer Fachmedien Wiesbaden."},{"key":"ref_11","unstructured":"NHTSA (2023, March 30). Review of Simulation Frameworks and Standards Related to Driving Scenarios; pp. 1\u201340, Available online: https:\/\/rosap.ntl.bts.gov\/view\/dot\/43621."},{"key":"ref_12","unstructured":"ASAM (2023, March 28). OpenLABEL Version 1.0.0 User Guide. Available online: https:\/\/www.asam.net\/standards\/detail\/openlabel\/."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ko, W., Park, S., Yun, J., Park, S., and Yun, I. (2022). Development of a framework for generating driving safety assessment scenarios for automated vehicles. Sensors, 22.","DOI":"10.3390\/s22166031"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Park, S., Park, S., Jeong, H., Yun, I., and So, J. (2021). Scenario-mining for level 4 automated vehicle safety assessment from real accident situations in urban areas using a natural language process. Sensors, 21.","DOI":"10.3390\/s21206929"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"102820","DOI":"10.1016\/j.trc.2020.102820","article-title":"Risk assessment based collision avoidance decision-making for autonomous vehicles in multi-scenarios","volume":"122","author":"Li","year":"2021","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Solmaz, S., and Holzinger, F. (2019, January 4\u20138). A novel testbench for development, calibration and functional testing of ADAS\/AD functions. Proceedings of the 2019 IEEE International Conference on Connected Vehicles and Expo, Graz, Austria.","DOI":"10.1109\/ICCVE45908.2019.8965225"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1109\/JSTSP.2022.3221271","article-title":"Mobility-aware cooperative caching in vehicular edge computing based on asynchronous federated and deep reinforcement learning","volume":"17","author":"Wu","year":"2023","journal-title":"IEEE J. Sel. Top. Signal Process."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1573","DOI":"10.1109\/TITS.2020.2972211","article-title":"Testing scenario library generation for connected and automated vehicles, part I: Methodology","volume":"22","author":"Feng","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Kerber, J., Wagner, S., Groh, K., Notz, D., K\u00fchbeck, T., Watzenig, D., and Knoll, A. (November, January 19). Clustering of the scenario space for the assessment of automated driving. Proceedings of the 2020 IEEE Intelligent Vehicles Symposium, Las Vegas, NV, USA.","DOI":"10.1109\/IV47402.2020.9304646"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Kl\u00fcck, F., Li, Y., Nica, M., Tao, J., and Wotawa, F. (2018, January 15\u201318). Using ontologies for test suites generation for automated and autonomous driving functions. Proceedings of the 2018 IEEE International Symposium on Software Reliability Engineering Workshops, Memphis, TN, USA.","DOI":"10.1109\/ISSREW.2018.00-20"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"117940","DOI":"10.1016\/j.eswa.2022.117940","article-title":"An objective evaluation method for automated vehicle virtual test","volume":"206","author":"Zhao","year":"2022","journal-title":"Expert Syst. Appl."},{"key":"ref_22","unstructured":"UNECE (2023, March 30). New Assessment\/Test Method for Automated Driving (NATM) Master Document (Final Draft). 4 (February); pp. 1\u20135. Available online: https:\/\/unece.org\/sites\/default\/files\/2021-01\/GRVA-09-07e.pdf."},{"key":"ref_23","unstructured":"American Association of State Highway and Transportation Officials (AASHTO) (2023, March 30). Available online: https:\/\/www.fpp.uni-lj.si\/mma\/HSM.pdf\/2019060611143076\/?m=1559812470."},{"key":"ref_24","first-page":"1594","article-title":"Spatial analysis methods for identifying hazardous locations on expressways in Korea","volume":"22","author":"Park","year":"2015","journal-title":"Sci. Iran."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"631","DOI":"10.1007\/978-981-13-1328-8_81","article-title":"K-Means and CRP-Based Characteristic Investigating Method of Traffic Accidents with Automated Speed Enforcement Cameras","volume":"Volume 12","author":"Park","year":"2019","journal-title":"Advanced Multimedia and Ubiquitous Engineering: MUE\/FutureTech 2018"},{"key":"ref_26","first-page":"90","article-title":"Review on determining number of Cluster in K-Means Clustering","volume":"1","author":"Kodinariya","year":"2013","journal-title":"Int. J."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"80716","DOI":"10.1109\/ACCESS.2020.2988796","article-title":"Unsupervised K-means clustering algorithm","volume":"8","author":"Sinaga","year":"2020","journal-title":"IEEE Access"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1016\/j.tra.2020.03.024","article-title":"Automated drivability: Toward an assessment of the spatial deployment of level 4 automated vehicles","volume":"136","author":"Soteropoulos","year":"2020","journal-title":"Transp. Res. Part A Policy Pract."},{"key":"ref_29","unstructured":"ASAM (2023, March 28). OpenODD Version 1.0.0 Concept Paper. Available online: https:\/\/www.asam.net\/standards\/detail\/openodd\/."},{"key":"ref_30","unstructured":"ASAM (2023, March 28). Using Metrics with ODD Specification. Available online: https:\/\/www.asam.net\/index.php?eID=dumpFile&t=f&f=4550&token=cd56bf3f926bba1c795bbaf66848de73f6b4ad67."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"10694","DOI":"10.3182\/20140824-6-ZA-1003.00353","article-title":"Collision avoidance with automatic braking and swerving","volume":"47","author":"Ackermann","year":"2014","journal-title":"IFAC Proc. Vol."},{"key":"ref_32","unstructured":"ISO (2023, March 28). Road Vehicles\u2014Scenario-Based Safety Evaluation Framework for Automated Driving Systems; pp. 1\u201394. Available online: https:\/\/www.iso.org\/standard\/78951.html."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1016\/j.trf.2014.06.015","article-title":"On computing time-to-collision for automation scenarios","volume":"27","author":"Schwarz","year":"2014","journal-title":"Transp. Res. Part F Traffic Psychol. Behav."},{"key":"ref_34","first-page":"2881","article-title":"Driving safety performance assessment metrics for ads-equipped vehicles","volume":"2","author":"Wishart","year":"2020","journal-title":"SAE Tech. Pap."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Wang, F., Franco-Penya, H.-H., Kelleher, J.D., Pugh, J., and Ross, R. (2017, January 15\u201320). An analysis of the application of simplified silhouette to the evaluation of k-means clustering validity. Proceedings of the Machine Learning and Data Mining in Pattern Recognition: 13th International Conference, New York, NY, USA.","DOI":"10.1007\/978-3-319-62416-7_21"},{"key":"ref_36","unstructured":"Yang, X., Wang, Y., Wu, D., and Ma, A. (December, January 30). K-means based clustering on mobile usage for social network analysis purpose. Proceedings of the 2010 6th International Conference on Advanced Information Management and Service, Seoul, Republic of Korea."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Deveaux, D., Higuchi, T., U\u00e7ar, S., Wang, C.-H., H\u00e4rri, J., and Altintas, O. (2021, January 19\u201322). Extraction of risk knowledge from time to collision variation in roundabouts. Proceedings of the 2021 IEEE International Intelligent Transportation Systems Conference, Indianapolis, IN, USA.","DOI":"10.1109\/ITSC48978.2021.9564570"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Wachenfeld, W., Junietz, P., Wenzel, R., and Winner, H. (2016, January 19\u201322). The worst-time-to-collision metric for situation identification. Proceedings of the 2016 IEEE Intelligent Vehicles Symposium, Gothenburg, Sweden.","DOI":"10.1109\/IVS.2016.7535468"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1016\/j.iatssr.2017.02.001","article-title":"Application of proximal surrogate indicators for safety evaluation: A review of recent developments and research needs","volume":"41","author":"Mahmud","year":"2017","journal-title":"IATSS Res."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/19\/8271\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:01:58Z","timestamp":1760130118000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/19\/8271"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,6]]},"references-count":39,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2023,10]]}},"alternative-id":["s23198271"],"URL":"https:\/\/doi.org\/10.3390\/s23198271","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,10,6]]}}}