{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,4,6]],"date-time":"2025-04-06T04:21:58Z","timestamp":1743913318037,"version":"3.40.3"},"reference-count":46,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,12,16]],"date-time":"2024-12-16T00:00:00Z","timestamp":1734307200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,12,16]],"date-time":"2024-12-16T00:00:00Z","timestamp":1734307200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100007210","name":"RWTH Aachen University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100007210","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Autom Softw Eng"],"published-print":{"date-parts":[[2025,5]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>The scenario-based verification and validation of highly automated driving functions requires extensive testing using a mix of interconnected test methods with varying degrees of virtualization, ranging from faster-than-real-time scenario exploration in software-in-the-loop simulations to in-vehicle testing. The efficiency of scenario-based test procedures is continuously improving, especially in the simulation domain with the introduction of parallel execution in the cloud, parameter variation algorithms and established standards for driving scenario specification. In contrast, current test case specifications are very tool- and project-specific and often not machine-readable, hindering the exchange, reuse and automation of scenario-based tests across all test platforms. This paper presents a novel machine-readable test specification format for scenario-based testing implemented as an XML schema. Its data structure incorporates logical or concrete scenarios within preconditions, inputs and pass criteria for automated driving functions following established test standards such as ISO\/IEC\/IEEE 29119. The format enables a tool-agnostic specification, reuse and exchange of scenario-based tests for simulation-based and in-vehicle testing. A cloud-simulation workflow has been developed that exploits the automation potential offered by the format. By means of testing a highway ramp-on function, a logical test case is generated and automatically imported into the simulator. For efficiently exploring the parameter space of the logical test case, a novel parameter variation method is applied. The combination of a dedicated test case format, intelligent scenario exploration methods and a state-of-the-art cloud simulation platform results in a highly efficient scenario-based test procedure.<\/jats:p>","DOI":"10.1007\/s10515-024-00481-6","type":"journal-article","created":{"date-parts":[[2024,12,16]],"date-time":"2024-12-16T18:29:05Z","timestamp":1734373745000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Machine-readable specification and intelligent cloud-based execution of logical test cases for automated driving functions"],"prefix":"10.1007","volume":"32","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4420-0357","authenticated-orcid":false,"given":"Max-Arno","family":"Meyer","sequence":"first","affiliation":[]},{"given":"Mehdi","family":"Zouari","sequence":"additional","affiliation":[]},{"given":"Sebastian","family":"Bannenberg","sequence":"additional","affiliation":[]},{"given":"Markus","family":"Deppe","sequence":"additional","affiliation":[]},{"given":"S\u00e9bastien","family":"Christiaens","sequence":"additional","affiliation":[]},{"given":"Sung-Yong","family":"Lee","sequence":"additional","affiliation":[]},{"given":"Jakob","family":"Andert","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,12,16]]},"reference":[{"key":"481_CR1","unstructured":"ASAM e.V.: ASAM AE expression\u2014general expression syntax. ASAM GES V1.0.1 (2017)"},{"key":"481_CR2","unstructured":"ASAM e.V.: Automotive test exchange format. ASAM ATX V1.0.1 (2018)"},{"key":"481_CR3","unstructured":"ASAM e.V.: Evolving landscapes of collaborative testing for ADAS & AD\u2014ASAM test specification study group report 2022. https:\/\/www.asam.net\/standards\/asam-test-specification (2022a). Accessed 16 July 2024"},{"key":"481_CR4","unstructured":"ASAM e.V.: Generic simulator interface. ASAM XIL V2.2.0 (2022b)"},{"key":"481_CR5","unstructured":"ASAM e.V.: Static road network description. ASAM OpenDRIVE V1.8.0 (2023a)"},{"key":"481_CR6","unstructured":"ASAM e.V.: Open test sequence eXchange format. ASAM OTX Extensions V3.2.0 (2023b)"},{"key":"481_CR7","unstructured":"ASAM e.V.: ASAM OpenTestSpecification\u2014concept paper. https:\/\/www.asam.net\/standards\/asam-test-specification (2024a). Accessed 16 July 2024"},{"key":"481_CR8","unstructured":"ASAM e.V.: Dynamic scenario description. ASAM OpenSCENARIO XML V1.3.0 (2024b)"},{"key":"481_CR9","doi-asserted-by":"publisher","unstructured":"Baumann, D., Pfeffer, R., Sax, E.: Automatic generation of critical test cases for the development of highly automated driving functions. In: IEEE 93rd Vehicular Technology Conference (VTC2021-Spring), Helsinki, Finland, pp. 1\u20135 (2021). https:\/\/doi.org\/10.1109\/VTC2021-Spring51267.2021.9448686","DOI":"10.1109\/VTC2021-Spring51267.2021.9448686"},{"issue":"1","key":"481_CR10","doi-asserted-by":"publisher","DOI":"10.1088\/1742-6596\/910\/1\/012016","volume":"910","author":"X Chen","year":"2017","unstructured":"Chen, X., Ji, Z., Fan, Y., Zhan, Y.: Restful API architecture based on Laravel framework. J. Phys. Conf. Ser. 910(1), 012016 (2017). https:\/\/doi.org\/10.1088\/1742-6596\/910\/1\/012016","journal-title":"J. Phys. Conf. Ser."},{"key":"481_CR11","volume-title":"Grundlagen der Wahrscheinlichkeitsrechnung und Statistik","author":"E Cramer","year":"2008","unstructured":"Cramer, E., Kamps, U.: Grundlagen der Wahrscheinlichkeitsrechnung und Statistik. Springer, Berlin, Heidelberg (2008)"},{"issue":"10","key":"481_CR12","doi-asserted-by":"publisher","first-page":"18794","DOI":"10.1109\/TITS.2022.3154774","volume":"23","author":"E de Gelder","year":"2022","unstructured":"de Gelder, E., Hof, J., Cator, E., Paardekooper, J.-P., Op den Camp, O., Ploeg, J., et al.: Scenario parameter generation method and scenario representativeness metric for scenario-based assessment of automated vehicles. IEEE T-ITS 23(10), 18794\u201318807 (2022). https:\/\/doi.org\/10.1109\/TITS.2022.3154774","journal-title":"IEEE T-ITS"},{"key":"481_CR100","doi-asserted-by":"publisher","first-page":"1031","DOI":"10.1007\/s10817-018-09509-5","volume":"63","author":"T Dreossi","year":"2019","unstructured":"Dreossi, T., Donz\u00e9, A. & Seshia, S.A.: Compositional falsification of cyber-physical systems with machine learning components. J Autom Reasoning 63, 1031\u20131053 (2019). https:\/\/doi.org\/10.1007\/s10817-018-09509-5","journal-title":"J Autom Reasoning"},{"key":"481_CR13","unstructured":"D\u00f6rr, D., Hordys, G., Kr\u00fcgel, K., Tsuzuku K.: Efficient validation of AD\/ADAS functions through combination of test methods. In: 2023 JSAE annual congress (Spring), Yokohama, Japan (2023)"},{"key":"481_CR14","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1007\/BF01386213","volume":"2","author":"JH Halton","year":"1960","unstructured":"Halton, J.H.: On the efficiency of certain quasi-random sequences of points in evaluating multi-dimensional integrals. Numer. Math. 2, 84\u201390 (1960). https:\/\/doi.org\/10.1007\/BF01386213","journal-title":"Numer. Math."},{"key":"481_CR15","doi-asserted-by":"publisher","unstructured":"Hue, C.T.M., Hanh, D.D., Binh, N.N.: A transformation-based method for test case automatic generation from use cases. In: 10th International Conference on Knowledge and Systems Engineering (KSE), Ho Chi Minh City, Vietnam, pp. 252\u2013257 (2018). https:\/\/doi.org\/10.1109\/KSE.2018.8573372","DOI":"10.1109\/KSE.2018.8573372"},{"key":"481_CR16","unstructured":"ISO: Road vehicles\u2014functional safety. ISO 26262 (2018)"},{"key":"481_CR17","unstructured":"ISO: Road vehicles\u2014safety of the intended functionality. ISO 21448 (2022)"},{"key":"481_CR18","unstructured":"ISO\/IEC: Systems and software engineering\u2014systems and software quality requirements and evaluation (SQuaRE). ISO\/IEC 25010 (2011)"},{"key":"481_CR19","unstructured":"ISO\/IEC\/IEEE: Software and systems engineering\u2014software testing\u2014part 4: test techniques. ISO\/IEC\/IEEE 29119-4 (2021)"},{"key":"481_CR20","unstructured":"ISO\/IEC\/IEEE: Software and systems engineering \u2013 Software testing \u2013 Part 1: General concepts. ISO\/IEC\/IEEE 29119\u20131 (2022)"},{"issue":"1","key":"481_CR21","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s00158-001-0160-4","volume":"23","author":"R Jin","year":"2001","unstructured":"Jin, R., Chen, W., Simpson, T.W.: Comparative studies of metamodeling techniques under multiple modelling criteria. Struct. Multidiscip. Optim. 23(1), 1\u201313 (2001)","journal-title":"Struct. Multidiscip. Optim."},{"key":"481_CR22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.5381\/jot.2021.20.3.a5","volume":"5","author":"K Juhnke","year":"2021","unstructured":"Juhnke, K., Nikic, A., Tichy, M.: Clustering natural language test case instructions as input for deriving automotive testing DSLs. J. Object Technol. 5, 1\u201314 (2021). https:\/\/doi.org\/10.5381\/jot.2021.20.3.a5","journal-title":"J. Object Technol."},{"key":"481_CR23","doi-asserted-by":"publisher","unstructured":"Kibalama, D., Tulpule, P., Chen, B.: AV\/ADAS safety-critical testing scenario generation from vehicle crash data. In: SAE Technical Paper 2022-01-0104 (2022). https:\/\/doi.org\/10.4271\/2022-01-0104","DOI":"10.4271\/2022-01-0104"},{"key":"481_CR24","unstructured":"Kugler, C., Granrath, C., Pischinger, F., Malvankar, A., D\u00e4sch, C., Meyer, M.-A., Andert, J.: Ausleitung von Testf\u00e4llen f\u00fcr automatisierte Fahrfunktionen auf Basis semiformaler Systemspezifikation. Tag des Systems Engineering 2022, Paderborn, Germany, pp. 65\u201369, (2022)"},{"issue":"12","key":"481_CR25","doi-asserted-by":"publisher","first-page":"342","DOI":"10.3390\/machines9120342","volume":"9","author":"J Ma","year":"2021","unstructured":"Ma, J., Che, X., Li, Y.: Lai EM-K: traffic scenarios for automated vehicle testing: a review of description languages and systems. Machines 9(12), 342 (2021). https:\/\/doi.org\/10.3390\/machines9120342","journal-title":"Machines"},{"key":"481_CR26","unstructured":"Mankins, J.C.: Technology readiness levels. White paper (1995)"},{"issue":"2","key":"481_CR27","first-page":"239","volume":"21","author":"MD McKay","year":"1979","unstructured":"McKay, M.D., Beckman, R.J., Conover, W.J.: A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21(2), 239\u2013245 (1979)","journal-title":"Technometrics"},{"key":"481_CR28","doi-asserted-by":"publisher","first-page":"373","DOI":"10.1007\/s42154-021-00161-1","volume":"4","author":"M-A Meyer","year":"2021","unstructured":"Meyer, M.-A., Sauter, L., Granrath, C., Hadj-Amor, H., Andert, J.: Simulator coupled with distributed co-simulation protocol for automated driving tests. Automot. Innov. 4, 373\u2013389 (2021). https:\/\/doi.org\/10.1007\/s42154-021-00161-1","journal-title":"Automot. Innov."},{"key":"481_CR29","doi-asserted-by":"publisher","unstructured":"Meyer, M.-A., Silberg, S., Granrath, C., Kugler, C., Wachtmeister, L., Rumpe, B. et al.: Scenario- and model-based systems engineering procedure for the SOTIF-compliant design of automated driving functions. In: 2022 IEEE Intelligent Vehicles Symposium (IV), Aachen, Germany, pp. 1599\u20131604 (2022a). https:\/\/doi.org\/10.1109\/IV51971.2022.9827151","DOI":"10.1109\/IV51971.2022.9827151"},{"key":"481_CR30","unstructured":"Meyer, M.-A., Christiaens, S., Phadke, A.: Datenstruktur zum Testen autonom fahrender Kraftfahrzeuge. German patent Apl. No. DE102022103909A1 (2022b)"},{"key":"481_CR31","doi-asserted-by":"publisher","unstructured":"Miettinen, K., Hakanen, J., Podkopaev, D.: interactive nonlinear multiobjective optimization methods. In: Greco, S., Ehrgott, M., Figueira, J. (eds.) Multiple Criteria Decision Analysis. International series in operations research & management science, vol. 233. Springer, New York (2016). https:\/\/doi.org\/10.1007\/978-1-4939-3094-4_22","DOI":"10.1007\/978-1-4939-3094-4_22"},{"key":"481_CR32","unstructured":"Otte, D., Michael, J., Carl, H.: Scientific approach and methodology of a new in-depth investigation study in Germany called GIDAS. In: International Technical Conference on the Enhanced Safety of Vehicles, Eindhoven, Netherlands (2003)"},{"key":"481_CR33","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-81455-7_23","volume-title":"German Success Stories in Industrial Mathematics. Mathematics in Industry","author":"S Peitz","year":"2021","unstructured":"Peitz, S., Dellnitz, M., Bannenberg, S.: Efficient virtual design and testing of autonomous vehicles. In: Bock, H.G., K\u00fcfer, K.H., Maass, P., Milde, A., Schulz, V. (eds.) German Success Stories in Industrial Mathematics. Mathematics in Industry, vol. 35. Springer, Cham (2021). https:\/\/doi.org\/10.1007\/978-3-030-81455-7_23"},{"key":"481_CR34","doi-asserted-by":"publisher","unstructured":"Peperhowe, M., Bannenberg, S., Rasche, R.: Smart testing of intelligent systems. In: VDI Wissensforum GmbH (eds) SIMVEC\u2013Simulation und Erprobung in der Fahrzeugentwicklung. VDI Verlag, D\u00fcsseldorf, Germany (2018). https:\/\/doi.org\/10.51202\/9783181023334","DOI":"10.51202\/9783181023334"},{"key":"481_CR35","unstructured":"Petzold, M., Unger, T., Liers, H.: Transfer of reconstructed real-world accident data into scenario catalogues for the development and test of ADAS and ADS. In: 27th International Technical Conference on the Enhanced Safety of Vehicles (ESV), Yokohama, Japan (2023)"},{"key":"481_CR36","doi-asserted-by":"publisher","first-page":"16","DOI":"10.1007\/s35658-023-1530-0","volume":"18","author":"R Pfeffer","year":"2023","unstructured":"Pfeffer, R., Herrmann, M., Kemper, H.: Effiziente Fehlerratenbestimmung durch KI-basierte Simulation kritischer Szenarien. ATZ Elektron 18, 16\u201321 (2023). https:\/\/doi.org\/10.1007\/s35658-023-1530-0","journal-title":"ATZ Elektron"},{"key":"481_CR37","doi-asserted-by":"publisher","unstructured":"Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D. et al.: Global sensitivity analysis. The Primer. John Wiley and Sons (2008). https:\/\/doi.org\/10.1002\/9780470725184","DOI":"10.1002\/9780470725184"},{"key":"481_CR38","doi-asserted-by":"publisher","unstructured":"Sippl, C., Bock, F., Lauer, C., Heinz, A., Neumayer, T., German, R.: Scenario-based systems engineering: an approach towards automated driving function development. In: 2019 IEEE International Systems Conference (SysCon), Orlando, FL, USA, 2019, pp. 1-8. https:\/\/doi.org\/10.1109\/SYSCON.2019.8836763","DOI":"10.1109\/SYSCON.2019.8836763"},{"issue":"5","key":"481_CR39","doi-asserted-by":"publisher","first-page":"124","DOI":"10.1145\/3579642","volume":"32","author":"S Tang","year":"2023","unstructured":"Tang, S., Zhang, Z., Zhang, Y., Zhou, J., Guo, Y., Liu, S., et al.: A survey on automated driving system testing: landscapes and trends. ACM Trans. Softw. Eng. Methodol. 32(5), 124 (2023). https:\/\/doi.org\/10.1145\/3579642","journal-title":"ACM Trans. Softw. Eng. Methodol."},{"issue":"2","key":"481_CR40","doi-asserted-by":"publisher","first-page":"265","DOI":"10.1109\/TIV.2019.2955903","volume":"5","author":"CE Tuncali","year":"2020","unstructured":"Tuncali, C.E., Fainekos, G., Prokhorov, D., Ito, H., Kapinski, J.: Requirements-driven test generation for autonomous vehicles with machine learning components. IEEE Trans Intell Veh 5(2), 265\u2013280 (2020). https:\/\/doi.org\/10.1109\/TIV.2019.2955903","journal-title":"IEEE Trans Intell Veh"},{"key":"481_CR41","unstructured":"W3C XML Schema Working Group: W3C XML schema definition language (XSD), XSD 1.1, (2011)"},{"key":"481_CR42","unstructured":"Wood, M., Wittmann, D., Liu, S., Knobel, C., Syguda, S., Wiltschko, T., et al.: Safety first for automated driving. White paper. https:\/\/www.connectedautomateddriving.eu\/wp-content\/uploads\/2019\/09\/Safety_First_for_Automated_Driving.pdf (2019). Accessed 16 July 2024"},{"key":"481_CR43","doi-asserted-by":"publisher","unstructured":"Zarazua, P.R., Alrousan, Q., Alzu\u2019bi, H., Tasky, T.: Operational design domain feature optimization route planning tool for automated vehicle open road testing. In: SAE Technical Paper 2023-01-0686 (2023). https:\/\/doi.org\/10.4271\/2023-01-0686","DOI":"10.4271\/2023-01-0686"},{"issue":"2","key":"481_CR44","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1145\/3460082","volume":"5","author":"Q Zhang","year":"2021","unstructured":"Zhang, Q., Ke Hong, D., Zhang, Z., Chen, Q.A., Mahlke, S., Mao, Z.M.: A systematic framework to identify violations of scenario-dependent driving rules in autonomous vehicle software. Proc. ACM Meas. Anal. Comput. Syst. 5(2), 15 (2021). https:\/\/doi.org\/10.1145\/3460082","journal-title":"Proc. ACM Meas. Anal. Comput. Syst."},{"issue":"6","key":"481_CR45","doi-asserted-by":"publisher","first-page":"3391","DOI":"10.1109\/TSE.2023.3254142","volume":"49","author":"Y Zhou","year":"2023","unstructured":"Zhou, Y., Sun, Y., Tang, Y., Chen, Y., Sun, J., Poskitt, C.M., et al.: Specification-based autonomous driving system testing. IEEE Trans. Software Eng. 49(6), 3391\u20133410 (2023). https:\/\/doi.org\/10.1109\/TSE.2023.3254142","journal-title":"IEEE Trans. Software Eng."}],"container-title":["Automated Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10515-024-00481-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10515-024-00481-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10515-024-00481-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,4,6]],"date-time":"2025-04-06T01:38:07Z","timestamp":1743903487000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10515-024-00481-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,16]]},"references-count":46,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,5]]}},"alternative-id":["481"],"URL":"https:\/\/doi.org\/10.1007\/s10515-024-00481-6","relation":{},"ISSN":["0928-8910","1573-7535"],"issn-type":[{"type":"print","value":"0928-8910"},{"type":"electronic","value":"1573-7535"}],"subject":[],"published":{"date-parts":[[2024,12,16]]},"assertion":[{"value":"31 July 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 December 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 December 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}],"article-number":"10"}}