{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,6]],"date-time":"2026-05-06T16:00:55Z","timestamp":1778083255212,"version":"3.51.4"},"reference-count":20,"publisher":"Springer Science and Business Media LLC","issue":"6","license":[{"start":{"date-parts":[[2024,11,1]],"date-time":"2024-11-01T00:00:00Z","timestamp":1730419200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,11,6]],"date-time":"2024-11-06T00:00:00Z","timestamp":1730851200000},"content-version":"vor","delay-in-days":5,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100008332","name":"Graz University of Technology","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100008332","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Elektrotech. Inftech."],"published-print":{"date-parts":[[2024,11]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Automated and autonomous driving systems are increasingly integrated into modern vehicles to support the vision of safe, efficient, and comfortable transportation. Given the complexity of these systems, thorough testing and close monitoring of their behavior is inevitable to prevent hazardous situations and unexpected behaviors. This paper presents different approaches for testing ADAS\/ADS, focusing on generating critical scenarios for virtual validation and monitoring approaches applied during operation. Specifically, we provide a\u00a0comprehensive overview of our previous work on combinatorial and search-based testing methodologies, highlighting their application in generating robust test suites. Additionally, we summarize our work on intelligent monitoring approaches to detect operational issues. Our findings emphasize the necessity of advanced testing solutions and continuous monitoring to identify and mitigate potential failures, demonstrating their applicability in enhancing the safety and trustworthiness of ADAS\/AD.<\/jats:p>","DOI":"10.1007\/s00502-024-01242-9","type":"journal-article","created":{"date-parts":[[2024,11,6]],"date-time":"2024-11-06T07:06:10Z","timestamp":1730876770000},"page":"392-399","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Testing ADAS\/ADS \u2013 from critical scenarios to automated testing oracles","Testen von ADAS\/ADS \u2013 von kritischen Szenarien zum automatisierten Testorakel"],"prefix":"10.1007","volume":"141","author":[{"given":"Lorenz","family":"Klampfl","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Florian","family":"Kl\u00fcck","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mihai","family":"Nica","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jianbo","family":"Tao","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0462-2283","authenticated-orcid":false,"given":"Franz","family":"Wotawa","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,11,6]]},"reference":[{"key":"1242_CR1","doi-asserted-by":"publisher","unstructured":"Koopman, P., Wagner, M.: Challenges in autonomous vehicle testing and validation. SAE Int. J. Trans. Safety 4, 15\u201324 (2016) https:\/\/doi.org\/10.4271\/2016-01-0128","DOI":"10.4271\/2016-01-0128"},{"key":"1242_CR2","doi-asserted-by":"publisher","unstructured":"Wotawa, F.: In: Watzenig, D., Horn, M. (eds.) Testing Autonomous and Highly Configurable Systems: Challenges and Feasible Solutions, pp.\u00a0519\u2013532. Springer, Cham (2017). https:\/\/doi.org\/10.1007\/978-3-319-31895-0_22","DOI":"10.1007\/978-3-319-31895-0_22"},{"key":"1242_CR3","unstructured":"Wotawa, F.: On the use of available testing methods for verification  validation of ai-based software and systems. In: SafeAI@AAAI. CEUR Workshop Proceedings, vol. 2808 (2021)"},{"key":"1242_CR4","doi-asserted-by":"publisher","unstructured":"Kalra, N., Paddock, S.M.: Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability? Transportation Research Part A: Policy and Practice 94, 182\u2013193 (2016) https:\/\/doi.org\/10.1016\/j.tra.2016.09.010","DOI":"10.1016\/j.tra.2016.09.010"},{"key":"1242_CR5","doi-asserted-by":"publisher","unstructured":"Zhang, X., Tao, J., Tan, K., T\u00f6rngren, M., S\u00e1nchez, J.M.G., Ramli, M.R., Tao, X., Gyllenhammar, M., Wotawa, F., Mohan, N., Nica, M., Felbinger, H.: Finding critical scenarios for automated driving systems: A systematic mapping study. IEEE Trans. Software Eng. 49(3), 991\u20131026 (2023) https:\/\/doi.org\/10.1109\/TSE.2022.3170122","DOI":"10.1109\/TSE.2022.3170122"},{"key":"1242_CR6","doi-asserted-by":"crossref","unstructured":"Kuhn, D.R., Bryce, R., Duan, F., Ghandehari, L.S., Lei, Y., Kacker, R.N.: Combinatorial testing: Theory and practice. In: Advances in Computers vol. 99, pp.\u00a01\u201366 (2015)","DOI":"10.1016\/bs.adcom.2015.05.003"},{"key":"1242_CR7","doi-asserted-by":"publisher","unstructured":"McMinn, P.: Search-based software test data generation: a\u00a0survey. Software Testing, Verification and Reliability 14(2), 105\u2013156 (2004) https:\/\/doi.org\/10.1002\/stvr.294https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/stvr.294","DOI":"10.1002\/stvr.294"},{"key":"1242_CR8","doi-asserted-by":"publisher","unstructured":"Harman, M., Jones, B.F.: Search-based software engineering. Information and Software Technology 43(14), 833\u2013839 (2001) https:\/\/doi.org\/10.1016\/S0950-5849(01)00189-6","DOI":"10.1016\/S0950-5849(01)00189-6"},{"key":"1242_CR9","doi-asserted-by":"publisher","unstructured":"Wotawa, F., Peischl, B., Kl\u00fcck, F., Nica, M.: Quality assurance methodologies for automated driving. Elektrotech. Informationstechnik 135(4-5), 322\u2013327 (2018) https:\/\/doi.org\/10.1007\/S00502-018-0630-7","DOI":"10.1007\/S00502-018-0630-7"},{"key":"1242_CR10","doi-asserted-by":"publisher","unstructured":"Li, Y., Tao, J., Wotawa, F.: Ontology-based test generation for automated and autonomous driving functions. Information and Software Technology 117(October 2019) (2020) https:\/\/doi.org\/10.1016\/j.infsof.2019.106200","DOI":"10.1016\/j.infsof.2019.106200"},{"key":"1242_CR11","doi-asserted-by":"crossref","unstructured":"Kuhn, D.R., Kacker, R.N., Lei, Y., Hunter, J.: Combinatorial software testing. Computer, 94\u201396 (2009)","DOI":"10.1109\/MC.2009.253"},{"key":"1242_CR12","doi-asserted-by":"publisher","unstructured":"Kl\u00fcck, F., Zimmermann, M., Wotawa, F., Nica, M.: Genetic algorithm-based test parameter optimization for adas system testing. In: 2019 IEEE 19th International Conference on Software Quality, Reliability and Security (QRS), pp.\u00a0418\u2013425 (2019). https:\/\/doi.org\/10.1109\/QRS.2019.00058","DOI":"10.1109\/QRS.2019.00058"},{"key":"1242_CR13","doi-asserted-by":"crossref","unstructured":"Kl\u00fcck, F., Zimmermann, M., Wotawa, F., Nica, M.: Performance comparison of two search-based testing strategies for adas system validation. In: Gaston, C., Kosmatov, N., Le\u00a0Gall, P. (eds.) Testing Software and Systems, pp.\u00a0140\u2013156. Springer, Cham (2019)","DOI":"10.1007\/978-3-030-31280-0_9"},{"key":"1242_CR14","doi-asserted-by":"publisher","unstructured":"Klampfl, L., Kl\u00fcck, F., Wotawa, F.: Using genetic algorithms for automating automated lane-keeping system testing. J. Softw. Evol. Process. 36(3) (2024) https:\/\/doi.org\/10.1002\/SMR.2520","DOI":"10.1002\/SMR.2520"},{"key":"1242_CR15","doi-asserted-by":"publisher","unstructured":"Kl\u00fcck, F., Li, Y., Tao, J., Wotawa, F.: An empirical comparison of combinatorial testing and search-based testing in the context of automated and autonomous driving systems. Inf. Softw. Technol. 160, 107225 (2023) https:\/\/doi.org\/10.1016\/J.INFSOF.2023.107225","DOI":"10.1016\/J.INFSOF.2023.107225"},{"key":"1242_CR16","doi-asserted-by":"crossref","unstructured":"Wotawa, F., Lewitschnig, H.: Monitoring hierarchical systems for safety assurance. In: IDC. Studies in Computational Intelligence, vol. 1026, pp.\u00a0331\u2013340. Springer, Cham (2021)","DOI":"10.1007\/978-3-030-96627-0_30"},{"key":"1242_CR17","doi-asserted-by":"publisher","unstructured":"Stettinger, G., Wotawa, F.: Smart monitoring for safety-assurance in autonomous driving (S). In: Chang, S. (ed.) The 35th International Conference on Software Engineering and Knowledge Engineering, SEKE 2023, KSIR Virtual Conference Center, USA, July 1\u201110, 2023, pp.\u00a0542\u2013547. KSI Research Inc., USA (2023). https:\/\/doi.org\/10.18293\/SEKE2023-144","DOI":"10.18293\/SEKE2023-144"},{"key":"1242_CR18","doi-asserted-by":"publisher","unstructured":"Wotawa, F., Klampfl, L.: Explaining object motion using answer set programming. In: Helic, D., Leitner, G., Stettinger, M., Felfernig, A., Ras, Z.W. (eds.) Foundations of Intelligent Systems \u2013 25th International Symposium, ISMIS 2020, Graz, Austria, September 23-25, 2020, Proceedings. Lecture Notes in Computer Science, vol. 12117, pp.\u00a0298\u2013307. Springer, Cham (2020). https:\/\/doi.org\/10.1007\/978-3-030-59491-6_28","DOI":"10.1007\/978-3-030-59491-6_28"},{"key":"1242_CR19","unstructured":"Klampfl, L., Wotawa, F.: On the Use of Qualitative Reasoning in Autonomous Driving. (2022). 35th International Workshop on Qualitative Reasoning ; Conference date: 23-07-2022 Through 23-07-2022. https:\/\/www.esade.edu\/faculty-research\/en\/institute-for-data-driven-decisions\/research\/juice-research-group\/35th-international-workshop-on-qualitative-reasoning"},{"key":"1242_CR20","doi-asserted-by":"publisher","unstructured":"Klampfl, L., Wotawa, F.: Identifying critical scenarios in autonomous driving during operation. In: Nowaczyk, S., Biecek, P., Chung, N.C., Vallati, M., Skruch, P., Jaworek-Korjakowska, J., Parkinson, S., Nikitas, A., Atzm\u00fcller, M., Kliegr, T., Schmid, U., Bobek, S., Lavrac, N., Peeters, M., Dierendonck, R., Robben, S., Mercier-Laurent, E., Kayakutlu, G., Owoc, M.L., Mason, K., Wahid, A., Bruno, P., Calimeri, F., Cauteruccio, F., Terracina, G., Wolter, D., Leidner, J.L., Kohlhase, M., Dimitrova, V. (eds.) Artificial Intelligence. ECAI 2023 International Workshops \u2013 $$\\text{XAI}^{3}$$, TACTIFUL, XI-ML, SEDAMI, RAAIT, AI4S, HYDRA, AI4AI, Krak\u00f3w, Poland, September 30\u2013October 4, 2023, Proceedings, Part I. Communications in Computer and Information Science, vol. 1947, pp.\u00a0156\u2013172. Springer, Cham (2023). https:\/\/doi.org\/10.1007\/978-3-031-50396-2_9","DOI":"10.1007\/978-3-031-50396-2_9"}],"container-title":["e+i Elektrotechnik und Informationstechnik"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00502-024-01242-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00502-024-01242-9\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00502-024-01242-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,14]],"date-time":"2024-11-14T13:07:21Z","timestamp":1731589641000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00502-024-01242-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11]]},"references-count":20,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2024,11]]}},"alternative-id":["1242"],"URL":"https:\/\/doi.org\/10.1007\/s00502-024-01242-9","relation":{},"ISSN":["0932-383X","1613-7620"],"issn-type":[{"value":"0932-383X","type":"print"},{"value":"1613-7620","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11]]},"assertion":[{"value":"21 June 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 September 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 November 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Funding: Partial financial support was received from the Austrian Federal Ministry for Digital and Economic Affairs, the National Foundation for Research, Technology and Development, Austria and the Christian Doppler Research Association.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interests"}},{"value":"Competing interests: There are no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interests"}},{"value":"Financial interests: The authors declare they have no financial interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interests"}},{"value":"Non-financial interests: none.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interests"}}]}}