{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,3]],"date-time":"2026-07-03T16:40:48Z","timestamp":1783096848458,"version":"3.54.6"},"reference-count":93,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2024,5,17]],"date-time":"2024-05-17T00:00:00Z","timestamp":1715904000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,5,17]],"date-time":"2024-05-17T00:00:00Z","timestamp":1715904000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100010663","name":"H2020 European Research Council","doi-asserted-by":"publisher","award":["787703"],"award-info":[{"award-number":["787703"]}],"id":[{"id":"10.13039\/100010663","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2024,7]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Simulation-based testing represents an important step to ensure the reliability of autonomous driving software. In practice, when companies rely on third-party general-purpose simulators, either for in-house or outsourced testing, the generalizability of testing results to real autonomous vehicles is at stake. In this paper, we enhance simulation-based testing by introducing the notion of<jats:italic>digital siblings<\/jats:italic>\u2014a multi-simulator approach that tests a given autonomous vehicle on multiple general-purpose simulators built with different technologies, that operate collectively as an ensemble in the testing process. We exemplify our approach on a case study focused on testing the lane-keeping component of an autonomous vehicle. We use two open-source simulators as digital siblings, and we empirically compare such a multi-simulator approach against a digital twin of a physical scaled autonomous vehicle on a large set of test cases. Our approach requires generating and running test cases for each individual simulator, in the form of sequences of road points. Then, test cases are migrated between simulators, using feature maps to characterize the exercised driving conditions. Finally, the joint predicted failure probability is computed, and a failure is reported only in cases of agreement among the siblings. Our empirical evaluation shows that the ensemble failure predictor by the digital siblings is superior to each individual simulator at predicting the failures of the digital twin. We discuss the findings of our case study and detail how our approach can help researchers interested in automated testing of autonomous driving software.<\/jats:p>","DOI":"10.1007\/s10664-024-10458-4","type":"journal-article","created":{"date-parts":[[2024,5,17]],"date-time":"2024-05-17T09:11:06Z","timestamp":1715937066000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":19,"title":["Two is better than one: digital siblings to improve autonomous driving testing"],"prefix":"10.1007","volume":"29","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7825-3409","authenticated-orcid":false,"given":"Matteo","family":"Biagiola","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrea","family":"Stocco","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Vincenzo","family":"Riccio","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Paolo","family":"Tonella","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,5,17]]},"reference":[{"key":"10458_CR1","doi-asserted-by":"crossref","unstructured":"Afzal A, Katz DS, Le\u00a0Goues C, Timperley CS (2021) Simulation for robotics test automation: Developer perspectives. In: 2021 14th IEEE conference on software testing, verification and validation (ICST). IEEE, pp 263\u2013274","DOI":"10.1109\/ICST49551.2021.00036"},{"issue":"1","key":"10458_CR2","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1109\/MCOMSTD.011.2100004","volume":"5","author":"S Almeaibed","year":"2021","unstructured":"Almeaibed S, Al-Rubaye S, Tsourdos A, Avdelidis NP (2021) Digital twin analysis to promote safety and security in autonomous vehicles. IEEE Commun Stand Mag 5(1):40\u201346. https:\/\/doi.org\/10.1109\/MCOMSTD.011.2100004","journal-title":"IEEE Commun Stand Mag"},{"key":"10458_CR3","doi-asserted-by":"crossref","unstructured":"Amini MH, Naseri S, Nejati S (2023) Evaluating the impact of flaky simulators on testing autonomous driving systems","DOI":"10.1007\/s10664-023-10433-5"},{"key":"10458_CR4","unstructured":"Arjovsky M, Chintala S, Bottou L (2017) Wasserstein generative adversarial networks. In: International conference on machine learning. PMLR, pp. 214\u2013223"},{"key":"10458_CR5","doi-asserted-by":"crossref","unstructured":"Barosan I, Basmenj AA, Chouhan SGR, Manrique D (2020) Development of a virtual simulation environment and a digital twin of an autonomous driving truck for a distribution center. Software architecture. Springer, Cham, pp 542\u2013557","DOI":"10.1007\/978-3-030-59155-7_39"},{"key":"10458_CR6","volume-title":"A recursive evaluation algorithm for a class of catmull-rom splines","author":"PJ Barry","year":"1988","unstructured":"Barry PJ, Goldman RN (1988) A recursive evaluation algorithm for a class of catmull-rom splines. SIGGRAPH Comput, Graph"},{"key":"10458_CR7","unstructured":"BeamNG.research (2022) BeamNG GmbH. https:\/\/www.beamng.gmbh\/research"},{"key":"10458_CR8","doi-asserted-by":"crossref","unstructured":"Ben Abdessalem R, Nejati SC, Briand L, Stifter T (2018) Testing vision-based control systems using learnable evolutionary algorithms. In: 2018 IEEE\/ACM 40th International Conference on Software Engineering (ICSE)","DOI":"10.1145\/3180155.3180160"},{"key":"10458_CR9","doi-asserted-by":"crossref","unstructured":"Bewley A, Rigley J, Liu Y, Hawke J, Shen R, Lam VD, Kendall A (2019) Learning to drive from simulation without real world labels. In: 2019 International conference on robotics and automation (ICRA). IEEE, pp 4818\u20134824","DOI":"10.1109\/ICRA.2019.8793668"},{"key":"10458_CR10","unstructured":"BGR\u00a0Media L (2018) Waymo\u2019s self-driving cars hit 10 million miles. https:\/\/techcrunch.com\/2018\/10\/10\/waymos-self-driving-cars-hit-10-million-miles"},{"key":"10458_CR11","doi-asserted-by":"crossref","unstructured":"Biagiola M, Klikovits S, Peltomaki J, Riccio V (2023) Sbft tool competition 2023-cyber-physical systems track. In: 16th IEEE\/ACM international workshop on Search-Based And Fuzz Testing, SBFT","DOI":"10.1109\/SBFT59156.2023.00010"},{"key":"10458_CR12","unstructured":"Bojarski M, Del\u00a0Testa D, Dworakowski D, Firner B, Flepp B, Goyal P, Jackel LD, Monfort M, Muller U, Zhang J, Zhang X, Zhao J, Zieba K (2016) End to end learning for self-driving cars. CoRR abs\/1604.07316"},{"key":"10458_CR13","doi-asserted-by":"crossref","unstructured":"Borg M, Abdessalem RB, Nejati S, Jegeden FX, Shin D (2021) Digital twins are not monozygotic\u2013cross-replicating adas testing in two industry-grade automotive simulators. In: ICST \u201921. IEEE","DOI":"10.1109\/ICST49551.2021.00050"},{"key":"10458_CR14","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1016\/j.cviu.2018.10.009","volume":"179","author":"A Borji","year":"2019","unstructured":"Borji A (2019) Pros and cons of gan evaluation measures. Comput Vision Image Understand 179:41\u201365","journal-title":"Comput Vision Image Understand"},{"key":"10458_CR15","unstructured":"Bottou L, Bousquet O (2007) The tradeoffs of large scale learning. In: Proceedings of NIPS \u201907"},{"key":"10458_CR16","unstructured":"Boutan E (2020) Autonomous driving market overview. https:\/\/medium.com\/swlh\/autonomous-driving-market-overview-b8c71d81c072"},{"key":"10458_CR17","doi-asserted-by":"crossref","unstructured":"Cerf VG (2018) A comprehensive self-driving car test. Commun ACM 61(2)","DOI":"10.1145\/3177753"},{"key":"10458_CR18","doi-asserted-by":"publisher","unstructured":"Cheng M, Zhou Y, Xie X (2023) Behavexplor: Behavior diversity guided testing for autonomous driving systems. In: Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis, pp 488\u2013500. Association for Computing Machinery. https:\/\/doi.org\/10.1145\/3597926.3598072","DOI":"10.1145\/3597926.3598072"},{"key":"10458_CR19","unstructured":"Conover WJ (1999) Practical nonparametric statistics, vol. 350. John Wiley & Sons"},{"key":"10458_CR20","unstructured":"DeepHyperion (2022) Replication package. https:\/\/github.com\/testingautomated-usi\/DeepHyperion"},{"key":"10458_CR21","unstructured":"U.S. Department\u00a0of Transportation NHTSA (2007) Pre-crash scenario typology for crash avoidance research"},{"key":"10458_CR22","unstructured":"U.S. Department\u00a0of\u00a0Transportation UD (2018) A framework for automated driving system testable cases and scenarios. https:\/\/rosap.ntl.bts.gov\/view\/dot\/38824\/dot_38824_DS1.pdf"},{"key":"10458_CR23","unstructured":"Donkey Car (2021). https:\/\/www.donkeycar.com\/"},{"issue":"2","key":"10458_CR24","first-page":"356","volume":"18","author":"W Farag","year":"2020","unstructured":"Farag W (2020) Complex trajectory tracking using pid control for autonomous driving. Int J Intell Transp Syst Res 18(2):356\u2013366","journal-title":"Int J Intell Transp Syst Res"},{"issue":"2","key":"10458_CR25","doi-asserted-by":"publisher","first-page":"276","DOI":"10.1109\/TSE.2012.14","volume":"39","author":"G Fraser","year":"2012","unstructured":"Fraser G, Arcuri A (2012) Whole test suite generation. IEEE Trans Softw Eng 39(2):276\u2013291","journal-title":"IEEE Trans Softw Eng"},{"key":"10458_CR26","doi-asserted-by":"crossref","unstructured":"Gambi A, Jahangirova G, Riccio V, Zampetti F (2022) SBST tool competition 2022. In: 2022 IEEE\/ACM 15th international workshop on Search-Based Software Testing (SBST). IEEE, pp 25\u201332","DOI":"10.1145\/3526072.3527538"},{"key":"10458_CR27","unstructured":"Gambi A, Maul P, Mueller M, Stamatogiannakis L, Fischer T, Panichella S (2019) Soft-body simulation and procedural generation for the development and testing of cyber-physical systems. Tech. rep, BeamNG"},{"key":"10458_CR28","doi-asserted-by":"crossref","unstructured":"Gambi A, Mueller M, Fraser G (2019) Automatically testing self-driving cars with search-based procedural content generation. In: Proceedings of ISSTA \u201919","DOI":"10.1145\/3293882.3330566"},{"key":"10458_CR29","doi-asserted-by":"crossref","unstructured":"Garc\u00eda S, Str\u00fcber D, Brugali D, Berger T, Pelliccione P (2020) Robotics software engineering: A perspective from the service robotics domain. In: Proceedings of ESEC\/FSE \u201920. pp 593\u2013604","DOI":"10.1145\/3368089.3409743"},{"key":"10458_CR30","doi-asserted-by":"crossref","unstructured":"Grewal R, Tonella P, Stocco A (2024) Predicting safety misbehaviours in autonomous driving systems using uncertainty quantification p 12","DOI":"10.1109\/ICST60714.2024.00016"},{"issue":"3","key":"10458_CR31","doi-asserted-by":"publisher","first-page":"362","DOI":"10.1002\/rob.21918","volume":"37","author":"S Grigorescu","year":"2020","unstructured":"Grigorescu S, Trasnea B, Cocias T, Macesanu G (2020) A survey of deep learning techniques for autonomous driving. J Field Robot 37(3):362\u2013386","journal-title":"J Field Robot"},{"key":"10458_CR32","unstructured":"Group E (2021) Esi prosivic. https:\/\/myesi.esi-group.com\/downloads\/software-downloads\/pro-sivic-2021.0"},{"key":"10458_CR33","doi-asserted-by":"publisher","unstructured":"Haq FU, Shin D, Briand LC (2022) Efficient online testing for dnn-enabled systems using surrogate-assisted and many-objective optimization. In: 44th IEEE\/ACM 44th International Conference on Software Engineering, ICSE 2022. ACM, Pittsburgh, PA, USA, May 25-27, 2022, pp 811\u2013822. https:\/\/doi.org\/10.1145\/3510003.3510188","DOI":"10.1145\/3510003.3510188"},{"key":"10458_CR34","doi-asserted-by":"crossref","unstructured":"Haq FU, Shin D, Nejati S, Briand L (2021) Can offline testing of deep neural networks replace their online testing? Empir Softw Eng","DOI":"10.1007\/s10664-021-09982-4"},{"key":"10458_CR35","unstructured":"Hu X, Li S, Huang T, Tang B, Chen L (2023) Sim2real and digital twins in autonomous driving: A survey"},{"key":"10458_CR36","doi-asserted-by":"crossref","unstructured":"Jahangirova G, Stocco A, Tonella P (2021) Quality metrics and oracles for autonomous vehicles testing. In: Proceedings of 14th IEEE International conference on software testing, verification and validation, ICST \u201921. IEEE","DOI":"10.1109\/ICST49551.2021.00030"},{"key":"10458_CR37","doi-asserted-by":"crossref","unstructured":"Jha S, Banerjee SS, Tsai T, Hari SKS, Sullivan MB, Kalbarczyk ZT, Keckler SW, Iyer RK (2019) Ml-based fault injection for autonomous vehicles: A case for bayesian fault injection. In: 2019 49th annual IEEE\/IFIP international conference on dependable systems and networks (DSN), pp. 112\u2013124. https:\/\/api.semanticscholar.org\/CorpusID:195776612","DOI":"10.1109\/DSN.2019.00025"},{"key":"10458_CR38","unstructured":"Kapteyn MG, Pretorius JVR, Willcox KE (2020) A probabilistic graphical model foundation for enabling predictive digital twins at scale. CoRR abs\/2012.05841"},{"key":"10458_CR39","doi-asserted-by":"crossref","unstructured":"Kaur P, Taghavi S, Tian Z, Shi W (2021) A survey on simulators for testing self-driving cars. CoRR abs\/2101.05337. arXiv:2101.05337","DOI":"10.1109\/MetroCAD51599.2021.00018"},{"key":"10458_CR40","doi-asserted-by":"crossref","unstructured":"Kim S, Liu M, Rhee JJ, Jeon Y, Kwon Y, Kim CH (2022) Drivefuzz. In: Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security. ACM. https:\/\/doi.org\/10.11452F3548606.3560558","DOI":"10.1145\/3548606.3560558"},{"key":"10458_CR41","unstructured":"Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv:1412.6980"},{"key":"10458_CR42","doi-asserted-by":"crossref","unstructured":"Kong Z, Guo J, Li A, Liu C (2020) Physgan: Generating physical-world-resilient adversarial examples for autonomous driving. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition. pp 14,254\u201314,263","DOI":"10.1109\/CVPR42600.2020.01426"},{"key":"10458_CR43","doi-asserted-by":"crossref","unstructured":"Koopman P, Wagner M (2016) Challenges in autonomous vehicle testing and validation. SAE Int J Transp Safety","DOI":"10.4271\/2016-01-0128"},{"key":"10458_CR44","unstructured":"Kothlow C (2021) The power of a multi-purpose digital twin. https:\/\/blogs.sw.siemens.com\/simcenter\/the-power-of-a-multi-purpose-digital-twin\/"},{"key":"10458_CR45","doi-asserted-by":"crossref","unstructured":"Lambertenghi SC, Stocco A (2024) Assessing quality metrics for neural reality gap input mitigation in autonomous driving testing p 12","DOI":"10.1109\/ICST60714.2024.00024"},{"key":"10458_CR46","doi-asserted-by":"publisher","unstructured":"Li G, Li Y, Jha S, Tsai T, Sullivan M, Hari SKS, Kalbarczyk Z, Iyer R (2020) Av-fuzzer: Finding safety violations in autonomous driving systems. In: 2020 IEEE 31st international symposium on software reliability engineering (ISSRE), pp. 25\u201336. https:\/\/doi.org\/10.1109\/ISSRE5003.2020.00012","DOI":"10.1109\/ISSRE5003.2020.00012"},{"key":"10458_CR47","doi-asserted-by":"crossref","unstructured":"Lukasczyk S, Kroi\u00df F, Fraser G (2020) Automated unit test generation for python. In: International symposium on search based software engineering. Springer, pp 9\u201324","DOI":"10.1007\/978-3-030-59762-7_2"},{"key":"10458_CR48","unstructured":"Luke S (2013) Essentials of metaheuristics, vol.\u00a02. Lulu Raleigh"},{"key":"10458_CR49","unstructured":"May C (2019) Why automotive companies outsource software development services. https:\/\/medium.datadriveninvestor.com\/why-automotive-companies-outsource-software-development-services-54a806458b4?gi=9d9b4f45e9ba"},{"key":"10458_CR50","doi-asserted-by":"crossref","unstructured":"Moghadam MH, Borg M, Saadatmand M, Mousavirad SJ, Bohlin M, Lisper B (2022) Machine learning testing in an adas case study using simulation-integrated bio-inspired search-based testing","DOI":"10.1002\/smr.2591"},{"key":"10458_CR51","unstructured":"Mouret JB, Clune J (2015) Illuminating search spaces by mapping elites. arXiv:1504.04909"},{"key":"10458_CR52","unstructured":"Nvidia PhysX (2022) https:\/\/developer.nvidia.com\/physx-sdk"},{"issue":"2","key":"10458_CR53","doi-asserted-by":"publisher","first-page":"122","DOI":"10.1109\/TSE.2017.2663435","volume":"44","author":"A Panichella","year":"2017","unstructured":"Panichella A, Kifetew FM, Tonella P (2017) Automated test case generation as a many-objective optimisation problem with dynamic selection of the targets. IEEE Trans Softw Eng 44(2):122\u2013158","journal-title":"IEEE Trans Softw Eng"},{"key":"10458_CR54","doi-asserted-by":"crossref","unstructured":"Panichella S, Gambi A, Zampetti F, Riccio V (2021) SBST tool competition 2021. In: 2021 IEEE\/ACM 14th international workshop on Search-Based Software Testing (SBST). IEEE, pp 20\u201327","DOI":"10.1109\/SBST52555.2021.00011"},{"issue":"2","key":"10458_CR55","doi-asserted-by":"publisher","first-page":"47","DOI":"10.3390\/e19020047","volume":"19","author":"A Ramdas","year":"2017","unstructured":"Ramdas A, Garc\u00eda Trillos N, Cuturi M (2017) On wasserstein two-sample testing and related families of nonparametric tests. Entropy 19(2):47","journal-title":"Entropy"},{"key":"10458_CR56","unstructured":"Replication package (2023) https:\/\/github.com\/testingautomated-usi\/maxitwo"},{"key":"10458_CR57","doi-asserted-by":"crossref","unstructured":"Riccio V, Tonella P (2020) Model-based exploration of the frontier of behaviours for deep learning system testing. In: Proceedings of ESEC\/FSE","DOI":"10.1145\/3368089.3409730"},{"key":"10458_CR58","doi-asserted-by":"publisher","unstructured":"Rosique F, Navarro PJ, Fern\u00e1ndez C, Padilla A (2019) A systematic review of perception system and simulators for autonomous vehicles research. Sensors 19(3). https:\/\/doi.org\/10.3390\/s19030648","DOI":"10.3390\/s19030648"},{"key":"10458_CR59","unstructured":"Saad D (1998) Online algorithms and stochastic approximations. Online Learn"},{"key":"10458_CR60","doi-asserted-by":"crossref","unstructured":"Saito T, Rehmsmeier M (2015) The precision-recall plot is more informative than the roc plot when evaluating binary classifiers on imbalanced datasets. PloS one 10(3):e0118,432","DOI":"10.1371\/journal.pone.0118432"},{"issue":"5","key":"10458_CR61","doi-asserted-by":"publisher","first-page":"307","DOI":"10.1038\/s43588-021-00077-0","volume":"1","author":"O San","year":"2021","unstructured":"San O (2021) The digital twin revolution. Nat Comput Sci 1(5):307\u2013308","journal-title":"Nat Comput Sci"},{"key":"10458_CR62","unstructured":"Software SDI (2022) Simcenter prescan. https:\/\/www.plm.automation.siemens.com\/global\/en\/products\/simcenter\/prescan.html"},{"key":"10458_CR63","doi-asserted-by":"crossref","unstructured":"Stocco A, Nunes PJ, d\u2019Amorim M, Tonella P (2022) Thirdeye: Attention maps for safe autonomous driving systems. In: Proceedings of 37th IEEE\/ACM international conference on automated software engineering, ASE \u201922. IEEE\/ACM","DOI":"10.1145\/3551349.3556968"},{"key":"10458_CR64","unstructured":"Stocco A, Pulfer B, Tonella P (2022) Mind the gap! A study on the transferability of virtual vs physical-world testing of autonomous driving systems. IEEE Trans Softw Eng. https:\/\/ieeexplore.ieee.org\/document\/9869302"},{"key":"10458_CR65","doi-asserted-by":"crossref","unstructured":"Stocco A, Pulfer B, Tonella P (2023) Model vs system level testing of autonomous driving systems: A replication and extension study. Empir Softw Eng","DOI":"10.1007\/s10664-023-10306-x"},{"key":"10458_CR66","doi-asserted-by":"crossref","unstructured":"Stocco A, Tonella P (2020) Towards anomaly detectors that learn continuously. In: Proceedings of 31st International Symposium on Software Reliability Engineering Workshops, ISSREW 2020. IEEE","DOI":"10.1109\/ISSREW51248.2020.00073"},{"key":"10458_CR67","doi-asserted-by":"publisher","unstructured":"Stocco A, Tonella P (2021) Confidence-driven weighted retraining for predicting safety-critical failures in autonomous driving systems. J Softw: Evol Process. https:\/\/doi.org\/10.1002\/smr.2386","DOI":"10.1002\/smr.2386"},{"key":"10458_CR68","doi-asserted-by":"crossref","unstructured":"Stocco A, Weiss M, Calzana M, Tonella P (2020) Misbehaviour prediction for autonomous driving systems. In: Proceedings of 42nd International Conference on Software Engineering, ICSE \u201920. ACM","DOI":"10.1145\/3377811.3380353"},{"key":"10458_CR69","doi-asserted-by":"crossref","unstructured":"Talwar D, Guruswamy S, Ravipati N, Eirinaki M (2020) Evaluating validity of synthetic data in perception tasks for autonomous vehicles. In: 2020 IEEE international conference on Artificial Intelligence Testing (AITest). IEEE, pp 73\u201380","DOI":"10.1109\/AITEST49225.2020.00018"},{"key":"10458_CR70","doi-asserted-by":"publisher","unstructured":"Tang S, Zhang Z, Zhang Y, Zhou J, Guo Y, Liu S, Guo S, Li Y, Ma L, Xue Y, Liu Y (2022) A survey on automated driving system testing: Landscapes and trends. arXiv:2206.05961, https:\/\/doi.org\/10.48550\/arXiv.2206.05961","DOI":"10.48550\/arXiv.2206.05961"},{"key":"10458_CR71","unstructured":"Tawn\u00a0Kramer ME contributors (2022) Donkeycar. https:\/\/www.donkeycar.com\/"},{"key":"10458_CR72","unstructured":"Team Chauffeur (2016) \u201cSteering angle model: Chauffeur.\u201d. https:\/\/github.com\/udacity\/self-driving-car\/tree\/master\/steering-models\/community-models\/chauffeur"},{"key":"10458_CR73","unstructured":"Team Epoch (2016) \u201cSteering angle model: Epoch.\u201d. https:\/\/github.com\/udacity\/self-driving-car\/tree\/master\/steering-models\/community-models\/cg23"},{"key":"10458_CR74","unstructured":"Team U (2019) Udacity\u2019s self-driving car simulator. https:\/\/github.com\/tsigalko18\/self-driving-car-sim"},{"key":"10458_CR75","unstructured":"Team U (2020) Udacity self-driving car challenge. https:\/\/github.com\/udacity\/self-driving-car\/"},{"key":"10458_CR76","doi-asserted-by":"crossref","unstructured":"Tian Y, Pei K, Jana S, Ray B (2018) Deeptest: Automated testing of deep-neural-network-driven autonomous cars. In: Proceedings of ICSE \u201918. ACM","DOI":"10.1145\/3180155.3180220"},{"key":"10458_CR77","unstructured":"Udacity (2017) A self-driving car simulator built with Unity. https:\/\/github.com\/udacity\/self-driving-car-sim. Online; accessed 18 August 2019"},{"key":"10458_CR78","unstructured":"Unity3d (2021) https:\/\/unity.com"},{"key":"10458_CR79","doi-asserted-by":"crossref","unstructured":"van Dinter R, Tekinerdogan B, Catal C (2022) Predictive maintenance using digital twins: A systematic literature review. Inf Softw Technol","DOI":"10.1016\/j.infsof.2022.107008"},{"key":"10458_CR80","doi-asserted-by":"crossref","unstructured":"Veledar O, Damjanovic-Behrendt V, Macher G (2019) Digital twins for dependability improvement of autonomous driving. In: Systems, software and services process improvement: 26th European Conference, EuroSPI 2019, Edinburgh, UK, September 18\u201320, 2019, Proceedings 26. Springer, pp 415\u2013426","DOI":"10.1007\/978-3-030-28005-5_32"},{"key":"10458_CR81","doi-asserted-by":"publisher","unstructured":"Verma A, Bagkar S, Allam NVS, Raman A, Schmid M, Krovi VN (2021) Implementation and Validation of Behavior Cloning Using Scaled Vehicles. In: SAE WCX digital summit. SAE international. https:\/\/doi.org\/10.4271\/2021-01-0248","DOI":"10.4271\/2021-01-0248"},{"key":"10458_CR82","unstructured":"Viitala A, Boney R, Kannala J (2020) Learning to drive small scale cars from scratch. CoRR abs\/2008.00715. arXiv:2008.00715"},{"key":"10458_CR83","unstructured":"Waabi World (2022) https:\/\/waabi.ai\/waabi-world\/"},{"key":"10458_CR84","unstructured":"Waymo Simulation City (2021) https:\/\/waymo.com\/blog\/2021\/06\/SimulationCity.html"},{"key":"10458_CR85","unstructured":"Wayve (2022) Introducing wayve infinity simulator. https:\/\/wayve.ai\/blog\/introducing-wayve-infinity-simulator\/"},{"key":"10458_CR86","doi-asserted-by":"publisher","unstructured":"Yun H, Park D (2021) Simulation of self-driving system by implementing digital twin with gta5. In: 2021 International Conference on Electronics, Information, and Communication (ICEIC). pp 1\u20132. https:\/\/doi.org\/10.1109\/ICEIC51217.2021.9369807","DOI":"10.1109\/ICEIC51217.2021.9369807"},{"key":"10458_CR87","doi-asserted-by":"crossref","unstructured":"Yurtsever E, Lambert J, Carballo A, Takeda K (2020) A survey of autonomous driving: Common practices and emerging technologies. IEEE Access 8:58443\u201358469","DOI":"10.1109\/ACCESS.2020.2983149"},{"key":"10458_CR88","doi-asserted-by":"crossref","unstructured":"Zhang M, Zhang Y, Zhang L, Liu C, Khurshid S (2018) Deeproad: Gan-based metamorphic testing and input validation framework for autonomous driving systems. In: Proceedings of ASE \u201918","DOI":"10.1145\/3238147.3238187"},{"key":"10458_CR89","unstructured":"Zhong Z, Kaiser G, Ray B (2021) Neural network guided evolutionary fuzzing for finding traffic violations of autonomous vehicles"},{"key":"10458_CR90","doi-asserted-by":"publisher","unstructured":"Zhou H, Chen X, Zhang G, Zhou W (2021) Deep reinforcement learning for autonomous driving by transferring visual features. In: 2020 25th International Conference on Pattern Recognition (ICPR). https:\/\/doi.org\/10.1109\/ICPR48806.2021.9412011","DOI":"10.1109\/ICPR48806.2021.9412011"},{"key":"10458_CR91","doi-asserted-by":"crossref","unstructured":"Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Computer vision (ICCV), 2017 IEEE international conference on","DOI":"10.1109\/ICCV.2017.244"},{"key":"10458_CR92","doi-asserted-by":"crossref","unstructured":"Zohdinasab T, Riccio V, Gambi A, Tonella P (2021) Deephyperion: exploring the feature space of deep learning-based systems through illumination search. In: Proceedings of ISSTA \u201921","DOI":"10.1145\/3460319.3464811"},{"key":"10458_CR93","doi-asserted-by":"crossref","unstructured":"Zohdinasab T, Riccio V, Gambi A, Tonella P (2022) Efficient and effective feature space exploration for testing deep learning systems. ACM Trans Softw Eng Methodol","DOI":"10.1145\/3544792"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-024-10458-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-024-10458-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-024-10458-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,19]],"date-time":"2024-11-19T08:06:25Z","timestamp":1732003585000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-024-10458-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,17]]},"references-count":93,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2024,7]]}},"alternative-id":["10458"],"URL":"https:\/\/doi.org\/10.1007\/s10664-024-10458-4","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,17]]},"assertion":[{"value":"7 February 2024","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 May 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declared that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflicts of interests\/Competing interests"}}],"article-number":"72"}}