{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,25]],"date-time":"2026-07-25T20:14:21Z","timestamp":1785010461885,"version":"3.55.0"},"reference-count":89,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2023,4,4]],"date-time":"2023-04-04T00:00:00Z","timestamp":1680566400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Horizon 2020","award":["957254-COSMOS"],"award-info":[{"award-number":["957254-COSMOS"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2023,4,30]]},"abstract":"<jats:p>Testing with simulation environments helps to identify critical failing scenarios for self-driving cars (SDCs). Simulation-based tests are safer than in-field operational tests and allow detecting software defects before deployment. However, these tests are very expensive and are too many to be run frequently within limited time constraints.<\/jats:p>\n          <jats:p>\n            In this article, we investigate test case prioritization techniques to increase the ability to detect SDC regression faults with virtual tests earlier. Our approach, called\n            <jats:italic>SDC-Prioritizer<\/jats:italic>\n            , prioritizes virtual tests for SDCs according to static features of the roads we designed to be used within the driving scenarios. These features can be collected without running the tests, which means that they do not require past execution results. We introduce two evolutionary approaches to prioritize the test cases using diversity metrics (black-box heuristics) computed on these static features. These two approaches, called\n            <jats:italic>SO-SDC-Prioritizer<\/jats:italic>\n            and\n            <jats:italic>MO-SDC-Prioritizer<\/jats:italic>\n            , use single-objective and multi-objective\n            <jats:bold>genetic algorithms<\/jats:bold>\n            (\n            <jats:bold>GA<\/jats:bold>\n            ), respectively, to find trade-offs between executing the less expensive tests and the most diverse test cases earlier.\n          <\/jats:p>\n          <jats:p>\n            Our empirical study conducted in the SDC domain shows that\n            <jats:italic>MO-SDC-Prioritizer<\/jats:italic>\n            significantly (\n            <jats:italic>P<\/jats:italic>\n            - value &lt;=0.1\n            <jats:italic>e<\/jats:italic>\n            -10) improves the ability to detect safety-critical failures at the same level of execution time compared to baselines: random and greedy-based test case orderings. Besides, our study indicates that multi-objective meta-heuristics outperform single-objective approaches when prioritizing simulation-based tests for SDCs.\n          <\/jats:p>\n          <jats:p>\n            <jats:italic>MO-SDC-Prioritizer<\/jats:italic>\n            prioritizes test cases with a large improvement in fault detection while its overhead (up to 0.45% of the test execution cost) is negligible.\n          <\/jats:p>","DOI":"10.1145\/3533818","type":"journal-article","created":{"date-parts":[[2022,5,24]],"date-time":"2022-05-24T07:10:54Z","timestamp":1653376254000},"page":"1-30","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":67,"title":["Single and Multi-objective Test Cases Prioritization for Self-driving Cars in Virtual Environments"],"prefix":"10.1145","volume":"32","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3987-0276","authenticated-orcid":false,"given":"Christian","family":"Birchler","sequence":"first","affiliation":[{"name":"Zurich University of Applied Science, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0354-9747","authenticated-orcid":false,"given":"Sajad","family":"Khatiri","sequence":"additional","affiliation":[{"name":"Zurich University of Applied Science &amp; Software Institute - USI, Lugano, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3549-9019","authenticated-orcid":false,"given":"Pouria","family":"Derakhshanfar","sequence":"additional","affiliation":[{"name":"Delft University of Technology, Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4120-626X","authenticated-orcid":false,"given":"Sebastiano","family":"Panichella","sequence":"additional","affiliation":[{"name":"Zurich University of Applied Science, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7395-3588","authenticated-orcid":false,"given":"Annibale","family":"Panichella","sequence":"additional","affiliation":[{"name":"Delft University of Technology, Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,4,4]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"2020. NVIDIA DRIVE Constellation. Retrieved 16 June 2022 from https:\/\/developer.nvidia.com\/drive\/drive-constellation."},{"key":"e_1_3_2_3_2","first-page":"143","volume-title":"Proceedings of the IEEE\/ACM International Conference on Automated Software Engineering","author":"Abdessalem Raja Ben","year":"2018","unstructured":"Raja Ben Abdessalem, Annibale Panichella, Shiva Nejati, Lionel C. Briand, and Thomas Stifter. 2018. Testing autonomous cars for feature interaction failures using many-objective search. In Proceedings of the IEEE\/ACM International Conference on Automated Software Engineering. IEEE, 143\u2013154."},{"key":"e_1_3_2_4_2","volume-title":"A 21st Century Cyber-Physical Systems Education","author":"Sciences Academies of","year":"2017","unstructured":"Academies of Sciences. 2017. A 21st Century Cyber-Physical Systems Education. National Academies Press."},{"key":"e_1_3_2_5_2","article-title":"A study on the challenges of using robotics simulators for testing","author":"Afzal Afsoon","year":"2020","unstructured":"Afsoon Afzal, Deborah S. Katz, Claire Le Goues, and Christopher S. Timperley. 2020. A study on the challenges of using robotics simulators for testing. CoRR, arXiv:2004.07368. Retrieved from https:\/\/arxiv.org\/abs\/2004.07368.","journal-title":"CoRR"},{"key":"e_1_3_2_6_2","first-page":"197","volume-title":"Proceedings of the 18th International Software Product Line Conference","author":"Al-Hajjaji Mustafa","year":"2014","unstructured":"Mustafa Al-Hajjaji, Thomas Th\u00fcm, Jens Meinicke, Malte Lochau, and Gunter Saake. 2014. Similarity-based prioritization in software product-line testing. In Proceedings of the 18th International Software Product Line Conference, Stefania Gnesi, Alessandro Fantechi, Patrick Heymans, Julia Rubin, Krzysztof Czarnecki, and Deepak Dhungana (Eds.). ACM, 197\u2013206. DOI:10.1145\/2648511.2648532"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.1486"},{"issue":"3","key":"e_1_3_2_8_2","doi-asserted-by":"crossref","first-page":"594","DOI":"10.1007\/s10664-013-9249-9","article-title":"Parameter tuning or default values? An empirical investigation in search-based software engineering","volume":"18","author":"Arcuri Andrea","year":"2013","unstructured":"Andrea Arcuri and Gordon Fraser. 2013. Parameter tuning or default values? An empirical investigation in search-based software engineering. Empirical Software Engineering 18, 3 (2013), 594\u2013623.","journal-title":"Empirical Software Engineering"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11219-016-9341-7"},{"key":"e_1_3_2_10_2","doi-asserted-by":"crossref","first-page":"1411","DOI":"10.1145\/3205455.3205490","volume-title":"Proceedings of the Genetic and Evolutionary Computation Conference","author":"Arrieta Aitor","year":"2018","unstructured":"Aitor Arrieta, Shuai Wang, Ainhoa Arruabarrena, Urtzi Markiegi, Goiuria Sagardui, and Leire Etxeberria. 2018. Multi-objective black-box test case selection for cost-effectively testing simulation models. In Proceedings of the Genetic and Evolutionary Computation Conference. 1411\u20131418."},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/TII.2017.2788019"},{"key":"e_1_3_2_12_2","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1145\/2934466.2946046","volume-title":"Proceedings of the 20th International Systems and Software Product Line Conference","author":"Arrieta Aitor","year":"2016","unstructured":"Aitor Arrieta, Shuai Wang, Goiuria Sagardui, and Leire Etxeberria. 2016. Search-based test case selection of cyber-physical system product lines for simulation-based validation. In Proceedings of the 20th International Systems and Software Product Line Conference, Hong Mei (Ed.). ACM, 297\u2013306. DOI:10.1145\/2934466.2946046"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2018.09.055"},{"issue":"1","key":"e_1_3_2_14_2","first-page":"161","article-title":"Cyber-physical systems","volume":"12","author":"Baheti Radhakisan","year":"2011","unstructured":"Radhakisan Baheti and Helen Gill. 2011. Cyber-physical systems. The Impact of Control Technology 12, 1 (2011), 161\u2013166.","journal-title":"The Impact of Control Technology"},{"key":"e_1_3_2_15_2","unstructured":"BeamNG.research. [n.d.]. BeamNG GmbH \u201cBeamNG.research \u2013 BeamNG.\u201d Retrieved April 2021 from https:\/\/beamng.gmbh\/research\/."},{"key":"e_1_3_2_16_2","volume-title":"Proceedings of the 29th IEEE International Conference on Software Analysis, Evolution, and Reengineering","author":"Birchler Christian","year":"2022","unstructured":"Christian Birchler, Nicolas Ganz, Sajad Khatiri, Alessio Gambi, and Sebastiano Panichella. 2022. Cost-effective simulation-based test selection in self-driving cars software with SDC-scissor. 2022. In Proceedings of the 29th IEEE International Conference on Software Analysis, Evolution, and Reengineering. To appear."},{"key":"e_1_3_2_17_2","first-page":"40:1\u201340:12","volume-title":"Proceedings of the 1st ACM SIGCAS Conference on Computing and Sustainable Societies","author":"Bondi Elizabeth","year":"2018","unstructured":"Elizabeth Bondi, Debadeepta Dey, Ashish Kapoor, Jim Piavis, Shital Shah, Fei Fang, Bistra Dilkina, Robert Hannaford, Arvind Iyer, Lucas Joppa, and Milind Tambe. 2018. AirSim-W: A simulation environment for wildlife conservation with UAVs. In Proceedings of the 1st ACM SIGCAS Conference on Computing and Sustainable Societies, Ellen W. Zegura (Ed.). ACM, 40:1\u201340:12. DOI:10.1145\/3209811.3209880"},{"key":"e_1_3_2_18_2","series-title":"Proceedings of the 8th International Parallel Problem Solving from Nature","first-page":"722","volume":"3242","author":"Branke J\u00fcrgen","year":"2004","unstructured":"J\u00fcrgen Branke, Kalyanmoy Deb, Henning Dierolf, and Matthias Osswald. 2004. Finding knees in multi-objective optimization. In Proceedings of the 8th International Parallel Problem Solving from NatureLNCS, Vol. 3242. Springer, Berlin, 722\u2013731."},{"issue":"2","key":"e_1_3_2_19_2","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1007\/s10710-006-9003-9","article-title":"Using genetic algorithms for early schedulability analysis and stress testing in real-time systems","volume":"7","author":"Briand Lionel C.","year":"2006","unstructured":"Lionel C. Briand, Yvan Labiche, and Marwa Shousha. 2006. Using genetic algorithms for early schedulability analysis and stress testing in real-time systems. Genetic Programming and Evolvable Machines 7, 2 (2006), 145\u2013170.","journal-title":"Genetic Programming and Evolvable Machines"},{"key":"e_1_3_2_20_2","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1016\/j.jss.2017.09.031","article-title":"Test case prioritization for object-oriented software: An adaptive random sequence approach based on clustering","volume":"135","author":"Chen Jinfu","year":"2018","unstructured":"Jinfu Chen, Lili Zhu, Tsong Yueh Chen, Dave Towey, Fei-Ching Kuo, Rubing Huang, and Yuchi Guo. 2018. Test case prioritization for object-oriented software: An adaptive random sequence approach based on clustering. Journal of Systems and Software 135 (2018), 107\u2013125.","journal-title":"Journal of Systems and Software"},{"issue":"3","key":"e_1_3_2_21_2","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1016\/S0020-0190(96)00135-4","article-title":"Dividing strategies for the optimization of a test suite","volume":"60","author":"Chen Tsong Yueh","year":"1996","unstructured":"Tsong Yueh Chen and Man Fai Lau. 1996. Dividing strategies for the optimization of a test suite. Information Processing Letters 60, 3 (Nov.1996), 135\u2013141.","journal-title":"Information Processing Letters"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.5771017"},{"key":"e_1_3_2_23_2","unstructured":"OpenStax CNX. 2021. OpenStax University Physics. Retrieved 16 June 2022 from http:\/\/cnx.org\/contents\/d50f6e32-0fda-46ef-a362-9bd36ca7c97d@10.16."},{"key":"e_1_3_2_24_2","doi-asserted-by":"crossref","DOI":"10.21236\/ADA294075","volume-title":"Genetic Algorithms for Tracking Changing Environments.","author":"Cobb Helen G.","year":"1993","unstructured":"Helen G. Cobb and John J. Grefenstette. 1993. Genetic Algorithms for Tracking Changing Environments.Technical Report. Naval Research Lab Washington DC."},{"key":"e_1_3_2_25_2","volume-title":"Evolutionary Algorithms for Solving Multi-Objective Problems (Genetic and Evolutionary Computation)","author":"Coello Carlos A. Coello","year":"2006","unstructured":"Carlos A. Coello Coello, Gary B. Lamont, and David A. Van Veldhuizen. 2006. Evolutionary Algorithms for Solving Multi-Objective Problems (Genetic and Evolutionary Computation). Springer-Verlag New York, Inc., Secaucus, NJ, USA."},{"issue":"3","key":"e_1_3_2_26_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2480741.2480752","article-title":"Exploration and exploitation in evolutionary algorithms: A survey","volume":"45","author":"\u010crepin\u0161ek Matej","year":"2013","unstructured":"Matej \u010crepin\u0161ek, Shih-Hsi Liu, and Marjan Mernik. 2013. Exploration and exploitation in evolutionary algorithms: A survey. ACM computing surveys 45, 3 (2013), 1\u201333.","journal-title":"ACM computing surveys"},{"issue":"2","key":"e_1_3_2_27_2","first-page":"182","article-title":"A fast elitist multi-objective genetic algorithm: NSGA-II","volume":"6","author":"Deb Kalyanmoy","year":"2000","unstructured":"Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal, and T. Meyarivan. 2000. A fast elitist multi-objective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation 6, 2 (2000), 182\u2013197.","journal-title":"IEEE Transactions on Evolutionary Computation"},{"issue":"2","key":"e_1_3_2_28_2","doi-asserted-by":"crossref","first-page":"182","DOI":"10.1109\/4235.996017","article-title":"A fast and elitist multiobjective genetic algorithm: NSGA-II","volume":"6","author":"Deb Kalyanmoy","year":"2002","unstructured":"Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal, and TAMT Meyarivan. 2002. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE transactions on evolutionary computation 6, 2 (2002), 182\u2013197.","journal-title":"IEEE transactions on evolutionary computation"},{"key":"e_1_3_2_29_2","article-title":"A test case prioritization genetic algorithm guided by the hypervolume indicator","author":"Nucci Dario Di","year":"2018","unstructured":"Dario Di Nucci, Annibale Panichella, Andy Zaidman, and Andrea De Lucia. 2018. A test case prioritization genetic algorithm guided by the hypervolume indicator. IEEE Transactions on Software Engineering 46, 6 (2018), 674\u2013696.","journal-title":"IEEE Transactions on Software Engineering"},{"issue":"1","key":"e_1_3_2_30_2","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1111\/j.1600-0587.2012.07348.x","article-title":"Collinearity: A review of methods to deal with it and a simulation study evaluating their performance","volume":"36","author":"Dormann Carsten F.","year":"2013","unstructured":"Carsten F. Dormann, Jane Elith, Sven Bacher, Carsten Buchmann, Gudrun Carl, Gabriel Carr\u00e9, Jaime R. Garc\u00eda Marqu\u00e9z, Bernd Gruber, Bruno Lafourcade, Pedro J. Leitao, and Munkemuller T. 2013. Collinearity: A review of methods to deal with it and a simulation study evaluating their performance. Ecography 36, 1 (2013), 27\u201346.","journal-title":"Ecography"},{"key":"e_1_3_2_31_2","first-page":"1","volume-title":"Proceedings of the 1st Annual Conference on Robot Learning","author":"Dosovitskiy Alexey","year":"2017","unstructured":"Alexey Dosovitskiy, Germ\u00e1n Ros, Felipe Codevilla, Antonio M. L\u00f3pez, and Vladlen Koltun. 2017. CARLA: An open urban driving simulator. In Proceedings of the 1st Annual Conference on Robot Learning. PMLR, 1\u201316. Retrieved from http:\/\/proceedings.mlr.press\/v78\/dosovitskiy17a.html."},{"key":"e_1_3_2_32_2","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1016\/j.knosys.2016.02.001","article-title":"Study on density peaks clustering based on k-nearest neighbors and principal component analysis","volume":"99","author":"Du Mingjing","year":"2016","unstructured":"Mingjing Du, Shifei Ding, and Hongjie Jia. 2016. Study on density peaks clustering based on k-nearest neighbors and principal component analysis. Knowledge-Based Systems 99 (2016), 135\u2013145.","journal-title":"Knowledge-Based Systems"},{"key":"e_1_3_2_33_2","first-page":"329","volume-title":"Proceedings of the 23rd International Conference on Software Engineering.","author":"Elbaum Sebastian","year":"2001","unstructured":"Sebastian Elbaum, Alexey Malishevsky, and Gregg Rothermel. 2001. Incorporating varying test costs and fault severities into test case prioritization. In Proceedings of the 23rd International Conference on Software Engineering. IEEE, 329\u2013338."},{"key":"e_1_3_2_34_2","doi-asserted-by":"crossref","first-page":"234","DOI":"10.1145\/2771783.2771788","volume-title":"Proceedings of the 2015 International Symposium on Software Testing and Analysis","author":"Epitropakis Michael G.","year":"2015","unstructured":"Michael G. Epitropakis, Shin Yoo, Mark Harman, and Edmund K. Burke. 2015. Empirical evaluation of pareto efficient multi-objective regression test case prioritisation. In Proceedings of the 2015 International Symposium on Software Testing and Analysis. ACM, 234\u2013245."},{"key":"e_1_3_2_35_2","first-page":"223","volume-title":"Proceedings of the 2016 IEEE International Conference on Software Testing, Verification and Validation","author":"Feldt Robert","year":"2016","unstructured":"Robert Feldt, Simon Poulding, David Clark, and Shin Yoo. 2016. Test set diameter: Quantifying the diversity of sets of test cases. In Proceedings of the 2016 IEEE International Conference on Software Testing, Verification and Validation. IEEE, 223\u2013233."},{"key":"e_1_3_2_36_2","doi-asserted-by":"crossref","first-page":"238","DOI":"10.1007\/978-3-030-57997-5_28","volume-title":"Proceedings of the Advances in Production Management Systems. Towards Smart and Digital Manufacturing","author":"Flores-Garc\u00eda Erik","year":"2020","unstructured":"Erik Flores-Garc\u00eda, Goo-Young Kim, Jinho Yang, Magnus Wiktorsson, and Sang Do Noh. 2020. Analyzing the characteristics of digital twin and discrete event simulation in cyber physical systems. In Proceedings of the Advances in Production Management Systems. Towards Smart and Digital Manufacturing, Bojan Lalic, Vidosav D. Majstorovic, Ugljesa Marjanovic, Gregor von Cieminski, and David Romero (Eds.). Springer, 238\u2013244. DOI:10.1007\/978-3-030-57997-5_28"},{"key":"e_1_3_2_37_2","first-page":"257","volume-title":"Proceedings of the ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering","author":"Gambi Alessio","year":"2019","unstructured":"Alessio Gambi, Tri Huynh, and Gordon Fraser. 2019. Generating effective test cases for self-driving cars from police reports. In Proceedings of the ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Marlon Dumas, Dietmar Pfahl, Sven Apel, and Alessandra Russo (Eds.). ACM, 257\u2013267. DOI:10.1145\/3338906.3338942"},{"key":"e_1_3_2_38_2","volume-title":"Proceedings of the 2019 IEEE\/ACM 41st International Conference on Software Engineering: Companion Proceedings","author":"Gambi Alessio","year":"2019","unstructured":"Alessio Gambi, Marc Mueller, and Gordon Fraser. 2019. AsFault: Testing self-driving car software using search-based procedural content generation. In Proceedings of the 2019 IEEE\/ACM 41st International Conference on Software Engineering: Companion Proceedings. IEEE. DOI:10.1109\/icse-companion.2019.00030"},{"key":"e_1_3_2_39_2","doi-asserted-by":"crossref","first-page":"318","DOI":"10.1145\/3293882.3330566","volume-title":"Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis","author":"Gambi Alessio","year":"2019","unstructured":"Alessio Gambi, Marc Mueller, and Gordon Fraser. 2019. Automatically testing self-driving cars with search-based procedural content generation. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis. ACM, New York, NY, 318\u2013328. DOI:10.1145\/3293882.3330566"},{"key":"e_1_3_2_40_2","volume-title":"Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems","author":"G\u00e9ron Aur\u00e9lien","year":"2019","unstructured":"Aur\u00e9lien G\u00e9ron. 2019. Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems. O\u2019Reilly Media."},{"key":"e_1_3_2_41_2","volume-title":"Genetic Algorithms","author":"Goldberg David E.","year":"2006","unstructured":"David E. Goldberg. 2006. Genetic Algorithms. Pearson Education India."},{"key":"e_1_3_2_42_2","doi-asserted-by":"crossref","first-page":"176","DOI":"10.1145\/3239372.3239409","volume-title":"Proceedings of the 21th ACM\/IEEE International Conference on Model Driven Engineering Languages and Systems","author":"Gonz\u00e1lez Carlos A.","year":"2018","unstructured":"Carlos A. Gonz\u00e1lez, Mojtaba Varmazyar, Shiva Nejati, Lionel C. Briand, and Yago Isasi. 2018. Enabling model testing of cyber-physical systems. In Proceedings of the 21th ACM\/IEEE International Conference on Model Driven Engineering Languages and Systems. ACM, New York, NY, 176\u2013186. DOI:10.1145\/3239372.3239409"},{"key":"e_1_3_2_43_2","volume-title":"Proceedings of the International Conference on Software Analysis, Evolution and Reengineering","author":"Grano Giovanni","year":"2018","unstructured":"Giovanni Grano, Adelina Ciurumelea, Sebastiano Panichella, Fabio Palomba, and Harald Gall. 2018. Exploring the integration of user feedback in automated testing of Android applications. In Proceedings of the International Conference on Software Analysis, Evolution and Reengineering."},{"key":"e_1_3_2_44_2","unstructured":"The Guardian. 2018. Self-driving Uber kills Arizona woman in first fatal crash involving pedestrian. Retrieved 16 June 2022 from https:\/\/www.theguardian.com\/technology\/2018\/mar\/19\/uber-self-driving-car-kills-woman-arizona-tempe."},{"issue":"1","key":"e_1_3_2_45_2","first-page":"6","article-title":"Achieving scalable model-based testing through test case diversity","volume":"22","author":"Hemmati Hadi","year":"2013","unstructured":"Hadi Hemmati, Andrea Arcuri, and Lionel Briand. 2013. Achieving scalable model-based testing through test case diversity. ACM Transactions on Software Engineering and Methodology 22, 1 (2013), 6.","journal-title":"ACM Transactions on Software Engineering and Methodology"},{"key":"e_1_3_2_46_2","first-page":"523","volume-title":"Proceedings of the 2016 IEEE\/ACM 38th International Conference on Software Engineering","author":"Henard Christopher","year":"2016","unstructured":"Christopher Henard, Mike Papadakis, Mark Harman, Yue Jia, and Yves Le Traon. 2016. Comparing white-box and black-box test prioritization. In Proceedings of the 2016 IEEE\/ACM 38th International Conference on Software Engineering. IEEE, 523\u2013534."},{"key":"e_1_3_2_47_2","doi-asserted-by":"crossref","first-page":"405","DOI":"10.1007\/s10664-005-3861-2","article-title":"Supporting controlled experimentation with testing techniques: An infrastructure and its potential impact.","volume":"10","author":"Do Sebastian G. Elbaum Hyunsook","year":"2005","unstructured":"Sebastian G. Elbaum Hyunsook Do and Gregg Rothermel. 2005. Supporting controlled experimentation with testing techniques: An infrastructure and its potential impact. Empirical Software Engineering 10 (2005), 405\u2013435.","journal-title":"Empirical Software Engineering"},{"key":"e_1_3_2_48_2","first-page":"321","volume-title":"Proceedings of the 3rd IEEE International Conference on Robotic Computing","author":"Ingrand F\u00e9lix","year":"2019","unstructured":"F\u00e9lix Ingrand. 2019. Recent trends in formal validation and verification of autonomous robots software. In Proceedings of the 3rd IEEE International Conference on Robotic Computing. 321\u2013328."},{"key":"e_1_3_2_49_2","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1109\/ASE.2009.77","volume-title":"Proceedings of the 2009 IEEE\/ACM International Conference on Automated Software Engineering","author":"Jiang Bo","year":"2009","unstructured":"Bo Jiang, Zhenyu Zhang, Wing Kwong Chan, and TH Tse. 2009. Adaptive random test case prioritization. In Proceedings of the 2009 IEEE\/ACM International Conference on Automated Software Engineering. IEEE, 233\u2013244."},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.tra.2016.09.010"},{"key":"e_1_3_2_51_2","volume-title":"Proceedings of the 2019 IEEE International Conference on Software Testing, Verification and Validation Workshops","author":"Kim Jiseob","year":"2019","unstructured":"Jiseob Kim, Sunil Chon, and Jihwan Park. 2019. Suggestion of testing method for industrial level cyber-physical system in complex environment. In Proceedings of the 2019 IEEE International Conference on Software Testing, Verification and Validation Workshops. IEEE. DOI:10.1109\/icstw.2019.00043"},{"issue":"6","key":"e_1_3_2_52_2","doi-asserted-by":"crossref","first-page":"1101","DOI":"10.1002\/nme.1620240606","article-title":"Norm methods and partial weighting in multicriterion optimization of structures","volume":"24","author":"Koski Juhani","year":"1987","unstructured":"Juhani Koski and Risto Silvennoinen. 1987. Norm methods and partial weighting in multicriterion optimization of structures. Internaternational Journal for Numerical Methods in Engineering 24, 6 (1987), 1101\u20131121.","journal-title":"Internaternational Journal for Numerical Methods in Engineering"},{"issue":"1","key":"e_1_3_2_53_2","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1007\/s10515-011-0093-0","article-title":"Prioritizing test cases with string distances","volume":"19","author":"Ledru Yves","year":"2012","unstructured":"Yves Ledru, Alexandre Petrenko, Sergiy Boroday, and Nadine Mandran. 2012. Prioritizing test cases with string distances. Automated Software Engineering 19, 1 (2012), 65\u201395.","journal-title":"Automated Software Engineering"},{"key":"e_1_3_2_54_2","doi-asserted-by":"crossref","first-page":"696","DOI":"10.1007\/11881070_94","article-title":"Genetic algorithm based on the orthogonal design for multidimensional knapsack problems","volume":"4221","author":"Li Hong","year":"2006","unstructured":"Hong Li, Yong-Chang Jiao, Li Zhang, and Ze-Wei Gu. 2006. Genetic algorithm based on the orthogonal design for multidimensional knapsack problems. Advances in Natural Computation 4221 (2006), 696\u2013705.","journal-title":"Advances in Natural Computation"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2007.38"},{"issue":"1","key":"e_1_3_2_56_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TRO.2019.2942989","article-title":"Deep drone racing: From simulation to reality with domain randomization","volume":"36","author":"Loquercio A.","year":"2020","unstructured":"A. Loquercio, E. Kaufmann, R. Ranftl, A. Dosovitskiy, V. Koltun, and D. Scaramuzza. 2020. Deep drone racing: From simulation to reality with domain randomization. IEEE Transactions on Robotics 36, 1 (2020), 1\u201314.","journal-title":"IEEE Transactions on Robotics"},{"key":"e_1_3_2_57_2","volume-title":"Cost-cognizant Test Case Prioritization","author":"Malishevsky Alexey G.","year":"2006","unstructured":"Alexey G. Malishevsky, Joseph R. Ruthruff, Gregg Rothermel, and Sebastian Elbaum. 2006. Cost-cognizant Test Case Prioritization. Technical Report. Technical Report TR-UNL-CSE-2006-0004, University of Nebraska-Lincoln."},{"issue":"10","key":"e_1_3_2_58_2","doi-asserted-by":"crossref","first-page":"918","DOI":"10.1109\/TSE.2015.2510633","article-title":"A multi-objective technique to prioritize test cases","volume":"42","author":"Marchetto Alessandro","year":"2015","unstructured":"Alessandro Marchetto, Md Mahfuzul Islam, Waseem Asghar, Angelo Susi, and Giuseppe Scanniello. 2015. A multi-objective technique to prioritize test cases. IEEE Transactions on Software Engineering 42, 10 (2015), 918\u2013940.","journal-title":"IEEE Transactions on Software Engineering"},{"key":"e_1_3_2_59_2","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1007\/s00158-003-0368-6","article-title":"Survey of multi-objective optimization methods for engineering","volume":"26","author":"Marler R. T.","year":"2004","unstructured":"R. T. Marler and J. S. Arora. 2004. Survey of multi-objective optimization methods for engineering. Structural and Multidisciplinary Optimization 26 (2004), 369\u2013395.","journal-title":"Structural and Multidisciplinary Optimization"},{"key":"e_1_3_2_60_2","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1007\/978-3-642-39742-4_12","volume-title":"Proceedings of the International Symposium on Search Based Software Engineering","author":"Matinnejad Reza","year":"2013","unstructured":"Reza Matinnejad, Shiva Nejati, Lionel Briand, Thomas Bruckmann, and Claude Poull. 2013. Automated model-in-the-loop testing of continuous controllers using search. In Proceedings of the International Symposium on Search Based Software Engineering. Springer, 141\u2013157."},{"key":"e_1_3_2_61_2","first-page":"2690","volume-title":"Proceedings of the Winter Simulation Conference","author":"Mesit Jaruwan","year":"2011","unstructured":"Jaruwan Mesit and Ratan K. Guha. 2011. A general model for soft body simulation in motion. In Proceedings of the Winter Simulation Conference, S. Jain, Roy R. Creasey Jr., Jan Himmelspach, K. Preston White, and Michael C. Fu 0001 (Eds.). IEEE, 2690\u20132702. Retrieved from http:\/\/dl.acm.org\/citation.cfm?id=2431518."},{"key":"e_1_3_2_62_2","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1145\/3196321.3196340","volume-title":"Proceedings of the 26th Conference on Program Comprehension","author":"Messaoudi Salma","year":"2018","unstructured":"Salma Messaoudi, Annibale Panichella, Domenico Bianculli, Lionel Briand, and Raimondas Sasnauskas. 2018. A search-based approach for accurate identification of log message formats. In Proceedings of the 26th Conference on Program Comprehension. 167\u2013177."},{"key":"e_1_3_2_63_2","first-page":"222","volume-title":"Proceedings of the 2018 IEEE\/ACM 40th International Conference on Software Engineering (ICSE)","author":"Miranda Breno","year":"2018","unstructured":"Breno Miranda, Emilio Cruciani, Roberto Verdecchia, and Antonia Bertolino. 2018. Fast approaches to scalable similarity-based test case prioritization. In Proceedings of the 2018 IEEE\/ACM 40th International Conference on Software Engineering (ICSE). IEEE, 222\u2013232."},{"key":"e_1_3_2_64_2","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1145\/2330163.2330233","volume-title":"Proceedings of the 14th Annual Conference on Genetic and Evolutionary Computation","author":"Nagy R\u00e9ka","year":"2012","unstructured":"R\u00e9ka Nagy, Mihai A. Suciu, and Dumitru Dumitrescu. 2012. Lorenz equilibrium: Equitability in non-cooperative games. In Proceedings of the 14th Annual Conference on Genetic and Evolutionary Computation. ACM, New York, NY, 489\u2013496."},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2018.2868082"},{"key":"e_1_3_2_66_2","first-page":"1","volume-title":"Proceedings of the 8th IEEE International Conference on Software Testing, Verification and Validation","author":"Panichella Annibale","year":"2015","unstructured":"Annibale Panichella, Fitsum M. Kifetew, and Paolo Tonella. 2015. Reformulating branch coverage as a many-objective optimization problem. In Proceedings of the 8th IEEE International Conference on Software Testing, Verification and Validation. IEEE, Piscataway, NJ, 1\u201310."},{"key":"e_1_3_2_67_2","volume-title":"Proceedings of the International Conference on Software Engineering, Workshops, 2021","author":"Panichella Sebastiano","year":"2021","unstructured":"Sebastiano Panichella, Alessio Gambi, Fiorella Zampetti, and Vincenzo Riccio. 2021. SBST tool competition 2021. In Proceedings of the International Conference on Software Engineering, Workshops, 2021. ACM."},{"key":"e_1_3_2_68_2","first-page":"281","volume-title":"Proceedings of the International Conference on Software Maintenance and Evolution","author":"Panichella Sebastiano","year":"2015","unstructured":"Sebastiano Panichella, Andrea Di Sorbo, Emitza Guzman, Corrado Aaron Visaggio, Gerardo Canfora, and Harald C. Gall. 2015. How can i improve my app? Classifying user reviews for software maintenance and evolution. In Proceedings of the International Conference on Software Maintenance and Evolution, Rainer Koschke, Jens Krinke, and Martin P. Robillard (Eds.). IEEE Computer Society, 281\u2013290. DOI:10.1109\/ICSM.2015.7332474"},{"key":"e_1_3_2_69_2","doi-asserted-by":"crossref","unstructured":"Singiresu S. Rao and TI Freiheit. 1991. A modified game theory approach to multiobjective optimization. J ournal of Mechanical Design 113 3 (1991) 286\u2013291.","DOI":"10.1115\/1.2912781"},{"key":"e_1_3_2_70_2","first-page":"13 pages","volume-title":"Proceedings of the ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering","author":"Riccio Vincenzo","year":"2020","unstructured":"Vincenzo Riccio and Paolo Tonella. 2020. Model-based exploration of the frontier of behaviours for deep learning system testing. In Proceedings of the ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ACM, 13 pages. DOI:10.1145\/3368089.3409730"},{"key":"e_1_3_2_71_2","first-page":"34","volume-title":"Proceedings of the International Conference on Software Maintenance","author":"Rothermel Gregg","year":"1998","unstructured":"Gregg Rothermel, Mary Jean Harrold, Jeffery Ostrin, and Christie Hong. 1998. An empirical study of the effects of minimization on the fault detection capabilities of test suites. In Proceedings of the International Conference on Software Maintenance. IEEE CS Press, 34\u201344."},{"key":"e_1_3_2_72_2","first-page":"179","volume-title":"Proceedings of the IEEE International Conference on Software Maintenance.","author":"Rothermel G.","year":"1999","unstructured":"G. Rothermel, R. H. Untch, Chengyun Chu, and M. J. Harrold. 1999. Test case prioritization: An empirical study. In Proceedings of the IEEE International Conference on Software Maintenance.179\u2013188. DOI:10.1109\/ICSM.1999.792604"},{"key":"e_1_3_2_73_2","first-page":"179","volume-title":"Proceedings of the IEEE International Conference on Software Maintenance","author":"Rothermel Gregg","year":"1999","unstructured":"Gregg Rothermel, Roland H. Untch, Chengyun Chu, and Mary Jean Harrold. 1999. Test case prioritization: An empirical study. In Proceedings of the IEEE International Conference on Software Maintenance. IEEE, 179\u2013188."},{"key":"e_1_3_2_74_2","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1109\/RESER.2013.6","volume-title":"Proceedings of the 2013 3rd International Workshop on Replication in Empirical Software Engineering Research","author":"Sayyad Abdel Salam","year":"2013","unstructured":"Abdel Salam Sayyad, Katerina Goseva-Popstojanova, Tim Menzies, and Hany Ammar. 2013. On parameter tuning in search based software engineering: A replicated empirical study. In Proceedings of the 2013 3rd International Workshop on Replication in Empirical Software Engineering Research. IEEE, 84\u201390."},{"key":"e_1_3_2_75_2","first-page":"51","volume-title":"Proceedings of the 3rd International Conference on Genetic Algorithms","author":"Schaffer J. David","year":"1989","unstructured":"J. David Schaffer, Rich Caruana, Larry J. Eshelman, and Rajarshi Das. 1989. A study of control parameters affecting online performance of genetic algorithms for function optimization. In Proceedings of the 3rd International Conference on Genetic Algorithms. 51\u201360."},{"key":"e_1_3_2_76_2","doi-asserted-by":"publisher","DOI":"10.1162\/EVCO_a_00006"},{"key":"e_1_3_2_77_2","first-page":"357","volume-title":"Proceedings of the International Conference on Model Driven Engineering Languages and Systems","author":"Shin Seung Yeob","year":"2018","unstructured":"Seung Yeob Shin, Karim Chaouch, Shiva Nejati, Mehrdad Sabetzadeh, Lionel C. Briand, and Frank Zimmer. 2018. HITECS: A UML profile and analysis framework for hardware-in-the-loop testing of cyber physical systems. In Proceedings of the International Conference on Model Driven Engineering Languages and Systems. ACM, 357\u2013367."},{"key":"e_1_3_2_78_2","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1145\/3213846.3213852","volume-title":"Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis","author":"Shin Seung Yeob","year":"2018","unstructured":"Seung Yeob Shin, Shiva Nejati, Mehrdad Sabetzadeh, Lionel C. Briand, and Frank Zimmer. 2018. Test case prioritization for acceptance testing of cyber physical systems: A multi-objective search-based approach. In Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis, Frank Tip and Eric Bodden (Eds.). ACM, 49\u201360. DOI:10.1145\/3213846.3213852"},{"key":"e_1_3_2_79_2","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2017.2742141"},{"key":"e_1_3_2_80_2","first-page":"499","volume-title":"Proceedings of the International Symposium on Foundations of Software Engineering","author":"Sorbo Andrea Di","year":"2016","unstructured":"Andrea Di Sorbo, Sebastiano Panichella, Carol V. Alexandru, Junji Shimagaki, Corrado Aaron Visaggio, Gerardo Canfora, and Harald C. Gall. 2016. What would users change in my app? summarizing app reviews for recommending software changes. In Proceedings of the International Symposium on Foundations of Software Engineering, Thomas Zimmermann, Jane Cleland-Huang, and Zhendong Su (Eds.). ACM, 499\u2013510. DOI:10.1145\/2950290.2950299"},{"key":"e_1_3_2_81_2","unstructured":"The-Washington-Post. 2019. Uber\u2019s radar detected Elaine Herzberg nearly 6 seconds before she was fatally struck but \u201cthe system design did not include a consideration for jaywalking pedestrians\u201d so it didn\u2019t react as if she were a person. Retrieved 16 June 2022 from https:\/\/mobile.twitter.com\/faizsays\/status\/1191885955088519168."},{"issue":"1","key":"e_1_3_2_82_2","doi-asserted-by":"crossref","first-page":"182","DOI":"10.1007\/s10664-012-9219-7","article-title":"Static test case prioritization using topic models","volume":"19","author":"Thomas Stephen W.","year":"2014","unstructured":"Stephen W. Thomas, Hadi Hemmati, Ahmed E. Hassan, and Dorothea Blostein. 2014. Static test case prioritization using topic models. Empirical Software Engineering 19, 1 (2014), 182\u2013212.","journal-title":"Empirical Software Engineering"},{"issue":"2","key":"e_1_3_2_83_2","first-page":"101","article-title":"A critique and improvement of the CL common language effect size statistics of McGraw and Wong","volume":"25","author":"Vargha Andr\u00e1s","year":"2000","unstructured":"Andr\u00e1s Vargha and Harold D. Delaney. 2000. A critique and improvement of the CL common language effect size statistics of McGraw and Wong. Journal of Educational and Behavioral Statistics 25, 2 (2000), 101\u2013132.","journal-title":"Journal of Educational and Behavioral Statistics"},{"key":"e_1_3_2_84_2","first-page":"5049","volume-title":"Proceedings of the 2019 IEEE\/RSJ International Conference on Intelligent Robots and Systems","author":"Xu Jiaxuan","year":"2019","unstructured":"Jiaxuan Xu, Qi Luo, Kecheng Xu, Xiangquan Xiao, Siyang Yu, Jiangtao Hu, Jinghao Miao, and Jingao Wang. 2019. An automated learning-based procedure for large-scale vehicle dynamics modeling on baidu apollo platform. In Proceedings of the 2019 IEEE\/RSJ International Conference on Intelligent Robots and Systems. IEEE, 5049\u20135056. DOI:10.1109\/IROS40897.2019.8968102"},{"key":"e_1_3_2_85_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.3016826"},{"key":"e_1_3_2_86_2","volume-title":"Proceedings of the 2nd International Symposium on Search-Based Software Engineering","author":"Yoo Shin","year":"2010","unstructured":"Shin Yoo. 2010. A novel mask-coding representation for set cover problems with applications in test suite minimisation. In Proceedings of the 2nd International Symposium on Search-Based Software Engineering. IEEE."},{"issue":"4","key":"e_1_3_2_87_2","doi-asserted-by":"crossref","first-page":"689","DOI":"10.1016\/j.jss.2009.11.706","article-title":"Using hybrid algorithm for Pareto efficient multi-objective test suite minimisation","volume":"83","author":"Yoo Shin","year":"2010","unstructured":"Shin Yoo and Mark Harman. 2010. Using hybrid algorithm for Pareto efficient multi-objective test suite minimisation. Journal of Systems and Software 83, 4 (2010), 689\u2013701.","journal-title":"Journal of Systems and Software"},{"issue":"2","key":"e_1_3_2_88_2","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1002\/stv.430","article-title":"Regression testing minimization, selection and prioritization: A survey","volume":"22","author":"Yoo S.","year":"2012","unstructured":"S. Yoo and M. Harman. 2012. Regression testing minimization, selection and prioritization: A survey. Software Testing, Verification and Reliability 22, 2 (March2012), 67\u2013120.","journal-title":"Software Testing, Verification and Reliability"},{"key":"e_1_3_2_89_2","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1007\/978-3-030-60508-7_9","volume-title":"Proceedings of the Runtime Verification","author":"Zapridou Eleni","year":"2020","unstructured":"Eleni Zapridou, Ezio Bartocci, and Panagiotis Katsaros. 2020. Runtime verification of autonomous driving systems in CARLA. In Proceedings of the Runtime Verification, Jyotirmoy Deshmukh and Dejan Ni\u010dkovi\u0107 (Eds.). Springer International Publishing, Cham, 172\u2013183."},{"key":"e_1_3_2_90_2","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1109\/SCAM.2010.14","volume-title":"Proceedings of the 2010 10th IEEE Working Conference on Source Code Analysis and Manipulation","author":"\u00dajh\u00e1zi B\u00e9la","year":"2010","unstructured":"B\u00e9la \u00dajh\u00e1zi, Rudolf Ferenc, Denys Poshyvanyk, and Tibor Gyim\u00f3thy. 2010. New conceptual coupling and cohesion metrics for object-oriented systems. In Proceedings of the 2010 10th IEEE Working Conference on Source Code Analysis and Manipulation. 33\u201342. DOI:10.1109\/SCAM.2010.14"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3533818","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3533818","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T18:43:41Z","timestamp":1750272221000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3533818"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,4]]},"references-count":89,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2023,4,30]]}},"alternative-id":["10.1145\/3533818"],"URL":"https:\/\/doi.org\/10.1145\/3533818","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,4]]},"assertion":[{"value":"2021-07-13","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-04-27","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-04-04","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}