{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,17]],"date-time":"2026-10-17T17:09:22Z","timestamp":1792256962893,"version":"4.4.0"},"reference-count":84,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2022,1,11]],"date-time":"2022-01-11T00:00:00Z","timestamp":1641859200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,1,11]],"date-time":"2022-01-11T00:00:00Z","timestamp":1641859200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100004359","name":"Vetenskapsr\u00e5det","doi-asserted-by":"publisher","award":["2019-05275"],"award-info":[{"award-number":["2019-05275"]}],"id":[{"id":"10.13039\/501100004359","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002835","name":"Chalmers University of Technology","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100002835","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2022,3]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Search-based test generation is guided by feedback from one or more fitness functions\u2014scoring functions that judge solution optimality. Choosing informative fitness functions is crucial to meeting the goals of a tester. Unfortunately, many goals\u2014such as forcing the class-under-test to throw exceptions, increasing test suite diversity, and attaining Strong Mutation Coverage\u2014<jats:italic>do not<\/jats:italic> have effective fitness function formulations. We propose that meeting such goals requires treating fitness function identification as a secondary optimization step. An <jats:italic>adaptive<\/jats:italic> algorithm that can vary the selection of fitness functions could adjust its selection throughout the generation process to maximize goal attainment, based on the current population of test suites. To test this hypothesis, we have implemented two reinforcement learning algorithms in the EvoSuite unit test generation framework, and used these algorithms to dynamically set the fitness functions used during generation for the three goals identified above. We have evaluated our framework, EvoSuiteFIT, on a set of Java case examples. EvoSuiteFIT techniques attain significant improvements for two of the three goals, and show limited improvements on the third when the number of generations of evolution is fixed. Additionally, for two of the three goals, EvoSuiteFIT detects faults missed by the other techniques. The ability to adjust fitness functions allows strategic choices that efficiently produce more effective test suites, and examining these choices offers insight into how to attain our testing goals. We find that adaptive fitness function selection is a powerful technique to apply when an effective fitness function does not already exist for achieving a testing goal.<\/jats:p>","DOI":"10.1007\/s10664-021-10048-8","type":"journal-article","created":{"date-parts":[[2022,1,11]],"date-time":"2022-01-11T09:02:57Z","timestamp":1641891777000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":26,"title":["Learning how to search: generating effective test cases through adaptive fitness function selection"],"prefix":"10.1007","volume":"27","author":[{"given":"Hussein","family":"Almulla","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6794-9585","authenticated-orcid":false,"given":"Gregory","family":"Gay","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,1,11]]},"reference":[{"key":"10048_CR1","doi-asserted-by":"crossref","unstructured":"Ahmed BS, Enoiu E, Afzal W, Zamli KZ (2020) An evaluation of monte carlo-based hyper-heuristic for interaction testing of industrial embedded software applications. Soft Comput. https:\/\/doi.org\/10.1007\/s00500-020-04769-z","DOI":"10.1007\/s00500-020-04769-z"},{"key":"10048_CR2","doi-asserted-by":"crossref","unstructured":"Albunian NM Menzies T, Petke J (eds) (2017) Diversity in search-based unit test suite generation. Springer International Publishing, Cham","DOI":"10.1007\/978-3-319-66299-2_17"},{"issue":"6","key":"10048_CR3","doi-asserted-by":"publisher","first-page":"742","DOI":"10.1109\/TSE.2009.52","volume":"36","author":"S Ali","year":"2010","unstructured":"Ali S, Briand LC, Hemmati H, Panesar-Walawege RK (2010) A systematic review of the application and empirical investigation of search-based test case generation. IEEE Trans Softw Eng 36(6):742\u2013762","journal-title":"IEEE Trans Softw Eng"},{"key":"10048_CR4","doi-asserted-by":"crossref","unstructured":"Almasi MM, Hemmati H, Fraser G, Arcuri A, Benefelds J (2017) An industrial evaluation of unit test generation: Finding real faults in a financial application. In: Proceedings of the 39th IEEE\/ACM international conference on software engineering (ICSE)\u2014software engineering in practice track (SEIP), ICSE 2017. ACM, New York","DOI":"10.1109\/ICSE-SEIP.2017.27"},{"key":"10048_CR5","doi-asserted-by":"crossref","unstructured":"Almulla H, Gay G (2020) Learning how to search: Generating exception-triggering tests through adaptive fitness function selection. In: 13th IEEE international conference on software testing, validation and verification","DOI":"10.1109\/ICST46399.2020.00017"},{"key":"10048_CR6","doi-asserted-by":"crossref","unstructured":"Almulla H, Gay G (2020) Generating diverse test suites for Gson through adaptive fitness function selection. In: Proceedings of the symposium on search-based software engineering, SSBSE 2020. Springer","DOI":"10.1007\/978-3-030-59762-7_18"},{"key":"10048_CR7","doi-asserted-by":"publisher","unstructured":"Alshahwan N, Harman M (2014) Coverage and fault detection of the output-uniqueness test selection criteria. In: Proceedings of the 2014 international symposium on software testing and analysis, ISSTA 2014. https:\/\/doi.org\/10.1145\/2610384.2610413. ACM, New York, pp 181\u2013192","DOI":"10.1145\/2610384.2610413"},{"issue":"8","key":"10048_CR8","doi-asserted-by":"publisher","first-page":"1978","DOI":"10.1016\/j.jss.2013.02.061","volume":"86","author":"S Anand","year":"2013","unstructured":"Anand S, Burke E, Chen TY, Clark J, Cohen MB, Grieskamp W, Harman M, Harrold MJ, McMinn P (2013) An orchestrated survey of methodologies for automated software test case generation. J Syst Softw 86(8):1978\u20132001","journal-title":"J Syst Softw"},{"issue":"2","key":"10048_CR9","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1002\/stvr.457","volume":"23","author":"A Arcuri","year":"2013","unstructured":"Arcuri A (2013) It really does matter how you normalize the branch distance in search-based software testing. Softw Test Verification Reliab 23(2):119\u2013147","journal-title":"Softw Test Verification Reliab"},{"key":"10048_CR10","doi-asserted-by":"publisher","first-page":"176","DOI":"10.1016\/j.infsof.2019.06.012","volume":"114","author":"JM Balera","year":"2019","unstructured":"Balera JM, de Santiago J\u00fanior VA (2019) A systematic mapping addressing hyper-heuristics within search-based software testing. Inf Softw Technol 114:176\u2013189","journal-title":"Inf Softw Technol"},{"issue":"5","key":"10048_CR11","doi-asserted-by":"publisher","first-page":"507","DOI":"10.1109\/TSE.2014.2372785","volume":"41","author":"E Barr","year":"2015","unstructured":"Barr E, Harman M, McMinn P, Shahbaz M, Yoo S (2015) The oracle problem in software testing: A survey. IEEE Trans Softw Eng 41 (5):507\u2013525. https:\/\/doi.org\/10.1109\/TSE.2014.2372785","journal-title":"IEEE Trans Softw Eng"},{"key":"10048_CR12","unstructured":"Bauersfeld S, Vos T (2012) A reinforcement learning approach to automated gui robustness testing. In: Fast abstracts of the 4th symposium on search-based software engineering (SSBSE 2012). pp 7\u201312"},{"key":"10048_CR13","doi-asserted-by":"publisher","unstructured":"Bauersfeld S, Vos TEJ (2012) Guitest: a java library for fully automated gui robustness testing. In: 2012 Proceedings of the 27th IEEE\/ACM international conference on automated software engineering. pp 330\u2013333. https:\/\/doi.org\/10.1145\/2351676.2351739","DOI":"10.1145\/2351676.2351739"},{"issue":"2","key":"10048_CR14","doi-asserted-by":"publisher","first-page":"239","DOI":"10.1007\/s11047-008-9098-4","volume":"8","author":"L Bianchi","year":"2009","unstructured":"Bianchi L, Dorigo M, Gambardella L, Gutjahr W (2009) A survey on metaheuristics for stochastic combinatorial optimization. Nat Comput 8 (2):239\u2013287. https:\/\/doi.org\/10.1007\/s11047-008-9098-4","journal-title":"Nat Comput"},{"key":"10048_CR15","doi-asserted-by":"publisher","unstructured":"Blasi A, Goffi A, Kuznetsov K, Gorla A, Ernst MD, Pezz\u00e8 M, Castellanos SD (2018) Translating code comments to procedure specifications. In: Proceedings of the 27th ACM SIGSOFT international symposium on software testing and analysis, ISSTA 2018. https:\/\/doi.org\/10.1145\/3213846.3213872. Association for Computing Machinery, New York, pp 242\u2013253","DOI":"10.1145\/3213846.3213872"},{"key":"10048_CR16","first-page":"453","volume-title":"A classification of hyper-heuristic approaches: revisited","author":"E Burke","year":"2019","unstructured":"Burke E, Hyde MR, Kendall G, Ochoa G, \u00d6zcan E., Woodward JR (2019) A classification of hyper-heuristic approaches: revisited. Springer International Publishing, Cham, pp 453\u2013477. https:\/\/doi.org\/10.1007\/978-3-319-91086-4_14"},{"key":"10048_CR17","doi-asserted-by":"publisher","unstructured":"Bu\u015foniu L., Ernst D, De Schutter B, Babuska\u030c R. (2011) Approximate reinforcement learning: An overview. In: 2011 IEEE symposium on adaptive dynamic programming and reinforcement learning (ADPRL). pp 1\u20138. https:\/\/doi.org\/10.1109\/ADPRL.2011.5967353","DOI":"10.1109\/ADPRL.2011.5967353"},{"issue":"5","key":"10048_CR18","doi-asserted-by":"publisher","first-page":"1690","DOI":"10.1016\/j.eswa.2012.09.013","volume":"40","author":"B Crawford","year":"2013","unstructured":"Crawford B, Soto R, Monfroy E, Palma W, Castro C, Paredes F (2013) Parameter tuning of a choice-function based hyperheuristic using particle swarm optimization. Expert Syst Appl 40(5):1690\u20131695. https:\/\/doi.org\/10.1016\/j.eswa.2012.09.013. http:\/\/www.sciencedirect.com\/science\/article\/pii\/S0957417412010676http:\/\/www.sciencedirect.com\/science\/article\/pii\/S0957417412010676","journal-title":"Expert Syst Appl"},{"key":"10048_CR19","doi-asserted-by":"crossref","unstructured":"De Oliveira Neto FG, Feldt R, Erlenhov L, Nunes JBDS (2018) Visualizing test diversity to support test optimisation. In: 2018 25th Asia-pacific software engineering conference (APSEC), pp. 149\u2013158","DOI":"10.1109\/APSEC.2018.00029"},{"key":"10048_CR20","doi-asserted-by":"crossref","unstructured":"Din F, Alsewari ARA, Zamli KZ (2017) A parameter free choice function based hyper-heuristic strategy for pairwise test generation. In: 2017 IEEE international conference on software quality, reliability and security companion (QRS-C). pp 85\u201391","DOI":"10.1109\/QRS-C.2017.22"},{"issue":"10","key":"10048_CR21","doi-asserted-by":"publisher","first-page":"7333","DOI":"10.1166\/asl.2018.12938","volume":"24","author":"F Din","year":"2018","unstructured":"Din F, Zamli KZ (2018) Hyper-heuristic based strategy for pairwise test case generation. Adv Sci Lett 24 (10):7333\u20137338. https:\/\/doi.org\/10.1166\/asl.2018.12938, https:\/\/www.ingentaconnect.com\/content\/asp\/asl\/2018\/00000024\/00000010\/art00068","journal-title":"Adv Sci Lett"},{"issue":"1","key":"10048_CR22","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1109\/4235.585892","volume":"1","author":"M Dorigo","year":"1997","unstructured":"Dorigo M, Gambardella LM (1997) Ant colony system: a cooperative learning approach to the traveling salesman problem. IEEE Trans Evol Comput 1 (1):53\u201366","journal-title":"IEEE Trans Evol Comput"},{"issue":"2","key":"10048_CR23","doi-asserted-by":"publisher","first-page":"405","DOI":"10.1016\/j.ejor.2019.07.073","volume":"285","author":"JH Drake","year":"2020","unstructured":"Drake JH, Kheiri A, \u00d6zcan E, Burke E (2020) Recent advances in selection hyper-heuristics. Eur J Oper Res 285(2):405\u2013428","journal-title":"Eur J Oper Res"},{"key":"10048_CR24","doi-asserted-by":"publisher","unstructured":"Feldt R, Poulding S (2015) Broadening the search in search-based software testing: It need not be evolutionary. In: 2015 IEEE\/ACM 8th international workshop on Search-based software testing (SBST). pp 1\u20137. https:\/\/doi.org\/10.1109\/SBST.2015.8","DOI":"10.1109\/SBST.2015.8"},{"key":"10048_CR25","doi-asserted-by":"crossref","unstructured":"Feldt R, Poulding S, Clark D, Yoo S (2016) Test set diameter: Quantifying the diversity of sets of test cases. In: 2016 IEEE international conference on software testing, verification and validation (ICST). pp 223\u2013233","DOI":"10.1109\/ICST.2016.33"},{"issue":"2","key":"10048_CR26","doi-asserted-by":"publisher","first-page":"34","DOI":"10.1109\/MCI.2017.2670461","volume":"12","author":"TN Ferreira","year":"2017","unstructured":"Ferreira TN, Lima JAP, Strickler A, Kuk JN, Vergilio SR, Pozo A (2017) Hyper-heuristic based product selection for software product line testing. IEEE Comput Intell Mag 12(2):34\u201345","journal-title":"IEEE Comput Intell Mag"},{"key":"10048_CR27","doi-asserted-by":"publisher","unstructured":"Filho HLJ, Ferreira TN, Vergilio SR (2018) Multiple objective test set selection for software product line testing: Evaluating different preference-based algorithms. In: Proceedings of the XXXII brazilian symposium on software engineering, SBES \u201918. https:\/\/doi.org\/10.1145\/3266237.3266275. Association for Computing Machinery, New York, pp 162\u2013171","DOI":"10.1145\/3266237.3266275"},{"key":"10048_CR28","doi-asserted-by":"publisher","unstructured":"Filho HLJ, Lima JAP, Vergilio SR (2017) Automatic generation of search-based algorithms applied to the feature testing of software product lines. In: Proceedings of the 31st Brazilian symposium on software engineering, SBES\u201917. https:\/\/doi.org\/10.1145\/3131151.3131152. Association for Computing Machinery, New York, pp 114\u2013123","DOI":"10.1145\/3131151.3131152"},{"key":"10048_CR29","doi-asserted-by":"crossref","unstructured":"Fraser G (2018) A tutorial on using and extending the evosuite search-based test generator. In: Search-based software engineering, Springer, pp 106\u2013130","DOI":"10.1007\/978-3-319-99241-9_5"},{"issue":"3","key":"10048_CR30","doi-asserted-by":"publisher","first-page":"783","DOI":"10.1007\/s10664-013-9299-z","volume":"20","author":"G Fraser","year":"2014","unstructured":"Fraser G, Arcuri A (2014) Achieving scalable mutation-based generation of whole test suites. Empir Softw Eng 20(3):783\u2013812","journal-title":"Empir Softw Eng"},{"key":"10048_CR31","doi-asserted-by":"crossref","unstructured":"Fraser G, Arcuri A (2017) Evosuite at the sbst 2017 tool competition. In: 10th international workshop on search-based software testing (SBST\u201917) at ICSE\u201917. pp 39\u201342","DOI":"10.1109\/SBST.2017.6"},{"key":"10048_CR32","doi-asserted-by":"publisher","unstructured":"Fraser G, Staats M, McMinn P, Arcuri A, Padberg F (2013) Does automated white-box test generation really help software testers?. In: Proceedings of the 2013 international symposium on software testing and analysis, ISSTA. https:\/\/doi.org\/10.1145\/2483760.2483774. ACM, New York, pp 291\u2013301","DOI":"10.1145\/2483760.2483774"},{"key":"10048_CR33","unstructured":"Gay G, Whalen MW, Heimdahl MP, Staats M (2014) The risks of coverage directed test case generation. Currently under submission, draft available from http:\/\/greggay.com\/pdf\/14risks.pdf"},{"key":"10048_CR34","doi-asserted-by":"crossref","unstructured":"Gay G (2017) The fitness function for the job: Search-based generation of test suites that detect real faults. In: Proceedings of the international conference on software testing, ICST 2017. IEEE","DOI":"10.1109\/ICST.2017.38"},{"key":"10048_CR35","doi-asserted-by":"crossref","unstructured":"Gay G (2017) Generating effective test suites by combining coverage criteria. In: Proceedings of the symposium on search-based software engineering, SSBSE 2017. Springer","DOI":"10.1007\/978-3-319-66299-2_5"},{"key":"10048_CR36","doi-asserted-by":"crossref","unstructured":"Gay G (2018) To call, or not to call: Contrasting direct and indirect branch coverage in test generation. In: Proceedings of the 11th international workshop on search-based software testing, SBST 2018. ACM, New York","DOI":"10.1145\/3194718.3194719"},{"issue":"Y","key":"10048_CR37","first-page":"1","volume":"X","author":"G Gay","year":"2018","unstructured":"Gay G (2018) Choosing the fitness function for the job: Automated generation of test suites that detect real faults. Under revision. J Softw Test Verif Reliab X(Y):1\u201320. Draft available from http:\/\/greggay.com\/pdf\/18fitness.pdf","journal-title":"J Softw Test Verif Reliab"},{"key":"10048_CR38","doi-asserted-by":"publisher","unstructured":"Gay G, Staats M, Whalen M, Heimdahl M (2015) The risks of coverage-directed test case generation. IEEE Trans Softw Eng PP(99). https:\/\/doi.org\/10.1109\/TSE.2015.2421011","DOI":"10.1109\/TSE.2015.2421011"},{"key":"10048_CR39","doi-asserted-by":"publisher","unstructured":"Goffi A, Gorla A, Ernst MD, Pezz\u00e8 M (2016) Automatic generation of oracles for exceptional behaviors. In: Proceedings of the 25th international symposium on software testing and analysis, ISSTA 2016. https:\/\/doi.org\/10.1145\/2931037.2931061. Association for Computing Machinery, New York, pp 213\u2013224","DOI":"10.1145\/2931037.2931061"},{"key":"10048_CR40","doi-asserted-by":"crossref","unstructured":"Grechanik M, Fu C, Xie Q (2012) Automatically finding performance problems with feedback-directed learning software testing. In: 2012 34th international conference on software engineering (ICSE). pp 156\u2013166","DOI":"10.1109\/ICSE.2012.6227197"},{"key":"10048_CR41","doi-asserted-by":"crossref","unstructured":"Guizzo G, Bazargani M, Paix\u00e3o M, Drake JH (2017) A hyper-heuristic for multi-objective integration and test ordering in google guava. In: SSBSE","DOI":"10.1007\/978-3-319-66299-2_15"},{"key":"10048_CR42","doi-asserted-by":"publisher","unstructured":"Guizzo G, Fritsche GM, Vergilio SR, Pozo ATR (2015) A hyper-heuristic for the multi-objective integration and test order problem. In: Proceedings of the 2015 annual conference on genetic and evolutionary computation, GECCO \u201915. https:\/\/doi.org\/10.1145\/2739480.2754725. ACM, New York, pp 1343\u20131350","DOI":"10.1145\/2739480.2754725"},{"key":"10048_CR43","doi-asserted-by":"crossref","unstructured":"Guizzo G, Vergilio SR (2018) A pattern-driven solution for designing multi-objective evolutionary algorithms. Nat Comput 1\u201314","DOI":"10.1007\/s11047-018-9677-y"},{"key":"10048_CR44","doi-asserted-by":"publisher","unstructured":"Guizzo G, Vergilio SR, Pozo ATR (2015) Evaluating a multi-objective hyper-heuristic for the integration and test order problem. In: 2015 Brazilian conference on intelligent Systems (BRACIS). https:\/\/doi.org\/10.1109\/BRACIS.2015.11, pp 1\u20136","DOI":"10.1109\/BRACIS.2015.11"},{"issue":"C","key":"10048_CR45","doi-asserted-by":"publisher","first-page":"331","DOI":"10.1016\/j.asoc.2017.03.012","volume":"56","author":"G Guizzo","year":"2017","unstructured":"Guizzo G, Vergilio SR, Pozo A, Fritsche GM (2017) A multi-objective and evolutionary hyper-heuristic applied to the integration and test order problem. Appl Soft Comput 56(C):331\u2013344. https:\/\/doi.org\/10.1016\/j.asoc.2017.03.012","journal-title":"Appl Soft Comput"},{"key":"10048_CR46","doi-asserted-by":"publisher","unstructured":"Harman M, Jia Y, Langdon WB (2011) Strong higher order mutation-based test data generation. In: Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on foundations of software engineering, ESEC\/FSE \u201911. https:\/\/doi.org\/10.1145\/2025113.2025144. Association for Computing Machinery, New York, pp 212\u2013222","DOI":"10.1145\/2025113.2025144"},{"key":"10048_CR47","doi-asserted-by":"publisher","first-page":"833","DOI":"10.1016\/S0950-5849(01)00189-6","volume":"43","author":"M Harman","year":"2001","unstructured":"Harman M, Jones B (2001) Search-based software engineering. J Inf Softw Technol 43:833\u2013839","journal-title":"J Inf Softw Technol"},{"key":"10048_CR48","doi-asserted-by":"crossref","unstructured":"Helali Moghadam M, Saadatmand M, Borg M, Bohlin M, Lisper B (2019) Machine learning to guide performance testing: An autonomous test framework. In: 2019 IEEE international conference on software testing, verification and validation workshops (ICSTW), pp 164\u2013167","DOI":"10.1109\/ICSTW.2019.00046"},{"key":"10048_CR49","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/1090.001.0001","volume-title":"Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence","author":"JH Holland","year":"1992","unstructured":"Holland JH (1992) Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. MIT press, Cambridge"},{"key":"10048_CR50","unstructured":"Iqbal MS, Kotthoff L, Jamshidi P (2019) Transfer learning for performance modeling of deep neural network systems. In: 2019 {USENIX} conference on operational machine learning (OpML 19). pp 43\u201346"},{"key":"10048_CR51","doi-asserted-by":"crossref","unstructured":"Jia Y (2015) Hyperheuristic search for sbst. In: Proceedings of the Eighth international workshop on search-based software testing, SBST \u201915. http:\/\/dl.acm.org\/citation.cfm?id=2821339.2821343. IEEE Press, Piscataway, pp 15\u201316","DOI":"10.1109\/SBST.2015.10"},{"key":"10048_CR52","doi-asserted-by":"crossref","unstructured":"Jia Y, Cohen MB, Harman M, Petke J (2015) Learning combinatorial interaction test generation strategies using hyperheuristic search. In: Proceedings of the 37th international conference on software engineering - volume 1, ICSE \u201915. IEEE Press, Piscataway, pp. 540\u2013550. http:\/\/dl.acm.org\/citation.cfm?id=2818754.2818821","DOI":"10.1109\/ICSE.2015.71"},{"key":"10048_CR53","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2019.00029","volume-title":"Directing a search towards execution properties with a learned fitness function","author":"L Joffe","year":"2019","unstructured":"Joffe L, Clark D (2019) Directing a search towards execution properties with a learned fitness function. IEEE, Washington"},{"key":"10048_CR54","doi-asserted-by":"publisher","unstructured":"Just R, Jalali D, Ernst MD (2014) Defects4J: A database of existing faults to enable controlled testing studies for Java programs. In: Proceedings of the 2014 international symposium on software testing and analysis, ISSTA 2014. https:\/\/doi.org\/10.1145\/2610384.2628055. ACM, New York, pp 437\u2013440","DOI":"10.1145\/2610384.2628055"},{"issue":"2","key":"10048_CR55","doi-asserted-by":"publisher","first-page":"262","DOI":"10.1287\/moor.12.2.262","volume":"12","author":"MN Katehakis","year":"1987","unstructured":"Katehakis MN, Veinott AF Jr (1987) The multi-armed bandit problem: decomposition and computation. Math Oper Res 12(2):262\u2013268","journal-title":"Math Oper Res"},{"key":"10048_CR56","doi-asserted-by":"publisher","first-page":"384","DOI":"10.1016\/j.jss.2016.04.007","volume":"117","author":"AC Kumari","year":"2016","unstructured":"Kumari AC, Srinivas K (2016) Hyper-heuristic approach for multi-objective software module clustering. J Syst Softw 117:384\u2013401. https:\/\/doi.org\/10.1016\/j.jss.2016.04.007. http:\/\/www.sciencedirect.com\/science\/article\/pii\/S0164121216300231","journal-title":"J Syst Softw"},{"key":"10048_CR57","doi-asserted-by":"crossref","unstructured":"Lindstrom B, Mrki A (2016) On strong mutation and subsuming mutants. In: 2016 IEEE Ninth international conference on software testing, verification and validation workshops (ICSTW). pp 112\u2013121","DOI":"10.1109\/ICSTW.2016.28"},{"key":"10048_CR58","doi-asserted-by":"crossref","unstructured":"Luiz Jakubovski Filho H, Nascimento Ferreira T, Regina Vergilio S (2018) Incorporating user preferences in a software product line testing hyper-heuristic approach. In: 2018 IEEE congress on evolutionary computation (CEC), pp. 1\u20138","DOI":"10.1109\/CEC.2018.8477803"},{"key":"10048_CR59","doi-asserted-by":"publisher","first-page":"162","DOI":"10.1016\/j.infsof.2018.07.001","volume":"103","author":"L Ma","year":"2018","unstructured":"Ma L, Wu P, Chen TY (2018) Diversity driven adaptive test generation for concurrent data structures. Inf Softw Technol 103:162\u2013173. http:\/\/www.sciencedirect.com\/science\/article\/pii\/S0950584918301356","journal-title":"Inf Softw Technol"},{"key":"10048_CR60","doi-asserted-by":"publisher","unstructured":"Malburg J, Fraser G (2011) Combining search-based and constraint-based testing. In: Proceedings of the 2011 26th IEEE\/ACM international conference on automated software engineering, ASE \u201911. https:\/\/doi.org\/10.1109\/ASE.2011.6100092. IEEE Computer Society, Washington, pp 436\u2013439","DOI":"10.1109\/ASE.2011.6100092"},{"key":"10048_CR61","doi-asserted-by":"publisher","unstructured":"Mariani T, Guizzo G, Vergilio SR, Pozo A (2016) Grammatical evolution for the multi-objective integration and test order problem. In: Proceedings of the genetic and evolutionary computation conference 2016, GECCO \u201916. https:\/\/doi.org\/10.1145\/2908812.2908816. Association for Computing Machinery, New York, pp 1069\u20131076","DOI":"10.1145\/2908812.2908816"},{"key":"10048_CR62","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1002\/stvr.294","volume":"14","author":"P McMinn","year":"2004","unstructured":"McMinn P (2004) Search-based software test data generation: A survey. Softw Test Verification Reliab 14:105\u2013156","journal-title":"Softw Test Verification Reliab"},{"key":"10048_CR63","doi-asserted-by":"crossref","unstructured":"do Nascimento Ferreira T, Kuk JN, Pozo A, Vergilio SR (2016) Product selection based on upper confidence bound moea\/d-dra for testing software product lines. In: 2016 IEEE congress on evolutionary computation (CEC). pp 4135\u20134142","DOI":"10.1109\/CEC.2016.7744315"},{"issue":"1","key":"10048_CR64","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1145\/375360.375365","volume":"33","author":"G Navarro","year":"2001","unstructured":"Navarro G (2001) A guided tour to approximate string matching. ACM Comput Surv 33(1):31\u201388. https:\/\/doi.org\/10.1145\/375360.375365","journal-title":"ACM Comput Surv"},{"key":"10048_CR65","doi-asserted-by":"publisher","unstructured":"Ochoa G, Vazquez-Rodriguez JA, Petrovic S, Burke E (2009) Dispatching rules for production scheduling: A hyper-heuristic landscape analysis. In: 2009 IEEE congress on evolutionary computation. pp 1873\u20131880. https:\/\/doi.org\/10.1109\/CEC.2009.4983169","DOI":"10.1109\/CEC.2009.4983169"},{"key":"10048_CR66","doi-asserted-by":"publisher","unstructured":"Orso A, Rothermel G (2014) Software testing: A research travelogue (2000\u20132014). In: Proceedings of the on future of software engineering, FOSE 2014. https:\/\/doi.org\/10.1145\/2593882.2593885. ACM, New York, pp 117\u2013132","DOI":"10.1145\/2593882.2593885"},{"issue":"1","key":"10048_CR67","doi-asserted-by":"publisher","first-page":"121","DOI":"10.1186\/2193-1801-2-121","volume":"2","author":"M Papadakis","year":"2013","unstructured":"Papadakis M, Malevris N (2013) Searching and generating test inputs for mutation testing. SpringerPlus 2(1):121. https:\/\/doi.org\/10.1186\/2193-1801-2-121","journal-title":"SpringerPlus"},{"key":"10048_CR68","volume-title":"Software Test and Analysis, Process, Principles, and Techniques","author":"M Pezze","year":"2006","unstructured":"Pezze M, Young M (2006) Software Test and Analysis, Process, Principles, and Techniques. Wiley, Hoboken"},{"key":"10048_CR69","doi-asserted-by":"publisher","unstructured":"Robillard MP, Murphy GC (2000) Designing robust java programs with exceptions. In: Proceedings of the 8th ACM SIGSOFT international symposium on foundations of software engineering: twenty-first century applications, SIGSOFT \u201900\/FSE-8. https:\/\/doi.org\/10.1145\/355045.355046. ACM, New York, pp 2\u201310","DOI":"10.1145\/355045.355046"},{"key":"10048_CR70","doi-asserted-by":"publisher","unstructured":"Rojas JM, Campos J, Vivanti M, Fraser G, Arcuri A (2015) Combining multiple coverage criteria in search-based unit test generation. In: Barros M, Labiche Y (eds) Search-based software engineering, lecture notes in computer science, vol. 9275. Springer International Publishing, pp 93\u2013108. https:\/\/doi.org\/10.1007\/978-3-319-22183-0_7","DOI":"10.1007\/978-3-319-22183-0_7"},{"issue":"2","key":"10048_CR71","doi-asserted-by":"publisher","first-page":"852","DOI":"10.1007\/s10664-015-9424-2","volume":"22","author":"JM Rojas","year":"2017","unstructured":"Rojas JM, Vivanti M, Arcuri A, Fraser G (2017) A detailed investigation of the effectiveness of whole test suite generation. Empir Softw Eng 22 (2):852\u2013893. https:\/\/doi.org\/10.1007\/s10664-015-9424-2","journal-title":"Empir Softw Eng"},{"key":"10048_CR72","doi-asserted-by":"crossref","unstructured":"Romano D, Penta MD, Antoniol G (2011) An approach for search based testing of null pointer exceptions. In: 2011 Fourth IEEE international conference on software testing, verification and validation, pp 160\u2013169","DOI":"10.1109\/ICST.2011.49"},{"issue":"4-5","key":"10048_CR73","doi-asserted-by":"publisher","first-page":"e1701","DOI":"10.1002\/stvr.1701","volume":"29","author":"A Salahirad","year":"2019","unstructured":"Salahirad A, Almulla H, Gay G (2019) Choosing the fitness function for the job: Automated generation of test suites that detect real faults. Softw Test Verification Reliab 29(4-5):e1701. https:\/\/doi.org\/10.1002\/stvr.1701","journal-title":"Softw Test Verification Reliab"},{"key":"10048_CR74","unstructured":"Shahbazi A (2015) Diversity-based automated test case generation. Ph.D. thesis, University of Alberta"},{"key":"10048_CR75","doi-asserted-by":"crossref","unstructured":"Shamshiri S, Just R, Rojas JM, Fraser G, McMinn P, Arcuri A (2015) Do automatically generated unit tests find real faults? an empirical study of effectiveness and challenges. In: Proceedings of the 30th IEEE\/ACM international conference on automated software engineering (ASE), ASE 2015. ACM, New York","DOI":"10.1109\/ASE.2015.86"},{"key":"10048_CR76","doi-asserted-by":"crossref","unstructured":"Souza FCM, Papadakis M, Le Traon Y, Delamaro ME (2016) Strong mutation-based test data generation using hill climbing. In: Proceedings of the 9th international workshop on search-based software testing. ACM, pp 45\u201354","DOI":"10.1145\/2897010.2897012"},{"key":"10048_CR77","doi-asserted-by":"publisher","first-page":"1232","DOI":"10.1016\/j.asoc.2016.07.059","volume":"49","author":"A Strickler","year":"2016","unstructured":"Strickler A, Prado Lima JA, Vergilio SR, Pozo A (2016) Deriving products for variability test of feature models with a hyper-heuristic approach. Appl Soft Comput 49:1232\u20131242. https:\/\/doi.org\/10.1016\/j.asoc.2016.07.059. http:\/\/www.sciencedirect.com\/science\/article\/pii\/S1568494616303994","journal-title":"Appl Soft Comput"},{"key":"10048_CR78","volume-title":"Reinforcement learning: an introduction, vol 1","author":"RS Sutton","year":"1998","unstructured":"Sutton RS, Barto AG (1998) Reinforcement learning: an introduction, vol 1. MIT press, Cambridge"},{"key":"10048_CR79","unstructured":"Sutton RS, Barto AG (2018) Reinforcement learning: An introduction, MIT press, Cambridge"},{"issue":"2","key":"10048_CR80","doi-asserted-by":"publisher","first-page":"101","DOI":"10.3102\/10769986025002101","volume":"25","author":"A Vargha","year":"2000","unstructured":"Vargha A, Delaney HD (2000) A critique and improvement of the cl common language effect size statistics of mcgraw and wong. J Educ Behav Stat 25 (2):101\u2013132. https:\/\/doi.org\/10.3102\/10769986025002101","journal-title":"J Educ Behav Stat"},{"key":"10048_CR81","doi-asserted-by":"crossref","unstructured":"Vogel T, Tran C, Grunske L (2019) Does diversity improve the test suite generation for mobile applications?. In: International symposium on search based software engineering. Springer, pp 58\u201374","DOI":"10.1007\/978-3-030-27455-9_5"},{"key":"10048_CR82","doi-asserted-by":"publisher","unstructured":"Xu D, Shrestha R, Shen N (2020) Automated strong mutation testing of xacml policies. In: Proceedings of the 25th ACM symposium on access control models and technologies, SACMAT \u201920. https:\/\/doi.org\/10.1145\/3381991.3395599. Association for Computing Machinery, New York, pp 105\u2013116","DOI":"10.1145\/3381991.3395599"},{"issue":"C","key":"10048_CR83","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1016\/j.asoc.2016.03.021","volume":"44","author":"KZ Zamli","year":"2016","unstructured":"Zamli KZ, Alkazemi BY, Kendall G (2016) A tabu search hyper-heuristic strategy for t-way test suite generation. Appl Soft Comput 44(C):57\u201374. https:\/\/doi.org\/10.1016\/j.asoc.2016.03.021","journal-title":"Appl Soft Comput"},{"key":"10048_CR84","doi-asserted-by":"publisher","first-page":"121","DOI":"10.1016\/j.ins.2017.03.007","volume":"399","author":"KZ Zamli","year":"2017","unstructured":"Zamli KZ, Din F, Kendall G, Ahmed BS (2017) An experimental study of hyper-heuristic selection and acceptance mechanism for combinatorial t-way test suite generation. Inf Sci 399:121\u2013153. https:\/\/doi.org\/10.1016\/j.ins.2017.03.007. http:\/\/www.sciencedirect.com\/science\/article\/pii\/S0020025517305820","journal-title":"Inf Sci"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-021-10048-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-021-10048-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-021-10048-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,4,12]],"date-time":"2022-04-12T13:23:58Z","timestamp":1649769838000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-021-10048-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,11]]},"references-count":84,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2022,3]]}},"alternative-id":["10048"],"URL":"https:\/\/doi.org\/10.1007\/s10664-021-10048-8","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1,11]]},"assertion":[{"value":"1 September 2021","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 January 2022","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"38"}}