{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,1]],"date-time":"2026-11-01T00:40:59Z","timestamp":1793493659030,"version":"4.5.0"},"reference-count":99,"publisher":"Springer Science and Business Media LLC","issue":"7","license":[{"start":{"date-parts":[[2022,10,1]],"date-time":"2022-10-01T00:00:00Z","timestamp":1664582400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,10,1]],"date-time":"2022-10-01T00:00:00Z","timestamp":1664582400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001711","name":"Schweizerischer Nationalfonds zur F\u00f6rderung der Wissenschaftlichen Forschung","doi-asserted-by":"publisher","award":["PZ00P2\u02d9186090"],"award-info":[{"award-number":["PZ00P2\u02d9186090"]}],"id":[{"id":"10.13039\/501100001711","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003407","name":"Ministero dell\u2019Istruzione, dell\u2019Universit\u00e0 e della Ricerca","doi-asserted-by":"publisher","award":["2020W3A5FY"],"award-info":[{"award-number":["2020W3A5FY"]}],"id":[{"id":"10.13039\/501100003407","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100007065","name":"Universit\u00e0 degli Studi di Salerno","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100007065","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2022,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Test flakiness is a phenomenon occurring when a test case is non-deterministic and exhibits both a passing and failing behavior when run against the same code. Over the last years, the problem has been closely investigated by researchers and practitioners, who all have shown its relevance in practice. The software engineering research community has been working toward defining approaches for detecting and addressing test flakiness. Despite being quite accurate, most of these approaches rely on expensive dynamic steps, e.g., the computation of code coverage information. Consequently, they might suffer from scalability issues that possibly preclude their practical use. This limitation has been recently targeted through machine learning solutions that could predict the flakiness of tests using various features, like source code vocabulary or a mixture of static and dynamic metrics computed on individual snapshots of the system. In this paper, we aim to perform a step forward and predict test flakiness <jats:italic>only using static metrics<\/jats:italic>. We propose a large-scale experiment on 70 Java projects coming from the <jats:sc>iDFlakies<\/jats:sc> and <jats:sc>FlakeFlagger<\/jats:sc> datasets. First, we statistically assess the differences between flaky and non-flaky tests in terms of 25 test and production code metrics and smells, analyzing both their individual and combined effects. Based on the results achieved, we experiment with a machine learning approach that predicts test flakiness solely based on static features, comparing it with two state-of-the-art approaches. The key results of the study show that the static approach has performance comparable to those of the baselines. In addition, we found that the characteristics of the production code might impact the performance of the flaky test prediction models.<\/jats:p>","DOI":"10.1007\/s10664-022-10227-1","type":"journal-article","created":{"date-parts":[[2022,10,1]],"date-time":"2022-10-01T08:02:32Z","timestamp":1664611352000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["Static test flakiness prediction: How Far Can We Go?"],"prefix":"10.1007","volume":"27","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6012-9947","authenticated-orcid":false,"given":"Valeria","family":"Pontillo","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fabio","family":"Palomba","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Filomena","family":"Ferrucci","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,10,1]]},"reference":[{"key":"10227_CR1","doi-asserted-by":"crossref","unstructured":"Alshammari A, Morris C, Hilton M, Bell J (2021) Flakeflagger: Predicting flakiness without rerunning tests. In: ICSE 2021, IEEE, pp 1572\u20131584","DOI":"10.1109\/ICSE43902.2021.00140"},{"key":"10227_CR2","unstructured":"Association IS (1998) 829-1998 IEEE standard for software test documentation, Tech. rep., Technical report"},{"key":"10227_CR3","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1016\/j.infsof.2018.12.009","volume":"108","author":"MI Azeem","year":"2019","unstructured":"Azeem MI, Palomba F, Shi L, Wang Q (2019) Machine learning techniques for code smell detection: a systematic literature review and meta-analysis. Inf Softw Technol 108:115\u2013138","journal-title":"Inf Softw Technol"},{"issue":"2","key":"10227_CR4","first-page":"18","volume":"2","author":"B Azhagusundari","year":"2013","unstructured":"Azhagusundari B, Thanamani AS, et al. (2013) Feature selection based on information gain. Int J Innov Technol Exploring Eng (IJITEE) 2(2):18\u201321","journal-title":"Int J Innov Technol Exploring Eng (IJITEE)"},{"key":"10227_CR5","doi-asserted-by":"crossref","unstructured":"Banko M, Brill E (2001) Scaling to very very large corpora for natural language disambiguation. In: Proceedings of the 39th annual meeting of the association for computational linguistics, pp 26\u201333","DOI":"10.3115\/1073012.1073017"},{"key":"10227_CR6","doi-asserted-by":"crossref","unstructured":"Bell J, Kaiser G, Melski E, Dattatreya M (2015) Efficient dependency detection for safe java test acceleration. In: Proceedings of the 2015 10th joint meeting on foundations of software engineering, pp 770\u2013781","DOI":"10.1145\/2786805.2786823"},{"key":"10227_CR7","unstructured":"Bell J, Legunsen O, Hilton M, Eloussi L, Yung T, Marinov D (2018) Deflaker: automatically detecting flaky tests. In: ICSE 2018, IEEE, pp 433\u2013444"},{"key":"10227_CR8","first-page":"1089","volume":"5","author":"Y Bengio","year":"2004","unstructured":"Bengio Y, Grandvalet Y (2004) No unbiased estimator of the variance of k-fold cross-validation. J Mach Learn Res 5:1089\u20131105","journal-title":"J Mach Learn Res"},{"issue":"2","key":"10227_CR9","first-page":"281","volume":"13","author":"J Bergstra","year":"2012","unstructured":"Bergstra J, Bengio Y (2012) Random search for hyper-parameter optimization. J Mach Learn Res 13(2):281\u2013305","journal-title":"J Mach Learn Res"},{"key":"10227_CR10","doi-asserted-by":"publisher","first-page":"76119","DOI":"10.1109\/ACCESS.2021.3082424","volume":"9","author":"A Bertolino","year":"2021","unstructured":"Bertolino A, Cruciani E, Miranda B, Verdecchia R (2021) Know your neighbor: fast static prediction of test flakiness. IEEE Access 9:76119\u201376134","journal-title":"IEEE Access"},{"key":"10227_CR11","doi-asserted-by":"crossref","unstructured":"Camara B, Silva M, Endo A, Vergilio S (2021) On the use of test smells for prediction of flaky tests. In: Brazilian symposium on systematic and automated software testing, pp 46\u201354","DOI":"10.1145\/3482909.3482916"},{"key":"10227_CR12","doi-asserted-by":"crossref","unstructured":"Camara B, Silva M, Endo A, Vergilio S (2021) What is the vocabulary of flaky tests? an extended replication. pp 444\u2013454","DOI":"10.1109\/ICPC52881.2021.00052"},{"key":"10227_CR13","doi-asserted-by":"crossref","unstructured":"Catolino G, Palomba F, Zaidman A, Ferrucci F (2019) How the experience of development teams relates to assertion density of test classes. In: ICSME 2019, IEEE, pp 223\u2013234","DOI":"10.1109\/ICSME.2019.00034"},{"key":"10227_CR14","doi-asserted-by":"publisher","first-page":"321","DOI":"10.1613\/jair.953","volume":"16","author":"NV Chawla","year":"2002","unstructured":"Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) Smote: synthetic minority over-sampling technique. J Artif Intell Res 16:321\u2013357","journal-title":"J Artif Intell Res"},{"issue":"6","key":"10227_CR15","doi-asserted-by":"publisher","first-page":"476","DOI":"10.1109\/32.295895","volume":"20","author":"S Chidamber","year":"1994","unstructured":"Chidamber S, Kemerer C (1994) A metrics suite for object oriented design. IEEE TSE 20(6):476\u2013493. https:\/\/doi.org\/10.1109\/32.295895","journal-title":"IEEE TSE"},{"key":"10227_CR16","doi-asserted-by":"crossref","unstructured":"Cordy M, Rwemalika R, Franci A, Papadakis M, Harman M (2022) Flakime: laboratory-controlled test flakiness impact assessment","DOI":"10.1145\/3510003.3510194"},{"key":"10227_CR17","doi-asserted-by":"crossref","unstructured":"Daniel B, Jagannath V, Dig D, Marinov D (2009) Reassert: suggesting repairs for broken unit tests. In: ASE 2009, IEEE, pp 433\u2013444","DOI":"10.1109\/ASE.2009.17"},{"key":"10227_CR18","doi-asserted-by":"crossref","unstructured":"de Almeida Filho FG, Martins ADF, da Silva Vinuto T, Monteiro JM, de Sousa \u00cdP, de Castro Machado J, Rocha LS (2019) Prevalence of bad smells in pl\/sql projects. In: 2019 IEEE\/ACM 27Th international conference on program comprehension (ICPC), IEEE, pp 116\u2013121","DOI":"10.1109\/ICPC.2019.00025"},{"issue":"1","key":"10227_CR19","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1109\/TSE.2018.2880977","volume":"47","author":"EV de Paulo Sobrinho","year":"2018","unstructured":"de Paulo Sobrinho EV, De Lucia A, de Almeida Maia M (2018) A systematic literature review on bad smells\u20145 w\u2019s: which, when, what, who, where. IEEE Trans Softw Eng 47(1):17\u201366","journal-title":"IEEE Trans Softw Eng"},{"key":"10227_CR20","doi-asserted-by":"publisher","first-page":"47","DOI":"10.1007\/s11831-021-09566-x","volume":"29","author":"JP dos Reis","year":"2021","unstructured":"dos Reis JP, e Abreu FB, de Figueiredo Carneiro G, Anslow C (2021) Code smells detection and visualization: a systematic literature review. Arch Comput Methods Eng 29:47\u201394","journal-title":"Arch Comput Methods Eng"},{"key":"10227_CR21","doi-asserted-by":"crossref","unstructured":"Dutta S, Shi A, Choudhary R, Zhang Z, Jain A, Misailovic S (2020) Detecting flaky tests in probabilistic and machine learning applications. In: Proceedings of the 29th ACM SIGSOFT international symposium on software testing and analysis, pp 211\u2013224","DOI":"10.1145\/3395363.3397366"},{"key":"10227_CR22","doi-asserted-by":"crossref","unstructured":"Eck M, Palomba F, Castelluccio M, Bacchelli A (2019) Understanding flaky tests: the developer\u2019s perspective. In: ESEC\/FSE 2019, pp 830\u2013840","DOI":"10.1145\/3338906.3338945"},{"key":"10227_CR23","unstructured":"Fowler M (2011) Eradicating non-determinism in tests. Martin Fowler Personal Blog. https:\/\/martinfowler.com\/articles\/nonDeterminism.html"},{"key":"10227_CR24","unstructured":"Fowler M (2018) Refactoring: improving the design of existing code. Addison-Wesley Professional"},{"key":"10227_CR25","unstructured":"Freund Y, Mason L (1999) The alternating decision tree learning algorithm. In: Icml, vol 99. Citeseer, pp 124\u2013133"},{"key":"10227_CR26","unstructured":"Garson G (2012) Testing statistical assumptions. Asheboro NC: Statistical Associates Publishing"},{"key":"10227_CR27","doi-asserted-by":"crossref","unstructured":"Grano G, De Iaco C, Palomba F, Gall H (2020) Pizza versus pinsa: on the perception and measurability of unit test code quality. In: ICSME 2020, IEEE, pp 336\u2013347","DOI":"10.1109\/ICSME46990.2020.00040"},{"key":"10227_CR28","unstructured":"Grano G, Palomba F, Gall H (2019) Lightweight assessment of test-case effectiveness using source-code-quality indicators IEEE TSE"},{"key":"10227_CR29","doi-asserted-by":"crossref","unstructured":"Greiler M, Van Deursen A, Storey MA (2013) Automated detection of test fixture strategies and smells. In: 2013 IEEE sixth international conference on software testing, verification and validation, IEEE, pp 322\u2013331","DOI":"10.1109\/ICST.2013.45"},{"key":"10227_CR30","unstructured":"Gruber M, Fraser G (2022)"},{"key":"10227_CR31","doi-asserted-by":"crossref","unstructured":"Gruber M, Lukasczyk S, Kroi\u00df F, Fraser G (2021) An empirical study of flaky tests in python. In: 2021 14Th IEEE conference on software testing, verification and validation (ICST), IEEE, pp 148\u2013158","DOI":"10.1109\/ICST49551.2021.00026"},{"key":"10227_CR32","doi-asserted-by":"crossref","unstructured":"Gyori A, Shi A, Hariri F, Marinov D (2015) Reliable testing: detecting state-polluting tests to prevent test dependency. In: Proceedings of the 2015 international symposium on software testing and analysis, pp 223\u2013233","DOI":"10.1145\/2771783.2771793"},{"key":"10227_CR33","unstructured":"Habchi S, Haben G, Papadakis M, Cordy M, Traon YL (2021)"},{"key":"10227_CR34","doi-asserted-by":"crossref","unstructured":"Haben G, Habchi S, Papadakis M, Cordy M, Le Traon Y (2021) A replication study on the usability of code vocabulary in predicting flaky tests. In: MSR 2021","DOI":"10.1109\/MSR52588.2021.00034"},{"issue":"6","key":"10227_CR35","doi-asserted-by":"publisher","first-page":"1276","DOI":"10.1109\/TSE.2011.103","volume":"38","author":"T Hall","year":"2011","unstructured":"Hall T, Beecham S, Bowes D, Gray D, Counsell S (2011) A systematic literature review on fault prediction performance in software engineering. IEEE Trans Softw Eng 38(6):1276\u20131304","journal-title":"IEEE Trans Softw Eng"},{"key":"10227_CR36","doi-asserted-by":"crossref","unstructured":"Han H, Wang WY, Mao BH (2005) Borderline-smote: a new over-sampling method in imbalanced data sets learning. In: International conference on intelligent computing, Springer, pp 878\u2013887","DOI":"10.1007\/11538059_91"},{"issue":"4","key":"10227_CR37","first-page":"83","volume":"5","author":"J Han","year":"2011","unstructured":"Han J, Kamber M, Pei J (2011) Data mining concepts and techniques third edition. Morgan Kaufmann Ser Data Manag Syst 5(4):83\u2013124","journal-title":"Morgan Kaufmann Ser Data Manag Syst"},{"key":"10227_CR38","doi-asserted-by":"publisher","unstructured":"Harman M, O\u2019Hearn P (2018) From start-ups to scale-ups: Opportunities and open problems for static and dynamic program analysis. In: 2018 IEEE 18Th international working conference on source code analysis and manipulation (SCAM), pp 1\u201323. https:\/\/doi.org\/10.1109\/SCAM.2018.00009","DOI":"10.1109\/SCAM.2018.00009"},{"key":"10227_CR39","unstructured":"He H, Bai Y, Garcia EA, Li S (2008) Adasyn: adaptive synthetic sampling approach for imbalanced learning. In: 2008 IEEE international joint conference on neural networks (IEEE world congress on computational intelligence), IEEE, pp 1322\u20131328"},{"key":"10227_CR40","unstructured":"Ho TK (1995) Random decision forests. In: Proceedings of 3rd international conference on document analysis and recognition, vol 1. IEEE, pp 278\u2013282"},{"issue":"3","key":"10227_CR41","doi-asserted-by":"publisher","first-page":"243","DOI":"10.1007\/s10664-011-9171-y","volume":"17","author":"F Khomh","year":"2012","unstructured":"Khomh F, Penta MD, Gu\u00e9h\u00e9neuc YG, Antoniol G (2012) An exploratory study of the impact of antipatterns on class change-and fault-proneness. Empir Softw Eng 17(3):243\u2013275","journal-title":"Empir Softw Eng"},{"key":"10227_CR42","unstructured":"Kohavi R (1995) A study of cross-validation and bootstrap for accuracy estimation and model selection. In: Proceedings of the 14th international joint conference on artificial intelligence - volume 2, IJCAI\u201995. Morgan Kaufmann Publishers Inc., San Francisco, pp 1137\u20131143"},{"issue":"3","key":"10227_CR43","doi-asserted-by":"publisher","first-page":"397","DOI":"10.1016\/j.ijforecast.2004.10.003","volume":"21","author":"AJ Koning","year":"2005","unstructured":"Koning AJ, Franses PH, Hibon M, Stekler HO (2005) The m3 competition: Statistical tests of the results. Int J Forecast 21(3):397\u2013409","journal-title":"Int J Forecast"},{"key":"10227_CR44","doi-asserted-by":"crossref","unstructured":"Kramer O (2016) Scikit-learn. In: Machine learning for evolution strategies, Springer, pp 45\u201353","DOI":"10.1007\/978-3-319-33383-0_5"},{"key":"10227_CR45","doi-asserted-by":"crossref","unstructured":"Lacoste F (2009) Killing the gatekeeper: introducing a continuous integration system. In: 2009 Agile conference, IEEE, pp 387\u2013392","DOI":"10.1109\/AGILE.2009.35"},{"key":"10227_CR46","doi-asserted-by":"crossref","unstructured":"Lam W, Oei R, Shi A, Marinov D, Xie T (2019) Idflakies: a framework for detecting and partially classifying flaky tests. In: ICST 2019, IEEE, pp 312\u2013322","DOI":"10.1109\/ICST.2019.00038"},{"key":"10227_CR47","doi-asserted-by":"crossref","unstructured":"Lam W, Winter S, Astorga A, Stodden V, Marinov D (2020) Understanding reproducibility and characteristics of flaky tests through test reruns in java projects. In: ISSRE 2020, IEEE, pp 403\u2013413","DOI":"10.1109\/ISSRE5003.2020.00045"},{"issue":"OOPSLA","key":"10227_CR48","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3428270","volume":"4","author":"W Lam","year":"2020","unstructured":"Lam W, Winter S, Wei A, Xie T, Marinov D, Bell J (2020) A large-scale longitudinal study of flaky tests. Proc ACM Prog Lang 4(OOPSLA):1\u201329","journal-title":"Proc ACM Prog Lang"},{"key":"10227_CR49","doi-asserted-by":"crossref","unstructured":"Lambiase S, Cupito A, Pecorelli F, De Lucia A, Palomba F (2020) Just-in-time test smell detection and refactoring: the darts project. In: Proceedings of the 28th international conference on program comprehension, pp 441\u2013445","DOI":"10.1145\/3387904.3389296"},{"key":"10227_CR50","doi-asserted-by":"crossref","unstructured":"Lampel J, Just S, Apel S, Zeller A (2021) When life gives you oranges: detecting and diagnosing intermittent job failures at mozilla. In: 29Th ACM joint meeting on european software engineering conference and symposium on the foundations of software engineering, pp 1381\u20131392","DOI":"10.1145\/3468264.3473931"},{"key":"10227_CR51","doi-asserted-by":"crossref","unstructured":"Luo Q, Hariri F, Eloussi L, Marinov D (2014) An empirical analysis of flaky tests. In: ESEC\/FSE 2014, pp 643\u2013653","DOI":"10.1145\/2635868.2635920"},{"issue":"4","key":"10227_CR52","doi-asserted-by":"publisher","first-page":"308","DOI":"10.1109\/TSE.1976.233837","volume":"SE-2","author":"T McCabe","year":"1976","unstructured":"McCabe T (1976) A complexity measure. IEEE TSE SE-2(4):308\u2013320. https:\/\/doi.org\/10.1109\/TSE.1976.233837","journal-title":"IEEE TSE"},{"key":"10227_CR53","doi-asserted-by":"crossref","unstructured":"Memon A, Cohen M (2013) Automated testing of gui applications: models, tools, and controlling flakiness. In: ICSE 2013, IEEE, pp 1479\u20131480","DOI":"10.1109\/ICSE.2013.6606750"},{"key":"10227_CR54","unstructured":"Micco J (2017) The state of continuous integration testing@ google. https:\/\/research.google\/pubs\/pub45880\/"},{"issue":"1","key":"10227_CR55","first-page":"20","volume":"36","author":"N Moha","year":"2009","unstructured":"Moha N, Gu\u00e9h\u00e9neuc Y, Duchien L, Le Meur A (2009) Decor: a method for the specification and detection of code and design smells. IEEE TSE 36(1):20\u201336","journal-title":"IEEE TSE"},{"key":"10227_CR56","doi-asserted-by":"crossref","unstructured":"Murillo-Morera J, Jenkins M (2015) A software defect-proneness prediction framework: a new approach using genetic algorithms to generate learning schemes. In: SEKE, pp 445\u2013450","DOI":"10.18293\/SEKE2015-099"},{"key":"10227_CR57","doi-asserted-by":"crossref","unstructured":"Myers L, Sirois MJ (2004) Spearman correlation coefficients, differences between. Encycl Stat Sci :12","DOI":"10.1002\/0471667196.ess5050"},{"issue":"3","key":"10227_CR58","doi-asserted-by":"publisher","first-page":"370","DOI":"10.2307\/2344614","volume":"135","author":"J Nelder","year":"1972","unstructured":"Nelder J, Wedderburn R (1972) Generalized linear models. J R Stat Soc Ser A (Gen) 135(3):370\u2013384","journal-title":"J R Stat Soc Ser A (Gen)"},{"key":"10227_CR59","unstructured":"Nemenyi PB (1963) Distribution-free multiple comparisons Princeton University"},{"issue":"12","key":"10227_CR60","doi-asserted-by":"publisher","first-page":"1565","DOI":"10.1038\/nbt1206-1565","volume":"24","author":"WS Noble","year":"2006","unstructured":"Noble WS (2006) What is a support vector machine? Nat Biotechnol 24(12):1565\u20131567","journal-title":"Nat Biotechnol"},{"issue":"5","key":"10227_CR61","doi-asserted-by":"publisher","first-page":"673","DOI":"10.1007\/s11135-006-9018-6","volume":"41","author":"R O\u2019brien","year":"2007","unstructured":"O\u2019brien R (2007) A caution regarding rules of thumb for variance inflation factors. Qual Quant 41(5):673\u2013690","journal-title":"Qual Quant"},{"key":"10227_CR62","unstructured":"Palomba F (2019) Flaky tests: problems, solutions, and challenges. In: BENEVOL"},{"issue":"3","key":"10227_CR63","doi-asserted-by":"publisher","first-page":"1188","DOI":"10.1007\/s10664-017-9535-z","volume":"23","author":"F Palomba","year":"2018","unstructured":"Palomba F, Bavota G, Di Penta M, Fasano F, Oliveto R, De Lucia A (2018) On the diffuseness and the impact on maintainability of code smells: a large scale empirical investigation. Empir Softw Eng 23(3):1188\u20131221","journal-title":"Empir Softw Eng"},{"issue":"5","key":"10227_CR64","doi-asserted-by":"publisher","first-page":"462","DOI":"10.1109\/TSE.2014.2372760","volume":"41","author":"F Palomba","year":"2014","unstructured":"Palomba F, Bavota G, Di Penta M, Oliveto R, Poshyvanyk D, De Lucia A (2014) Mining version histories for detecting code smells. IEEE Trans Softw Eng 41(5):462\u2013489","journal-title":"IEEE Trans Softw Eng"},{"issue":"10","key":"10227_CR65","doi-asserted-by":"publisher","first-page":"977","DOI":"10.1109\/TSE.2017.2752171","volume":"44","author":"F Palomba","year":"2017","unstructured":"Palomba F, Panichella A, Zaidman A, Oliveto R, De Lucia A (2017) The scent of a smell: an extensive comparison between textual and structural smells. IEEE Trans Softw Eng 44(10):977\u20131000","journal-title":"IEEE Trans Softw Eng"},{"key":"10227_CR66","doi-asserted-by":"crossref","unstructured":"Palomba F, Zaidman A, De Lucia A (2018) Automatic test smell detection using information retrieval techniques. In: 2018 IEEE international conference on software maintenance and evolution (ICSME), IEEE, pp 311\u2013322","DOI":"10.1109\/ICSME.2018.00040"},{"issue":"1","key":"10227_CR67","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3476105","volume":"31","author":"O Parry","year":"2021","unstructured":"Parry O, Kapfhammer GM, Hilton M, McMinn P (2021) A survey of flaky tests. ACM Trans Softw Eng Methodol (TOSEM) 31(1):1\u201374","journal-title":"ACM Trans Softw Eng Methodol (TOSEM)"},{"issue":"2","key":"10227_CR68","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10664-021-10059-5","volume":"27","author":"F Pecorelli","year":"2022","unstructured":"Pecorelli F, Catolino G, Ferrucci F, De Lucia A, Palomba F (2022) Software testing and android applications: a large-scale empirical study. Empir Softw Eng 27(2):1\u201341","journal-title":"Empir Softw Eng"},{"key":"10227_CR69","doi-asserted-by":"crossref","unstructured":"Pecorelli F, Di Lillo G, Palomba F, De Lucia A (2020) Vitrum: a plug-in for the visualization of test-related metrics. In: AVI 2020, pp 1\u20133","DOI":"10.1145\/3399715.3399954"},{"key":"10227_CR70","doi-asserted-by":"crossref","unstructured":"Pecorelli F, Palomba F, De Lucia A (2021) The relation of test-related factors to software quality: a case study on apache systems, vol 26","DOI":"10.1007\/s10664-020-09891-y"},{"key":"10227_CR71","doi-asserted-by":"crossref","unstructured":"Pecorelli F, Palomba F, Di Nucci D, De Lucia A (2019) Comparing heuristic and machine learning approaches for metric-based code smell detection. In: 2019 IEEE\/ACM 27Th international conference on program comprehension (ICPC), IEEE, pp 93\u2013104","DOI":"10.1109\/ICPC.2019.00023"},{"key":"10227_CR72","doi-asserted-by":"crossref","unstructured":"Perez A, Abreu R, van Deursen A (2017) A test-suite diagnosability metric for spectrum-based fault localization approaches. In: ICSE 2017, IEEE, pp 654\u2013664","DOI":"10.1109\/ICSE.2017.66"},{"key":"10227_CR73","doi-asserted-by":"crossref","unstructured":"Peruma A, Almalki K, Newman CD, Mkaouer MW, Ouni A, Palomba F (2020) Tsdetect: an open source test smells detection tool. In: Proceedings of the 28th ACM joint meeting on european software engineering conference and symposium on the foundations of software engineering, pp 1650\u20131654","DOI":"10.1145\/3368089.3417921"},{"key":"10227_CR74","volume-title":"Software testing and analysis: process, principles, and techniques","author":"M Pezze","year":"2008","unstructured":"Pezze M, Young M (2008) Software testing and analysis: process, principles, and techniques. Wiley, New Jersey"},{"key":"10227_CR75","doi-asserted-by":"crossref","unstructured":"Pinto G, Miranda B, Dissanayake S, D\u2019Amorim M, Treude C, Bertolino A (2020) What is the vocabulary of flaky tests?. In: MSR 2020, pp 492\u2013502","DOI":"10.1145\/3379597.3387482"},{"key":"10227_CR76","doi-asserted-by":"publisher","unstructured":"Pontillo V, Palomba F, Ferrucci F (2021) Toward static test flakiness prediction: a feasibility study. In: MaLTESQuE 2021, Association for Computing Machinery, New York, NY, USA, pp 19\u201324. https:\/\/doi.org\/10.1145\/3472674.3473981","DOI":"10.1145\/3472674.3473981"},{"key":"10227_CR77","doi-asserted-by":"publisher","unstructured":"Pontillo V, Palomba F, Ferrucci F (2022) Static test flakiness prediction: how far can we go? - online appendix -. https:\/\/doi.org\/10.6084\/m9.figshare.17080946","DOI":"10.6084\/m9.figshare.17080946"},{"issue":"1","key":"10227_CR78","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1007\/BF00116251","volume":"1","author":"JR Quinlan","year":"1986","unstructured":"Quinlan JR (1986) Induction of decision trees. Mach Learn 1 (1):81\u2013106","journal-title":"Mach Learn"},{"issue":"11","key":"10227_CR79","doi-asserted-by":"publisher","first-page":"1167","DOI":"10.1002\/smr.1573","volume":"25","author":"A Qusef","year":"2013","unstructured":"Qusef A, Bavota G, Oliveto R, Lucia AD, Binkley D (2013) Evaluating test-to-code traceability recovery methods through controlled experiments. J Softw Evol Process 25(11):1167\u20131191","journal-title":"J Softw Evol Process"},{"key":"10227_CR80","doi-asserted-by":"crossref","unstructured":"Rehman MHU, Rigby PC (2021) Quantifying no-fault-found test failures to prioritize inspection of flaky tests at ericsson. In: 29Th ACM joint meeting on european software engineering conference and symposium on the foundations of software engineering, pp 1371\u20131380","DOI":"10.1145\/3468264.3473930"},{"key":"10227_CR81","doi-asserted-by":"crossref","unstructured":"Schapire RE (2013) Explaining adaboost. In: Empirical inference, Springer, pp 37\u201352","DOI":"10.1007\/978-3-642-41136-6_5"},{"key":"10227_CR82","doi-asserted-by":"crossref","unstructured":"Shabtai A, Elovici Y, Rokach L (2012) A survey of data leakage detection and prevention solutions. Springer Science & Business Media","DOI":"10.1007\/978-1-4614-2053-8"},{"issue":"3","key":"10227_CR83","doi-asserted-by":"publisher","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","volume":"27","author":"CE Shannon","year":"1948","unstructured":"Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27(3):379\u2013423","journal-title":"Bell Syst Tech J"},{"key":"10227_CR84","doi-asserted-by":"crossref","unstructured":"Shi A, Lam W, Oei R, Xie T, Marinov D (2019) Ifixflakies: a framework for automatically fixing order-dependent flaky tests. In: ESEC\/FSE 2019, pp 545\u2013555","DOI":"10.1145\/3338906.3338925"},{"key":"10227_CR85","doi-asserted-by":"crossref","unstructured":"Spadini D, Palomba F, Zaidman A, Bruntink M, Bacchelli A (2018) On the relation of test smells to software code quality. In: 2018 IEEE international conference on software maintenance and evolution (ICSME), IEEE, pp 1\u201312","DOI":"10.1109\/ICSME.2018.00010"},{"key":"10227_CR86","doi-asserted-by":"crossref","unstructured":"Taud H, Mas J (2018) Multilayer perceptron (mlp). In: Geomatic approaches for modeling land change scenarios, Springer, pp 451\u2013455","DOI":"10.1007\/978-3-319-60801-3_27"},{"key":"10227_CR87","doi-asserted-by":"crossref","unstructured":"Terragni V, Salza P, Ferrucci F (2020) A container-based infrastructure for fuzzy-driven root causing of flaky tests. In: ICSE 2020, pp 69\u201372","DOI":"10.1145\/3377816.3381742"},{"key":"10227_CR88","doi-asserted-by":"crossref","unstructured":"Thorve S, Sreshtha C, Meng N (2018) An empirical study of flaky tests in android apps. In: ICSME 2018, IEEE, pp 534\u2013538","DOI":"10.1109\/ICSME.2018.00062"},{"issue":"11","key":"10227_CR89","doi-asserted-by":"publisher","first-page":"1063","DOI":"10.1109\/TSE.2017.2653105","volume":"43","author":"M Tufano","year":"2017","unstructured":"Tufano M, Palomba F, Bavota G, Oliveto R, Di Penta M, De Lucia A, Poshyvanyk D (2017) When and why your code starts to smell bad (and whether the smells go away). IEEE Trans Softw Eng 43(11):1063\u20131088","journal-title":"IEEE Trans Softw Eng"},{"key":"10227_CR90","unstructured":"van Deursen A, Moonen L, Van Den Bergh A, Kok G (2001) Refactoring test code. In: XP 2001, Citeseer, pp 92\u201395"},{"issue":"12","key":"10227_CR91","doi-asserted-by":"publisher","first-page":"800","DOI":"10.1109\/TSE.2007.70745","volume":"33","author":"B Van Rompaey","year":"2007","unstructured":"Van Rompaey B, Du Bois B, Demeyer S, Rieger M (2007) On the detection of test smells: a metrics-based approach for general fixture and eager test. IEEE Trans Softw Eng 33(12):800\u2013817","journal-title":"IEEE Trans Softw Eng"},{"issue":"2","key":"10227_CR92","doi-asserted-by":"publisher","first-page":"1419","DOI":"10.1007\/s10664-019-09750-5","volume":"25","author":"C Vassallo","year":"2020","unstructured":"Vassallo C, Panichella S, Palomba F, Proksch S, Gall HC, Zaidman A (2020) How developers engage with static analysis tools in different contexts. Empir Softw Eng 25(2):1419\u20131457","journal-title":"Empir Softw Eng"},{"key":"10227_CR93","doi-asserted-by":"crossref","unstructured":"Vavrov\u00e1 N, Zaytsev V (2017) Does python smell like java? tool support for design defect discovery in python. arXiv:1703.10882","DOI":"10.22152\/programming-journal.org\/2017\/1\/11"},{"key":"10227_CR94","doi-asserted-by":"crossref","unstructured":"Wang T, Golubev Y, Smirnov O, Li J, Bryksin T, Ahmed I (2021) Pynose: a test smell detector for python. In: 2021 36Th IEEE\/ACM international conference on automated software engineering (ASE), IEEE, pp 593\u2013605","DOI":"10.1109\/ASE51524.2021.9678615"},{"key":"10227_CR95","first-page":"713","volume":"15","author":"GI Webb","year":"2010","unstructured":"Webb GI, Keogh E, Miikkulainen R (2010) Na\u00efve bayes. Encycl Mach Learn 15:713\u2013714","journal-title":"Encycl Mach Learn"},{"key":"10227_CR96","doi-asserted-by":"crossref","unstructured":"Wong WE, Horgan JR, London S, Agrawal H (1997) A study of effective regression testing in practice. In: PROCEEDINGS the eighth international symposium on software reliability engineering, IEEE, pp 264\u2013274","DOI":"10.1109\/ISSRE.1997.630875"},{"key":"10227_CR97","doi-asserted-by":"crossref","unstructured":"Yen SJ, Lee YS (2006) Under-sampling approaches for improving prediction of the minority class in an imbalanced dataset. In: Intelligent control and automation, Springer, pp 731\u2013740","DOI":"10.1007\/978-3-540-37256-1_89"},{"key":"10227_CR98","doi-asserted-by":"crossref","unstructured":"Zhang S, Jalali D, Wuttke J, Mus\u0307lu K, Lam W, Ernst M, Notkin D (2014) Empirically revisiting the test independence assumption. In: ISSTA 2014, pp 385\u2013396","DOI":"10.1145\/2610384.2610404"},{"key":"10227_CR99","doi-asserted-by":"crossref","unstructured":"Zheng W, Liu G, Zhang M, Chen X, Zhao W (2021) Research progress of flaky tests. In: 2021 IEEE international conference on software analysis, evolution and reengineering (SANER), IEEE, pp 639\u2013646","DOI":"10.1109\/SANER50967.2021.00081"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-022-10227-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-022-10227-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-022-10227-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,11,21]],"date-time":"2022-11-21T02:17:56Z","timestamp":1668997076000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-022-10227-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,1]]},"references-count":99,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2022,12]]}},"alternative-id":["10227"],"URL":"https:\/\/doi.org\/10.1007\/s10664-022-10227-1","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,10,1]]},"assertion":[{"value":"18 August 2022","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 October 2022","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"<!--Emphasis Type='Bold' removed-->Competing interests"}}],"article-number":"187"}}