{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,25]],"date-time":"2026-02-25T18:11:11Z","timestamp":1772043071253,"version":"3.50.1"},"reference-count":75,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2021,9,28]],"date-time":"2021-09-28T00:00:00Z","timestamp":1632787200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["61872008 and 62002256"],"award-info":[{"award-number":["61872008 and 62002256"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2022,1,31]]},"abstract":"<jats:p>Although regression testing is important to guarantee the software quality in software evolution, it suffers from the widely known cost problem. To address this problem, existing researchers made dedicated efforts on test prioritization, which optimizes the execution order of tests to detect faults earlier; while practitioners in industry leveraged more computing resources to save the time cost of regression testing. By combining these two orthogonal solutions, in this article, we define the problem of parallel test prioritization, which is to conduct test prioritization in the scenario of parallel test execution to reduce the cost of regression testing.<\/jats:p>\n          <jats:p>Different from traditional sequential test prioritization, parallel test prioritization aims at generating a set of test sequences, each of which is allocated in an individual computing resource and executed in parallel. In particular, we propose eight parallel test prioritization techniques by adapting the existing four sequential test prioritization techniques, by including and excluding testing time in prioritization.<\/jats:p>\n          <jats:p>\n            To investigate the performance of the eight parallel test prioritization techniques, we conducted an extensive study on 54 open-source projects and a case study on 16 commercial projects from\n            <jats:italic>Baidu<\/jats:italic>\n            , a famous search service provider with 600M monthly active users. According to the two studies, parallel test prioritization does improve the efficiency of regression testing, and cost-aware additional parallel test prioritization technique significantly outperforms the other techniques, indicating that this technique is a good choice for practical parallel testing. Besides, we also investigated the influence of two external factors, the number of computing resources and time allowed for parallel testing, and find that more computing resources indeed improve the performance of parallel test prioritization. In addition, we investigated the influence of two more factors, test granularity and coverage criterion, and find that parallel test prioritization can still accelerate regression testing in parallel scenario. Moreover, we investigated the benefit of parallel test prioritization on the regression testing process of continuous integration, considering both the cumulative acceleration performance and the overhead of prioritization techniques, and the results demonstrate the superiority of parallel test prioritization.\n          <\/jats:p>","DOI":"10.1145\/3471906","type":"journal-article","created":{"date-parts":[[2021,9,28]],"date-time":"2021-09-28T20:49:24Z","timestamp":1632862164000},"page":"1-50","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["Parallel Test Prioritization"],"prefix":"10.1145","volume":"31","author":[{"given":"Jianyi","family":"Zhou","sequence":"first","affiliation":[{"name":"Key Laboratory of High Confidence Software Technologies (Peking University), Ministry of Education, Beijing, China"}]},{"given":"Junjie","family":"Chen","sequence":"additional","affiliation":[{"name":"College of Intelligence and Computing, Tianjin University, Tianjin, China"}]},{"given":"Dan","family":"Hao","sequence":"additional","affiliation":[{"name":"Key Laboratory of High Confidence Software Technologies (Peking University), Ministry of Education, Beijing, China"}]}],"member":"320","published-online":{"date-parts":[[2021,9,28]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Proceedings of the 27th International Conference on Software Engineering. ACM, 402\u2013411","author":"Andrews James H.","year":"2005","unstructured":"James H. Andrews , Lionel C. Briand , and Yvan Labiche . 2005 . Is mutation an appropriate tool for testing experiments? In Proceedings of the 27th International Conference on Software Engineering. ACM, 402\u2013411 . James H. Andrews, Lionel C. Briand, and Yvan Labiche. 2005. Is mutation an appropriate tool for testing experiments? In Proceedings of the 27th International Conference on Software Engineering. ACM, 402\u2013411."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11219-012-9181-z"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183440.3183456"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2017.70"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2016.19"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2016.19"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2017.45"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3236024.3236053"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2018.2889771"},{"key":"e_1_2_1_10_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3394112","article-title":"Practical accuracy estimation for efficient deep neural network testing","volume":"29","author":"Chen Junjie","year":"2020","unstructured":"Junjie Chen , Zhuo Wu , Zan Wang , Hanmo You , Lingming Zhang , and Ming Yan . 2020 . Practical accuracy estimation for efficient deep neural network testing . ACM Transactions on Software Engineering and Methodology 29 , 4 (2020), 1 \u2013 35 . Junjie Chen, Zhuo Wu, Zan Wang, Hanmo You, Lingming Zhang, and Ming Yan. 2020. Practical accuracy estimation for efficient deep neural network testing. ACM Transactions on Software Engineering and Methodology 29, 4 (2020), 1\u201335.","journal-title":"ACM Transactions on Software Engineering and Methodology"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2009.4"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2010.58"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2006.92"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.5555\/381473.381508"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2635868.2635910"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2009.07.001"},{"key":"e_1_2_1_17_1","volume-title":"Proceedings of the 2015 International Symposium on Software Testing and Analysis. ACM, 234\u2013245","author":"Epitropakis Michael G.","unstructured":"Michael G. Epitropakis , Shin Yoo , Mark Harman , and Edmund K. Burke . 2015. Empirical evaluation of Pareto efficient multi-objective regression test case prioritisation . In Proceedings of the 2015 International Symposium on Software Testing and Analysis. ACM, 234\u2013245 . Michael G. Epitropakis, Shin Yoo, Mark Harman, and Edmund K. Burke. 2015. Empirical evaluation of Pareto efficient multi-objective regression test case prioritisation. In Proceedings of the 2015 International Symposium on Software Testing and Analysis. ACM, 234\u2013245."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2889160.2889222"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1037\/a0024338"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1002\/j.1538-7305.1966.tb01709.x"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.5555\/302163.302182"},{"key":"e_1_2_1_22_1","unstructured":"Pooja Gupta Mark Ivey and John Penix. 2011. Testing at the speed and scale of Google. Google Engineering Tools Blog. http:\/\/google-engtools.blogspot.com\/2011\/06\/testing-at-speed-and-scale-of-google.html.  Pooja Gupta Mark Ivey and John Penix. 2011. Testing at the speed and scale of Google. Google Engineering Tools Blog. http:\/\/google-engtools.blogspot.com\/2011\/06\/testing-at-speed-and-scale-of-google.html."},{"key":"e_1_2_1_23_1","volume-title":"Proceedings of the 31st International Conference on Very Large Data Bases. Klemens B\u00f6hm, Christian S. Jensen, Laura M. Haas, Martin L. Kersten, Per-\u00c5ke Larson, and Beng Chin Ooi (Eds.), ACM, 589\u2013600","author":"Haftmann Florian","year":"2005","unstructured":"Florian Haftmann , Donald Kossmann , and Eric Lo . 2005 . Parallel execution of test runs for database application systems . In Proceedings of the 31st International Conference on Very Large Data Bases. Klemens B\u00f6hm, Christian S. Jensen, Laura M. Haas, Martin L. Kersten, Per-\u00c5ke Larson, and Beng Chin Ooi (Eds.), ACM, 589\u2013600 . Florian Haftmann, Donald Kossmann, and Eric Lo. 2005. Parallel execution of test runs for database application systems. In Proceedings of the 31st International Conference on Very Large Data Bases. Klemens B\u00f6hm, Christian S. Jensen, Laura M. Haas, Martin L. Kersten, Per-\u00c5ke Larson, and Beng Chin Ooi (Eds.), ACM, 589\u2013600."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-006-0028-8"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11704-016-6112-3"},{"key":"e_1_2_1_26_1","volume-title":"IEEE Transactions on Software Engineering 42, 5","author":"Hao Dan","year":"2016","unstructured":"Dan Hao , Lu Zhang , Lei Zang , Yanbo Wang , Xingxia Wu , and Tao Xie . 2016. IEEE Transactions on Software Engineering 42, 5 ( 2016 ), 490\u2013505. Dan Hao, Lu Zhang, Lei Zang, Yanbo Wang, Xingxia Wu, and Tao Xie. 2016. IEEE Transactions on Software Engineering 42, 5 (2016), 490\u2013505."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2685614"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884791"},{"key":"e_1_2_1_29_1","volume-title":"C","author":"Jiang Bo","year":"2015","unstructured":"Bo Jiang and Wing Kwong Chan . 2015. Input-based adaptive randomized test case prioritization: A local beam search approach. Journal of Systems and Software 105 , C ( 2015 ), 91\u2013106. Bo Jiang and Wing Kwong Chan. 2015. Input-based adaptive randomized test case prioritization: A local beam search approach. Journal of Systems and Software 105, C (2015), 91\u2013106."},{"key":"e_1_2_1_30_1","volume-title":"Proceedings of the 2009 IEEE\/ACM International Conference on Automated Software Engineering. IEEE Computer Society, 233\u2013244","author":"Jiang Bo","unstructured":"Bo Jiang , Zhenyu Zhang , Wing Kwong Chan , and T. H. Tse . 2009. Adaptive random test case prioritization . In Proceedings of the 2009 IEEE\/ACM International Conference on Automated Software Engineering. IEEE Computer Society, 233\u2013244 . Bo Jiang, Zhenyu Zhang, Wing Kwong Chan, and T. H. Tse. 2009. Adaptive random test case prioritization. In Proceedings of the 2009 IEEE\/ACM International Conference on Automated Software Engineering. IEEE Computer Society, 233\u2013244."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2635868.2635929"},{"key":"e_1_2_1_32_1","unstructured":"Cem Kaner. 1997. Improving the maintainability of automated test suites. International Software Quality Week. http:\/\/www.kaner.com\/pdfs\/autosqa.pdf.  Cem Kaner. 1997. Improving the maintainability of automated test suites. International Software Quality Week. http:\/\/www.kaner.com\/pdfs\/autosqa.pdf."},{"key":"e_1_2_1_33_1","volume-title":"Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering. ACM, 257\u2013267","author":"Peter Kim Chang Hwan","year":"2013","unstructured":"Chang Hwan Peter Kim , Darko Marinov , Sarfraz Khurshid , Don Batory , Sabrina Souto , Paulo Barros , and Marcelo d' Amorim . 2013 . SPLat: Lightweight dynamic analysis for reducing combinatorics in testing configurable systems . In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering. ACM, 257\u2013267 . Chang Hwan Peter Kim, Darko Marinov, Sarfraz Khurshid, Don Batory, Sabrina Souto, Paulo Barros, and Marcelo d'Amorim. 2013. SPLat: Lightweight dynamic analysis for reducing combinatorics in testing configurable systems. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering. ACM, 257\u2013267."},{"key":"e_1_2_1_34_1","volume-title":"Int. Softw. Develop. Conf. Presentation slides are available at: qconsf.com\/sf2010\/. [Online]. Available: Presentation slides are available at: https:\/\/qconsf.com\/sf2010\/.","author":"Kumar Ashish","year":"2010","unstructured":"Ashish Kumar . 2010 . Development at the speed and scale of Google . Int. Softw. Develop. Conf. Presentation slides are available at: qconsf.com\/sf2010\/. [Online]. Available: Presentation slides are available at: https:\/\/qconsf.com\/sf2010\/. Ashish Kumar. 2010. Development at the speed and scale of Google. Int. Softw. Develop. Conf. Presentation slides are available at: qconsf.com\/sf2010\/. [Online]. Available: Presentation slides are available at: https:\/\/qconsf.com\/sf2010\/."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/APSEC.2014.12"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2019.00038"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF01585745"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-39742-4_10"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2007.38"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180213"},{"key":"e_1_2_1_41_1","volume-title":"Advances in Computers.","author":"Lou Yiling","unstructured":"Yiling Lou , Junjie Chen , Lingming Zhang , and Dan Hao . 2019. A survey on regression test-case prioritization . In Advances in Computers. Vol. 113 , Atif M. Memon (Eds.). Elsevier , 1\u201346. Yiling Lou, Junjie Chen, Lingming Zhang, and Dan Hao. 2019. A survey on regression test-case prioritization. In Advances in Computers. Vol. 113, Atif M. Memon (Eds.). Elsevier, 1\u201346."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2015.7381798"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884874"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/2635868.2635920"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2018.2822270"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2018.2822270"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2011.106"},{"key":"e_1_2_1_49_1","volume-title":"Tools for continuous integration at Google scale. Google Tech Talk","author":"Micco J.","unstructured":"J. Micco . 2012. Tools for continuous integration at Google scale. Google Tech Talk , Google Inc. Presentation video is available at: https:\/\/www.youtube.com\/watch?v=KH2_sB1A6lA. J. Micco. 2012. Tools for continuous integration at Google scale. Google Tech Talk, Google Inc. Presentation video is available at: https:\/\/www.youtube.com\/watch?v=KH2_sB1A6lA."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180210"},{"key":"e_1_2_1_51_1","volume-title":"Proceedings of the 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT International Symposium on Foundations of Software Engineering","author":"Misailovic Sasa","unstructured":"Sasa Misailovic , Aleksandar Milicevic , Nemanja Petrovic , Sarfraz Khurshid , and Darko Marinov . 2007. Parallel test generation and execution with Korat . In Proceedings of the 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT International Symposium on Foundations of Software Engineering . Ivica Crnkovic and Antonia Bertolino (Eds.), ACM , 135\u2013144. Sasa Misailovic, Aleksandar Milicevic, Nemanja Petrovic, Sarfraz Khurshid, and Darko Marinov. 2007. Parallel test generation and execution with Korat. In Proceedings of the 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT International Symposium on Foundations of Software Engineering. Ivica Crnkovic and Antonia Bertolino (Eds.), ACM, 135\u2013144."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2015.7381799"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2018.2868082"},{"key":"e_1_2_1_54_1","volume-title":"Communications in Computer and Information Science","author":"Parashar Prem","unstructured":"Prem Parashar , Arvind Kalia , and Rajesh Bhatia . 2012. Pair-wise time-aware test case prioritization for regression testing . In Information Systems, Technology and Management. Sumeet Dua, Aryya Gangopadhyay, Parimala Thulasiraman, Umberto Straccia, Michael A. Shepherd, and Benno Stein (Eds.), Communications in Computer and Information Science , Vol. 285 . Springer , 176\u2013186. Prem Parashar, Arvind Kalia, and Rajesh Bhatia. 2012. Pair-wise time-aware test case prioritization for regression testing. In Information Systems, Technology and Management. Sumeet Dua, Aryya Gangopadhyay, Parimala Thulasiraman, Umberto Straccia, Michael A. Shepherd, and Benno Stein (Eds.), Communications in Computer and Information Science, Vol. 285. Springer, 176\u2013186."},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.536955"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/248233.248262"},{"key":"e_1_2_1_57_1","volume-title":"Proceedings IEEE International Conference on Software Maintenance. IEEE, 179\u2013188","author":"Rothermel Gregg","unstructured":"Gregg Rothermel , Roland H. Untch , Chengyun Chu, and Mary Jean Harrold. 1999. Test case prioritization: An empirical study . In Proceedings IEEE International Conference on Software Maintenance. IEEE, 179\u2013188 . Gregg Rothermel, Roland H. Untch, Chengyun Chu, and Mary Jean Harrold. 1999. Test case prioritization: An empirical study. In Proceedings IEEE International Conference on Software Maintenance. IEEE, 179\u2013188."},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.962562"},{"key":"e_1_2_1_59_1","volume-title":"37th IEEE\/ACM International Conference on Software Engineering. Antonia Bertolino, Gerardo Canfora, and Sebastian G. Elbaum (Eds.)","volume":"1","author":"Saha Ripon K.","unstructured":"Ripon K. Saha , Lingming Zhang , Sarfraz Khurshid , and Dewayne E. Perry . 2015. An information retrieval approach for regression test prioritization based on program changes . In 37th IEEE\/ACM International Conference on Software Engineering. Antonia Bertolino, Gerardo Canfora, and Sebastian G. Elbaum (Eds.) , Vol. 1 , IEEE Computer Society, 268\u2013279. Ripon K. Saha, Lingming Zhang, Sarfraz Khurshid, and Dewayne E. Perry. 2015. An information retrieval approach for regression test prioritization based on program changes. In 37th IEEE\/ACM International Conference on Software Engineering. Antonia Bertolino, Gerardo Canfora, and Sebastian G. Elbaum (Eds.), Vol. 1, IEEE Computer Society, 268\u2013279."},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.2307\/2333709"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/3293882.3330568"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/2635868.2635921"},{"key":"e_1_2_1_63_1","volume-title":"Proceedings of the 30th IEEE International Symposium on Software Reliability Engineering","author":"Shi August","unstructured":"August Shi , Peiyuan Zhao , and Darko Marinov . 2019. Understanding and improving regression test selection in continuous integration . In Proceedings of the 30th IEEE International Symposium on Software Reliability Engineering . Katinka Wolter, Ina Schieferdecker, Barbara Gallina, Michel Cukier, Roberto Natella, Naghmeh Ramezani Ivaki, and Nuno Laranjeiro (Eds.), IEEE , 228\u2013238. August Shi, Peiyuan Zhao, and Darko Marinov. 2019. Understanding and improving regression test selection in continuous integration. In Proceedings of the 30th IEEE International Symposium on Software Reliability Engineering. Katinka Wolter, Ina Schieferdecker, Barbara Gallina, Michel Cukier, Roberto Natella, Naghmeh Ramezani Ivaki, and Nuno Laranjeiro (Eds.), IEEE, 228\u2013238."},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/3213846.3213852"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-012-9219-7"},{"key":"e_1_2_1_66_1","first-page":"19","article-title":"The need to report effect size estimates revisited. An overview of some recommended measures of effect size","volume":"1","author":"Tomczak Maciej","year":"2014","unstructured":"Maciej Tomczak and Tomczak Ewa . 2014 . The need to report effect size estimates revisited. An overview of some recommended measures of effect size . Trends in Sport Sciences 1 , 21 (2014), 19 \u2013 25 . Maciej Tomczak and Tomczak Ewa. 2014. The need to report effect size estimates revisited. An overview of some recommended measures of effect size. Trends in Sport Sciences 1, 21 (2014), 19\u201325.","journal-title":"Trends in Sport Sciences"},{"key":"e_1_2_1_67_1","volume-title":"Proceedings of the ACM\/SIGSOFT International Symposium on Software Testing and Analysis.","author":"Walcott Kristen R.","unstructured":"Kristen R. Walcott , Mary Lou Soffa , Gregory M. Kapfhammer , and Robert S. Roos . 2006. Time-aware test suite prioritization . In Proceedings of the ACM\/SIGSOFT International Symposium on Software Testing and Analysis. Kristen R. Walcott, Mary Lou Soffa, Gregory M. Kapfhammer, and Robert S. Roos. 2006. Time-aware test suite prioritization. In Proceedings of the ACM\/SIGSOFT International Symposium on Software Testing and Analysis."},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/3106237.3106258"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00046"},{"key":"e_1_2_1_70_1","volume-title":"Breakthroughs in Statistics","author":"Wilcoxon Frank","unstructured":"Frank Wilcoxon . 1992. Individual comparisons by ranking methods . In Breakthroughs in Statistics . S. Kotz and N. L. Johnson (Eds.), Springer Series in Statistics (Perspectives in Statistics), Springer , 196\u2013202. Frank Wilcoxon. 1992. Individual comparisons by ranking methods. In Breakthroughs in Statistics. S. Kotz and N. L. Johnson (Eds.), Springer Series in Statistics (Perspectives in Statistics), Springer, 196\u2013202."},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1002\/stv.430"},{"key":"e_1_2_1_72_1","volume-title":"Proceedings of the 8th European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering.","author":"Yoo Shin","year":"2011","unstructured":"Shin Yoo , Robert Nilsson , and Mark Harman . 2011 . Faster fault finding at Google using multi objective regression test optimisation . In Proceedings of the 8th European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering. Shin Yoo, Robert Nilsson, and Mark Harman. 2011. Faster fault finding at Google using multi objective regression test optimisation. In Proceedings of the 8th European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering."},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2018.2809496"},{"key":"e_1_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.1145\/1572272.1572297"},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1145\/1572272.1572297"},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSTW.2017.74"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3471906","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3471906","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:24:49Z","timestamp":1750195489000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3471906"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,28]]},"references-count":75,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,1,31]]}},"alternative-id":["10.1145\/3471906"],"URL":"https:\/\/doi.org\/10.1145\/3471906","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,28]]},"assertion":[{"value":"2020-09-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-06-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-09-28","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}