{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,18]],"date-time":"2026-07-18T05:05:22Z","timestamp":1784351122761,"version":"3.55.0"},"reference-count":130,"publisher":"Association for Computing Machinery (ACM)","issue":"13s","license":[{"start":{"date-parts":[[2023,7,13]],"date-time":"2023-07-13T00:00:00Z","timestamp":1689206400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Comput. Surv."],"published-print":{"date-parts":[[2023,12,31]]},"abstract":"<jats:p>\n            <jats:bold>Context:<\/jats:bold>\n            Software regression testing refers to rerunning test cases after the system under test is modified, ascertaining that the changes have not (re-)introduced failures. Not all researchers\u2019 approaches consider applicability and scalability concerns, and not many have produced an impact in practice.\n            <jats:bold>Objective:<\/jats:bold>\n            One goal is to investigate industrial relevance and applicability of proposed approaches. Another is providing a live review, open to continuous updates by the community.\n            <jats:bold>Method:<\/jats:bold>\n            A systematic review of regression testing studies that are clearly motivated by or validated against industrial relevance and applicability is conducted. It is complemented by follow-up surveys with authors of the selected papers and 23 practitioners.\n            <jats:bold>Results:<\/jats:bold>\n            A set of 79 primary studies published between 2016\u20132022 is collected and classified according to approaches and metrics. Aspects relative to their relevance and impact are discussed, also based on their authors\u2019 feedback. All the data are made available from the live repository that accompanies the study.\n            <jats:bold>Conclusions:<\/jats:bold>\n            While widely motivated by industrial relevance and applicability, not many approaches are evaluated in industrial or large-scale open-source systems, and even fewer approaches have been adopted in practice. Some challenges hindering the implementation of relevant approaches are synthesized, also based on the practitioners\u2019 feedback.\n          <\/jats:p>","DOI":"10.1145\/3579851","type":"journal-article","created":{"date-parts":[[2023,1,12]],"date-time":"2023-01-12T15:39:10Z","timestamp":1673537950000},"page":"1-36","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":27,"title":["State of Practical Applicability of Regression Testing Research: A Live Systematic Literature Review"],"prefix":"10.1145","volume":"55","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0148-0662","authenticated-orcid":false,"given":"Renan","family":"Greca","sequence":"first","affiliation":[{"name":"Gran Sasso Science Institute, Italy and ISTI-CNR, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9608-9393","authenticated-orcid":false,"given":"Breno","family":"Miranda","sequence":"additional","affiliation":[{"name":"Universidade Federal de Pernambuco, Brazil"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8749-1356","authenticated-orcid":false,"given":"Antonia","family":"Bertolino","sequence":"additional","affiliation":[{"name":"ISTI-CNR, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,7,13]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSTW55395.2022.00034"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/3490700.3490710"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0264972"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/SEAA.2016.29"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2018.00025"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSTW.2017.6"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2021.3070549"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/CCAA.2018.8777692"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2938260"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380369"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-018-9670-1"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11219-016-9328-4"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/WICSA.2016.24"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/2950290.2983954"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/2372233.2372238"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11219-012-9181-z"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/3236024.3275527"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3106237.3106297"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3236024.3236053"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/APSEC53868.2021.00050"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.29007\/7z5n"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/MCI.2017.2670460"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1177\/001316446002000104"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/3338906.3341187"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00055"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2019.110398"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1016\/bs.adcom.2016.04.004"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-005-3861-2"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2014.04.002"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11219-019-09449-2"},{"key":"e_1_3_2_32_2","unstructured":"Sebastian Elbaum Andrew Mclaughlin and John Penix. 2014. The Google Dataset of Testing Results. Retrieved from https:\/\/code.google.com\/p\/google-shared-dataset-of-test-suite-results."},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3460319.3464834"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642- 13792-1_3"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2009.07.001"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1016\/bs.adcom.2015.11.003"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10009-015-0365-2"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2019.00039"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.i3507"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2017.3641116"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2018.06.007"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1609\/aimag.v38i1.2714"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.5220\/0007842502260237"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524481.3527223"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11390-019-1917-9"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2018.08.061"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11704-016-6112-3"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/FOSM.2008.4659253"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.3837\/tiis.2020.05.001"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.3390\/app11209709"},{"key":"e_1_3_2_51_2","unstructured":"Kim Herzig. 2016. Let\u2019s assume we had to pay for testing. Retrieved from https:\/\/www.slideshare.net\/kim.herzig\/keynote-ast-2016."},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2015.66"},{"key":"e_1_3_2_53_2","first-page":"55","volume-title":"Software Engineering (Workshops)","author":"Hirzel Matthias","year":"2016","unstructured":"Matthias Hirzel and Herbert Klaeren. 2016. Graph-walk-based selective regression testing of web applications created with Google web toolkit. In Software Engineering (Workshops). CEUR, 55\u201369. https:\/\/ceur-ws.org\/Vol-1559\/paper05.pdf."},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.1994.296778"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/2610384.2628055"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1145\/3057269"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2017.08.014"},{"key":"e_1_3_2_58_2","unstructured":"Stanislav Kirdey Kevin Cureton Scott Rick Sankar Ramanathan and Mrinal Shukla. 2019. Lerner \u2014 Using RL agents for test case scheduling. Retrieved from https:\/\/netflixtechblog.com\/lerner-using-rl-agents-for-test-case-scheduling-3e0686211198."},{"issue":"2004","key":"e_1_3_2_59_2","first-page":"1","article-title":"Procedures for performing systematic reviews","volume":"33","author":"Kitchenham Barbara","year":"2004","unstructured":"Barbara Kitchenham. 2004. Procedures for performing systematic reviews. Keele, UK, Keele Univ. 33, 2004 (2004), 1\u201326.","journal-title":"Keele, UK, Keele Univ."},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1109\/APSEC.2017.22"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1145\/3106237.3106288"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1109\/SMC.2019.8914505"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEIP.2019.00019"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.1989.65194"},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2021.3137929"},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2020.2992428"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1016\/bs.adcom.2018.10.001"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884874"},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-35510-4"},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEIP.2019.00018"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1145\/2993288.2993299"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2019.110430"},{"key":"e_1_3_2_73_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME.2016.77"},{"key":"e_1_3_2_74_2","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3473916"},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEIP.2017.16"},{"key":"e_1_3_2_76_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2020.110607"},{"key":"e_1_3_2_77_2","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180210"},{"key":"e_1_3_2_78_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10270-021-00924-8"},{"key":"e_1_3_2_79_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jksuci.2018.09.005"},{"key":"e_1_3_2_80_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEIP.2019.00031"},{"key":"e_1_3_2_81_2","doi-asserted-by":"publisher","DOI":"10.1145\/3530019.3530023"},{"key":"e_1_3_2_82_2","unstructured":"Bao N. Nguyen Tim Henderson John Micco and Sanjeev Dhanda. 2016. Google Journal Club. Retrieved from https:\/\/sites.google.com\/site\/gjournalclub\/."},{"key":"e_1_3_2_83_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-35510-4_4"},{"key":"e_1_3_2_84_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2015.7381799"},{"key":"e_1_3_2_85_2","doi-asserted-by":"publisher","DOI":"10.1145\/3526072.3527525"},{"key":"e_1_3_2_86_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11219-017-9398-y"},{"key":"e_1_3_2_87_2","doi-asserted-by":"publisher","DOI":"10.1145\/3457913.3457930"},{"key":"e_1_3_2_88_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-021-10066-6"},{"key":"e_1_3_2_89_2","doi-asserted-by":"publisher","DOI":"10.1145\/3395363.3397383"},{"key":"e_1_3_2_90_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00054"},{"key":"e_1_3_2_91_2","doi-asserted-by":"publisher","DOI":"10.1145\/2908812.2908850"},{"key":"e_1_3_2_92_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2020.106268"},{"key":"e_1_3_2_93_2","doi-asserted-by":"publisher","DOI":"10.1145\/2631685"},{"key":"e_1_3_2_94_2","doi-asserted-by":"publisher","DOI":"10.1145\/2972206.2972224"},{"key":"e_1_3_2_95_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-49421-0_10"},{"key":"e_1_3_2_96_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2809600"},{"key":"e_1_3_2_97_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-72904-2_11"},{"key":"e_1_3_2_98_2","doi-asserted-by":"publisher","DOI":"10.1142\/S0218194016300013"},{"key":"e_1_3_2_99_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-96147-3_18"},{"key":"e_1_3_2_100_2","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.1794"},{"key":"e_1_3_2_101_2","doi-asserted-by":"publisher","DOI":"10.14569\/IJACSA.2021.0120282"},{"key":"e_1_3_2_102_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2008.32"},{"key":"e_1_3_2_103_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2016.01.018"},{"key":"e_1_3_2_104_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME52107.2021.00053"},{"key":"e_1_3_2_105_2","doi-asserted-by":"publisher","DOI":"10.1145\/3213846.3213875"},{"key":"e_1_3_2_106_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2019.00031"},{"issue":"4","key":"e_1_3_2_107_2","article-title":"Systematic literature review on regression test prioritization techniques","volume":"36","author":"Singh Yogesh","year":"2012","unstructured":"Yogesh Singh, Arvinder Kaur, Bharti Suri, and Shweta Singhal. 2012. Systematic literature review on regression test prioritization techniques. Informatica 36, 4 (2012).","journal-title":"Informatica"},{"key":"e_1_3_2_108_2","doi-asserted-by":"publisher","DOI":"10.1145\/3092703.3092709"},{"key":"e_1_3_2_109_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2016.06.017"},{"key":"e_1_3_2_110_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2015.09.002"},{"key":"e_1_3_2_111_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2016.23"},{"key":"e_1_3_2_112_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-49094-6_17"},{"key":"e_1_3_2_113_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSTW.2016.14"},{"key":"e_1_3_2_114_2","doi-asserted-by":"publisher","DOI":"10.1145\/3106237.3117763"},{"key":"e_1_3_2_115_2","doi-asserted-by":"publisher","DOI":"10.1145\/2896941.2896951"},{"key":"e_1_3_2_116_2","doi-asserted-by":"publisher","DOI":"10.1145\/2889160.2889240"},{"key":"e_1_3_2_117_2","doi-asserted-by":"publisher","DOI":"10.1145\/3361242.3361258"},{"key":"e_1_3_2_118_2","doi-asserted-by":"publisher","DOI":"10.1109\/COMPSAC51774.2021.00182"},{"key":"e_1_3_2_119_2","doi-asserted-by":"publisher","DOI":"10.1109\/SCAM.2019.00032"},{"key":"e_1_3_2_120_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2022.3184842"},{"key":"e_1_3_2_121_2","volume-title":"CEUR Workshop Proceedings","volume":"2201","author":"Yilmaz U.","year":"2018","unstructured":"U. Yilmaz and A. Tarhan. 2018. A case study to compare regression test selection techniques on open-source software projects. In CEUR Workshop Proceedings, Vol. 2201, CEUR."},{"key":"e_1_3_2_122_2","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.430"},{"key":"e_1_3_2_123_2","doi-asserted-by":"publisher","DOI":"10.1145\/2950290.2983937"},{"key":"e_1_3_2_124_2","doi-asserted-by":"publisher","DOI":"10.1145\/3338906.3340448"},{"key":"e_1_3_2_125_2","doi-asserted-by":"publisher","DOI":"10.17706\/jsw.10.8.971-990"},{"key":"e_1_3_2_126_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524481.3527230"},{"key":"e_1_3_2_127_2","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180198"},{"key":"e_1_3_2_128_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2019.00052"},{"key":"e_1_3_2_129_2","doi-asserted-by":"publisher","DOI":"10.1145\/3471906"},{"key":"e_1_3_2_130_2","doi-asserted-by":"publisher","DOI":"10.1109\/TR.2020.2979815"},{"key":"e_1_3_2_131_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME.2018.00016"}],"container-title":["ACM Computing Surveys"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3579851","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3579851","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:51:27Z","timestamp":1750182687000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3579851"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,13]]},"references-count":130,"journal-issue":{"issue":"13s","published-print":{"date-parts":[[2023,12,31]]}},"alternative-id":["10.1145\/3579851"],"URL":"https:\/\/doi.org\/10.1145\/3579851","relation":{},"ISSN":["0360-0300","1557-7341"],"issn-type":[{"value":"0360-0300","type":"print"},{"value":"1557-7341","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,13]]},"assertion":[{"value":"2022-02-15","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-01-03","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-07-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}