{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,17]],"date-time":"2026-03-17T00:54:46Z","timestamp":1773708886250,"version":"3.50.1"},"reference-count":88,"publisher":"Association for Computing Machinery (ACM)","issue":"7","license":[{"start":{"date-parts":[[2024,9,27]],"date-time":"2024-09-27T00:00:00Z","timestamp":1727395200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2024,9,30]]},"abstract":"<jats:p>In recent years, Automated Program Repair (APR) has been extensively studied in academia and even drawn wide attention from the industry. However, APR techniques can be extremely time consuming since (1) a large number of patches can be generated for a given bug, and (2) each patch needs to be executed on the original tests to ensure its correctness. In the literature, various techniques (e.g., based on learning, mining, and constraint solving) have been proposed\/studied to reduce the number of patches. Intuitively, every patch can be treated as a software revision during regression testing; thus, traditional Regression Test Selection (RTS) techniques can be leveraged to only execute the tests affected by each patch (as the other tests would keep the same outcomes) to further reduce patch execution time. However, few APR systems actually adopt RTS and there is still a lack of systematic studies demonstrating the benefits of RTS and the impact of different RTS strategies on APR. To this end, this article presents the first extensive study of widely used RTS techniques at different levels (i.e., class\/method\/statement levels) for 12 state-of-the-art APR systems on over 2M patches. Our study reveals various practical guidelines for bridging the gap between APR and regression testing, including: (1) the number of patches widely used for measuring APR efficiency can incur skewed conclusions, and the use of inconsistent RTS configurations can further skew the conclusions; (2) all studied RTS techniques can substantially improve APR efficiency and should be considered in future APR work; (3) method- and statement-level RTS outperform class-level RTS substantially and should be preferred; (4) RTS techniques can substantially outperform state-of-the-art test prioritization techniques for APR, and combining them can further improve APR efficiency; and (5) traditional Regression Test Prioritization (RTP) widely studied in regression testing performs even better than APR-specific test prioritization when combined with most RTS techniques. Furthermore, we also present the detailed impact of different patch categories and patch validation strategies on our findings.<\/jats:p>","DOI":"10.1145\/3672450","type":"journal-article","created":{"date-parts":[[2024,6,13]],"date-time":"2024-06-13T17:56:59Z","timestamp":1718301419000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["When Automated Program Repair Meets Regression Testing\u2014An Extensive Study on Two Million Patches"],"prefix":"10.1145","volume":"33","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4066-3365","authenticated-orcid":false,"given":"Yiling","family":"Lou","sequence":"first","affiliation":[{"name":"Fudan University, Shanghai, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-3657-6169","authenticated-orcid":false,"given":"Jun","family":"Yang","sequence":"additional","affiliation":[{"name":"University of Illinois Urbana-Champaign, Champaign, IL, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0592-802X","authenticated-orcid":false,"given":"Samuel","family":"Benton","sequence":"additional","affiliation":[{"name":"The University of Texas at Dallas, Richardson, TX, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8295-303X","authenticated-orcid":false,"given":"Dan","family":"Hao","sequence":"additional","affiliation":[{"name":"Peking University, Beijing, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6690-8332","authenticated-orcid":false,"given":"Lin","family":"Tan","sequence":"additional","affiliation":[{"name":"Purdue University, West Lafayette, IN, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4765-1893","authenticated-orcid":false,"given":"Zhenpeng","family":"Chen","sequence":"additional","affiliation":[{"name":"Nanyang Technological University, Singapore, Singapore"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7453-2339","authenticated-orcid":false,"given":"Lu","family":"Zhang","sequence":"additional","affiliation":[{"name":"Peking University, Beijing, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8681-0287","authenticated-orcid":false,"given":"Lingming","family":"Zhang","sequence":"additional","affiliation":[{"name":"University of Illinois Urbana-Champaign, West Lafayette, IN, USA"}]}],"member":"320","published-online":{"date-parts":[[2024,9,27]]},"reference":[{"key":"e_1_3_1_2_2","unstructured":"Replication package. 2023. Retrieved from https:\/\/github.com\/anomynousdata\/RTAPR"},{"key":"e_1_3_1_3_2","unstructured":"Apache Camel. 2020a. Retrieved from http:\/\/camel.apache.org\/"},{"key":"e_1_3_1_4_2","unstructured":"Apache Commons Math. 2020b. Retrieved from https:\/\/commons.apache.org\/proper\/commons-math\/"},{"key":"e_1_3_1_5_2","unstructured":"Apache CXF. 2020c. Retrieved from https:\/\/cxf.apache.org\/"},{"key":"e_1_3_1_6_2","unstructured":"ASM Java bytecode manipulation and analysis framework. 2020. Retrieved from http:\/\/asm.ow2.org"},{"key":"e_1_3_1_7_2","unstructured":"JavaAgent. 2020. Retrieved from https:\/\/docs.oracle.com\/javase\/7\/docs\/api\/java\/lang\/instrument\/package-summary.html"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.5555\/1308173.1308264"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3324884.3416566"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2021.3125203"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/3236024.3275527"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.5555\/3155562.3155642"},{"key":"e_1_3_1_13_2","unstructured":"Lingchao Chen and Lingming Zhang. 2020. Fast and precise on-the-fly patch validation for all. arXiv: 2007.11449."},{"key":"e_1_3_1_14_2","first-page":"1943","article-title":"Sequencer: Sequence-to-sequence learning for end-to-end program repair","volume":"47","author":"Chen Zimin","year":"2019","unstructured":"Zimin Chen, Steve James Kommrusch, Michele Tufano, Louis-No\u00ebl Pouchet, Denys Poshyvanyk, and Martin Monperrus. 2019. Sequencer: Sequence-to-sequence learning for end-to-end program repair. IEEE Trans Softw. Eng 47 (2019), 1943\u20131959.","journal-title":"IEEE Trans Softw. Eng"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2017.7884635"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/2896921.2896931"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/2635868.2635910"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/3293882.3330559"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/2771783.2771784"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11219-013-9208-0"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00120"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/504282.504305"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180245"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2019.00033"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/3213846.3213871"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00111"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00107"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/2610384.2610388"},{"key":"e_1_3_1_29_2","first-page":"437","article-title":"Defects4J: A database of existing faults to enable controlled testing studies for Java programs","author":"Just Ren\u00e9","year":"2014","unstructured":"Ren\u00e9 Just, Darioush Jalali, and Michael D. Ernst. 2014b. Defects4J: A database of existing faults to enable controlled testing studies for Java programs. In International Symposium on Software Testing and Analysis (ISSTA \u201914). 437\u2013440.","journal-title":"International Symposium on Software Testing and Analysis (ISSTA \u201914)"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2021.3067156"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2013.6606626"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-019-09742-5"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-019-09780-z"},{"issue":"2","key":"e_1_3_1_34_2","first-page":"51","article-title":"Class firewall, test order, and regression testing of object-oriented programs","volume":"8","author":"Kung David Chenho","year":"1995","unstructured":"David Chenho Kung, Jerry Gao, Pei Hsia, Jeremy Lin, and Yasufumi Toyoshima. 1995. Class firewall, test order, and regression testing of object-oriented programs. J. Object Oriented Program. 8, 2 (1995), 51\u201365.","journal-title":"J. Object Oriented Program."},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3106237.3106309"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2016.76"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/2950290.2950361"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380345"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510177"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2007.38"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2019.00020"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2019.8667970"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3293882.3330577"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/APSEC.2018.00085"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380338"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2018.8330202"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/3106237.3106253"},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884872"},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1016\/bs.adcom.2018.10.001"},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/3395363.3397351"},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884874"},{"key":"e_1_3_1_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/3395363.3397369"},{"key":"e_1_3_1_53_2","doi-asserted-by":"publisher","DOI":"10.1145\/2931037.2948705"},{"key":"e_1_3_1_54_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-99241-9_3"},{"key":"e_1_3_1_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/3241980"},{"key":"e_1_3_1_56_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2018.00031"},{"key":"e_1_3_1_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEIP.2017.16"},{"key":"e_1_3_1_58_2","unstructured":"Martin Monperrus. 2020. The living review on automated program repair. (2020)."},{"key":"e_1_3_1_59_2","doi-asserted-by":"publisher","DOI":"10.1145\/1029894.1029928"},{"key":"e_1_3_1_60_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.2013.29"},{"key":"e_1_3_1_61_2","doi-asserted-by":"publisher","DOI":"10.1145\/1028976.1029012"},{"key":"e_1_3_1_62_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.1993.366926"},{"key":"e_1_3_1_63_2","doi-asserted-by":"publisher","DOI":"10.1145\/248233.248262"},{"key":"e_1_3_1_64_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.1999.792604"},{"key":"e_1_3_1_65_2","doi-asserted-by":"publisher","DOI":"10.5555\/3155562.3155643"},{"key":"e_1_3_1_66_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00020"},{"key":"e_1_3_1_67_2","doi-asserted-by":"publisher","DOI":"10.1145\/3360613"},{"key":"e_1_3_1_68_2","doi-asserted-by":"publisher","DOI":"10.1145\/2786805.2786825"},{"key":"e_1_3_1_69_2","doi-asserted-by":"publisher","DOI":"10.1145\/3106237.3106258"},{"key":"e_1_3_1_70_2","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3616271"},{"key":"e_1_3_1_71_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2013.6693094"},{"key":"e_1_3_1_72_2","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180233"},{"key":"e_1_3_1_73_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377816.3381743"},{"key":"e_1_3_1_74_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2019.8668043"},{"key":"e_1_3_1_75_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-4380-9_16"},{"key":"e_1_3_1_76_2","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3468600"},{"key":"e_1_3_1_77_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00129"},{"key":"e_1_3_1_78_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549101"},{"key":"e_1_3_1_79_2","doi-asserted-by":"publisher","DOI":"10.5555\/3155562.3155644"},{"key":"e_1_3_1_80_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2017.45"},{"key":"e_1_3_1_81_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2016.2560811"},{"key":"e_1_3_1_82_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510222"},{"key":"e_1_3_1_83_2","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3182517"},{"key":"e_1_3_1_84_2","doi-asserted-by":"publisher","DOI":"10.1002\/stv.430"},{"key":"e_1_3_1_85_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2018.2874648"},{"key":"e_1_3_1_86_2","doi-asserted-by":"publisher","DOI":"10.1145\/3360004"},{"key":"e_1_3_1_87_2","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180198"},{"key":"e_1_3_1_88_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2019.00052"},{"key":"e_1_3_1_89_2","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3468544"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3672450","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3672450","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:58:01Z","timestamp":1750294681000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3672450"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,27]]},"references-count":88,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2024,9,30]]}},"alternative-id":["10.1145\/3672450"],"URL":"https:\/\/doi.org\/10.1145\/3672450","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,9,27]]},"assertion":[{"value":"2023-02-08","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-05-14","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-09-27","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}