{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,14]],"date-time":"2026-04-14T03:58:10Z","timestamp":1776139090197,"version":"3.50.1"},"reference-count":61,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2024,6,29]],"date-time":"2024-06-29T00:00:00Z","timestamp":1719619200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Science Foundation of China","doi-asserted-by":"crossref","award":["61932021"],"award-info":[{"award-number":["61932021"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Hong Kong Research Grant Council\/General Research Fund","award":["16207120"],"award-info":[{"award-number":["16207120"]}]},{"name":"Hong Kong Research Grant Council\/Research Impact Fund","award":["R5034-18"],"award-info":[{"award-number":["R5034-18"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2024,7,31]]},"abstract":"<jats:p>Metamorphic Testing (MT) alleviates the oracle problem by defining oracles based on metamorphic relations (MRs) that govern multiple related inputs and their outputs. However, designing MRs is challenging, as it requires domain-specific knowledge. This hinders the widespread adoption of MT. We observe that developer-written test cases can embed domain knowledge that encodes MRs. Such encoded MRs could be synthesized for testing not only their original programs but also other programs that share similar functionalities.<\/jats:p>\n          <jats:p>\n            In this article, we propose\n            <jats:sc>MR-Scout<\/jats:sc>\n            to automatically synthesize MRs from test cases in open-source software (OSS) projects.\n            <jats:sc>MR-Scout<\/jats:sc>\n            first discovers MR-encoded test cases (MTCs), and then synthesizes the encoded MRs into parameterized methods (called\n            <jats:italic>codified MRs<\/jats:italic>\n            ), and filters out MRs that demonstrate poor quality for new test case generation.\n            <jats:sc>MR-Scout<\/jats:sc>\n            discovered over 11,000 MTCs from 701 OSS projects. Experimental results show that over 97% of codified MRs are of high quality for automated test case generation, demonstrating the practical applicability of\n            <jats:sc>MR-Scout<\/jats:sc>\n            . Furthermore, codified-MRs-based tests effectively enhance the test adequacy of programs with developer-written tests, leading to 13.52% and 9.42% increases in line coverage and mutation score, respectively. Our qualitative study shows that 55.76% to 76.92% of codified MRs are easily comprehensible for developers.\n          <\/jats:p>","DOI":"10.1145\/3656340","type":"journal-article","created":{"date-parts":[[2024,4,9]],"date-time":"2024-04-09T11:53:09Z","timestamp":1712663589000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":12,"title":["MR-Scout: Automated Synthesis of Metamorphic Relations from Existing Test Cases"],"prefix":"10.1145","volume":"33","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-2887-1690","authenticated-orcid":false,"given":"Congying","family":"Xu","sequence":"first","affiliation":[{"name":"The Hong Kong University of Science and Technology, Hong Kong, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5885-9297","authenticated-orcid":false,"given":"Valerio","family":"Terragni","sequence":"additional","affiliation":[{"name":"The University of Auckland, Auckland, New Zealand"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3082-5957","authenticated-orcid":false,"given":"Hengcheng","family":"Zhu","sequence":"additional","affiliation":[{"name":"The Hong Kong University of Science and Technology, Hong Kong, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6126-303X","authenticated-orcid":false,"given":"Jiarong","family":"Wu","sequence":"additional","affiliation":[{"name":"The Hong Kong University of Science and Technology, Hong Kong, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3508-7172","authenticated-orcid":false,"given":"Shing-Chi","family":"Cheung","sequence":"additional","affiliation":[{"name":"The Hong Kong University of Science and Technology, Hong Kong China"}]}],"member":"320","published-online":{"date-parts":[[2024,6,29]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEIP52600.2021.00023"},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE51524.2021.9678706"},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.1002\/STVR.1486"},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3473920"},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3520304.3534077"},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2021.111041."},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2018.09.016"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/2408776.2408795"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3490488"},{"key":"e_1_3_3_11_2","volume-title":"Metamorphic Testing: A New Approach for Generating Next Test Cases","author":"Chen T. Y.","year":"1998","unstructured":"T. Y. Chen, S. C. Cheung, and S. M. Yiu. 1998. Metamorphic Testing: A New Approach for Generating Next Test Cases. Technical Report. Technical Report HKUST-CS98-01, Department of Computer Science, The Hong Kong University of Science and Technology."},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/3143561"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2015.07.037"},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2022.3227418"},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/MET.2019.00008"},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/2896971.2896978"},{"key":"e_1_3_3_17_2","unstructured":"EvoSuite. 2023. EvoSuite. Retrieved August 20 2023 from https:\/\/www.evosuite.org\/"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/2025113.2025179"},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2013.51"},{"key":"e_1_3_3_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2012.14"},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/2001420.2001464"},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3526072.3527538"},{"key":"e_1_3_3_23_2","unstructured":"GitHub. 2023. GitHub. Retrieved August 20 2023 from https:\/\/github.com\/"},{"key":"e_1_3_3_24_2","unstructured":"Grammarly. 2023. Grammarly. Retrieved August 20 2023 from http:\/\/grammarly.com"},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2015.7102580"},{"key":"e_1_3_3_26_2","unstructured":"N. Alan Heckert James J. Filliben C. M. Croarkin B. Hembree William F. Guthrie P. Tobias and J. Prinz. 2002. Handbook 151: Nist\/sematech e-handbook of statistical methods. (2002)."},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-022-10131-8"},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/2931037.2931062"},{"key":"e_1_3_3_29_2","unstructured":"Junit. 2023. Junit4. Retrieved August 20 2023 from https:\/\/junit.org\/junit4\/javadoc\/4.13\/org\/junit\/Assert.html"},{"key":"e_1_3_3_30_2","unstructured":"Junit. 2023. Junit5. Retrieved August 20 2023 from https:\/\/junit.org\/junit5\/"},{"key":"e_1_3_3_31_2","unstructured":"Junit. 2023. Junit5 Assertions. Retrieved August 20 2023 from https:\/\/junit.org\/junit5\/docs\/5.0.3\/api\/org\/junit\/jupiter\/api\/Assertions.html"},{"key":"e_1_3_3_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-COMPANION.2019.00098"},{"key":"e_1_3_3_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2013.6698899"},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.1594"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3468619"},{"key":"e_1_3_3_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/MET.2017.6"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","unstructured":"Haoyang Ma Qingchao Shen Yongqiang Tian Junjie Chen and Shing-Chi Cheung. 2023. Fuzzing Deep Learning Compilers with HirGen. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis. 248\u2013260. DOI:DOI:10.1145\/3597926.3598053","DOI":"10.1145\/3597926.3598053"},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2020\/64"},{"key":"e_1_3_3_39_2","unstructured":"OpenAI. 2023. ChatGPT. Retrieved August 20 2023 from https:\/\/openai.com\/blog\/chatgpt"},{"key":"e_1_3_3_40_2","unstructured":"Oracle. 2023. Java Language Specification. Retrieved August 20 2023 from https:\/\/docs.oracle.com\/javase\/specs\/"},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/1297846.1297902"},{"key":"e_1_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00202"},{"key":"e_1_3_3_43_2","unstructured":"PITest. 2023. PITest. Retrieved August 20 2023 from https:\/\/pitest.org\/"},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2020.3009698"},{"key":"e_1_3_3_45_2","volume-title":"Mathematical Statistics and Data Analysis","author":"Rice John A.","year":"2006","unstructured":"John A. Rice. 2006. Mathematical Statistics and Data Analysis. Cengage Learning."},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/MET.2017.3"},{"key":"e_1_3_3_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2016.2532875"},{"key":"e_1_3_3_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3182528"},{"key":"e_1_3_3_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2934848"},{"key":"e_1_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/2896971.2896974"},{"key":"e_1_3_3_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409758"},{"key":"e_1_3_3_52_2","unstructured":"TestNG. 2023. TestNG. Retrieved August 20 2023 from https:\/\/testng.org"},{"key":"e_1_3_3_53_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-19811-3_21"},{"key":"e_1_3_3_54_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-021-09985-1"},{"key":"e_1_3_3_55_2","unstructured":"MR-Scout. 2023. MR-Scout. Retrieved August 20 2023 from https:\/\/mr-scout.github.io"},{"key":"e_1_3_3_56_2","doi-asserted-by":"publisher","unstructured":"Shuai Wang and Zhendong Su. 2020. Metamorphic object insertion for testing object detection systems. In Proceedings of the 35th IEEE\/ACM International Conference on Automated Software Engineering. 1053\u20131065. DOI:DOI:10.1145\/3324884.3416584","DOI":"10.1145\/3324884.3416584"},{"key":"e_1_3_3_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME46990.2020.00014"},{"key":"e_1_3_3_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/3508035"},{"key":"e_1_3_3_59_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME.2019.00035"},{"key":"e_1_3_3_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/2642937.2642994"},{"key":"e_1_3_3_61_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2018.2876433"},{"key":"e_1_3_3_62_2","doi-asserted-by":"publisher","DOI":"10.1145\/3324884.3416539"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3656340","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3656340","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:48:59Z","timestamp":1750286939000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3656340"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,29]]},"references-count":61,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2024,7,31]]}},"alternative-id":["10.1145\/3656340"],"URL":"https:\/\/doi.org\/10.1145\/3656340","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,29]]},"assertion":[{"value":"2023-08-23","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-03-20","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-06-29","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}