{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,23]],"date-time":"2026-04-23T11:08:00Z","timestamp":1776942480845,"version":"3.51.4"},"reference-count":60,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2023,11,23]],"date-time":"2023-11-23T00:00:00Z","timestamp":1700697600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62272037 and 61872039"],"award-info":[{"award-number":["62272037 and 61872039"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100012226","name":"Fundamental Research Funds for the Central Universities","doi-asserted-by":"crossref","award":["FRF-GF-19-B19"],"award-info":[{"award-number":["FRF-GF-19-B19"]}],"id":[{"id":"10.13039\/501100012226","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Beijing Natural Science Foundation","award":["4162040"],"award-info":[{"award-number":["4162040"]}]},{"name":"Aeronautical Science Foundation of China","award":["2016ZD74004"],"award-info":[{"award-number":["2016ZD74004"]}]},{"DOI":"10.13039\/501100004739","name":"Youth Innovation Promotion Association of the Chinese Academy of Sciences","doi-asserted-by":"crossref","award":["Y2021041"],"award-info":[{"award-number":["Y2021041"]}],"id":[{"id":"10.13039\/501100004739","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Key Research Program of Frontier Sciences, CAS","award":["ZDBS-LY-7006"],"award-info":[{"award-number":["ZDBS-LY-7006"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2024,1,31]]},"abstract":"<jats:p>\n            Concurrent programs are normally composed of multiple concurrent threads sharing memory space. These threads are often interleaved, which may lead to some non-determinism in execution results, even for the same program input. This poses huge challenges to the testing of concurrent programs, especially on the test result verification\u2014that is, the prevalent existence of the oracle problem. In this article, we investigate the application of metamorphic testing (MT), a mainstream technique to address the oracle problem, into the testing of concurrent programs. Based on the unique features of interleaved executions in concurrent programming, we propose an extended notion of metamorphic relations, the core part of MT, which are particularly designed for the testing of concurrent programs. A comprehensive testing approach, namely\n            <jats:italic>ConMT<\/jats:italic>\n            , is thus developed and a tool is built to automate its implementation on concurrent programs written in Java. Empirical studies have been conducted to evaluate the performance of ConMT, and the experimental results show that in addition to addressing the oracle problem, ConMT outperforms the baseline traditional testing techniques with respect to a higher degree of automation, better bug detection capability, and shorter testing time. It is clear that ConMT can significantly improve the cost-effectiveness for the testing of concurrent programs and thus advances the state of the art in the field. The study also brings novelty into MT, hence promoting the fundamental research of software testing.\n          <\/jats:p>","DOI":"10.1145\/3607182","type":"journal-article","created":{"date-parts":[[2023,7,3]],"date-time":"2023-07-03T12:13:28Z","timestamp":1688386408000},"page":"1-21","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["An Interleaving Guided Metamorphic Testing Approach for Concurrent Programs"],"prefix":"10.1145","volume":"33","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3696-6176","authenticated-orcid":false,"given":"Chang-Ai","family":"Sun","sequence":"first","affiliation":[{"name":"University of Science and Technology Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5507-3053","authenticated-orcid":false,"given":"Hepeng","family":"Dai","sequence":"additional","affiliation":[{"name":"University of Science and Technology Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-6279-1610","authenticated-orcid":false,"given":"Ning","family":"Geng","sequence":"additional","affiliation":[{"name":"University of Science and Technology Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3125-4399","authenticated-orcid":false,"given":"Huai","family":"Liu","sequence":"additional","affiliation":[{"name":"Swinburne University of Technology, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3578-0994","authenticated-orcid":false,"given":"Tsong Yueh","family":"Chen","sequence":"additional","affiliation":[{"name":"Swinburne University of Technology, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4931-0566","authenticated-orcid":false,"given":"Peng","family":"Wu","sequence":"additional","affiliation":[{"name":"Chinese Academy of Sciences and University of Chinese Academy of Sciences, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4768-0272","authenticated-orcid":false,"given":"Yan","family":"Cai","sequence":"additional","affiliation":[{"name":"Chinese Academy of Sciences and University of Chinese Academy of Sciences, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-8748-8466","authenticated-orcid":false,"given":"Jinqiu","family":"Wang","sequence":"additional","affiliation":[{"name":"Chinese Academy of Sciences and University of Chinese Academy of Sciences, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,11,23]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEIP52600.2021.00023"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/1985793.1985795"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2014.2372785"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2017.2707089"},{"key":"e_1_3_2_6_2","volume-title":"SWEBOC V3.0: Guide to the Software Engineering Body of Knowledge","author":"Bourque Pierre","year":"2014","unstructured":"Pierre Bourqueand Richard E. Fairley (Eds.). 2014. SWEBOC V3.0: Guide to the Software Engineering Body of Knowledge. IEEE, Los Alamitos, CA."},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3468549"},{"key":"e_1_3_2_8_2","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1007\/978-3-540-73230-3_7","volume-title":"Reliable Software Technologies\u2014Ada Europe 2007)","author":"Chan W. K.","year":"2007","unstructured":"W. K. Chan, T. Y. Chen, S. C. Cheung, T. H. Tse, and Zhenyu Zhang. 2007. Towards the testing of power-aware software applications for wireless sensor networks. In Reliable Software Technologies\u2014Ada Europe 2007). Lecture Notes in Computer Science, Vol. 4498. Springer, 84\u201399."},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/QSIC.2005.67"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2105-10-24"},{"issue":"1","key":"e_1_3_2_11_2","first-page":"Article 4, 27 p","article-title":"Metamorphic testing: A review of challenges and opportunities","volume":"51","author":"Chen Tsong Yueh","year":"2018","unstructured":"Tsong Yueh Chen, Fei-Ching Kuo, Huai Liu, Pak-Lok Poon, Dave Towey, T. H. Tse, and Zhi Quan Zhou. 2018. Metamorphic testing: A review of challenges and opportunities. ACM Computing Surveys 51, 1 (2018), Article 4, 27 pages.","journal-title":"ACM Computing Surveys"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/MET.2019.00013"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/SSIRI.2010.25"},{"key":"e_1_3_2_14_2","unstructured":"Anurag Dwarakanath Sanjay Podder Neville Dubash Kishore P. Durg Manish Ahuja Raghotham M. Rao Samarth Sikand and Jagadeesh Chandra Bose Rantham Prabhakara. 2019. Verification by metamorphic testing of applications that utilize artificial intelligence. US Patent 20190108443. Filed Sept. 28 2018. Issued Oct. 8 2019."},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.5555\/1228965.1228973"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2004.1303339"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/964001.964023"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/1542476.1542490"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/263699.263717"},{"key":"e_1_3_2_20_2","volume-title":"Concurrency at Microsoft: An Exploratory Survey","author":"Godefroid Patrice","year":"2008","unstructured":"Patrice Godefroid and Nachi Nagappan. 2008. Concurrency at Microsoft: An Exploratory Survey. Technical Report MSR-TR-2008-75. Microsoft. https:\/\/www.microsoft.com\/en-us\/research\/publication\/concurrency-at-microsoft-an-exploratory-survey\/."},{"key":"e_1_3_2_21_2","volume-title":"Java Concurrency in Practice","author":"Goetz Brian","year":"2006","unstructured":"Brian Goetz, Tim Peierls, Doug Lea, Joshua Bloch, Joseph Bowbeer, and David Holmes. 2006. Java Concurrency in Practice. Pearson Education."},{"key":"e_1_3_2_22_2","unstructured":"GraphicsFuzz. 2018. GraphicsFuzz is acquired by Google. GraphicsFuzz . Retrieved February 16 2021 from https:\/\/www.graphicsfuzz.com\/."},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1002\/0471028959.sof268"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/1368088.1368120"},{"key":"e_1_3_2_25_2","unstructured":"S. Y. Huang. 2020. Maximal-Causality-Reduction (MCR) Java Version. Retrieved July 9 2023 from https:\/\/github.com\/parasol-aser\/JMCR."},{"key":"e_1_3_2_26_2","volume-title":"Proceedings of the 31st European Conference on Object-Oriented Programming (ECOOP\u201917)","author":"Huang Shiyou","year":"2017","unstructured":"Shiyou Huang and Jeff Huang. 2017. Speeding up maximal causality reduction with static dependency analysis. In Proceedings of the 31st European Conference on Object-Oriented Programming (ECOOP\u201917)."},{"key":"e_1_3_2_27_2","unstructured":"Jacob Jackson. 2012. Nasdaq\u2019s Facebook glitch came from \u2018race conditions\u2019. COMPUTERWORLD . Retrieved February 16 2021 from https:\/\/www.computerworld.com\/article\/2504676\/nasdaq-s-facebook-glitch-came-from--race-conditions-.html."},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/1542476.1542489"},{"key":"e_1_3_2_29_2","unstructured":"JUnit. 2020. Junit 5. Retrieved July 9 2023 from https:\/\/junit.org\/junit5\/."},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/3276516"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.1594"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/1806799.1806836"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/359545.359563"},{"key":"e_1_3_2_34_2","doi-asserted-by":"crossref","unstructured":"Vu Le Mehrdad Afshari and Zhendong Su. 2014. Compiler validation via equivalence modulo inputs. ACM SIGPLAN Notices 49 (2014) 216\u2013226.","DOI":"10.1145\/2666356.2594334"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/MC.1993.274940"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2015.348"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2013.46"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISCA.2008.4"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2018.07.001"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3276515"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/TR.2018.2847247"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/6.809121"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/781498.781528"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/1453101.1453121"},{"key":"e_1_3_2_45_2","article-title":"Fast, sound and effectively complete dynamic race prediction","author":"Pavlogiannis Andreas","year":"2019","unstructured":"Andreas Pavlogiannis. 2019. Fast, sound and effectively complete dynamic race prediction. arXiv preprint arXiv: 1901.08857 (2019).","journal-title":"arXiv preprint arXiv: 1901.08857"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2020.3009698"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/2555243.2555262"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/265924.265927"},{"key":"e_1_3_2_49_2","unstructured":"Kevin Poulsen. 2004. Software bug contributed to blackout. The Register . Retrieved July 9 2023 from https:\/\/www.theregister.com\/2004\/02\/12\/software_bug_contributed_to_blackout\/."},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2016.2532875"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/1375581.1375584"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jpdc.2007.01.006"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2934848"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICWS.2011.65"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1002\/spe.2324"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180220"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/CMPSAC.2004.1342879"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2006.1599419"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2019.00073"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2022.3144480"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2015.2478001"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3607182","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3607182","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:37:34Z","timestamp":1750178254000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3607182"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,23]]},"references-count":60,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,1,31]]}},"alternative-id":["10.1145\/3607182"],"URL":"https:\/\/doi.org\/10.1145\/3607182","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,23]]},"assertion":[{"value":"2022-07-30","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-06-12","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-11-23","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}