{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,12]],"date-time":"2026-09-12T01:35:46Z","timestamp":1789176946009,"version":"build-2803163510"},"reference-count":83,"publisher":"Association for Computing Machinery (ACM)","issue":"ISSTA","license":[{"start":{"date-parts":[[2025,6,22]],"date-time":"2025-06-22T00:00:00Z","timestamp":1750550400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/publication-rights-and-licensing-policy"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["Grant Nos. U24B20149 and 62272271"],"award-info":[{"award-number":["Grant Nos. U24B20149 and 62272271"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"the Natural Science Foundation of Shandong Province","award":["Grant No. ZR2024QF093"],"award-info":[{"award-number":["Grant No. ZR2024QF093"]}]},{"name":"the Taishan Scholars Program","award":["Grant No. tsqn202408009"],"award-info":[{"award-number":["Grant No. tsqn202408009"]}]},{"name":"the Research Grants Council of Hong Kong, and the Industry Research Project funds","award":["Grant Nos. 6000871, 6000796, 9229109, 9229098, 9220103, and 9229029"],"award-info":[{"award-number":["Grant Nos. 6000871, 6000796, 9229109, 9229098, 9220103, and 9229029"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Softw. Eng."],"published-print":{"date-parts":[[2025,6,22]]},"abstract":"<jats:p>In recent years, Large Language Models (LLMs) have dramatically advanced the performance of automated code translation, making their computational accuracy score reach up to over 80% on many previous benchmarks. However, most code samples in these benchmarks are short, standalone, statement\/method-level, and algorithmic, which is not aligned with practical coding tasks. Therefore, it is still unknown the actual capability of LLMs in translating code samples written for daily development.<\/jats:p>\n                  <jats:p>To achieve this, we construct a class-level code translation benchmark, ClassEval-T, and make the first attempt to extensively assess recent LLMs\u2019 performance on class-level code translation. ClassEval-T is extended from ClassEval, a well-known class-level Python code generation benchmark consisting of multiple practical coding topics, such as database operation and game design, and diverse contextual dependencies (e.g., fields, methods, and libraries). It cost us 360 person-hours to accomplish the manual migration to Java and C++ with complete code samples and associated test suites. Subsequently, we design three translation strategies (i.e., holistic, min-dependency, and standalone) for class-level code translations and evaluate eight recent LLMs of commercial, general, and code kinds in diverse families and sizes on ClassEval-T. Experimental results demonstrate a remarkable performance drop compared with the most widely studied method-level code translation benchmark, and obvious discrepancies among LLMs appear, showing the effectivenes of ClassEval-T in measuring recent LLMs. Afterwards, we further discuss the usage scenarios for diverse translation strategies and LLMs\u2019 ability to dependency awareness when translating class samples. Finally, 1,243 failure cases made by the best-performing LLM under test are thoroughly analyzed and categorized in this paper for practical guidance and future enlightenment.<\/jats:p>","DOI":"10.1145\/3728940","type":"journal-article","created":{"date-parts":[[2025,6,22]],"date-time":"2025-06-22T10:52:56Z","timestamp":1750589576000},"page":"1421-1444","source":"Crossref","is-referenced-by-count":11,"title":["ClassEval-T: Evaluating Large Language Models in Class-Level Code Translation"],"prefix":"10.1145","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-3395-9575","authenticated-orcid":false,"given":"Pengyu","family":"Xue","sequence":"first","affiliation":[{"name":"Shandong University, Qingdao, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-7624-156X","authenticated-orcid":false,"given":"Linhao","family":"Wu","sequence":"additional","affiliation":[{"name":"Shandong University, Qingdao, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0670-4538","authenticated-orcid":false,"given":"Zhen","family":"Yang","sequence":"additional","affiliation":[{"name":"Shandong University, Qingdao, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-4153-6774","authenticated-orcid":false,"given":"Chengyi","family":"Wang","sequence":"additional","affiliation":[{"name":"Shandong University, Qingdao, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-3473-8221","authenticated-orcid":false,"given":"Xiang","family":"Li","sequence":"additional","affiliation":[{"name":"Shandong University, Qingdao, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-5366-9140","authenticated-orcid":false,"given":"Yuxiang","family":"Zhang","sequence":"additional","affiliation":[{"name":"Shandong University, Qingdao, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5579-8852","authenticated-orcid":false,"given":"Jia","family":"Li","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-9253-7906","authenticated-orcid":false,"given":"Ruikai","family":"Jin","sequence":"additional","affiliation":[{"name":"Shandong University, Qingdao, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-1351-0565","authenticated-orcid":false,"given":"Yifei","family":"Pei","sequence":"additional","affiliation":[{"name":"Shandong University, Qingdao, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9526-6634","authenticated-orcid":false,"given":"Zhaoyan","family":"Shen","sequence":"additional","affiliation":[{"name":"Shandong University, Qingdao, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-8861-8907","authenticated-orcid":false,"given":"Xiran","family":"Lyu","sequence":"additional","affiliation":[{"name":"Shandong University, Qingdao, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3803-9600","authenticated-orcid":false,"given":"Jacky Wai","family":"Keung","sequence":"additional","affiliation":[{"name":"City University of Hong Kong, Hong Kong, Hong Kong"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,6,22]]},"reference":[{"key":"e_1_3_1_2_2","unstructured":"Claude 3.5 Sonnet. n.d.. Anthropic. https:\/\/www.anthropic.com\/claude\/sonnet. Online; accessed 2025-02-17."},{"key":"e_1_3_1_3_2","unstructured":"Codeforces. n.d.. Codeforces. https:\/\/codeforces.com\/. Online; accessed 2025-02-21."},{"key":"e_1_3_1_4_2","unstructured":"GitHub. n.d.. GitHub. https:\/\/github.com\/wLinHoo\/ClassEval-T. Online; accessed 2025-04-09."},{"key":"e_1_3_1_5_2","unstructured":"OpenAI. 2025. OpenAI. https:\/\/openai.com\/."},{"key":"e_1_3_1_6_2","doi-asserted-by":"crossref","unstructured":"Asma Ben Abacha et al. 2024. Medec: A benchmark for medical error detection and correction in clinical notes. arXiv preprint arXiv:2412.19260.","DOI":"10.18653\/v1\/2025.findings-acl.1159"},{"key":"e_1_3_1_7_2","first-page":"2268","article-title":"AVATAR: A Parallel Corpus for Java-Python Program Translation","author":"Ahmad Wasi","year":"2023","unstructured":"Wasi Ahmad, et al. 2023. AVATAR: A Parallel Corpus for Java-Python Program Translation. Findings of the Association for Computational Linguistics: ACL 2023, 2268\u20132281.","journal-title":"Findings of the Association for Computational Linguistics: ACL 2023"},{"key":"e_1_3_1_8_2","doi-asserted-by":"crossref","unstructured":"Toufique Ahmed Premkumar Devanbu. 2022. Few-shot training LLMs for project-specific code-summarization. Proceedings of the 37th IEEE\/ACM International Conference on Automated Software Engineering 1\u20135.","DOI":"10.1145\/3551349.3559555"},{"key":"e_1_3_1_9_2","unstructured":"Apache Maven. 2024. Apache Maven - Java Project Management and Comprehension Tool. https:\/\/maven.apache.org\/. Accessed on 10\/31\/2024."},{"key":"e_1_3_1_10_2","unstructured":"Baidu Qianfan. 2024. Baidu Qianfan Platform - Open API Interface. https:\/\/qianfan.cloud.baidu.com\/. Accessed on 10\/31\/2024."},{"key":"e_1_3_1_11_2","unstructured":"Mark Chen et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374."},{"key":"e_1_3_1_12_2","unstructured":"Cluplus. 2024. Cluplus Reference - Standard C++ Library Reference. https:\/\/cluplus.com\/reference\/. Accessed on 10\/31\/2024."},{"issue":"8","key":"e_1_3_1_13_2","first-page":"725","article-title":"Recommended steps for thematic synthesis in software engineering","volume":"53","author":"Cruz Daniela S.","year":"2011","unstructured":"Daniela S. Cruz, Tore Dyba. 2011. Recommended steps for thematic synthesis in software engineering. Information and Software Technology, 53, 8, 725\u2013841.","journal-title":"Information and Software Technology"},{"key":"e_1_3_1_14_2","unstructured":"Deepseek. 2024. Deepseek - Official API Interface. https:\/\/www.deepseek.com\/zh. Accessed on 10\/27\/2024."},{"key":"e_1_3_1_15_2","unstructured":"DeepSeek Team. 2024. DeepSeek. https:\/\/www.deepseek.com\/. Accessed on 10\/27\/2024."},{"key":"e_1_3_1_16_2","first-page":"205","article-title":"A search-and-fill strategy to code generation for complex software requirements","volume":"177","author":"Dong Yukun","year":"2025","unstructured":"Yukun Dong, et al. 2025. A search-and-fill strategy to code generation for complex software requirements. Information and Software Technology, 177, 205\u2013107584.","journal-title":"Information and Software Technology"},{"key":"e_1_3_1_17_2","doi-asserted-by":"crossref","unstructured":"Xueying Du et al. 2024. Evaluating large language models in class-level code generation. Proceedings of the IEEE\/ACM 46th International Conference on Software Engineering 1\u201313.","DOI":"10.1145\/3597503.3639219"},{"key":"e_1_3_1_18_2","unstructured":"Abhimanyu Dube et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783."},{"key":"e_1_3_1_19_2","unstructured":"Hasan Ferit Eniser et al. 2024. Towards Translating Real-World Code with LLMs: A Study of Translating to Rust. arXiv preprint arXiv:2405.11514."},{"key":"e_1_3_1_20_2","unstructured":"Jean-Marie Favre. 1995. The CPP Paradox. Proceedings of the European Workshop on Software Maintenance."},{"key":"e_1_3_1_21_2","unstructured":"Geeks for Geeks. 2024. Geeks for Geeks - A Computer Science Portal for Geeks. https:\/\/www.geeksforgeeks.org\/. Accessed on 10\/31\/2024."},{"key":"e_1_3_1_22_2","unstructured":"GitHub Octoverse. 2024. The top programming languages | The State of the Octoverse. https:\/\/octoverse.github.com\/2022\/top-programming-languages?trk=em660b43a71404a918183182941226. Accessed on 10\/27\/2024."},{"key":"e_1_3_1_23_2","unstructured":"Google. 2024. GoogleTest - C++ Testing Framework. https:\/\/google.github.io\/googletest\/. Accessed on 10\/31\/2024."},{"key":"e_1_3_1_24_2","unstructured":"Ali Reza Ibrahimzada et al. 2024. Repository-level compositional code translation and validation. arXiv preprint arXiv:2410.24117."},{"key":"e_1_3_1_25_2","unstructured":"AtCoder. n.d.. AtCoder. https:\/\/atcoder.jp\/. Online; accessed 2025-02-22."},{"key":"e_1_3_1_26_2","doi-asserted-by":"crossref","unstructured":"Siyuan Jiang et al. 2024. aiXcoder-7B: A Lightweight and Effective Large Language Model for Code Completion. arXiv preprint arXiv:2410.13187.","DOI":"10.32388\/ATAHD0"},{"key":"e_1_3_1_27_2","doi-asserted-by":"crossref","unstructured":"Mingsheng Jiao et al. 2023. On the evaluation of neural code translation: Taxonomy and benchmark. Proceedings of the IEEE\/ACM International Conference on Automated Software Engineering (ASE) 1529\u20131541.","DOI":"10.1109\/ASE56229.2023.00114"},{"key":"e_1_3_1_28_2","unstructured":"JUnit 5. 2024. JUnit 5 - Testing Framework for Java. https:\/\/junit.org\/junit5\/. Accessed on 10\/31\/2024."},{"key":"e_1_3_1_29_2","doi-asserted-by":"crossref","unstructured":"Mohammad Abdullah Matin Khan et al. 2024. XCodeeval: An execution-based large scale multilingual multitask benchmark for code understanding generation translation and retrieval. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) 6766\u20136805.","DOI":"10.18653\/v1\/2024.acl-long.367"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/3690635"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","unstructured":"Jia Li et al. 2024. EvoCodeBench: An Evolving Code Generation Benchmark Aligned with Real-World Code Repositories. CoRR abs\/2404.00599. https:\/\/doi.org\/10.48550\/arXiv.2404.00599 10.48550\/arXiv.2404.00599.","DOI":"10.48550\/arXiv.2404.00599"},{"key":"e_1_3_1_32_2","doi-asserted-by":"crossref","unstructured":"Jia Li et al. 2024. DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositories. arXiv preprint arXiv:2405.19856.","DOI":"10.18653\/v1\/2024.findings-acl.214"},{"key":"e_1_3_1_33_2","doi-asserted-by":"crossref","unstructured":"Jia Li et al. 2024. DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositories. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics.","DOI":"10.18653\/v1\/2024.findings-acl.214"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3675395"},{"key":"e_1_3_1_35_2","article-title":"StarCoder: May the Source Be With You!","author":"Li R.","year":"2023","unstructured":"R. Li, et al. 2023. StarCoder: May the Source Be With You!. Transactions on Machine Learning Research.","journal-title":"Transactions on Machine Learning Research"},{"key":"e_1_3_1_36_2","unstructured":"Zhen Li et al. 2022. Towards robust code authorship attribution via automatic coding style transformation. Proceedings of the 44th International Conference on Software Engineering 1906\u20131918."},{"key":"e_1_3_1_37_2","unstructured":"Aixin Liu et al. 2024. Deepseek-v2: A strong economical and efficient mixture-of-experts language model. arXiv preprint arXiv:2405.04434."},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","unstructured":"Fang Liu Jia Li Lin Zhang. 2023. Syntax and Domain Aware Model for Unsupervised Program Translation. Proceedings of the IEEE\/ACM 45th International Conference on Software Engineering (ICSE) 755\u2013767. https:\/\/doi.org\/10.1109\/ICSE48619.2023.00072 10.1109\/ICSE48619.2023.00072.","DOI":"10.1109\/ICSE48619.2023.00072"},{"key":"e_1_3_1_39_2","unstructured":"Fang Liu et al. 2024. Exploring and evaluating hallucinations in llm-powered code generation. arXiv preprint arXiv:2404.00971."},{"key":"e_1_3_1_40_2","article-title":"Is your code generated by chatgpt really correct?","volume":"36","author":"Liu Jiawei","year":"2024","unstructured":"Jiawei Liu, et al. 2024. Is your code generated by chatgpt really correct?. Advances in Neural Information Processing Systems, 36.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_41_2","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1162\/tacl_a_00638","article-title":"Lost in the middle: How language models use long contexts","volume":"12","author":"Li Nelson F.","year":"2024","unstructured":"Nelson F. Li, et al. 2024. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12, 157\u2013173.","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"e_1_3_1_42_2","unstructured":"Llama Team. 2024. Llama 3. https:\/\/www.llama.com\/. Accessed on 10\/27\/2024."},{"key":"e_1_3_1_43_2","unstructured":"Natural Language Toolkit (NLTK). 2024. NLTK -- Natural Language Toolkit. https:\/\/www.nltk.org\/. Accessed on 10\/31\/2024."},{"key":"e_1_3_1_44_2","doi-asserted-by":"crossref","unstructured":"Anh Tuan Nguyen Tung Thanh Nguyen Tien N Nguyen. 2015. Divide-and-conquer approach for multi-phase statistical migration for source code. Proceedings of the 30th IEEE\/ACM International Conference on Automated Software Engineering (ASE) 585\u2013596.","DOI":"10.1109\/ASE.2015.74"},{"key":"e_1_3_1_45_2","unstructured":"Vikram Nitin Baishakhi Ray. 2024. SpecTra: Enhancing the Code Translation Ability of Language Models by Generating Multi-Modal Specifications. arXiv preprint arXiv:2405.18574."},{"key":"e_1_3_1_46_2","unstructured":"NuGet. 2024. NuGet - Package Manager for .NET. https:\/\/www.nuget.org\/. Accessed on 10\/31\/2024."},{"key":"e_1_3_1_47_2","unstructured":"NVIDIA. 2024. NVIDIA Open API Interface. https:\/\/build.nvidia.com\/explore\/discover. Accessed on 10\/31\/2024."},{"key":"e_1_3_1_48_2","unstructured":"OpenAI. 2024. ChatGPT - AI Chatbot by OpenAI. https:\/\/openai.com\/index\/chatgpt\/. Accessed on 10\/31\/2024."},{"key":"e_1_3_1_49_2","unstructured":"Oracle. 2024. Java SE 8 Documentation. https:\/\/docs.oracle.com\/javase\/8\/docs\/. Accessed on 10\/31\/2024."},{"key":"e_1_3_1_50_2","unstructured":"Guangsheng Ou et al. 2024. Repository-level Code Translation Benchmark Targeting Rust. arXiv preprint arXiv:2411.13990."},{"key":"e_1_3_1_51_2","doi-asserted-by":"crossref","unstructured":"Yicheng Ouyang et al. 2024. Benchmarking Automated Program Repair: An Extensive Study on Both Real-World and Artificial Bugs. Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis 440\u2013452.","DOI":"10.1145\/3650212.3652140"},{"key":"e_1_3_1_52_2","unstructured":"Jialing Pan et al. 2023. SteloCoder: A Decoder-Only LLM for Multi-Language to Python Code Translation. arXiv preprint arXiv:2310.15539."},{"key":"e_1_3_1_53_2","doi-asserted-by":"crossref","unstructured":"Sobojev Raju Pavuluri et al. 2024. Lost in translation: A study of bugs introduced by large language models while translating code. Proceedings of the IEEE\/ACM 46th International Conference on Software Engineering 1\u201313.","DOI":"10.1145\/3597503.3639226"},{"key":"e_1_3_1_54_2","unstructured":"Ruchir Puri et al. n.d.. CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Tasks."},{"key":"e_1_3_1_55_2","unstructured":"PyPDF2 Documentation. 2024. Welcome to PyPDF2 \u2014 PyPDF2 documentation. https:\/\/pypdf2.readthedocs.io\/en\/3.x\/. Accessed on 10\/31\/2024."},{"key":"e_1_3_1_56_2","unstructured":"Python Documentation. 2024. Python Documentation - Official Python Language Reference. https:\/\/docs.python.org. Accessed on 10\/31\/2024."},{"key":"e_1_3_1_57_2","unstructured":"Python Documentation. 2024. tkinter \u2014 Python interface to Tcl\/Tk \u2014 Python 3.13.0 documentation. https:\/\/docs.python.org\/3\/library\/tkinter.html. Accessed on 10\/31\/2024."},{"key":"e_1_3_1_58_2","unstructured":"Python Documentation. 2024. unittest \u2014 Python\u2019s Unit Testing Framework. https:\/\/docs.python.org\/3\/library\/unittest.html. Accessed on 10\/31\/2024."},{"key":"e_1_3_1_59_2","unstructured":"Quoc Le. 2024. Quora - Q&A Platform. https:\/\/www.quora.com\/. Accessed on 10\/31\/2024."},{"key":"e_1_3_1_60_2","unstructured":"Achut Reddy et al. 2000. JavaTM Coding Style Guide. Sun Microsystems."},{"key":"e_1_3_1_61_2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2006.03511"},{"key":"e_1_3_1_62_2","doi-asserted-by":"publisher","unstructured":"Baptiste Roziere et al. 2021. Leveraging Automated Unit Tests for Unsupervised Code Translation. International Conference on Learning Representations. https:\/\/doi.org\/10.48550\/arXiv.2110.06773 10.48550\/arXiv.2110.06773.","DOI":"10.48550\/arXiv.2110.06773"},{"key":"e_1_3_1_63_2","unstructured":"Baptiste Roziere et al. 2023. Code Llama: Open Foundation Models for Code. arXiv preprint arXiv:2308.12950."},{"key":"e_1_3_1_64_2","unstructured":"Stack Overflow. 2024. Stack Overflow - Programming Q&A Site. https:\/\/stackoverflow.com\/. Accessed on 10\/31\/2024."},{"key":"e_1_3_1_65_2","unstructured":"Herb Sutter Andrei Alexandrescu. 2004. C++ Coding Standards: 101 Rules Guidelines and Best Practices. Pearson Education."},{"key":"e_1_3_1_66_2","unstructured":"Marc Szafraniec et al. 2023. Code Translation with Compiler Representations. The Eleventh International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=XamUEU3eNSQ."},{"key":"e_1_3_1_67_2","unstructured":"Qingxiao Tao et al. 2024. Unraveling the Potential of Large Language Models in Code Translation. arXiv preprint arXiv:2401.09812."},{"key":"e_1_3_1_68_2","unstructured":"CodeGemma Team. 2024. Codegemma: Open code models based on gemma. arXiv preprint arXiv:2406.11409."},{"key":"e_1_3_1_69_2","unstructured":"Gemma Team. 2024. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295."},{"key":"e_1_3_1_70_2","unstructured":"Hugo Touvron et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288."},{"key":"e_1_3_1_71_2","doi-asserted-by":"crossref","unstructured":"Wei Wu Yann-Ga\u00ebl Gu\u00e9h\u00e9neuc Giuliano Antoniol Miryung Kim. 2010. Aura: a hybrid approach to identify framework evolution. Proceedings of the 32nd ACM\/IEEE International Conference on Software Engineering - Volume 1 325\u2013334.","DOI":"10.1145\/1806799.1806848"},{"key":"e_1_3_1_72_2","first-page":"1482","article-title":"Automated program repair in the era of large pre-trained language models","volume":"49","author":"Xia Chunqiu Steven","year":"2023","unstructured":"Chunqiu Steven Xia, Yuxiang Wei, Lingming Zhang. 2023. Automated program repair in the era of large pre-trained language models. IEEE\/ACM Transactions on Software Engineering, 49, 1482\u20131494.","journal-title":"IEEE\/ACM Transactions on Software Engineering"},{"key":"e_1_3_1_73_2","unstructured":"Pengyu Xue et al. 2024. Exploring and Lifting the Robustness of LLM-powered Automated Program Repair with Metamorphic Testing. arXiv preprint arXiv:2410.07516."},{"key":"e_1_3_1_74_2","unstructured":"Pengyu Xue et al. 2024. Automated Commit Message Generation with Large Language Models: An Empirical Study and Beyond. arXiv preprint arXiv:2404.14824."},{"key":"e_1_3_1_75_2","unstructured":"Weixiang Yan et al. 2023. Codescope: An execution-based multilingual multitask multidimensional benchmark for evaluating llms on code understanding and generation. arXiv preprint arXiv:2311.08588."},{"key":"e_1_3_1_76_2","unstructured":"Weixiang Yan et al. 2023. Codetranscean: A comprehensive multilingual benchmark for code translation. arXiv preprint arXiv:2310.04951."},{"key":"e_1_3_1_77_2","unstructured":"Aidan ZH et al. 2024. Vert: Verified equivalent rust transpilation with few-shot learning. arXiv preprint arXiv:2404.18522."},{"key":"e_1_3_1_78_2","first-page":"1","article-title":"A multi-modal transformer-based code summarization approach for smart contracts","volume":"2","author":"Yang Zhen","year":"2021","unstructured":"Zhen Yang, Jacky Keung, Xiao Yu, Xiaodong Gu, Zhengyuan Wei, Xiaoxue Ma, Miao Zhang. 2021. A multi-modal transformer-based code summarization approach for smart contracts. Proceedings of the ACM on Software Engineering, 2, ISSTA, ISSTA063, 1\u201312. Publication date: July 2025.","journal-title":"Proceedings of the ACM on Software Engineering"},{"key":"e_1_3_1_79_2","doi-asserted-by":"crossref","first-page":"1585","DOI":"10.1145\/3660778","article-title":"Exploring and unleashing the power of large language models in automated code translation","author":"Yang Zhen","year":"2024","unstructured":"Zhen Yang, Fang Liu, Zhongxing Yu, Jacky Wei Keung, Jia Li, Shuo Liu, Yifan Hong, Xiaoxue Ma, Zhi Jin, Ge Li. 2024. Exploring and unleashing the power of large language models in automated code translation. Proceedings of the ACM on Software Engineering, FSE, 1585\u20131608.","journal-title":"Proceedings of the ACM on Software Engineering"},{"key":"e_1_3_1_80_2","doi-asserted-by":"crossref","unstructured":"Xin Yin Chao Ni Tien N. Nguyen Shaohua Wang Xiaohu Yang. 2024. Rectifier: Code translation with corrector via llms. arXiv preprint arXiv:2407.07472.","DOI":"10.22541\/au.174548366.63392632\/v1"},{"key":"e_1_3_1_81_2","doi-asserted-by":"crossref","unstructured":"Hao Yu et al. 2024. Codereval: A benchmark of pragmatic code generation with generative pre-trained models. Proceedings of the 46th IEEE\/ACM International Conference on Software Engineering 1\u201312.","DOI":"10.1145\/3597503.3623316"},{"key":"e_1_3_1_82_2","unstructured":"Zhiqiang Yuan et al. 2024. TRANSAGENT: An LLM-Based Multi-Agent System for Code Translation. arXiv preprint arXiv:2409.19894."},{"key":"e_1_3_1_83_2","doi-asserted-by":"crossref","unstructured":"Zejun Zhang et al. 2024. Hard to Read and Understand Pythonic Idioms? Delidiom and Explain Them in Non-Idiomatic Equivalent Code. Proceedings of the IEEE\/ACM 46th International Conference on Software Engineering 1\u201312.","DOI":"10.1145\/3597503.3639101"},{"key":"e_1_3_1_84_2","doi-asserted-by":"crossref","unstructured":"Yang Zhao et al. 2024. Deciphering the Impact of Pretraining Data on Large Language Models through Machine Unlearning. arXiv preprint arXiv:2402.11537.","DOI":"10.18653\/v1\/2024.findings-acl.559"}],"container-title":["Proceedings of the ACM on Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3728940","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,23]],"date-time":"2026-08-23T13:48:16Z","timestamp":1787492896000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3728940"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,22]]},"references-count":83,"journal-issue":{"issue":"ISSTA","published-print":{"date-parts":[[2025,6,22]]}},"alternative-id":["10.1145\/3728940"],"URL":"https:\/\/doi.org\/10.1145\/3728940","relation":{},"ISSN":["2994-970X"],"issn-type":[{"value":"2994-970X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,22]]}}}