{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,2]],"date-time":"2026-10-02T10:35:37Z","timestamp":1790937337709,"version":"4.1.0"},"reference-count":72,"publisher":"Association for Computing Machinery (ACM)","issue":"FSE","license":[{"start":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T00:00:00Z","timestamp":1782777600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62502283"],"award-info":[{"award-number":["62502283"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["U24B20149"],"award-info":[{"award-number":["U24B20149"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100007129","name":"Natural Science Foundation of Shandong Province","doi-asserted-by":"publisher","award":["ZR2024QF093"],"award-info":[{"award-number":["ZR2024QF093"]}],"id":[{"id":"10.13039\/501100007129","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Young Talent of Lifting Engineering for Science and Technology in Shandong, China","award":["SDAST2025QTB031"],"award-info":[{"award-number":["SDAST2025QTB031"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Softw. Eng."],"published-print":{"date-parts":[[2026,6,30]]},"abstract":"<jats:p>In recent years, Large Language Models (LLMs) have been widely studied in the code translation field on the method, class, and even repository levels. However, most of these benchmarks are limited in terms of Third-Party Library (TPL) categories and scales, making TPL-related errors hard to expose and hindering the development of targeted solutions. Considering the high dependence (over 90%) on TPLs in practical programming, demystifying and analyzing LLMs' code translation performance involving various TPLs becomes imperative.   \nTo address this gap, we construct TransLibEval, the first benchmark dedicated to library-centric code translation. It consists of 200 real-world tasks across Python, Java, and C++, each explicitly involving TPLs from diverse categories such as data processing, machine learning, and web development, with comprehensive dependency coverage and high-coverage test suites. We evaluate seven recent LLMs of commercial, general, and code-specialized families under six translation strategies of three categories: Direct, IR-guided, and Retrieval-augmented. Experimental results show a dramatic performance drop compared with library-free settings (average CA decline over 60%), while diverse strategies demonstrate heterogeneous advantages. Furthermore, we analyze 4,831 failed cases from GPT-4o, one of the State-of-the-Art (SOTA) LLMs, revealing numerous third-party reference errors that were obscured previously. These findings highlight the unique challenges of library-centric translation and provide practical guidance for improving TPL-aware code intelligence.<\/jats:p>","DOI":"10.1145\/3797137","type":"journal-article","created":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T17:06:14Z","timestamp":1782839174000},"page":"920-942","source":"Crossref","is-referenced-by-count":2,"title":["TransLibEval: Demystify Large Language Models\u2019 Capability in Third-Party Library-Targeted Code Translation"],"prefix":"10.1145","volume":"3","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-3395-9575","authenticated-orcid":false,"given":"Pengyu","family":"Xue","sequence":"first","affiliation":[{"name":"Shandong University, School of Computer Science and Technology, Qingdao, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-7047-1981","authenticated-orcid":false,"given":"Kunwu","family":"Zheng","sequence":"additional","affiliation":[{"name":"Shandong University, School of Computer Science and Technology, Qingdao, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0670-4538","authenticated-orcid":false,"given":"Zhen","family":"Yang","sequence":"additional","affiliation":[{"name":"Shandong University, School of Computer Science and Technology, Qingdao, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-1351-0565","authenticated-orcid":false,"given":"Yifei","family":"Pei","sequence":"additional","affiliation":[{"name":"Shandong University, School of Computer Science and Technology, Qingdao, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-7624-156X","authenticated-orcid":false,"given":"Linhao","family":"Wu","sequence":"additional","affiliation":[{"name":"Shandong University, School of Computer Science and Technology, Qingdao, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-1307-3981","authenticated-orcid":false,"given":"Jiahui","family":"Dong","sequence":"additional","affiliation":[{"name":"Shandong University, School of Computer Science and Technology, Qingdao, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9082-3208","authenticated-orcid":false,"given":"Xiapu","family":"Luo","sequence":"additional","affiliation":[{"name":"Hong Kong Polytechnic University, Hong Kong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2563-083X","authenticated-orcid":false,"given":"Yan","family":"Xiao","sequence":"additional","affiliation":[{"name":"Sun Yat-sen University, Shenzhen, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-5722-1990","authenticated-orcid":false,"given":"Fei","family":"Liu","sequence":"additional","affiliation":[{"name":"Shandong University, School of Computer Science and Technology, Qingdao, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-4791-3770","authenticated-orcid":false,"given":"Yuxuan","family":"Zhang","sequence":"additional","affiliation":[{"name":"Shandong University, School of Computer Science and Technology, Qingdao, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-8861-8907","authenticated-orcid":false,"given":"Xiran","family":"Lyu","sequence":"additional","affiliation":[{"name":"Shandong University, School of Computer Science and Technology, Qingdao, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-8852-1215","authenticated-orcid":false,"given":"Xianhang","family":"Li","sequence":"additional","affiliation":[{"name":"Shandong University, School of Computer Science and Technology, Qingdao, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-9896-1647","authenticated-orcid":false,"given":"Xuanyu","family":"Zhu","sequence":"additional","affiliation":[{"name":"Shandong University, School of Computer Science and Technology, Qingdao, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-4153-6774","authenticated-orcid":false,"given":"Chengyi","family":"Wang","sequence":"additional","affiliation":[{"name":"Shandong University, School of Computer Science and Technology, Qingdao, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,6,30]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"https:\/\/codeforces.com\/ [Online","year":"2025","unstructured":"Codeforces. https:\/\/codeforces.com\/ [Online; accessed 2025-02-21]."},{"key":"e_1_2_1_2_1","volume-title":"https:\/\/api.stackexchange.com\/docs [Online","author":"Stack Exchange","year":"2025","unstructured":"Stack Exchange API. https:\/\/api.stackexchange.com\/docs [Online; accessed 2025-09-04]."},{"key":"e_1_2_1_3_1","volume-title":"https:\/\/openai.com\/ [Online","author":"AI.","year":"2025","unstructured":"2025. OpenAI. https:\/\/openai.com\/ [Online; accessed 2025-02-17]."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.findings-acl.143"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00061"},{"key":"e_1_2_1_6_1","unstructured":"Apache Maven. 2024. Maven -Java Project Management and Comprehension Tool. https:\/\/maven.apache.org\/ (Accessed on 10\/31\/2024)."},{"key":"e_1_2_1_7_1","unstructured":"Baidu Qianfan. 2024. Baidu Qianfan Platform -Open API Interface. https:\/\/qianfan.cloud.baidu.com\/ (Accessed on 10\/31\/2024)."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2107.03374"},{"key":"e_1_2_1_10_1","unstructured":"Deepseek. 2024. Deepseek -Official API Interface. https:\/\/www.deepseek.com\/zh (Accessed on 10\/31\/2024)."},{"key":"e_1_2_1_11_1","unstructured":"DeepSeek Team. 2024. DeepSeek. https:\/\/www.deepseek.com\/. (Accessed on 10\/27\/2024)."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/1111320.1111064"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639219"},{"key":"e_1_2_1_14_1","volume-title":"PyPI Download Stats. URL: https:\/\/pypistats. org\/top","author":"Flynn Christopher","year":"2024","unstructured":"Christopher Flynn. 2024. PyPI Download Stats. URL: https:\/\/pypistats. org\/top (2024)."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2409.16656"},{"key":"e_1_2_1_16_1","unstructured":"GitHub Octoverse. 2024. The top programming languages | The State of the Octoverse. https:\/\/octoverse.github.com\/ 2022\/top-programming-languages?src_trk=em6690b43a717440.491810831822941226. (Accessed on 10\/27\/2024)."},{"key":"e_1_2_1_17_1","unstructured":"Google. 2024. GoogleTest -C++ Testing Framework. https:\/\/google.github.io\/googletest\/ (Accessed on 10\/31\/2024)."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-022-10131-8"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3729379"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3672456"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","unstructured":"Junda Jiao Yelin Liu Zhuo Liu Yuxin Wang Yuxin Lu Yuchen Yao Binyuan Hui Ge Li and Zhi Jin. 2024. G- TransEval: A Granular-grained Code Translation Benchmark for General-purpose Languages. arXiv:2403.01831 [cs.SE] doi:10.48550\/ARXIV.2403.01831 10.48550\/ARXIV.2403.01831","DOI":"10.48550\/ARXIV.2403.01831"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE56229.2023.00114"},{"key":"e_1_2_1_23_1","unstructured":"JUnit 5. 2024. JUnit 5 -Testing Framework for Java. https:\/\/junit.org\/junit5\/ (Accessed on 10\/31\/2024)."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3690635"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","unstructured":"Aixin Liu Bei Feng Bin Wang Bingxuan Wang Bo Liu Chenggang Zhao Chengqi Dengr Chong Ruan Damai Dai Daya Guo et al. 2024. Deepseek-v2: A strong economical and efficient mixture-of-experts language model. arXiv preprint arXiv:2405.04434 (2024). doi:10.48550\/arXiv.2405.04434 10.48550\/arXiv.2405.04434","DOI":"10.48550\/arXiv.2405.04434"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2404.00971"},{"key":"e_1_2_1_28_1","unstructured":"Llama Team. 2024. Llama 3. https:\/\/www.llama.com\/. (Accessed on 10\/27\/2024)."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv"},{"key":"e_1_2_1_30_1","unstructured":"NuGet. 2024. NuGet -Package Manager for .NET. https:\/\/www.nuget.org\/ (Accessed on 10\/31\/2024)."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2411.13990"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2503.18305"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2310"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639226"},{"key":"e_1_2_1_35_1","volume-title":"d.]. PEP 8 -Style Guide for Python Code | peps.python.org. https:\/\/peps.python.org\/pep-0008\/#descriptivenaming-styles [Online","year":"2025","unstructured":"PEP8. [n. d.]. PEP 8 -Style Guide for Python Code | peps.python.org. https:\/\/peps.python.org\/pep-0008\/#descriptivenaming-styles [Online; accessed 2025-09-01]."},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/2.876288"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2105.12655"},{"key":"e_1_2_1_38_1","unstructured":"Python Documentation. 2024. unittest -Python's Unit Testing Framework. https:\/\/docs.python.org\/3\/library\/unittest. html (Accessed on 10\/31\/2024)."},{"key":"e_1_2_1_39_1","unstructured":"Achut Reddy et al. 2000. Java\u2122 coding style guide. Sun MicroSystems (2000)."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","unstructured":"Debarshi Roy and Marius Litoiu. 2023. A Survey on Evaluation Metrics for Large Language Models of Code. arXiv:2305.07439 [cs.SE] doi:10.48550\/ARXIV.2305.07439 10.48550\/ARXIV.2305.07439","DOI":"10.48550\/ARXIV.2305.07439"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.5555\/3495724.3497454"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2006.03511"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2110.06773"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2308.12950"},{"key":"e_1_2_1_45_1","unstructured":"Stack Overflow. 2024. Stack Overflow -Programming Q&A Site. https:\/\/stackoverflow.com\/ (Accessed on 10\/31\/2024)."},{"key":"e_1_2_1_46_1","unstructured":"Herb Sutter and Andrei Alexandrescu. 2004. C++ coding standards: 101 rules guidelines and best practices. Pearson Education."},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/APSEC65559.2024.00046"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","unstructured":"Zhiruo Tao Yida Zhai Zifeng Wang Kailong Wang Hong-Ning Dai Yong Li and Zhenhong Chen. 2023. Polyglot Human-Level Programming with Large Language Models. arXiv:2311.08225 [cs.CL] doi:10.48550\/ARXIV.2311.08225 10.48550\/ARXIV.2311.08225","DOI":"10.48550\/ARXIV.2311.08225"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2406.11409"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2403.08295"},{"key":"e_1_2_1_51_1","unstructured":"Qwen Team. 2025. Qwen. https:\/\/qwen.ai\/home. https:\/\/qwen.ai\/home Accessed: 2025-08-28."},{"key":"e_1_2_1_52_1","volume-title":"TIOBE Programming Community Index. https:\/\/www.tiobe.com\/tiobe-index\/","author":"Software TIOBE","year":"2025","unstructured":"TIOBE Software. 2025. TIOBE Programming Community Index. https:\/\/www.tiobe.com\/tiobe-index\/. Accessed: July 2025."},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2025.3562025"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","unstructured":"Hugo Touvron Louis Martin Kevin Stone Peter Albert Amjad Almahairi Yasmine Babaei Nikolay Bashlykov Soumya Batra Prajjwal Bhargava Shruti Bhosale et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 (2023). doi:10.48550\/arXiv.2307.09288 10.48550\/arXiv.2307.09288","DOI":"10.48550\/arXiv.2307.09288"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2504.14852"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2508.04295"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","unstructured":"Yanli Wang Yanlin Wang Suiquan Wang Daya Guo Jiachi Chen John Grundy Xilin Liu Yuchi Ma Mingzhi Mao Hongyu Zhang et al. 2024. RepoTransBench: A Real-World Benchmark for Repository-Level Code Translation. arXiv preprint arXiv:2412.17744 (2024). doi:10.48550\/arXiv.2412.17744 10.48550\/arXiv.2412.17744","DOI":"10.48550\/arXiv.2412.17744"},{"key":"e_1_2_1_58_1","unstructured":"Peter Wentworth Jeffrey Elkner Allen B Downey and Chris Meyers. 2011. How to think like a computer scientist."},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.19492850"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2504.02017"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/3728940"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2024.3478317"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.findings-emnlp.337"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC52881.2021.00010"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2506.02791"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/3660778"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2407.07472"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3623316"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2409.19894"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1145\/3660783"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2402.11537"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1145\/3728941"}],"container-title":["Proceedings of the ACM on Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3797137","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T18:54:11Z","timestamp":1782845651000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3797137"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,30]]},"references-count":72,"journal-issue":{"issue":"FSE","published-print":{"date-parts":[[2026,6,30]]}},"alternative-id":["10.1145\/3797137"],"URL":"https:\/\/doi.org\/10.1145\/3797137","relation":{},"ISSN":["2994-970X"],"issn-type":[{"value":"2994-970X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,30]]}}}