{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T11:48:15Z","timestamp":1791460095291,"version":"4.3.4"},"reference-count":83,"publisher":"Association for Computing Machinery (ACM)","issue":"5","funder":[{"name":"Key Research and Development Program of Jiangsu Province","award":["BE2021002-2"],"award-info":[{"award-number":["BE2021002-2"]}]},{"DOI":"10.13039\/501100004608","name":"Natural Science Foundation of Jiangsu Province","doi-asserted-by":"crossref","award":["BK20241195"],"award-info":[{"award-number":["BK20241195"]}],"id":[{"id":"10.13039\/501100004608","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Open Project of Key Laboratory of Industry and Information Technology Ministry for Software Fusion Application and Testing Verification, the National Natural Science Foundation of China","award":["62302210 and 62202219"],"award-info":[{"award-number":["62302210 and 62202219"]}]},{"name":"Innovation Projects and Overseas Open Projects of State Key Laboratory for Novel Software Technology","award":["ZZKT2025A12, ZZKT2025B18, ZZKT2025B20, ZZKT2025B22, KFKT2025A17, KFKT2025A19, KFKT2025A20, KFKT2024A02, KFKT2024A13, KFKT2024A14, KFKT2023A09, and KFKT2023A10"],"award-info":[{"award-number":["ZZKT2025A12, ZZKT2025B18, ZZKT2025B20, ZZKT2025B22, KFKT2025A17, KFKT2025A19, KFKT2025A20, KFKT2024A02, KFKT2024A13, KFKT2024A14, KFKT2023A09, and KFKT2023A10"]}]},{"name":"Swiss National Science Foundation through the SNF Project","award":["200021_197227"],"award-info":[{"award-number":["200021_197227"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2026,5,31]]},"abstract":"<jats:p>\n                    Large Language Model-Generated Code (LLMgCode) has become increasingly common in software development. So far LLMgCode has more quality issues than Human-Authored Code (HaCode). It is common for LLMgCode to mix with HaCode in a code change, while the change is signed by only human developers, without being carefully examined. Many automated methods have been proposed to detect LLMgCode from HaCode, in which the perplexity-based method (\n                    <jats:italic toggle=\"yes\">\n                      <jats:sc>Perplexity<\/jats:sc>\n                    <\/jats:italic>\n                    for short) is the state-of-the-art method. However, the efficacy evaluation of\n                    <jats:sc>Perplexity<\/jats:sc>\n                    has focused on detection accuracy. Yet it is unclear whether\n                    <jats:sc>Perplexity<\/jats:sc>\n                    is good enough in a wider range of realistic evaluation settings. To this end, we carry out a family of experiments to compare\n                    <jats:sc>Perplexity<\/jats:sc>\n                    against feature- and pre-training-based methods from three perspectives:\n                    <jats:italic toggle=\"yes\">detection accuracy<\/jats:italic>\n                    ,\n                    <jats:italic toggle=\"yes\">detection speed<\/jats:italic>\n                    , and\n                    <jats:italic toggle=\"yes\">generalization capability<\/jats:italic>\n                    . The experimental results show that\n                    <jats:sc>Perplexity<\/jats:sc>\n                    has the best generalization capability while having limited detection accuracy and detection speed. Based on that, we discuss the strengths and limitations of\n                    <jats:sc>Perplexity<\/jats:sc>\n                    , e.g.,\n                    <jats:sc>Perplexity<\/jats:sc>\n                    is unsuitable for high-level programming languages. Finally, we provide recommendations to improve\n                    <jats:sc>Perplexity<\/jats:sc>\n                    and apply it in practice. As the first large-scale investigation on detecting LLMgCode from HaCode, this article provides a wide range of findings for future improvement.\n                  <\/jats:p>","DOI":"10.1145\/3748506","type":"journal-article","created":{"date-parts":[[2025,7,14]],"date-time":"2025-07-14T19:54:11Z","timestamp":1752522851000},"page":"1-34","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["One Size Does Not Fit All: Investigating Efficacy of Perplexity in Detecting LLM-Generated Code"],"prefix":"10.1145","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0009-0004-5157-1118","authenticated-orcid":false,"given":"Jinwei","family":"Xu","sequence":"first","affiliation":[{"name":"Nanjing University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9159-5331","authenticated-orcid":false,"given":"He","family":"Zhang","sequence":"additional","affiliation":[{"name":"Nanjing University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-8789-4589","authenticated-orcid":false,"given":"Yanjing","family":"Yang","sequence":"additional","affiliation":[{"name":"Nanjing University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0406-2263","authenticated-orcid":false,"given":"Lanxin","family":"Yang","sequence":"additional","affiliation":[{"name":"Nanjing University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-5094-4427","authenticated-orcid":false,"given":"Zeru","family":"Cheng","sequence":"additional","affiliation":[{"name":"Nanjing University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9070-7269","authenticated-orcid":false,"given":"Jun","family":"Lyu","sequence":"additional","affiliation":[{"name":"Nanjing University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0146-5411","authenticated-orcid":false,"given":"Bohan","family":"Liu","sequence":"additional","affiliation":[{"name":"Nanjing University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3263-1275","authenticated-orcid":false,"given":"Xin","family":"Zhou","sequence":"additional","affiliation":[{"name":"Nanjing University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0193-6823","authenticated-orcid":false,"given":"Alberto","family":"Bacchelli","sequence":"additional","affiliation":[{"name":"University of Zurich, Zurich, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1107-7719","authenticated-orcid":false,"given":"Yin Kia","family":"Chiam","sequence":"additional","affiliation":[{"name":"University of Malaya, Kuala Lumpur, Malaysia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4272-7860","authenticated-orcid":false,"given":"Thiam Kian","family":"Chiew","sequence":"additional","affiliation":[{"name":"University of Malaya, Kuala Lumpur, Malaysia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,4,24]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/3243734.3243738"},{"key":"e_1_3_2_3_2","unstructured":"Mayank Agarwal C. Talamadupula Fernando Martinez Stephanie Houde Michael Muller John Richards Steven I. Ross and Justin D. Weisz. 2021. Using document similarity methods to create parallel datasets for code translation. arXiv:2110.05423. Retrieved from https:\/\/arxiv.org\/abs\/2110.05423"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/1134285.1134336"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-023-10380-1"},{"key":"e_1_3_2_6_2","unstructured":"Jacob Austin Augustus Odena Maxwell Nye Maarten Bosma Henryk Michalewski David Dohan Ellen Jiang Carrie Cai Michael Terry Quoc Le et al. 2021. Program synthesis with large language models. arXiv:2108.07732. Retrieved from https:\/\/arxiv.org\/abs\/2108.07732"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/2025113.2025119"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3468606"},{"key":"e_1_3_2_9_2","unstructured":"Rishi Bommasani Drew A. Hudson Ehsan Adeli Russ Altman Simran Arora Sydney von Arx Michael S. Bernstein Jeannette Bohg Antoine Bosselut Emma Brunskill et al. 2021. On the opportunities and risks of foundation models. arXiv:2108.07258. Retrieved from https:\/\/arxiv.org\/abs\/2108.07258"},{"key":"e_1_3_2_10_2","first-page":"1877","volume-title":"Proceedings of the 33rd Annual Conference on Neural Information Processing Systems","author":"Brown Tom","year":"2020","unstructured":"Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D. Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. In Proceedings of the 33rd Annual Conference on Neural Information Processing Systems. Curran Associates, Inc., 1877\u20131901."},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/3605770.3625215"},{"key":"e_1_3_2_12_2","first-page":"32","volume-title":"Proceedings of the 12th Australasian Document Computing Symposium","author":"Burrows Steven","year":"2007","unstructured":"Steven Burrows and Seyed M. M. Tahaghoghi. 2007. Source code authorship attribution using n-grams. In Proceedings of the 12th Australasian Document Computing Symposium. Citeseer, 32\u201339."},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.5555\/2831143.2831160"},{"key":"e_1_3_2_14_2","unstructured":"Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Ponde De Oliveira Pinto Jared Kaplan Harri Edwards Yuri Burda Nicholas Joseph Greg Brockman et al. 2021. Evaluating large language models trained on code. arXiv:2107.03374. Retrieved from https:\/\/arxiv.org\/abs\/2107.03374"},{"issue":"70","key":"e_1_3_2_15_2","first-page":"1","article-title":"Scaling instruction-finetuned language models","volume":"25","author":"Chung Hyung Won","year":"2024","unstructured":"Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2024. Scaling instruction-finetuned language models. Journal of Machine Learning Research 25, 70 (2024), 1\u201353.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3661167.3661221"},{"key":"e_1_3_2_17_2","unstructured":"Copyleaks. 2025. AI Detector By Copyleaks. Retrieved February 15 2025 from https:\/\/copyleaks.com\/codeleaks"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2023.111734"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3183440.3195007"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3672459"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639219"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/QRS62785.2024.00033"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/1806799.1806856"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3544902.3546253"},{"key":"e_1_3_2_25_2","unstructured":"Yujia Fu Peng Liang Amjed Tahir Zengyang Li Mojtaba Shahin Jiaxin Yu and Jinfu Chen. 2023. Security weaknesses of copilot generated code in GitHub. arXiv:2310.02059. Retrieved from https:\/\/arxiv.org\/abs\/2310.02059"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-3019"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/IWPSE.2005.21"},{"key":"e_1_3_2_28_2","unstructured":"GitHub. 2025. Copilot: The AI Editor for Everyone. Retrieved February 15 2025 from https:\/\/github.com\/features\/copilot"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524610.3527892"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/3672445"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jnlssr.2023.08.001"},{"key":"e_1_3_2_32_2","unstructured":"Xinwei Guo Yujun Li Yafeng Peng and Xuetao Wei. 2024. Copyleft for alleviating AIGC copyright dilemma: What-if analysis public perception and implications. arXiv:2402.12216. Retrieved from https:\/\/arxiv.org\/abs\/2402.12216"},{"key":"e_1_3_2_33_2","unstructured":"Craig Hale. 2022. Microsoft Is Being Sued over GitHub Copilot Piracy. Retrieved February 15 2025 from https:\/\/www.techradar.com\/news\/microsoft-is-being-sued-over-github-copilot-piracy"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2022.3158543"},{"key":"e_1_3_2_35_2","first-page":"1","volume-title":"Proceedings of the 12th International Conference on Learning Representations","author":"Hong Sirui","year":"2024","unstructured":"Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, et al. 2024. MetaGPT: Meta programming for multi-agent collaborative framework. In Proceedings of the 12th International Conference on Learning Representations. OpenReview.net, 1\u201328."},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/3643991.3644926"},{"key":"e_1_3_2_37_2","first-page":"1808","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Ippolito Daphne","year":"2019","unstructured":"Daphne Ippolito, Daniel Duckworth, Chris Callison-Burch, and Douglas Eck. 2019. Automatic detection of generated text is easiest when humans are fooled. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. ACL, 1808\u20131822."},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR59073.2023.00082"},{"key":"e_1_3_2_39_2","unstructured":"Juyong Jiang Fan Wang Jiasi Shen Sungju Kim and Sunghun Kim. 2024. A survey on large language models for code generation. arXiv:2406.00515. Retrieved from https:\/\/arxiv.org\/abs\/2406.00515"},{"key":"e_1_3_2_40_2","first-page":"1","volume-title":"Proceedings of the 12th International Conference on Learning Representations. OpenReview.net","author":"Jimenez Carlos E.","year":"2023","unstructured":"Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. 2023. SWE-bench: Can language models resolve real-world GitHub issues? In Proceedings of the 12th International Conference on Learning Representations. OpenReview.net, 1\u201346."},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/ITNG.2007.17"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00179"},{"key":"e_1_3_2_43_2","unstructured":"Fang Liu Yang Liu Lin Shi Houkun Huang Ruifeng Wang Zhen Yang Li Zhang Zhongqi Li and Yuchi Ma. 2024. Exploring and evaluating hallucinations in LLM-powered code generation. arXiv:2404.00971. Retrieved from https:\/\/arxiv.org\/abs\/2404.00971"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2024.3392499"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-011-9170-z"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00181"},{"key":"e_1_3_2_47_2","first-page":"24950","volume-title":"Proceedings of the 40th International Conference on Machine Learning. PMLR","author":"Mitchell Eric","year":"2023","unstructured":"Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn. 2023. DetectGPT: Zero-shot machine-generated text detection using probability curvature. In Proceedings of the 40th International Conference on Machine Learning. PMLR, 24950\u201324962."},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-22137-8_29"},{"key":"e_1_3_2_49_2","first-page":"41386","volume-title":"Proceedings of the 36th Annual Conference on Neural Information Processing Systems","author":"Naik Shounak","year":"2023","unstructured":"Shounak Naik, Rajaswa Patil, Swati Agarwal, and Veeky Baths. 2023. Large language models of code fail at completing code with potential bugs. In Proceedings of the 36th Annual Conference on Neural Information Processing Systems. Curran, 41386\u201341412."},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2024.112059"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1002\/smr.1704"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/RE59067.2024.00038"},{"key":"e_1_3_2_53_2","unstructured":"OpenAI. 2025. GPT-4. Retrieved February 15 2025 from https:\/\/openai.com\/index\/gpt-4\/"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639226"},{"key":"e_1_3_2_55_2","first-page":"754","volume-title":"Proceedings of the 43rd IEEE Symposium on Security and Privacy","author":"Pearce Hammond","year":"2022","unstructured":"Hammond Pearce, Baleegh Ahmad, Benjamin Tan, Brendan Dolan-Gavitt, and Ramesh Karri. 2022. Asleep at the keyboard? Assessing the security of GitHub Copilot\u2019s code contributions. In Proceedings of the 43rd IEEE Symposium on Security and Privacy. IEEE, 754\u2013768."},{"key":"e_1_3_2_56_2","first-page":"2785","volume-title":"Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security","author":"Perry Neil","year":"2024","unstructured":"Neil Perry, Megha Srivastava, Deepak Kumar, and Dan Boneh. 2024. Do users write more insecure code with AI assistants? In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security. ACM, 2785\u20132799."},{"key":"e_1_3_2_57_2","first-page":"1","volume-title":"Proceedings of the 2021 Neural Information Processing Systems Track on Datasets and Benchmarks","author":"Puri Ruchir","year":"2021","unstructured":"Ruchir Puri, David S. Kung, Geert Janssen, Wei Zhang, Giacomo Domeniconi, Vladimir Zolotov, Julian Dolby, Jie Chen, Mihir Choudhury, Lindsey Decker, et al. 2021. CodeNet: A large-scale AI for code dataset for learning a diversity of coding tasks. In Proceedings of the 2021 Neural Information Processing Systems Track on Datasets and Benchmarks. Curran, 1\u201313."},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/1985793.1985860"},{"key":"e_1_3_2_59_2","first-page":"2205","volume-title":"Proceedings of the 32nd USENIX Security Symposium","author":"Sandoval Gustavo","year":"2023","unstructured":"Gustavo Sandoval, Hammond Pearce, Teo Nys, Ramesh Karri, Siddharth Garg, and Brendan Dolan-Gavitt. 2023. Lost at C: A user study on the security implications of large language model code assistants. In Proceedings of the 32nd USENIX Security Symposium. USENIX, 2205\u20132222."},{"key":"e_1_3_2_60_2","first-page":"1559","volume-title":"Proceedings of the 30th USENIX Security Symposium","author":"Schuster Roei","year":"2021","unstructured":"Roei Schuster, Congzheng Song, Eran Tromer, and Vitaly Shmatikov. 2021. You autocomplete me: Poisoning vulnerabilities in neural code completion. In Proceedings of the 30th USENIX Security Symposium. USENIX, 1559\u20131575."},{"key":"e_1_3_2_61_2","unstructured":"Inbal Shani and GitHub Staff. 2023. Survey Reveals AI\u2019s Impact on the Developer Experience. Retrieved February 15 2025 from https:\/\/github.blog\/news-insights\/research\/survey-reveals-ais-impact-on-the-developer-experience\/"},{"key":"e_1_3_2_62_2","first-page":"145","volume-title":"Proceedings of the 15th Australasian Computing Education Conference","author":"Sheard Judithe","year":"2013","unstructured":"Judithe Sheard, Angela Carbone, Donald Chinn, Tony Clear, Malcolm Corney, Daryl D\u2019Souza, Joel Fenwick, James Harland, Mikko-Jussi Laakso, Donna Teague, et al. 2013. How difficult are exams? A framework for assessing the complexity of introductory programming exams. In Proceedings of the 15th Australasian Computing Education Conference. Australian Computer Society, 145\u2013154."},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE55347.2025.00005"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1145\/3691621.3694934"},{"key":"e_1_3_2_65_2","unstructured":"Irene Solaiman Miles Brundage Jack Clark Amanda Askell Ariel Herbert-Voss Jeff Wu Alec Radford Gretchen Krueger Jong Wook Kim Sarah Kreps et al. 2019. Release strategies and the social impacts of language models. arXiv:1908.09203. Retrieved from https:\/\/arxiv.org\/abs\/1908.09203"},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1037\/11491-005"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE55347.2025.00040"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.findings-emnlp.827"},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE55347.2025.00064"},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1145\/3650212.3680347"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884852"},{"key":"e_1_3_2_72_2","first-page":"1","article-title":"Emergent abilities of large language models","author":"Wei Jason","year":"2022","unstructured":"Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. 2022. Emergent abilities of large language models. Transactions on Machine Learning Research (2022), 1\u201330.","journal-title":"Transactions on Machine Learning Research"},{"key":"e_1_3_2_73_2","unstructured":"Shijie Wu Ozan Irsoy Steven Lu Vadim Dabravolski Mark Dredze Sebastian Gehrmann Prabhanjan Kambadur David Rosenberg and Gideon Mann. 2023. BloombergGPT: A large language model for finance. arXiv:2303.17564. Retrieved from https:\/\/arxiv.org\/abs\/2303.17564"},{"issue":"91","key":"e_1_3_2_74_2","first-page":"1","article-title":"Distinguishing LLM-generated from human-written code by contrastive learning","volume":"34","author":"Xu Xiaodan","year":"2025","unstructured":"Xiaodan Xu, Chao Ni, Xinrong Guo, Shaoxuan Liu, Xiaoya Wang, Kui Liu, and Xiaohu Yang. 2025. Distinguishing LLM-generated from human-written code by contrastive learning. ACM Transactions on Software Engineering and Methodology 34, 91 (2025), 1\u201331.","journal-title":"ACM Transactions on Software Engineering and Methodology"},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v38i21.30361"},{"key":"e_1_3_2_76_2","doi-asserted-by":"publisher","DOI":"10.1145\/2025113.2025121"},{"key":"e_1_3_2_77_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3623316"},{"key":"e_1_3_2_78_2","unstructured":"Xiao Yu Lei Liu Xing Hu Jacky Wai Keung Jin Liu and Xin Xia 2024. Where are large language models for code generation on GitHub? arXiv:2406.19544. Retrieved from https:\/\/arxiv.org\/abs\/2406.19544"},{"key":"e_1_3_2_79_2","first-page":"40373","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Yu Zhiyuan","year":"2023","unstructured":"Zhiyuan Yu, Yuhao Wu, Ning Zhang, Chenguang Wang, Yevgeniy Vorobeychik, and Chaowei Xiao. 2023. CodeiPPrompt: Intellectual property infringement assessment of code language models. In Proceedings of the International Conference on Machine Learning. PMLR, 40373\u201340389."},{"key":"e_1_3_2_80_2","first-page":"1","volume-title":"Proceedings of the 11th International Conference on Learning Representations","author":"Zhang Shun","year":"2023","unstructured":"Shun Zhang, Zhenfang Chen, Yikang Shen, Mingyu Ding, Joshua B. Tenenbaum, and Chuang Gan. 2023. Planning with large language models for code generation. In Proceedings of the 11th International Conference on Learning Representations. OpenReview.net, 1\u201328."},{"key":"e_1_3_2_81_2","unstructured":"Wayne Xin Zhao Kun Zhou Junyi Li Tianyi Tang Xiaolei Wang Yupeng Hou Yingqian Min Beichen Zhang Junjie Zhang Zican Dong et al. 2023. A survey of large language models. arXiv:2303.18223. Retrieved from https:\/\/arxiv.org\/abs\/2303.18223"},{"key":"e_1_3_2_82_2","first-page":"1","volume-title":"Proceedings of the 11th International Conference on Learning Representations","author":"Zhou Shuyan","year":"2023","unstructured":"Shuyan Zhou, Uri Alon, Frank F. Xu, Zhiruo Wang, Zhengbao Jiang, and Graham Neubig. 2023. DocPrompting: Generating code by retrieving the docs. In Proceedings of the 11th International Conference on Learning Representations. OpenReview.net, 1\u201319."},{"key":"e_1_3_2_83_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2009.32"},{"key":"e_1_3_2_84_2","doi-asserted-by":"publisher","DOI":"10.1145\/3520312.3534864"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3748506","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,25]],"date-time":"2026-04-25T06:34:09Z","timestamp":1777098849000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3748506"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,24]]},"references-count":83,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2026,5,31]]}},"alternative-id":["10.1145\/3748506"],"URL":"https:\/\/doi.org\/10.1145\/3748506","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,24]]},"assertion":[{"value":"2025-03-24","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-06-24","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-04-24","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}