{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,21]],"date-time":"2026-07-21T12:19:07Z","timestamp":1784636347732,"version":"3.55.0"},"reference-count":111,"publisher":"Association for Computing Machinery (ACM)","issue":"6","funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62202420"],"award-info":[{"award-number":["62202420"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"National Research Foundation, under its Investigatorship","award":["NRF-NRFI08-2022-0002"],"award-info":[{"award-number":["NRF-NRFI08-2022-0002"]}]},{"DOI":"10.13039\/501100001381","name":"National Research Foundation, Singapore","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001381","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Zhejiang University Education Foundation Qizhen Scholar Foundation"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2025,7,31]]},"abstract":"<jats:p>Developers deal with code-change-related tasks daily, e.g., reviewing code. Pre-trained code and code-change-oriented models have been adapted to help developers with such tasks. Recently, large language models (LLMs) have shown their effectiveness in code-related tasks. However, existing LLMs for code focus on general code syntax and semantics rather than the differences between two code versions. Thus, it is an open question how LLMs perform on code-change-related tasks.<\/jats:p>\n          <jats:p>\n            To answer this question, we conduct an empirical study using\n            <jats:inline-formula content-type=\"math\/tex\">\n              <jats:tex-math notation=\"LaTeX\" version=\"MathJax\">\\(&gt;\\)<\/jats:tex-math>\n            <\/jats:inline-formula>\n            1B parameters LLMs on three code-change-related tasks, i.e., code review generation, commit message generation, and just-in-time comment update, with in-context learning (ICL) and parameter-efficient fine-tuning (PEFT, including LoRA and prefix-tuning). We observe that the performance of LLMs is poor without examples and generally improves with examples, but more examples do not always lead to better performance. LLMs tuned with LoRA have comparable performance to the state-of-the-art small pre-trained models. Larger models are not always better, but\n            <jats:sc>Llama 2<\/jats:sc>\n            and\n            <jats:sc>Code Llama<\/jats:sc>\n            families are always the best. The best LLMs outperform small pre-trained models on the code changes that only modify comments and perform comparably on other code changes. We suggest future work should focus more on guiding LLMs to learn the knowledge specific to the changes related to code rather than comments for code-change-related tasks.\n          <\/jats:p>","DOI":"10.1145\/3709358","type":"journal-article","created":{"date-parts":[[2024,12,23]],"date-time":"2024-12-23T14:48:06Z","timestamp":1734965286000},"page":"1-36","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":35,"title":["Exploring the Capabilities of LLMs for Code-Change-Related Tasks"],"prefix":"10.1145","volume":"34","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-4602-4296","authenticated-orcid":false,"given":"Lishui","family":"Fan","sequence":"first","affiliation":[{"name":"The State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7273-6709","authenticated-orcid":false,"given":"Jiakun","family":"Liu","sequence":"additional","affiliation":[{"name":"Singapore Management University, Singapore, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1981-1626","authenticated-orcid":false,"given":"Zhongxin","family":"Liu","sequence":"additional","affiliation":[{"name":"The State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou, China and Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security, Hangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4367-7201","authenticated-orcid":false,"given":"David","family":"Lo","sequence":"additional","affiliation":[{"name":"Singapore Management University, Singapore, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6302-3256","authenticated-orcid":false,"given":"Xin","family":"Xia","sequence":"additional","affiliation":[{"name":"Zhejiang University, Hangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2615-9792","authenticated-orcid":false,"given":"Shanping","family":"Li","sequence":"additional","affiliation":[{"name":"The State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,7]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","unstructured":"Loubna Ben Allal Raymond Li Denis Kocetkov Chenghao Mou Christopher Akiki Carlos Munoz Ferrandis Niklas Muennighoff Mayank Mishra Alex Gu Manan Dey et al. 2023. Santacoder: Don\u2019t reach for the stars! arXiv:2301.03988. Retrieved from 10.48550\/arXiv.2301.03988","DOI":"10.48550\/arXiv.2301.03988"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","unstructured":"Shamil Ayupov and Nadezhda Chirkova. 2022. Parameter-efficient finetuning of transformers for source code. arXiv:2212.05901. Retrieved from 10.48550\/arXiv.2212.05901","DOI":"10.48550\/arXiv.2212.05901"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","unstructured":"Sid Black Leo Gao Phil Wang Connor Leahy and Stella Biderman. 2021. GPT-Neo: Large scale autoregressive language modeling with mesh-tensorflow. DOI: 10.5281\/zenodo.5297715","DOI":"10.5281\/zenodo.5297715"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/ESEM.2013.23"},{"key":"e_1_3_2_6_2","first-page":"1877","article-title":"Language models are few-shot learners","volume":"33","author":"Brown Tom","year":"2020","unstructured":"Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D. Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems, Vol. 33, 1877\u20131901.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/1858996.1859005"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.findings-acl.676"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3649329.3657356"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-main.168"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.findings-emnlp.745"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-main.50"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00091"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/3637528.3671452"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-023-00626-4"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510069"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377812.3381395"},{"key":"e_1_3_2_18_2","first-page":"575","volume-title":"Software Pioneers.","author":"Fagan Michael E.","year":"2022","unstructured":"Michael E. Fagan. 2022. Design and code inspections to reduce errors in program development (reprint). In Software Pioneers. Manfred Broy and Ernst Denert (Eds.), Springer, Berlin, 575\u2013607."},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2007.70731"},{"key":"e_1_3_2_20_2","first-page":"1","volume-title":"The Eleventh International Conference on Learning Representations (ICLR \u201923)","author":"Fried Daniel","year":"2023","unstructured":"Daniel Fried, Armen Aghajanyan, Jessy Lin, Sida Wang, Eric Wallace, Freda Shi, Ruiqi Zhong, Scott Yih, Luke Zettlemoyer, and Mike Lewis. 2023. Incoder: A generative model for code infilling and synthesis. In The Eleventh International Conference on Learning Representations (ICLR \u201923), 1\u201326."},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/1806799.1806828"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE56229.2023.00109"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3608134"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3697012"},{"key":"e_1_3_2_25_2","unstructured":"Daya Guo Qihao Zhu Dejian Yang Zhenda Xie Kai Dong Wentao Zhang Guanting Chen Xiao Bi Yu Wu Y. K. Li et al. 2024. Deepseek-Coder: When the large language model meets programming\u2013the rise of code intelligence. arXiv:2401.14196. Retrieved from https:\/\/arxiv.org\/abs\/2401.14196"},{"key":"e_1_3_2_26_2","unstructured":"Qi Guo Junming Cao Xiaofei Xie Shangqing Liu Xiaohong Li Bihuan Chen and Xin Peng. 2023. Exploring the potential of chatgpt in automated code refinement: An empirical study. arXiv:2309.08221. Retrieved from https:\/\/arxiv.org\/abs\/2309.08221"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/PAAP56126.2022.10010469"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3576915.3623175"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380361"},{"key":"e_1_3_2_30_2","first-page":"1","volume-title":"The Tenth International Conference on Learning Representations (ICLR \u201922)","author":"Hu Edward J.","year":"2022","unstructured":"Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. Lora: Low-rank adaptation of large language models. In The Tenth International Conference on Learning Representations (ICLR \u201922), 1\u201326."},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3502853"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","unstructured":"Hamel Husain Ho-Hsiang Wu Tiferet Gazit Miltiadis Allamanis and Marc Brockschmidt. 2019. Codesearchnet challenge: Evaluating the state of semantic code search. arXiv:1909.09436. Retrieved from 10.48550\/arXiv.1909.09436","DOI":"10.48550\/arXiv.1909.09436"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639223"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2017.8115626"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3613891"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-015-9400-x"},{"key":"e_1_3_2_37_2","unstructured":"Jared Kaplan Sam McCandlish Tom Henighan Tom B. Brown Benjamin Chess Rewon Child Scott Gray Alec Radford Jeffrey Wu and Dario Amodei. 2020. Scaling laws for neural language models. arXiv:2001.08361. Retrieved from https:\/\/arxiv.org\/abs\/2001.08361"},{"key":"e_1_3_2_38_2","first-page":"262","volume-title":"Proceedings of the 22nd International Conference on Program Comprehension","author":"Kirinuki Hiroyuki","year":"2024","unstructured":"Hiroyuki Kirinuki, Yoshiki Higo, Keisuke Hotta, and Shinji Kusumoto. 2024. Hey! Are you committing tangled changes? In Proceedings of the 22nd International Conference on Program Comprehension, 262\u2013265."},{"key":"e_1_3_2_39_2","first-page":"388","volume-title":"Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing","author":"Koehn Philipp","year":"2004","unstructured":"Philipp Koehn. 2004. Statistical significance tests for machine translation evaluation. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, 388\u2013395."},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3195836.3195842"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3661167.3661210"},{"key":"e_1_3_2_42_2","first-page":"21314","article-title":"Coderl: Mastering code generation through pretrained models and deep reinforcement learning","volume":"35","author":"Le Hung","year":"2022","unstructured":"Hung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese, and Steven Chu Hong Hoi. 2022. Coderl: Mastering code generation through pretrained models and deep reinforcement learning. In Advances in Neural Information Processing Systems, Vol. 35, 21314\u201321328.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_43_2","unstructured":"Jia Li Ge Li Chongyang Tao Huangzhao Zhang Fang Liu and Zhi Jin. 2023. Large language model-aware in-context learning for code generation. arXiv:2310.09748. Retrieved from https:\/\/arxiv.org\/abs\/2310.09748"},{"key":"e_1_3_2_44_2","unstructured":"Jia Li Yunfei Zhao Yongmin Li Ge Li and Zhi Jin. 2023. Towards enhancing in-context learning for code generation. arXiv:2303.17780. Retrieved from https:\/\/arxiv.org\/abs\/2303.17780"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2024.107527"},{"key":"e_1_3_2_46_2","unstructured":"Raymond Li Loubna Ben Allal Yangtian Zi Niklas Muennighoff Denis Kocetkov Chenghao Mou Marc Marone Christopher Akiki Jia Li Jenny Chim et al. 2023. Starcoder: May the source be with you! arXiv:2305.06161. Retrieved from https:\/\/arxiv.org\/abs\/2305.06161"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.acl-long.353"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549081"},{"key":"e_1_3_2_49_2","unstructured":"Vladislav Lialin Vijeta Deshpande and Anna Rumshisky. 2023. Scaling down to scale up: A guide to parameter-efficient fine-tuning. arXiv:2303.15647. Retrieved from https:\/\/arxiv.org\/abs\/2303.15647"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC52881.2021.00013"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3616339"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2022.3185458"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.1988.10169"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-main.616"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/3637528.3671609"},{"key":"e_1_3_2_56_2","unstructured":"Chia-Wei Liu Ryan Lowe Iulian V. Serban Michael Noseworthy Laurent Charlin and Joelle Pineau. 2016. How not to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. arXiv:1603.08023. Retrieved from https:\/\/arxiv.org\/abs\/1603.08023"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3649594"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2020.3038681"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","unstructured":"Shangqing Liu Yanzhou Li and Yang Liu. 2022. Commitbart: A large pre-trained model for github commits. arXiv:2208.08100. Retrieved from 10.48550\/arXiv.2208.08100","DOI":"10.48550\/arXiv.2208.08100"},{"key":"e_1_3_2_60_2","doi-asserted-by":"crossref","unstructured":"Shuo Liu Jacky Keung Zhen Yang Fang Liu Qilin Zhou and Yihan Liao. 2024. Delving into parameter-efficient fine-tuning in code change learning: An empirical study. arXiv:2402.06247. Retrieved from https:\/\/arxiv.org\/abs\/2402.06247","DOI":"10.1109\/SANER60148.2024.00055"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238190"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2021.3138909"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1145\/3324884.3416581"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1145\/3324884.3416581"},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P17-2045"},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE59848.2023.00026"},{"key":"e_1_3_2_67_2","first-page":"1","volume-title":"The Twelfth International Conference on Learning Representations (ICLR \u201924)","author":"Luo Ziyang","year":"2024","unstructured":"Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. 2024. Wizardcoder: Empowering code large language models with evol-instruct. In The Twelfth International Conference on Learning Representations (ICLR \u201924), 1\u201313."},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.11613\/BM.2012.031"},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.findings-emnlp.796"},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-main.759"},{"key":"e_1_3_2_71_2","doi-asserted-by":"crossref","unstructured":"Swaroop Mishra Daniel Khashabi Chitta Baral and Hannaneh Hajishirzi. 2021. Cross-task generalization via natural language crowdsourcing instructions. arXiv:2104.08773. Retrieved from https:\/\/arxiv.org\/abs\/2104.08773","DOI":"10.18653\/v1\/2022.acl-long.244"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.acl-long.891"},{"key":"e_1_3_2_73_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639187"},{"key":"e_1_3_2_74_2","first-page":"1","volume-title":"The Eleventh International Conference on Learning Representations (ICLR \u201923)","author":"Nijkamp Erik","year":"2023","unstructured":"Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. 2023. Codegen: An open large language model for code with multi-turn program synthesis. In The Eleventh International Conference on Learning Representations (ICLR \u201923), 1\u201325."},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i1.16119"},{"key":"e_1_3_2_76_2","first-page":"311","volume-title":"Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics","author":"Papineni Kishore","year":"2002","unstructured":"Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, 311\u2013318."},{"key":"e_1_3_2_77_2","unstructured":"Indraneil Paul Jun Luo Goran Glava\u0161 and Iryna Gurevych. 2024. Ircoder: Intermediate representations make language models robust multilingual code generators. arXiv:2403.03894. Retrieved from https:\/\/arxiv.org\/abs\/2403.03894"},{"key":"e_1_3_2_78_2","unstructured":"Baolin Peng Chunyuan Li Pengcheng He Michel Galley and Jianfeng Gao. 2023. Instruction tuning with GPT-4. arXiv:2304.03277. Retrieved from https:\/\/arxiv.org\/abs\/2304.03277"},{"key":"e_1_3_2_79_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.494"},{"key":"e_1_3_2_80_2","doi-asserted-by":"publisher","DOI":"10.5555\/3455716.3455856"},{"key":"e_1_3_2_81_2","doi-asserted-by":"publisher","DOI":"10.1145\/2889160.2889244"},{"key":"e_1_3_2_82_2","unstructured":"Sebastian Raschka. 2018. Model evaluation model selection and algorithm selection in machine learning. arXiv:1811.12808. Retrieved from https:\/\/arxiv.org\/abs\/1811.12808"},{"key":"e_1_3_2_83_2","unstructured":"Baptiste Roziere Jonas Gehring Fabian Gloeckle Sten Sootla Itai Gat Xiaoqing Ellen Tan Yossi Adi Jingyu Liu Romain Sauvestre and Tal Remez. 2023. Code LLaMA: Open foundation models for code. arXiv:2308.12950. Retrieved from https:\/\/arxiv.org\/abs\/2308.12950"},{"key":"e_1_3_2_84_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-main.372"},{"key":"e_1_3_2_85_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER48275.2020.9054794"},{"key":"e_1_3_2_86_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10515-024-00421-4"},{"key":"e_1_3_2_87_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-022-10219-1"},{"key":"e_1_3_2_88_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.findings-emnlp.825"},{"key":"e_1_3_2_89_2","unstructured":"Yi Tay Mostafa Dehghani Jinfeng Rao William Fedus Samira Abnar Hyung Won Chung Sharan Narang Dani Yogatama Ashish Vaswani and Donald Metzler. 2021. Scale efficiently: Insights from pre-training and fine-tuning transformers. arXiv:2109.10686. Retrieved from https:\/\/arxiv.org\/abs\/2109.10686"},{"key":"e_1_3_2_90_2","unstructured":"Hugo Touvron Louis Martin Kevin Stone Peter Albert Amjad Almahairi Yasmine Babaei Nikolay Bashlykov Soumya Batra Prajjwal Bhargava Shruti Bhosale et al. 2023. LLaMA 2: Open foundation and fine-tuned chat models. arXiv:2307.09288. Retrieved from https:\/\/arxiv.org\/abs\/2307.09288"},{"key":"e_1_3_2_91_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-Companion58688.2023.00053"},{"key":"e_1_3_2_92_2","unstructured":"Ben Wang and Aran Komatsuzaki. 2021. GPT-J-6B: A 6 billion parameter autoregressive language model. Retrieved from https:\/\/github.com\/kingoflolz\/mesh-transformer-jax"},{"key":"e_1_3_2_93_2","doi-asserted-by":"crossref","unstructured":"Yanlin Wang Yanxian Huang Daya Guo Hongyu Zhang and Zibin Zheng. 2024. Sparsecoder: Identifier-aware sparse transformer for file-level code summarization. arXiv:2401.14727. Retrieved from https:\/\/arxiv.org\/abs\/2401.14727","DOI":"10.1109\/SANER60148.2024.00068"},{"key":"e_1_3_2_94_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.685"},{"key":"e_1_3_2_95_2","first-page":"1","article-title":"Emergent abilities of large language models","volume":"2022","author":"Wei Jason","year":"2022","unstructured":"Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. 2022. Emergent abilities of large language models. Transactions on Machine Learning 2022 (2022), 1\u201330.","journal-title":"Transactions on Machine Learning"},{"key":"e_1_3_2_96_2","unstructured":"Martin Weyssow Xin Zhou Kisub Kim David Lo and Houari Sahraoui. 2023. Exploring parameter-efficient fine-tuning techniques for code generation with large language models. arXiv:2308.10462. Retrieved from https:\/\/arxiv.org\/abs\/2308.10462"},{"key":"e_1_3_2_97_2","first-page":"1","volume-title":"The Forty-first International Conference on Machine Learning (ICML \u201924)","author":"Wu Di","year":"2024","unstructured":"Di Wu, Wasi Uddin Ahmad, Dejiao Zhang, Murali Krishna Ramanathan, and Xiaofei Ma. 2024. Repoformer: Selective retrieval for repository-level code completion. In The Forty-first International Conference on Machine Learning (ICML \u201924), 1\u201325."},{"key":"e_1_3_2_98_2","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN60899.2024.10651243"},{"key":"e_1_3_2_99_2","doi-asserted-by":"publisher","DOI":"10.1145\/3551349.3556909"},{"key":"e_1_3_2_100_2","unstructured":"Lingling Xu Haoran Xie Si-Zhao Joe Qin Xiaohui Tao and Fu Lee Wang. 2023. Parameter-efficient fine-tuning methods for pretrained language models: A critical review and assessment. arXiv:2312.12148. Retrieved from https:\/\/arxiv.org\/abs\/2312.12148"},{"key":"e_1_3_2_101_2","doi-asserted-by":"publisher","DOI":"10.1109\/IACC.2016.25"},{"key":"e_1_3_2_102_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3623342"},{"key":"e_1_3_2_103_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2024.3440503"},{"key":"e_1_3_2_104_2","doi-asserted-by":"publisher","unstructured":"Zhiqiang Yuan Junwei Liu Qiancheng Zi Mingwei Liu Xin Peng and Yiling Lou. 2023. Evaluating instruction-tuned large language models on code comprehension and generation. arXiv:2308.01240. Retrieved from 10.48550\/arXiv.2308.01240","DOI":"10.48550\/arXiv.2308.01240"},{"key":"e_1_3_2_105_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.acl-long.411"},{"issue":"14","key":"e_1_3_2_106_2","first-page":"5165","article-title":"An efficient optimized framework for analyzing the performance of breast cancer using machine learning algorithms","volume":"100","author":"Zeid Mae","year":"2022","unstructured":"Mae Zeid, Khaled El-Bahnasy, and S. E. Abu-Youssef. 2022. An efficient optimized framework for analyzing the performance of breast cancer using machine learning algorithms. Journal of Theoretical and Applied Information Technology 100, 14 (2022), 5165\u20135178.","journal-title":"Journal of Theoretical and Applied Information Technology"},{"key":"e_1_3_2_107_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.151"},{"key":"e_1_3_2_108_2","doi-asserted-by":"publisher","DOI":"10.1145\/3551349.3556955"},{"issue":"1","key":"e_1_3_2_109_2","first-page":"295","article-title":"LLM-cloud complete: Leveraging cloud computing for efficient large language model-based code completion","volume":"5","author":"Zhang Mingxuan","year":"2024","unstructured":"Mingxuan Zhang, Bo Yuan, Hanzhe Li, and Kangming Xu. 2024. LLM-cloud complete: Leveraging cloud computing for efficient large language model-based code completion. Journal of Artificial Intelligence General Science 5, 1 (2024), 295\u2013326.","journal-title":"Journal of Artificial Intelligence General Science"},{"key":"e_1_3_2_110_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME58846.2023.00028"},{"key":"e_1_3_2_111_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524610.3527901"},{"key":"e_1_3_2_112_2","first-page":"344","volume-title":"Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics","author":"Mutton Andrew","year":"2015","unstructured":"Andrew Mutton, Mark Dras, Stephen Wan, and Robert Dale. 2015. GLEU: Automatic evaluation of sentence-level fluency. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, 344\u2013351."}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3709358","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,1]],"date-time":"2025-07-01T13:29:10Z","timestamp":1751376550000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3709358"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7]]},"references-count":111,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2025,7,31]]}},"alternative-id":["10.1145\/3709358"],"URL":"https:\/\/doi.org\/10.1145\/3709358","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7]]},"assertion":[{"value":"2024-06-19","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-12-09","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-07-01","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}