{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,10]],"date-time":"2026-04-10T21:22:00Z","timestamp":1775856120775,"version":"3.50.1"},"reference-count":92,"publisher":"Association for Computing Machinery (ACM)","issue":"7","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2025,9,30]]},"abstract":"<jats:p>Large language models (LLMs) demonstrate impressive capabilities to generate accurate code snippets given natural language intents in a zero-shot manner, i.e., without the need for specific fine-tuning. While prior studies have highlighted the advantages of fine-tuning LLMs, this process incurs high computational costs, making it impractical in resource-scarce environments, particularly for models with billions of parameters. To address these challenges, previous research explored in-context learning (ICL) and retrieval-augmented generation (RAG) as strategies to guide the LLM generative process with task-specific prompt examples. However, ICL and RAG introduce inconveniences, such as the need for designing contextually relevant prompts and the absence of learning task-specific parameters, thereby limiting downstream task performance. In this context, we foresee parameter-efficient fine-tuning (PEFT) as a promising approach to efficiently specialize LLMs to task-specific data while maintaining reasonable resource consumption. In this article, we deliver a comprehensive study of PEFT techniques for LLMs in the context of automated code generation. Our comprehensive investigation of PEFT techniques for LLMs reveals their superiority and potential over ICL and RAG across a diverse set of LLMs and three representative Python code generation datasets: Conala, CodeAlpacaPy, and APPS. Furthermore, our study highlights the potential for tuning larger LLMs and significant reductions in memory usage by combining PEFT with quantization. Therefore, this study opens opportunities for broader applications of PEFT in software engineering scenarios.<\/jats:p>","DOI":"10.1145\/3714461","type":"journal-article","created":{"date-parts":[[2025,1,22]],"date-time":"2025-01-22T08:26:55Z","timestamp":1737534415000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":35,"title":["Exploring Parameter-Efficient Fine-Tuning Techniques for Code Generation with Large Language Models"],"prefix":"10.1145","volume":"34","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5987-850X","authenticated-orcid":false,"given":"Martin","family":"Weyssow","sequence":"first","affiliation":[{"name":"DIRO, University of Montreal, Montreal, Quebec, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4558-0622","authenticated-orcid":false,"given":"Xin","family":"Zhou","sequence":"additional","affiliation":[{"name":"Singapore Management University, Singapore, Singapore"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4462-6916","authenticated-orcid":false,"given":"Kisub","family":"Kim","sequence":"additional","affiliation":[{"name":"Singapore Management University, Singapore, Singapore"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4367-7201","authenticated-orcid":false,"given":"David","family":"Lo","sequence":"additional","affiliation":[{"name":"Singapore Management University, Singapore, Singapore"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6304-9926","authenticated-orcid":false,"given":"Houari","family":"Sahraoui","sequence":"additional","affiliation":[{"name":"DIRO, University of Montreal, Montreal, Quebec, Canada"}]}],"member":"320","published-online":{"date-parts":[[2025,8,17]]},"reference":[{"key":"e_1_3_1_2_2","first-page":"245","article-title":"Structural language models of code","author":"Alon Uri","year":"2020","unstructured":"Uri Alon, Roy Sadaka, Omer Levy, and Eran Yahav. 2020. Structural language models of code. In Proceedings of the International Conference on Machine Learning. PMLR, 245\u2013256.","journal-title":"Proceedings of the International Conference on Machine Learning"},{"key":"e_1_3_1_3_2","unstructured":"Ben Athiwaratkun Sanjay Krishna Gouda Zijian Wang Xiaopeng Li Yuchen Tian Ming Tan Wasi Uddin Ahmad Shiqi Wang Qing Sun Mingyue Shang et\u00a0al. 2022. Multi-lingual evaluation of code generation models. arXiv:2210.14868. Retrieved from https:\/\/arxiv.org\/abs\/2210.14868"},{"key":"e_1_3_1_4_2","unstructured":"Jacob Austin Augustus Odena Maxwell Nye Maarten Bosma Henryk Michalewski David Dohan Ellen Jiang Carrie Cai Michael Terry Quoc Le et\u00a0al. 2021. Program synthesis with large language models. arXiv:2108.07732. Retrieved from https:\/\/arxiv.org\/abs\/2108.07732"},{"key":"e_1_3_1_5_2","unstructured":"Matej Balog Alexander L. Gaunt Marc Brockschmidt Sebastian Nowozin and Daniel Tarlow. 2016. Deepcoder: Learning to write programs. arXiv:1611.01989. Retrieved from https:\/\/arxiv.org\/abs\/1611.01989"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.5555\/3495724.3495883"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2023.3267446"},{"key":"e_1_3_1_8_2","first-page":"1372","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Chappuis Christel","year":"2022","unstructured":"Christel Chappuis, Val\u00e9rie Zermatten, Sylvain Lobry, Bertrand Le Saux, and Devis Tuia. 2022. Prompt-RSVQA: Prompting visual context to a language model for remote sensing visual question answering. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, 1372\u20131381."},{"key":"e_1_3_1_9_2","unstructured":"Sahil Chaudhary. 2023. Code Alpaca: An Instruction-Following LLaMA Model for Code Generation. Retrieved from https:\/\/github.com\/sahil280114\/codealpaca"},{"key":"e_1_3_1_10_2","unstructured":"Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Ponde de Oliveira Pinto Jared Kaplan Harri Edwards Yuri Burda Nicholas Joseph Greg Brockman et\u00a0al. 2021. Evaluating large language models trained on code. arXiv:2107.03374. Retrieved from https:\/\/arxiv.org\/abs\/2107.03374"},{"key":"e_1_3_1_11_2","doi-asserted-by":"crossref","first-page":"5282","DOI":"10.18653\/v1\/2023.findings-acl.325","article-title":"CodePrompt: Task-agnostic prefix tuning for program and language generation","author":"Choi YunSeok","year":"2023","unstructured":"YunSeok Choi and Jee-Hyong Lee. 2023. CodePrompt: Task-agnostic prefix tuning for program and language generation. In Findings of the Association for Computational Linguistics (ACL \u201923), 5282\u20135297.","journal-title":"Findings of the Association for Computational Linguistics (ACL \u201923)"},{"key":"e_1_3_1_12_2","unstructured":"Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra Adam Roberts Paul Barham Hyung Won Chung Charles Sutton Sebastian Gehrmann et\u00a0al. 2022. Palm: Scaling language modeling with pathways. arXiv:2204.02311. Retrieved from https:\/\/arxiv.org\/abs\/2204.02311"},{"key":"e_1_3_1_13_2","unstructured":"Tim Dettmers Mike Lewis Younes Belkada and Luke Zettlemoyer. 2022. LLM.int8() 8-bit Matrix multiplication for transformers at scale. arXiv:2208.07339. Retrieved from https:\/\/arxiv.org\/abs\/2208.07339"},{"key":"e_1_3_1_14_2","unstructured":"Tim Dettmers Artidoro Pagnoni Ari Holtzman and Luke Zettlemoyer. 2023. QLoRA: Efficient finetuning of quantized LLMs. arXiv:2305.14314. Retrieved from https:\/\/arxiv.org\/abs\/2305.14314"},{"key":"e_1_3_1_15_2","unstructured":"Ning Ding Yujia Qin Guang Yang Fuchao Wei Zonghan Yang Yusheng Su Shengding Hu Yulin Chen Chi-Min Chan Weize Chen et\u00a0al. 2022. Delta tuning: A comprehensive study of parameter efficient methods for pre-trained language models. arXiv:2203.06904. Retrieved from https:\/\/arxiv.org\/abs\/2203.06904"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-023-00626-4"},{"key":"e_1_3_1_17_2","doi-asserted-by":"crossref","unstructured":"Zhangyin Feng Daya Guo Duyu Tang Nan Duan Xiaocheng Feng Ming Gong Linjun Shou Bing Qin Ting Liu Daxin Jiang et\u00a0al. 2020. Codebert: A pre-trained model for programming and natural languages. arXiv:2002.08155. Retrieved from https:\/\/arxiv.org\/abs\/2002.08155","DOI":"10.18653\/v1\/2020.findings-emnlp.139"},{"key":"e_1_3_1_18_2","unstructured":"Shuzheng Gao Xin-Cheng Wen Cuiyun Gao Wenxuan Wang and Michael R. Lyu. 2023. Constructing effective in-context demonstration for code intelligence tasks: An empirical study. arXiv:2304.07575. Retrieved from https:\/\/arxiv.org\/abs\/2304.07575"},{"key":"e_1_3_1_19_2","unstructured":"Shuzheng Gao Hongyu Zhang Cuiyun Gao and Chaozheng Wang. 2023. Keeping pace with ever-increasing data: Towards continual learning of code intelligence models. arXiv:2302.03482. Retrieved from https:\/\/arxiv.org\/abs\/2302.03482"},{"key":"e_1_3_1_20_2","article-title":"Large language models are few-shot summarizers: Multi-intent comment generation via in-context learning","author":"Geng Mingyang","year":"2024","unstructured":"Mingyang Geng, Shangwen Wang, Dezun Dong, Haotian Wang, Ge Li, Zhi Jin, Xiaoguang Mao, and Xiangke Liao. 2024. Large language models are few-shot summarizers: Multi-intent comment generation via in-context learning. In Proceedings of the IEEE\/ACM 46th International Conference on Software Engineering (ICSE \u201924).","journal-title":"Proceedings of the IEEE\/ACM 46th International Conference on Software Engineering (ICSE \u201924)"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524610.3527892"},{"key":"e_1_3_1_22_2","unstructured":"Shirley Anugrah Hayati Raphael Olivier Pravalika Avvaru Pengcheng Yin Anthony Tomasic and Graham Neubig. 2018. Retrieval-based neural code generation. arXiv:1808.10025 Retrieved from https:\/\/arxiv.org\/abs\/1808.10025"},{"key":"e_1_3_1_23_2","article-title":"Measuring coding challenge competence with APPS","author":"Hendrycks Dan","year":"2021","unstructured":"Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, and Jacob Steinhardt. 2021. Measuring coding challenge competence with APPS. In Proceedings of the 38th Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2).","journal-title":"Proceedings of the 38th Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2)"},{"key":"e_1_3_1_24_2","first-page":"2790","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Houlsby Neil","year":"2019","unstructured":"Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for NLP. In Proceedings of the International Conference on Machine Learning. PMLR, 2790\u20132799."},{"key":"e_1_3_1_25_2","unstructured":"Edward J Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv:2106.09685. Retrieved from https:\/\/arxiv.org\/abs\/2106.09685"},{"key":"e_1_3_1_26_2","unstructured":"Zhiqiang Hu Yihuai Lan Lei Wang Wanyu Xu Ee-Peng Lim Roy Ka-Wei Lee Lidong Bing and Soujanya Poria. 2023. LLM-adapters: An adapter family for parameter-efficient fine-tuning of large language models. arXiv:2304.01933. Retrieved from https:\/\/arxiv.org\/abs\/2304.01933"},{"key":"e_1_3_1_27_2","unstructured":"Chengsong Huang Qian Liu Bill Yuchen Lin Tianyu Pang Chao Du and Min Lin. 2023. Lorahub: Efficient cross-task generalization via dynamic Lora composition. arXiv:2307.13269. Retrieved from https:\/\/arxiv.org\/abs\/2307.13269"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v37i4.25642"},{"key":"e_1_3_1_29_2","unstructured":"Denis Kocetkov Raymond Li Loubna Ben Allal Jia Li Chenghao Mou Carlos Mu\u00f1oz Ferrandis Yacine Jernite Margaret Mitchell Sean Hughes Thomas Wolf et\u00a0al. 2022. The Stack: 3 TB of permissively licensed source code. arXiv:2211.15533. Retrieved from https:\/\/arxiv.org\/abs\/2211.15533"},{"key":"e_1_3_1_30_2","first-page":"22199","article-title":"Large language models are zero-shot reasoners","author":"Kojima Takeshi","year":"2022","unstructured":"Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. In Proceedings of the 36th International Conference on Neural Information Processing Systems, 22199\u201322213.","journal-title":"Proceedings of the 36th International Conference on Neural Information Processing Systems"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.243"},{"key":"e_1_3_1_32_2","first-page":"9459","article-title":"Retrieval-augmented generation for knowledge-intensive Nlp tasks","author":"Lewis Patrick","year":"2020","unstructured":"Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K\u00fcttler, Mike Lewis, Wen-tau Yih, Tim Rockt\u00e4schel, et\u00a0al. 2020. Retrieval-augmented generation for knowledge-intensive Nlp tasks. In Proceedings of the 34th International Conference on Neural Information Processing Systems, 9459\u20139474.","journal-title":"Proceedings of the 34th International Conference on Neural Information Processing Systems"},{"key":"e_1_3_1_33_2","unstructured":"Jia Li Yongmin Li Ge Li Zhi Jin Yiyang Hao and Xing Hu. 2023. Skcoder: A sketch-based approach for automatic code generation. arXiv:2302.06144. Retrieved from https:\/\/arxiv.org\/abs\/2302.06144"},{"key":"e_1_3_1_34_2","unstructured":"Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. arXiv:2101.00190. Retrieved from https:\/\/arxiv.org\/abs\/2101.00190"},{"key":"e_1_3_1_35_2","article-title":"2023","author":"Li Zehan","unstructured":"Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. 2023. Towards general text embeddings with multi-stage contrastive learning. arXiv:2308.03281. Retrieved from https:\/\/arxiv.org\/abs\/2308.03281","journal-title":"Towards general text embeddings with multi-stage contrastive learning"},{"key":"e_1_3_1_36_2","unstructured":"Percy Liang Rishi Bommasani Tony Lee Dimitris Tsipras Dilara Soylu Michihiro Yasunaga Yian Zhang Deepak Narayanan Yuhuai Wu Ananya Kumar et\u00a0al. 2022. Holistic evaluation of language models. arXiv:2211.09110. Retrieved from https:\/\/arxiv.org\/abs\/2211.09110"},{"key":"e_1_3_1_37_2","doi-asserted-by":"crossref","unstructured":"W. Liang M. Yuksekgonul Y. Mao E. Wu and J. Zou. 2023. GPT detectors are biased against non-native English writers arXiv:2304.02819. Retrieved from https:\/\/arxiv.org\/abs\/2304.02819","DOI":"10.1016\/j.patter.2023.100779"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.5555\/3600270.3600412"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE56229.2023.00125"},{"key":"e_1_3_1_40_2","unstructured":"Shangqing Liu Yu Chen Xiaofei Xie Jingkai Siow and Yang Liu. 2020. Retrieval-augmented generation for code summarization via hybrid GNN. arXiv:2006.05405. Retrieved from https:\/\/arxiv.org\/abs\/2006.05405"},{"key":"e_1_3_1_41_2","doi-asserted-by":"crossref","unstructured":"Shuo Liu Jacky Keung Zhen Yang Fang Liu Qilin Zhou and Yihan Liao. 2024. Delving Into parameter-efficient fine-tuning in code change learning: An empirical study. arXiv:2402.06247. Retrieved from https:\/\/arxiv.org\/abs\/2402.06247","DOI":"10.1109\/SANER60148.2024.00055"},{"key":"e_1_3_1_42_2","unstructured":"Shuai Lu Nan Duan Hojae Han Daya Guo Seung-won Hwang and Alexey Svyatkovskiy. 2022. Reacc: A retrieval-augmented code completion framework. arXiv:2203.07722. Retrieved from https:\/\/arxiv.org\/abs\/2203.07722"},{"key":"e_1_3_1_43_2","unstructured":"Shuai Lu Daya Guo Shuo Ren Junjie Huang Alexey Svyatkovskiy Ambrosio Blanco Colin Clement Dawn Drain Daxin Jiang Duyu Tang et\u00a0al. 2021. CodeXGLUE: A machine learning benchmark dataset for code understanding and generation. arXiv:2102.04664. Retrieved from https:\/\/arxiv.org\/abs\/2102.04664"},{"key":"e_1_3_1_44_2","unstructured":"Sourab Mangrulkar Sylvain Gugger Lysandre Debut Younes Belkada Sayak Paul and Benjamin Bossan. 2022. PEFT: State-of-the-Art Parameter-Efficient Fine-Tuning Methods. Retrieved from https:\/\/github.com\/huggingface\/peft"},{"key":"e_1_3_1_45_2","article-title":"Recent advances in natural language processing via large pre-trained language models: A survey","author":"Min Bonan","year":"2021","unstructured":"Bonan Min, Hayley Ross, Elior Sulem, Amir Pouran Ben Veyseh, Thien Huu Nguyen, Oscar Sainz, Eneko Agirre, Ilana Heintz, and Dan Roth. 2021. Recent advances in natural language processing via large pre-trained language models: A survey. Computing Surveys (2021).","journal-title":"Computing Surveys"},{"key":"e_1_3_1_46_2","doi-asserted-by":"crossref","unstructured":"Sewon Min Mike Lewis Luke Zettlemoyer and Hannaneh Hajishirzi. 2021. MetaiCL: Learning to learn in context. arXiv:2110.15943. Retrieved from https:\/\/arxiv.org\/abs\/2110.15943","DOI":"10.18653\/v1\/2022.naacl-main.201"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00205"},{"key":"e_1_3_1_48_2","unstructured":"Erik Nijkamp Hiroaki Hayashi Caiming Xiong Silvio Savarese and Yingbo Zhou. 2023. CodeGen2: Lessons for training LLMs on programming and natural languages. arXiv:2305.02309. Retrieved from https:\/\/arxiv.org\/abs\/2305.02309"},{"key":"e_1_3_1_49_2","article-title":"2023","author":"Nijkamp Erik","unstructured":"Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. 2023. CodeGen: An open large language model for code with multi-turn program synthesis. arXiv:2203.13474. Retrieved from https:\/\/arxiv.org\/abs\/2203.13474","journal-title":"CodeGen: An open large language model for code with multi-turn program synthesis"},{"key":"e_1_3_1_50_2","first-page":"776","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing","author":"Norouzi Sajad","year":"2021","unstructured":"Sajad Norouzi, Keyi Tang, and Yanshuai Cao. 2021. Code generation from natural language with less prior knowledge and more monolingual data. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Vol. 2: Short Papers), 776\u2013785."},{"key":"e_1_3_1_51_2","unstructured":"OpenAI. 2023. GPT-4 technical report. arXiv:2303.08774. Retrieved from https:\/\/arxiv.org\/abs\/2303.08774"},{"key":"e_1_3_1_52_2","first-page":"27730","article-title":"Training language models to follow instructions with human feedback","author":"Ouyang Long","year":"2022","unstructured":"Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et\u00a0al. 2022. Training language models to follow instructions with human feedback. In Proceedings of the 36th International Conference on Neural Information Processing Systems, 27730\u201327744.","journal-title":"Proceedings of the 36th International Conference on Neural Information Processing Systems"},{"key":"e_1_3_1_53_2","doi-asserted-by":"crossref","unstructured":"Md Rizwan Parvez Wasi Uddin Ahmad Saikat Chakraborty Baishakhi Ray and Kai-Wei Chang. 2021. Retrieval augmented code generation and summarization. arXiv:2108.11601. Retrieved from https:\/\/arxiv.org\/abs\/2108.11601","DOI":"10.18653\/v1\/2021.findings-emnlp.232"},{"key":"e_1_3_1_54_2","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1145\/3524459.3527351","volume-title":"Proceedings of the 3rd International Workshop on Automated Program Repair","author":"Prenner Julian Aron","year":"2022","unstructured":"Julian Aron Prenner, Hlib Babii, and Romain Robbes. 2022. Can OpenAI's codex fix bugs? An evaluation on QuixBugs. In Proceedings of the 3rd International Workshop on Automated Program Repair, 69\u201375."},{"issue":"8","key":"e_1_3_1_55_2","first-page":"9","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford Alec","year":"2019","unstructured":"Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9.","journal-title":"OpenAI blog"},{"key":"e_1_3_1_56_2","unstructured":"Shuo Ren Daya Guo Shuai Lu Long Zhou Shujie Liu Duyu Tang Neel Sundaresan Ming Zhou Ambrosio Blanco and Shuai Ma. 2020. Codebleu: A method for automatic evaluation of code synthesis. arXiv:2009.10297. Retrieved from https:\/\/arxiv.org\/abs\/2009.10297"},{"key":"e_1_3_1_57_2","unstructured":"Baptiste Roziere Jonas Gehring Fabian Gloeckle Sten Sootla Itai Gat Xiaoqing Ellen Tan Yossi Adi Jingyu Liu Tal Remez J\u00e9r\u00e9my Rapin et\u00a0al. 2023. Code Llama: Open foundation models for code. arXiv:2308.12950. Retrieved from https:\/\/arxiv.org\/abs\/2308.12950"},{"issue":"4","key":"e_1_3_1_58_2","doi-asserted-by":"crossref","first-page":"94","DOI":"10.1007\/s10664-024-10457-5","article-title":"Utilization of pre-trained language models for adapter-based knowledge transfer in software engineering","volume":"29","author":"Saberi Iman","year":"2024","unstructured":"Iman Saberi, Fatemeh Fard, and Fuxiang Chen. 2024. Utilization of pre-trained language models for adapter-based knowledge transfer in software engineering. Empirical Software Engineering 29, 4 (2024), 94.","journal-title":"Empirical Software Engineering"},{"key":"e_1_3_1_59_2","unstructured":"Iman Saberi and Fatemeh H. Fard. 2023. Model-agnostic syntactical information for pre-trained programming language models. arXiv:2303.06233. Retrieved from https:\/\/arxiv.org\/abs\/2303.06233"},{"key":"e_1_3_1_60_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.01438"},{"key":"e_1_3_1_61_2","first-page":"4596","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Shazeer Noam","year":"2018","unstructured":"Noam Shazeer and Mitchell Stern. 2018. Adafactor: Adaptive learning rates with sublinear memory cost. In Proceedings of the International Conference on Machine Learning. PMLR, 4596\u20134604."},{"key":"e_1_3_1_62_2","first-page":"31693","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Shrivastava Disha","year":"2023","unstructured":"Disha Shrivastava, Hugo Larochelle, and Daniel Tarlow. 2023. Repository-level prompt generation for large language models of code. In Proceedings of the International Conference on Machine Learning. PMLR, 31693\u201331715."},{"key":"e_1_3_1_63_2","doi-asserted-by":"crossref","unstructured":"Emma Strubell Ananya Ganesh and Andrew McCallum. 2019. Energy and policy considerations for deep learning in NLP. arXiv:1906.02243. Retrieved from https:\/\/arxiv.org\/abs\/1906.02243","DOI":"10.18653\/v1\/P19-1355"},{"key":"e_1_3_1_64_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i05.6430"},{"key":"e_1_3_1_65_2","unstructured":"Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux Timoth\u00e9e Lacroix Baptiste Rozi\u00e8re Naman Goyal Eric Hambro Faisal Azhar et\u00a0al. 2023. Llama: Open and efficient foundation language models. arXiv:2302.13971. Retrieved from https:\/\/arxiv.org\/abs\/2302.13971"},{"key":"e_1_3_1_66_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00577"},{"key":"e_1_3_1_67_2","first-page":"1","article-title":"Expectation vs. experience: Evaluating the usability of code generation tools powered by large language models","author":"Vaithilingam Priyan","year":"2022","unstructured":"Priyan Vaithilingam, Tianyi Zhang, and Elena L. Glassman. 2022. Expectation vs. experience: Evaluating the usability of code generation tools powered by large language models. In Proceedings of the CHI Conference on Human Factors in Computing Systems Extended Abstracts, 1\u20137.","journal-title":"Proceedings of the CHI Conference on Human Factors in Computing Systems Extended Abstracts"},{"key":"e_1_3_1_68_2","article-title":"Attention is all you need","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems.","journal-title":"Proceedings of the 31st International Conference on Neural Information Processing Systems"},{"key":"e_1_3_1_69_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549113"},{"key":"e_1_3_1_70_2","doi-asserted-by":"crossref","unstructured":"Deze Wang Boxing Chen Shanshan Li Wei Luo Shaoliang Peng Wei Dong and Xiangke Liao. 2023. One adapter for all programming languages? Adapter tuning for code search and summarization. arXiv:2303.15822. Retrieved from https:\/\/arxiv.org\/abs\/2303.15822","DOI":"10.1109\/ICSE48619.2023.00013"},{"key":"e_1_3_1_71_2","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3616256"},{"key":"e_1_3_1_72_2","doi-asserted-by":"crossref","unstructured":"Yue Wang Hung Le Akhilesh Deepak Gotmare Nghi D. Q. Bui Junnan Li and Steven C. H. Hoi. 2023. CodeT5+: Open code large language models for code understanding and generation. arXiv:2305.07922. Retrieved from https:\/\/arxiv.org\/abs\/2305.07922","DOI":"10.18653\/v1\/2023.emnlp-main.68"},{"key":"e_1_3_1_73_2","doi-asserted-by":"crossref","unstructured":"Yue Wang Weishi Wang Shafiq Joty and Steven C. H. Hoi. 2021. CodeT5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation. arXiv:2109.00859. Retrieved from https:\/\/arxiv.org\/abs\/2109.00859","DOI":"10.18653\/v1\/2021.emnlp-main.685"},{"key":"e_1_3_1_74_2","unstructured":"Zhiruo Wang Shuyan Zhou Daniel Fried and Graham Neubig. 2022. Execution-based evaluation for open-domain code generation. arXiv:2212.10481. Retrieved from https:\/\/arxiv.org\/abs\/2212.10481"},{"key":"e_1_3_1_75_2","doi-asserted-by":"crossref","unstructured":"Albert Webson and Ellie Pavlick. 2021. Do prompt-based models really understand the meaning of their prompts? arXiv:2109.01247. Retrieved from https:\/\/arxiv.org\/abs\/2109.01247","DOI":"10.18653\/v1\/2022.naacl-main.167"},{"key":"e_1_3_1_76_2","unstructured":"Jason Wei Maarten Bosma Vincent Y. Zhao Kelvin Guu Adams Wei Yu Brian Lester Nan Du Andrew M. Dai and Quoc V. Le. 2021. Finetuned language models are zero-shot learners. arXiv:2109.01652. Retrieved from https:\/\/arxiv.org\/abs\/2109.01652"},{"key":"e_1_3_1_77_2","unstructured":"Jason Wei Yi Tay Rishi Bommasani Colin Raffel Barret Zoph Sebastian Borgeaud Dani Yogatama Maarten Bosma Denny Zhou Donald Metzler et\u00a0al. 2022. Emergent abilities of large language models. arXiv:2206.07682. Retrieved from https:\/\/arxiv.org\/abs\/2206.07682"},{"key":"e_1_3_1_78_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510455.3512771"},{"key":"e_1_3_1_79_2","doi-asserted-by":"crossref","unstructured":"Martin Weyssow Xin Zhou Kisub Kim David Lo and Houari Sahraoui. 2023. On the usage of continual learning for out-of-distribution generalization in pre-trained language models of code. arXiv:2305.04106. Retrieved from https:\/\/arxiv.org\/abs\/2305.04106","DOI":"10.1145\/3611643.3616244"},{"key":"e_1_3_1_80_2","doi-asserted-by":"crossref","unstructured":"Thomas Wolf Lysandre Debut Victor Sanh Julien Chaumond Clement Delangue Anthony Moi Pierric Cistac Tim Rault R\u00e9mi Louf Morgan Funtowicz et\u00a0al. 2019. Huggingface's transformers: State-of-the-art natural language processing. arXiv:1910.03771. Retrieved from https:\/\/arxiv.org\/abs\/1910.03771","DOI":"10.18653\/v1\/2020.emnlp-demos.6"},{"key":"e_1_3_1_81_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00129"},{"key":"e_1_3_1_82_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549101"},{"key":"e_1_3_1_83_2","doi-asserted-by":"crossref","unstructured":"Frank F. Xu Uri Alon Graham Neubig and Vincent Josua Hellendoorn. 2022. A systematic evaluation of large language models of code (MAPS \u201922). ACM New York NY 1\u201310. DOI: https:\/\/doi.org\/10.1145\/3520312.3534862","DOI":"10.1145\/3520312.3534862"},{"key":"e_1_3_1_84_2","unstructured":"Prateek Yadav Qing Sun Hantian Ding Xiaopeng Li Dejiao Zhang Ming Tan Xiaofei Ma Parminder Bhatia Ramesh Nallapati Murali Krishna Ramanathan et\u00a0al. 2023. Exploring continual learning for code generation models. arXiv:2307.02435. Retrieved from https:\/\/arxiv.org\/abs\/2307.02435"},{"key":"e_1_3_1_85_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2022.111577"},{"key":"e_1_3_1_86_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.01839"},{"key":"e_1_3_1_87_2","doi-asserted-by":"publisher","DOI":"10.1145\/3196398.3196408"},{"key":"e_1_3_1_88_2","first-page":"476","article-title":"Learning to mine aligned code and natural language pairs from stack overflow","author":"Yin Pengcheng","year":"2018","unstructured":"Pengcheng Yin, Bowen Deng, Edgar Chen, Bogdan Vasilescu, and Graham Neubig. 2018. Learning to mine aligned code and natural language pairs from stack overflow. In Proceedings of the IEEE\/ACM 15th International Conference on Mining Software Repositories (MSR \u201918). IEEE, 476\u2013486.","journal-title":"Proceedings of the IEEE\/ACM 15th International Conference on Mining Software Repositories (MSR \u201918)"},{"key":"e_1_3_1_89_2","unstructured":"Zhiqiang Yuan Junwei Liu Qiancheng Zi Mingwei Liu Xin Peng and Yiling Lou. 2023. Evaluating instruction-tuned large language models on code comprehension and generation. arXiv:2308.01240. Retrieved from https:\/\/arxiv.org\/abs\/2308.01240"},{"key":"e_1_3_1_90_2","unstructured":"Bowen Zhang Xianghua Fu Daijun Ding Hu Huang Yangyang Li and Liwen Jing. 2023. Investigating chain-of-thought with ChatGPT for stance detection on social media. arXiv:2304.03087. Retrieved from https:\/\/arxiv.org\/abs\/2304.03087"},{"key":"e_1_3_1_91_2","first-page":"12697","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Zhao Zihao","year":"2021","unstructured":"Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. Calibrate before use: Improving few-shot performance of language models. In Proceedings of the International Conference on Machine Learning. PMLR, 12697\u201312706."},{"key":"e_1_3_1_92_2","volume-title":"Proceedings of the 11th International Conference on Learning Representations","author":"Zhou Shuyan","year":"2023","unstructured":"Shuyan Zhou, Uri Alon, Frank F. Xu, Zhengbao Jiang, and Graham Neubig. 2023. Docprompting: Generating code by retrieving the docs. In Proceedings of the 11th International Conference on Learning Representations."},{"key":"e_1_3_1_93_2","first-page":"425","volume-title":"Proceedings of the IEEE International Conference on Software Maintenance and Evolution (ICSME \u201921)","author":"Zhou Xin","year":"2021","unstructured":"Xin Zhou, DongGyun Han, and David Lo. 2021. Assessing generalizability of codebert. In Proceedings of the IEEE International Conference on Software Maintenance and Evolution (ICSME \u201921). IEEE, 425\u2013436."}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3714461","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,18]],"date-time":"2025-08-18T02:55:27Z","timestamp":1755485727000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3714461"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,17]]},"references-count":92,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2025,9,30]]}},"alternative-id":["10.1145\/3714461"],"URL":"https:\/\/doi.org\/10.1145\/3714461","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,17]]},"assertion":[{"value":"2024-01-16","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-01-09","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-08-17","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}