{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T08:46:17Z","timestamp":1784277977241,"version":"3.55.0"},"reference-count":108,"publisher":"Association for Computing Machinery (ACM)","issue":"7","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2025,9,30]]},"abstract":"<jats:p>Pre-trained models (PTMs) have succeeded in various software engineering (SE) tasks following the \u201cpre-train then fine-tune\u201d paradigm. As fully fine-tuning all parameters of PTMs can be computationally expensive, a potential solution is parameter-efficient fine-tuning (PEFT), which freezes PTMs while introducing extra parameters. Although PEFT methods have been applied to SE tasks, researchers often focus on specific scenarios and lack a comprehensive comparison of PTMs from different aspects such as field, size, and architecture. To fill this gap, we have conducted an empirical study on six PEFT methods, eight PTMs, and four SE tasks. The experimental results reveal several noteworthy findings. For example, model architecture has little impact on PTM performance when using PEFT methods. Additionally, we provide a comprehensive discussion of PEFT methods from three perspectives. First, we analyze the effectiveness and efficiency of PEFT methods. Second, we explore the impact of the scaling factor hyperparameter. Finally, we investigate the application of PEFT methods on the latest open source large language model, Llama 3.2. These findings provide valuable insights to guide future researchers in effectively applying PEFT methods to SE tasks.<\/jats:p>","DOI":"10.1145\/3722107","type":"journal-article","created":{"date-parts":[[2025,3,7]],"date-time":"2025-03-07T15:57:21Z","timestamp":1741363041000},"page":"1-34","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Experimental Evaluation of Parameter-Efficient Fine-Tuning for Software Engineering Tasks"],"prefix":"10.1145","volume":"34","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-3385-7607","authenticated-orcid":false,"given":"Wentao","family":"Zou","sequence":"first","affiliation":[{"name":"National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-2492-2530","authenticated-orcid":false,"given":"Zongwen","family":"Shen","sequence":"additional","affiliation":[{"name":"National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-1495-2697","authenticated-orcid":false,"given":"Qi","family":"Li","sequence":"additional","affiliation":[{"name":"National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1773-0942","authenticated-orcid":false,"given":"Jidong","family":"Ge","sequence":"additional","affiliation":[{"name":"National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9270-5072","authenticated-orcid":false,"given":"Chuanyi","family":"Li","sequence":"additional","affiliation":[{"name":"National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1180-3891","authenticated-orcid":false,"given":"Xiang","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence and Computer Science, Nantong University, Nantong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0217-2469","authenticated-orcid":false,"given":"Xiaoyu","family":"Shen","sequence":"additional","affiliation":[{"name":"Eastern Institute for Advanced Study, Ningbo, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7790-0195","authenticated-orcid":false,"given":"Liguo","family":"Huang","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Southern Methodist University, Dallas, Texas, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-1102-9584","authenticated-orcid":false,"given":"Bin","family":"Luo","sequence":"additional","affiliation":[{"name":"National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,8,14]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.naacl-main.223"},{"key":"e_1_3_3_3_2","doi-asserted-by":"crossref","unstructured":"Wasi Uddin Ahmad Saikat Chakraborty Baishakhi Ray and Kai-Wei Chang. 2021. Unified pre-training for program understanding and generation. arXiv:2103.06333. Retrieved from https:\/\/arxiv.org\/abs\/2103.06333","DOI":"10.18653\/v1\/2021.naacl-main.211"},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510049"},{"key":"e_1_3_3_5_2","unstructured":"Muhammad Azeem Akbar Arif Ali Khan and Peng Liang. 2023. Ethical aspects of ChatGPT in software engineering research. arXiv:2306.07557. Retrieved from https:\/\/arxiv.org\/abs\/2306.07557"},{"key":"e_1_3_3_6_2","unstructured":"Shamil Ayupov and Nadezhda Chirkova. 2022. Parameter-efficient finetuning of transformers for source code. arXiv:2212.05901. Retrieved from https:\/\/arxiv.org\/abs\/2212.05901"},{"key":"e_1_3_3_7_2","unstructured":"Jinze Bai Shuai Bai Yunfei Chu Zeyu Cui Kai Dang Xiaodong Deng Yang Fan Wenbin Ge Yu Han Fei Huang et\u00a0al. 2023. Qwen technical report. arXiv:2309.16609. Retrieved from https:\/\/arxiv.org\/abs\/2309.16609"},{"key":"e_1_3_3_8_2","unstructured":"Rishi Bommasani Drew A. Hudson Ehsan Adeli Russ Altman Simran Arora Sydney von Arx Michael S. Bernstein Jeannette Bohg Antoine Bosselut Emma Brunskill et\u00a0al. 2021. On the opportunities and risks of foundation models. arXiv:2108.07258. Retrieved from https:\/\/arxiv.org\/abs\/2108.07258"},{"key":"e_1_3_3_9_2","first-page":"1877","article-title":"Language models are few-shot learners","volume":"33","author":"Brown Tom","year":"2020","unstructured":"Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D. Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et\u00a0al. 2020. Language models are few-shot learners. Advances in Neural Information Processing Systems 33 (2020), 1877\u20131901.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.inlg-1.36"},{"key":"e_1_3_3_11_2","first-page":"27","volume-title":"Proceedings on I (Still) Can\u2019t Believe It\u2019s Not Better! Workshop at NeurIPS 2021","author":"Chatelain Am\u00e9lie","year":"2022","unstructured":"Am\u00e9lie Chatelain, Amine Djeghri, Daniel Hesslow, and Julien Launay. 2022. Is the number of trainable parameters all that actually matters? In Proceedings on I (Still) Can\u2019t Believe It\u2019s Not Better! Workshop at NeurIPS 2021. PMLR, 27\u201332."},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524610.3527917"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3468611"},{"key":"e_1_3_3_14_2","first-page":"10088","article-title":"Qlora: Efficient finetuning of quantized llms","volume":"36","author":"Dettmers Tim","year":"2023","unstructured":"Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. Qlora: Efficient finetuning of quantized llms. Advances in Neural Information Processing Systems 36 (2023), 10088\u201310115.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_15_2","unstructured":"Jacob Devlin Ming-Wei Chang Kenton Lee and Kristina Toutanova. 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805. Retrieved from https:\/\/arxiv.org\/abs\/1810.04805"},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-023-00626-4"},{"key":"e_1_3_3_17_2","unstructured":"Abhimanyu Dubey Abhinav Jauhri Abhinav Pandey Abhishek Kadian Ahmad Al-Dahle Aiesha Letman Akhil Mathur Alan Schelten Amy Yang Angela Fan et\u00a0al. 2024. The Llama 3 herd of models. arXiv:2407.21783. Retrieved from https:\/\/arxiv.org\/abs\/2407.21783"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.139"},{"key":"e_1_3_3_19_2","volume-title":"Proceedings of the 11th International Conference on Learning Representations (ICLR \u201923)","author":"Fried Daniel","year":"2023","unstructured":"Daniel Fried, Armen Aghajanyan, Jessy Lin, Sida Wang, Eric Wallace, Freda Shi, Ruiqi Zhong, Scott Yih, Luke Zettlemoyer, and Mike Lewis. 2023. InCoder: A generative model for code infilling and synthesis. In Proceedings of the 11th International Conference on Learning Representations (ICLR \u201923). Retrieved from https:\/\/openreview.net\/forum?id=hQwb-lbM6EL"},{"key":"e_1_3_3_20_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v37i11.26505"},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524610.3527892"},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-021-01453-z"},{"key":"e_1_3_3_23_2","doi-asserted-by":"crossref","unstructured":"Daya Guo Shuai Lu Nan Duan Yanlin Wang Ming Zhou and Jian Yin. 2022. UniXcoder: Unified cross-modal pre-training for code representation. arXiv:2203.03850. Retrieved from https:\/\/arxiv.org\/abs\/2203.03850","DOI":"10.18653\/v1\/2022.acl-long.499"},{"key":"e_1_3_3_24_2","unstructured":"Daya Guo Shuo Ren Shuai Lu Zhangyin Feng Duyu Tang Shujie Liu Long Zhou Nan Duan Alexey Svyatkovskiy Shengyu Fu et\u00a0al. 2020. GraphCodeBERT: Pre-training code representations with data flow. arXiv:2009.08366. Retrieved from https:\/\/arxiv.org\/abs\/2009.08366"},{"key":"e_1_3_3_25_2","unstructured":"Demi Guo Alexander M. Rush and Yoon Kim. 2020. Parameter-efficient transfer learning with diff pruning. arXiv:2012.07463. Retrieved from https:\/\/arxiv.org\/abs\/2012.07463"},{"key":"e_1_3_3_26_2","unstructured":"Daya Guo Qihao Zhu Dejian Yang Zhenda Xie Kai Dong Wentao Zhang Guanting Chen Xiao Bi Y. Wu Y. K. Li Fuli Luo Yingfei Xiong and Wenfeng Liang. 2024. DeepSeek-Coder: When the large language model meets programming\u2014The rise of code intelligence. arXiv:2401.14196. Retrieved from https:\/\/arxiv.org\/abs\/2401.14196"},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3623306"},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3379597.3387449"},{"key":"e_1_3_3_29_2","volume-title":"Proceedings of the International Conference on Learning Representations","author":"He Junxian","year":"2021","unstructured":"Junxian He, Chunting Zhou, Xuezhe Ma, Taylor Berg-Kirkpatrick, and Graham Neubig. 2021. Towards a unified view of parameter-efficient transfer learning. In Proceedings of the International Conference on Learning Representations."},{"key":"e_1_3_3_30_2","first-page":"2790","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Houlsby Neil","year":"2019","unstructured":"Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for NLP. In Proceedings of the International Conference on Machine Learning. PMLR, 2790\u20132799."},{"key":"e_1_3_3_31_2","unstructured":"Edward J. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang and Weizhu Chen. 2021. LoRA: Low-rank adaptation of large language models. arXiv:2106.09685. Retrieved from https:\/\/arxiv.org\/abs\/2106.09685"},{"key":"e_1_3_3_32_2","unstructured":"Hamel Husain Ho-Hsiang Wu Tiferet Gazit Miltiadis Allamanis and Marc Brockschmidt. 2019. CodeSearchNet challenge: Evaluating the state of semantic code search. arXiv:1909.09436. Retrieved from https:\/\/arxiv.org\/abs\/1909.09436"},{"key":"e_1_3_3_33_2","unstructured":"Srinivasan Iyer Ioannis Konstas Alvin Cheung and Luke Zettlemoyer. 2018. Mapping language to code in programmatic context. arXiv:1808.09588. Retrieved from https:\/\/arxiv.org\/abs\/1808.09588"},{"key":"e_1_3_3_34_2","unstructured":"Juyong Jiang Fan Wang Jiasi Shen Sungju Kim and Sunghun Kim. 2024. A survey on large language models for code generation. arXiv:2406.00515. Retrieved from https:\/\/arxiv.org\/abs\/2406.00515"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.nlp.2023.100048"},{"key":"e_1_3_3_36_2","first-page":"5110","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Kanade Aditya","year":"2020","unstructured":"Aditya Kanade, Petros Maniatis, Gogul Balakrishnan, and Kensen Shi. 2020. Learning and evaluating contextual embedding of source code. In Proceedings of the International Conference on Machine Learning. PMLR, 5110\u20135121."},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.243"},{"key":"e_1_3_3_38_2","doi-asserted-by":"crossref","unstructured":"Mike Lewis Yinhan Liu Naman Goyal Marjan Ghazvininejad Abdelrahman Mohamed Omer Levy Ves Stoyanov and Luke Zettlemoyer. 2019. BART: Denoising sequence-to-sequence pre-training for natural language generation translation and comprehension. arXiv:1910.13461. Retrieved from https:\/\/arxiv.org\/abs\/1910.13461","DOI":"10.18653\/v1\/2020.acl-main.703"},{"key":"e_1_3_3_39_2","unstructured":"Raymond Li Loubna Ben Allal Yangtian Zi Niklas Muennighoff Denis Kocetkov Chenghao Mou Marc Marone Christopher Akiki Jia Li Jenny Chim et al . 2023. Starcoder: May the source be with you! arXiv:2305.06161. Retrieved from https:\/\/arxiv.org\/abs\/2305.06161"},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.acl-long.353"},{"key":"e_1_3_3_41_2","unstructured":"Vladislav Lialin Vijeta Deshpande and Anna Rumshisky. 2023. Scaling down to scale up: A guide to parameter-efficient fine-tuning. arXiv:2303.15647. Retrieved from https:\/\/arxiv.org\/abs\/2303.15647"},{"key":"e_1_3_3_42_2","first-page":"1950","article-title":"Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning","volume":"35","author":"Liu Haokun","year":"2022","unstructured":"Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A. Raffel. 2022. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. Advances in Neural Information Processing Systems 35 (2022), 1950\u20131965.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE56229.2023.00125"},{"key":"e_1_3_3_44_2","doi-asserted-by":"crossref","unstructured":"Pengfei Liu Weizhe Yuan Jinlan Fu Zhengbao Jiang Hiroaki Hayashi and Graham Neubig. 2023. Pre-train prompt and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys 55 9 (2023) 1\u201335.","DOI":"10.1145\/3560815"},{"key":"e_1_3_3_45_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.aiopen.2023.08.012"},{"key":"e_1_3_3_46_2","unstructured":"Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy Mike Lewis Luke Zettlemoyer and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv:1907.11692. Retrieved from https:\/\/arxiv.org\/abs\/1907.11692"},{"key":"e_1_3_3_47_2","doi-asserted-by":"crossref","unstructured":"Zihan Liu Genta Indra Winata and Pascale Fung. 2021. Continual mixed-language pre-training for extremely low-resource neural machine translation. arXiv:2105.03953. Retrieved from https:\/\/arxiv.org\/abs\/2105.03953","DOI":"10.18653\/v1\/2021.findings-acl.239"},{"key":"e_1_3_3_48_2","unstructured":"Shuai Lu Daya Guo Shuo Ren Junjie Huang Alexey Svyatkovskiy Ambrosio Blanco Colin Clement Dawn Drain Daxin Jiang Duyu Tang et\u00a0al. 2021. CodeXGLUE: A machine learning benchmark dataset for code understanding and generation. arXiv:2102.04664. Retrieved from https:\/\/arxiv.org\/abs\/2102.04664"},{"key":"e_1_3_3_49_2","unstructured":"Wei Ma Shangqing Liu Wenhan Wang Qiang Hu Ye Liu Cen Zhang Liming Nie and Yang Liu. 2023. The scope of ChatGPT in software engineering: A thorough investigation. arXiv:2305.12138. Retrieved from https:\/\/arxiv.org\/abs\/2305.12138"},{"key":"e_1_3_3_50_2","doi-asserted-by":"crossref","unstructured":"Antonio Mastropaolo Luca Pascarella Emanuela Guglielmi Matteo Ciniselli Simone Scalabrino Rocco Oliveto and Gabriele Bavota. 2023. On the robustness of code generation techniques: An empirical study on GitHub copilot. arXiv:2302.00438. Retrieved from https:\/\/arxiv.org\/abs\/2302.00438","DOI":"10.1109\/ICSE48619.2023.00181"},{"key":"e_1_3_3_51_2","unstructured":"Aditya Mogadala Xiaoyu Shen and Dietrich Klakow. 2020. Integrating image captioning with rule-based entity masking. arXiv:2007.11690. Retrieved from https:\/\/arxiv.org\/abs\/2007.11690"},{"key":"e_1_3_3_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2019.00099"},{"key":"e_1_3_3_53_2","volume-title":"Proceedings of the 11th International Conference on Learning Representations","author":"Nijkamp Erik","year":"2022","unstructured":"Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. 2022. CodeGen: An open large language model for code with multi-turn program synthesis. In Proceedings of the 11th International Conference on Learning Representations."},{"key":"e_1_3_3_54_2","unstructured":"Changan Niu Chuanyi Li Bin Luo and Vincent Ng. 2022. Deep learning meets software engineering: A survey on pre-trained models of source code. arXiv:2205.11739. Retrieved from https:\/\/arxiv.org\/abs\/2205.11739"},{"key":"e_1_3_3_55_2","unstructured":"Changan Niu Chuanyi Li Vincent Ng Dongxiao Chen Jidong Ge and Bin Luo. 2023. An empirical comparison of pre-trained models of source code. arXiv:2302.04026. Retrieved from https:\/\/arxiv.org\/abs\/2302.04026"},{"key":"e_1_3_3_56_2","unstructured":"Changan Niu Chuanyi Li Vincent Ng Jidong Ge Liguo Huang and Bin Luo. 2022. SPT-code: Sequence-to-sequence pre-training for learning the representation of source code. arXiv:2201.01549. Retrieved from https:\/\/arxiv.org\/abs\/2201.01549"},{"key":"e_1_3_3_57_2","first-page":"311","volume-title":"Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics","author":"Papineni Kishore","year":"2002","unstructured":"Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, 311\u2013318."},{"key":"e_1_3_3_58_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.617"},{"key":"e_1_3_3_59_2","unstructured":"Alec Radford Karthik Narasimhan Tim Salimans and Ilya Sutskever. 2018. Improving language understanding by generative pre-training (2018). Retrieved from https:\/\/openai.com\/index\/language-unsupervised\/"},{"issue":"8","key":"e_1_3_3_60_2","first-page":"9","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford Alec","year":"2019","unstructured":"Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 (2019), 9.","journal-title":"OpenAI Blog"},{"key":"e_1_3_3_61_2","doi-asserted-by":"publisher","DOI":"10.5555\/3455716.3455856"},{"key":"e_1_3_3_62_2","first-page":"506","article-title":"Learning multiple visual domains with residual adapters","volume":"30","author":"Rebuffi Sylvestre-Alvise","year":"2017","unstructured":"Sylvestre-Alvise Rebuffi, Hakan Bilen, and Andrea Vedaldi. 2017. Learning multiple visual domains with residual adapters. Advances in Neural Information Processing Systems 30 (2017), 506\u2013516.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_63_2","unstructured":"Shuo Ren Daya Guo Shuai Lu Long Zhou Shujie Liu Duyu Tang Neel Sundaresan Ming Zhou Ambrosio Blanco and Shuai Ma. 2020. CodeBLEU: A method for automatic evaluation of code synthesis. arXiv:2009.10297. Retrieved from https:\/\/arxiv.org\/abs\/2009.10297"},{"key":"e_1_3_3_64_2","unstructured":"Baptiste Roziere Jonas Gehring Fabian Gloeckle Sten Sootla Itai Gat Xiaoqing Ellen Tan Yossi Adi Jingyu Liu Tal Remez J\u00e9r\u00e9my Rapin et\u00a0al. 2023. Code Llama: Open foundation models for code. arXiv:2308.12950. Retrieved from https:\/\/arxiv.org\/abs\/2308.12950"},{"key":"e_1_3_3_65_2","doi-asserted-by":"crossref","unstructured":"Iman Saberi Fatemeh Fard and Fuxiang Chen. 2023. Utilization of pre-trained language model for adapter-based knowledge transfer in software engineering. arXiv:2307.08540. Retrieved from https:\/\/arxiv.org\/abs\/2307.08540","DOI":"10.1007\/s10664-024-10457-5"},{"key":"e_1_3_3_66_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR59073.2023.00036"},{"key":"e_1_3_3_67_2","first-page":"111","volume-title":"Proceedings of the 5th Workshop on e-Commerce and NLP (ECNLP 5)","author":"Shen Xiaoyu","unstructured":"Xiaoyu Shen, Gianni Barlacchi, Marco Del Tredici, Weiwei Cheng, and Adri\u00e0 de Gispert. 2022. semiPQA: A study on product question answering over semi-structured data. In Proceedings of the 5th Workshop on e-Commerce and NLP (ECNLP 5), 111\u2013120."},{"key":"e_1_3_3_68_2","doi-asserted-by":"crossref","unstructured":"Xiaoyu Shen Youssef Oualil Clayton Greenberg Mittul Singh and Dietrich Klakow. 2017. Estimation of gap between current language models and human performance. In INTERSPEECH 553\u2013557","DOI":"10.21437\/Interspeech.2017-729"},{"key":"e_1_3_3_69_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00185"},{"key":"e_1_3_3_70_2","unstructured":"Ensheng Shi Yanlin Wang Hongyu Zhang Lun Du Shi Han Dongmei Zhang and Hongbin Sun. 2023. Towards efficient fine-tuning of pre-trained code models: An experimental study and beyond. arXiv:2304.05216. Retrieved from https:\/\/arxiv.org\/abs\/2304.05216"},{"key":"e_1_3_3_71_2","doi-asserted-by":"publisher","DOI":"10.1145\/3551349.3556964"},{"key":"e_1_3_3_72_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.is.2021.101718"},{"key":"e_1_3_3_73_2","doi-asserted-by":"crossref","unstructured":"Yisheng Song Ting Wang Puyu Cai Subrota K. Mondal and Jyoti Prakash Sahoo. 2023. A comprehensive survey of few-shot learning: Evolution applications challenges and opportunities. ACM Computing Surveys 55 13s (2023) 1\u201340.","DOI":"10.1145\/3582688"},{"key":"e_1_3_3_74_2","unstructured":"Benjamin Steenhoek Md Mahbubur Rahman Richard Jiles and Wei Le. 2022. An empirical study of deep learning models for vulnerability detection. arXiv:2212.08109. Retrieved from https:\/\/arxiv.org\/abs\/2212.08109"},{"key":"e_1_3_3_75_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00188"},{"key":"e_1_3_3_76_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.535"},{"key":"e_1_3_3_77_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.acl-long.65"},{"key":"e_1_3_3_78_2","unstructured":"Hui Su Xiao Zhou Houjin Yu Xiaoyu Shen Yuwen Chen Zilin Zhu Yang Yu and Jie Zhou. 2022. WeLM: A well-read pre-trained language model for Chinese. arXiv:2209.10372. Retrieved from https:\/\/arxiv.org\/abs\/2209.10372"},{"key":"e_1_3_3_79_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME.2014.77"},{"key":"e_1_3_3_80_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3417058"},{"key":"e_1_3_3_81_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR52588.2021.00045"},{"key":"e_1_3_3_82_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE56229.2023.00076"},{"key":"e_1_3_3_83_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE51524.2021.9678882"},{"key":"e_1_3_3_84_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510224"},{"key":"e_1_3_3_85_2","unstructured":"Hugo Touvron Louis Martin Kevin Stone Peter Albert Amjad Almahairi Yasmine Babaei Nikolay Bashlykov Soumya Batra Prajjwal Bhargava Shruti Bhosale et\u00a0al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv:2307.09288. Retrieved from https:\/\/arxiv.org\/abs\/2307.09288"},{"key":"e_1_3_3_86_2","doi-asserted-by":"crossref","unstructured":"Rosalia Tufano Luca Pascarella and Gabriele Bavota. 2023. Automating code-related tasks through transformers: The impact of pre-training. arXiv:2302.04048. Retrieved from https:\/\/arxiv.org\/abs\/2302.04048","DOI":"10.1109\/ICSE48619.2023.00203"},{"key":"e_1_3_3_87_2","first-page":"6000","article-title":"Attention is all you need","volume":"30","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in Neural Information Processing Systems 30 (2017), 6000\u20136010.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_88_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00013"},{"key":"e_1_3_3_89_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639217"},{"key":"e_1_3_3_90_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.685"},{"key":"e_1_3_3_91_2","doi-asserted-by":"publisher","DOI":"10.1186\/s40537-016-0043-6"},{"key":"e_1_3_3_92_2","unstructured":"Martin Weyssow Xin Zhou Kisub Kim David Lo and Houari Sahraoui. 2023. Exploring parameter-efficient fine-tuning techniques for code generation with large language models. arXiv:2308.10462. Retrieved from https:\/\/arxiv.org\/abs\/2308.10462"},{"key":"e_1_3_3_93_2","doi-asserted-by":"publisher","DOI":"10.1109\/JAS.2023.123618"},{"key":"e_1_3_3_94_2","unstructured":"Hu Xu Bing Liu Lei Shu and Philip S. Yu. 2019. BERT post-training for review reading comprehension and aspect-based sentiment analysis. arXiv:1904.02232. Retrieved from https:\/\/arxiv.org\/abs\/1904.02232"},{"key":"e_1_3_3_95_2","unstructured":"HanXiang Xu ShenAo Wang Ningke Li Kailong Wang Yanjie Zhao Kai Chen Ting Yu Yang Liu and HaoYu Wang. 2024. Large language models for cyber security: A systematic literature review. arXiv:2405.04760. Retrieved from https:\/\/arxiv.org\/abs\/2405.04760"},{"key":"e_1_3_3_96_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER48275.2020.9054840"},{"key":"e_1_3_3_97_2","first-page":"5753","article-title":"XLNet: Generalized autoregressive pretraining for language understanding","volume":"32","author":"Yang Zhilin","year":"2019","unstructured":"Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R. Salakhutdinov, and Quoc V. Le. 2019. XLNet: Generalized autoregressive pretraining for language understanding. Advances in Neural Information Processing Systems 32 (2019), 5753\u20135763.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_98_2","doi-asserted-by":"crossref","unstructured":"Elad Ben Zaken Shauli Ravfogel and Yoav Goldberg. 2021. BitFit: Simple parameter-efficient fine-tuning for transformer-based masked language-models. arXiv:2106.10199. Retrieved from https:\/\/arxiv.org\/abs\/2106.10199","DOI":"10.18653\/v1\/2022.acl-short.1"},{"key":"e_1_3_3_99_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.acl-long.411"},{"key":"e_1_3_3_100_2","doi-asserted-by":"publisher","DOI":"10.1145\/3533767.3534390"},{"key":"e_1_3_3_101_2","doi-asserted-by":"crossref","unstructured":"Fengji Zhang Bei Chen Yue Zhang Jacky Keung Jin Liu Daoguang Zan Yi Mao Jian-Guang Lou and Weizhu Chen. 2023. RepoCoder: Repository-level code completion through iterative retrieval and generation. arXiv:2303.12570. Retrieved from https:\/\/arxiv.org\/abs\/2303.12570","DOI":"10.18653\/v1\/2023.emnlp-main.151"},{"key":"e_1_3_3_102_2","unstructured":"Qingru Zhang Minshuo Chen Alexander Bukharin Pengcheng He Yu Cheng Weizhu Chen and Tuo Zhao. 2023. Adaptive budget\u00a0allocation for parameter-efficient fine-tuning. arXiv:2303.10512. Retrieved from https:\/\/arxiv.org\/abs\/2303.10512"},{"key":"e_1_3_3_103_2","unstructured":"Qingyu Zhang Xiaoyu Shen Ernie Chang Jidong Ge and Pengke Chen. 2022. MDIA: A benchmark for multilingual dialogue generation in 46 languages. arXiv:2208.13078. Retrieved from https:\/\/arxiv.org\/abs\/2208.13078"},{"key":"e_1_3_3_104_2","doi-asserted-by":"publisher","DOI":"10.1093\/nsr\/nwx105"},{"key":"e_1_3_3_105_2","unstructured":"Zibin Zheng Kaiwen Ning Jiachi Chen Yanlin Wang Wenqing Chen Lianghong Guo and Weicheng Wang. 2023. Towards an understanding of large language models in software engineering tasks. arXiv:2308.11396. Retrieved from https:\/\/arxiv.org\/abs\/2308.11396"},{"key":"e_1_3_3_106_2","doi-asserted-by":"publisher","DOI":"10.1145\/3551349.3556943"},{"key":"e_1_3_3_107_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME52107.2021.00044"},{"key":"e_1_3_3_108_2","article-title":"Devign: Effective vulnerability identification by learning comprehensive program semantics via graph neural networks","volume":"32","author":"Zhou Yaqin","year":"2019","unstructured":"Yaqin Zhou, Shangqing Liu, Jingkai Siow, Xiaoning Du, and Yang Liu. 2019. Devign: Effective vulnerability identification by learning comprehensive program semantics via graph neural networks. Advances in Neural Information Processing Systems 32 (2019).","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_109_2","unstructured":"Terry Yue Zhuo Armel Zebaze Nitchakarn Suppattarachai Leandro von Werra Harm de Vries Qian Liu and Niklas Muennighoff. 2024. Astraios: Parameter-efficient instruction tuning code large language models. arXiv:2401.00788. Retrieved from https:\/\/arxiv.org\/abs\/2401.00788"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3722107","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,25]],"date-time":"2025-08-25T06:24:19Z","timestamp":1756103059000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3722107"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,14]]},"references-count":108,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2025,9,30]]}},"alternative-id":["10.1145\/3722107"],"URL":"https:\/\/doi.org\/10.1145\/3722107","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,14]]},"assertion":[{"value":"2024-04-26","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-03-01","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-08-14","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}