{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T08:56:52Z","timestamp":1790585812946,"version":"4.1.0"},"reference-count":91,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2026,9,25]],"date-time":"2026-09-25T00:00:00Z","timestamp":1790294400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,9,25]],"date-time":"2026-09-25T00:00:00Z","timestamp":1790294400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"publisher","award":["RGPIN-2019-05175"],"award-info":[{"award-number":["RGPIN-2019-05175"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2027,2]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    Large Language Models (LLMs) have shown high capabilities in several software development-related tasks such as program repair, documentation, code refactoring, debugging, and testing. However, training these models requires massive amount of data and significant computational resources. Adapters are specialized, small modules designed for parameter efficient fine-tuning of LLMs for specific tasks, domains, or applications without requiring extensive retraining of the entire model. These adapters offer a more efficient way to customize LLMs for particular needs, leveraging the pre-existing capabilities of the large model. Model (and adapter) merging have emerged as a technique to develop one model capable of multiple tasks, with minimal or no training required. Although model and adapter merging has shown promising performance in domains such as natural language processing and computer vision, its applicability to software engineering tasks remains underexplored. In this paper, we investigate the effectiveness of merged adapters within the context of software engineering, with a particular focus on the Automated Program Repair (APR) task, through our approach,\n                    <jats:bold>MergeRepair<\/jats:bold>\n                    . In particular, we merge multiple task-specific adapters using three different merging methods, including weight-averaging, TIES-Merging, and DARE-TIES, and evaluate the performance of the merged adapter on the APR task. We introduce a\n                    <jats:bold>continual merging<\/jats:bold>\n                    approach, a novel method in which we sequentially merge the task-specific adapters where the order and weight of the merged adapters play a significant role. We further compare the performance of our approach with a baseline method consisting of equal-weight merging applied on parameters of different adapters, where all adapters are of equal importance. Through continual merging, we explore the capability of merged adapters and the effect of task order, as it occurs in real-world software projects. To evaluate our approach, we consider two LLMs, StarCoder2 and Granite models on an APR benchmark, HumanEvalFix, with various combinations of the merged adapters trained on a subset of the CommitPackFT dataset that contains four tasks and APR in Python. The results show that merging task-specific adapters can enhance the performance of models on the APR task,\n                    <jats:italic>without additional training<\/jats:italic>\n                    . In particular, our approach achieves up to 2.38% performance improvement over APR task in terms of pass@1 score when merging APR with Improvement and Misc task-adapters on top of the StarCoder2 model. Performance rises to 4.01% for pass@10 score with the same model and merged adapters. This improvement could be significantly influenced by the base model and the specific task adapters merged, rather than only the number of adapters. The proposed continual merging approach can be effective if the order of merging is efficiently determined. Results show that performance improves when the most effective task-adapter is added last in a continual merging process. Moreover, we can achieve on-par or even better performance when APR adapter is not used in the merged model, emphasizing the generalizeability of the approach. These findings suggest that leveraging knowledge from other code-related tasks through adapter merging can improve performance on new tasks like APR, even with limited data.\n                  <\/jats:p>","DOI":"10.1007\/s10664-026-10943-y","type":"journal-article","created":{"date-parts":[[2026,9,25]],"date-time":"2026-09-25T12:15:25Z","timestamp":1790338525000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["MergeRepair: An Exploratory Study on Merging Task-Specific Adapters in Code LLMs for Automated Program Repair"],"prefix":"10.1007","volume":"32","author":[{"given":"Meghdad","family":"Dehghan","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7895-2023","authenticated-orcid":false,"given":"Jie JW","family":"Wu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fatemeh H.","family":"Fard","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ali","family":"Ouni","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,9,25]]},"reference":[{"key":"10943_CR1","unstructured":"Ainsworth S, Hayase J, Srinivasa S (2022) Git re-basin: merging models modulo permutation symmetries. The Eleventh international conference on learning representations"},{"key":"10943_CR2","doi-asserted-by":"crossref","unstructured":"Akiba T, Shing M, Tang Y, Sun Q, Ha D (2025) Evolutionary optimization of model merging recipes. Nature Machine Intelligence 1\u201310","DOI":"10.1038\/s42256-024-00975-8"},{"key":"10943_CR3","doi-asserted-by":"crossref","unstructured":"Bai J, Zhou K, Xue G, Zha H, Sun G, Tseng B, Zheng Z, Chang Y (2009) Multi-task learning for learning to rank in web search. In: Proceedings of the 18th ACM conference on Information and knowledge management, pp 1549\u20131552","DOI":"10.1145\/1645953.1646169"},{"key":"10943_CR4","unstructured":"Bavarian M, Jun H, Tezak N, Schulman J, McLeavey C, Tworek J, Chen M (2022) Efficient training of language models to fill in the middle. arXiv preprint arXiv:2207.14255"},{"key":"10943_CR5","unstructured":"Beeching E, Fourrier C, Habib N, Han S, Lambert N, Rajani N, Sanseviero O, Tunstall L, Wolf T (2023) Open LLM leaderboard. https:\/\/huggingface.co\/spaces\/HuggingFaceH4\/open_llm_leaderboard"},{"key":"10943_CR6","doi-asserted-by":"publisher","unstructured":"Ben\u00a0Zaken E, Goldberg Y, Ravfogel S (2022) BitFit: Simple parameter-efficient fine-tuning for transformer-based masked language-models. In: Muresan S, Nakov P, Villavicencio A (eds) Proceedings of the 60th annual meeting of the association for computational linguistics (Volume 2: Short Papers), Association for Computational Linguistics, Dublin, Ireland, pp 1\u20139. https:\/\/doi.org\/10.18653\/v1\/2022.acl-short.1","DOI":"10.18653\/v1\/2022.acl-short.1"},{"issue":"9","key":"10943_CR7","first-page":"1943","volume":"47","author":"Z Chen","year":"2019","unstructured":"Chen Z, Kommrusch S, Tufano M, Pouchet LN, Poshyvanyk D, Monperrus M (2019) Sequencer: sequence-to-sequence learning for end-to-end program repair. IEEE Trans Softw Eng 47(9):1943\u20131959","journal-title":"IEEE Trans Softw Eng"},{"key":"10943_CR8","unstructured":"Chen M, Tworek J, Jun H, Yuan Q, Pinto HPdO, Kaplan J, Edwards H, Burda Y, Joseph N, Brockman G, et\u00a0al. (2021) Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374"},{"key":"10943_CR9","doi-asserted-by":"crossref","unstructured":"Chen H, Tao R, Zhang H, Wang Y, Li X, Ye W, Wang J, Hu G, Savvides M (2024) Conv-adapter: exploring parameter efficient transfer learning for convnets. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp 1551\u20131561","DOI":"10.1109\/CVPRW63382.2024.00162"},{"key":"10943_CR10","doi-asserted-by":"publisher","unstructured":"Chronopoulou A, Peters M, Fraser A, Dodge J (2023) AdapterSoup: weight averaging to improve generalization of pretrained language models. In: Vlachos A, Augenstein I (eds) Findings of the association for computational linguistics: EACL 2023, Association for Computational Linguistics, Dubrovnik, Croatia, pp 2054\u2013206https:\/\/doi.org\/10.18653\/v1\/2023.findings-eacl.153","DOI":"10.18653\/v1\/2023.findings-eacl.153"},{"issue":"3","key":"10943_CR11","doi-asserted-by":"publisher","first-page":"494","DOI":"10.1037\/0033-2909.114.3.494","volume":"114","author":"N Cliff","year":"1993","unstructured":"Cliff N (1993) Dominance statistics: ordinal analyses to answer ordinal questions. Psychol Bull 114(3):494\u2013509","journal-title":"Psychol Bull"},{"issue":"3","key":"10943_CR12","doi-asserted-by":"publisher","first-page":"220","DOI":"10.1038\/s42256-023-00626-4","volume":"5","author":"N Ding","year":"2023","unstructured":"Ding N, Qin Y, Yang G, Wei F, Yang Z, Su Y, Hu S, Chen Y, Chan CM, Chen W et al (2023) Parameter-efficient fine-tuning of large-scale pre-trained language models. Nat Mach Intell 5(3):220\u2013235","journal-title":"Nat Mach Intell"},{"key":"10943_CR13","unstructured":"Li R et\u00a0al (2023) Starcoder: may the source be with you! Transactions on Machine Learning Research. https:\/\/openreview.net\/forum?id=KoFOg41haE, reproducibility Certification"},{"key":"10943_CR14","doi-asserted-by":"crossref","unstructured":"Fu M, Tantithamthavorn C, Le T, Nguyen V, Phung D (2022) Vulrepair: a t5-based automated software vulnerability repair. In: Proceedings of the 30th ACM joint European software engineering conference and symposium on the foundations of software engineering, pp 935\u2013947","DOI":"10.1145\/3540250.3549098"},{"key":"10943_CR15","doi-asserted-by":"crossref","unstructured":"Goddard C, Siriwardhana S, Ehghaghi M, Meyers L, Karpukhin V, Benedict B, McQuade M, Solawetz J (2024) Arcee\u2019s mergekit: a toolkit for merging large language models. arXiv preprint arXiv:2403.13257","DOI":"10.18653\/v1\/2024.emnlp-industry.36"},{"key":"10943_CR16","doi-asserted-by":"crossref","unstructured":"Goues L, Dewey-Vogt C, Forrest M, Weimer S, W (2012) A systematic study of automated program repair: Fixing 55 out of 105 bugs for \\$8 each. In: 34th International conference on software engineering (ICSE). IEEE, pp 3\u201313","DOI":"10.1109\/ICSE.2012.6227211"},{"key":"10943_CR17","doi-asserted-by":"crossref","unstructured":"Haque MZ, Afrin S, Mastropaolo A (2025) A systematic literature review of parameter-efficient fine-tuning for large code models. arXiv:2504.21569 arXiv preprint","DOI":"10.1145\/3796522"},{"key":"10943_CR18","unstructured":"Houlsby N, Giurgiu A, Jastrzebski S, Morrone B, De\u00a0Laroussilhe Q, Gesmundo A, Attariyan M, Gelly S (2019a) Parameter-efficient transfer learning for nlp. In: International conference on machine learning, PMLR, pp 2790\u20132799"},{"key":"10943_CR19","unstructured":"Houlsby N, Giurgiu A, Jastrzebski S, Morrone B, De\u00a0Laroussilhe Q, Gesmundo A, Attariyan M, Gelly S (2019b) Parameter-efficient transfer learning for nlp. In: International conference on machine learning, PMLR, pp 2790\u20132799"},{"key":"10943_CR20","doi-asserted-by":"publisher","unstructured":"Howard J, Ruder S (2018) Universal language model fine-tuning for text classification. In: Gurevych I, Miyao Y (eds) Proceedings of the 56th annual meeting of the association for computational linguistics (Volume 1: Long Papers), Association for Computational Linguistics, Melbourne, Australia, pp 328\u2013339. https:\/\/doi.org\/10.18653\/v1\/P18-1031","DOI":"10.18653\/v1\/P18-1031"},{"key":"10943_CR21","doi-asserted-by":"publisher","first-page":"2179","DOI":"10.1007\/s10664-019-09730-9","volume":"25","author":"X Hu","year":"2020","unstructured":"Hu X, Li G, Xia X, Lo D, Jin Z (2020) Deep code comment generation with hybrid lexical and syntactical information. Empir Softw Eng 25:2179\u20132217","journal-title":"Empir Softw Eng"},{"key":"10943_CR22","unstructured":"Hu EJ, Shen Y, Wallis P, Allen-Zhu Z, Li Y, Wang S, Wang L, Chen W (2022) LoRA: low-rank adaptation of large language models. In: International conference on learning representations. https:\/\/openreview.net\/forum?id=nZeVKeeFYf9"},{"key":"10943_CR23","unstructured":"Huang C, Liu Q, Lin BY, Pang T, Du C, Lin M (2023) Lorahub: efficient cross-task generalization via dynamic lora composition. arXiv preprint arXiv:2307.13269"},{"key":"10943_CR24","doi-asserted-by":"publisher","first-page":"122741","DOI":"10.52202\/079017-3900","volume":"37","author":"C Huang","year":"2024","unstructured":"Huang C, Ye P, Chen T, He T, Yue X, Ouyang W (2024) Emr-merging: tuning-free high-performance model merging. Adv Neural Inf Process Syst 37:122741\u2013122769","journal-title":"Adv Neural Inf Process Syst"},{"key":"10943_CR25","unstructured":"Ilharco G, Ribeiro MT, Wortsman M, Gururangan S, Schmidt L, Hajishirzi H, Farhadi A (2022) Editing models with task arithmetic. arXiv:2212.04089 arXiv preprint"},{"key":"10943_CR26","unstructured":"Izmailov P, Podoprikhin D, Garipov T, Vetrov D, Wilson AG (2018) Averaging weights leads to wider optima and better generalization. arXiv:1803.05407 arXiv preprint"},{"key":"10943_CR27","doi-asserted-by":"crossref","unstructured":"Jobstmann B, Griesmayer A, Bloem R (2005) Program repair as a game. In: Computer aided verification: 17th international conference, CAV 2005, Edinburgh, Scotland, UK, July 6-10, 2005. Proceedings 17, Springer, pp 226\u2013238","DOI":"10.1007\/11513988_23"},{"issue":"6","key":"10943_CR28","doi-asserted-by":"publisher","first-page":"540","DOI":"10.4097\/kjae.2015.68.6.540","volume":"68","author":"TK Kim","year":"2015","unstructured":"Kim TK (2015) T test as a parametric statistic. Korean J Anesthesiol 68(6):540\u2013546","journal-title":"Korean J Anesthesiol"},{"key":"10943_CR29","doi-asserted-by":"publisher","first-page":"1980","DOI":"10.1007\/s10664-019-09780-z","volume":"25","author":"A Koyuncu","year":"2020","unstructured":"Koyuncu A, Liu K, Bissyand\u00e9 TF, Kim D, Klein J, Monperrus M, Le Traon Y (2020) Fixminer: mining relevant fix patterns for automated program repair. Empir Softw Eng 25:1980\u20132024","journal-title":"Empir Softw Eng"},{"key":"10943_CR30","doi-asserted-by":"publisher","unstructured":"Li XL, Liang P (2021) Prefix-tuning: optimizing continuous prompts for generation. In: Zong C, Xia F, Li W, Navigli R (eds) Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing (Volume 1: Long Papers), Association for Computational Linguistics, Online, pp 4582\u20134597. https:\/\/doi.org\/10.18653\/v1\/2021.acl-long.353","DOI":"10.18653\/v1\/2021.acl-long.353"},{"key":"10943_CR31","doi-asserted-by":"crossref","unstructured":"Li Y, Mehrish A, Bhardwaj R, Majumder N, Cheng B, Zhao S, Zadeh A, Mihalcea R, Poria S (2023) Evaluating parameter-efficient transfer learning approaches on sure benchmark for speech understanding. In: ICASSP 2023\u20132023 IEEE international conference on acoustics. Speech and Signal Processing (ICASSP), IEEE, pp 1\u20135","DOI":"10.1109\/ICASSP49357.2023.10095656"},{"key":"10943_CR32","doi-asserted-by":"publisher","unstructured":"Liu X, Gao J, He X, Deng L, Duh K, Wang Yy (2015) Representation learning using multi-task deep neural networks for semantic classification and information retrieval. In: Mihalcea R, Chai J, Sarkar A (eds) Proceedings of the 2015 conference of the North American chapter of the association for computational linguistics: human language technologies, Association for Computational Linguistics, Denver, Colorado, pp 912\u2013921. https:\/\/doi.org\/10.3115\/v1\/N15-1092","DOI":"10.3115\/v1\/N15-1092"},{"issue":"5","key":"10943_CR33","doi-asserted-by":"publisher","first-page":"1800","DOI":"10.1109\/TSE.2020.3038681","volume":"48","author":"S Liu","year":"2020","unstructured":"Liu S, Gao C, Chen S, Nie LY, Liu Y (2020) Atom: commit message generation based on abstract syntax tree and hybrid ranking. IEEE Trans Softw Eng 48(5):1800\u20131817","journal-title":"IEEE Trans Softw Eng"},{"key":"10943_CR34","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2020.110817","volume":"171","author":"K Liu","year":"2021","unstructured":"Liu K, Li L, Koyuncu A, Kim D, Liu Z, Klein J, Bissyand\u00e9 TF (2021) A critical review on the evaluation of automated program repair systems. J Syst Softw 171:110817","journal-title":"J Syst Softw"},{"key":"10943_CR35","doi-asserted-by":"publisher","first-page":"1950","DOI":"10.52202\/068431-0142","volume":"35","author":"H Liu","year":"2022","unstructured":"Liu H, Tam D, Muqeeth M, Mohta J, Huang T, Bansal M, Raffel CA (2022) Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. Adv Neural Inf Process Syst 35:1950\u20131965","journal-title":"Adv Neural Inf Process Syst"},{"key":"10943_CR36","doi-asserted-by":"crossref","unstructured":"Liu S, Keung J, Yang Z, Liu F, Zhou Q, Liao Y (2024) Delving into parameter-efficient fine-tuning in code change learning: an empirical study. arXiv preprint arXiv:2402.06247","DOI":"10.1109\/SANER60148.2024.00055"},{"key":"10943_CR37","unstructured":"Lozhkov A, Li R, Allal LB, Cassano F, Lamy-Poirier J, Tazi N, Tang A, Pykhtar D, Liu J, Wei Y, et\u00a0al. (2024) Starcoder 2 and the stack v2: the next generation. arXiv preprint arXiv:2402.19173"},{"key":"10943_CR38","unstructured":"Luong MT, Le QV, Sutskever I, Vinyals O, Kaiser L (2015) Multi-task sequence to sequence learning. arXiv:1511.06114 arXiv preprint"},{"key":"10943_CR39","doi-asserted-by":"crossref","unstructured":"Lutellier T, Pham HV, Pang L, Li Y, Wei M, Tan L (2020) Coconut: combining context-aware neural translation models using ensemble for program repair. In: Proceedings of the 29th ACM SIGSOFT international symposium on software testing and analysis. pp 101\u2013114","DOI":"10.1145\/3395363.3397369"},{"key":"10943_CR40","unstructured":"M\u00e4nnist\u00f6 J, Attieh J, Tiedemann J (2025) A comparative study of PEFT methods for python code generation. In: Johansson R, Stymne S (eds) Proceedings of the joint 25th nordic conference on computational linguistics and 11th baltic conference on human language technologies (NoDaLiDa\/Baltic-HLT 2025), University of Tartu Library, Tallinn, Estonia, pp 390\u2013396. https:\/\/aclanthology.org\/2025.nodalida-1.42\/"},{"key":"10943_CR41","doi-asserted-by":"publisher","first-page":"17703","DOI":"10.52202\/068431-1287","volume":"35","author":"MS Matena","year":"2022","unstructured":"Matena MS, Raffel CA (2022) Merging models with fisher-weighted averaging. Adv Neural Inf Process Syst 35:17703\u201317716","journal-title":"Adv Neural Inf Process Syst"},{"key":"10943_CR42","doi-asserted-by":"crossref","unstructured":"Mesbah A, Rice A, Johnston E, Glorioso N, Aftandilian E (2019) Deepdelta: learning to repair compilation errors. In: Proceedings of the 2019 27th ACM joint meeting on european software engineering conference and symposium on the foundations of software engineering, pp 925\u2013936","DOI":"10.1145\/3338906.3340455"},{"key":"10943_CR43","unstructured":"Mishra M, Stallone M, Zhang G, Shen Y, Prasad A, Soria AM, Merler M, Selvam P, Surendran S, Singh S (2024) Granite code models: a family of open foundation models for code intelligence. arXiv:2405.04324 arXiv preprint"},{"key":"10943_CR44","unstructured":"Motwani M, Brun Y (2020) Automatically repairing programs using both tests and bug reports. arXiv:2011.08340 arXiv preprint"},{"key":"10943_CR45","unstructured":"Muennighoff N, Liu Q, Zebaze AR, Zheng Q, Hui B, Zhuo TY, Singh S, Tang X, Werra V, Longpre L, S (2023) Octopack: instruction tuning code large language models. In: The twelfth international conference on learning representations"},{"key":"10943_CR46","doi-asserted-by":"crossref","unstructured":"Nguyen T, Le T (2024) Generalizability of mixture of domain-specific adapters from the lens of signed weight directions and its application to effective model pruning. arXiv:2402.10639 arXiv preprint","DOI":"10.18653\/v1\/2024.acl-long.700"},{"key":"10943_CR47","doi-asserted-by":"crossref","unstructured":"Nguyen H, Qi D, Roychoudhury A, Chandra S (2013) Semfix: program repair via semantic analysis. In: 35th International conference on software engineering (ICSE). IEEE. pp 772\u2013781","DOI":"10.1109\/ICSE.2013.6606623"},{"key":"10943_CR48","unstructured":"Nijkamp E, Pang B, Hayashi H, Tu L, Wang H, Zhou Y, Savarese S, Xiong C (2023) Codegen: an open large language model for code with multi-turn program synthesis. In: The eleventh international conference on learning representations. https:\/\/openreview.net\/forum?id=iaYcJKpY2B_"},{"key":"10943_CR49","doi-asserted-by":"crossref","unstructured":"Otake S, Kawakami R, Inoue N (2023) Parameter efficient transfer learning for various speech processing tasks. In: ICASSP 2023\u20132023 IEEE international conference on acoustics. speech and signal processing (ICASSP), IEEE, pp 1\u20135","DOI":"10.1109\/ICASSP49357.2023.10096311"},{"key":"10943_CR50","doi-asserted-by":"crossref","unstructured":"Ouyang L, Wu J, Jiang X, Almeida D, Wainwright C, Mishkin P, Zhang C, Agarwal S, Slama K, Ray A et al (2022a) Training language models to follow instructions with human feedback. Adv Neural Inf Process Syst 35:27730\u201327744","DOI":"10.52202\/068431-2011"},{"key":"10943_CR51","doi-asserted-by":"crossref","unstructured":"Ouyang L, Wu J, Jiang X, Almeida D, Wainwright C, Mishkin P, Zhang C, Agarwal S, Slama K, Ray A et al (2022b) Training language models to follow instructions with human feedback. Adv Neural Inf Process Syst 35:27730\u201327744","DOI":"10.52202\/068431-2011"},{"key":"10943_CR52","doi-asserted-by":"publisher","first-page":"113085","DOI":"10.1016\/j.eswa.2019.113085","volume":"144","author":"SK Pandey","year":"2020","unstructured":"Pandey SK, Mishra RB, Tripathi AK (2020) Bpdet: an effective software bug prediction model using deep representation and ensemble learning techniques. Expert Syst Appl 144:113085","journal-title":"Expert Syst Appl"},{"key":"10943_CR53","doi-asserted-by":"crossref","unstructured":"Papineni K, Roukos S, Ward T, Zhu WJ (2002) Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th annual meeting of the association for computational linguistics, pp 311\u2013318","DOI":"10.3115\/1073083.1073135"},{"key":"10943_CR54","doi-asserted-by":"publisher","unstructured":"Pfeiffer J, Kamath A, R\u00fcckl\u00e9 A, Cho K, Gurevych I (2021) AdapterFusion: non-destructive task composition for transfer learning. In: Merlo P, Tiedemann J, Tsarfaty R (eds) Proceedings of the 16th conference of the European chapter of the association for computational linguistics: main volume, Association for Computational Linguistics, Online, pp 487\u2013503. https:\/\/doi.org\/10.18653\/v1\/2021.eacl-main.39","DOI":"10.18653\/v1\/2021.eacl-main.39"},{"key":"10943_CR55","doi-asserted-by":"crossref","unstructured":"Poth C, Sterz H, Paul I, Purkayastha S, Engl\u00e4nder L, Imhof T, Vuli\u0107 I, Ruder S, Gurevych I, Pfeiffer J (2023) Adapters: a unified library for parameter-efficient and modular transfer learning. arXiv:2311.11077 arXiv preprint","DOI":"10.18653\/v1\/2023.emnlp-demo.13"},{"key":"10943_CR56","unstructured":"Prabhakar A, Li Y, Narasimhan K, Kakade S, Malach E, Jelassi S (2024) Lora soups: merging loras for practical skill composition tasks. arXiv:2410.13025 arXiv preprint"},{"issue":"140","key":"10943_CR57","first-page":"1","volume":"21","author":"C Raffel","year":"2020","unstructured":"Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou Y, Li W, Liu PJ (2020) Exploring the limits of transfer learning with a unified text-to-text transformer. J Mach Learn Res 21(140):1\u201367","journal-title":"J Mach Learn Res"},{"key":"10943_CR58","unstructured":"Ren S, Guo D, Lu S, Zhou L, Liu S, Tang D, Sundaresan N, Zhou M, Blanco A, Ma S (2020) Codebleu: a method for automatic evaluation of code synthesis. arXiv preprint arXiv:2009.10297"},{"key":"10943_CR59","doi-asserted-by":"crossref","unstructured":"Shi E, Wang Y, Zhang H, Du L, Han S, Zhang D, Sun H (2023) Towards efficient fine-tuning of pre-trained code models: an experimental study and beyond. In: Proceedings of the 32nd ACM SIGSOFT international symposium on software testing and analysis, pp 39\u201351","DOI":"10.1145\/3597926.3598036"},{"key":"10943_CR60","doi-asserted-by":"crossref","unstructured":"Shi J, Yang Z, Lo D (2024) Efficient and green large language models for software engineering: vision and the road ahead. ACM Transactions on Software Engineering and Methodology","DOI":"10.1145\/3708525"},{"key":"10943_CR61","unstructured":"Silva A, Fang S, Monperrus M (2023) Repairllama: efficient representations and fine-tuned adapters for program repair. arXiv:2312.15698 arXiv preprint"},{"key":"10943_CR62","first-page":"22045","volume":"33","author":"SP Singh","year":"2020","unstructured":"Singh SP, Jaggi M (2020) Model fusion via optimal transport. Adv Neural Inf Process Syst 33:22045\u201322055","journal-title":"Adv Neural Inf Process Syst"},{"key":"10943_CR63","unstructured":"Stoica G, Bolya D, Bjorner JB, Ramesh P, Hearn T, Hoffman J (2024) Zipit! merging models from different tasks without training. In: The twelfth international conference on learning representations. https:\/\/openreview.net\/forum?id=LEYUkvdUhq"},{"key":"10943_CR64","doi-asserted-by":"crossref","unstructured":"Sung YL, Cho J, Bansal M (2022) Vl-adapter: parameter-efficient transfer learning for vision-and-language tasks. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp 5227\u20135237","DOI":"10.1109\/CVPR52688.2022.00516"},{"key":"10943_CR65","doi-asserted-by":"crossref","unstructured":"Torrey L, Shavlik J (2010) Transfer learning. Handbook of research on machine learning applications and trends: algorithms, methods, and techniques. IGI global, pp 242\u2013264","DOI":"10.4018\/978-1-60566-766-9.ch011"},{"key":"10943_CR66","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser \u0141, Polosukhin I (2017) Attention is all you need. Advances in Neural Information Processing Systems 30"},{"key":"10943_CR67","doi-asserted-by":"publisher","unstructured":"Wang Y, Agarwal S, Mukherjee S, Liu X, Gao J, Awadallah AH, Gao J (2022) AdaMix: Mixture-of-adaptations for parameter-efficient model tuning. In: Goldberg Y, Kozareva Z, Zhang Y (eds) Proceedings of the 2022 conference on empirical methods in natural language processing, Association for Computational Linguistics, Abu Dhabi, United Arab Emirates, pp 5744\u20135760. https:\/\/doi.org\/10.18653\/v1\/2022.emnlp-main.388","DOI":"10.18653\/v1\/2022.emnlp-main.388"},{"key":"10943_CR68","doi-asserted-by":"crossref","unstructured":"Wang D, Chen B, Li S, Luo W, Peng S, Dong W, Liao X (2023a) One adapter for all programming languages? Adapter tuning for code search and summarization. In: 2023 IEEE\/ACM 45th international conference on software engineering (ICSE), IEEE, pp 5\u201316","DOI":"10.1109\/ICSE48619.2023.00013"},{"key":"10943_CR69","doi-asserted-by":"publisher","unstructured":"Wang S, Li Z, Qian H, Yang C, Wang Z, Shang M, Kumar V, Tan S, Ray B, Bhatia P, Nallapati R, Ramanathan MK, Roth D, Xiang B (2023b) ReCode: robustness evaluation of code generation models. In: Rogers A, Boyd-Graber J, Okazaki N (eds) Proceedings of the 61st annual meeting of the association for computational linguistics (Volume 1: Long Papers), Association for Computational Linguistics, Toronto, Canada, pp 13818\u201313843.https:\/\/doi.org\/10.18653\/v1\/2023.acl-long.773","DOI":"10.18653\/v1\/2023.acl-long.773"},{"key":"10943_CR70","doi-asserted-by":"crossref","unstructured":"Wang Y, Kordi Y, Mishra S, Liu A, Smith NA, Khashabi D, Hajishirzi H (2023c) Self-instruct: Aligning language models with self-generated instructions. In: Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: long papers), pp 13484\u201313508","DOI":"10.18653\/v1\/2023.acl-long.754"},{"key":"10943_CR71","doi-asserted-by":"crossref","unstructured":"Wei Y, Pei Y, Furia CA, Silva LS, Buchholz S, Meyer B, Zeller A (2010) Automated fixing of programs with contracts. In: Proceedings of the 19th international symposium on Software testing and analysis, pp 61\u201372","DOI":"10.1145\/1831708.1831716"},{"key":"10943_CR72","doi-asserted-by":"crossref","unstructured":"Weimer W, Nguyen T, Goues L, Forrest C, S (2009) Automatically finding patches using genetic programming. In: IEEE 31st international conference on software engineering. IEEE, pp 364\u2013374","DOI":"10.1109\/ICSE.2009.5070536"},{"issue":"5","key":"10943_CR73","doi-asserted-by":"publisher","first-page":"109","DOI":"10.1145\/1735223.1735249","volume":"53","author":"W Weimer","year":"2010","unstructured":"Weimer W, Forrest S, Le Goues C, Nguyen T (2010) Automatic program repair with evolutionary computation. Commun ACM 53(5):109\u2013116","journal-title":"Commun ACM"},{"key":"10943_CR74","unstructured":"Weyssow M, Zhou X, Kim K, Lo D, Sahraoui H (2023a) Exploring parameter-efficient fine-tuning techniques for code generation with large language models. arXiv preprint arXiv:2308.10462"},{"key":"10943_CR75","doi-asserted-by":"crossref","unstructured":"Weyssow M, Zhou X, Kim K, Lo D, Sahraoui H (2023b) On the usage of continual learning for out-of-distribution generalization in pre-trained language models of code. In: Proceedings of the 31st ACM joint European software engineering conference and symposium on the foundations of software engineering, pp 1470\u20131482","DOI":"10.1145\/3611643.3616244"},{"key":"10943_CR76","doi-asserted-by":"crossref","unstructured":"Wong CP, Santiesteban P, K\u00e4stner C, Goues L, C (2021) Varfix: balancing edit expressiveness and search effectiveness in automated program repair. In; Proceedings of the 29th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering, pp 354\u2013366","DOI":"10.1145\/3468264.3468600"},{"key":"10943_CR77","unstructured":"Wortsman M, Ilharco G, Gadre SY, Roelofs R, Gontijo-Lopes R, Morcos AS, Namkoong H, Farhadi A, Carmon Y, Kornblith S (2022) Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In: International conference on machine learning. PMLR, pp 23965\u201323998"},{"key":"10943_CR78","unstructured":"Wu X, Huang S, Wei F (2024) Mixture of lora experts. arXiv:2404.13628 arXiv preprint"},{"key":"10943_CR79","doi-asserted-by":"publisher","unstructured":"Xia CS, Wei Y, Zhang L (2023) Automated program repair in the era of large pre-trained language models. In: Proceedings of the 45th international conference on software engineering, IEEE Press, ICSE \u201923, pp 1482\u20131494. https:\/\/doi.org\/10.1109\/ICSE48619.2023.00129","DOI":"10.1109\/ICSE48619.2023.00129"},{"key":"10943_CR80","doi-asserted-by":"publisher","first-page":"80522","DOI":"10.52202\/079017-2560","volume":"37","author":"Y Xin","year":"2024","unstructured":"Xin Y, Luo S, Liu X, Zhou H, Cheng X, Lee CE, Du J, Wang H, Chen M, Liu T et al (2024) V-petl bench: a unified visual parameter-efficient transfer learning benchmark. Adv Neural Inf Process Syst 37:80522\u201380535","journal-title":"Adv Neural Inf Process Syst"},{"issue":"1","key":"10943_CR81","doi-asserted-by":"publisher","first-page":"34","DOI":"10.1109\/TSE.2016.2560811","volume":"43","author":"J Xuan","year":"2016","unstructured":"Xuan J, Martinez M, Demarco F, Clement M, Marcote SL, Durieux T, Le Berre D, Monperrus M (2016) Nopol: automatic repair of conditional statement bugs in java programs. IEEE Trans Softw Eng 43(1):34\u201355","journal-title":"IEEE Trans Softw Eng"},{"key":"10943_CR82","doi-asserted-by":"crossref","unstructured":"Yadav P, Tam D, Choshen L, Raffel CA, Bansal M (2024) Ties-merging: resolving interference when merging models. Advances in Neural Information Processing Systems p 36","DOI":"10.52202\/075280-0310"},{"key":"10943_CR83","unstructured":"Yang E, Wang Z, Shen L, Liu S, Guo G, Wang X, Tao D (2023) Adamerging: adaptive model merging for multi-task learning. arXiv preprint arXiv:2310.02575"},{"key":"10943_CR84","unstructured":"Yang E, Shen L, Guo G, Wang X, Cao X, Zhang J, Tao D (2024) Model merging in llms, mllms, and beyond: methods, theories, applications and opportunities. arXiv:2408.07666 arXiv preprint"},{"key":"10943_CR85","unstructured":"Yu L, Yu B, Yu H, Huang F, Li Y (2023) Language models are super mario: absorbing abilities from homologous models as a free lunch. arXiv preprint arXiv:2311.03099"},{"key":"10943_CR86","doi-asserted-by":"crossref","unstructured":"Yuan C, Hu W, Tian G, Yang S, Wang H (2013) Multi-task sparse learning with beta process prior for action recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 423\u2013429","DOI":"10.1109\/CVPR.2013.61"},{"issue":"1","key":"10943_CR87","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1093\/nsr\/nwx105","volume":"5","author":"Y Zhang","year":"2018","unstructured":"Zhang Y, Yang Q (2018) An overview of multi-task learning. Natl Sci Rev 5(1):30\u201343","journal-title":"Natl Sci Rev"},{"key":"10943_CR88","doi-asserted-by":"publisher","first-page":"12589","DOI":"10.52202\/075280-0552","volume":"36","author":"J Zhang","year":"2023","unstructured":"Zhang J, Liu J, He J et al (2023) Composing parameter-efficient modules with arithmetic operation. Adv Neural Inf Process Syst 36:12589\u201312610","journal-title":"Adv Neural Inf Process Syst"},{"key":"10943_CR89","doi-asserted-by":"crossref","unstructured":"Zhao Y, Zhang W, Wang H, Kawaguchi K, Bing L (2024) Adamergex: cross-lingual transfer with large language models via adaptive adapter merging. arXiv:2402.18913 arXiv preprint","DOI":"10.18653\/v1\/2025.naacl-long.493"},{"key":"10943_CR90","unstructured":"Zhuo TY, Zebaze A, Suppattarachai N, von Werra L, de\u00a0Vries H, Liu Q, Muennighoff N (2024) Astraios: parameter-efficient instruction tuning code large language models. arXiv preprint arXiv:2401.00788"},{"key":"10943_CR91","doi-asserted-by":"crossref","unstructured":"Zou W, Shen Z, Li Q, Ge J, Li C, Chen X, Shen X, Huang L, Luo B (2025) Experimental evaluation of parameter-efficient fine-tuning for software engineering tasks. ACM Transactions on Software Engineering and Methodology","DOI":"10.1145\/3722107"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-026-10943-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-026-10943-y","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-026-10943-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T08:13:54Z","timestamp":1790583234000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-026-10943-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,25]]},"references-count":91,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2027,2]]}},"alternative-id":["10943"],"URL":"https:\/\/doi.org\/10.1007\/s10664-026-10943-y","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,25]]},"assertion":[{"value":"30 May 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 July 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 September 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Not applicable.","order":1,"name":"Ethics","label":"Ethical Approval","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","label":"Informed Consent","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":3,"name":"Ethics","label":"Conflicts of Interest","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":4,"name":"Ethics","label":"Clinical Trial Number","group":{"name":"EthicsHeading","label":"Declarations"}}],"article-number":"37"}}