{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T15:42:13Z","timestamp":1782834133188,"version":"3.54.5"},"reference-count":292,"publisher":"Association for Computing Machinery (ACM)","issue":"8","funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62576364, 62576083, 62432003, U25A20431, 62411540034"],"award-info":[{"award-number":["62576364, 62576083, 62432003, U25A20431, 62411540034"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Shenzhen Science and Technology Program","award":["CJGJZD20240729141505007"],"award-info":[{"award-number":["CJGJZD20240729141505007"]}]},{"name":"Shenzhen Basic Research Key Project","award":["JCYJ20241202124430041"],"award-info":[{"award-number":["JCYJ20241202124430041"]}]},{"DOI":"10.13039\/501100002858","name":"China Postdoctoral Science Foundation","doi-asserted-by":"crossref","award":["2025M781535"],"award-info":[{"award-number":["2025M781535"]}],"id":[{"id":"10.13039\/501100002858","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Manufacturing, Trade and Connectivity (MTC) Industry Alignment Fund-Pre-Positioning","award":["M23L4a0001"],"award-info":[{"award-number":["M23L4a0001"]}]},{"name":"MTC Programmatic","award":["M23L9b0052"],"award-info":[{"award-number":["M23L9b0052"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Comput. Surv."],"published-print":{"date-parts":[[2026,6,30]]},"abstract":"<jats:p>\n                    Model merging is an efficient empowerment technique in the machine learning community that does not require the collection of raw training data and does not require expensive computation. As model merging becomes increasingly prevalent across various fields, it is crucial to understand the available model merging techniques comprehensively. However, there is a significant gap in the literature regarding a systematic and thorough review of these techniques. This survey provides a comprehensive overview of model merging methods and theories, their applications in various domains and settings, and future research directions. Specifically, we first propose a new taxonomic approach that exhaustively discusses existing model merging methods. Secondly, we discuss the application of model merging techniques in large language models, multimodal large language models, and more than ten machine learning subfields, including continual learning, multi-task learning, few-shot learning, and so on. Finally, we highlight the remaining challenges of model merging and discuss future research directions. A comprehensive list of papers about model merging is available at\n                    <jats:italic toggle=\"yes\">\n                      <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" xlink:href=\"https:\/\/github.com\/EnnengYang\/Awesome-Model-Merging-Methods-Theories-Applications\">https:\/\/github.com\/EnnengYang\/Awesome-Model-Merging-Methods-Theories-Applications<\/jats:ext-link>\n                    <\/jats:italic>\n                    .\n                  <\/jats:p>","DOI":"10.1145\/3787849","type":"journal-article","created":{"date-parts":[[2026,1,10]],"date-time":"2026-01-10T20:37:43Z","timestamp":1768077463000},"page":"1-41","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":13,"title":["Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications, and Opportunities"],"prefix":"10.1145","volume":"58","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5419-5286","authenticated-orcid":false,"given":"Enneng","family":"Yang","sequence":"first","affiliation":[{"name":"Shenzhen Campus of Sun Yat-sen University","place":["China, China"]},{"name":"Northeastern University","place":["China, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5659-3464","authenticated-orcid":false,"given":"Li","family":"Shen","sequence":"additional","affiliation":[{"name":"Shenzhen Campus of Sun Yat-sen University","place":["Shenzhen, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1709-5056","authenticated-orcid":false,"given":"Guibing","family":"Guo","sequence":"additional","affiliation":[{"name":"Northeastern University","place":["Shenyang, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8668-3524","authenticated-orcid":false,"given":"Xingwei","family":"Wang","sequence":"additional","affiliation":[{"name":"Northeastern University","place":["Shenyang, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7141-708X","authenticated-orcid":false,"given":"Xiaochun","family":"Cao","sequence":"additional","affiliation":[{"name":"Shenzhen Campus of Sun Yat-sen University","place":["Shenzhen, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8996-7581","authenticated-orcid":false,"given":"Jie","family":"Zhang","sequence":"additional","affiliation":[{"name":"Nanyang Technological University","place":["Singapore, Singapore"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7225-5449","authenticated-orcid":false,"given":"Dacheng","family":"Tao","sequence":"additional","affiliation":[{"name":"Nanyang Technological University","place":["Singapore, Singapore"]},{"name":"The University of Sydney","place":["Singapore, Singapore"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,2,9]]},"reference":[{"key":"e_1_3_2_2_2","article-title":"Strong copyright protection for language models via adaptive model fusion","author":"Abad Javier","year":"2024","unstructured":"Javier Abad, Konstantin Donhauser, Francesco Pinto, and Fanny Yang. 2024. Strong copyright protection for language models via adaptive model fusion. ICML arXiv preprint arXiv:2407.20105 (2024).","journal-title":"ICML"},{"key":"e_1_3_2_3_2","unstructured":"Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad Ilge Akkaya Florencia Leoni Aleman Diogo Almeida Janko Altenschmidt Sam Altman Shyamal Anadkat et\u00a0al. 2023. Gpt-4 technical report. arXiv:2303.08774. Retrieved from https:\/\/arxiv.org\/abs\/2303.08774"},{"key":"e_1_3_2_4_2","unstructured":"Linara Adilova Maksym Andriushchenko Michael Kamp Asja Fischer and Martin Jaggi. 2023. Layer-wise linear mode connectivity. arXiv preprint arXiv:2307.06966 (2023)."},{"key":"e_1_3_2_5_2","article-title":"Jointly training large autoregressive multimodal models","author":"Aiello Emanuele","year":"2024","unstructured":"Emanuele Aiello, Lili Yu, Yixin Nie, Armen Aghajanyan, and Barlas Oguz. 2024. Jointly training large autoregressive multimodal models. ICLR. 1\u201322. https:\/\/openreview.net\/forum?id=5jcav5RcKw","journal-title":"ICLR"},{"key":"e_1_3_2_6_2","volume-title":"ICLR","author":"Ainsworth Samuel","year":"2023","unstructured":"Samuel Ainsworth, Jonathan Hayase, and Siddhartha Srinivasa. 2023. Git Re-basin: Merging models modulo permutation symmetries. In ICLR. 1\u201329. https:\/\/openreview.net\/forum?id=CQsmMYmlP5T"},{"issue":"2","key":"e_1_3_2_7_2","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1038\/s42256-024-00975-8","article-title":"Evolutionary optimization of model merging recipes","volume":"7","author":"Akiba Takuya","year":"2025","unstructured":"Takuya Akiba, Makoto Shing, Yujin Tang, Qi Sun, and David Ha. 2025. Evolutionary optimization of model merging recipes. Nature Machine Intelligence 7, 2 (2025), 195\u2013204.","journal-title":"Nature Machine Intelligence"},{"key":"e_1_3_2_8_2","doi-asserted-by":"crossref","unstructured":"Anton Alexandrov Veselin Raychev Mark Niklas M\u00fcller Ce Zhang Martin Vechev and Kristina Toutanova. 2024. Mitigating Catastrophic Forgetting in Language Transfer via Model Merging. arXiv:2407.08699. Retrieved from https:\/\/arxiv.org\/abs\/2407.08699","DOI":"10.18653\/v1\/2024.findings-emnlp.1000"},{"key":"e_1_3_2_9_2","first-page":"8265","article-title":"Ensemble of averages: Improving model selection and boosting performance in domain generalization","volume":"35","author":"Arpit Devansh","year":"2022","unstructured":"Devansh Arpit, Huan Wang, Yingbo Zhou, and Caiming Xiong. 2022. Ensemble of averages: Improving model selection and boosting performance in domain generalization. NeurIPS 35 (2022), 8265\u20138277.","journal-title":"NeurIPS"},{"key":"e_1_3_2_10_2","unstructured":"Nader Asadi Mahdi Beitollahi Yasser Khalil Yinchuan Li Guojun Zhang and Xi Chen. 2024. Does combining parameter-efficient modules improve few-shot transfer accuracy? arXiv:2402.15414. Retrieved from https:\/\/arxiv.org\/abs\/2402.15414"},{"key":"e_1_3_2_11_2","first-page":"205","volume-title":"ECCV","author":"Avrahami Omri","year":"2022","unstructured":"Omri Avrahami, Dani Lischinski, and Ohad Fried. 2022. GAN cocktail: Mixing GANs without dataset access. In ECCV. Springer, 205\u2013221."},{"key":"e_1_3_2_12_2","doi-asserted-by":"crossref","unstructured":"Robert Belanec Simon Ostermann Ivan Srba and Maria Bielikova. 2024. Task prompt vectors: Effective initialization through multi-task soft-prompt transfer. arXiv:2408.01119. Retrieved from https:\/\/arxiv.org\/abs\/2408.01119","DOI":"10.1007\/978-3-662-72243-5_5"},{"key":"e_1_3_2_13_2","first-page":"769","volume-title":"ICML","author":"Benton Gregory","year":"2021","unstructured":"Gregory Benton, Wesley Maddox, Sanae Lotfi, and Andrew Gordon Gordon Wilson. 2021. Loss surface simplexes for mode connecting volumes and fast ensembling. In ICML. PMLR, 769\u2013779."},{"key":"e_1_3_2_14_2","article-title":"Language models are homer simpson! safety re-alignment of fine-tuned language models through task arithmetic","author":"Bhardwaj Rishabh","year":"2024","unstructured":"Rishabh Bhardwaj, Do Duc Anh, and Soujanya Poria. 2024. Language models are homer simpson! safety re-alignment of fine-tuned language models through task arithmetic. arXiv preprint arXiv:2402.11746 (2024).","journal-title":"arXiv preprint"},{"key":"e_1_3_2_15_2","article-title":"Diffusion soup: Model merging for text-to-image diffusion models","author":"Biggs Benjamin","year":"2024","unstructured":"Benjamin Biggs, Arjun Seshadri, Yang Zou, Achin Jain, Aditya Golatkar, Yusheng Xie, Alessandro Achille, Ashwin Swaminathan, and Stefano Soatto. 2024. Diffusion soup: Model merging for text-to-image diffusion models. arXiv preprint arXiv:2406.08431 (2024).","journal-title":"arXiv preprint"},{"key":"e_1_3_2_16_2","first-page":"491","volume-title":"WWW","author":"Borkan Daniel","year":"2019","unstructured":"Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2019. Nuanced metrics for measuring unintended bias with real data for text classification. In WWW. 491\u2013500."},{"key":"e_1_3_2_17_2","first-page":"3495","volume-title":"ICML","author":"Cai Ruisi","year":"2023","unstructured":"Ruisi Cai, Zhenyu Zhang, and Zhangyang Wang. 2023. Robust weight signatures: Gaining robustness as easy as patching weights?. In ICML. PMLR, 3495\u20133506."},{"key":"e_1_3_2_18_2","article-title":"Param \\(\\Delta\\)  for Direct weight mixing: Post-train large language model at zero cost","volume":"2504","author":"Cao Sheng","year":"2025","unstructured":"Sheng Cao, Mingrui Wu, Karthik Prasad, Yuandong Tian, and Zechun Liu. 2025. Param \\(\\Delta\\) for Direct weight mixing: Post-train large language model at zero cost. CoRR abs\/2504.21023 (2025).","journal-title":"CoRR"},{"key":"e_1_3_2_19_2","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1023\/A:1007379606734","article-title":"Multitask learning","volume":"28","author":"Caruana Rich","year":"1997","unstructured":"Rich Caruana. 1997. Multitask learning. Machine Learning 28, 1 (1997), 41\u201375.","journal-title":"Machine Learning"},{"key":"e_1_3_2_20_2","first-page":"22405","article-title":"Swad: Domain generalization by seeking flat minima","volume":"34","author":"Cha Junbum","year":"2021","unstructured":"Junbum Cha, Sanghyuk Chun, Kyungjae Lee, Han-Cheol Cho, Seunghyun Park, Yunsung Lee, and Sungrae Park. 2021. Swad: Domain generalization by seeking flat minima. NeurIPS 34 (2021), 22405\u201322418.","journal-title":"NeurIPS"},{"key":"e_1_3_2_21_2","doi-asserted-by":"crossref","unstructured":"Chi Chen Yiyang Du Zheng Fang Ziyue Wang Fuwen Luo Peng Li Ming Yan Ji Zhang Fei Huang Maosong Sun et\u00a0al. 2024. Model composition for multimodal large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) ACL 2024. Association for Computational Linguistics 11246\u201311262.","DOI":"10.18653\/v1\/2024.acl-long.606"},{"key":"e_1_3_2_22_2","first-page":"11276","volume-title":"AAAI","author":"Chen Guangyao","year":"2024","unstructured":"Guangyao Chen, Peixi Peng, Yangru Huang, Mengyue Geng, and Yonghong Tian. 2024. Adaptive discovering and merging for incremental novel class discovery. In AAAI, Vol. 38. 11276\u201311284."},{"key":"e_1_3_2_23_2","volume-title":"ICML","author":"Chen Weiyu","year":"2025","unstructured":"Weiyu Chen and James Kwok. 2025. Pareto merging: Multi-objective optimization for preference-aware model merging. In ICML."},{"key":"e_1_3_2_24_2","article-title":"Do not think that much for 2+ 3=? on the overthinking of o1-like llms","author":"Chen Xingyu","year":"2024","unstructured":"Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, et\u00a0al. 2024. Do not think that much for 2+ 3=? on the overthinking of o1-like llms. arXiv preprint arXiv:2412.21187 (2024).","journal-title":"arXiv preprint"},{"key":"e_1_3_2_25_2","first-page":"794","volume-title":"ICML","author":"Chen Zhao","year":"2018","unstructured":"Zhao Chen, Vijay Badrinarayanan, Chen-Yu Lee, and Andrew Rabinovich. 2018. Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In ICML. PMLR, 794\u2013803."},{"key":"e_1_3_2_26_2","article-title":"DAM: Dynamic adapter merging for continual video QA learning","author":"Cheng Feng","year":"2024","unstructured":"Feng Cheng, Ziyang Wang, Yi-Lin Sung, Yan-Bo Lin, Mohit Bansal, and Gedas Bertasius. 2024. DAM: Dynamic adapter merging for continual video QA learning. arXiv preprint arXiv:2403.08755 (2024).","journal-title":"arXiv preprint"},{"key":"e_1_3_2_27_2","volume-title":"ICML","author":"Cheng Runxi","year":"2025","unstructured":"Runxi Cheng, Feng Xiong, Yongxian Wei, Wanyun Zhu, and Chun Yuan. 2025. Whoever started the interference should end it: Guiding data-free model merging via task vectors. In ICML."},{"key":"e_1_3_2_28_2","article-title":"Task arithmetic with LoRA for continual learning","author":"Chitale Rajas","year":"2023","unstructured":"Rajas Chitale, Ankit Vaidya, Aditya Kane, and Archana Ghotkar. 2023. Task arithmetic with LoRA for continual learning. arXiv preprint arXiv:2311.02428 (2023).","journal-title":"arXiv preprint"},{"issue":"240","key":"e_1_3_2_29_2","first-page":"1","article-title":"Palm: Scaling language modeling with pathways","volume":"24","author":"Chowdhery Aakanksha","year":"2023","unstructured":"Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et\u00a0al. 2023. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research 24, 240 (2023), 1\u2013113.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_30_2","first-page":"2009","volume-title":"Findings of the Association for Computational Linguistics: EACL 2023","author":"Chronopoulou Alexandra","year":"2023","unstructured":"Alexandra Chronopoulou, Matthew E. Peters, Alexander Fraser, and Jesse Dodge. 2023. AdapterSoup: Weight averaging to improve generalization of pretrained language models. In Findings of the Association for Computational Linguistics: EACL 2023. 2009\u20132018."},{"key":"e_1_3_2_31_2","article-title":"Language and task arithmetic with parameter-efficient layers for zero-shot summarization","author":"Chronopoulou Alexandra","year":"2023","unstructured":"Alexandra Chronopoulou, Jonas Pfeiffer, Joshua Maynez, Xinyi Wang, Sebastian Ruder, and Priyanka Agrawal. 2023. Language and task arithmetic with parameter-efficient layers for zero-shot summarization. arXiv preprint arXiv:2311.09344 (2023).","journal-title":"arXiv preprint"},{"key":"e_1_3_2_32_2","article-title":"Have you merged my model? on the robustness of large language model IP protection methods against model merging","author":"Cong Tianshuo","year":"2024","unstructured":"Tianshuo Cong, Delong Ran, Zesen Liu, Xinlei He, Jinyuan Liu, Yichen Gong, Qi Li, Anyu Wang, and Xiaoyun Wang. 2024. Have you merged my model? on the robustness of large language model IP protection methods against model merging. arXiv preprint arXiv:2404.05188 (2024).","journal-title":"arXiv preprint"},{"key":"e_1_3_2_33_2","article-title":"\\(C^{2}M^{3}\\) : Cycle-consistent multi-model merging","author":"Crisostomi Donato","year":"2024","unstructured":"Donato Crisostomi, Marco Fumero, Daniele Baieri, Florian Bernard, and Emanuele Rodol\u00e0. 2024. \\(C^{2}M^{3}\\) : Cycle-consistent multi-model merging. arXiv preprint arXiv:2405.17897 (2024).","journal-title":"arXiv preprint"},{"key":"e_1_3_2_34_2","first-page":"12313","volume-title":"CVPR","author":"Croce Francesco","year":"2023","unstructured":"Francesco Croce, Sylvestre-Alvise Rebuffi, Evan Shelhamer, and Sven Gowal. 2023. Seasoning model soups for robustness to adversarial and natural distribution shifts. In CVPR. 12313\u201312323."},{"key":"e_1_3_2_35_2","volume-title":"ICLR","author":"Daheim Nico","year":"2024","unstructured":"Nico Daheim, Thomas M\u00f6llenhoff, Edoardo Ponti, Iryna Gurevych, and Mohammad Emtiyaz Khan. 2024. Model merging by uncertainty-based gradient matching. In ICLR. 1\u201319."},{"key":"e_1_3_2_36_2","article-title":"Deepseekmoe: Towards ultimate expert specialization in mixture-of-experts language models","author":"Dai Damai","year":"2024","unstructured":"Damai Dai, Chengqi Deng, Chenggang Zhao, RX Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, Yu Wu, et\u00a0al. 2024. Deepseekmoe: Towards ultimate expert specialization in mixture-of-experts language models. arXiv preprint arXiv:2401.06066 (2024).","journal-title":"arXiv preprint"},{"key":"e_1_3_2_37_2","volume-title":"ICLR","author":"Dai Rui","year":"2025","unstructured":"Rui Dai, Sile Hu, Xu Shen, Yonggang Zhang, Xinmei Tian, and Jieping Ye. 2025. Leveraging submodule linearity enhances task arithmetic performance in LLMs. In ICLR. 1\u201328."},{"key":"e_1_3_2_38_2","article-title":"Model breadcrumbs: Scaling multi-task model merging with sparse masks","author":"Davari MohammadReza","year":"2023","unstructured":"MohammadReza Davari and Eugene Belilovsky. 2023. Model breadcrumbs: Scaling multi-task model merging with sparse masks. arXiv preprint arXiv:2312.06795 (2023).","journal-title":"arXiv preprint"},{"key":"e_1_3_2_39_2","article-title":"DELLA-merging: Reducing interference in model merging through magnitude-based sampling","author":"Deep Pala Tej","year":"2024","unstructured":"Pala Tej Deep, Rishabh Bhardwaj, and Soujanya Poria. 2024. DELLA-merging: Reducing interference in model merging through magnitude-based sampling. arXiv preprint arXiv:2406.11617 (2024).","journal-title":"arXiv preprint"},{"key":"e_1_3_2_40_2","article-title":"Controlled text generation via language model arithmetic","author":"Dekoninck Jasper","year":"2024","unstructured":"Jasper Dekoninck, Marc Fischer, Luca Beurer-Kellner, and Martin Vechev. 2024. Controlled text generation via language model arithmetic. ICLR (2024), 1\u201328.","journal-title":"ICLR"},{"key":"e_1_3_2_41_2","article-title":"Adaptive stochastic weight averaging","author":"Demir Caglar","year":"2024","unstructured":"Caglar Demir, Arnab Sharma, and Axel-Cyrille Ngonga Ngomo. 2024. Adaptive stochastic weight averaging. JMLR (2024).","journal-title":"JMLR"},{"issue":"1","key":"e_1_3_2_42_2","first-page":"110","article-title":"Ensemble learning","volume":"2","author":"Dietterich Thomas G.","year":"2002","unstructured":"Thomas G. Dietterich et\u00a0al. 2002. Ensemble learning. The Handbook of Brain Theory and Neural Networks 2, 1 (2002), 110\u2013125.","journal-title":"The Handbook of Brain Theory and Neural Networks"},{"key":"e_1_3_2_43_2","article-title":"Mitigating social biases in language models through unlearning","author":"Dige Omkar","year":"2024","unstructured":"Omkar Dige, Diljot Singh, Tsz Fung Yau, Qixuan Zhang, Borna Bolandraftar, Xiaodan Zhu, and Faiza Khan Khattak. 2024. Mitigating social biases in language models through unlearning. arXiv preprint arXiv:2406.13551 (2024).","journal-title":"arXiv preprint"},{"issue":"3","key":"e_1_3_2_44_2","doi-asserted-by":"crossref","first-page":"220","DOI":"10.1038\/s42256-023-00626-4","article-title":"Parameter-efficient fine-tuning of large-scale pre-trained language models","volume":"5","author":"Ding Ning","year":"2023","unstructured":"Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zonghan Yang, Yusheng Su, Shengding Hu, Yulin Chen, Chi-Min Chan, Weize Chen, et\u00a0al. 2023. Parameter-efficient fine-tuning of large-scale pre-trained language models. Nature Machine Intelligence 5, 3 (2023), 220\u2013235.","journal-title":"Nature Machine Intelligence"},{"key":"e_1_3_2_45_2","article-title":"Avoiding copyright infringement via machine unlearning","author":"Dou Guangyao","year":"2024","unstructured":"Guangyao Dou, Zheyuan Liu, Qing Lyu, Kaize Ding, and Eric Wong. 2024. Avoiding copyright infringement via machine unlearning. arXiv preprint arXiv:2406.10952 (2024).","journal-title":"arXiv preprint"},{"key":"e_1_3_2_46_2","first-page":"1309","volume-title":"ICML","author":"Draxler Felix","year":"2018","unstructured":"Felix Draxler, Kambis Veschgini, Manfred Salmhofer, and Fred Hamprecht. 2018. Essentially no barriers in neural network energy landscape. In ICML. PMLR, 1309\u20131318."},{"key":"e_1_3_2_47_2","first-page":"84746","article-title":"Parameter competition balancing for model merging","volume":"37","author":"Du Guodong","year":"2024","unstructured":"Guodong Du, Junlin Lee, Jing Li, Runhua Jiang, Yifei Guo, Shuyang Yu, Hanting Liu, Sim K Goh, Ho-Kin Tang, Daojing He, et\u00a0al. 2024. Parameter competition balancing for model merging. NeurIPS 37 (2024), 84746\u201384776.","journal-title":"NeurIPS"},{"key":"e_1_3_2_48_2","article-title":"The role of permutation invariance in linear mode connectivity of neural networks","author":"Entezari Rahim","year":"2022","unstructured":"Rahim Entezari, Hanie Sedghi, Olga Saukh, and Behnam Neyshabur. 2022. The role of permutation invariance in linear mode connectivity of neural networks. ICLR (2022), 1\u201324.","journal-title":"ICLR"},{"key":"e_1_3_2_49_2","first-page":"3853","volume-title":"AISTATS (Proceedings of Machine Learning Research, Vol. 238)","author":"Ferbach Damien","year":"2024","unstructured":"Damien Ferbach, Baptiste Goujaud, Gauthier Gidel, and Aymeric Dieuleveut. 2024. Proving linear mode connectivity of neural networks via optimal transport. In AISTATS (Proceedings of Machine Learning Research, Vol. 238). PMLR, 3853\u20133861."},{"issue":"594","key":"e_1_3_2_50_2","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1098\/rsta.1922.0009","article-title":"On the mathematical foundations of theoretical statistics","volume":"222","author":"Fisher Ronald A.","year":"1922","unstructured":"Ronald A. Fisher. 1922. On the mathematical foundations of theoretical statistics. Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character 222, 594-604 (1922), 309\u2013368.","journal-title":"Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character"},{"key":"e_1_3_2_51_2","article-title":"Sharpness-aware minimization for efficiently improving generalization","author":"Foret Pierre","year":"2021","unstructured":"Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. 2021. Sharpness-aware minimization for efficiently improving generalization. ICLR (2021), 1\u201319.","journal-title":"ICLR"},{"key":"e_1_3_2_52_2","article-title":"WASH: Train your ensemble with communication-efficient weight shuffling, then average","author":"Fournier Louis","year":"2024","unstructured":"Louis Fournier, Adel Nabli, Masih Aminbeidokhti, Marco Pedersoli, Eugene Belilovsky, and Edouard Oyallon. 2024. WASH: Train your ensemble with communication-efficient weight shuffling, then average. arXiv preprint arXiv:2405.17517 (2024).","journal-title":"arXiv preprint"},{"key":"e_1_3_2_53_2","first-page":"3259","volume-title":"ICML","author":"Frankle Jonathan","year":"2020","unstructured":"Jonathan Frankle, Gintare Karolina Dziugaite, Daniel Roy, and Michael Carbin. 2020. Linear mode connectivity and the lottery ticket hypothesis. In ICML. PMLR, 3259\u20133269."},{"key":"e_1_3_2_54_2","doi-asserted-by":"crossref","unstructured":"Tingchen Fu Deng Cai Lemao Liu Shuming Shi and Rui Yan. 2024. Disperse-then-merge: Pushing the limits of instruction tuning via alignment tax reduction. In Findings of the Association for Computational Linguistics ACL 2024. Association for Computational Linguistics 2967\u20132985.","DOI":"10.18653\/v1\/2024.findings-acl.175"},{"key":"e_1_3_2_55_2","article-title":"Merging improves self-critique against jailbreak attacks","author":"Gallego Victor","year":"2024","unstructured":"Victor Gallego. 2024. Merging improves self-critique against jailbreak attacks. arXiv preprint arXiv:2406.07188 (2024).","journal-title":"arXiv preprint arXiv:2406.07188"},{"key":"e_1_3_2_56_2","article-title":"Ethos: Rectifying language models in orthogonal parameter space","author":"Gao Lei","year":"2024","unstructured":"Lei Gao, Yue Niu, Tingting Tang, Salman Avestimehr, and Murali Annavaram. 2024. Ethos: Rectifying language models in orthogonal parameter space. arXiv preprint arXiv:2403.08994 (2024).","journal-title":"arXiv preprint"},{"key":"e_1_3_2_57_2","first-page":"18695","volume-title":"CVPR","author":"Gargiulo Antonio Andrea","year":"2025","unstructured":"Antonio Andrea Gargiulo, Donato Crisostomi, Maria Sofia Bucarelli, Simone Scardapane, Fabrizio Silvestri, and Emanuele Rodola. 2025. Task singular vectors: Reducing task interference in model merging. In CVPR. 18695\u201318705."},{"key":"e_1_3_2_58_2","article-title":"Loss surfaces, mode connectivity, and fast ensembling of dnns","volume":"31","author":"Garipov Timur","year":"2018","unstructured":"Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry P. Vetrov, and Andrew G. Wilson. 2018. Loss surfaces, mode connectivity, and fast ensembling of dnns. NeurIPS 31 (2018), 8803\u20138812.","journal-title":"NeurIPS"},{"key":"e_1_3_2_59_2","article-title":"Arcee\u2019s mergekit: A toolkit for merging large language models","author":"Goddard Charles","year":"2024","unstructured":"Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vlad Karpukhin, Brian Benedict, Mark McQuade, and Jacob Solawetz. 2024. Arcee\u2019s mergekit: A toolkit for merging large language models. arXiv preprint arXiv:2403.13257 (2024).","journal-title":"arXiv preprint"},{"key":"e_1_3_2_60_2","article-title":"Arcee\u2019s mergekit: A toolkit for merging large language models","author":"Goddard Charles","year":"2024","unstructured":"Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vlad Karpukhin, Brian Benedict, Mark McQuade, and Jacob Solawetz. 2024. Arcee\u2019s mergekit: A toolkit for merging large language models. arXiv preprint arXiv:2403.13257 (2024).","journal-title":"arXiv preprint"},{"key":"e_1_3_2_61_2","article-title":"Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning","author":"Guo Daya","year":"2025","unstructured":"Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et\u00a0al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948 (2025).","journal-title":"arXiv preprint arXiv:2501.12948"},{"key":"e_1_3_2_62_2","volume-title":"ICLR","author":"Gupta Vipul","year":"2020","unstructured":"Vipul Gupta, Santiago Akle Serrano, and Dennis DeCoste. 2020. Stochastic weight averaging in parallel: Large-batch training that generalizes well. In ICLR. OpenReview.net."},{"key":"e_1_3_2_63_2","article-title":"Model merging and safety alignment: One bad model spoils the bunch","author":"Hammoud Hasan Abed Al Kader","year":"2024","unstructured":"Hasan Abed Al Kader Hammoud, Umberto Michieli, Fabio Pizzati, Philip Torr, Adel Bibi, Bernard Ghanem, and Mete Ozay. 2024. Model merging and safety alignment: One bad model spoils the bunch. arXiv preprint arXiv:2406.14563 (2024).","journal-title":"arXiv preprint arXiv:2406.14563"},{"key":"e_1_3_2_64_2","first-page":"1225","volume-title":"ICML","author":"Hardt Moritz","year":"2016","unstructured":"Moritz Hardt, Ben Recht, and Yoram Singer. 2016. Train faster, generalize better: Stability of stochastic gradient descent. In ICML. PMLR, 1225\u20131234."},{"key":"e_1_3_2_65_2","article-title":"Safety arithmetic: A framework for test-time safety alignment of language models by steering parameters and activations","author":"Hazra Rima","year":"2024","unstructured":"Rima Hazra, Sayan Layek, Somnath Banerjee, and Soujanya Poria. 2024. Safety arithmetic: A framework for test-time safety alignment of language models by steering parameters and activations. arXiv preprint arXiv:2406.11801 (2024).","journal-title":"arXiv preprint arXiv:2406.11801"},{"key":"e_1_3_2_66_2","article-title":"MerA: Merging pretrained adapters for few-shot learning","author":"He Shwai","year":"2023","unstructured":"Shwai He, Run-Ze Fan, Liang Ding, Li Shen, Tianyi Zhou, and Dacheng Tao. 2023. MerA: Merging pretrained adapters for few-shot learning. arXiv preprint arXiv:2308.15982 (2023).","journal-title":"arXiv preprint arXiv:2308.15982"},{"key":"e_1_3_2_67_2","volume-title":"EMNLP","author":"He Shwai","year":"2023","unstructured":"Shwai He, Run-Ze Fan, Liang Ding, Li Shen, Tianyi Zhou, and Dacheng Tao. 2023. Merging experts into one: Improving computational efficiency of mixture of experts. In EMNLP. Association for Computational Linguistics, 14685\u201314691."},{"key":"e_1_3_2_68_2","article-title":"Localize-and-stitch: Efficient model merging via sparse task arithmetic","author":"He Yifei","year":"2025","unstructured":"Yifei He, Yuzheng Hu, Yong Lin, Tong Zhang, and Han Zhao. 2025. Localize-and-stitch: Efficient model merging via sparse task arithmetic. Transactions on Machine Learning Research 2025 (2025).","journal-title":"Transactions on Machine Learning Research"},{"key":"e_1_3_2_69_2","article-title":"MergeBench: A benchmark for merging domain-specialized LLMs","author":"He Yifei","year":"2025","unstructured":"Yifei He, Siqi Zeng, Yuzheng Hu, Rui Yang, Tong Zhang, and Han Zhao. 2025. MergeBench: A benchmark for merging domain-specialized LLMs. NeurIPS 2025 Datasets and Benchmarks Track (2025). arXiv preprint arXiv:2505.10833.","journal-title":"NeurIPS 2025 Datasets and Benchmarks Track"},{"key":"e_1_3_2_70_2","article-title":"Benchmarking neural network robustness to common corruptions and perturbations","author":"Hendrycks Dan","year":"2019","unstructured":"Dan Hendrycks and Thomas Dietterich. 2019. Benchmarking neural network robustness to common corruptions and perturbations. ICLR (2019), 1\u201316.","journal-title":"ICLR"},{"key":"e_1_3_2_71_2","article-title":"A systematic study of model merging techniques in large language models","author":"Hitit O\u011fuz Ka\u011fan","year":"2025","unstructured":"O\u011fuz Ka\u011fan Hitit, Leander Girrbach, and Zeynep Akata. 2025. A systematic study of model merging techniques in large language models. arXiv preprint arXiv:2511.21437 (2025).","journal-title":"arXiv preprint arXiv:2511.21437"},{"key":"e_1_3_2_72_2","first-page":"6840","article-title":"Denoising diffusion probabilistic models","volume":"33","author":"Ho Jonathan","year":"2020","unstructured":"Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. NeurIPS 33 (2020), 6840\u20136851.","journal-title":"NeurIPS"},{"issue":"1","key":"e_1_3_2_73_2","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1080\/13600834.2019.1573501","article-title":"The European union general data protection regulation: What it is and what it means","volume":"28","author":"Hoofnagle Chris Jay","year":"2019","unstructured":"Chris Jay Hoofnagle, Bart Van Der Sloot, and Frederik Zuiderveen Borgesius. 2019. The European union general data protection regulation: What it is and what it means. Information & Communications Technology Law 28, 1 (2019), 65\u201398.","journal-title":"Information & Communications Technology Law"},{"key":"e_1_3_2_74_2","volume-title":"ICML","author":"Horoi Stefan","year":"2024","unstructured":"Stefan Horoi, Albert Manuel Orozco Camacho, Eugene Belilovsky, and Guy Wolf. 2024. Harmony in diversity: Merging neural networks with canonical correlation analysis. In ICML."},{"key":"e_1_3_2_75_2","volume-title":"ICLR","author":"Hu Edward J.","year":"2022","unstructured":"Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In ICLR. 1\u201313. https:\/\/openreview.net\/forum?id=nZeVKeeFYf9"},{"key":"e_1_3_2_76_2","first-page":"18252","volume-title":"AAAI","author":"Hu Xinshuo","year":"2024","unstructured":"Xinshuo Hu, Dongfang Li, Baotian Hu, Zihao Zheng, Zhenyu Liu, and Min Zhang. 2024. Separate the wheat from the chaff: Model deficiency unlearning via parameter-efficient module operation. In AAAI, Vol. 38. 18252\u201318260."},{"key":"e_1_3_2_77_2","unstructured":"Chengsong Huang Qian Liu Bill Yuchen Lin Tianyu Pang Chao Du and Min Lin. 2023. LoraHub: Efficient cross-task generalization via dynamic LoRA composition. arXiv preprint arXiv:2307.13269 (2023)."},{"key":"e_1_3_2_78_2","doi-asserted-by":"crossref","unstructured":"Chenyu Huang Peng Ye Tao Chen Tong He Xiangyu Yue and Wanli Ouyang. 2024. EMR-merging: Tuning-free high-performance model merging. NeurIPS 37 (2024) 122741\u2013122769.","DOI":"10.52202\/079017-3900"},{"key":"e_1_3_2_79_2","doi-asserted-by":"crossref","unstructured":"Shih-Cheng Huang Pin-Zu Li Yu-Chi Hsu Kuang-Ming Chen Yu Tung Lin Shih-Kai Hsiao Richard Tzong-Han Tsai and Hung-yi Lee. 2024. Chat vector: A Simple Approach to Equip LLMs with Instruction Following and Model Alignment in New Languages. arXiv:2310.04799.","DOI":"10.18653\/v1\/2024.acl-long.590"},{"key":"e_1_3_2_80_2","volume-title":"ICLR","author":"Ilharco Gabriel","year":"2023","unstructured":"Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. 2023. Editing models with task arithmetic. In ICLR. 1\u201331."},{"key":"e_1_3_2_81_2","first-page":"29262","article-title":"Patching open-vocabulary models by interpolating weights","volume":"35","author":"Ilharco Gabriel","year":"2022","unstructured":"Gabriel Ilharco, Mitchell Wortsman, Samir Yitzhak Gadre, Shuran Song, Hannaneh Hajishirzi, Simon Kornblith, Ali Farhadi, and Ludwig Schmidt. 2022. Patching open-vocabulary models by interpolating weights. NeurIPS 35 (2022), 29262\u201329277.","journal-title":"NeurIPS"},{"key":"e_1_3_2_82_2","article-title":"Transformer fusion with optimal transport","author":"Imfeld Moritz","year":"2024","unstructured":"Moritz Imfeld, Jacopo Graldi, Marco Giordano, Thomas Hofmann, Sotiris Anagnostidis, and Sidak Pal Singh. 2024. Transformer fusion with optimal transport. ICLR (2024), 1\u201324.","journal-title":"ICLR"},{"key":"e_1_3_2_83_2","volume-title":"ICLR","author":"Iurada Leonardo","year":"2025","unstructured":"Leonardo Iurada, Marco Ciccone, and Tatiana Tommasi. 2025. Efficient model editing with task-localized sparse fine-tuning. In ICLR. 1\u201325."},{"key":"e_1_3_2_84_2","first-page":"876","volume-title":"UAIarXiv preprint arXiv:1803.05407","author":"Izmailov P.","year":"2018","unstructured":"P. Izmailov, A. G. Wilson, D. Podoprikhin, D. Vetrov, and T. Garipov. 2018. Averaging weights leads to wider optima and better generalization, In UAI. arXiv preprint arXiv:1803.05407, 876\u2013885."},{"key":"e_1_3_2_85_2","article-title":"Neural tangent kernel: Convergence and generalization in neural networks","volume":"31","author":"Jacot Arthur","year":"2018","unstructured":"Arthur Jacot, Franck Gabriel, and Cl\u00e9ment Hongler. 2018. Neural tangent kernel: Convergence and generalization in neural networks. NeurIPS 31 (2018), 8580\u20138589.","journal-title":"NeurIPS"},{"issue":"223","key":"e_1_3_2_86_2","first-page":"1","article-title":"Parallelizing stochastic gradient descent for least squares regression: Mini-batching, averaging, and model misspecification","volume":"18","author":"Jain Prateek","year":"2018","unstructured":"Prateek Jain, Sham M. Kakade, Rahul Kidambi, Praneeth Netrapalli, and Aaron Sidford. 2018. Parallelizing stochastic gradient descent for least squares regression: Mini-batching, averaging, and model misspecification. JMLR 18, 223 (2018), 1\u201342.","journal-title":"JMLR"},{"key":"e_1_3_2_87_2","first-page":"16048","volume-title":"CVPR","author":"Jain Samyak","year":"2023","unstructured":"Samyak Jain, Sravanti Addepalli, Pawan Kumar Sahu, Priyam Dey, and R Venkatesh Babu. 2023. Dart: Diversify-aggregate-repeat training improves generalization of neural networks. In CVPR. 16048\u201316059."},{"key":"e_1_3_2_88_2","article-title":"Model stock: All we need is just a few fine-tuned models","author":"Jang Dong-Hwan","year":"2024","unstructured":"Dong-Hwan Jang, Sangdoo Yun, and Dongyoon Han. 2024. Model stock: All we need is just a few fine-tuned models. arXiv preprint arXiv:2403.19522 (2024).","journal-title":"arXiv preprint arXiv:2403.19522"},{"key":"e_1_3_2_89_2","article-title":"Personalized soups: Personalized large language model alignment via post-hoc parameter merging","author":"Jang Joel","year":"2023","unstructured":"Joel Jang, Seungone Kim, Bill Yuchen Lin, Yizhong Wang, Jack Hessel, Luke Zettlemoyer, Hannaneh Hajishirzi, Yejin Choi, and Prithviraj Ammanabrolu. 2023. Personalized soups: Personalized large language model alignment via post-hoc parameter merging. arXiv preprint arXiv:2310.11564 (2023).","journal-title":"arXiv preprint arXiv:2310.11564"},{"key":"e_1_3_2_90_2","first-page":"14702","volume-title":"ICML","author":"Jang Joel","year":"2023","unstructured":"Joel Jang, Seungone Kim, Seonghyeon Ye, Doyoung Kim, Lajanugen Logeswaran, Moontae Lee, Kyungjae Lee, and Minjoon Seo. 2023. Exploring the benefits of training expert language models over instruction tuning. In ICML. PMLR, 14702\u201314729."},{"key":"e_1_3_2_91_2","first-page":"13","volume-title":"ISIT","author":"Jhunjhunwala Divyansh","year":"2024","unstructured":"Divyansh Jhunjhunwala, Neharika Jali, Gauri Joshi, and Shiqiang Wang. 2024. Erasure coded neural network inference via fisher averaging. In ISIT. IEEE, 13\u201318."},{"key":"e_1_3_2_92_2","first-page":"1612","volume-title":"AISTATS","author":"Jhunjhunwala Divyansh","year":"2024","unstructured":"Divyansh Jhunjhunwala, Shiqiang Wang, and Gauri Joshi. 2024. FedFisher: Leveraging fisher information for one-shot federated learning. In AISTATS. PMLR, 1612\u20131620."},{"key":"e_1_3_2_93_2","first-page":"1","volume-title":"IJCNN","author":"Ji Shaoxiong","year":"2019","unstructured":"Shaoxiong Ji, Shirui Pan, Guodong Long, Xue Li, Jing Jiang, and Zi Huang. 2019. Learning private neural language modeling with attentive aggregation. In IJCNN. IEEE, 1\u20138."},{"key":"e_1_3_2_94_2","article-title":"Mixtral of experts","author":"Jiang Albert Q.","year":"2024","unstructured":"Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et\u00a0al. 2024. Mixtral of experts. arXiv preprint arXiv:2401.04088 (2024).","journal-title":"arXiv preprint arXiv:2401.04088"},{"key":"e_1_3_2_95_2","article-title":"ForkMerge: Mitigating negative transfer in auxiliary-task learning","volume":"36","author":"Jiang Junguang","year":"2023","unstructured":"Junguang Jiang, Baixu Chen, Junwei Pan, Ximei Wang, Dapeng Liu, Jie Jiang, and Mingsheng Long. 2023. ForkMerge: Mitigating negative transfer in auxiliary-task learning. NeurIPS 36 (2023), 30367\u201330389.","journal-title":"NeurIPS"},{"key":"e_1_3_2_96_2","unstructured":"Ruochen Jin Bojian Hou Jiancong Xiao Weijie Su and Li Shen. 2024. Fine-Tuning Linear Layers Only Is a Simple yet Effective Way for Task Arithmetic. arxiv:2407.07089"},{"key":"e_1_3_2_97_2","volume-title":"ICLR","author":"Jin Xisen","year":"2023","unstructured":"Xisen Jin, Xiang Ren, Daniel Preotiuc-Pietro, and Pengxiang Cheng. 2023. Dataless knowledge fusion by merging weights of language models. In ICLR. 1\u201319."},{"key":"e_1_3_2_98_2","article-title":"Population parameter averaging (papa)","author":"Jolicoeur-Martineau Alexia","year":"2023","unstructured":"Alexia Jolicoeur-Martineau, Emy Gervais, Kilian Fatras, Yan Zhang, and Simon Lacoste-Julien. 2023. Population parameter averaging (papa). TMLR (2024). arXiv preprint arXiv:2304.03094.","journal-title":"TMLR"},{"key":"e_1_3_2_99_2","article-title":"Repair: Renormalizing permuted activations for interpolation repair","author":"Jordan Keller","year":"2023","unstructured":"Keller Jordan, Hanie Sedghi, Olga Saukh, Rahim Entezari, and Behnam Neyshabur. 2023. Repair: Renormalizing permuted activations for interpolation repair. ICLR (2023), 1\u201317.","journal-title":"ICLR"},{"key":"e_1_3_2_100_2","article-title":"Stop wasting my time! saving days of imagenet and bert training with latest weight averaging","author":"Kaddour Jean","year":"2022","unstructured":"Jean Kaddour. 2022. Stop wasting my time! saving days of imagenet and bert training with latest weight averaging. NeurIPS Workshop (2022). arXiv preprint arXiv:2209.14981.","journal-title":"NeurIPS Workshop"},{"key":"e_1_3_2_101_2","first-page":"16577","article-title":"When do flat minima optimizers work?","volume":"35","author":"Kaddour Jean","year":"2022","unstructured":"Jean Kaddour, Linqing Liu, Ricardo Silva, and Matt J. Kusner. 2022. When do flat minima optimizers work? NeurIPS 35 (2022), 16577\u201316595.","journal-title":"NeurIPS"},{"key":"e_1_3_2_102_2","article-title":"Self-MoE: Towards compositional large language models with self-specialized experts","author":"Kang Junmo","year":"2024","unstructured":"Junmo Kang, Leonid Karlinsky, Hongyin Luo, Zhen Wang, Jacob Hansen, James Glass, David Cox, Rameswar Panda, Rogerio Feris, and Alan Ritter. 2024. Self-MoE: Towards compositional large language models with self-specialized experts. arXiv preprint arXiv:2406.12034 (2024).","journal-title":"arXiv preprint arXiv:2406.12034"},{"key":"e_1_3_2_103_2","article-title":"Ctrl: A conditional transformer language model for controllable generation","author":"Keskar Nitish Shirish","year":"2019","unstructured":"Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, Caiming Xiong, and Richard Socher. 2019. Ctrl: A conditional transformer language model for controllable generation. arXiv preprint arXiv:1909.05858 (2019).","journal-title":"arXiv preprint arXiv:1909.05858"},{"key":"e_1_3_2_104_2","article-title":"Solar 10.7 b: Scaling large language models with simple yet effective depth up-scaling","author":"Kim Dahyun","year":"2023","unstructured":"Dahyun Kim, Chanjun Park, Sanghoon Kim, Wonsung Lee, Wonho Song, Yunsu Kim, Hyeonwoo Kim, Yungi Kim, Hyeonju Lee, Jihoo Kim, et\u00a0al. 2023. Solar 10.7 b: Scaling large language models with simple yet effective depth up-scaling. arXiv preprint arXiv:2312.15166 (2023).","journal-title":"arXiv preprint arXiv:2312.15166"},{"issue":"13","key":"e_1_3_2_105_2","doi-asserted-by":"crossref","first-page":"3521","DOI":"10.1073\/pnas.1611835114","article-title":"Overcoming catastrophic forgetting in neural networks","volume":"114","author":"Kirkpatrick James","year":"2017","unstructured":"James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et\u00a0al. 2017. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences 114, 13 (2017), 3521\u20133526.","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"e_1_3_2_106_2","article-title":"No train but gain: Language arithmetic for training-free language adapters enhancement","author":"Klimaszewski Mateusz","year":"2024","unstructured":"Mateusz Klimaszewski, Piotr Andruszkiewicz, and Alexandra Birch. 2024. No train but gain: Language arithmetic for training-free language adapters enhancement. arXiv preprint arXiv:2404.15737 (2024).","journal-title":"arXiv preprint arXiv:2404.15737"},{"key":"e_1_3_2_107_2","article-title":"Activated parameter locating via causal intervention for model merging","author":"Kong Fanshuang","year":"2024","unstructured":"Fanshuang Kong, Richong Zhang, and Ziqiao Wang. 2024. Activated parameter locating via causal intervention for model merging. arXiv preprint arXiv:2408.09485 (2024).","journal-title":"arXiv preprint arXiv:2408.09485"},{"key":"e_1_3_2_108_2","article-title":"GeDi: Generative discriminator guided sequence generation","author":"Krause Ben","year":"2021","unstructured":"Ben Krause, Akhilesh Deepak Gotmare, Bryan McCann, Nitish Shirish Keskar, Shafiq Joty, Richard Socher, and Nazneen Fatema Rajani. 2021. GeDi: Generative discriminator guided sequence generation. EMNLP (2021), 4929\u20134952.","journal-title":"EMNLP"},{"key":"e_1_3_2_109_2","article-title":"Explaining landscape connectivity of low-cost solutions for multilayer nets","volume":"32","author":"Kuditipudi Rohith","year":"2019","unstructured":"Rohith Kuditipudi, Xiang Wang, Holden Lee, Yi Zhang, Zhiyuan Li, Wei Hu, Rong Ge, and Sanjeev Arora. 2019. Explaining landscape connectivity of low-cost solutions for multilayer nets. NeurIPS 32 (2019), 14574\u201314583.","journal-title":"NeurIPS"},{"key":"e_1_3_2_110_2","volume-title":"ICLR","author":"Lee Yeoreum","year":"2025","unstructured":"Yeoreum Lee, Jinwook Jung, and Sungyong Baik. 2025. Mitigating parameter interference in model merging via sharpness-aware fine-tuning. In ICLR. 1\u201323."},{"key":"e_1_3_2_111_2","article-title":"It\u2019s morphing time: Unleashing the potential of multiple LLMs via multi-objective optimization","author":"Li Bingdong","year":"2024","unstructured":"Bingdong Li, Zixiang Di, Yanting Yang, Hong Qian, Peng Yang, Hao Hao, Ke Tang, and Aimin Zhou. 2024. It\u2019s morphing time: Unleashing the potential of multiple LLMs via multi-objective optimization. arXiv preprint arXiv:2407.00487 (2024).","journal-title":"arXiv preprint arXiv:2407.00487"},{"key":"e_1_3_2_112_2","article-title":"Visualizing the loss landscape of neural nets","volume":"31","author":"Li Hao","year":"2018","unstructured":"Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. 2018. Visualizing the loss landscape of neural nets. NeurIPS 31 (2018), 6391\u20136401.","journal-title":"NeurIPS"},{"key":"e_1_3_2_113_2","article-title":"Watermarking LLMs with weight quantization","author":"Li Linyang","year":"2023","unstructured":"Linyang Li, Botian Jiang, Pengyu Wang, Ke Ren, Hang Yan, and Xipeng Qiu. 2023. Watermarking LLMs with weight quantization. arXiv preprint arXiv:2310.11237 (2023).","journal-title":"arXiv preprint arXiv:2310.11237"},{"key":"e_1_3_2_114_2","article-title":"Sub-MoE: Efficient mixture-of-expert LLMs compression via subspace expert merging","author":"Li Lujun","year":"2025","unstructured":"Lujun Li, Zhu Qiyuan, Jiacheng Wang, Wei Li, Hao Gu, Sirui Han, and Yike Guo. 2025. Sub-MoE: Efficient mixture-of-expert LLMs compression via subspace expert merging. arXiv preprint arXiv:2506.23266 (2025).","journal-title":"arXiv preprint arXiv:2506.23266"},{"key":"e_1_3_2_115_2","unstructured":"Lu Li Tianyu Zhang Zhiqi Bu Suyuchen Wang Huan He Jie Fu Yonghui Wu Jiang Bian Yong Chen and Yoshua Bengio. 2024. MAP: Low-compute model merging with amortized pareto fronts via quadratic approximation. arxiv:2406.07529 [cs.LG]"},{"key":"e_1_3_2_116_2","article-title":"Branch-train-merge: Embarrassingly parallel training of expert language models","author":"Li Margaret","year":"2022","unstructured":"Margaret Li, Suchin Gururangan, Tim Dettmers, Mike Lewis, Tim Althoff, Noah A. Smith, and Luke Zettlemoyer. 2022. Branch-train-merge: Embarrassingly parallel training of expert language models. arXiv preprint arXiv:2208.03306 (2022).","journal-title":"arXiv preprint arXiv:2208.03306"},{"key":"e_1_3_2_117_2","article-title":"Merge, then compress: Demystify efficient SMoE with hints from its routing policy","author":"Li Pingzhi","year":"2024","unstructured":"Pingzhi Li, Zhenyu Zhang, Prateek Yadav, Yi-Lin Sung, Yu Cheng, Mohit Bansal, and Tianlong Chen. 2024. Merge, then compress: Demystify efficient SMoE with hints from its routing policy. ICLR (2024), 1\u201323.","journal-title":"ICLR"},{"key":"e_1_3_2_118_2","volume-title":"ICLR","author":"Li Tao","year":"2023","unstructured":"Tao Li, Zhehao Huang, Qinghua Tao, Yingwen Wu, and Xiaolin Huang. 2023. Trainable weight averaging: Efficient training by optimizing historical solutions. In ICLR. 1\u201316."},{"key":"e_1_3_2_119_2","article-title":"Scalable learned model soup on a single GPU: An efficient subspace training strategy","author":"Li Tao","year":"2024","unstructured":"Tao Li, Weisen Jiang, Fanghui Liu, Xiaolin Huang, and James T. Kwok. 2024. Scalable learned model soup on a single GPU: An efficient subspace training strategy. arXiv preprint arXiv:2407.03641 (2024).","journal-title":"arXiv preprint arXiv:2407.03641"},{"issue":"3","key":"e_1_3_2_120_2","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1109\/MSP.2020.2975749","article-title":"Federated learning: Challenges, methods, and future directions","volume":"37","author":"Li Tian","year":"2020","unstructured":"Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. 2020. Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine 37, 3 (2020), 50\u201360.","journal-title":"IEEE Signal Processing Magazine"},{"key":"e_1_3_2_121_2","article-title":"Training-Free Model Merging for Multi-target Domain Adaptation","author":"Li Wenyi","year":"2024","unstructured":"Wenyi Li, Huan-ang Gao, Mingju Gao, Beiwen Tian, Rong Zhi, and Hao Zhao. 2024. Training-Free Model Merging for Multi-target Domain Adaptation. arXiv preprint arXiv:2407.13771 (2024).","journal-title":"arXiv preprint arXiv:2407.13771"},{"key":"e_1_3_2_122_2","article-title":"Deep model fusion: A survey","author":"Li Weishi","year":"2023","unstructured":"Weishi Li, Yong Peng, Miao Zhang, Liang Ding, Han Hu, and Li Shen. 2023. Deep model fusion: A survey. arXiv preprint arXiv:2309.15698 (2023).","journal-title":"arXiv preprint arXiv:2309.15698"},{"key":"e_1_3_2_123_2","unstructured":"Xuechen Li Tianyi Zhang Yann Dubois Rohan Taori Ishaan Gulrajani Carlos Guestrin Percy Liang and Tatsunori B. Hashimoto. 2023. Alpacaeval: An automatic evaluator of instruction-following models. https:\/\/github.com\/tatsu-lab\/alpaca_eval"},{"key":"e_1_3_2_124_2","article-title":"Model merging in pre-training of large language models","author":"Li Yunshui","year":"2025","unstructured":"Yunshui Li, Yiyuan Ma, Shen Yan, Chaoyi Zhang, Jing Liu, Jianqiao Lu, Ziwen Xu, Mengzhao Chen, Minrui Wang, Shiyi Zhan, et\u00a0al. 2025. Model merging in pre-training of large language models. arXiv preprint arXiv:2505.12082 (2025).","journal-title":"arXiv preprint arXiv:2505.12082"},{"key":"e_1_3_2_125_2","article-title":"DogeRM: Equipping reward models with domain knowledge through model merging","author":"Lin Tzu-Han","year":"2024","unstructured":"Tzu-Han Lin, Chen-An Li, Hung-yi Lee, and Yun-Nung Chen. 2024. DogeRM: Equipping reward models with domain knowledge through model merging. arXiv preprint arXiv:2407.01470 (2024).","journal-title":"arXiv preprint arXiv:2407.01470"},{"key":"e_1_3_2_126_2","first-page":"13857","volume-title":"ICML","author":"Liu Chang","year":"2022","unstructured":"Chang Liu, Chenfei Lou, Runzhong Wang, Alan Yuhan Xi, Li Shen, and Junchi Yan. 2022. Deep neural network fusion via graph matching with applications to model ensemble and federated learning. In ICML. PMLR, 13857\u201313869."},{"key":"e_1_3_2_127_2","article-title":"Checkpoint merging via bayesian optimization in LLM pretraining","author":"Liu Deyuan","year":"2024","unstructured":"Deyuan Liu, Zecheng Wang, Bingning Wang, Weipeng Chen, Chunshan Li, Zhiying Tu, Dianhui Chu, Bo Li, and Dianbo Sui. 2024. Checkpoint merging via bayesian optimization in LLM pretraining. arXiv preprint arXiv:2403.19390 (2024).","journal-title":"arXiv preprint arXiv:2403.19390"},{"key":"e_1_3_2_128_2","article-title":"Linear combination of saved checkpoints makes consistency and diffusion models better","author":"Liu Enshu","year":"2024","unstructured":"Enshu Liu, Junyi Zhu, Zinan Lin, Xuefei Ning, Matthew B. Blaschko, Sergey Yekhanin, Shengen Yan, Guohao Dai, Huazhong Yang, and Yu Wang. 2024. Linear combination of saved checkpoints makes consistency and diffusion models better. arXiv preprint arXiv:2404.02241 (2024).","journal-title":"arXiv preprint arXiv:2404.02241"},{"key":"e_1_3_2_129_2","article-title":"LoRA-as-an-attack! piercing LLM safety under the share-and-play scenario","author":"Liu Hongyi","year":"2024","unstructured":"Hongyi Liu, Zirui Liu, Ruixiang Tang, Jiayi Yuan, Shaochen Zhong, Yu-Neng Chuang, Li Li, Rui Chen, and Xia Hu. 2024. LoRA-as-an-attack! piercing LLM safety under the share-and-play scenario. arXiv preprint arXiv:2403.00108 (2024).","journal-title":"arXiv preprint arXiv:2403.00108"},{"key":"e_1_3_2_130_2","first-page":"1950","article-title":"Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning","volume":"35","author":"Liu Haokun","year":"2022","unstructured":"Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A. Raffel. 2022. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. NeurIPS 35 (2022), 1950\u20131965.","journal-title":"NeurIPS"},{"key":"e_1_3_2_131_2","volume-title":"ICLR","author":"Liu Tian Yu","year":"2024","unstructured":"Tian Yu Liu, Aditya Golatkar, and Stefano Soatto. 2024. Tangent transformers for composition, privacy and removal. In ICLR. 1\u201318."},{"key":"e_1_3_2_132_2","first-page":"18676","volume-title":"ICCV","author":"Liu Tian Yu","year":"2023","unstructured":"Tian Yu Liu and Stefano Soatto. 2023. Tangent model composition for ensembling and continual fine-tuning. In ICCV. 18676\u201318686."},{"key":"e_1_3_2_133_2","article-title":"Towards safer large language models through machine unlearning","author":"Liu Zheyuan","year":"2024","unstructured":"Zheyuan Liu, Guangyao Dou, Zhaoxuan Tan, Yijun Tian, and Meng Jiang. 2024. Towards safer large language models through machine unlearning. arXiv preprint arXiv:2402.10058 (2024).","journal-title":"arXiv preprint arXiv:2402.10058"},{"key":"e_1_3_2_134_2","article-title":"Online merging optimizers for boosting rewards and mitigating tax in alignment","author":"Lu Keming","year":"2024","unstructured":"Keming Lu, Bowen Yu, Fei Huang, Yang Fan, Runji Lin, and Chang Zhou. 2024. Online merging optimizers for boosting rewards and mitigating tax in alignment. arXiv preprint arXiv:2405.17931 (2024).","journal-title":"arXiv preprint arXiv:2405.17931"},{"key":"e_1_3_2_135_2","article-title":"Twin-merging: Dynamic integration of modular expertise in model merging","author":"Lu Zhenyi","year":"2024","unstructured":"Zhenyi Lu, Chenghao Fan, Wei Wei, Xiaoye Qu, Dangyang Chen, and Yu Cheng. 2024. Twin-merging: Dynamic integration of modular expertise in model merging. arXiv preprint arXiv:2406.15479 (2024).","journal-title":"arXiv preprint arXiv:2406.15479"},{"key":"e_1_3_2_136_2","first-page":"1930","volume-title":"SIGKDD","author":"Ma Jiaqi","year":"2018","unstructured":"Jiaqi Ma, Zhe Zhao, Xinyang Yi, Jilin Chen, Lichan Hong, and Ed H. Chi. 2018. Modeling task relationships in multi-task learning with multi-gate mixture-of-experts. In SIGKDD. ACM, 1930\u20131939."},{"key":"e_1_3_2_137_2","article-title":"Souper-model: How simple arithmetic unlocks state-of-the-art LLM performance","author":"Maiti Shalini","year":"2025","unstructured":"Shalini Maiti, Amar Budhiraja, Bhavul Gauri, Gaurav Chaurasia, Anton Protopopov, Alexis Audran-Reiss, Michael Slater, Despoina Magka, Tatiana Shavrina, Roberta Raileanu, et\u00a0al. 2025. Souper-model: How simple arithmetic unlocks state-of-the-art LLM performance. arXiv preprint arXiv:2511.13254 (2025).","journal-title":"arXiv preprint arXiv:2511.13254"},{"key":"e_1_3_2_138_2","first-page":"1485","volume-title":"ACM MM","author":"Marcel S\u00e9bastien","year":"2010","unstructured":"S\u00e9bastien Marcel and Yann Rodriguez. 2010. Torchvision the machine-vision package of torch. In ACM MM. 1485\u20131488."},{"key":"e_1_3_2_139_2","volume-title":"ICML","author":"Marczak Daniel","year":"2025","unstructured":"Daniel Marczak, Simone Magistri, Sebastian Cygert, Bart\u0142omiej Twardowski, Andrew D. Bagdanov, and Joost van de Weijer. 2025. No task left behind: Isotropic model merging with common and task-specific subspaces. In ICML."},{"key":"e_1_3_2_140_2","doi-asserted-by":"crossref","unstructured":"Daniel Marczak Bart\u0142omiej Twardowski Tomasz Trzci\u0144ski and Sebastian Cygert. 2024. MagMax: Leveraging model merging for seamless continual learning. In European Conference on Computer Vision. Springer 379\u2013395.","DOI":"10.1007\/978-3-031-73013-9_22"},{"key":"e_1_3_2_141_2","first-page":"17703","article-title":"Merging models with fisher-weighted averaging","volume":"35","author":"Matena Michael S.","year":"2022","unstructured":"Michael S. Matena and Colin A. Raffel. 2022. Merging models with fisher-weighted averaging. NeurIPS 35 (2022), 17703\u201317716.","journal-title":"NeurIPS"},{"key":"e_1_3_2_142_2","first-page":"1273","volume-title":"AISTATS","author":"McMahan Brendan","year":"2017","unstructured":"Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. In AISTATS. PMLR, 1273\u20131282."},{"key":"e_1_3_2_143_2","article-title":"MergeMoE: Efficient compression of MoE models via expert output merging","author":"Miao Ruijie","year":"2025","unstructured":"Ruijie Miao, Yilun Yao, Zihan Wang, Zhiming Wang, Bairen Yi, LingJun Liu, Yikai Zhao, and Tong Yang. 2025. MergeMoE: Efficient compression of MoE models via expert output merging. arXiv preprint arXiv:2510.14436 (2025).","journal-title":"arXiv preprint arXiv:2510.14436"},{"key":"e_1_3_2_144_2","article-title":"Mergenetic: A simple evolutionary model merging library","author":"Minut Adrian Robert","year":"2025","unstructured":"Adrian Robert Minut, Tommaso Mencattini, Andrea Santilli, Donato Crisostomi, and Emanuele Rodol\u00e0. 2025. Mergenetic: A simple evolutionary model merging library. arXiv preprint arXiv:2505.11427 (2025).","journal-title":"arXiv preprint arXiv:2505.11427"},{"key":"e_1_3_2_145_2","first-page":"3470","volume-title":"ACL","author":"Mishra Swaroop","year":"2022","unstructured":"Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. 2022. Cross-task generalization via natural language crowdsourcing instructions. In ACL. Association for Computational Linguistics (ACL), 3470\u20133487."},{"key":"e_1_3_2_146_2","unstructured":"Mohammed Muqeeth Haokun Liu Yufan Liu and Colin Raffel. 2024. Learning to route among specialized experts for zero-shot generalization. In ICML. PMLR 36829\u201336846."},{"key":"e_1_3_2_147_2","article-title":"Soft merging of experts with adaptive routing","author":"Muqeeth Mohammed","year":"2024","unstructured":"Mohammed Muqeeth, Haokun Liu, and Colin Raffel. 2024. Soft merging of experts with adaptive routing. TMLR (2024), 1\u201339.","journal-title":"TMLR"},{"key":"e_1_3_2_148_2","article-title":"Uniform convergence may be unable to explain generalization in deep learning","volume":"32","author":"Nagarajan Vaishnavh","year":"2019","unstructured":"Vaishnavh Nagarajan and J. Zico Kolter. 2019. Uniform convergence may be unable to explain generalization in deep learning. NeurIPS 32 (2019), 11615\u201311626.","journal-title":"NeurIPS"},{"key":"e_1_3_2_149_2","article-title":"MaxFusion: Plug&play multi-modal generation in text-to-image diffusion models","author":"Nair Nithin Gopalakrishnan","year":"2024","unstructured":"Nithin Gopalakrishnan Nair, Jeya Maria Jose Valanarasu, and Vishal M. Patel. 2024. MaxFusion: Plug&play multi-modal generation in text-to-image diffusion models. arXiv preprint arXiv:2404.09977 (2024).","journal-title":"arXiv preprint arXiv:2404.09977"},{"key":"e_1_3_2_150_2","article-title":"Equivariant deep weight space alignment","author":"Navon Aviv","year":"2024","unstructured":"Aviv Navon, Aviv Shamsian, Ethan Fetaya, Gal Chechik, Nadav Dym, and Haggai Maron. 2024. Equivariant deep weight space alignment. ICML (2024), 37376\u201337395.","journal-title":"ICML"},{"key":"e_1_3_2_151_2","first-page":"1","volume-title":"ICASSP","author":"Nguyen Dang","year":"2023","unstructured":"Dang Nguyen, Trang Nguyen, Khai Nguyen, Dinh Phung, Hung Bui, and Nhat Ho. 2023. On cross-layer alignment for model fusion of heterogeneous neural networks. In ICASSP. IEEE, 1\u20135."},{"key":"e_1_3_2_152_2","article-title":"Expert merging in sparse mixture of experts with nash bargaining","author":"Nguyen Dung V.","year":"2025","unstructured":"Dung V. Nguyen, Anh T. Nguyen, Minh H. Nguyen, Luc Q. Nguyen, Shiqi Jiang, Ethan Fetaya, Linh Duy Tran, Gal Chechik, and Tan M. Nguyen. 2025. Expert merging in sparse mixture of experts with nash bargaining. arXiv preprint arXiv:2510.16138 (2025).","journal-title":"arXiv preprint arXiv:2510.16138"},{"key":"e_1_3_2_153_2","article-title":"Forgetting before Learning: Utilizing parametric arithmetic for knowledge updating in large language models","author":"Ni Shiwen","year":"2023","unstructured":"Shiwen Ni, Dingwei Chen, Chengming Li, Xiping Hu, Ruifeng Xu, and Min Yang. 2023. Forgetting before Learning: Utilizing parametric arithmetic for knowledge updating in large language models. arXiv preprint arXiv:2311.08011 (2023).","journal-title":"arXiv preprint arXiv:2311.08011"},{"key":"e_1_3_2_154_2","unstructured":"Guillermo Ortiz-Jimenez Alessandro Favero and Pascal Frossard. 2023. Task arithmetic in the tangent space: Improved editing of pre-trained models. NeurIPS 36 (2023) 66727\u201366754."},{"key":"e_1_3_2_155_2","article-title":"Towards modular llms by building and reusing a library of loras","author":"Ostapenko Oleksiy","year":"2024","unstructured":"Oleksiy Ostapenko, Zhan Su, Edoardo Maria Ponti, Laurent Charlin, Nicolas Le Roux, Matheus Pereira, Lucas Caccia, and Alessandro Sordoni. 2024. Towards modular llms by building and reusing a library of loras. ICML. PMLR, 38885\u201338904.","journal-title":"ICML"},{"key":"e_1_3_2_156_2","article-title":"Lottery ticket adaptation: Mitigating destructive interference in LLMs","author":"Panda Ashwinee","year":"2024","unstructured":"Ashwinee Panda, Berivan Isik, Xiangyu Qi, Sanmi Koyejo, Tsachy Weissman, and Prateek Mittal. 2024. Lottery ticket adaptation: Mitigating destructive interference in LLMs. arXiv preprint arXiv:2406.16797 (2024).","journal-title":"arXiv preprint arXiv:2406.16797"},{"key":"e_1_3_2_157_2","first-page":"68","article-title":"The california consumer privacy act: Towards a european-style privacy regime in the united states","volume":"23","author":"Pardau Stuart L.","year":"2018","unstructured":"Stuart L. Pardau. 2018. The california consumer privacy act: Towards a european-style privacy regime in the united states. J. Tech. L. & Pol\u2019y 23 (2018), 68.","journal-title":"J. Tech. L. & Pol\u2019y"},{"key":"e_1_3_2_158_2","volume-title":"ICML","author":"Patil Shishir G","year":"2025","unstructured":"Shishir G Patil, Huanzhi Mao, Fanjia Yan, Charlie Cheng-Jie Ji, Vishnu Suresh, Ion Stoica, and Joseph E. Gonzalez. 2025. The Berkeley function calling leaderboard (BFCL): From tool use to agentic evaluation of large language models. In ICML."},{"key":"e_1_3_2_159_2","first-page":"20237","volume-title":"CVPR","author":"Pe\u00f1a Fidel A. Guerrero","year":"2023","unstructured":"Fidel A. Guerrero Pe\u00f1a, Heitor Rapela Medeiros, Thomas Dubail, Masih Aminbeidokhti, Eric Granger, and Marco Pedersoli. 2023. Re-basin via implicit Sinkhorn differentiation. In CVPR. 20237\u201320246."},{"key":"e_1_3_2_160_2","article-title":"PAFT: A parallel training paradigm for effective LLM fine-tuning","author":"Pentyala Shiva Kumar","year":"2024","unstructured":"Shiva Kumar Pentyala, Zhichao Wang, Bin Bi, Kiran Ramnath, Xiang-Bo Mao, Regunathan Radhakrishnan, Sitaram Asur, et\u00a0al. 2024. PAFT: A parallel training paradigm for effective LLM fine-tuning. arXiv preprint arXiv:2406.17923 (2024).","journal-title":"arXiv preprint arXiv:2406.17923"},{"key":"e_1_3_2_161_2","first-page":"1015","volume-title":"ACM MM","author":"Piczak Karol J","year":"2015","unstructured":"Karol J Piczak. 2015. ESC: Dataset for environmental sound classification. In ACM MM. 1015\u20131018."},{"key":"e_1_3_2_162_2","article-title":"A second-order perspective on compositionality and incremental learning","author":"Porrello Angelo","year":"2024","unstructured":"Angelo Porrello, Lorenzo Bonicelli, Pietro Buzzega, Monica Millunzi, Simone Calderara, and Rita Cucchiara. 2024. A second-order perspective on compositionality and incremental learning. arXiv preprint arXiv:2405.16350 (2024).","journal-title":"arXiv preprint arXiv:2405.16350"},{"key":"e_1_3_2_163_2","article-title":"DynaMMo: Dynamic model merging for efficient class incremental learning for medical images","author":"Qazi Mohammad Areeb","year":"2024","unstructured":"Mohammad Areeb Qazi, Ibrahim Almakky, Anees Ur Rehman Hashmi, Santosh Sanjeev, and Mohammad Yaqub. 2024. DynaMMo: Dynamic model merging for efficient class incremental learning for medical images. arXiv preprint arXiv:2404.14099 (2024).","journal-title":"arXiv preprint arXiv:2404.14099"},{"key":"e_1_3_2_164_2","first-page":"4200","volume-title":"EMNLP","author":"Qiu Haiquan","year":"2025","unstructured":"Haiquan Qiu, You Wu, Dong Li, Jianmin Guo, and Quanming Yao. 2025. Superpose task-specific features for model merging. In EMNLP. 4200\u20134214."},{"issue":"8","key":"e_1_3_2_165_2","first-page":"9","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford Alec","year":"2019","unstructured":"Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et\u00a0al. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 (2019), 9.","journal-title":"OpenAI Blog"},{"key":"e_1_3_2_166_2","article-title":"Direct preference optimization: Your language model is secretly a reward model","volume":"36","author":"Rafailov Rafael","year":"2023","unstructured":"Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. NeurIPS 36 (2023), 53728\u201353741.","journal-title":"NeurIPS"},{"key":"e_1_3_2_167_2","article-title":"DEM: Distribution edited model for training with mixed data distributions","author":"Ram Dhananjay","year":"2024","unstructured":"Dhananjay Ram, Aditya Rawal, Momchil Hardalov, Nikolaos Pappas, and Sheng Zha. 2024. DEM: Distribution edited model for training with mixed data distributions. arXiv preprint arXiv:2406.15570 (2024).","journal-title":"arXiv preprint arXiv:2406.15570"},{"key":"e_1_3_2_168_2","first-page":"28656","volume-title":"ICML","author":"Ram\u00e9 Alexandre","year":"2023","unstructured":"Alexandre Ram\u00e9, Kartik Ahuja, Jianyu Zhang, Matthieu Cord, L\u00e9on Bottou, and David Lopez-Paz. 2023. Model ratatouille: Recycling diverse models for out-of-distribution generalization. In ICML. PMLR, 28656\u201328679."},{"key":"e_1_3_2_169_2","article-title":"Rewarded soups: Towards pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards","volume":"36","author":"Rame Alexandre","year":"2023","unstructured":"Alexandre Rame, Guillaume Couairon, Corentin Dancette, Jean-Baptiste Gaya, Mustafa Shukor, Laure Soulier, and Matthieu Cord. 2023. Rewarded soups: Towards pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards. NeurIPS 36 (2023), 71095\u201371134.","journal-title":"NeurIPS"},{"key":"e_1_3_2_170_2","article-title":"WARP: On the benefits of weight averaged rewarded policies","author":"Ram\u00e9 Alexandre","year":"2024","unstructured":"Alexandre Ram\u00e9, Johan Ferret, Nino Vieillard, Robert Dadashi, L\u00e9onard Hussenot, Pierre-Louis Cedoz, Pier Giuseppe Sessa, Sertan Girgin, Arthur Douillard, and Olivier Bachem. 2024. WARP: On the benefits of weight averaged rewarded policies. arXiv preprint arXiv:2406.16768 (2024).","journal-title":"arXiv preprint arXiv:2406.16768"},{"key":"e_1_3_2_171_2","first-page":"10821","article-title":"Diverse weight averaging for out-of-distribution generalization","volume":"35","author":"Rame Alexandre","year":"2022","unstructured":"Alexandre Rame, Matthieu Kirchmeyer, Thibaud Rahier, Alain Rakotomamonjy, Patrick Gallinari, and Matthieu Cord. 2022. Diverse weight averaging for out-of-distribution generalization. NeurIPS 35 (2022), 10821\u201310836.","journal-title":"NeurIPS"},{"key":"e_1_3_2_172_2","unstructured":"Alexandre Ram\u00e9 Nino Vieillard L\u00e9onard Hussenot Robert Dadashi Geoffrey Cideron Olivier Bachem and Johan Ferret. 2024. Warm: On the benefits of weight averaged reward models. In ICML. PMLR 42048\u201342073."},{"key":"e_1_3_2_173_2","article-title":"Revisiting adapters with adversarial training","author":"Rebuffi Sylvestre-Alvise","year":"2023","unstructured":"Sylvestre-Alvise Rebuffi, Francesco Croce, and Sven Gowal. 2023. Revisiting adapters with adversarial training. ICLR. 1\u201320.","journal-title":"ICLR"},{"key":"e_1_3_2_174_2","doi-asserted-by":"crossref","DOI":"10.1080\/09540099550039318","article-title":"Catastrophic forgetting, rehearsal and pseudorehearsal","author":"Robins Anthony","year":"1995","unstructured":"Anthony Robins. 1995. Catastrophic forgetting, rehearsal and pseudorehearsal. Connection Science 7, 2 (1995), 123\u2013146.","journal-title":"Connection Science"},{"key":"e_1_3_2_175_2","first-page":"10684","volume-title":"CVPR","author":"Rombach Robin","year":"2022","unstructured":"Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj\u00f6rn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In CVPR. 10684\u201310695."},{"key":"e_1_3_2_176_2","unstructured":"Simo Ryu. [n. d.]. Merging loras. https:\/\/github.com\/cloneofsimo\/lora"},{"issue":"4","key":"e_1_3_2_177_2","first-page":"e1249","article-title":"Ensemble learning: A survey","volume":"8","author":"Sagi Omer","year":"2018","unstructured":"Omer Sagi and Lior Rokach. 2018. Ensemble learning: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 8, 4 (2018), e1249.","journal-title":"Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery"},{"key":"e_1_3_2_178_2","volume-title":"NeurIPS Workshop","author":"Sanyal Sunny","year":"2023","unstructured":"Sunny Sanyal, Atula Tejaswi Neerkaje, Jean Kaddour, Abhishek Kumar, et\u00a0al. 2023. Early weight averaging meets high learning rates for LLM pre-training. In NeurIPS Workshop."},{"key":"e_1_3_2_179_2","first-page":"2846","volume-title":"EACL","author":"Schumann Raphael","year":"2024","unstructured":"Raphael Schumann, Elman Mansimov, Yi-An Lai, Nikolaos Pappas, Xibin Gao, and Yi Zhang. 2024. Backward compatibility during data updates by weight interpolation. In EACL. 2846\u20132861."},{"key":"e_1_3_2_180_2","first-page":"525","volume-title":"NeurIPS","author":"Sener Ozan","year":"2018","unstructured":"Ozan Sener and Vladlen Koltun. 2018. Multi-task learning as multi-objective optimization. In NeurIPS. 525\u2013536."},{"key":"e_1_3_2_181_2","article-title":"Ziplora: Any subject in any style by effectively merging loras","author":"Shah Viraj","year":"2023","unstructured":"Viraj Shah, Nataniel Ruiz, Forrester Cole, Erika Lu, Svetlana Lazebnik, Yuanzhen Li, and Varun Jampani. 2023. Ziplora: Any subject in any style by effectively merging loras. arXiv preprint arXiv:2311.13600 (2023).","journal-title":"arXiv preprint arXiv:2311.13600"},{"key":"e_1_3_2_182_2","first-page":"245","volume-title":"Proceedings of the 12th Annual Conference on Computer Graphics and Interactive Techniques","author":"Shoemake Ken","year":"1985","unstructured":"Ken Shoemake. 1985. Animating rotation with quaternion curves. In Proceedings of the 12th Annual Conference on Computer Graphics and Interactive Techniques. 245\u2013254."},{"key":"e_1_3_2_183_2","unstructured":"Mustafa Shukor Corentin Dancette Alexandre Rame and Matthieu Cord. 2023. UnIVAL: Unified model for image video audio and language tasks. arXiv preprint arXiv:2307.16184 (2023)."},{"key":"e_1_3_2_184_2","first-page":"9722","volume-title":"ICML","author":"Simsek Berfin","year":"2021","unstructured":"Berfin Simsek, Fran\u00e7ois Ged, Arthur Jacot, Francesco Spadaro, Cl\u00e9ment Hongler, Wulfram Gerstner, and Johanni Brea. 2021. Geometry of the loss landscape in overparameterized neural networks: Symmetries and invariances. In ICML. PMLR, 9722\u20139732."},{"key":"e_1_3_2_185_2","first-page":"22045","article-title":"Model fusion via optimal transport","volume":"33","author":"Singh Sidak Pal","year":"2020","unstructured":"Sidak Pal Singh and Martin Jaggi. 2020. Model fusion via optimal transport. NeurIPS 33 (2020), 22045\u201322055.","journal-title":"NeurIPS"},{"key":"e_1_3_2_186_2","article-title":"Subspace-boosted model merging","author":"Skorobogat Ronald","year":"2025","unstructured":"Ronald Skorobogat, Karsten Roth, and Mariana-Iuliana Georgescu. 2025. Subspace-boosted model merging. arXiv preprint arXiv:2506.16506 (2025).","journal-title":"arXiv preprint arXiv:2506.16506"},{"key":"e_1_3_2_187_2","first-page":"731","volume-title":"2017 16th IEEE ICML and Applications (ICMLA)","author":"Smith Joshua","year":"2017","unstructured":"Joshua Smith and Michael Gashler. 2017. An investigation of how neural networks learn from the experiences of peers through periodic weight averaging. In 2017 16th IEEE ICML and Applications (ICMLA). IEEE, 731\u2013736."},{"key":"e_1_3_2_188_2","first-page":"32211","volume-title":"ICML","author":"Song Yang","year":"2023","unstructured":"Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. 2023. Consistency models. In ICML. 32211\u201332252."},{"key":"e_1_3_2_189_2","article-title":"ZipIt! merging models from different tasks without training","author":"Stoica George","year":"2024","unstructured":"George Stoica, Daniel Bolya, Jakob Bjorner, Taylor Hearn, and Judy Hoffman. 2024. ZipIt! merging models from different tasks without training. ICLR (2024), 1\u201323.","journal-title":"ICLR"},{"key":"e_1_3_2_190_2","volume-title":"ICLR","author":"Stoica George","year":"2024","unstructured":"George Stoica, Pratik Ramesh, Boglarka Ecsedi, Leshem Choshen, and Judy Hoffman. 2024. Model merging with SVD to tie the Knots. In ICLR. 1\u201319."},{"key":"e_1_3_2_191_2","article-title":"Branch-train-mix: Mixing expert LLMs into a mixture-of-experts LLM","author":"Sukhbaatar Sainbayar","year":"2024","unstructured":"Sainbayar Sukhbaatar, Olga Golovneva, Vasu Sharma, Hu Xu, Xi Victoria Lin, Baptiste Rozi\u00e8re, Jacob Kahn, Daniel Li, Wen-tau Yih, Jason Weston, et\u00a0al. 2024. Branch-train-mix: Mixing expert LLMs into a mixture-of-experts LLM. arXiv preprint arXiv:2403.07816 (2024).","journal-title":"arXiv preprint arXiv:2403.07816"},{"key":"e_1_3_2_192_2","volume-title":"NeurIPS","author":"Sun Wenju","year":"2025","unstructured":"Wenju Sun, Qingyong Li, Wen Wang, Yang Liu, Yangli-ao Geng, and Boyang Li. 2025. Towards minimizing feature drift in model merging: Layer-wise task vector fusion for adaptive knowledge integration. In NeurIPS."},{"key":"e_1_3_2_193_2","first-page":"8728","article-title":"Adashare: Learning what to share for efficient deep multi-task learning","volume":"33","author":"Sun Ximeng","year":"2020","unstructured":"Ximeng Sun, Rameswar Panda, Rogerio Feris, and Kate Saenko. 2020. Adashare: Learning what to share for efficient deep multi-task learning. NeurIPS 33 (2020), 8728\u20138740.","journal-title":"NeurIPS"},{"key":"e_1_3_2_194_2","doi-asserted-by":"crossref","unstructured":"Anirudh S. Sundar Chao-Han Huck Yang David M. Chan Shalini Ghosh Venkatesh Ravichandran and Phani Sankar Nidadavolu. 2024. Multimodal attention merging for improved speech recognition and audio event classification. In ICASSPW. IEEE 655\u2013659.","DOI":"10.1109\/ICASSPW62465.2024.10627466"},{"key":"e_1_3_2_195_2","article-title":"An empirical study of multimodal model merging","author":"Sung Yi-Lin","year":"2023","unstructured":"Yi-Lin Sung, Linjie Li, Kevin Lin, Zhe Gan, Mohit Bansal, and Lijuan Wang. 2023. An empirical study of multimodal model merging. EMNLP (2023), 1563\u20131575.","journal-title":"EMNLP"},{"key":"e_1_3_2_196_2","article-title":"Merging by matching models in task subspaces","author":"Tam Derek","year":"2023","unstructured":"Derek Tam, Mohit Bansal, and Colin Raffel. 2023. Merging by matching models in task subspaces. arXiv preprint arXiv:2312.04339 (2023).","journal-title":"arXiv preprint arXiv:2312.04339"},{"key":"e_1_3_2_197_2","article-title":"Realistic evaluation of model merging for compositional generalization","author":"Tam Derek","year":"2024","unstructured":"Derek Tam, Yash Kant, Brian Lester, Igor Gilitschenski, and Colin Raffel. 2024. Realistic evaluation of model merging for compositional generalization. arXiv preprint arXiv:2409.18314 (2024).","journal-title":"arXiv preprint arXiv:2409.18314"},{"key":"e_1_3_2_198_2","volume-title":"NeurIPS 2024 Competition Track","author":"Tam Derek","year":"2024","unstructured":"Derek Tam, Margaret Li, Prateek Yadav, Rickard Br\u00fcel Gabrielsson, Jiacheng Zhu, Kristjan Greenewald, Mikhail Yurochkin, Mohit Bansal, Colin Raffel, and Leshem Choshen. 2024. Llm merging: Building llms efficiently through merging. In NeurIPS 2024 Competition Track."},{"key":"e_1_3_2_199_2","article-title":"Concrete subspace learning based interference elimination for multi-task model fusion","author":"Tang Anke","year":"2023","unstructured":"Anke Tang, Li Shen, Yong Luo, Liang Ding, Han Hu, Bo Du, and Dacheng Tao. 2023. Concrete subspace learning based interference elimination for multi-task model fusion. arXiv preprint arXiv:2312.06173 (2023).","journal-title":"arXiv preprint arXiv:2312.06173"},{"key":"e_1_3_2_200_2","article-title":"Towards efficient pareto set approximation via mixture of experts based model fusion","author":"Tang Anke","year":"2024","unstructured":"Anke Tang, Li Shen, Yong Luo, Shiwei Liu, Han Hu, and Bo Du. 2024. Towards efficient pareto set approximation via mixture of experts based model fusion. arXiv preprint arXiv:2406.09770 (2024).","journal-title":"arXiv preprint arXiv:2406.09770"},{"key":"e_1_3_2_201_2","volume-title":"Journal of Machine Learning Research","author":"Tang Anke","year":"2025","unstructured":"Anke Tang, Li Shen, Yong Luo, Enneng Yang, Han Hu, Leifei Zhang, Bo Do, and Dacheng Tao. 2025. FusionBench: A comprehensive benchmark of deep model fusion. In Journal of Machine Learning Research."},{"key":"e_1_3_2_202_2","article-title":"Merging multi-task models via weight-ensembling mixture of experts","author":"Tang Anke","year":"2024","unstructured":"Anke Tang, Li Shen, Yong Luo, Nan Yin, Lefei Zhang, and Dacheng Tao. 2024. Merging multi-task models via weight-ensembling mixture of experts. ICML (2024), 47778\u201347799.","journal-title":"ICML"},{"key":"e_1_3_2_203_2","article-title":"Parameter efficient multi-task model fusion with partial linearization","author":"Tang Anke","year":"2024","unstructured":"Anke Tang, Li Shen, Yong Luo, Yibing Zhan, Han Hu, Bo Du, Yixin Chen, and Dacheng Tao. 2024. Parameter efficient multi-task model fusion with partial linearization. ICLR (2024), 1\u201331.","journal-title":"ICLR"},{"key":"e_1_3_2_204_2","volume-title":"NeurIPS","author":"Tang Anke","year":"2025","unstructured":"Anke Tang, Enneng Yang, Li Shen, Yong Luo, Han Hu, Lefei Zhang, Bo Du, and Dacheng Tao. 2025. Merging on the fly without retraining: A sequential approach to scalable continual model merging. In NeurIPS."},{"key":"e_1_3_2_205_2","article-title":"Unlocking the potential of model merging for low-resource languages","author":"Tao Mingxu","year":"2024","unstructured":"Mingxu Tao, Chen Zhang, Quzhe Huang, Tianyao Ma, Songfang Huang, Dongyan Zhao, and Yansong Feng. 2024. Unlocking the potential of model merging for low-resource languages. arXiv preprint arXiv:2407.03994 (2024).","journal-title":"arXiv preprint arXiv:2407.03994"},{"key":"e_1_3_2_206_2","article-title":"Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results","volume":"30","author":"Tarvainen Antti","year":"2017","unstructured":"Antti Tarvainen and Harri Valpola. 2017. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. NeurIPS 30 (2017), 1195\u20131204.","journal-title":"NeurIPS"},{"key":"e_1_3_2_207_2","first-page":"15300","article-title":"Optimizing mode connectivity via neuron alignment","volume":"33","author":"Tatro Norman","year":"2020","unstructured":"Norman Tatro, Pin-Yu Chen, Payel Das, Igor Melnyk, Prasanna Sattigeri, and Rongjie Lai. 2020. Optimizing mode connectivity via neuron alignment. NeurIPS 33 (2020), 15300\u201315311.","journal-title":"NeurIPS"},{"key":"e_1_3_2_208_2","article-title":"Gemini: A family of highly capable multimodal models","author":"Team Gemini","year":"2023","unstructured":"Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M. Dai, Anja Hauth, et\u00a0al. 2023. Gemini: A family of highly capable multimodal models. arXiv preprint arXiv:2312.11805 (2023).","journal-title":"arXiv preprint arXiv:2312.11805"},{"key":"e_1_3_2_209_2","unstructured":"Kimi Team Angang Du Bofei Gao Bowei Xing Changjiu Jiang Cheng Chen Cheng Li Chenjun Xiao Chenzhuang Du Chonghua Liao et\u00a0al. 2025. Kimi k1. 5: Scaling reinforcement learning with LLMs. arXiv preprint arXiv:2501.12599 (2025)."},{"key":"e_1_3_2_210_2","unstructured":"Qwen Team. 2025. Qwq-32b: Embracing the power of reinforcement learning. https:\/\/qwenlm.github.io\/blog\/qwq-32b\/"},{"key":"e_1_3_2_211_2","article-title":"A unified module for accelerating stable-diffusion: Lcm-lora","author":"Thakur Ayush","year":"2024","unstructured":"Ayush Thakur and Rashmi Vashisth. 2024. A unified module for accelerating stable-diffusion: Lcm-lora. arXiv preprint arXiv:2403.16024 (2024).","journal-title":"arXiv preprint arXiv:2403.16024"},{"key":"e_1_3_2_212_2","first-page":"7349","volume-title":"LREC-COLING","author":"Thennal D. K.","year":"2024","unstructured":"D. K. Thennal, Ganesh Nathan, and M. S. Suchithra. 2024. Fisher mask nodes for language model merging. In LREC-COLING. 7349\u20137355."},{"key":"e_1_3_2_213_2","first-page":"303","volume-title":"EuroS&P","author":"Thudi Anvith","year":"2022","unstructured":"Anvith Thudi, Gabriel Deza, Varun Chandrasekaran, and Nicolas Papernot. 2022. Unrolling sgd: Understanding factors influencing machine unlearning. In EuroS&P. IEEE, 303\u2013319."},{"key":"e_1_3_2_214_2","article-title":"WSM: Decay-free learning rate schedule via checkpoint merging for LLM pre-training","author":"Tian Changxin","year":"2025","unstructured":"Changxin Tian, Jiapeng Wang, Qian Zhao, Kunlong Chen, Jia Liu, Ziqi Liu, Jiaxin Mao, Wayne Xin Zhao, Zhiqiang Zhang, and Jun Zhou. 2025. WSM: Decay-free learning rate schedule via checkpoint merging for LLM pre-training. arXiv preprint arXiv:2507.17634 (2025).","journal-title":"arXiv preprint arXiv:2507.17634"},{"key":"e_1_3_2_215_2","first-page":"12512","volume-title":"CVPR","author":"Tian Jie","year":"2025","unstructured":"Jie Tian, Xiaoye Qu, Zhenyi Lu, Wei Wei, Sichen Liu, and Yu Cheng. 2025. Extrapolating and decoupling image-to-video generation models: Motion modeling is easier than you think. In CVPR. 12512\u201312521."},{"key":"e_1_3_2_216_2","article-title":"Llama: Open and efficient foundation language models","author":"Touvron Hugo","year":"2023","unstructured":"Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth\u00e9e Lacroix, Baptiste Rozi\u00e8re, Naman Goyal, Eric Hambro, Faisal Azhar, et\u00a0al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023).","journal-title":"arXiv preprint arXiv:2302.13971"},{"key":"e_1_3_2_217_2","article-title":"Llama 2: Open foundation and fine-tuned chat models","author":"Touvron Hugo","year":"2023","unstructured":"Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et\u00a0al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 (2023).","journal-title":"arXiv preprint arXiv:2307.09288"},{"key":"e_1_3_2_218_2","first-page":"133","volume-title":"AAAI Workshop","author":"Utans Joachim","year":"1996","unstructured":"Joachim Utans. 1996. Weight averaging for neural networks and local resampling schemes. In AAAI Workshop. Citeseer, 133\u2013138."},{"issue":"7","key":"e_1_3_2_219_2","first-page":"3614","article-title":"Multi-task learning for dense prediction tasks: A survey","volume":"44","author":"Vandenhende Simon","year":"2021","unstructured":"Simon Vandenhende, Stamatios Georgoulis, Wouter Van Gansbeke, Marc Proesmans, Dengxin Dai, and Luc Van Gool. 2021. Multi-task learning for dense prediction tasks: A survey. TPAMI 44, 7 (2021), 3614\u20133633.","journal-title":"TPAMI"},{"key":"e_1_3_2_220_2","article-title":"Attention is all you need","volume":"30","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. NeurIPS 30 (2017), 5998\u20136008.","journal-title":"NeurIPS"},{"key":"e_1_3_2_221_2","volume-title":"ICLR","author":"Oswald Johannes von","year":"2021","unstructured":"Johannes von Oswald, Seijin Kobayashi, Alexander Meulemans, Christian Henning, Benjamin F. Grewe, and Jo\u00e3o Sacramento. 2021. Neural networks with late-phase weights. In ICLR. 1\u201325."},{"key":"e_1_3_2_222_2","volume-title":"The Twelfth International Conference on Learning Representations","author":"Wan Fanqi","unstructured":"Fanqi Wan, Xinting Huang, Deng Cai, Xiaojun Quan, Wei Bi, and Shuming Shi. 2024. Knowledge fusion of large language models. In The Twelfth International Conference on Learning Representations. 1\u201320."},{"key":"e_1_3_2_223_2","article-title":"Knowledge fusion of large language models","author":"Wan Fanqi","year":"2024","unstructured":"Fanqi Wan, Xinting Huang, Deng Cai, Xiaojun Quan, Wei Bi, and Shuming Shi. 2024. Knowledge fusion of large language models. ICLR (2024). arXiv preprint arXiv:2401.10491.","journal-title":"ICLR"},{"key":"e_1_3_2_224_2","article-title":"FuseChat: Knowledge fusion of chat models","author":"Wan Fanqi","year":"2024","unstructured":"Fanqi Wan, Ziyi Yang, Longguang Zhong, Xiaojun Quan, Xinting Huang, and Wei Bi. 2024. FuseChat: Knowledge fusion of chat models. arXiv preprint arXiv:2402.16107 (2024).","journal-title":"arXiv preprint arXiv:2402.16107"},{"key":"e_1_3_2_225_2","article-title":"Openchat: Advancing open-source language models with mixed-quality data","author":"Wang Guan","year":"2023","unstructured":"Guan Wang, Sijie Cheng, Xianyuan Zhan, Xiangang Li, Sen Song, and Yang Liu. 2023. Openchat: Advancing open-source language models with mixed-quality data. arXiv preprint arXiv:2309.11235 (2023).","journal-title":"arXiv preprint arXiv:2309.11235"},{"key":"e_1_3_2_226_2","doi-asserted-by":"crossref","unstructured":"Hanqing Wang Bowen Ping Shuo Wang Xu Han Yun Chen Zhiyuan Liu and Maosong Sun. 2024. LoRA-flow: Dynamic LoRA fusion for large language models in generative tasks. In ACL. 12871\u201312882.","DOI":"10.18653\/v1\/2024.acl-long.695"},{"key":"e_1_3_2_227_2","volume-title":"ICLR","author":"Wang Hongyi","year":"2020","unstructured":"Hongyi Wang, Mikhail Yurochkin, Yuekai Sun, Dimitris Papailiopoulos, and Yasaman Khazaeni. 2020. Federated learning with matched averaging. In ICLR. 1\u201316."},{"key":"e_1_3_2_228_2","first-page":"7611","article-title":"Tackling the objective inconsistency problem in heterogeneous federated optimization","volume":"33","author":"Wang Jianyu","year":"2020","unstructured":"Jianyu Wang, Qinghua Liu, Hao Liang, Gauri Joshi, and H. Vincent Poor. 2020. Tackling the objective inconsistency problem in heterogeneous federated optimization. NeurIPS 33 (2020), 7611\u20137623.","journal-title":"NeurIPS"},{"key":"e_1_3_2_229_2","volume-title":"ICLR","author":"Wang Ke","year":"2025","unstructured":"Ke Wang, Nikolaos Dimitriadis, Alessandro Favero, Guillermo Ortiz-Jimenez, Fran\u00e7ois Fleuret, and Pascal Frossard. 2025. LiNeS: Post-training layer scaling prevents forgetting and enhances model merging. In ICLR. 1\u201334."},{"key":"e_1_3_2_230_2","article-title":"Localizing task information for improved model merging and compression","author":"Wang Ke","year":"2024","unstructured":"Ke Wang, Nikolaos Dimitriadis, Guillermo Ortiz-Jimenez, Fran\u00e7ois Fleuret, and Pascal Frossard. 2024. Localizing task information for improved model merging and compression. ICML (2024), 50268\u201350287.","journal-title":"ICML"},{"key":"e_1_3_2_231_2","article-title":"Conditioned language policy: A general framework for steerable multi-objective finetuning","author":"Wang Kaiwen","year":"2024","unstructured":"Kaiwen Wang, Rahul Kidambi, Ryan Sullivan, Alekh Agarwal, Christoph Dann, Andrea Michi, Marco Gelmi, Yunxuan Li, Raghav Gupta, Avinava Dubey, et\u00a0al. 2024. Conditioned language policy: A general framework for steerable multi-objective finetuning. arXiv preprint arXiv:2407.15762 (2024).","journal-title":"arXiv preprint arXiv:2407.15762"},{"key":"e_1_3_2_232_2","volume-title":"ICML","author":"Wang Peng","year":"2024","unstructured":"Peng Wang, Li Shen, Zerui Tao, Shuaida He, and Dacheng Tao. 2024. Generalization analysis of stochastic weight averaging with general sampling. In ICML. 51442\u201351464."},{"key":"e_1_3_2_233_2","volume-title":"ICLR","author":"Wang Xiang","year":"2022","unstructured":"Xiang Wang, Annie N. Wang, Mo Zhou, and Rong Ge. 2022. Plateau in monotonic linear interpolation\u2014a\u201c Biased\u201d view of loss landscape for deep networks. In ICLR. 1\u201337."},{"key":"e_1_3_2_234_2","article-title":"Model merging scaling laws in large language models","author":"Wang Yuanyi","year":"2025","unstructured":"Yuanyi Wang, Yanggan Gu, Yiming Zhang, Qi Zhou, Zhaoyi Yan, Congkai Xie, Xinyao Wang, Jianbo Yuan, and Hongxia Yang. 2025. Model merging scaling laws in large language models. arXiv preprint arXiv:2509.24244 (2025).","journal-title":"arXiv preprint arXiv:2509.24244"},{"key":"e_1_3_2_235_2","article-title":"Why do more experts fail? a theoretical analysis of model merging","author":"Wang Zijing","year":"2025","unstructured":"Zijing Wang, Xingle Xu, Yongkang Liu, Yiqun Zhang, Peiqin Lin, Shi Feng, Xiaocui Yang, Daling Wang, and Hinrich Sch\u00fctze. 2025. Why do more experts fail? a theoretical analysis of model merging. arXiv preprint arXiv:2505.21226 (2025).","journal-title":"arXiv preprint arXiv:2505.21226"},{"key":"e_1_3_2_236_2","doi-asserted-by":"crossref","unstructured":"Zhenyi Wang Enneng Yang Li Shen and Heng Huang. 2025. A comprehensive survey of forgetting in deep learning beyond continual learning. TPAMI 47 3 (2025) 1464\u20131483.","DOI":"10.1109\/TPAMI.2024.3498346"},{"key":"e_1_3_2_237_2","volume-title":"ICML","author":"Wei Qi","year":"2025","unstructured":"Qi Wei, Shuo He, Enneng Yang, Tingcong Liu, Haobo Wang, Lei Feng, and Bo An. 2025. Representation surgery in model merging with probabilistic modeling. In ICML."},{"key":"e_1_3_2_238_2","article-title":"Unifying multimodal large language model capabilities and modalities via model merging","author":"Wei Yongxian","year":"2025","unstructured":"Yongxian Wei, Runxi Cheng, Weike Jin, Enneng Yang, Li Shen, Lu Hou, Sinan Du, Chun Yuan, Xiaochun Cao, and Dacheng Tao. 2025. Unifying multimodal large language model capabilities and modalities via model merging. arXiv preprint arXiv:2505.19892 (2025).","journal-title":"arXiv preprint arXiv:2505.19892"},{"key":"e_1_3_2_239_2","volume-title":"ICML","author":"Wei Yongxian","year":"2025","unstructured":"Yongxian Wei, Anke Tang, Li Shen, Zixuan Hu, Chun Yuan, and Xiaochun Cao. 2025. Modeling multi-task model merging as adaptive projective gradient descent. In ICML."},{"key":"e_1_3_2_240_2","article-title":"PyTorch Image Models","author":"Wightman Ross","year":"2019","unstructured":"Ross Wightman. 2019. PyTorch Image Models. Retrieved from https:\/\/github.com\/rwightman\/pytorch-image-models","journal-title":"https:\/\/github.com\/rwightman\/pytorch-image-models"},{"key":"e_1_3_2_241_2","article-title":"Huggingface\u2019s transformers: State-of-the-art natural language processing","author":"Wolf Thomas","year":"2019","unstructured":"Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R\u00e9mi Louf, Morgan Funtowicz, et\u00a0al. 2019. Huggingface\u2019s transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771 (2019).","journal-title":"arXiv preprint arXiv:1910.03771"},{"key":"e_1_3_2_242_2","first-page":"23965","volume-title":"ICML","author":"Wortsman Mitchell","year":"2022","unstructured":"Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S. Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, et\u00a0al. 2022. Model soups: Averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In ICML. PMLR, 23965\u201323998."},{"key":"e_1_3_2_243_2","first-page":"7959","volume-title":"CVPR","author":"Wortsman Mitchell","year":"2022","unstructured":"Mitchell Wortsman, Gabriel Ilharco, Jong Wook Kim, Mike Li, Simon Kornblith, Rebecca Roelofs, Raphael Gontijo Lopes, Hannaneh Hajishirzi, Ali Farhadi, Hongseok Namkoong, et\u00a0al. 2022. Robust fine-tuning of zero-shot models. In CVPR. 7959\u20137971."},{"key":"e_1_3_2_244_2","volume-title":"NeurIPS Datasets and Benchmarks Track","author":"Wu Baoyuan","year":"2022","unstructured":"Baoyuan Wu, Hongrui Chen, Mingda Zhang, Zihao Zhu, Shaokui Wei, Danni Yuan, and Chao Shen. 2022. BackdoorBench: A comprehensive benchmark of backdoor learning. In NeurIPS Datasets and Benchmarks Track 35 (2022), 10546\u201310559."},{"key":"e_1_3_2_245_2","first-page":"37713","volume-title":"ICML","author":"Wu Chengyue","year":"2023","unstructured":"Chengyue Wu, Teng Wang, Yixiao Ge, Zeyu Lu, Ruisong Zhou, Ying Shan, and Ping Luo. 2023. pi-tuning: Transferring multimodal foundation models with optimal multi-task interpolation. In ICML. PMLR, 37713\u201337727."},{"key":"e_1_3_2_246_2","article-title":"Unlocking efficient long-to-short llm reasoning with model merging","author":"Wu Han","year":"2025","unstructured":"Han Wu, Yuxuan Yao, Shuqi Liu, Zehua Liu, Xiaojin Fu, Xiongwei Han, Xing Li, Hui-Ling Zhen, Tao Zhong, and Mingxuan Yuan. 2025. Unlocking efficient long-to-short llm reasoning with model merging. arXiv preprint arXiv:2503.20641 (2025).","journal-title":"arXiv preprint arXiv:2503.20641"},{"key":"e_1_3_2_247_2","article-title":"Revisiting model interpolation for efficient reasoning","author":"Wu Taiqiang","year":"2025","unstructured":"Taiqiang Wu, Runming Yang, Tao Liu, Jiahao Wang, and Ngai Wong. 2025. Revisiting model interpolation for efficient reasoning. arXiv preprint arXiv:2510.10977 (2025).","journal-title":"arXiv preprint arXiv:2510.10977"},{"key":"e_1_3_2_248_2","volume-title":"ICLR","author":"Wu Xun","year":"2024","unstructured":"Xun Wu, Shaohan Huang, and Furu Wei. 2024. MoLE: Mixture of LoRA experts. In ICLR. 1\u201317."},{"key":"e_1_3_2_249_2","doi-asserted-by":"crossref","unstructured":"Shitao Xiao Zheng Liu Peitian Zhang and Xingrun Xing. 2024. Lm-Cocktail: Resilient tuning of language models via model merging. In Findings of the ACL 2024. 2474\u20132488.","DOI":"10.18653\/v1\/2024.findings-acl.145"},{"key":"e_1_3_2_250_2","article-title":"Multi-task model merging via adaptive weight disentanglement","author":"Xiong Feng","year":"2024","unstructured":"Feng Xiong, Runxi Cheng, Wang Chen, Zhanqiu Zhang, Yiwen Guo, Chun Yuan, and Ruifeng Xu. 2024. Multi-task model merging via adaptive weight disentanglement. arXiv preprint arXiv:2411.18729 (2024).","journal-title":"arXiv preprint arXiv:2411.18729"},{"key":"e_1_3_2_251_2","article-title":"Instructional fingerprinting of large language models","author":"Xu Jiashu","year":"2024","unstructured":"Jiashu Xu, Fei Wang, Mingyu Derek Ma, Pang Wei Koh, Chaowei Xiao, and Muhao Chen. 2024. Instructional fingerprinting of large language models. arXiv preprint arXiv:2401.12255 (2024).","journal-title":"arXiv preprint arXiv:2401.12255"},{"key":"e_1_3_2_252_2","doi-asserted-by":"crossref","unstructured":"Zhengqi Xu Ke Yuan Huiqiong Wang Yong Wang Mingli Song and Jie Song. 2024. Training-free pretrained model merging. In CVPR. 5915\u20135925.","DOI":"10.1109\/CVPR52733.2024.00565"},{"key":"e_1_3_2_253_2","unstructured":"Prateek Yadav Derek Tam Leshem Choshen Colin Raffel and Mohit Bansal. 2023. Ties-merging: Resolving interference when merging models. NeurIPS 36 (2023) 7093\u20137115."},{"key":"e_1_3_2_254_2","article-title":"What matters for model merging at scale?","author":"Yadav Prateek","year":"2024","unstructured":"Prateek Yadav, Tu Vu, Jonathan Lai, Alexandra Chronopoulou, Manaal Faruqui, Mohit Bansal, and Tsendsuren Munkhdalai. 2024. What matters for model merging at scale? arXiv preprint arXiv:2410.03617 (2024).","journal-title":"arXiv preprint arXiv:2410.03617"},{"key":"e_1_3_2_255_2","volume-title":"ICML","author":"Yan Kunda","year":"2025","unstructured":"Kunda Yan, Min Zhang, Sen Cui, Qu Zikun, Bo Jiang, Feng Liu, and Changshui Zhang. 2025. CALM: Consensus-aware localized merging for multi-task learning. In ICML."},{"key":"e_1_3_2_256_2","article-title":"Baichuan 2: Open large-scale language models","author":"Yang Aiyuan","year":"2023","unstructured":"Aiyuan Yang, Bin Xiao, Bingning Wang, Borong Zhang, Ce Bian, Chao Yin, Chenxu Lv, Da Pan, Dian Wang, Dong Yan, et\u00a0al. 2023. Baichuan 2: Open large-scale language models. arXiv preprint arXiv:2309.10305 (2023).","journal-title":"arXiv preprint arXiv:2309.10305"},{"key":"e_1_3_2_257_2","article-title":"Kwai keye-vl 1.5 technical report","author":"Yang Biao","year":"2025","unstructured":"Biao Yang, Bin Wen, Boyang Ding, Changyi Liu, Chenglong Chu, Chengru Song, Chongling Rao, Chuan Yi, Da Li, Dunju Zang, et\u00a0al. 2025. Kwai keye-vl 1.5 technical report. arXiv preprint arXiv:2509.01563 (2025).","journal-title":"arXiv preprint arXiv:2509.01563"},{"key":"e_1_3_2_258_2","first-page":"10745","volume-title":"AAAI","author":"Yang Enneng","year":"2023","unstructured":"Enneng Yang, Junwei Pan, Ximei Wang, Haibin Yu, Li Shen, Xihua Chen, Lei Xiao, Jie Jiang, and Guibing Guo. 2023. AdaTask: A task-aware adaptive learning rate approach to multi-task learning. In AAAI, Vol. 37. 10745\u201310753."},{"key":"e_1_3_2_259_2","unstructured":"Enneng Yang Li Shen Zhenyi Wang Guibing Guo Xiaojun Chen Xingwei Wang and Dacheng Tao. 2024. Representation surgery for multi-task model merging. In ICML. 56332\u201356356."},{"key":"e_1_3_2_260_2","article-title":"Surgeryv2: Bridging the gap between model merging and multi-task learning with deep representation surgery","author":"Yang Enneng","year":"2024","unstructured":"Enneng Yang, Li Shen, Zhenyi Wang, Guibing Guo, Xingwei Wang, Xiaocun Cao, Jie Zhang, and Dacheng Tao. 2024. Surgeryv2: Bridging the gap between model merging and multi-task learning with deep representation surgery. arXiv preprint arXiv:2410.14389 (2024).","journal-title":"arXiv preprint arXiv:2410.14389"},{"key":"e_1_3_2_261_2","article-title":"Continual model merging without data: Dual projections for balancing stability and plasticity","author":"Yang Enneng","year":"2025","unstructured":"Enneng Yang, Anke Tang, Li Shen, Guibing Guo, Xingwei Wang, Xiaochun Cao, and Jie Zhang. 2025. Continual model merging without data: Dual projections for balancing stability and plasticity. NeurIPS (2025).","journal-title":"NeurIPS"},{"key":"e_1_3_2_262_2","article-title":"AdaMerging: Adaptive model merging for multi-task learning","author":"Yang Enneng","year":"2024","unstructured":"Enneng Yang, Zhenyi Wang, Li Shen, Shiwei Liu, Guibing Guo, Xingwei Wang, and Dacheng Tao. 2024. AdaMerging: Adaptive model merging for multi-task learning. ICLR (2024), 1\u201321.","journal-title":"ICLR"},{"key":"e_1_3_2_263_2","first-page":"7015","volume-title":"ICML","author":"Yang Guandao","year":"2019","unstructured":"Guandao Yang, Tianyi Zhang, Polina Kirichenko, Junwen Bai, Andrew Gordon Wilson, and Chris De Sa. 2019. SWALP: Stochastic weight averaging in low precision training. In ICML. PMLR, 7015\u20137024."},{"key":"e_1_3_2_264_2","doi-asserted-by":"crossref","unstructured":"Yuanshun Yao Xiaojun Xu and Yang Liu. 2024. Large language model unlearning. NeurIPS 37 (2024) 105425\u2013105475.","DOI":"10.52202\/079017-3346"},{"key":"e_1_3_2_265_2","doi-asserted-by":"crossref","unstructured":"Peng Ye Chenyu Huang Mingzhu Shen Tao Chen Yongqi Huang and Wanli Ouyang. 2025. Dynamic model merging with mixture of weights. IEEE Trans. Circuits Syst. Video Technol. 35 8 (2025) 7925\u20137939.","DOI":"10.1109\/TCSVT.2025.3544680"},{"key":"e_1_3_2_266_2","article-title":"Merging vision transformers from different tasks and domains","author":"Ye Peng","year":"2023","unstructured":"Peng Ye, Chenyu Huang, Mingzhu Shen, Tao Chen, Yongqi Huang, Yuning Zhang, and Wanli Ouyang. 2023. Merging vision transformers from different tasks and domains. arXiv preprint arXiv:2312.16240 (2023).","journal-title":"arXiv preprint arXiv:2312.16240"},{"key":"e_1_3_2_267_2","first-page":"7163","volume-title":"EMNLP","author":"Ye Qinyuan","year":"2021","unstructured":"Qinyuan Ye, Bill Yuchen Lin, and Xiang Ren. 2021. CrossFit: A few-shot learning challenge for cross-task generalization in NLP. In EMNLP. 7163\u20137189."},{"key":"e_1_3_2_268_2","article-title":"A safety realignment framework via subspace-oriented model fusion for large language models","author":"Yi Xin","year":"2024","unstructured":"Xin Yi, Shunfan Zheng, Linlin Wang, Xiaoling Wang, and Liang He. 2024. A safety realignment framework via subspace-oriented model fusion for large language models. arXiv preprint arXiv:2405.09055 (2024).","journal-title":"arXiv preprint arXiv:2405.09055"},{"key":"e_1_3_2_269_2","article-title":"Extend model merging from fine-tuned to pre-trained large language models via weight disentanglement","author":"Yu Le","year":"2024","unstructured":"Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. 2024. Extend model merging from fine-tuned to pre-trained large language models via weight disentanglement. arXiv preprint arXiv:2408.03092 (2024).","journal-title":"arXiv preprint arXiv:2408.03092"},{"key":"e_1_3_2_270_2","article-title":"Language models are super mario: Absorbing abilities from homologous models as a free lunch","author":"Yu Le","year":"2024","unstructured":"Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. 2024. Language models are super mario: Absorbing abilities from homologous models as a free lunch. ICML (2024), 57755\u201357775.","journal-title":"ICML"},{"key":"e_1_3_2_271_2","first-page":"5824","article-title":"Gradient surgery for multi-task learning","volume":"33","author":"Yu Tianhe","year":"2020","unstructured":"Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. 2020. Gradient surgery for multi-task learning. NeurIPS 33 (2020), 5824\u20135836.","journal-title":"NeurIPS"},{"key":"e_1_3_2_272_2","first-page":"7252","volume-title":"ICML","author":"Yurochkin Mikhail","year":"2019","unstructured":"Mikhail Yurochkin, Mayank Agarwal, Soumya Ghosh, Kristjan Greenewald, Nghia Hoang, and Yasaman Khazaeni. 2019. Bayesian nonparametric federated learning of neural networks. In ICML. PMLR, 7252\u20137261."},{"key":"e_1_3_2_273_2","article-title":"Fuse to forget: Bias reduction and selective memorization through model fusion","author":"Zaman Kerem","year":"2023","unstructured":"Kerem Zaman, Leshem Choshen, and Shashank Srivastava. 2023. Fuse to forget: Bias reduction and selective memorization through model fusion. arXiv preprint arXiv:2311.07682 (2023).","journal-title":"arXiv preprint arXiv:2311.07682"},{"key":"e_1_3_2_274_2","volume-title":"ICML","author":"Zhang Binchi","year":"2025","unstructured":"Binchi Zhang, Zaiyi Zheng, Zhengzhang Chen, and Jundong Li. 2025. Beyond the permutation symmetry of transformers: The role of rotation for model fusion. In ICML."},{"key":"e_1_3_2_275_2","article-title":"Knowledge composition using task vectors with learned anisotropic scaling","author":"Zhang Frederic Z.","year":"2024","unstructured":"Frederic Z. Zhang, Paul Albert, Cristian Rodriguez-Opazo, Anton van den Hengel, and Ehsan Abbasnejad. 2024. Knowledge composition using task vectors with learned anisotropic scaling. arXiv preprint arXiv:2407.02880 (2024).","journal-title":"arXiv preprint arXiv:2407.02880"},{"key":"e_1_3_2_276_2","article-title":"Swa object detection","author":"Zhang Haoyang","year":"2020","unstructured":"Haoyang Zhang, Ying Wang, Feras Dayoub, and Niko S\u00fcnderhauf. 2020. Swa object detection. arXiv preprint arXiv:2012.12645 (2020).","journal-title":"arXiv preprint arXiv:2012.12645"},{"key":"e_1_3_2_277_2","article-title":"Unraveling LoRA interference: Orthogonal subspaces for robust model merging","author":"Zhang Haobo","year":"2025","unstructured":"Haobo Zhang and Jiayu Zhou. 2025. Unraveling LoRA interference: Orthogonal subspaces for robust model merging. arXiv preprint arXiv:2505.22934 (2025).","journal-title":"arXiv preprint arXiv:2505.22934"},{"key":"e_1_3_2_278_2","volume-title":"NeurIPS","author":"Zhang Jinghan","year":"2023","unstructured":"Jinghan Zhang, Shiqi Chen, Junteng Liu, and Junxian He. 2023. Composing parameter-efficient modules with arithmetic operations. In NeurIPS 36 (2023), 12589\u201312610."},{"key":"e_1_3_2_279_2","doi-asserted-by":"crossref","unstructured":"Jinghuai Zhang Jianfeng Chi Zheng Li Kunlin Cai Yang Zhang and Yuan Tian. 2024. Badmerging: Backdoor attacks against model merging. In CCS. 4450\u20134464.","DOI":"10.1145\/3658644.3690284"},{"key":"e_1_3_2_280_2","article-title":"Lookaround optimizer: k steps around, 1 step average","volume":"36","author":"Zhang Jiangtao","year":"2023","unstructured":"Jiangtao Zhang, Shunyu Liu, Jie Song, Tongtian Zhu, Zhengqi Xu, and Mingli Song. 2023. Lookaround optimizer: k steps around, 1 step average. NeurIPS 36 (2023), 28678\u201328698.","journal-title":"NeurIPS"},{"key":"e_1_3_2_281_2","volume-title":"ICML","author":"Zhao Bo","year":"2025","unstructured":"Bo Zhao, Nima Dehmamy, Robin Walters, and Rose Yu. 2025. Understanding mode connectivity via parameter space symmetry. In ICML."},{"key":"e_1_3_2_282_2","first-page":"13349","article-title":"Model-GLUE: Democratized LLM scaling for a large model zoo in the wild","volume":"37","author":"Zhao Xinyu","year":"2024","unstructured":"Xinyu Zhao, Guoheng Sun, Ruisi Cai, Yukun Zhou, Pingzhi Li, Peihao Wang, Bowen Tan, Yexiao He, Li Chen, Yi Liang, et\u00a0al. 2024. Model-GLUE: Democratized LLM scaling for a large model zoo in the wild. NeurIPS 37 (2024), 13349\u201313371.","journal-title":"NeurIPS"},{"key":"e_1_3_2_283_2","article-title":"AdaMergeX: Cross-lingual transfer with large language models via adaptive adapter merging","author":"Zhao Yiran","year":"2024","unstructured":"Yiran Zhao, Wenxuan Zhang, Huiming Wang, Kenji Kawaguchi, and Lidong Bing. 2024. AdaMergeX: Cross-lingual transfer with large language models via adaptive adapter merging. arXiv preprint arXiv:2402.18913 (2024).","journal-title":"arXiv preprint arXiv:2402.18913"},{"key":"e_1_3_2_284_2","doi-asserted-by":"crossref","unstructured":"Ziyu Zhao Leilei Gan Guoyin Wang Wangchunshu Zhou Hongxia Yang Kun Kuang and Fei Wu. 2024. LoraRetriever: Input-aware LoRA retrieval and composition for mixed tasks in the wild. In Findings of the ACL. 4447\u20134462.","DOI":"10.18653\/v1\/2024.findings-acl.263"},{"key":"e_1_3_2_285_2","article-title":"Weak-to-strong extrapolation expedites alignment","author":"Zheng Chujie","year":"2024","unstructured":"Chujie Zheng, Ziqi Wang, Heng Ji, Minlie Huang, and Nanyun Peng. 2024. Weak-to-strong extrapolation expedites alignment. arXiv preprint arXiv:2404.16792 (2024).","journal-title":"arXiv preprint arXiv:2404.16792"},{"key":"e_1_3_2_286_2","article-title":"Learn from model beyond fine-tuning: A survey","author":"Zheng Hongling","year":"2023","unstructured":"Hongling Zheng, Li Shen, Anke Tang, Yong Luo, Han Hu, Bo Du, and Dacheng Tao. 2023. Learn from model beyond fine-tuning: A survey. arXiv preprint arXiv:2310.08184 (2023).","journal-title":"arXiv preprint arXiv:2310.08184"},{"key":"e_1_3_2_287_2","article-title":"On task vectors and gradients","author":"Zhou Luca","year":"2025","unstructured":"Luca Zhou, Daniele Solombrino, Donato Crisostomi, Maria Sofia Bucarelli, Giuseppe Alessio D\u2019Inverno, Fabrizio Silvestri, and Emanuele Rodol\u00e0. 2025. On task vectors and gradients. arXiv preprint arXiv:2508.16082 (2025).","journal-title":"arXiv preprint arXiv:2508.16082"},{"key":"e_1_3_2_288_2","article-title":"MetaGPT: Merging large language models using model exclusive task arithmetic","author":"Zhou Yuyan","year":"2024","unstructured":"Yuyan Zhou, Liang Song, Bingning Wang, and Weipeng Chen. 2024. MetaGPT: Merging large language models using model exclusive task arithmetic. arXiv preprint arXiv:2406.11385 (2024).","journal-title":"arXiv preprint arXiv:2406.11385"},{"key":"e_1_3_2_289_2","article-title":"Cross-task linearity emerges in the pretraining-finetuning paradigm","author":"Zhou Zhanpeng","year":"2024","unstructured":"Zhanpeng Zhou, Zijun Chen, Yilan Chen, Bo Zhang, and Junchi Yan. 2024. Cross-task linearity emerges in the pretraining-finetuning paradigm. arXiv preprint arXiv:2402.03660v1 (2024).","journal-title":"arXiv preprint arXiv:2402.03660v1"},{"key":"e_1_3_2_290_2","first-page":"60853","article-title":"Going beyond linear mode connectivity: The layerwise linear feature connectivity","volume":"36","author":"Zhou Zhanpeng","year":"2023","unstructured":"Zhanpeng Zhou, Yongyi Yang, Xiaojiang Yang, Junchi Yan, and Wei Hu. 2023. Going beyond linear mode connectivity: The layerwise linear feature connectivity. NeurIPS 36 (2023), 60853\u201360877.","journal-title":"NeurIPS"},{"key":"e_1_3_2_291_2","unstructured":"Didi Zhu Zhongyi Sun Zexi Li Tao Shen Ke Yan Shouhong Ding Kun Kuang and Chao Wu. 2024. Model tailor: Mitigating catastrophic forgetting in multi-modal large language models. In ICML. PMLR 62581\u201362598."},{"key":"e_1_3_2_292_2","article-title":"Fine-tuning language models from human preferences","author":"Ziegler Daniel M.","year":"2019","unstructured":"Daniel M. Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B. Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. 2019. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593 (2019).","journal-title":"arXiv preprint arXiv:1909.08593"},{"key":"e_1_3_2_293_2","volume-title":"ICLR","author":"Zimmer Max","year":"2024","unstructured":"Max Zimmer, Christoph Spiegel, and Sebastian Pokutta. 2024. Sparse model soups: A recipe for improved pruning via model averaging. In ICLR. 1\u201331."}],"container-title":["ACM Computing Surveys"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3787849","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,9]],"date-time":"2026-02-09T13:20:50Z","timestamp":1770643250000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3787849"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,9]]},"references-count":292,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2026,6,30]]}},"alternative-id":["10.1145\/3787849"],"URL":"https:\/\/doi.org\/10.1145\/3787849","relation":{},"ISSN":["0360-0300","1557-7341"],"issn-type":[{"value":"0360-0300","type":"print"},{"value":"1557-7341","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,9]]},"assertion":[{"value":"2024-09-06","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-12-26","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-02-09","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}