{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,22]],"date-time":"2025-10-22T23:35:33Z","timestamp":1761176133522,"version":"build-2065373602"},"reference-count":0,"publisher":"IOS Press","isbn-type":[{"value":"9781643686318","type":"electronic"}],"license":[{"start":{"date-parts":[[2025,10,21]],"date-time":"2025-10-21T00:00:00Z","timestamp":1761004800000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,10,21]]},"abstract":"<jats:p>Parameter-efficient transfer learning (PETL) has emerged as a promising solution to adapt large-scale pre-trained models to downstream tasks. Nevertheless, these methods have not thoroughly explored the characteristics of PETL methods to optimize the fine-tuning performance with miminal volume of parameters. In this paper, we first reveal that, compared to pre-trained models, PETL tends to generate similar features via homogeneous feature transformations across different layers. Subsequently, we propose a Global Share Local Transform Mixture-of-Experts framework, namely GLEAM, that decomposes the adapter into a shared component and layer-specific local components to simultaneously reduce the redundancy in layer-wise parameter matrices for homogeneous feature transformations and fine-tune the locally specific parameters for minimizing performance loss. Specifically, we develop a shared mixture of convolution that introduces shared multi-scale sparse MoE to enable diverse transformations for suppressing the homogeneity issue of feature transformations in PETL. GLEAM is evaluated on more than 20 datasets for image classification and few-shot learning. Extensive experimental results demonstrate that it achieves comparable performance with existing PETL methods like LoRA with only 3% of its parameters and further yields competitive performance using only 0.07M parameters.<\/jats:p>","DOI":"10.3233\/faia250864","type":"book-chapter","created":{"date-parts":[[2025,10,22]],"date-time":"2025-10-22T09:44:24Z","timestamp":1761126264000},"source":"Crossref","is-referenced-by-count":0,"title":["GLEAM: Parameter-Efficient Transfer Learning via Global Share Local Transform Mixture-of-Experts"],"prefix":"10.3233","author":[{"given":"Jiarui","family":"Zhang","sequence":"first","affiliation":[{"name":"Department of Computer Science and Engineering, Shanghai Jiao Tong University, jr.zhang@sjtu.edu.cn, daiwenrui@sjtu.edu.cn, zoujunni@sjtu.edu.cn"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yue","family":"Xin","sequence":"additional","affiliation":[{"name":"Department of Electronic Engineering, Shanghai Jiao Tong University, xinyuexiong@sjtu.edu.cn, zhengziyang@sjtu.edu.cn, lcl1985@sjtu.edu.cn, xionghongkai@sjtu.edu.cn"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yaoming","family":"Wang","sequence":"additional","affiliation":[{"name":"Meituan Inc, wangyaoming03@meituan.com"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wenrui","family":"Dai","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, Shanghai Jiao Tong University, jr.zhang@sjtu.edu.cn, daiwenrui@sjtu.edu.cn, zoujunni@sjtu.edu.cn"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ziyang","family":"Zheng","sequence":"additional","affiliation":[{"name":"Department of Electronic Engineering, Shanghai Jiao Tong University, xinyuexiong@sjtu.edu.cn, zhengziyang@sjtu.edu.cn, lcl1985@sjtu.edu.cn, xionghongkai@sjtu.edu.cn"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chenglin","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Electronic Engineering, Shanghai Jiao Tong University, xinyuexiong@sjtu.edu.cn, zhengziyang@sjtu.edu.cn, lcl1985@sjtu.edu.cn, xionghongkai@sjtu.edu.cn"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Junni","family":"Zou","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, Shanghai Jiao Tong University, jr.zhang@sjtu.edu.cn, daiwenrui@sjtu.edu.cn, zoujunni@sjtu.edu.cn"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hongkai","family":"Xiong","sequence":"additional","affiliation":[{"name":"Department of Electronic Engineering, Shanghai Jiao Tong University, xinyuexiong@sjtu.edu.cn, zhengziyang@sjtu.edu.cn, lcl1985@sjtu.edu.cn, xionghongkai@sjtu.edu.cn"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"7437","container-title":["Frontiers in Artificial Intelligence and Applications","ECAI 2025"],"original-title":[],"link":[{"URL":"https:\/\/ebooks.iospress.nl\/pdf\/doi\/10.3233\/FAIA250864","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,22]],"date-time":"2025-10-22T09:44:25Z","timestamp":1761126265000},"score":1,"resource":{"primary":{"URL":"https:\/\/ebooks.iospress.nl\/doi\/10.3233\/FAIA250864"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,21]]},"ISBN":["9781643686318"],"references-count":0,"URL":"https:\/\/doi.org\/10.3233\/faia250864","relation":{},"ISSN":["0922-6389","1879-8314"],"issn-type":[{"value":"0922-6389","type":"print"},{"value":"1879-8314","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,21]]}}}