{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,22]],"date-time":"2025-10-22T23:37:14Z","timestamp":1761176234895,"version":"build-2065373602"},"reference-count":0,"publisher":"IOS Press","isbn-type":[{"value":"9781643686318","type":"electronic"}],"license":[{"start":{"date-parts":[[2025,10,21]],"date-time":"2025-10-21T00:00:00Z","timestamp":1761004800000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,10,21]]},"abstract":"<jats:p>Different from existing federated fine-tuning (FFT) methods for foundation models, hybrid heterogeneous federated fine-tuning (HHFFT) is an under-explored scenario where clients exhibit double heterogeneity in model architectures and downstream tasks. This hybrid heterogeneity introduces two significant challenges: 1) heterogeneous matrix aggregation, where clients adopt different large-scale foundation models based on their task requirements and resource limitations, leading to dimensional mismatches during LoRA parameter aggregation; and 2) multi-task knowledge interference, where local shared parameters, trained with both task-shared and task-specific knowledge, cannot ensure only task-shared knowledge is transferred between clients. To address these challenges, we propose H2Tune, a federated foundation model fine-tuning with hybrid heterogeneity. Our framework H2Tune consists of three key components: (i) sparsified triple matrix decomposition to align hidden dimensions across clients through constructing rank-consistent middle matrices, with adaptive sparsification based on client resources; (ii) relation-guided matrix layer alignment to handle heterogeneous layer structures and representation capabilities; and (iii) alternating task-knowledge disentanglement mechanism to decouple shared and specific knowledge of local model parameters through alternating optimization. Theoretical analysis proves a convergence rate of O(1\/\u221aT). Extensive experiments show our method achieves up to 15.4% accuracy improvement compared to state-of-the-art baselines.<\/jats:p>","DOI":"10.3233\/faia251183","type":"book-chapter","created":{"date-parts":[[2025,10,22]],"date-time":"2025-10-22T09:53:55Z","timestamp":1761126835000},"source":"Crossref","is-referenced-by-count":0,"title":["H2Tune: Federated Foundation Model Fine-Tuning with Hybrid Heterogeneity"],"prefix":"10.3233","author":[{"given":"Wei","family":"Guo","sequence":"first","affiliation":[{"name":"School of Artificial Intelligence, Beihang University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Siyuan","family":"Lu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Heilongjiang University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yiqi","family":"Tong","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Beihang University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhaojun","family":"Hu","sequence":"additional","affiliation":[{"name":"School of Statistics, Renmin University of China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fuzhen","family":"Zhuang","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence, Beihang University"},{"name":"Zhongguancun Laboratory, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiao","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Shandong University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tao","family":"Fan","sequence":"additional","affiliation":[{"name":"WeBank Co., Ltd, Shenzhen, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jin","family":"Dong","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence, Beihang University"},{"name":"Beijing Academy of Blockchain and Edge Computing"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"7437","container-title":["Frontiers in Artificial Intelligence and Applications","ECAI 2025"],"original-title":[],"link":[{"URL":"https:\/\/ebooks.iospress.nl\/pdf\/doi\/10.3233\/FAIA251183","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,22]],"date-time":"2025-10-22T09:53:55Z","timestamp":1761126835000},"score":1,"resource":{"primary":{"URL":"https:\/\/ebooks.iospress.nl\/doi\/10.3233\/FAIA251183"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,21]]},"ISBN":["9781643686318"],"references-count":0,"URL":"https:\/\/doi.org\/10.3233\/faia251183","relation":{},"ISSN":["0922-6389","1879-8314"],"issn-type":[{"value":"0922-6389","type":"print"},{"value":"1879-8314","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,21]]}}}