{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T17:04:10Z","timestamp":1764695050696,"version":"3.46.0"},"publisher-location":"New York, NY, USA","reference-count":32,"publisher":"ACM","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2025,11,4]]},"DOI":"10.1145\/3737899.3768528","type":"proceedings-article","created":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T17:01:43Z","timestamp":1764694903000},"page":"85-92","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["FedHO: Memory-Efficient Federated Fine-Tuning for Large Models via Hybrid Gradient Computation"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0004-5943-5647","authenticated-orcid":false,"given":"Yinan","family":"Zhang","sequence":"first","affiliation":[{"name":"The Hong Kong Polytechnic University Hong Kong, Hong Kong"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2725-2529","authenticated-orcid":false,"given":"Jiannong","family":"Cao","sequence":"additional","affiliation":[{"name":"The Hong Kong Polytechnic University Hong Kong, Hong Kong"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1653-108X","authenticated-orcid":false,"given":"Mingjin","family":"Zhang","sequence":"additional","affiliation":[{"name":"The Hong Kong Polytechnic University Hong Kong, Hong Kong"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9483-4000","authenticated-orcid":false,"given":"Ruosong","family":"Yang","sequence":"additional","affiliation":[{"name":"China Mobile (Hong Kong) Innovation and Research Institute Hong Kong, Hong Kong"}]}],"member":"320","published-online":{"date-parts":[[2025,12,2]]},"reference":[{"key":"e_1_3_2_1_1_1","first-page":"54496","article-title":"Fine-grained theoretical analysis of federated zeroth-order optimization","volume":"36","author":"Chen Jun","year":"2023","unstructured":"Jun Chen, Hong Chen, Bin Gu, and Hao Deng. 2023. Fine-grained theoretical analysis of federated zeroth-order optimization. Advances in Neural Information Processing Systems 36 (2023), 54496--54508.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_1_2_1","volume-title":"Free Dolly: Introducing the World's First Truly Open Instruction-Tuned LLM. https:\/\/www.databricks.com\/blog\/2023\/04\/12\/dolly-first-open-commercially-viable-instruction-tuned-llm","author":"Conover Mike","year":"2023","unstructured":"Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. 2023. Free Dolly: Introducing the World's First Truly Open Instruction-Tuned LLM. https:\/\/www.databricks.com\/blog\/2023\/04\/12\/dolly-first-open-commercially-viable-instruction-tuned-llm"},{"key":"e_1_3_2_1_3_1","volume-title":"Qlora: Efficient finetuning of quantized llms. Advances in neural information processing systems 36","author":"Dettmers Tim","year":"2023","unstructured":"Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. Qlora: Efficient finetuning of quantized llms. Advances in neural information processing systems 36 (2023), 10088--10115."},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"crossref","first-page":"5058","DOI":"10.1109\/TSP.2022.3214122","article-title":"Communication-efficient stochastic zerothorder optimization for federated learning","volume":"70","author":"Fang Wenzhi","year":"2022","unstructured":"Wenzhi Fang, Ziyi Yu, Yuning Jiang, Yuanming Shi, Colin N Jones, and Yong Zhou. 2022. Communication-efficient stochastic zerothorder optimization for federated learning. IEEE Transactions on Signal Processing 70 (2022), 5058--5073.","journal-title":"IEEE Transactions on Signal Processing"},{"key":"e_1_3_2_1_5_1","volume-title":"Meta-learning for low-resource neural machine translation. arXiv preprint arXiv:1808.08437","author":"Gu Jiatao","year":"2018","unstructured":"Jiatao Gu, Yong Wang, Yun Chen, Kyunghyun Cho, and Victor OK Li. 2018. Meta-learning for low-resource neural machine translation. arXiv preprint arXiv:1808.08437 (2018)."},{"key":"e_1_3_2_1_6_1","volume-title":"Parameter-efficient fine-tuning for large models: A comprehensive survey. arXiv preprint arXiv:2403.14608","author":"Han Zeyu","year":"2024","unstructured":"Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. 2024. Parameter-efficient fine-tuning for large models: A comprehensive survey. arXiv preprint arXiv:2403.14608 (2024)."},{"key":"e_1_3_2_1_7_1","volume-title":"International Conference on Machine Learning. PMLR, 4387--4398","author":"Hsieh Kevin","year":"2020","unstructured":"Kevin Hsieh, Amar Phanishayee, Onur Mutlu, and Phillip Gibbons. 2020. The non-iid data quagmire of decentralized machine learning. In International Conference on Machine Learning. PMLR, 4387--4398."},{"key":"e_1_3_2_1_8_1","volume-title":"Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685","author":"Hu Edward J","year":"2021","unstructured":"Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 (2021)."},{"key":"e_1_3_2_1_9_1","volume-title":"Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al.","author":"Kairouz Peter","year":"2021","unstructured":"Peter Kairouz, H Brendan McMahan, Brendan Avent, Aur\u00e9lien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. 2021. Advances and open problems in federated learning. Foundations and trends\u00ae in machine learning 14, 1--2 (2021), 1--210."},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3637528.3671573"},{"key":"e_1_3_2_1_11_1","volume-title":"Proceedings of Machine learning and systems 2","author":"Li Tian","year":"2020","unstructured":"Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. 2020. Federated optimization in heterogeneous networks. Proceedings of Machine learning and systems 2 (2020), 429--450."},{"key":"e_1_3_2_1_12_1","volume-title":"Super tickets in pre-trained language models: From model compression to improving generalization. arXiv preprint arXiv:2105.12002","author":"Liang Chen","year":"2021","unstructured":"Chen Liang, Simiao Zuo, Minshuo Chen, Haoming Jiang, Xiaodong Liu, Pengcheng He, Tuo Zhao, and Weizhu Chen. 2021. Super tickets in pre-trained language models: From model compression to improving generalization. arXiv preprint arXiv:2105.12002 (2021)."},{"key":"e_1_3_2_1_13_1","volume-title":"Rouge: A package for automatic evaluation of summaries. In Text summarization branches out. 74-81.","author":"Lin Chin-Yew","year":"2004","unstructured":"Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out. 74-81."},{"key":"e_1_3_2_1_14_1","volume-title":"Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 1827-1838","author":"Ling Zhenqing","year":"2024","unstructured":"Zhenqing Ling, Daoyuan Chen, Liuyi Yao, Yaliang Li, and Ying Shen. 2024. On the convergence of zeroth-order federated tuning for large language models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 1827-1838."},{"key":"e_1_3_2_1_15_1","first-page":"53038","article-title":"Fine-tuning language models with just forward passes","volume":"36","author":"Malladi Sadhika","year":"2023","unstructured":"Sadhika Malladi, Tianyu Gao, Eshaan Nichani, Alex Damian, Jason D Lee, Danqi Chen, and Sanjeev Arora. 2023. Fine-tuning language models with just forward passes. Advances in Neural Information Processing Systems 36 (2023), 53038--53075.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_1_16_1","unstructured":"Brendan McMahan Eider Moore Daniel Ramage Seth Hampson and Blaise Aguera y Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics. PMLR 1273--1282."},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.01152"},{"key":"e_1_3_2_1_18_1","volume-title":"Federated full-parameter tuning of billion-sized language models with communication cost under 18 kilobytes. arXiv preprint arXiv:2312.06353","author":"Qin Zhen","year":"2023","unstructured":"Zhen Qin, Daoyuan Chen, Bingchen Qian, Bolin Ding, Yaliang Li, and Shuiguang Deng. 2023. Federated full-parameter tuning of billion-sized language models with communication cost under 18 kilobytes. arXiv preprint arXiv:2312.06353 (2023)."},{"key":"e_1_3_2_1_19_1","volume-title":"Movement pruning: Adaptive sparsity by fine-tuning. Advances in neural information processing systems 33","author":"Sanh Victor","year":"2020","unstructured":"Victor Sanh, Thomas Wolf, and Alexander Rush. 2020. Movement pruning: Adaptive sparsity by fine-tuning. Advances in neural information processing systems 33 (2020), 20378--20389."},{"key":"e_1_3_2_1_20_1","volume-title":"Federated zeroth-order optimization using trajectory-informed surrogate gradients. arXiv preprint arXiv:2308.04077","author":"Shu Yao","year":"2023","unstructured":"Yao Shu, Xiaoqiang Lin, Zhongxiang Dai, and Bryan Kian Hsiang Low. 2023. Federated zeroth-order optimization using trajectory-informed surrogate gradients. arXiv preprint arXiv:2308.04077 (2023)."},{"key":"e_1_3_2_1_21_1","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 7782-7790","author":"Soni Sunny","year":"2024","unstructured":"Sunny Soni, Aaqib Saeed, and Yuki M Asano. 2024. Federated Learning with a Single Shared Image. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 7782-7790."},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2022.3160699"},{"key":"e_1_3_2_1_23_1","volume-title":"Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971","author":"Touvron Hugo","year":"2023","unstructured":"Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth\u00e9e Lacroix, Baptiste Rozi\u00e8re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023)."},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3703631"},{"key":"e_1_3_2_1_25_1","volume-title":"Atharva Naik, David Stap, et al.","author":"Wang Yizhong","year":"2022","unstructured":"Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al. 2022. Super-naturalinstructions: Generalization via declarative instructions on 1600+ nlp tasks. arXiv preprint arXiv:2204.07705 (2022)."},{"key":"e_1_3_2_1_26_1","first-page":"579","volume-title":"2024 USENIX Annual Technical Conference (USENIX ATC 24)","author":"Xu Mengwei","year":"2024","unstructured":"Mengwei Xu, Dongqi Cai, Yaozong Wu, Xiang Li, and Shangguang Wang. 2024. {FwdLLM}: Efficient Federated Finetuning of Large Language Models with Perturbed Inferences. In 2024 USENIX Annual Technical Conference (USENIX ATC 24). 579-596."},{"key":"e_1_3_2_1_27_1","volume-title":"Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 6137-6147","author":"Ye Rui","year":"2024","unstructured":"Rui Ye, Wenhao Wang, Jingyi Chai, Dihan Li, Zexi Li, Yinda Xu, Yaxin Du, Yanfeng Wang, and Siheng Chen. 2024. Openfedllm: Training large language models on decentralized private data via federated learning. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 6137-6147."},{"key":"e_1_3_2_1_28_1","volume-title":"2022 IEEE International Conference on Service-Oriented System Engineering (SOSE). IEEE, 165--175","author":"Zhang Mingjin","year":"2022","unstructured":"Mingjin Zhang, Jiannong Cao, Yuvraj Sahni, Qianyi Chen, Shan Jiang, and Tao Wu. 2022. Eaas: A service-oriented edge computing framework towards distributed intelligence. In 2022 IEEE International Conference on Service-Oriented System Engineering (SOSE). IEEE, 165--175."},{"key":"e_1_3_2_1_29_1","volume-title":"Resource-efficient parallel split learning in heterogeneous edge computing. arXiv preprint arXiv:2403.15815","author":"Zhang Mingjin","year":"2024","unstructured":"Mingjin Zhang, Jiannong Cao, Yuvraj Sahni, Xiangchun Chen, and Shan Jiang. 2024. Resource-efficient parallel split learning in heterogeneous edge computing. arXiv preprint arXiv:2403.15815 (2024)."},{"key":"e_1_3_2_1_30_1","volume-title":"International conference on machine learning. PMLR, 26809--26823","author":"Zhang Qingru","year":"2022","unstructured":"Qingru Zhang, Simiao Zuo, Chen Liang, Alexander Bukharin, Pengcheng He, Weizhu Chen, and Tuo Zhao. 2022. Platon: Pruning large transformer models with upper confidence bound of weight importance. In International conference on machine learning. PMLR, 26809--26823."},{"key":"e_1_3_2_1_31_1","unstructured":"Yihua Zhang Pingzhi Li Junyuan Hong Jiaxiang Li Yimeng Zhang Wenqing Zheng Pin-Yu Chen Jason D Lee Wotao Yin Mingyi Hong et al. 2024. Revisiting zeroth-order optimization for memory-efficient llm fine-tuning: A benchmark. arXiv preprint arXiv:2402.11592 (2024)."},{"key":"e_1_3_2_1_32_1","volume-title":"When foundation model meets federated learning: Motivations, challenges, and future directions. arXiv preprint arXiv:2306.15546","author":"Zhuang Weiming","year":"2023","unstructured":"Weiming Zhuang, Chen Chen, and Lingjuan Lyu. 2023. When foundation model meets federated learning: Motivations, challenges, and future directions. arXiv preprint arXiv:2306.15546 (2023)."}],"event":{"name":"ACM MobiCom '25: The 31st Annual International Conference on Mobile Computing and Networking","location":"Hong Kong China","acronym":"FLEdge-AI '25","sponsor":["SIGMOBILE ACM Special Interest Group on Mobility of Systems, Users, Data and Computing"]},"container-title":["Proceedings of the Federated Learning and Edge AI for Privacy and Mobility"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3737899.3768528","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T17:01:52Z","timestamp":1764694912000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3737899.3768528"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,4]]},"references-count":32,"alternative-id":["10.1145\/3737899.3768528","10.1145\/3737899"],"URL":"https:\/\/doi.org\/10.1145\/3737899.3768528","relation":{},"subject":[],"published":{"date-parts":[[2025,11,4]]},"assertion":[{"value":"2025-12-02","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}