{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,28]],"date-time":"2026-03-28T17:54:25Z","timestamp":1774720465167,"version":"3.50.1"},"reference-count":32,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2022,8,4]],"date-time":"2022-08-04T00:00:00Z","timestamp":1659571200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62104125"],"award-info":[{"award-number":["62104125"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100021171","name":"Guangdong Basic and Applied Basic Research Foundation","doi-asserted-by":"crossref","award":["2020A1515110887"],"award-info":[{"award-number":["2020A1515110887"]}],"id":[{"id":"10.13039\/501100021171","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Institute for Guo Qiang of Tsinghua University","award":["2020GQG1004"],"award-info":[{"award-number":["2020GQG1004"]}]},{"DOI":"10.13039\/100018913","name":"Tsinghua Shenzhen International Graduate School","doi-asserted-by":"crossref","award":["HW202 and QD2021013C"],"award-info":[{"award-number":["HW202 and QD2021013C"]}],"id":[{"id":"10.13039\/100018913","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Hong Kong RGC","award":["ECS 21212419"],"award-info":[{"award-number":["ECS 21212419"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2022,8,31]]},"abstract":"<jats:p>Federated learning (FL) has attracted tremendous attentions in recent years due to its privacy-preserving measures and great potential in some distributed but privacy-sensitive applications, such as finance and health. However, high communication overloads for transmitting high-dimensional networks and extra security masks remain a bottleneck of FL. This article proposes a communication-efficient FL framework with an Adaptive Quantized Gradient (AQG), which adaptively adjusts the quantization level based on a local gradient\u2019s update to fully utilize the heterogeneity of local data distribution for reducing unnecessary transmissions. In addition, client dropout issues are taken into account and an Augmented AQG is developed, which could limit the dropout noise with an appropriate amplification mechanism for transmitted gradients. Theoretical analysis and experiment results show that the proposed AQG leads to 18% to 50% of additional transmission reduction as compared with existing popular methods, including Quantized Gradient Descent (QGD) and Lazily Aggregated Quantized (LAQ) gradient-based methods without deteriorating convergence properties. Experiments with heterogenous data distributions corroborate a more significant transmission reduction compared with independent identical data distributions. The proposed AQG is robust to a client dropping rate up to 90% empirically, and the Augmented AQG manages to further improve the FL system\u2019s communication efficiency with the presence of moderate-scale client dropouts commonly seen in practical FL scenarios.<\/jats:p>","DOI":"10.1145\/3510587","type":"journal-article","created":{"date-parts":[[2022,3,23]],"date-time":"2022-03-23T14:43:53Z","timestamp":1648046633000},"page":"1-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":78,"title":["Communication-Efficient Federated Learning with Adaptive Quantization"],"prefix":"10.1145","volume":"13","author":[{"given":"Yuzhu","family":"Mao","sequence":"first","affiliation":[{"name":"Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, Guangdong Province, China and Tsinghua-Berkeley Shenzhen Institute (TBSI), Tsinghua University, Shenzhen, Guangdong Province, China"}]},{"given":"Zihao","family":"Zhao","sequence":"additional","affiliation":[{"name":"Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, Guangdong Province, China and Tsinghua-Berkeley Shenzhen Institute (TBSI), Tsinghua University, Shenzhen, Guangdong Province, China"}]},{"given":"Guangfeng","family":"Yan","sequence":"additional","affiliation":[{"name":"City University of Hong Kong, Kowloon, Hong Kong, China and City University of Hong Kong Shenzhen Research Institute, Shenzhen, Guangdong Province, China"}]},{"given":"Yang","family":"Liu","sequence":"additional","affiliation":[{"name":"Institute for AI Industry Research, Tsinghua University, Beijing, China"}]},{"given":"Tian","family":"Lan","sequence":"additional","affiliation":[{"name":"George Washington University, Washington, DC, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2756-4984","authenticated-orcid":false,"given":"Linqi","family":"Song","sequence":"additional","affiliation":[{"name":"City University of Hong Kong, Kowloon, Hong Kong, China and City University of Hong Kong Shenzhen Research Institute, Shenzhen, Guangdong Province, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0597-4512","authenticated-orcid":false,"given":"Wenbo","family":"Ding","sequence":"additional","affiliation":[{"name":"Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, Guangdong Province, China and Tsinghua-Berkeley Shenzhen Institute (TBSI), Tsinghua University, Shenzhen, Guangdong Province, China"}]}],"member":"320","published-online":{"date-parts":[[2022,8,4]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/2976749.2978318"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D17-1045"},{"key":"e_1_3_2_4_2","volume-title":"Proceedings of the 31st International Conference on Neural Information Processing Systems","volume":"30","author":"Alistarh Dan","year":"2017","unstructured":"Dan Alistarh, Demjan Grubic, Jerry Li, Ryota Tomioka, and Milan Vojnovic. 2017. QSGD: Communication-efficient SGD via gradient quantization and encoding. In Proceedings of the 31st International Conference on Neural Information Processing Systems, Vol. 30. Curran Associates, Inc., Long Beach, CA, USA."},{"key":"e_1_3_2_5_2","first-page":"560","volume-title":"Proceedings of the 35th International Conference on Machine Learning","volume":"80","author":"Bernstein Jeremy","year":"2018","unstructured":"Jeremy Bernstein, Yu Xiang Wang, Kamyar Azizzadenesheli, and Animashree Anandkumar. 2018. signSGD: Compressed optimisation for non-convex problems. In Proceedings of the 35th International Conference on Machine Learning, Vol. 80. PMLR, Stockholmsm\u00e4ssan, Stockholm, Sweden, 560\u2013569."},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3133956.3133982"},{"key":"e_1_3_2_7_2","doi-asserted-by":"crossref","unstructured":"Nader Bouacida Jiahui Hou Hui Zang and Xin Liu. 2020. Adaptive federated dropout: Improving communication efficiency and generalization for federated learning. arXiv preprint arXiv:2011.04050.","DOI":"10.1109\/INFOCOMWKSHPS51825.2021.9484526"},{"key":"e_1_3_2_8_2","unstructured":"Tianyi Chen Georgios B. Giannakis Tao Sun and Wotao Yin. 2018. LAG: Lazily aggregated gradient for communication-efficient distributed learning. arXiv preprint arXiv:1805.09965."},{"key":"e_1_3_2_9_2","unstructured":"Kewei Cheng Tao Fan Yilun Jin Yang Liu Tianjian Chen and Qiang Yang. 2019. Secureboost: A lossless federated learning framework. arXiv preprint arXiv:1901.08755."},{"key":"e_1_3_2_10_2","unstructured":"Enmao Diao Jie Ding and Vahid Tarokh. 2020. HeteroFL: Computation and communication efficient federated learning for heterogeneous clients. arXiv preprint arXiv:2010.01264."},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/OJCS.2020.2992630"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0933-3657(98)00028-1"},{"key":"e_1_3_2_13_2","unstructured":"Andrew Hard Kanishka Rao Rajiv Mathews Swaroop Ramaswamy Fran\u00e7oise Beaufays Sean Augenstein Hubert Eichner Chlo\u00e9 Kiddon and Daniel Ramage. 2019. Federated learning for mobile keyboard prediction. arXiv preprint arXiv:1811.03604."},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/GLOBECOM42002.2020.9322155"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP39728.2021.9413697"},{"key":"e_1_3_2_16_2","article-title":"Lazily aggregated quantized gradient innovation for communication-efficient federated learning","author":"Jun Sun","year":"2022","unstructured":"Sun Jun, Chen Tianyi, Georgios B. Giannakis, Yang Qinmin, and Yang Zaiyue. 2022. Lazily aggregated quantized gradient innovation for communication-efficient federated learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 4 (2022), 2031\u20132044.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-020-0186-1"},{"key":"e_1_3_2_18_2","unstructured":"Ron Kohavi. 1996. Scaling up the accuracy of naive-Bayes classifiers: A decision-tree hybrid. Proceedings of the Second International Conference on Knowledge Discovery and Data Mining 96 (1996) 202\u2013207."},{"key":"e_1_3_2_19_2","first-page":"429","volume-title":"Proceedings of Machine Learning and Systems","volume":"2","author":"Li Tian","year":"2020","unstructured":"Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. 2020. Federated optimization in heterogeneous networks. In Proceedings of Machine Learning and Systems, Vol. 2. 429\u2013450."},{"key":"e_1_3_2_20_2","volume-title":"Proceedings of International Conference on Learning Representations","author":"Lin Yujun","year":"2018","unstructured":"Yujun Lin, Song Han, Huizi Mao, Yu Wang, and William J. Dally. 2018. Deep gradient compression: Reducing the communication bandwidth for distributed training. In Proceedings of International Conference on Learning Representations, Vancouver, BC, Canada."},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i08.7021"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/MIS.2020.2988525"},{"key":"e_1_3_2_23_2","first-page":"1273","volume-title":"Proceedings of the 20th International Conference on Artificial Intelligence and Statistics","volume":"54","author":"McMahan Brendan","year":"2017","unstructured":"Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, Vol. 54. PMLR, Fort Lauderdale, FL, 1273\u20131282."},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICC.2019.8761315"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2019.2944481"},{"key":"e_1_3_2_26_2","doi-asserted-by":"crossref","unstructured":"Frank Seide Hao Fu Jasha Droppo Gang Li and Dong Yu. 2014. 1-Bit stochastic gradient descent and its application to data-parallel distributed training of speech DNNs. In Fifteenth Annual Conference of the International Speech Communication Association . Citeseer Singapore 1058\u20131062.","DOI":"10.21437\/Interspeech.2014-274"},{"issue":"3","key":"e_1_3_2_27_2","first-page":"262","article-title":"Classification of radar returns from the ionosphere using neural networks","volume":"10","author":"Sigillito Vincent G.","year":"1989","unstructured":"Vincent G. Sigillito, Simon P. Wing, Larrie V. Hutton, and Kile B. Baker. 1989. Classification of radar returns from the ionosphere using neural networks. Johns Hopkins APL Technical Digest 10, 3 (1989), 262\u2013266.","journal-title":"Johns Hopkins APL Technical Digest"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/CDC42340.2020.9303828"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.5555\/3327345.3327357"},{"key":"e_1_3_2_30_2","first-page":"9872","volume-title":"Proceedings of the 32nd International Conference on Neural Information Processing Systems.","author":"Wang Hongyi","year":"2018","unstructured":"Hongyi Wang, Scott Sievert, Zachary Charles, Shengchao Liu, Stephen Wright, and Dimitris Papailiopoulos. 2018. ATOMO: Communication-efficient learning via atomic sparsification. In Proceedings of the 32nd International Conference on Neural Information Processing Systems. Montr\u00e9al, Quebec, Canada, Curran Associates Inc., Red Hook, 9872\u20139883."},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.5555\/3326943.3327063"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/3298981"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/INFOCOM41043.2020.9155418"}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3510587","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3510587","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:12:20Z","timestamp":1750191140000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3510587"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,4]]},"references-count":32,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2022,8,31]]}},"alternative-id":["10.1145\/3510587"],"URL":"https:\/\/doi.org\/10.1145\/3510587","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"value":"2157-6904","type":"print"},{"value":"2157-6912","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,8,4]]},"assertion":[{"value":"2021-04-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-01-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-08-04","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}