{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T15:07:27Z","timestamp":1764688047445,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":27,"publisher":"ACM","license":[{"start":{"date-parts":[[2023,5,8]],"date-time":"2023-05-08T00:00:00Z","timestamp":1683504000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"King Abdullah University of Science and Technology (KAUST)","award":["ORA-CRG10-2021-4699"],"award-info":[{"award-number":["ORA-CRG10-2021-4699"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,5,8]]},"DOI":"10.1145\/3578356.3592590","type":"proceedings-article","created":{"date-parts":[[2023,5,4]],"date-time":"2023-05-04T19:44:37Z","timestamp":1683229477000},"page":"123-130","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["A First Look at the Impact of Distillation Hyper-Parameters in Federated Knowledge Distillation"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7095-3726","authenticated-orcid":false,"given":"Norah","family":"Alballa","sequence":"first","affiliation":[{"name":"KAUST, Thuwal, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5051-4283","authenticated-orcid":false,"given":"Marco","family":"Canini","sequence":"additional","affiliation":[{"name":"KAUST, Thuwal, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,5,8]]},"reference":[{"key":"e_1_3_2_1_1_1","unstructured":"Rohan Anil Gabriel Pereyra Alexandre Passos Robert Ormandi George E Dahl and Geoffrey E Hinton. 2018. Large scale distributed neural network training through online distillation. arXiv:1804.03235 [cs.LG]  Rohan Anil Gabriel Pereyra Alexandre Passos Robert Ormandi George E Dahl and Geoffrey E Hinton. 2018. Large scale distributed neural network training through online distillation. arXiv:1804.03235 [cs.LG]"},{"key":"e_1_3_2_1_2_1","volume-title":"Titouan Parcollet, Pedro Porto Buarque de Gusm\u00e3o, and Nicholas D. Lane.","author":"Beutel Daniel J.","year":"2022","unstructured":"Daniel J. Beutel , Taner Topal , Akhil Mathur , Xinchi Qiu , Javier Fernandez-Marques , Yan Gao , Lorenzo Sani , Kwing Hei Li , Titouan Parcollet, Pedro Porto Buarque de Gusm\u00e3o, and Nicholas D. Lane. 2022 . Flower : A Friendly Federated Learning Research Framework . arXiv:2007.14390 [cs.LG] Daniel J. Beutel, Taner Topal, Akhil Mathur, Xinchi Qiu, Javier Fernandez-Marques, Yan Gao, Lorenzo Sani, Kwing Hei Li, Titouan Parcollet, Pedro Porto Buarque de Gusm\u00e3o, and Nicholas D. Lane. 2022. Flower: A Friendly Federated Learning Research Framework. arXiv:2007.14390 [cs.LG]"},{"key":"e_1_3_2_1_3_1","unstructured":"Ilai Bistritz Ariana Mann and Nicholas Bambos. 2020. Distributed distillation for on-device learning. In NeurIPS.  Ilai Bistritz Ariana Mann and Nicholas Bambos. 2020. Distributed distillation for on-device learning. In NeurIPS."},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"crossref","unstructured":"Cristian Bucilu\u01ce Rich Caruana and Alexandru Niculescu-Mizil. 2006. Model compression. In KDD.  Cristian Bucilu\u01ce Rich Caruana and Alexandru Niculescu-Mizil. 2006. Model compression. In KDD.","DOI":"10.1145\/1150402.1150464"},{"key":"e_1_3_2_1_5_1","volume-title":"Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfer. arXiv:1912.11279 [stat.ML]","author":"Chang Hongyan","year":"2019","unstructured":"Hongyan Chang , Virat Shejwalkar , Reza Shokri , and Amir Houmansadr . 2019 . Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfer. arXiv:1912.11279 [stat.ML] Hongyan Chang, Virat Shejwalkar, Reza Shokri, and Amir Houmansadr. 2019. Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfer. arXiv:1912.11279 [stat.ML]"},{"key":"e_1_3_2_1_6_1","unstructured":"Hong-You Chen and Wei-Lun Chao. 2021. FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning. arXiv:2009.01974 [cs.LG]  Hong-You Chen and Wei-Lun Chao. 2021. FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning. arXiv:2009.01974 [cs.LG]"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"crossref","unstructured":"Xuan Gong Abhishek Sharma Srikrishna Karanam Ziyan Wu Terrence Chen David Doermann and Arun Innanje. 2021. Ensemble attention distillation for privacy-preserving federated learning. In ICCV.  Xuan Gong Abhishek Sharma Srikrishna Karanam Ziyan Wu Terrence Chen David Doermann and Arun Innanje. 2021. Ensemble attention distillation for privacy-preserving federated learning. In ICCV.","DOI":"10.1109\/ICCV48922.2021.01480"},{"key":"e_1_3_2_1_8_1","article-title":"Knowledge distillation: A survey","volume":"129","author":"Gou Jianping","year":"2021","unstructured":"Jianping Gou , Baosheng Yu , Stephen J Maybank , and Dacheng Tao . 2021 . Knowledge distillation: A survey . International Journal of Computer Vision 129 , 6 (2021). Jianping Gou, Baosheng Yu, Stephen J Maybank, and Dacheng Tao. 2021. Knowledge distillation: A survey. International Journal of Computer Vision 129, 6 (2021).","journal-title":"International Journal of Computer Vision"},{"key":"e_1_3_2_1_9_1","unstructured":"Neel Guha Ameet Talwalkar and Virginia Smith. 2019. One-Shot Federated Learning. arXiv:1902.11175 [cs.LG]  Neel Guha Ameet Talwalkar and Virginia Smith. 2019. One-Shot Federated Learning. arXiv:1902.11175 [cs.LG]"},{"key":"e_1_3_2_1_10_1","unstructured":"Geoffrey Hinton Oriol Vinyals and Jeff Dean. 2015. Distilling the Knowledge in a Neural Network. arXiv:1503.02531 [stat.ML]  Geoffrey Hinton Oriol Vinyals and Jeff Dean. 2015. Distilling the Knowledge in a Neural Network. arXiv:1503.02531 [stat.ML]"},{"key":"e_1_3_2_1_11_1","volume-title":"MHAT: An efficient model-heterogenous aggregation training scheme for federated learning. Information Sciences 560","author":"Hu Li","year":"2021","unstructured":"Li Hu , Hongyang Yan , Lang Li , Zijie Pan , Xiaozhang Liu , and Zulong Zhang . 2021 . MHAT: An efficient model-heterogenous aggregation training scheme for federated learning. Information Sciences 560 (2021). Li Hu, Hongyang Yan, Lang Li, Zijie Pan, Xiaozhang Liu, and Zulong Zhang. 2021. MHAT: An efficient model-heterogenous aggregation training scheme for federated learning. Information Sciences 560 (2021)."},{"key":"e_1_3_2_1_12_1","article-title":"Distillation-based semi-supervised federated learning for communication-efficient collaborative training with non-iid private data","volume":"22","author":"Itahara Sohei","year":"2021","unstructured":"Sohei Itahara , Takayuki Nishio , Yusuke Koda , Masahiro Morikura , and Koji Yamamoto . 2021 . Distillation-based semi-supervised federated learning for communication-efficient collaborative training with non-iid private data . IEEE Transactions on Mobile Computing 22 , 1 (2021). Sohei Itahara, Takayuki Nishio, Yusuke Koda, Masahiro Morikura, and Koji Yamamoto. 2021. Distillation-based semi-supervised federated learning for communication-efficient collaborative training with non-iid private data. IEEE Transactions on Mobile Computing 22, 1 (2021).","journal-title":"IEEE Transactions on Mobile Computing"},{"key":"e_1_3_2_1_13_1","volume-title":"Tizhoosh","author":"Kalra Shivam","year":"2021","unstructured":"Shivam Kalra , Junfeng Wen , Jesse C. Cresswell , Maksims Volkovs , and Hamid R . Tizhoosh . 2021 . ProxyFL: Decentralized Federated Learning through Proxy Model Sharing . arXiv:2111.11343 [cs.LG] Shivam Kalra, Junfeng Wen, Jesse C. Cresswell, Maksims Volkovs, and Hamid R. Tizhoosh. 2021. ProxyFL: Decentralized Federated Learning through Proxy Model Sharing. arXiv:2111.11343 [cs.LG]"},{"key":"e_1_3_2_1_14_1","unstructured":"Alex Krizhevsky Geoffrey Hinton etal 2009. Learning multiple layers of features from tiny images. Technical Report. University of Toronto.  Alex Krizhevsky Geoffrey Hinton et al. 2009. Learning multiple layers of features from tiny images. Technical Report. University of Toronto."},{"key":"e_1_3_2_1_15_1","volume-title":"Fedscale: Benchmarking model and system performance of federated learning at scale. In ICML.","author":"Lai Fan","year":"2022","unstructured":"Fan Lai , Yinwei Dai , Sanjay Singapuram , Jiachen Liu , Xiangfeng Zhu , Harsha Madhyastha , and Mosharaf Chowdhury . 2022 . Fedscale: Benchmarking model and system performance of federated learning at scale. In ICML. Fan Lai, Yinwei Dai, Sanjay Singapuram, Jiachen Liu, Xiangfeng Zhu, Harsha Madhyastha, and Mosharaf Chowdhury. 2022. Fedscale: Benchmarking model and system performance of federated learning at scale. In ICML."},{"key":"e_1_3_2_1_16_1","article-title":"Decentralized federated learning via mutual knowledge transfer","volume":"9","author":"Li Chengxi","year":"2021","unstructured":"Chengxi Li , Gang Li , and Pramod K Varshney . 2021 . Decentralized federated learning via mutual knowledge transfer . IEEE Internet of Things Journal 9 , 2 (2021). Chengxi Li, Gang Li, and Pramod K Varshney. 2021. Decentralized federated learning via mutual knowledge transfer. IEEE Internet of Things Journal 9, 2 (2021).","journal-title":"IEEE Internet of Things Journal"},{"key":"e_1_3_2_1_17_1","unstructured":"Daliang Li and Junpu Wang. 2019. FedMD: Heterogenous Federated Learning via Model Distillation. arXiv:1910.03581 [cs.LG]  Daliang Li and Junpu Wang. 2019. FedMD: Heterogenous Federated Learning via Model Distillation. arXiv:1910.03581 [cs.LG]"},{"key":"e_1_3_2_1_18_1","unstructured":"Tao Lin Lingjing Kong Sebastian U. Stich and Martin Jaggi. 2020. Ensemble distillation for robust model fusion in federated learning. In NeurIPS.  Tao Lin Lingjing Kong Sebastian U. Stich and Martin Jaggi. 2020. Ensemble distillation for robust model fusion in federated learning. In NeurIPS."},{"key":"e_1_3_2_1_19_1","unstructured":"Mi Luo Fei Chen Dapeng Hu Yifan Zhang Jian Liang and Jiashi Feng. 2021. No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data. In NeurIPS.  Mi Luo Fei Chen Dapeng Hu Yifan Zhang Jian Liang and Jiashi Feng. 2021. No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data. In NeurIPS."},{"key":"e_1_3_2_1_20_1","unstructured":"Michael Matena and Colin Raffel. 2022. Merging Models with Fisher-Weighted Averaging. arXiv:2111.09832 [cs.LG]  Michael Matena and Colin Raffel. 2022. Merging Models with Fisher-Weighted Averaging. arXiv:2111.09832 [cs.LG]"},{"key":"e_1_3_2_1_21_1","volume-title":"Fedaux: Leveraging unlabeled auxiliary data in federated learning","author":"Sattler Felix","year":"2021","unstructured":"Felix Sattler , Tim Korjakow , Roman Rischke , and Wojciech Samek . 2021 . Fedaux: Leveraging unlabeled auxiliary data in federated learning . IEEE Transactions on Neural Networks and Learning Systems ( 2021). Felix Sattler, Tim Korjakow, Roman Rischke, and Wojciech Samek. 2021. Fedaux: Leveraging unlabeled auxiliary data in federated learning. IEEE Transactions on Neural Networks and Learning Systems (2021)."},{"key":"e_1_3_2_1_22_1","article-title":"Cfd: Communication-efficient federated distillation via soft-label quantization and delta coding","volume":"9","author":"Sattler Felix","year":"2021","unstructured":"Felix Sattler , Arturo Marban , Roman Rischke , and Wojciech Samek . 2021 . Cfd: Communication-efficient federated distillation via soft-label quantization and delta coding . IEEE Transactions on Network Science and Engineering 9 , 4 (2021). Felix Sattler, Arturo Marban, Roman Rischke, and Wojciech Samek. 2021. Cfd: Communication-efficient federated distillation via soft-label quantization and delta coding. IEEE Transactions on Network Science and Engineering 9, 4 (2021).","journal-title":"IEEE Transactions on Network Science and Engineering"},{"key":"e_1_3_2_1_23_1","unstructured":"J\u00fcrgen Schmidhuber. 1991. Neural sequence chunkers. Inst. f\u00fcr Informatik.  J\u00fcrgen Schmidhuber. 1991. Neural sequence chunkers. Inst. f\u00fcr Informatik."},{"key":"e_1_3_2_1_24_1","unstructured":"Samuel Stanton Pavel Izmailov Polina Kirichenko Alexander A Alemi and Andrew G Wilson. 2021. Does knowledge distillation really work?. In NeurIPS.  Samuel Stanton Pavel Izmailov Polina Kirichenko Alexander A Alemi and Andrew G Wilson. 2021. Does knowledge distillation really work?. In NeurIPS."},{"key":"e_1_3_2_1_25_1","volume-title":"Fedproto: Federated prototype learning across heterogeneous clients. In AAAI.","author":"Tan Yue","year":"2022","unstructured":"Yue Tan , Guodong Long , Lu Liu , Tianyi Zhou , Qinghua Lu , Jing Jiang , and Chengqi Zhang . 2022 . Fedproto: Federated prototype learning across heterogeneous clients. In AAAI. Yue Tan, Guodong Long, Lu Liu, Tianyi Zhou, Qinghua Lu, Jing Jiang, and Chengqi Zhang. 2022. Fedproto: Federated prototype learning across heterogeneous clients. In AAAI."},{"key":"e_1_3_2_1_26_1","article-title":"Self-distillation: Towards efficient and compact neural networks","volume":"44","author":"Zhang Linfeng","year":"2022","unstructured":"Linfeng Zhang , Chenglong Bao , and Kaisheng Ma . 2022 . Self-distillation: Towards efficient and compact neural networks . IEEE Transactions on Pattern Analysis and Machine Intelligence 44 , 8 (2022). Linfeng Zhang, Chenglong Bao, and Kaisheng Ma. 2022. Self-distillation: Towards efficient and compact neural networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 8 (2022).","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"crossref","unstructured":"Ying Zhang Tao Xiang Timothy M Hospedales and Huchuan Lu. 2018. Deep mutual learning. In CVPR.  Ying Zhang Tao Xiang Timothy M Hospedales and Huchuan Lu. 2018. Deep mutual learning. In CVPR.","DOI":"10.1109\/CVPR.2018.00454"}],"event":{"name":"EuroMLSys '23: 3rd Workshop on Machine Learning and Systems","sponsor":["SIGOPS ACM Special Interest Group on Operating Systems"],"location":"Rome Italy","acronym":"EuroMLSys '23"},"container-title":["Proceedings of the 3rd Workshop on Machine Learning and Systems"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3578356.3592590","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:46:51Z","timestamp":1750178811000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3578356.3592590"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,8]]},"references-count":27,"alternative-id":["10.1145\/3578356.3592590","10.1145\/3578356"],"URL":"https:\/\/doi.org\/10.1145\/3578356.3592590","relation":{},"subject":[],"published":{"date-parts":[[2023,5,8]]},"assertion":[{"value":"2023-05-08","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}