{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,27]],"date-time":"2026-04-27T08:52:59Z","timestamp":1777279979440,"version":"3.51.4"},"publisher-location":"New York, NY, USA","reference-count":23,"publisher":"ACM","license":[{"start":{"date-parts":[[2022,12,13]],"date-time":"2022-12-13T00:00:00Z","timestamp":1670889600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"NSFC","award":["61936011"],"award-info":[{"award-number":["61936011"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,12,13]]},"DOI":"10.1145\/3551626.3564955","type":"proceedings-article","created":{"date-parts":[[2022,12,7]],"date-time":"2022-12-07T00:55:45Z","timestamp":1670374545000},"page":"1-7","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Federated Knowledge Transfer for Heterogeneous Visual Models"],"prefix":"10.1145","author":[{"given":"Wenzhe","family":"Li","sequence":"first","affiliation":[{"name":"Tsinghua University, Shenzhen, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zirui","family":"Zhu","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tianchi","family":"Huang","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lifeng","family":"Sun","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chun","family":"Yuan","sequence":"additional","affiliation":[{"name":"Tsinghua University, Shenzhen, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,12,13]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01007"},{"key":"e_1_3_2_1_2_1","first-page":"22593","article-title":"Distributed distillation for on-device learning","volume":"33","author":"Bistritz Ilai","year":"2020","unstructured":"Ilai Bistritz , Ariana Mann , and Nicholas Bambos . 2020 . Distributed distillation for on-device learning . Advances in Neural Information Processing Systems 33 (2020), 22593 -- 22604 . Ilai Bistritz, Ariana Mann, and Nicholas Bambos. 2020. Distributed distillation for on-device learning. Advances in Neural Information Processing Systems 33 (2020), 22593--22604.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"e_1_3_2_1_4_1","unstructured":"Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai Thomas Unterthiner Mostafa Dehghani Matthias Minderer Georg Heigold Sylvain Gelly etal 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020).  Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai Thomas Unterthiner Mostafa Dehghani Matthias Minderer Georg Heigold Sylvain Gelly et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)."},{"key":"e_1_3_2_1_5_1","first-page":"14068","article-title":"Group knowledge transfer: Federated learning of large cnns at the edge","volume":"33","author":"He Chaoyang","year":"2020","unstructured":"Chaoyang He , Murali Annavaram , and Salman Avestimehr . 2020 . Group knowledge transfer: Federated learning of large cnns at the edge . Advances in Neural Information Processing Systems 33 (2020), 14068 -- 14080 . Chaoyang He, Murali Annavaram, and Salman Avestimehr. 2020. Group knowledge transfer: Federated learning of large cnns at the edge. Advances in Neural Information Processing Systems 33 (2020), 14068--14080.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_2_1_7_1","unstructured":"Geoffrey Hinton Oriol Vinyals Jeff Dean etal 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 2 7 (2015).  Geoffrey Hinton Oriol Vinyals Jeff Dean et al. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 2 7 (2015)."},{"key":"e_1_3_2_1_8_1","volume-title":"Measuring the effects of non-identical data distribution for federated visual classification. arXiv preprint arXiv:1909.06335","author":"Harry Hsu Tzu-Ming","year":"2019","unstructured":"Tzu-Ming Harry Hsu , Hang Qi , and Matthew Brown . 2019. Measuring the effects of non-identical data distribution for federated visual classification. arXiv preprint arXiv:1909.06335 ( 2019 ). Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown. 2019. Measuring the effects of non-identical data distribution for federated visual classification. arXiv preprint arXiv:1909.06335 (2019)."},{"key":"e_1_3_2_1_9_1","volume-title":"Distillation-based semi-supervised federated learning for communication-efficient collaborative training with non-iid private data. arXiv preprint arXiv:2008.06180","author":"Itahara Sohei","year":"2020","unstructured":"Sohei Itahara , Takayuki Nishio , Yusuke Koda , Masahiro Morikura , and Koji Yamamoto . 2020. Distillation-based semi-supervised federated learning for communication-efficient collaborative training with non-iid private data. arXiv preprint arXiv:2008.06180 ( 2020 ). Sohei Itahara, Takayuki Nishio, Yusuke Koda, Masahiro Morikura, and Koji Yamamoto. 2020. Distillation-based semi-supervised federated learning for communication-efficient collaborative training with non-iid private data. arXiv preprint arXiv:2008.06180 (2020)."},{"key":"e_1_3_2_1_10_1","volume-title":"Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al.","author":"Kairouz Peter","year":"2021","unstructured":"Peter Kairouz , H Brendan McMahan , Brendan Avent , Aur\u00e9lien Bellet , Mehdi Bennis , Arjun Nitin Bhagoji , Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. 2021 . Advances and open problems in federated learning. Foundations and Trends\u00ae in Machine Learning 14, 1--2 (2021), 1--210. Peter Kairouz, H Brendan McMahan, Brendan Avent, Aur\u00e9lien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. 2021. Advances and open problems in federated learning. Foundations and Trends\u00ae in Machine Learning 14, 1--2 (2021), 1--210."},{"key":"e_1_3_2_1_11_1","unstructured":"Alex Krizhevsky Geoffrey Hinton etal 2009. Learning multiple layers of features from tiny images. (2009).  Alex Krizhevsky Geoffrey Hinton et al. 2009. Learning multiple layers of features from tiny images. (2009)."},{"key":"e_1_3_2_1_12_1","volume-title":"On information and sufficiency. The annals of mathematical statistics 22, 1","author":"Kullback Solomon","year":"1951","unstructured":"Solomon Kullback and Richard A Leibler . 1951. On information and sufficiency. The annals of mathematical statistics 22, 1 ( 1951 ), 79--86. Solomon Kullback and Richard A Leibler. 1951. On information and sufficiency. The annals of mathematical statistics 22, 1 (1951), 79--86."},{"key":"e_1_3_2_1_13_1","volume-title":"Fedmd: Heterogenous federated learning via model distillation. arXiv preprint arXiv:1910.03581","author":"Li Daliang","year":"2019","unstructured":"Daliang Li and Junpu Wang . 2019 . Fedmd: Heterogenous federated learning via model distillation. arXiv preprint arXiv:1910.03581 (2019). Daliang Li and Junpu Wang. 2019. Fedmd: Heterogenous federated learning via model distillation. arXiv preprint arXiv:1910.03581 (2019)."},{"key":"e_1_3_2_1_14_1","first-page":"429","article-title":"Federated optimization in heterogeneous networks","volume":"2","author":"Li Tian","year":"2020","unstructured":"Tian Li , Anit Kumar Sahu , Manzil Zaheer , Maziar Sanjabi , Ameet Talwalkar , and Virginia Smith . 2020 . Federated optimization in heterogeneous networks . Proceedings of Machine Learning and Systems 2 (2020), 429 -- 450 . Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. 2020. Federated optimization in heterogeneous networks. Proceedings of Machine Learning and Systems 2 (2020), 429--450.","journal-title":"Proceedings of Machine Learning and Systems"},{"key":"e_1_3_2_1_15_1","volume-title":"Kumar Kshitij Patel, and Martin Jaggi","author":"Lin Tao","year":"2018","unstructured":"Tao Lin , Sebastian U Stich , Kumar Kshitij Patel, and Martin Jaggi . 2018 . Don 't use large mini-batches, use local sgd. arXiv preprint arXiv:1808.07217 (2018). Tao Lin, Sebastian U Stich, Kumar Kshitij Patel, and Martin Jaggi. 2018. Don't use large mini-batches, use local sgd. arXiv preprint arXiv:1808.07217 (2018)."},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.425"},{"key":"e_1_3_2_1_17_1","unstructured":"Brendan McMahan Eider Moore Daniel Ramage Seth Hampson and Blaise Aguera y Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics. PMLR 1273--1282.  Brendan McMahan Eider Moore Daniel Ramage Seth Hampson and Blaise Aguera y Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics. PMLR 1273--1282."},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00241"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v36i2.20103"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.3390\/electronics11040670"},{"key":"e_1_3_2_1_21_1","volume-title":"Attention is all you need. Advances in neural information processing systems 30","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani , Noam Shazeer , Niki Parmar , Jakob Uszkoreit , Llion Jones , Aidan N Gomez , \u0141ukasz Kaiser , and Illia Polosukhin . 2017. Attention is all you need. Advances in neural information processing systems 30 ( 2017 ). Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017)."},{"key":"e_1_3_2_1_22_1","volume-title":"Federated learning with matched averaging. arXiv preprint arXiv:2002.06440","author":"Wang Hongyi","year":"2020","unstructured":"Hongyi Wang , Mikhail Yurochkin , Yuekai Sun , Dimitris Papailiopoulos , and Yasaman Khazaeni . 2020. Federated learning with matched averaging. arXiv preprint arXiv:2002.06440 ( 2020 ). Hongyi Wang, Mikhail Yurochkin, Yuekai Sun, Dimitris Papailiopoulos, and Yasaman Khazaeni. 2020. Federated learning with matched averaging. arXiv preprint arXiv:2002.06440 (2020)."},{"key":"e_1_3_2_1_23_1","volume-title":"Distilled one-shot federated learning. arXiv preprint arXiv:2009.07999","author":"Zhou Yanlin","year":"2020","unstructured":"Yanlin Zhou , George Pu , Xiyao Ma , Xiaolin Li , and Dapeng Wu. 2020. Distilled one-shot federated learning. arXiv preprint arXiv:2009.07999 ( 2020 ). Yanlin Zhou, George Pu, Xiyao Ma, Xiaolin Li, and Dapeng Wu. 2020. Distilled one-shot federated learning. arXiv preprint arXiv:2009.07999 (2020)."}],"event":{"name":"MMAsia '22: ACM Multimedia Asia","location":"Tokyo Japan","acronym":"MMAsia '22","sponsor":["SIGMM ACM Special Interest Group on Multimedia"]},"container-title":["Proceedings of the 4th ACM International Conference on Multimedia in Asia"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3551626.3564955","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3551626.3564955","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:00:26Z","timestamp":1750186826000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3551626.3564955"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,13]]},"references-count":23,"alternative-id":["10.1145\/3551626.3564955","10.1145\/3551626"],"URL":"https:\/\/doi.org\/10.1145\/3551626.3564955","relation":{},"subject":[],"published":{"date-parts":[[2022,12,13]]},"assertion":[{"value":"2022-12-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}