{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:21:32Z","timestamp":1750220492550,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":28,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,10,15]],"date-time":"2021-10-15T00:00:00Z","timestamp":1634256000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"the Fundamental Research Funds for the Central Universities","award":["YD2150002001"],"award-info":[{"award-number":["YD2150002001"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,10,15]]},"DOI":"10.1145\/3497623.3497669","type":"proceedings-article","created":{"date-parts":[[2022,2,5]],"date-time":"2022-02-05T00:30:14Z","timestamp":1644021014000},"page":"283-289","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Rethinking Logits-Level Knowledge Distillation"],"prefix":"10.1145","author":[{"given":"Teng","family":"Gao","sequence":"first","affiliation":[{"name":"University of Science and Technology of China, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hong","family":"An","sequence":"additional","affiliation":[{"name":"University of Science and Technology of China, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,2,4]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"Do deep nets really need to be deep? arXiv preprint arXiv:1312.6184","author":"Ba Lei Jimmy","year":"2013","unstructured":"Lei Jimmy Ba and Rich Caruana . Do deep nets really need to be deep? arXiv preprint arXiv:1312.6184 , 2013 . Lei Jimmy Ba and Rich Caruana. Do deep nets really need to be deep? arXiv preprint arXiv:1312.6184, 2013."},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1150402.1150464"},{"key":"e_1_3_2_1_3_1","first-page":"255","volume-title":"2009 IEEE conference on computer vision and pattern recognition","author":"Deng Jia","unstructured":"Jia Deng , Wei Dong , Richard Socher , Li-Jia Li , Kai Li , and Li Fei-Fei . Imagenet : A large-scale hierarchical image database . In 2009 IEEE conference on computer vision and pattern recognition , pages 248\u2013 255 . Ieee, 2009. Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248\u2013255. Ieee, 2009."},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.668"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00201"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33013779"},{"key":"e_1_3_2_1_8_1","volume-title":"Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531","author":"Hinton Geoffrey","year":"2015","unstructured":"Geoffrey Hinton , Oriol Vinyals , and Jeff Dean . Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 , 2015 . Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015."},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.5555\/3327144.3327200"},{"key":"e_1_3_2_1_10_1","volume-title":"Learning multiple layers of features from tiny images. Handbook of Systemic Autoimmune Diseases, 1(4)","author":"Krizhevsky A.","year":"2009","unstructured":"A. Krizhevsky and G. Hinton . Learning multiple layers of features from tiny images. Handbook of Systemic Autoimmune Diseases, 1(4) , 2009 . A. Krizhevsky and G. Hinton. Learning multiple layers of features from tiny images. Handbook of Systemic Autoimmune Diseases, 1(4), 2009."},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.5555\/2999134.2999257"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01264-9_8"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.5555\/3454287.3454709"},{"key":"e_1_3_2_1_14_1","volume-title":"Pervasive label errors in test sets destabilize machine learning benchmarks. arXiv preprint arXiv:2103.14749","author":"Northcutt Curtis G","year":"2021","unstructured":"Curtis G Northcutt , Anish Athalye , and Jonas Mueller . Pervasive label errors in test sets destabilize machine learning benchmarks. arXiv preprint arXiv:2103.14749 , 2021 . Curtis G Northcutt, Anish Athalye, and Jonas Mueller. Pervasive label errors in test sets destabilize machine learning benchmarks. arXiv preprint arXiv:2103.14749, 2021."},{"key":"e_1_3_2_1_15_1","first-page":"3976","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Park Wonpyo","year":"2019","unstructured":"Wonpyo Park , Dongju Kim , Yan Lu , and Minsu Cho . Relational knowledge distilla- tion . In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition , pages 3967\u2013 3976 , 2019 . Wonpyo Park, Dongju Kim, Yan Lu, and Minsu Cho. Relational knowledge distilla- tion. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, pages 3967\u20133976, 2019."},{"key":"e_1_3_2_1_16_1","volume-title":"Antoine Chassang, Carlo Gatta, and Yoshua Bengio. Fitnets: Hints for thin deep nets. arXiv preprint arXiv:1412.6550","author":"Romero Adriana","year":"2014","unstructured":"Adriana Romero , Nicolas Ballas , Samira Ebrahimi Kahou , Antoine Chassang, Carlo Gatta, and Yoshua Bengio. Fitnets: Hints for thin deep nets. arXiv preprint arXiv:1412.6550 , 2014 . Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. Fitnets: Hints for thin deep nets. arXiv preprint arXiv:1412.6550, 2014."},{"key":"e_1_3_2_1_17_1","first-page":"4520","volume-title":"Proceedings of the IEEE conference on computer vision and pattern recognition","author":"Sandler Mark","year":"2018","unstructured":"Mark Sandler , Andrew Howard , Menglong Zhu , Andrey Zhmoginov , and Liang-Chieh Chen . Mobilenetv2 : Inverted residuals and linear bottlenecks . In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 4510\u2013 4520 , 2018 . Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4510\u20134520, 2018."},{"key":"e_1_3_2_1_18_1","volume-title":"Very deep convolutional networks for large- scale image recognition. arXiv preprint arXiv:1409.1556","author":"Simonyan Karen","year":"2014","unstructured":"Karen Simonyan and Andrew Zisserman . Very deep convolutional networks for large- scale image recognition. arXiv preprint arXiv:1409.1556 , 2014 . Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large- scale image recognition. arXiv preprint arXiv:1409.1556, 2014."},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.308"},{"key":"e_1_3_2_1_21_1","volume-title":"Contrastive representation distillation. arXiv preprint arXiv:1910.10699","author":"Tian Yonglong","year":"2019","unstructured":"Yonglong Tian , Dilip Krishnan , and Phillip Isola . Contrastive representation distillation. arXiv preprint arXiv:1910.10699 , 2019 . Yonglong Tian, Dilip Krishnan, and Phillip Isola. Contrastive representation distillation. arXiv preprint arXiv:1910.10699, 2019."},{"key":"e_1_3_2_1_22_1","volume-title":"Visualizing data using t-sne. Journal of machine learning research, 9(11)","author":"der Maaten Laurens Van","year":"2008","unstructured":"Laurens Van der Maaten and Geoffrey Hinton . Visualizing data using t-sne. Journal of machine learning research, 9(11) , 2008 . Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008."},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.754"},{"key":"e_1_3_2_1_24_1","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Yuan Li","year":"2020","unstructured":"Li Yuan , Francis EH Tay , Guilin Li , Tao Wang , and Jiashi Feng . Revisiting knowl- edge distillation via label smoothing regularization . In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , June 2020 . Li Yuan, Francis EH Tay, Guilin Li, Tao Wang, and Jiashi Feng. Revisiting knowl- edge distillation via label smoothing regularization. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020."},{"key":"e_1_3_2_1_25_1","volume-title":"Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. arXiv preprint arXiv:1612.03928","author":"Zagoruyko Sergey","year":"2016","unstructured":"Sergey Zagoruyko and Nikos Komodakis . Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. arXiv preprint arXiv:1612.03928 , 2016 . Sergey Zagoruyko and Nikos Komodakis. Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. arXiv preprint arXiv:1612.03928, 2016."},{"key":"e_1_3_2_1_26_1","volume-title":"Wide residual networks. arXiv preprint arXiv:1605.07146","author":"Zagoruyko Sergey","year":"2016","unstructured":"Sergey Zagoruyko and Nikos Komodakis . Wide residual networks. arXiv preprint arXiv:1605.07146 , 2016 . Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. arXiv preprint arXiv:1605.07146, 2016."},{"key":"e_1_3_2_1_27_1","volume-title":"mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412","author":"Zhang Hongyi","year":"2017","unstructured":"Hongyi Zhang , Moustapha Cisse , Yann N Dauphin , and David Lopez-Paz . mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412 , 2017 . Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412, 2017."},{"key":"e_1_3_2_1_28_1","volume-title":"Rethinking soft labels for knowledge distillation: A bias-variance tradeoff perspective. arXiv preprint arXiv:2102.00650","author":"Zhou Helong","year":"2021","unstructured":"Helong Zhou , Liangchen Song , Jiajie Chen , Ye Zhou , Guoli Wang , Junsong Yuan , and Qian Zhang . Rethinking soft labels for knowledge distillation: A bias-variance tradeoff perspective. arXiv preprint arXiv:2102.00650 , 2021 . Helong Zhou, Liangchen Song, Jiajie Chen, Ye Zhou, Guoli Wang, Junsong Yuan, and Qian Zhang. Rethinking soft labels for knowledge distillation: A bias-variance tradeoff perspective. arXiv preprint arXiv:2102.00650, 2021."}],"event":{"name":"ICCPR '21: 2021 10th International Conference on Computing and Pattern Recognition","acronym":"ICCPR '21","location":"Shanghai China"},"container-title":["2021 10th International Conference on Computing and Pattern Recognition"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3497623.3497669","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3497623.3497669","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:49:24Z","timestamp":1750193364000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3497623.3497669"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,15]]},"references-count":28,"alternative-id":["10.1145\/3497623.3497669","10.1145\/3497623"],"URL":"https:\/\/doi.org\/10.1145\/3497623.3497669","relation":{},"subject":[],"published":{"date-parts":[[2021,10,15]]},"assertion":[{"value":"2022-02-04","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}