{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,26]],"date-time":"2026-09-26T17:25:52Z","timestamp":1790443552393,"version":"4.1.0"},"reference-count":220,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2026,1,2]],"date-time":"2026-01-02T00:00:00Z","timestamp":1767312000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"European project AIRISE","award":["101092312"],"award-info":[{"award-number":["101092312"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Deep learning models, especially for object detection have gained immense popularity in computer vision. These models have demonstrated remarkable accuracy and performance, driving advancements across various applications. However, the high computational complexity and large storage requirements of state-of-the-art object detection models pose significant challenges for deployment on resource-constrained devices like mobile phones and embedded systems. Knowledge Distillation (KD) has emerged as a prominent solution to these challenges, effectively compressing large, complex teacher models into smaller, efficient student models. This technique maintains good accuracy while significantly reducing model size and computational demands, making object detection models more practical for real-world applications. This survey provides a comprehensive review of KD-based object detection models developed in recent years. It offers an in-depth analysis of existing techniques, highlighting their novelty and limitations, and explores future research directions. The survey covers the different distillation algorithms used in object detection. It also examines extended applications of knowledge distillation in object detection, such as improvements for lightweight models, addressing catastrophic forgetting in incremental learning, and enhancing small object detection. Furthermore, the survey also delves into the application of knowledge distillation in other domains such as image classification, semantic segmentation, 3D reconstruction, and document analysis.<\/jats:p>","DOI":"10.3390\/s26010292","type":"journal-article","created":{"date-parts":[[2026,1,2]],"date-time":"2026-01-02T09:56:23Z","timestamp":1767347783000},"page":"292","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Knowledge Distillation in Object Detection: A Survey from CNN to Transformer"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7052-979X","authenticated-orcid":false,"given":"Tahira","family":"Shehzadi","sequence":"first","affiliation":[{"name":"Department of Computer Science, RPTU Kaiserslautern-Landau, 67663 Kaiserslautern, Germany"},{"name":"Mindgarage Lab, RPTU Kaiserslautern-Landau, 67663 Kaiserslautern, Germany"},{"name":"German Research Institute for Artificial Intelligence (DFKI), 67663 Kaiserslautern, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rabya","family":"Noor","sequence":"additional","affiliation":[{"name":"Department of Computer Science, RPTU Kaiserslautern-Landau, 67663 Kaiserslautern, Germany"},{"name":"German Research Institute for Artificial Intelligence (DFKI), 67663 Kaiserslautern, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-5055-7352","authenticated-orcid":false,"given":"Ifza","family":"Ifza","sequence":"additional","affiliation":[{"name":"Department of Computer Science, RPTU Kaiserslautern-Landau, 67663 Kaiserslautern, Germany"},{"name":"German Research Institute for Artificial Intelligence (DFKI), 67663 Kaiserslautern, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4029-6574","authenticated-orcid":false,"given":"Marcus","family":"Liwicki","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Electrical and Space Engineering Lule\u00e5 University of Technology, 971 87 Lule\u00e5, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Didier","family":"Stricker","sequence":"additional","affiliation":[{"name":"Department of Computer Science, RPTU Kaiserslautern-Landau, 67663 Kaiserslautern, Germany"},{"name":"Mindgarage Lab, RPTU Kaiserslautern-Landau, 67663 Kaiserslautern, Germany"},{"name":"German Research Institute for Artificial Intelligence (DFKI), 67663 Kaiserslautern, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0536-6867","authenticated-orcid":false,"given":"Muhammad Zeshan","family":"Afzal","sequence":"additional","affiliation":[{"name":"Department of Computer Science, RPTU Kaiserslautern-Landau, 67663 Kaiserslautern, Germany"},{"name":"Mindgarage Lab, RPTU Kaiserslautern-Landau, 67663 Kaiserslautern, Germany"},{"name":"German Research Institute for Artificial Intelligence (DFKI), 67663 Kaiserslautern, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,1,2]]},"reference":[{"key":"ref_1","unstructured":"Hinton, G.E., Vinyals, O., and Dean, J. (2015). Distilling the Knowledge in a Neural Network. arXiv."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Sauer, A., Asaadi, S., and K\u00fcch, F. (2022, January 27). Knowledge distillation meets few-shot learning: An approach for few-shot intent classification within and across domains. Proceedings of the 4th Workshop on NLP for Conversational AI, Dublin, Ireland.","DOI":"10.18653\/v1\/2022.nlp4convai-1.10"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Kim, Y., and Rush, A. (2016). Sequence-level knowledge distillation. arXiv.","DOI":"10.18653\/v1\/D16-1139"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"e474","DOI":"10.7717\/peerj-cs.474","article-title":"Knowledge distillation in deep learning and its applications","volume":"7","author":"Alkhulaifi","year":"2021","journal-title":"PeerJ Comput. Sci."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Kothandaraman, D., and Nambiar, A. (2021, January 5\u20139). Domain adaptive knowledge distillation for driving scene semantic segmentation. Proceedings of the 2021 IEEE Winter Conference on Applications of Computer Vision Workshops (WACVW), Waikola, HI, USA.","DOI":"10.1109\/WACVW52041.2021.00019"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Ni, Y., Zhang, X., and Li, Y. (2023). Knowledge Distillation Facilitates the Lightweight and Efficient Plant Diseases Detection Model. Agriculture, 13.","DOI":"10.34133\/plantphenomics.0062"},{"key":"ref_7","first-page":"56","article-title":"An Incremental Learning of YOLOv3 Without Catastrophic Forgetting for Smart City Applications","volume":"10","author":"Ruan","year":"2022","journal-title":"IEEE Consum. Electron. Mag."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Pham, C., Hoang, T., and Do, T.T. (2023, January 7). Collaborative Multi-Teacher Knowledge Distillation for Learning Low Bit-width Deep Neural Networks. Proceedings of the 2023 IEEE\/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA.","DOI":"10.1109\/WACV56688.2023.00637"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Zhang, H., Chen, D., and Wang, C. (2023, January 10\u201314). Adaptive Multi-Teacher Knowledge Distillation with Meta-Learning. Proceedings of the 2023 IEEE International Conference on Multimedia and Expo (ICME), Brisbane, Australia.","DOI":"10.1109\/ICME55011.2023.00333"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Zhang, H., Chen, D., and Wang, C. (2022, January 23\u201327). Confidence-Aware Multi-Teacher Knowledge Distillation. Proceedings of the ICASSP 2022\u20142022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore.","DOI":"10.1109\/ICASSP43922.2022.9747534"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Yuan, F., Shou, L., Pei, J., Lin, W., Gong, M., Fu, Y., and Jiang, D. (2021, January 2\u20139). Reinforced Multi-Teacher Selection for Knowledge Distillation. Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada.","DOI":"10.1609\/aaai.v35i16.17680"},{"key":"ref_12","first-page":"12345","article-title":"Agree to Disagree: Adaptive Ensemble Knowledge Distillation in Gradient Space","volume":"Volume 33","author":"Larochelle","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1016\/j.neucom.2020.07.048","article-title":"Adaptive multi-teacher multi-level knowledge distillation","volume":"415","author":"Liu","year":"2020","journal-title":"Neurocomputing"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Li, L., Dong, P., Wei, Z., and Yang, Y. (2023, January 1\u20136). Automated Knowledge Distillation via Monte Carlo Tree Search. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Paris, France.","DOI":"10.1109\/ICCV51070.2023.01597"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Liu, Y., Chen, K., Liu, C., Qin, Z., Luo, Z., and Wang, J. (2019, January 15\u201320). Structured Knowledge Distillation for Semantic Segmentation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00271"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"7035","DOI":"10.1109\/TPAMI.2020.3001940","article-title":"Structured Knowledge Distillation for Dense Prediction","volume":"45","author":"Liu","year":"2023","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Yang, C., Zhou, H., An, Z., Jiang, X., Xu, Y., and Zhang, Q. (2022, January 19\u201324). Cross-Image Relational Knowledge Distillation for Semantic Segmentation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01200"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Yang, Z., Li, Z., Shao, M., Shi, D., Yuan, Z., and Yuan, C. (2022, January 23\u201327). Masked Generative Distillation. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel.","DOI":"10.1007\/978-3-031-20083-0_4"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Shu, C., Liu, Y., Gao, J., Yan, Z., and Shen, C. (2021, January 11\u201317). Channel-Wise Knowledge Distillation for Dense Prediction. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.00526"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Chen, J., Zhu, D., Qian, G., Ghanem, B., Yan, Z., Zhu, C., Xiao, F., Culatana, S.C., and Elhoseiny, M. (2023, January 1\u20136). Exploring Open-Vocabulary Semantic Segmentation from CLIP Vision Encoder Distillation Only. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Paris, France.","DOI":"10.1109\/ICCV51070.2023.00071"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Liu, L., Wang, Z., Phan, M.H., Zhang, B., Ge, J., and Liu, Y. (2024, January 7\u201311). BPKD: Boundary Privileged Knowledge Distillation for Semantic Segmentation. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, Hawaii.","DOI":"10.1109\/WACV57701.2024.00110"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Wang, Y., Zhou, W., Jiang, T., Bai, X., and Xu, Y. (2020, January 23\u201328). Intra-class Feature Variation Distillation for Semantic Segmentation. Proceedings of the Computer Vision\u2014ECCV Cham, Switzerland.","DOI":"10.1007\/978-3-030-58571-6_21"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Lin, S., Xie, H., Wang, B., Yu, K., Chang, X., Liang, X., and Wang, G. (2022, January 19\u201324). Knowledge Distillation via the Target-Aware Transformer. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01064"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Berrada, T., Couprie, C., Alahari, K., and Verbeek, J. (2024, January 7\u201311). Guided Distillation for Semi-Supervised Instance Segmentation. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, Hawaii.","DOI":"10.1109\/WACV57701.2024.00053"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Xu, Y., Yang, Y., and Zhang, L. (2023, January 1\u20136). Multi-Task Learning with Knowledge Distillation for Dense Prediction. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Paris, France.","DOI":"10.1109\/ICCV51070.2023.01970"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Yang, Z., Li, R., Ling, E., Zhang, C., Wang, Y., Huang, D., Ma, K.T., Hur, M., and Lin, G. (2023, January 1\u20136). Label-Guided Knowledge Distillation for Continual Semantic Segmentation on 2D Images and 3D Point Clouds. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Paris, France.","DOI":"10.1109\/ICCV51070.2023.01705"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Ji, D., Wang, H., Tao, M., Huang, J., Hua, X.S., and Lu, H. (2022, January 19\u201324). Structural and Statistical Texture Knowledge Distillation for Semantic Segmentation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01637"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Phan, M.H., Ta, T.A., Phung, S.L., Tran-Thanh, L., and Bouzerdoum, A. (2022, January 19\u201324). Class Similarity Weighted Knowledge Distillation for Continual Semantic Segmentation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01636"},{"key":"ref_29","first-page":"51359","article-title":"Augmentation-Free Dense Contrastive Knowledge Distillation for Efficient Semantic Segmentation","volume":"Volume 36","author":"Oh","year":"2023","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref_30","first-page":"10380","article-title":"Decomposed Knowledge Distillation for Class-Incremental Semantic Segmentation","volume":"Volume 35","author":"Koyejo","year":"2022","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Lin, J., Yuan, Y., Shao, T., and Zhou, K. (2020, January 13\u201319). Towards High-Fidelity 3D Face Reconstruction From In-the-Wild Images Using Graph Convolutional Networks. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00593"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Wang, S., Clark, R., Wen, H., and Trigoni, N. (June, January 29). Deepvo: Towards end-to-end visual odometry with deep recurrent convolutional neural networks. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989236"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Jackson, A.S., Bulat, A., Argyriou, V., and Tzimiropoulos, G. (2017, January 22\u201329). Large pose 3D face reconstruction from a single image via direct volumetric CNN regression. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.117"},{"key":"ref_34","unstructured":"Zheng, H., Cao, D., Xu, J., Ai, R., Gu, W., Yang, Y., and Liang, Y. (2024). Distilling Temporal Knowledge with Masked Feature Reconstruction for 3D Object Detection. arXiv."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"13864","DOI":"10.3934\/mbe.2023617","article-title":"IFKD: Implicit field knowledge distillation for single view reconstruction","volume":"20","author":"Wang","year":"2023","journal-title":"Math. Biosci. Eng."},{"key":"ref_36","unstructured":"Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. Towards Efficient 3D Object Detection with Knowledge Distillation. Advances in Neural Information Processing Systems, Curran Associates, Inc.. 28 November\u20139 December 2022."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Zhang, L., Dong, R., Tai, H.S., and Ma, K. (2023, January 19\u201324). PointDistiller: Structured Knowledge Distillation Towards Efficient and Compact 3D Detection. Proceedings of the 2023 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.02087"},{"key":"ref_38","unstructured":"Landeghem, J.V., Maity, S., Banerjee, A., Blaschko, M.B., Moens, M.F., Llad\u2019os, J., and Biswas, S. (2024). DistilDoc: Knowledge Distillation for Visually-Rich Document Applications. Document Analysis and Recognition\u2014ICDAR 2024: 18th International Conference, Athens, Greece, 30 August\u20134 September 2024, Springer."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Banerjee, A., Biswas, S., Llad\u2019\u00f3s, J., and Pal, U. (2024). GraphKD: Exploring Knowledge Distillation Towards Document Object Detection with Structured Graph Creation. Document Analysis and Recognition\u2014ICDAR 2024: 18th International Conference, Athens, Greece, 30 August\u20134 September 2024, Springer.","DOI":"10.1007\/978-3-031-70543-4_21"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Chen, X., He, B., Hui, K., Sun, L., and Sun, Y. (2020). Simplified TinyBERT: Knowledge Distillation for Document Retrieval. arXiv.","DOI":"10.1007\/978-3-030-72240-1_21"},{"key":"ref_41","unstructured":"Fink, G.A., Jain, R., Kise, K., and Zanibbi, R. (2023). Multi-teacher Knowledge Distillation for End-to-End Text Image Machine Translation. Proceedings of the Document Analysis and Recognition\u2014ICDAR 2023, Springer."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Elhanashi, A., Dini, P., Saponara, S., and Zheng, Q. (2023). Integration of Deep Learning into the IoT: A Survey of Techniques and Challenges for Real-World Applications. Electronics, 12.","DOI":"10.3390\/electronics12244925"},{"key":"ref_43","unstructured":"Cortes, C., Lawrence, N., Lee, D., Sugiyama, M., and Garnett, R. (2015). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. Advances in Neural Information Processing Systems, Curran Associates, Inc."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Tan, M., Pang, R., and Le, Q.V. (2020, January 14\u201319). EfficientDet: Scalable and Efficient Object Detection. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01079"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Doll\u00e1r, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017). Feature Pyramid Networks for Object Detection. arXiv, Available online: http:\/\/arxiv.org\/abs\/1612.03144.","DOI":"10.1109\/CVPR.2017.106"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016). You Only Look Once: Unified, Real-Time Object Detection. arXiv, Available online: http:\/\/arxiv.org\/abs\/1506.02640.","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Goyal, P., Girshick, R., He, K., and Dollar, P. (2017, January 22\u201329). Focal Loss for Dense Object Detection. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.324"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016). SSD: Single Shot MultiBox Detector. Lecture Notes in Computer Science, Springer International Publishing.","DOI":"10.1007\/978-3-319-46448-0_2"},{"key":"ref_49","unstructured":"He, K., Gkioxari, G., Doll\u00e1r, P., and Girshick, R. (2018). Mask R-CNN. arXiv, Available online: http:\/\/arxiv.org\/abs\/1703.06870."},{"key":"ref_50","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2021). An Image is Worth 16 \u00d7 16 Words: Transformers for Image Recognition at Scale. arXiv, Available online: http:\/\/arxiv.org\/abs\/2010.11929."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Vedaldi, A., Bischof, H., Brox, T., and Frahm, J.-M. (2020). End-to-End Object Detection with Transformers. Computer Vision\u2014ECCV 2020, Springer.","DOI":"10.1007\/978-3-030-58542-6"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Tian, Z., Shen, C., Chen, H., and He, T. (November, January 27). FCOS: Fully Convolutional One-Stage Object Detection. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00972"},{"key":"ref_53","unstructured":"Shehzadi, T., Hashmi, K.A., Sarode, S., Stricker, D., and Afzal, M.Z. (2025, January 12\u201319). STEP-DETR: Advancing DETR-based Semi-Supervised Object Detection with Super Teacher and Pseudo-Label Guided Text Queries. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Honolulu, HI, USA."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Wang, C.Y., Bochkovskiy, A., and Liao, H.Y.M. (2022). YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv.","DOI":"10.1109\/CVPR52729.2023.00721"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Law, H., and Deng, J. (2018, January 8\u201314). CornerNet: Detecting Objects as Paired Keypoints. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01264-9_45"},{"key":"ref_56","unstructured":"Duan, K., Bai, S., Xie, L., Qi, H., Huang, Q., and Tian, Q. (November, January 27). CenterNet: Keypoint Triplets for Object Detection. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea."},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Zhang, S., Wen, L., Bian, X., Lei, Z., and Li, S.Z. (2018, January 18\u201322). Single-Shot Refinement Neural Network for Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00442"},{"key":"ref_58","unstructured":"Li, Y., Chen, Y., Wang, N., and Zhang, Z. (November, January 27). Scale-Aware Trident Networks for Object Detection. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea."},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Chen, K., Pang, J., Wang, J., Xiong, Y., Li, X., Sun, S., Feng, W., Liu, Z., Shi, J., and Ouyang, W. (2019, January 16\u201320). Hybrid Task Cascade for Instance Segmentation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00511"},{"key":"ref_60","unstructured":"Zhu, X., Su, W., Lu, L., Li, B., Wang, X., and Dai, J. (2021). Deformable DETR: Deformable Transformers for End-to-End Object Detection. arXiv."},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 11\u201317). Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.00986"},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Vincent, O.R., and Folorunso, O. (2009, January 12\u201315). A Descriptive Algorithm for Sobel Image Edge Detection. Proceedings of the InSITE 2009: Informing Science + IT Education Conference, Macon, GA, USA.","DOI":"10.28945\/3351"},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"679","DOI":"10.1109\/TPAMI.1986.4767851","article-title":"A Computational Approach To Edge Detection","volume":"PAMI-8","author":"Canny","year":"1986","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_64","doi-asserted-by":"crossref","unstructured":"Harris, C.G., and Stephens, M.J. (1988, January 1\u20133). A Combined Corner and Edge Detector. Proceedings of the Alvey Vision Conference, London, UK.","DOI":"10.5244\/C.2.23"},{"key":"ref_65","unstructured":"Dalal, N., and Triggs, B. (2005, January 20\u201326). Histograms of oriented gradients for human detection. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR\u201905), San Diego, CA, USA."},{"key":"ref_66","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1023\/A:1022627411411","article-title":"Support-Vector Networks","volume":"20","author":"Cortes","year":"1995","journal-title":"Mach. Learn."},{"key":"ref_67","doi-asserted-by":"crossref","unstructured":"Girshick, R. (2015, January 7\u201313). Fast R-CNN. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.169"},{"key":"ref_68","first-page":"10469","article-title":"Optimized convolutional neural network architectures for efficient on-device vision-based object detection","volume":"34","author":"Campos","year":"2021","journal-title":"Neural Comput. Appl."},{"key":"ref_69","unstructured":"Du, Z., Zhang, R., Chang, M.F., Zhang, X., Liu, S., Chen, T., and Chen, Y. (2021, January 6\u201314). Distilling Object Detectors with Feature Richness. Proceedings of the Neural Information Processing Systems, Red Hook, NY, USA."},{"key":"ref_70","doi-asserted-by":"crossref","first-page":"3048","DOI":"10.1109\/TPAMI.2021.3055564","article-title":"Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks","volume":"44","author":"Wang","year":"2020","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_71","unstructured":"Chen, G., Choi, W., Yu, X., Han, T.X., and Chandraker, M. (2017, January 4\u20139). Learning Efficient Object Detection Models with Knowledge Distillation. Proceedings of the Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_72","doi-asserted-by":"crossref","unstructured":"Zhang, F., Zhu, X., and Ye, M. (2019, January 20). Fast Human Pose Estimation. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00363"},{"key":"ref_73","doi-asserted-by":"crossref","unstructured":"Meng, Z., Li, J., Zhao, Y., and Gong, Y. (2019, January 12\u201317). Conditional Teacher-student Learning. Proceedings of the ICASSP 2019\u20142019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK.","DOI":"10.1109\/ICASSP.2019.8683438"},{"key":"ref_74","unstructured":"Kim, S., and Kim, H.E. (2017, January 24\u201326). Transferring Knowledge to Smaller Network with Class-Distance Loss. Proceedings of the International Conference on Learning Representations, Toulon, France."},{"key":"ref_75","unstructured":"Romero, A., Ballas, N., Kahou, S.E., Chassang, A., Gatta, C., and Bengio, Y. (2015, January 7\u20139). FitNets: Hints for Thin Deep Nets. Proceedings of the ICLR, San Diego, CA, USA."},{"key":"ref_76","unstructured":"Yim, J., Joo, D., Bae, J., and Kim, J. (2017, January 21\u201326). Like What You Like: Knowledge Distill via Neuron Selectivity Transfer. Proceedings of the CVPR, Honolulu, HI, USA."},{"key":"ref_77","unstructured":"Zagoruyko, S., and Komodakis, N. (2017, January 24\u201326). Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer. Proceedings of the ICLR, Toulon, France."},{"key":"ref_78","unstructured":"Yim, J., Joo, D., Bae, J., and Kim, J. (2018, January 3\u20138). Paraphrasing Complex Network: Network Compression via Factor Transfer. Proceedings of the NIPS, Montreal, QC, Canada."},{"key":"ref_79","unstructured":"Heo, B., Lee, M., Yun, S., and Choi, J.Y. (February, January 27). Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons. Proceedings of the AAAI, Honolulu, HI, USA."},{"key":"ref_80","doi-asserted-by":"crossref","unstructured":"Zhou, G., Fan, Y., Cui, R., Bian, W., Zhu, X., and Gai, K. (2017, January 4\u20139). Rocket Launching: A Universal and Efficient Framework for Training Well-performing Light Net. Proceedings of the AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.","DOI":"10.1609\/aaai.v32i1.11601"},{"key":"ref_81","doi-asserted-by":"crossref","unstructured":"Liu, J., Wen, D., Gao, H., Tao, W., Chen, T.W., Osa, K., and Kato, M. (2019, January 16\u201320). Knowledge Representing: Efficient, Sparse Representation of Prior Knowledge for Knowledge Distillation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Long Beach, CA, USA.","DOI":"10.1109\/CVPRW.2019.00090"},{"key":"ref_82","first-page":"3068","article-title":"Amalgamating Knowledge towards Comprehensive Classification","volume":"33","author":"Shen","year":"2019","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"ref_83","unstructured":"Heo, B., Kim, J., Yun, S., Park, H., Kwak, N., and Choi, J.Y. (November, January 27). A Comprehensive Overhaul of Feature Distillation. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, South Korea."},{"key":"ref_84","doi-asserted-by":"crossref","unstructured":"Vedaldi, A., Bischof, H., Brox, T., and Frahm, J.M. (2020). Feature Normalized Knowledge Distillation for Image Classification. Proceedings of the Computer Vision\u2014ECCV 2020, Springer.","DOI":"10.1007\/978-3-030-58542-6"},{"key":"ref_85","doi-asserted-by":"crossref","unstructured":"Guan, Y., Zhao, P., Wang, B., Zhang, Y., Yao, C., Bian, K., and Tang, J. (2020). Differentiable Feature Aggregation Search for Knowledge Distillation. arXiv.","DOI":"10.1007\/978-3-030-58520-4_28"},{"key":"ref_86","unstructured":"Yang, J., Mart\u00ednez, B., Bulat, A., and Tzimiropoulos, G. (2020). Knowledge distillation via adaptive instance normalization. arXiv."},{"key":"ref_87","doi-asserted-by":"crossref","unstructured":"Wang, X., Fu, T., Liao, S., Wang, S., Lei, Z., and Mei, T. (2020, January 23\u201328). Exclusivity-Consistency Regularized Knowledge Distillation for Face Recognition. Proceedings of the European Conference on Computer Vision, Glasgow, UK.","DOI":"10.1007\/978-3-030-58586-0_20"},{"key":"ref_88","unstructured":"Passban, P., Wu, Y., Rezagholizadeh, M., and Liu, Q. (2020, January 7\u201312). ALP-KD: Attention-Based Layer Projection for Knowledge Distillation. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA."},{"key":"ref_89","unstructured":"Chen, D., Mei, J.P., Zhang, Y., Wang, C., Wang, Z., Feng, Y., and Chen, C. (2020, January 7\u201312). Cross-Layer Distillation with Semantic Calibration. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA."},{"key":"ref_90","doi-asserted-by":"crossref","unstructured":"Yim, J., Joo, D., Bae, J., and Kim, J. (2017, January 21\u201326). A Gift from Knowledge Distillation: Fast Optimization, Network Minimization and Transfer Learning. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.754"},{"key":"ref_91","doi-asserted-by":"crossref","unstructured":"You, S., Xu, C., Xu, C., and Tao, D. (2017, January 13\u201317). Learning from Multiple Teacher Networks. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada.","DOI":"10.1145\/3097983.3098135"},{"key":"ref_92","doi-asserted-by":"crossref","unstructured":"Zhang, C., and Peng, Y. (2018, January 13\u201319). Better and Faster: Knowledge Transfer from Multiple Self-supervised Learning Tasks via Graph Distillation for Video Classification. Proceedings of the International Joint Conference on Artificial Intelligence, Stockholm, Sweden.","DOI":"10.24963\/ijcai.2018\/158"},{"key":"ref_93","doi-asserted-by":"crossref","unstructured":"Chen, Y., Wang, N., and Zhang, Z. (2017). DarkRank: Accelerating Deep Metric Learning via Cross Sample Similarities Transfer. arXiv.","DOI":"10.1609\/aaai.v32i1.11783"},{"key":"ref_94","unstructured":"Lee, S., and Song, B.C. (2019, January 9\u201312). Graph-based Knowledge Distillation by Multi-head Attention Network. Proceedings of the British Machine Vision Conference, Cardiff, UK."},{"key":"ref_95","doi-asserted-by":"crossref","unstructured":"Park, W., Kim, D., Lu, Y., and Cho, M. (2019, January 29). Relational Knowledge Distillation. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00409"},{"key":"ref_96","doi-asserted-by":"crossref","unstructured":"Liu, Y., Cao, J., Li, B., Yuan, C., Hu, W., Li, Y., and feng Duan, Y. (2019, January 29). Knowledge Distillation via Instance Relationship Graph. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00726"},{"key":"ref_97","doi-asserted-by":"crossref","unstructured":"Tung, F., and Mori, G. (November, January 27). Similarity-Preserving Knowledge Distillation. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.","DOI":"10.1109\/ICCV.2019.00145"},{"key":"ref_98","doi-asserted-by":"crossref","unstructured":"Peng, B., Jin, X., Liu, J., Zhou, S., Wu, Y., Liu, Y., Li, D., and Zhang, Z. (November, January 27). Correlation Congruence for Knowledge Distillation. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.","DOI":"10.1109\/ICCV.2019.00511"},{"key":"ref_99","doi-asserted-by":"crossref","unstructured":"Yu, L., Yazici, V.O., Liu, X., van de Weijer, J., Cheng, Y., and Ramisa, A. (2019, January 15\u201320). Learning Metrics From Teachers: Compact Networks for Image Embedding. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00302"},{"key":"ref_100","doi-asserted-by":"crossref","first-page":"2030","DOI":"10.1109\/TNNLS.2020.2995884","article-title":"Probabilistic Knowledge Transfer for Lightweight Deep Representation Learning","volume":"32","author":"Passalis","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_101","doi-asserted-by":"crossref","unstructured":"Passalis, N., Tzelepi, M., and Tefas, A. (2020, January 13\u201319). Heterogeneous Knowledge Distillation Using Information Flow Modeling. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00241"},{"key":"ref_102","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1109\/TNNLS.2020.2970494","article-title":"Learning Student Networks via Feature Embedding","volume":"32","author":"Chen","year":"2018","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_103","doi-asserted-by":"crossref","unstructured":"Jiao, X., Yin, Y., Shang, L., Jiang, X., Chen, X., Li, L., Wang, F., and Liu, Q. (2019). TinyBERT: Distilling BERT for Natural Language Understanding. arXiv.","DOI":"10.18653\/v1\/2020.findings-emnlp.372"},{"key":"ref_104","unstructured":"Tang, R., Lu, Y., Liu, L., Mou, L., Vechtomova, O., and Lin, J.J. (2019). Distilling Task-Specific Knowledge from BERT into Simple Neural Networks. arXiv."},{"key":"ref_105","unstructured":"Allen-Zhu, Z., and Li, Y. (2020). Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning. arXiv."},{"key":"ref_106","doi-asserted-by":"crossref","first-page":"2935","DOI":"10.1109\/TPAMI.2017.2773081","article-title":"Learning without Forgetting","volume":"40","author":"Li","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_107","doi-asserted-by":"crossref","unstructured":"Peng, Z., Li, Z., Zhang, J., Li, Y., Qi, G.J., and Tang, J. (November, January 27). Few-Shot Image Recognition with Knowledge Transfer. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.","DOI":"10.1109\/ICCV.2019.00053"},{"key":"ref_108","unstructured":"Bagherinezhad, H., Horton, M., Rastegari, M., and Farhadi, A. (2018). Label Refinery: Improving ImageNet Classification through Label Progression. arXiv."},{"key":"ref_109","unstructured":"Chen, W.C., Chang, C.C., Lu, C.Y., and Lee, C.R. (2018, January 2\u20136). Knowledge Distillation with Feature Maps for Image Classification. Proceedings of the Asian Conference on Computer Vision, Perth, Australia."},{"key":"ref_110","doi-asserted-by":"crossref","first-page":"2168","DOI":"10.1109\/TVCG.2019.2903943","article-title":"DeepVID: Deep Visual Interpretation and Diagnosis for Image Classifiers via Knowledge Distillation","volume":"25","author":"Wang","year":"2019","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_111","doi-asserted-by":"crossref","unstructured":"Mukherjee, P., Das, A., Bhunia, A.K., and Roy, P.P. (2018, January 7\u201310). Cogni-Net: Cognitive Feature Learning Through Deep Visual Perception. Proceedings of the 2019 IEEE International Conference on Image Processing, Athens, Greece.","DOI":"10.1109\/ICIP.2019.8803717"},{"key":"ref_112","doi-asserted-by":"crossref","unstructured":"Zhu, M., Han, K., Zhang, C., Lin, J., and Wang, Y. (2019, January 12\u201317). Low-resolution Visual Recognition via Deep Feature Distillation. Proceedings of the ICASSP 2019\u20142019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK.","DOI":"10.1109\/ICASSP.2019.8682926"},{"key":"ref_113","unstructured":"Rijk, P.D., Schneider, L., Cordts, M., and Gavrila, D. (December, January 28). Structural Knowledge Distillation for Object Detection. Proceedings of the Advances in Neural Information Processing Systems, New Orleans, LA, USA."},{"key":"ref_114","unstructured":"Cao, W., Zhang, Y., Gao, J., Cheng, A., Cheng, K., and Cheng, J. (December, January 28). PKD: General Distillation Framework for Object Detectors via Pearson Correlation Coefficient. Proceedings of the Advances in Neural Information Processing Systems, New Orleans, LA, USA."},{"key":"ref_115","doi-asserted-by":"crossref","unstructured":"Zhao, B., Cui, Q., Song, R., Qiu, Y., and Liang, J. (2022, January 19\u201324). Decoupled Knowledge Distillation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01165"},{"key":"ref_116","doi-asserted-by":"crossref","unstructured":"Jin, Y., Wang, J., and Lin, D. (2023, January 19\u201324). Multi-Level Logit Distillation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.02325"},{"key":"ref_117","doi-asserted-by":"crossref","unstructured":"Feng, T., Wang, M., and Yuan, H. (2022, January 19\u201324). Overcoming Catastrophic Forgetting in Incremental Object Detection via Elastic Response Distillation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00921"},{"key":"ref_118","first-page":"16468","article-title":"Instance-Conditional Knowledge Distillation for Object Detection","volume":"Volume 34","author":"Ranzato","year":"2021","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref_119","first-page":"69490","article-title":"KD-Zero: Evolving Knowledge Distiller for Any Teacher-Student Pairs","volume":"Volume 36","author":"Oh","year":"2023","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref_120","doi-asserted-by":"crossref","unstructured":"Wang, L., Li, X., Liao, Y., Jiang, Z., Wu, J., Wang, F., Qian, C., and Liu, S. (2022, January 23\u201327). HEAD: HEtero-Assists Distillation for Heterogeneous Object Detectors. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel.","DOI":"10.1007\/978-3-031-20077-9_19"},{"key":"ref_121","doi-asserted-by":"crossref","unstructured":"Dai, X., Jiang, Z., Wu, Z., Bao, Y., Wang, Z., Liu, S., and Zhou, E. (2021, January 20\u201325). General Instance Distillation for Object Detection. Proceedings of the 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00775"},{"key":"ref_122","doi-asserted-by":"crossref","unstructured":"Guo, J., Han, K., Wang, Y., Wu, H., Chen, X., Xu, C., and Xu, C. (2021, January 20\u201325). Distilling Object Detectors via Decoupled Features. Proceedings of the 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00219"},{"key":"ref_123","doi-asserted-by":"crossref","unstructured":"Wang, T., Yuan, L., Zhang, X., and Feng, J. (2019, January 15\u201320). Distilling Object Detectors with Fine-Grained Feature Imitation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00507"},{"key":"ref_124","doi-asserted-by":"crossref","unstructured":"Yang, Z., Li, Z., Jiang, X., Gong, Y., Yuan, Z., Zhao, D., and Yuan, C. (2022, January 19\u201324). Focal and Global Knowledge Distillation for Detectors. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00460"},{"key":"ref_125","unstructured":"Zhang, L., and Ma, K. (2021, January 3\u20137). Improve Object Detection with Feature-based Knowledge Distillation: Towards Accurate and Efficient Detectors. Proceedings of the International Conference on Learning Representations, Virtual Event."},{"key":"ref_126","doi-asserted-by":"crossref","unstructured":"Wu, D., Chen, P., Yu, X., Li, G., Han, Z., and Jiao, J. (2023, January 1\u20136). Spatial Self-Distillation for Object Detection with Inaccurate Bounding Boxes. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Paris, France.","DOI":"10.1109\/ICCV51070.2023.00631"},{"key":"ref_127","doi-asserted-by":"crossref","unstructured":"He, Y., Zhu, C., Wang, J., Savvides, M., and Zhang, X. (2019, January 16\u201320). Bounding Box Regression with Uncertainty for Accurate Object Detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00300"},{"key":"ref_128","doi-asserted-by":"crossref","unstructured":"Bilen, H., and Vedaldi, A. (2016, January 27\u201330). Weakly Supervised Deep Detection Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.311"},{"key":"ref_129","doi-asserted-by":"crossref","unstructured":"Liu, C., Wang, K., Lu, H., Cao, Z., and Zhang, Z. (2022). Robust Object Detection with Inaccurate Bounding Boxes. arXiv, Available online: http:\/\/arxiv.org\/abs\/2207.09697.","DOI":"10.1007\/978-3-031-20080-9_4"},{"key":"ref_130","doi-asserted-by":"crossref","unstructured":"Chen, P., Yu, X., Han, X., Hassan, N., Wang, K., Li, J., Zhao, J., Shi, H., Han, Z., and Ye, Q. (2022, January 23\u201327). Point-to-Box Network for Accurate Object Detection via Single Point Supervision. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel.","DOI":"10.1007\/978-3-031-20077-9_4"},{"key":"ref_131","doi-asserted-by":"crossref","unstructured":"Chadwick, S., and Newman, P. (2019, January 9\u201312). Training Object Detectors with Noisy Data. Proceedings of the 2019 IEEE Intelligent Vehicles Symposium (IV), Paris, France.","DOI":"10.1109\/IVS.2019.8814137"},{"key":"ref_132","unstructured":"Li, J., Xiong, C., Socher, R., and Hoi, S.C.H. (2020). Towards Noise-resistant Object Detection with Noisy Annotations. arXiv."},{"key":"ref_133","doi-asserted-by":"crossref","unstructured":"Mao, J., Yu, Q., Yamakata, Y., and Aizawa, K. (2021). Noisy Annotation Refinement for Object Detection. arXiv.","DOI":"10.5244\/C.35.230"},{"key":"ref_134","doi-asserted-by":"crossref","first-page":"5782","DOI":"10.1109\/TIP.2021.3085208","article-title":"Training Robust Object Detectors From Noisy Category Labels and Imprecise Bounding Boxes","volume":"30","author":"Xu","year":"2021","journal-title":"IEEE Trans. Image Process."},{"key":"ref_135","unstructured":"Ma, Z., Luo, G., Gao, J., Li, L., Chen, Y., Wang, S., Zhang, C., and Hu, W. (2022, January 19\u201324). Open-Vocabulary One-Stage Detection with Hierarchical Visual-Language Knowledge Distillation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA."},{"key":"ref_136","doi-asserted-by":"crossref","unstructured":"Zareian, A., Rosa, K.D., Hu, D.H., and Chang, S.F. (2021, January 20\u201325). Open-Vocabulary Object Detection Using Captions. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.01416"},{"key":"ref_137","doi-asserted-by":"crossref","unstructured":"Bansal, A., Sikka, K., Sharma, G., Chellappa, R., and Divakaran, A. (2018, January 8\u201314). Zero-Shot Object Detection. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01246-5_24"},{"key":"ref_138","unstructured":"Gu, X., Lin, T.Y., Kuo, W., and Cui, Y. (2022, January 25\u201329). Open-vocabulary Object Detection via Vision and Language Knowledge Distillation. Proceedings of the International Conference on Learning Representations, Virtual Event."},{"key":"ref_139","unstructured":"Li, Z., Yao, L., Zhang, X., Wang, X., Kanhere, S.S., and Zhang, H. (27\u20131, January 27). Zero-Shot Object Detection with Textual Descriptions. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA."},{"key":"ref_140","unstructured":"Zheng, Y., Huang, R., Han, C., Huang, X., and Cui, L. (December, January 30). Background Learnable Cascade for Zero-Shot Object Detection. Proceedings of the Asian Conference on Computer Vision (ACCV), Kyoto, Japan."},{"key":"ref_141","doi-asserted-by":"crossref","unstructured":"Zheng, Y., Wu, J., Qin, Y., Zhang, F., and Cui, L. (2021, January 19\u201325). Zero-Shot Instance Segmentation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00262"},{"key":"ref_142","unstructured":"Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., and Clark, J. (2021). Learning Transferable Visual Models From Natural Language Supervision. arXiv, Available online: http:\/\/arxiv.org\/abs\/2103.00020."},{"key":"ref_143","doi-asserted-by":"crossref","unstructured":"Li, D., Tasci, S., Ghosh, S., Zhu, J., Zhang, J., and Heck, L. (2019, January 7\u20139). RILOD: Near real-time incremental learning for object detection at the edge. Proceedings of the 4th ACM\/IEEE Symposium on Edge Computing, Bellevue, WA, USA.","DOI":"10.1145\/3318216.3363317"},{"key":"ref_144","doi-asserted-by":"crossref","first-page":"10070","DOI":"10.1109\/TPAMI.2023.3248583","article-title":"Localization Distillation for Object Detection","volume":"45","author":"Zheng","year":"2023","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_145","unstructured":"Cho, J.H., and Hariharan, B. (November, January 27). On the Efficacy of Knowledge Distillation. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Korea."},{"key":"ref_146","unstructured":"Furlanello, T., Lipton, Z.C., Tschannen, M., Itti, L., and Anandkumar, A. (2018). Born Again Neural Networks. arXiv, Available online: http:\/\/arxiv.org\/abs\/1805.04770."},{"key":"ref_147","unstructured":"Mirzadeh, S.I., Farajtabar, M., Li, A., Levine, N., Matsukawa, A., and Ghasemzadeh, H. (February, January 27). Improved Knowledge Distillation via Teacher Assistant. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA."},{"key":"ref_148","doi-asserted-by":"crossref","unstructured":"Yang, C., Xie, L., Su, C., and Yuille, A.L. (2019, January 16\u201320). Snapshot Distillation: Teacher-Student Optimization in One Generation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00297"},{"key":"ref_149","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Xiang, T., Hospedales, T.M., and Lu, H. (2017, January 21\u201326). Deep Mutual Learning. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2018.00454"},{"key":"ref_150","doi-asserted-by":"crossref","unstructured":"Pham, C., Nguyen, V.A., Le, T., Phung, D., Carneiro, G., and Do, T.T. (2024, January 4\u20138). Frequency Attention for Knowledge Distillation. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA.","DOI":"10.1109\/WACV57701.2024.00227"},{"key":"ref_151","doi-asserted-by":"crossref","unstructured":"Ji, M., Heo, B., and Park, S. (2021). Show, Attend and Distill: Knowledge Distillation via Attention-based Feature Matching. arXiv.","DOI":"10.1609\/aaai.v35i9.16969"},{"key":"ref_152","doi-asserted-by":"crossref","unstructured":"Shin, S., Lee, J., Lee, J., Yu, Y., and Lee, K. (2022, January 23\u201327). Teaching Where to Look: Attention Similarity Knowledge Distillation for Low Resolution Face Recognition. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel.","DOI":"10.1007\/978-3-031-19775-8_37"},{"key":"ref_153","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1007\/BF00348431","article-title":"Gauss and the history of the fast Fourier transform","volume":"34","author":"Heideman","year":"1985","journal-title":"Arch. Hist. Exact Sci."},{"key":"ref_154","doi-asserted-by":"crossref","unstructured":"Li, Q., Jin, S., and Yan, J. (2017, January 21\u201326). Mimicking Very Efficient Network for Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.776"},{"key":"ref_155","doi-asserted-by":"crossref","unstructured":"Huang, J., and Guo, Z. (2023, January 1\u201310). Pixel-Wise Contrastive Distillation. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Paris, France.","DOI":"10.1109\/ICCV51070.2023.01499"},{"key":"ref_156","doi-asserted-by":"crossref","unstructured":"Lee, S.H., Kim, D.H., and Song, B.C. (2018, January 8\u201314). Self-supervised Knowledge Distillation Using Singular Value Decomposition. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01231-1_21"},{"key":"ref_157","unstructured":"Hadsell, R., Chopra, S., and LeCun, Y. (2006, January 17\u201322). Dimensionality Reduction by Learning an Invariant Mapping. Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR\u201906), New York, NY, USA."},{"key":"ref_158","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017, January 4\u20139). Attention Is All You Need. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS 2017), Long Beach, CA, USA."},{"key":"ref_159","doi-asserted-by":"crossref","unstructured":"He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R. (2020, January 13\u201319). Momentum Contrast for Unsupervised Visual Representation Learning. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00975"},{"key":"ref_160","first-page":"33716","article-title":"Knowledge Distillation from a Stronger Teacher","volume":"Volume 35","author":"Koyejo","year":"2022","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref_161","unstructured":"Zhou, H., Song, L., Chen, J., Zhou, Y., Wang, G., Yuan, J., and Zhang, Q. (2021). Rethinking Soft Labels for Knowledge Distillation: A Bias-Variance Tradeoff Perspective. arXiv."},{"key":"ref_162","doi-asserted-by":"crossref","unstructured":"Cao, Y., Xu, J., Lin, S., Wei, F., and Hu, H. (November, January 27). GCNet: Non-Local Networks Meet Squeeze-Excitation Networks and Beyond. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Republic of Korea.","DOI":"10.1109\/ICCVW.2019.00246"},{"key":"ref_163","doi-asserted-by":"crossref","unstructured":"Lao, S., Song, G., Liu, B., Liu, Y., and Yang, Y. (2023, January 1\u201310). UniKD: Universal Knowledge Distillation for Mimicking Homogeneous or Heterogeneous Object Detectors. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Paris, France.","DOI":"10.1109\/ICCV51070.2023.00585"},{"key":"ref_164","doi-asserted-by":"crossref","unstructured":"Lu, X., Li, Q., Li, B., and Yan, J. (2020). MimicDet: Bridging the Gap Between One-Stage and Two-Stage Object Detection. arXiv.","DOI":"10.1007\/978-3-030-58568-6_32"},{"key":"ref_165","doi-asserted-by":"crossref","unstructured":"Yao, L., Pi, R., Xu, H., Zhang, W., Li, Z., and Zhang, T. (2021, January 11\u201317). G-DetKD: Towards General Distillation Framework for Object Detectors via Contrastive and Semantic-guided Feature Imitation. Proceedings of the 2021 IEEE\/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.00357"},{"key":"ref_166","first-page":"65299","article-title":"Knowledge Diffusion for Distillation","volume":"Volume 36","author":"Oh","year":"2023","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref_167","unstructured":"Ho, J., Jain, A., and Abbeel, P. (2020). Denoising Diffusion Probabilistic Models. arXiv."},{"key":"ref_168","unstructured":"Song, J., Meng, C., and Ermon, S. (2021, January 3\u20137). Denoising Diffusion Implicit Models. Proceedings of the International Conference on Learning Representations, Vienna, Austria."},{"key":"ref_169","doi-asserted-by":"crossref","unstructured":"Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. (2022, January 18\u201324). High-Resolution Image Synthesis with Latent Diffusion Models. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01042"},{"key":"ref_170","unstructured":"He, X., Zhao, K., and Chu, X. (2019). AutoML: A Survey of the State-of-the-Art. arXiv."},{"key":"ref_171","first-page":"2497","article-title":"Monte Carlo Tree Search: A review of recent modifications and applications","volume":"56","author":"Godlewski","year":"2022","journal-title":"Artif. Intell. Rev."},{"key":"ref_172","doi-asserted-by":"crossref","unstructured":"Zheng, Z., Ye, R., Wang, P., Ren, D., Zuo, W., Hou, Q., and Cheng, M.M. (2022, January 18\u201324). Localization Distillation for Dense Object Detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00919"},{"key":"ref_173","first-page":"21002","article-title":"Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection","volume":"Volume 33","author":"Larochelle","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref_174","doi-asserted-by":"crossref","unstructured":"Qiu, H., Li, H., Wu, Q., and Shi, H. (2020, January 13\u201319). Offset Bin Classification Network for Accurate Object Detection. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01320"},{"key":"ref_175","doi-asserted-by":"crossref","unstructured":"Son, W., Na, J., Choi, J., and Hwang, W. (2021, January 11\u201317). Densely Guided Knowledge Distillation Using Multiple Teacher Assistants. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.00926"},{"key":"ref_176","doi-asserted-by":"crossref","unstructured":"Yang, L., Zhou, X., Li, X., Qiao, L., Li, Z., Yang, Z., Wang, G., and Li, X. (2023, January 1\u201310). Bridging Cross-task Protocol Inconsistency for Distillation in Dense Object Detection. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Paris, France.","DOI":"10.1109\/ICCV51070.2023.01575"},{"key":"ref_177","doi-asserted-by":"crossref","unstructured":"Guo, Q., Mu, Y., Chen, J., Wang, T., Yu, Y., and Luo, P. (2022, January 18\u201324). Scale-Equivalent Distillation for Semi-Supervised Object Detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01412"},{"key":"ref_178","unstructured":"Liu, Y.C., Ma, C.Y., He, Z., Kuo, C.W., Chen, K., Zhang, P., Wu, B., Kira, Z., and Vajda, P. (2021). Unbiased Teacher for Semi-Supervised Object Detection. arXiv, Available online: http:\/\/arxiv.org\/abs\/2102.09480."},{"key":"ref_179","unstructured":"Sohn, K., Zhang, Z., Li, C.L., Zhang, H., Lee, C.Y., and Pfister, T. (2020). A Simple Semi-Supervised Learning Framework for Object Detection. arXiv."},{"key":"ref_180","doi-asserted-by":"crossref","unstructured":"Yang, Q., Wei, X., Wang, B., Hua, X.S., and Zhang, L. (2021, January 20\u201325). Interactive Self-Training with Mean Teachers for Semi-Supervised Object Detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00588"},{"key":"ref_181","doi-asserted-by":"crossref","unstructured":"Zhou, Q., Yu, C., Wang, Z., Qian, Q., and Li, H. (2021, January 20\u201325). Instant-Teaching: An End-to-End Semi-Supervised Object Detection Framework. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00407"},{"key":"ref_182","doi-asserted-by":"crossref","unstructured":"Zhu, Y., Zhou, Q., Liu, N., Xu, Z., Ou, Z., Mou, X., and Tang, J. (2023, January 19\u201325). ScaleKD: Distilling Scale-Aware Knowledge in Small Object Detector. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.01889"},{"key":"ref_183","doi-asserted-by":"crossref","unstructured":"Cai, Z., and Vasconcelos, N. (2018, January 18\u201322). Cascade R-CNN: Delving Into High Quality Object Detection. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00644"},{"key":"ref_184","doi-asserted-by":"crossref","unstructured":"Yang, Z., Liu, S., Hu, H., Wang, L., and Lin, S. (November, January 27). RepPoints: Point Set Representation for Object Detection. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.","DOI":"10.1109\/ICCV.2019.00975"},{"key":"ref_185","doi-asserted-by":"crossref","unstructured":"He, R., Sun, S., Yang, J., Bai, S., and Qi, X. (2022, January 18\u201324). Knowledge Distillation As Efficient Pre-Training: Faster Convergence, Higher Data-Efficiency, and Better Transferability. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00895"},{"key":"ref_186","doi-asserted-by":"crossref","unstructured":"Joulin, A., Van Der Maaten, L., Jabri, A., and Vasilache, N. (2016, January 8\u201316). Learning Visual Features from Large Weakly Supervised Data. Proceedings of the European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46478-7_5"},{"key":"ref_187","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20\u201325). ImageNet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_188","doi-asserted-by":"crossref","first-page":"1956","DOI":"10.1007\/s11263-020-01316-z","article-title":"The Open Images Dataset V4","volume":"128","author":"Kuznetsova","year":"2018","journal-title":"Int. J. Comput. Vis."},{"key":"ref_189","doi-asserted-by":"crossref","unstructured":"Mahajan, D., Girshick, R., Ramanathan, V., He, K., Paluri, M., Li, Y., Bharambe, A., and van der Maaten, L. (2018, January 8\u201314). Exploring the Limits of Weakly Supervised Pretraining. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01216-8_12"},{"key":"ref_190","doi-asserted-by":"crossref","unstructured":"Sun, C., Shrivastava, A., Singh, S., and Gupta, A. (2017, January 22\u201329). Revisiting Unreasonable Effectiveness of Data in Deep Learning Era. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.97"},{"key":"ref_191","doi-asserted-by":"crossref","first-page":"164","DOI":"10.1109\/TAC.1980.1102314","article-title":"The singular value decomposition: Its computation and some applications","volume":"25","author":"Klema","year":"1980","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_192","doi-asserted-by":"crossref","unstructured":"Tomani, C., Cremers, D., and Buettner, F. (2022, January 23\u201327). Parameterized Temperature Scaling for Boosting the Expressive Power in Post-Hoc Uncertainty Calibration. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel.","DOI":"10.1007\/978-3-031-19778-9_32"},{"key":"ref_193","doi-asserted-by":"crossref","unstructured":"Liu, Z., Hu, X., and Nevatia, R. (2024, January 1\u20135). Efficient Feature Distillation for Zero-Shot Annotation Object Detection. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA.","DOI":"10.1109\/WACV57701.2024.00094"},{"key":"ref_194","doi-asserted-by":"crossref","unstructured":"Kang, M., Zhang, J., Zhang, J., Wang, X., Chen, Y., Ma, Z., and Huang, X. (2023, January 1\u20137). Alleviating Catastrophic Forgetting of Incremental Object Detection via Within-Class and Between-Class Knowledge Distillation. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Paris, France.","DOI":"10.1109\/ICCV51070.2023.01732"},{"key":"ref_195","doi-asserted-by":"crossref","first-page":"364","DOI":"10.1038\/s42256-019-0080-x","article-title":"Continual learning of context-dependent processing in neural networks","volume":"1","author":"Zeng","year":"2019","journal-title":"Nat. Mach. Intell."},{"key":"ref_196","doi-asserted-by":"crossref","first-page":"476","DOI":"10.1016\/j.neunet.2023.01.041","article-title":"Continual Object Detection: A Review of Definitions, Strategies, and Challenges","volume":"161","author":"Menezes","year":"2023","journal-title":"Neural Netw."},{"key":"ref_197","doi-asserted-by":"crossref","unstructured":"Wang, L., Liu, Y., Du, P., Ding, Z., Liao, Y., Qi, Q., Chen, B., and Liu, S. (2023, January 19\u201324). Object-Aware Distillation Pyramid for Open-Vocabulary Object Detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.01076"},{"key":"ref_198","doi-asserted-by":"crossref","unstructured":"Chang, J., Wang, S., Xu, H.M., Chen, Z., Yang, C., and Zhao, F. (2023, January 1\u20137). DETRDistill: A Universal Knowledge Distillation Framework for DETR-families. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Paris, France.","DOI":"10.1109\/ICCV51070.2023.00635"},{"key":"ref_199","doi-asserted-by":"crossref","unstructured":"Shehzadi, T., Hashmi, K.A., Liwicki, M., Stricker, D., and Afzal, M.Z. (2025). Object Detection with Transformers: A Review. Sensors, 25.","DOI":"10.3390\/s25196025"},{"key":"ref_200","doi-asserted-by":"crossref","unstructured":"Shehzadi, T., Hashmi, K.A., Stricker, D., and Afzal, M.Z. (2024, January 17\u201322). Sparse Semi-DETR: Sparse Learnable Queries for Semi-Supervised Object Detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vienna, Austria.","DOI":"10.1109\/CVPR52733.2024.00558"},{"key":"ref_201","doi-asserted-by":"crossref","unstructured":"Liu, Z., Liu, Q., Liu, T., Wang, Y., and Wen, W. (2018, January 18\u201322). Feature Distillation: DNN-Oriented JPEG Compression Against Adversarial Examples. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2019.00095"},{"key":"ref_202","doi-asserted-by":"crossref","unstructured":"Li, L., Miao, J., Shi, D., Tan, W., Ren, Y., Yang, Y., and Pu, S. (2023, January 1\u20137). Distilling DETR with Visual-Linguistic Knowledge for Open-Vocabulary Object Detection. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Paris, France.","DOI":"10.1109\/ICCV51070.2023.00598"},{"key":"ref_203","unstructured":"Jia, C., Yang, Y., Xia, Y., Chen, Y.T., Parekh, Z., Pham, H., Le, Q.V., Sung, Y.H., Li, Z., and Duerig, T. (2021, January 18\u201324). Scaling Up Visual and Vision-Language Representation Learning with Noisy Text Supervision. Proceedings of the 38th International Conference on Machine Learning (ICML), Virtual. Proceedings of Machine Learning Research."},{"key":"ref_204","unstructured":"Shehzadi, T., Stricker, D., and Afzal, M.Z. (2024). Semi-Supervised Object Detection: A Survey on Progress from CNN to Transformer. arXiv."},{"key":"ref_205","first-page":"98","article-title":"Protein active site prediction for early drug discovery and designing","volume":"13","author":"Yousaf","year":"2021","journal-title":"Int. Rev. Appl. Sci. Eng."},{"key":"ref_206","doi-asserted-by":"crossref","unstructured":"Shehzadi, T., Majid, A., Hameed, M., Farooq, A., and Yousaf, A. (2020, January 12\u201314). Intelligent predictor using cancer-related biologically information extraction from cancer transcriptomes. Proceedings of the 2020 International Symposium on Recent Advances in Electrical Engineering & Computer Sciences (RAEE & CS), Islamabad, Pakistan.","DOI":"10.1109\/RAEECS50817.2020.9265692"},{"key":"ref_207","doi-asserted-by":"crossref","unstructured":"Shehzadi, T., Hashmi, K.A., Pagani, A., Liwicki, M., Stricker, D., and Afzal, M.Z. (2022). Mask-Aware Semi-Supervised Object Detection in Floor Plans. Appl. Sci., 12.","DOI":"10.20944\/preprints202209.0025.v1"},{"key":"ref_208","doi-asserted-by":"crossref","unstructured":"Shehzadi, T., Ifza, I., Stricker, D., and Afzal, M.Z. (2025, January 15\u201317). FD-SSD: Semi-supervised Detection of Bone Fenestration and Dehiscence in Intraoral Images. Proceedings of the Medical Image Understanding and Analysis (MIUA), Leeds, UK.","DOI":"10.1007\/978-3-031-98691-8_1"},{"key":"ref_209","doi-asserted-by":"crossref","unstructured":"Xie, Q., Hovy, E.H., Luong, M.T., and Le, Q.V. (2020, January 13\u201319). Self-Training with Noisy Student Improves ImageNet Classification. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01070"},{"key":"ref_210","doi-asserted-by":"crossref","unstructured":"Zhu, Y., and Wang, Y. (2021, January 10\u201317). Student Customized Knowledge Distillation: Bridging the Gap Between Student and Teacher. Proceedings of the 2021 IEEE\/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.00501"},{"key":"ref_211","doi-asserted-by":"crossref","unstructured":"Beyer, L., Zhai, X., Royer, A., Markeeva, L., Anil, R., and Kolesnikov, A. (2022, January 18\u201324). Knowledge distillation: A good teacher is patient and consistent. Proceedings of the 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01065"},{"key":"ref_212","unstructured":"Li, Z., Li, X., Yang, L., Zhao, B., Song, R., Luo, L., Li, J., and Yang, J. (2023, January 7\u201314). Curriculum Temperature for Knowledge Distillation. Proceedings of the AAAI Conference on Artificial Intelligence, Washington, DC, USA."},{"key":"ref_213","unstructured":"Qiu, Z., Ma, X., Yang, K., Liu, C., Hou, J., Yi, S., and Ouyang, W. (2023, January 1\u20135). Better Teacher Better Student: Dynamic Prior Knowledge for Knowledge Distillation. Proceedings of the International Conference on Learning Representations (ICLR), Kigali, Rwanda."},{"key":"ref_214","doi-asserted-by":"crossref","unstructured":"Park, J., and No, A. (2022, January 23\u201327). Prune Your Model Before Distill It. Proceedings of the Computer Vision\u2014ECCV 2022, Tel Aviv, Israel.","DOI":"10.1007\/978-3-031-20083-0_8"},{"key":"ref_215","unstructured":"Wang, X., Li, Y., Chen, Z., and Zhao, H. (2025, January 16\u201321). SemiTabDETR: End-to-End Semi-supervised Table Detection with Transformer-Based Enhanced Query Approach. Proceedings of the Document Analysis and Recognition\u2014ICDAR, Wuhan, China."},{"key":"ref_216","unstructured":"Shehzadi, T., Stricker, D., and Afzal, M.Z. (September, January 30). A Hybrid Approach for Document Layout Analysis in Document Images. Proceedings of the International Conference on Document Analysis and Recognition (ICDAR), Athens, Greece."},{"key":"ref_217","doi-asserted-by":"crossref","unstructured":"Shehzadi, T., Hashmi, K.A., Stricker, D., Liwicki, M., and Afzal, M.Z. (2023, January 21\u201326). Towards End-to-End Semi-Supervised Table Detection with Deformable Transformer. Proceedings of the Document Analysis and Recognition\u2014ICDAR 2023, San Jos\u00e9, CA, USA.","DOI":"10.1007\/978-3-031-41679-8_4"},{"key":"ref_218","unstructured":"Shehzadi, T., Ifza, I., Stricker, D., and Afzal, M.Z. (2025, January 19\u201323). DocSemi: Efficient Document Layout Analysis with Guided Queries. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Honolulu, HI, USA."},{"key":"ref_219","unstructured":"Shehzadi, T., Ifza, I., Stricker, D., and Afzal, M.Z. (2025, January 19\u201325). Efficient Additive Attention for Transformer-based Semi-supervised Document Layout Analysis. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Honolulu, HI, USA."},{"key":"ref_220","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Maire, M., Belongie, S.J., Hays, J., Perona, P., Ramanan, D., Doll\u00e1r, P., and Zitnick, C.L. (2014, January 6\u201312). Microsoft COCO: Common Objects in Context. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10602-1_48"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/1\/292\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,5]],"date-time":"2026-01-05T12:45:25Z","timestamp":1767617125000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/1\/292"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1,2]]},"references-count":220,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,1]]}},"alternative-id":["s26010292"],"URL":"https:\/\/doi.org\/10.3390\/s26010292","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1,2]]}}}