{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,31]],"date-time":"2026-01-31T05:48:45Z","timestamp":1769838525009,"version":"3.49.0"},"reference-count":29,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2025,3,28]],"date-time":"2025-03-28T00:00:00Z","timestamp":1743120000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>There are two problems with traditional knowledge distillation methods in industrial anomaly detection: first, traditional methods mostly use feature alignment between the same layers. The second is that similar or even identical structures are usually used to build teacher-student models, thus limiting the ability to represent anomalies in multiple ways. To address these issues, this work proposes a Hierarchical Knowledge Transfer (HKT) framework for detecting industrial surface anomalies. First, HKT utilizes the deep knowledge of the highest feature layer in the teacher\u2019s network to guide student learning at every level, thus enabling cross-layer interactions. Multiple projectors are built inside the model to facilitate the teacher in transferring knowledge to each layer of the student. Second, the teacher-student structural symmetry is decoupled by embedding Convolutional Block Attention Modules (CBAM) in the student network. Finally, based on HKT, a more powerful anomaly detection model, HKT+, is developed. By adding two additional convolutional layers to the teacher and student networks of HKT, HKT+ achieves enhanced detection capabilities at the cost of a relatively small increase in model parameters. Experiments on the MVTec AD and BeanTech AD(BTAD) datasets show that HKT+ achieves state-of-the-art performance with average area under the receiver operating characteristic curve (AUROC) scores of 98.69% and 94.58%, respectively, which outperforms most current state-of-the-art methods.<\/jats:p>","DOI":"10.3390\/jimaging11040102","type":"journal-article","created":{"date-parts":[[2025,3,31]],"date-time":"2025-03-31T01:59:36Z","timestamp":1743386376000},"page":"102","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Hierarchical Knowledge Transfer: Cross-Layer Distillation for Industrial Anomaly Detection"],"prefix":"10.3390","volume":"11","author":[{"given":"Junning","family":"Xu","sequence":"first","affiliation":[{"name":"College of Electronics and Information Engineering, Shanghai University of Electric Power, Shanghai 201306, China"}]},{"given":"Sanxin","family":"Jiang","sequence":"additional","affiliation":[{"name":"College of Electronics and Information Engineering, Shanghai University of Electric Power, Shanghai 201306, China"}]}],"member":"1968","published-online":{"date-parts":[[2025,3,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"104","DOI":"10.1007\/s11633-023-1459-z","article-title":"Deep industrial image anomaly detection: A survey","volume":"21","author":"Liu","year":"2023","journal-title":"Mach. Intell. Res."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Yang, Q., and Guo, R. (2024). An Unsupervised Method for Industrial Image Anomaly Detection with Vision Transformer-Based Autoencoder. Sensors, 24.","DOI":"10.3390\/s24082440"},{"key":"ref_3","first-page":"1","article-title":"Deep learning for unsupervised anomaly localization in industrial images: A survey","volume":"71","author":"Tao","year":"2022","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_4","unstructured":"Liu, J., Xie, G., Chen, R., Li, X., Wang, J., Liu, Y., Wang, C., and Zheng, F. (2023). Real3d-ad: A dataset of point cloud anomaly detection. arXiv."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Roth, K., Pemula, L., Zepeda, J., Scholkopf, B., Brox, T., and Gehler, P. (2021, January 18\u201324). Towards total recall in industrial anomaly detection. Proceedings of the 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01392"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"3048","DOI":"10.1109\/TPAMI.2021.3055564","article-title":"Knowledge distillation and student teacher learning for visual intelligence: A review and new outlooks","volume":"44","author":"Wang","year":"2020","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Li, M., Lyu, X., and Guo, X. (2023, January 25\u201327). Knowledge distillation with attention mechanism for anomaly detection. Proceedings of the International Conference on Signal Processing and Computer Science, Guilin, China.","DOI":"10.1117\/12.3012320"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Li, Z., Ge, Y., Wang, X., Yue, X., and Meng, L. (2023, January 4\u20137). Industrial anomaly detection via teacher student network. Proceedings of the 2023 International Conference on Advanced Mechatronic Systems (ICAMechS), Melbourne, Australia.","DOI":"10.1109\/ICAMechS59878.2023.10272951"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Cai, F., and Xia, S. (2023, January 23\u201325). Mixed distillation for unsupervised anomaly detection. Proceedings of the 2023 18th International Conference on Machine Vision and Applications (MVA), Hamamatsu, Japan.","DOI":"10.23919\/MVA57639.2023.10215597"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Bergmann, P., Fauser, M., Sattlegger, D., and Steger, C. (2019, January 13\u201319). Uninformed students: Student-teacher anomaly detection with discriminative latent embeddings. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2020, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00424"},{"key":"ref_11","unstructured":"Wang, G., Han, S., Ding, E., and Huang, D. (2021, January 22\u201325). Student-teacher feature pyramid matching for anomaly detection. Proceedings of the British Machine Vision Conference, Online."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Deng, H., and Li, X. (2022, January 18\u201324). Anomaly detection via reverse distillation from one-class embedding. Proceedings of the 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00951"},{"key":"ref_13","unstructured":"Nguyen, V.-D., Bach, H.H., and Trang, L.H. (2023, January 3\u20135). An improved reverse distillation model for unsupervised anomaly detection. Proceedings of the 2023 17th International Conference on Ubiquitous Information Management and Communication (IMCOM), Seoul, Republic of Korea."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Batzner, K., Heckler, L., and K\u00f6nig, R. (2024, January 3\u20138). Efficientad: Accurate visual anomaly detection at millisecond-level latencies. Proceedings of the 2024 IEEE\/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, Hawaii, USA.","DOI":"10.1109\/WACV57701.2024.00020"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Woo, S., Park, J., Lee, J.-Y., and Kweon, I.S. (2018, January 8\u201314). Cbam: Convolutional block attention module. Proceedings of the European conference on computer vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01234-2_1"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1038","DOI":"10.1007\/s11263-020-01400-4","article-title":"Carsten Steger: The MVTec Anomaly Detection Dataset: A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection","volume":"129","author":"Bergmann","year":"2021","journal-title":"Int. J. Comput. Vis."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Bergmann, P., Fauser, M., and Sattlegger, D. (2019, January 15\u201320). Carsten Steger: MVTec AD\u2014A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00982"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Mishra, P., Verk, R., Fornasier, D., Piciarelli, C., and Foresti, G.L. (2021, January 20\u201323). Vt-adl: A vision transformer network for image anomaly detection and localization. Proceedings of the 2021 IEEE 30th International Symposium on Industrial Electronics (ISIE), Kyoto, Japan.","DOI":"10.1109\/ISIE45552.2021.9576231"},{"key":"ref_19","unstructured":"Vaswani, A., Shazeer, N.M., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017, January 4\u20139). Attention is all you need. Proceedings of the Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_20","unstructured":"Silva, A.L.V.E., Sim\u00f5es, F., Kowerko, D., Schlosser, T., Battisti, F., and Teichrieb, V. (2023, January 3\u20138). Attention modules improve image level anomaly detection for industrial inspection: A differnet case study. Proceedings of the 2024 IEEE\/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L., Li, K., and Li, F. (2009, January 20\u201325). Imagenet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2009), Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_22","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. CoRR."},{"key":"ref_23","unstructured":"Ling, C.X., Huang, J., and Zhang, H. (2003, January 9\u201315). Auc: A statistically consistent and more discriminating measure than accuracy. Proceedings of the International Joint Conference on Artificial Intelligence, Acapulco, Mexico."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Zavrtanik, V., Kristan, M., and Skocaj, D. (2021, January 10\u201317). Draem\u2014A discriminatively trained reconstruction embedding for surface anomaly detection. Proceedings of the 2021 IEEE\/CVF International Conference on Computer Vision (ICCV), Montreal, Canada.","DOI":"10.1109\/ICCV48922.2021.00822"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Salehi, M., Sadjadi, N., Baselizadeh, S., Rohban, M.H., and Rabiee, H.R. (2020, January 19\u201325). Multiresolution knowledge distillation for anomaly detection. Proceedings of the 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Online.","DOI":"10.1109\/CVPR46437.2021.01466"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Thomine, S., Snoussi, H., and Soua, M. (2023, January 10\u201312). Fable: Fabric anomaly detection automation process. Proceedings of the 2023 International Conference on Control, Automation and Diagnosis (ICCAD), Rome, Italy.","DOI":"10.1109\/ICCAD57653.2023.10152345"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Rudolph, M., Wehrbein, T., Rosenhahn, B., and Wandt, B. (2022, January 3\u20138). Fully Convolutional Cross-Scale-Flows for Image-based Defect Detection. Proceedings of the 2022 IEEE\/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA.","DOI":"10.1109\/WACV51458.2022.00189"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Gudovskiy, D., Ishizaka, S., and Kozuka, K. (2022, January 3\u20138). CFLOW-AD: Real-Time Unsupervised Anomaly Detection with Localization via Conditional Normalizing Flows. Proceedings of the 2022 IEEE\/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA.","DOI":"10.1109\/WACV51458.2022.00188"},{"key":"ref_29","unstructured":"Wang, P., Yao, H., and Yu, W. (2023). SCL-VI: Self-supervised context learning for visual inspection of industrial defects. arXiv."}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/11\/4\/102\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:04:44Z","timestamp":1760029484000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/11\/4\/102"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,28]]},"references-count":29,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2025,4]]}},"alternative-id":["jimaging11040102"],"URL":"https:\/\/doi.org\/10.3390\/jimaging11040102","relation":{},"ISSN":["2313-433X"],"issn-type":[{"value":"2313-433X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3,28]]}}}