{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,10]],"date-time":"2026-07-10T02:23:17Z","timestamp":1783650197341,"version":"3.55.0"},"reference-count":40,"publisher":"MDPI AG","issue":"23","license":[{"start":{"date-parts":[[2023,11,24]],"date-time":"2023-11-24T00:00:00Z","timestamp":1700784000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"China Post-Doctoral Science Foundation","award":["2021T140529"],"award-info":[{"award-number":["2021T140529"]}]},{"name":"China Post-Doctoral Science Foundation","award":["ZYTS23170"],"award-info":[{"award-number":["ZYTS23170"]}]},{"name":"Fundamental Research Funds for the Central Universities","award":["2021T140529"],"award-info":[{"award-number":["2021T140529"]}]},{"name":"Fundamental Research Funds for the Central Universities","award":["ZYTS23170"],"award-info":[{"award-number":["ZYTS23170"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In the production process of metal industrial products, the deficiencies and limitations of existing technologies and working conditions can have adverse effects on the quality of the final products, making surface defect detection particularly crucial. However, collecting a sufficient number of samples of defective products can be challenging. Therefore, treating surface defect detection as a semi-supervised problem is appropriate. In this paper, we propose a method based on a Transformer with pruned and merged multi-scale masked feature fusion. This method learns the semantic context from normal samples. We incorporate the Vision Transformer (ViT) into a generative adversarial network to jointly learn the generation in the high-dimensional image space and the inference in the latent space. We use an encoder\u2013decoder neural network with long skip connections to capture information between shallow and deep layers. During training and testing, we design block masks of different scales to obtain rich semantic context information. Additionally, we introduce token merging (ToMe) into the ViT to improve the training speed of the model without affecting the training results. In this paper, we focus on the problems of rust, scratches, and other defects on the metal surface. We conduct various experiments on five metal industrial product datasets and the MVTec AD dataset to demonstrate the superiority of our method.<\/jats:p>","DOI":"10.3390\/s23239381","type":"journal-article","created":{"date-parts":[[2023,11,24]],"date-time":"2023-11-24T03:54:35Z","timestamp":1700798075000},"page":"9381","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["Metal Surface Defect Detection Based on a Transformer with Multi-Scale Mask Feature Fusion"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-9601-5673","authenticated-orcid":false,"given":"Lin","family":"Zhao","sequence":"first","affiliation":[{"name":"School of Electrical and Control Engineering, Shaanxi University of Science and Technology, Xi\u2019an 710021, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yu","family":"Zheng","sequence":"additional","affiliation":[{"name":"School of Cyber Engineering, Xidian University, Xi\u2019an 710126, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tao","family":"Peng","sequence":"additional","affiliation":[{"name":"School of Electrical and Control Engineering, Shaanxi University of Science and Technology, Xi\u2019an 710021, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Enrang","family":"Zheng","sequence":"additional","affiliation":[{"name":"School of Electrical and Control Engineering, Shaanxi University of Science and Technology, Xi\u2019an 710021, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,11,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2679","DOI":"10.1109\/TIM.2018.2868490","article-title":"Deep Architecture for High-Speed Railway Insulator Surface Defect Detection: Denoising Autoencoder With Multitask Learning","volume":"68","author":"Kang","year":"2019","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"656","DOI":"10.1109\/TIM.2018.2853958","article-title":"A Coarse-to-Fine Model for Rail Surface Defect Detection","volume":"68","author":"Yu","year":"2019","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Chen, X., Lv, J., Fang, Y., and Du, S. (2022). Online Detection of Surface Defects Based on Improved YOLOV3. Sensors, 22.","DOI":"10.3390\/s22030817"},{"key":"ref_4","first-page":"53048","article-title":"Aero-Engine Blade Defect Detection: A Systematic Review of Deep Learning Models","volume":"11","author":"Abdulrahman","year":"2023","journal-title":"IEEE Access"},{"key":"ref_5","first-page":"1992415","article-title":"Aeroengine Blade Surface Defect Detection System Based on Improved Faster RCNN","volume":"2023","author":"Liu","year":"2023","journal-title":"Int. J. Intell. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2514411","DOI":"10.1109\/TIM.2023.3276026","article-title":"Surface Defect Detection of Aeroengine Blades Based on Cross-Layer Semantic Guidance","volume":"72","author":"Song","year":"2023","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_7","first-page":"3512712","article-title":"Tiny Defect Detection in High-Resolution Aero-Engine Blade Images via a Coarse-to-Fine Framework","volume":"70","author":"Li","year":"2021","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"103750","DOI":"10.1016\/j.compind.2022.103750","article-title":"Anomaly detection in smart agriculture systems","volume":"143","author":"Catalano","year":"2022","journal-title":"Comput. Ind."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Roth, K., Pemula, L., Zepeda, J., Sch\u00f6lkopf, B., Brox, T., and Gehler, P.V. (2022, January 18\u201324). Towards Total Recall in Industrial Anomaly Detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01392"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"114489","DOI":"10.1109\/ACCESS.2023.3324717","article-title":"Multi-Branch Deep Fusion Network-Based Automatic Detection of Weld Defects Using Non-Destructive Ultrasonic Test","volume":"11","author":"Kim","year":"2023","journal-title":"IEEE Access"},{"key":"ref_11","unstructured":"Chalapathy, R., Menon, A.K., and Chawla, S. (2018). Anomaly Detection using One-Class Neural Networks. arXiv."},{"key":"ref_12","first-page":"362","article-title":"Anomaly Detection in Activities of Daily Living Using One-Class Support Vector Machine","volume":"Volume 840","author":"Lotfi","year":"2018","journal-title":"Proceedings of the Advances in Computational Intelligence Systems\u2014Contributions Presented at the 18th UK Workshop on Computational Intelligence"},{"key":"ref_13","first-page":"146","article-title":"Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery","volume":"Volume 10265","author":"Niethammer","year":"2017","journal-title":"Proceedings of the Information Processing in Medical Imaging\u201425th International Conference, IPMI 2017"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Collin, A.S., and De Vleeschouwer, C. (2021, January 10\u201315). Improved anomaly detection by training an autoencoder with skip connections on images corrupted with Stain-shaped noise. Proceedings of the 2020 25th International Conference on Pattern Recognition (ICPR), Milan, Italy.","DOI":"10.1109\/ICPR48806.2021.9412842"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Youkachen, S., Ruchanurucks, M., Phatrapomnant, T., and Kaneko, H. (2019, January 25\u201327). Defect Segmentation of Hot-rolled Steel Strip Surface by using Convolutional Auto-Encoder and Conventional Image processing. Proceedings of the 2019 10th International Conference of Information and Communication Technology for Embedded Systems (IC-ICTES), Bangkok, Thailand.","DOI":"10.1109\/ICTEmSys.2019.8695928"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1016\/j.media.2019.01.010","article-title":"f-AnoGAN: Fast unsupervised anomaly detection with generative adversarial networks","volume":"54","author":"Schlegl","year":"2019","journal-title":"Med. Image Anal."},{"key":"ref_17","first-page":"622","article-title":"GANomaly: Semi-supervised Anomaly Detection via Adversarial Training","volume":"Volume 11363","author":"Jawahar","year":"2018","journal-title":"Proceedings of the Computer Vision\u2014ACCV 2018\u201414th Asian Conference on Computer Vision"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Ak\u00e7ay, S., Abarghouei, A.A., and Breckon, T.P. (2019, January 14\u201319). Skip-GANomaly: Skip Connected and Adversarially Trained Encoder-Decoder Anomaly Detection. Proceedings of the International Joint Conference on Neural Networks, IJCNN 2019, IEEE, Budapest, Hungary.","DOI":"10.1109\/IJCNN.2019.8851808"},{"key":"ref_19","unstructured":"Yan, X., Zhang, H., Xu, X., Hu, X., and Heng, P. (2021, January 2\u20139). Learning Semantic Context from Normal Samples for Unsupervised Anomaly Detection. Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"46717","DOI":"10.1109\/ACCESS.2022.3171559","article-title":"AnoViT: Unsupervised Anomaly Detection and Localization with Vision Transformer-Based Encoder-Decoder","volume":"10","author":"Lee","year":"2022","journal-title":"IEEE Access"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Mishra, P., Verk, R., Fornasier, D., Piciarelli, C., and Foresti, G.L. (2021, January 20\u201323). VT-ADL: A Vision Transformer Network for Image Anomaly Detection and Localization. Proceedings of the 30th IEEE International Symposium on Industrial Electronics, ISIE 2021, Kyoto, Japan.","DOI":"10.1109\/ISIE45552.2021.9576231"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Zavrtanik, V., Kristan, M., and Skocaj, D. (2021, January 10\u201317). DR\u00c6M\u2014A discriminatively trained reconstruction embedding for surface anomaly detection. Proceedings of the 2021 IEEE\/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.00822"},{"key":"ref_23","unstructured":"Wang, G., Han, S., Ding, E., and Huang, D. (2021, January 22\u201325). Student-Teacher Feature Pyramid Matching for Anomaly Detection. Proceedings of the 32nd British Machine Vision Conference 2021, BMVC 2021, Online."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Deng, H., and Li, X. (2022, January 18\u201324). Anomaly Detection via Reverse Distillation from One-Class Embedding. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00951"},{"key":"ref_25","first-page":"475","article-title":"PaDiM: A Patch Distribution Modeling Framework for Anomaly Detection and Localization","volume":"Volume 12664","author":"Bimbo","year":"2020","journal-title":"Proceedings of the Pattern Recognition. ICPR International Workshops and Challenges"},{"key":"ref_26","unstructured":"Cohen, N., and Hoshen, Y. (2020). Sub-Image Anomaly Detection with Deep Pyramid Correspondences. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Bao, F., Nie, S., Xue, K., Cao, Y., Li, C., Su, H., and Zhu, J. (2023, January 17\u201324). All are Worth Words: A ViT Backbone for Diffusion Models. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.02171"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Bolya, D., and Hoffman, J. (2023, January 17\u201324). Token Merging for Fast Stable Diffusion. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023\u2014Workshops, Vancouver, BC, Canada.","DOI":"10.1109\/CVPRW59228.2023.00484"},{"key":"ref_29","unstructured":"Ranzato, M., Beygelzimer, A., Dauphin, Y.N., Liang, P., and Vaughan, J.W. (2021, January 6\u201314). DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification. Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, Virtual."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Meng, L., Li, H., Chen, B., Lan, S., Wu, Z., Jiang, Y., and Lim, S. (2022, January 18\u201324). AdaViT: Adaptive Vision Transformers for Efficient Image Recognition. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01199"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Yin, H., Vahdat, A., \u00c1lvarez, J.M., Mallya, A., Kautz, J., and Molchanov, P. (2022, January 18\u201324). A-ViT: Adaptive Tokens for Efficient Vision Transformer. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01054"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"620","DOI":"10.1007\/978-3-031-20083-0_37","article-title":"SPViT: Enabling Faster Vision Transformers via Latency-Aware Soft Token Pruning","volume":"Volume 13671","author":"Avidan","year":"2022","journal-title":"Proceedings of the Computer Vision\u2014ECCV 2022\u201417th European Conference"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1038","DOI":"10.1007\/s11263-020-01400-4","article-title":"The MVTec Anomaly Detection Dataset: A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection","volume":"129","author":"Bergmann","year":"2021","journal-title":"Int. J. Comput. Vis."},{"key":"ref_34","unstructured":"Zhuang, Z., Liu, M., Cutkosky, A., and Orabona, F. (2022). Understanding AdamW through Proximal Methods and Scale-Freeness. arXiv."},{"key":"ref_35","unstructured":"Gottlob, G., and Walsh, T. (2003, January 9\u201315). AUC: A Statistically Consistent and more Discriminating Measure than Accuracy. Proceedings of the IJCAI-03, Eighteenth International Joint Conference on Artificial Intelligence, Acapulco, Mexico."},{"key":"ref_36","unstructured":"Allan, J., Croft, W.B., de Vries, A.P., and Zhai, C. (2015, January 27\u201330). Estimating the Uncertainty of Average F1 Scores. Proceedings of the 2015 International Conference on The Theory of Information Retrieval, ICTIR 2015, Northampton, MA, USA."},{"key":"ref_37","unstructured":"Gao, P., Ma, T., Li, H., Lin, Z., Dai, J., and Qiao, Y. (2022). ConvMAE: Masked Convolution Meets Masked Autoencoders. arXiv."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"78446","DOI":"10.1109\/ACCESS.2022.3193699","article-title":"CFA: Coupled-Hypersphere-Based Feature Adaptation for Target-Oriented Anomaly Localization","volume":"10","author":"Lee","year":"2022","journal-title":"IEEE Access"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Rudolph, M., Wehrbein, T., Rosenhahn, B., and Wandt, B. (2022, January 3\u20138). Fully Convolutional Cross-Scale-Flows for Image-based Defect Detection. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision, WACV 2022, Waikoloa, HI, USA.","DOI":"10.1109\/WACV51458.2022.00189"},{"key":"ref_40","unstructured":"Yu, J., Zheng, Y., Wang, X., Li, W., Wu, Y., Zhao, R., and Wu, L. (2021). FastFlow: Unsupervised Anomaly Detection and Localization via 2D Normalizing Flows. arXiv."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/23\/9381\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:29:06Z","timestamp":1760131746000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/23\/9381"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,24]]},"references-count":40,"journal-issue":{"issue":"23","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["s23239381"],"URL":"https:\/\/doi.org\/10.3390\/s23239381","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,24]]}}}