{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,4]],"date-time":"2026-03-04T16:45:03Z","timestamp":1772642703327,"version":"3.50.1"},"reference-count":38,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2024,4,25]],"date-time":"2024-04-25T00:00:00Z","timestamp":1714003200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62071499"],"award-info":[{"award-number":["62071499"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>The detection of cracks is extremely important for maintenance of concrete structures. Deep learning-based segmentation models have achieved high accuracy in crack segmentation. However, mainstream crack segmentation models have very high computational complexity, and therefore cannot be used in portable crack detection equipment. To address this problem, a knowledge distilling structure is designed by us. In this structure, a large teacher model named TBUNet is proposed to transfer crack knowledge to a student model with symmetry structure named ULNet. In the TBUNet, stacked transformer modules are used to capture dependency relationships between different crack positions in feature maps and achieve contextual awareness. In the ULNet, only a tiny U-Net with light-weighted parameters is used to maintain very low computational complexity. In addition, a mixed loss function is designed to ensure detail and global features extracted by the teacher model are consistent with those of the student model. Our designed experiments demonstrate that the ULNet can achieve accuracies of 96.2%, 87.6%, and 75.3%, and recall of 97.1%, 88.5%, and 76.2% on the Cracktree200, CRACK500, and MICrack datasets, respectively, which is 4\u20136% higher than most crack segmentation models. However, the ULNet only has a model size of 1 M, which is suitable for use in portable crack detection equipment.<\/jats:p>","DOI":"10.3390\/sym16050520","type":"journal-article","created":{"date-parts":[[2024,4,26]],"date-time":"2024-04-26T05:13:41Z","timestamp":1714108421000},"page":"520","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Distilling Knowledge from a Transformer-Based Crack Segmentation Model to a Light-Weighted Symmetry Model with Mixed Loss Function for Portable Crack Detection Equipment"],"prefix":"10.3390","volume":"16","author":[{"given":"Xiaohu","family":"Zhang","sequence":"first","affiliation":[{"name":"School of Electronic and Communication Engineering, Sun Yat-sen University, Guangzhou 510275, China"}]},{"given":"Haifeng","family":"Huang","sequence":"additional","affiliation":[{"name":"School of Electronic and Communication Engineering, Sun Yat-sen University, Guangzhou 510275, China"}]}],"member":"1968","published-online":{"date-parts":[[2024,4,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1016\/j.ssci.2019.04.020","article-title":"Concrete crack detection method based on optical fiber sensing network and microbending principle","volume":"117","author":"Wu","year":"2019","journal-title":"Saf. Sci."},{"key":"ref_2","unstructured":"Bradski, G., and Daebler, A. (2008). Learning OpenCV: Computer Vision with OpenCV Library, University of Arizona."},{"key":"ref_3","unstructured":"Meghana, R.K., Apoorva, S., and Chitkara, Y. (2018, January 15\u201316). Inspection, Identification and Repair Monitoring of Cracked Concrete Structure\u2014An Application of Image Processing. Proceedings of the 2018 3rd International Conference on Communication and Electronics Systems (ICCES), Coimbatore, India."},{"key":"ref_4","unstructured":"Dorafshan, S., Maguire, M., and Thomas, R.J. (2018). SDNET2018: A Concrete Crack Image Dataset for Machine Learning Applications, Utah State University."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Liu, J. (2023, January 10\u201312). Road Crack Detection Using HDD LOSS and Dual Attention Module with DeepLabv3+. Proceedings of the 2023 3rd International Conference on Digital Society and Intelligent Systems (DSInS), Chengdu, China.","DOI":"10.1109\/DSInS60115.2023.10455258"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Zhou, S., Wang, Q., Wu, H., Wang, Q., Meng, Y., and Shen, T. (August, January 30). ASSA-UNet: An Efficient UNet-Based Network for Chip Internal Defect Detection. Proceedings of the 2023 11th International Conference on Information Systems and Computing Technology (ISCTech), Qingdao, China.","DOI":"10.1109\/ISCTech60480.2023.00036"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"332","DOI":"10.1049\/itr2.12146","article-title":"RAO-UNet: A residual attention and octave UNet for road crack detection via balance loss","volume":"16","author":"Fan","year":"2022","journal-title":"IET Intell. Transp. Syst."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Zheng, S., Lu, J., Zhao, H., Zhu, X., Luo, Z., Wang, Y., Fu, Y., Feng, J., Xiang, T., and Torr, P.H. (2021, January 20\u201325). Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00681"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"14920","DOI":"10.3934\/mbe.2023668","article-title":"Multi-scale feature fusion for pavement crack detection based on Transformer","volume":"20","author":"Yang","year":"2023","journal-title":"Math. Biosci. Eng."},{"key":"ref_10","unstructured":"Aso, R., Shiota, S., and Kiya, H. (2023, January 10\u201313). Enhanced Security with Encrypted Vision Transformer in Federated Learning. Proceedings of the 2023 IEEE 12th Global Conference on Consumer Electronics (GCCE), Nara, Japan."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2255006","DOI":"10.1142\/S0218001422550060","article-title":"Enhanced Edge Detection for 3D Crack Segmentation and Depth Measurement with Laser Data","volume":"36","author":"Cao","year":"2022","journal-title":"Int. J. Pattern Recognit. Artif. Intell."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"104712","DOI":"10.1016\/j.autcon.2022.104712","article-title":"Unifying transformer and convolution for dam crack detection","volume":"147","author":"Zhang","year":"2023","journal-title":"Autom. Constr."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"487","DOI":"10.1109\/TMC.2021.3073969","article-title":"RF-Based Human Activity Recognition Using Signal Adapted Convolutional Neural Network","volume":"22","author":"Chen","year":"2021","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"4706315","DOI":"10.1109\/TGRS.2022.3165209","article-title":"DisOptNet: Distilling Semantic Knowledge From Optical Images for Weather-Independent Building Segmentation","volume":"60","author":"Kang","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"54564","DOI":"10.1109\/ACCESS.2020.2981561","article-title":"Crack Detection of Concrete Pavement With Cross-Entropy Loss Function and Improved VGG16 Network Model","volume":"8","author":"Qu","year":"2020","journal-title":"IEEE Access"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2180638","DOI":"10.1080\/10298436.2023.2180638","article-title":"A global context and pyramidal scale guided convolutional neural network for pavement crack detection","volume":"24","author":"Maurya","year":"2023","journal-title":"Int. J. Pavement Eng."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Mercioni, M.A., and Holban, S. (2020, January 5\u20136). P-Swish: Activation Function with Learnable Parameters Based on Swish Activation Function in Deep Learning. Proceedings of the 2020 International Symposium on Electronics and Telecommunications (ISETC), Timi\u0219oara, Romania.","DOI":"10.1109\/ISETC50328.2020.9301059"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"105816","DOI":"10.1016\/j.engappai.2022.105816","article-title":"Fast brain tumor detection using adaptive stochastic gradient descent on shared-memory parallel environment","volume":"120","author":"Qin","year":"2023","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_19","first-page":"15263","article-title":"Fully convolutional networks for semantic segmentation","volume":"25","author":"Long","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_20","unstructured":"Jenkins, M.D., Carr, T.A., Iglesias, M.I., Buggy, T., and Morison, G. (2018, January 3\u20137). A deep convolutional neural network for semantic pixel-wise segmentation of road and pavement surface cracks. Proceedings of the 2018 26th European Signal Processing Conference (EUSIPCO), Rome, Italy."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Nguyen, N.T.H., Le, T.H., Perry, S., and Nguyen, T.T. (2018, January 6\u20137). Pavement crack detection using convolutional neural network. Proceedings of the International Symposium on Information and Communication Technology, Da Nang, Vietnam.","DOI":"10.1145\/3287921.3287949"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Di Benedetto, A., Fiani, M., and Gujski, L.M. (2023). U-Net-Based CNN Architecture for Road Crack Segmentation. Infrastructures, 8.","DOI":"10.3390\/infrastructures8050090"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Deng, Z. (2022). Proceedings of 2021 Chinese Intelligent Automation Conference, Springer. Lecture Notes in Electrical Engineering.","DOI":"10.1007\/978-981-16-6372-7"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"22135","DOI":"10.1109\/TITS.2021.3095507","article-title":"Crackw-net: A novel pavement crack image segmentation convolutional neural network","volume":"23","author":"Han","year":"2022","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Zhang, C., Jiang, W., and Zhao, Q. (2021). Semantic segmentation of aerial imagery via split-attention networks with disentangled nonlocal and edge supervision. Remote Sens., 13.","DOI":"10.3390\/rs13061176"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"18392","DOI":"10.1109\/TITS.2022.3158670","article-title":"Dma-net: Deeplab with multi-scale attention for pavement crack segmentation","volume":"23","author":"Sun","year":"2022","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Jun, F., Li, J., Shi, Y., Zhao, Y., and Zhang, C. (2022, January 22\u201324). Acau-net: Atrous convolution and attention u-net model for pavement crack segmentation. Proceedings of the 2022 International Conference on Computer Engineering and Artificial Intelligence (ICCEAI), Shijiazhuang, China.","DOI":"10.1109\/ICCEAI55464.2022.00120"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Li, J., Liu, Y., Zhang, Y., and Zhang, Y. (2021). Cascaded attention denseunet (cadunet) for road extraction from very-high-resolution images. Int. J. Geo-Inf., 10.","DOI":"10.3390\/ijgi10050329"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Wang, Q., Wu, B., Zhu, P., Li, P., and Hu, Q. (2020, January 13\u201319). Eca-net: Efficient channel attention for deep convolutional neural networks. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01155"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Gao, Z., Peng, B., Li, T., and Gou, C. (2019, January 14\u201319). Generative adversarial networks for road crack image segmentation. Proceedings of the 2019 International Joint Conference on Neural Networks (IJCNN), Budapest, Hungary.","DOI":"10.1109\/IJCNN.2019.8851910"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"115718","DOI":"10.1016\/j.eswa.2021.115718","article-title":"Two-stage convolutional neural network for road crack detection and segmentation","volume":"186","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Zhang, X., and Huang, H. (2023). PSNet: Parallel-Convolution-Based U-Net for Crack Detection with Self-Gated Attention Block. Appl. Sci., 13.","DOI":"10.3390\/app13179875"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Zhang, X., and Huang, H. (2023). PHCNet: Pyramid Hierarchical-Convolution-Based U-Net for Crack Detection with Mixed Global Attention Module and Edge Feature Extractor. Appl. Sci., 13.","DOI":"10.3390\/app131810263"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Emara, T., Munim HE, A.E., and Abbas, H.M. (2019, January 2\u20134). LiteSeg: A Novel Lightweight ConvNet for Semantic Segmentation. Proceedings of the 2019 Digital Image Computing: Techniques and Applications (DICTA), Perth, Australia.","DOI":"10.1109\/DICTA47822.2019.8945975"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Wang, B., and Li, H.S. (2021, January 24\u201326). Lane detection algorithm based on MoblieNet + UNet lightweight network. Proceedings of the 2021 3rd International Symposium on Robotics & Intelligent Manufacturing Technology (ISRIMT), Changzhou, China.","DOI":"10.1109\/ISRIMT53730.2021.9596927"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1016\/j.neucom.2023.02.025","article-title":"BiSeNet V3: Bilateral segmentation network with coordinate attention for real-time semantic segmentation","volume":"532","author":"Tsai","year":"2023","journal-title":"Neurocomputing"},{"key":"ref_37","unstructured":"Ruan, J., Xie, M., Gao, J., Liu, T., and Fu, Y. (2023). International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Jiang, W., Xie, Z., Li, Y., Liu, C., and Lu, H. (2020, January 6\u201310). Lrnnet: A light-weighted network with efficient reduced non-local operation for real-time semantic segmentation. Proceedings of the 2020 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), London, UK.","DOI":"10.1109\/ICMEW46912.2020.9106038"}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/16\/5\/520\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:33:07Z","timestamp":1760106787000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/16\/5\/520"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,25]]},"references-count":38,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2024,5]]}},"alternative-id":["sym16050520"],"URL":"https:\/\/doi.org\/10.3390\/sym16050520","relation":{},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,25]]}}}