{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,18]],"date-time":"2026-03-18T22:33:07Z","timestamp":1773873187435,"version":"3.50.1"},"reference-count":45,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2023,1,7]],"date-time":"2023-01-07T00:00:00Z","timestamp":1673049600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Guizhou Provincial Science and Technology Foundation","award":["No. QKHJC-ZK[2021]Key001"],"award-info":[{"award-number":["No. QKHJC-ZK[2021]Key001"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>As a popular research subject in the field of computer vision, knowledge distillation (KD) is widely used in semantic segmentation (SS). However, based on the learning paradigm of the teacher\u2013student model, the poor quality of teacher network feature knowledge still hinders the development of KD technology. In this paper, we investigate the output features of the teacher\u2013student network and propose a feature condensation-based KD network (FCKDNet), which reduces pseudo-knowledge transfer in the teacher\u2013student network. First, combined with the pixel information entropy calculation rule, we design a feature condensation method to separate the foreground feature knowledge from the background noise of the teacher network outputs. Then, the obtained feature condensation matrix is applied to the original outputs of the teacher and student networks to improve the feature representation capability. In addition, after performing feature condensation on the teacher network, we propose a soft enhancement method of features based on spatial and channel dimensions to improve the dependency of pixels in the feature maps. Finally, we divide the outputs of the teacher network into spatial condensation features and channel condensation features and perform distillation loss calculation with the student network separately to assist the student network to converge faster. Extensive experiments on the public datasets Pascal VOC and Cityscapes demonstrate that our proposed method improves the baseline by 3.16% and 2.98% in terms of mAcc, and 2.03% and 2.30% in terms of mIoU, respectively, and has better segmentation performance and robustness than the mainstream methods.<\/jats:p>","DOI":"10.3390\/e25010125","type":"journal-article","created":{"date-parts":[[2023,1,9]],"date-time":"2023-01-09T02:09:39Z","timestamp":1673230179000},"page":"125","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["FCKDNet: A Feature Condensation Knowledge Distillation Network for Semantic Segmentation"],"prefix":"10.3390","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7769-8206","authenticated-orcid":false,"given":"Wenhao","family":"Yuan","sequence":"first","affiliation":[{"name":"College of Big Data and Information Engineering, Guizhou University, Guiyang 550025, China"}]},{"given":"Xiaoyan","family":"Lu","sequence":"additional","affiliation":[{"name":"College of Big Data and Information Engineering, Guizhou University, Guiyang 550025, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4706-889X","authenticated-orcid":false,"given":"Rongfen","family":"Zhang","sequence":"additional","affiliation":[{"name":"College of Big Data and Information Engineering, Guizhou University, Guiyang 550025, China"}]},{"given":"Yuhong","family":"Liu","sequence":"additional","affiliation":[{"name":"College of Big Data and Information Engineering, Guizhou University, Guiyang 550025, China"}]}],"member":"1968","published-online":{"date-parts":[[2023,1,7]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"3833","DOI":"10.1007\/s40747-021-00618-0","article-title":"Semantic segmentation of large-scale point clouds based on dilated nearest neighbors graph","volume":"8","author":"Wang","year":"2022","journal-title":"Complex Intell. Syst."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H. (2018, January 8). Encoder-decoder with atrous separable convolution for semantic image segmentation. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01234-2_49"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017, January 1\u201326). Pyramid scene parsing network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.660"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Lin, G., Milan, A., Shen, C., and Reid, I. (2017, January 21\u201326). Refinenet: Multi-path refinement networks for high-resolution semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.549"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Yuan, Y., Chen, X., and Wang, J. (2020, January 23\u201328). Object-contextual representations for semantic segmentation. Proceedings of the European Conference on Computer Vision, Glasgow, UK.","DOI":"10.1007\/978-3-030-58539-6_11"},{"key":"ref_6","unstructured":"Hinton, G., Vinyals, O., and Dean, J. (2015). Distilling the knowledge in a neural network. arXiv."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Pham, H., Dai, Z., Xie, Q., and Le, Q.V. (2021, January 19\u201325). Meta pseudo labels. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Virtual.","DOI":"10.1109\/CVPR46437.2021.01139"},{"key":"ref_8","unstructured":"Wu, G., and Gong, S. (2021, January 2\u20139). Peer collaborative learning for online knowledge distillation. Proceedings of the AAAI Conference on Artificial Intelligence, virtual."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Wang, K., Liu, X., Bagdanov, A.D., Herranz, L., Jui, S., and van de Weijer, J. (2022, January 19\u201320). Incremental Meta-Learning via Episodic Replay Distillation for Few-Shot Image Recognition. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPRW56347.2022.00417"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Kang, M., Park, J., and Han, B. (2022, January 19\u201324). Class-Incremental Learning by Knowledge Distillation with Adaptive Feature Consolidation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01560"},{"key":"ref_11","unstructured":"Yang, C., An, Z., Zhou, H., Xu, Y., and Zhan, Q. (2022). Online Knowledge Distillation via Mutual Contrastive Learning for Visual Recognition. arXiv."},{"key":"ref_12","first-page":"11920","article-title":"Mosaicking to distill: Knowledge distillation from out-of-domain data","volume":"34","author":"Fang","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Yang, L., and Xu, K. (2021, January 19\u201325). Cross modality knowledge distillation for multi-modal aerial view object classification. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Virtual.","DOI":"10.1109\/CVPRW53098.2021.00048"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Shen, Y., Xu, L., Yang, Y., Li, Y., and Guo, Y. (2022, January 19\u201324). Self-Distillation from the Last Mini-Batch for Consistency Regularization. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01164"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Dadashzadeh, A., Whone, A., and Mirmehdi, M. (2022, January 19\u201324). Auxiliary Learning for Self-Supervised Video Representation via Similarity-based Knowledge Distillation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPRW56347.2022.00468"},{"key":"ref_16","unstructured":"Tung, F., and Mori, G. (November, January 27). Similarity-preserving knowledge distillation. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Korea."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Heo, B., Lee, M., Yun, S., and Choi, J.Y. (2019, January 27\u201328). Knowledge transfer via distillation of activation boundaries formed by hidden neurons. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.","DOI":"10.1609\/aaai.v33i01.33013779"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Yim, J., Joo, D., Bae, J., and Kim, J. (2017, January 21\u201326). A gift from knowledge distillation: Fast optimization, network minimization and transfer learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.754"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Park, W., Kim, D., Lu, Y., and Cho, M. (2019, January 15\u201320). Relational knowledge distillation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00409"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Liu, Y., Chen, K., Liu, C., Qin, Z., Luo, Z., and Wang, J. (2019, January 15\u201320). Structured knowledge distillation for semantic segmentation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00271"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Wang, Y., Zhou, W., Jiang, T., Bai, X., and Xu, Y. (2020, January 23\u201328). Intra-class feature variation distillation for semantic segmentation. Proceedings of the European Conference on Computer Vision, Glasgow, UK.","DOI":"10.1007\/978-3-030-58571-6_21"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Shu, C., Liu, Y., Gao, J., Yan, Z., and Shen, C. (2021, January 11\u201317). Channel-wise knowledge distillation for dense prediction. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, BC, Canada.","DOI":"10.1109\/ICCV48922.2021.00526"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Arnaudo, E., Cermelli, F., Tavera, A., Rossi, C., and Caputo, B. (2022, January 23\u201327). A contrastive distillation approach for incremental semantic segmentation in aerial images. Proceedings of the International Conference on Image Analysis and Processing, Lecce, Italy.","DOI":"10.1007\/978-3-031-06430-2_62"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Yang, C., Zhou, H., An, Z., Jiang, X., Xu, Y., and Zhang, Q. (2022, January 19\u201320). Cross-image relational knowledge distillation for semantic segmentation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01200"},{"key":"ref_25","unstructured":"Huang, T., You, S., Wang, F., Qian, C., and Xu, C. (2022). Knowledge Distillation from A Stronger Teacher. arXiv."},{"key":"ref_26","unstructured":"Liu, R., Yang, K., Liu, H., Zhang, J., Peng, K., and Stiefelhagen, R. (2022). Transformer-based Knowledge Distillation for Efficient Semantic Segmentation of Road-driving Scenes. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Yuan, J., Ge, J., Qian, Q., Wang, Z., Wang, F., and Liu, Y. (2022). Semi-supervised Semantic Segmentation with Mutual Knowledge Distillation. arXiv.","DOI":"10.1145\/3581783.3611906"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Hou, Q., Zhou, D., and Feng, J. (2021, January 19\u201325). Coordinate attention for efficient mobile network design. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Virtual.","DOI":"10.1109\/CVPR46437.2021.01350"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Deng, F., Feng, H., Liang, M., Wang, H., Yang, Y., Gao, Y., Chen, J., Hu, J., Guo, X., and Lam, T.L. (October, January 27). FEANet: Feature-enhanced attention network for RGB-thermal real-time semantic segmentation. Proceedings of the 2021 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic.","DOI":"10.1109\/IROS51168.2021.9636084"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"399","DOI":"10.1016\/j.ins.2021.10.055","article-title":"LGCNet: A local-to-global context-aware feature augmentation network for salient object detection","volume":"584","author":"Ji","year":"2022","journal-title":"Inf. Sci."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"20541","DOI":"10.1007\/s11042-022-12249-9","article-title":"COMAL: Compositional multi-scale feature enhanced learning for crowd counting","volume":"81","author":"Zhou","year":"2022","journal-title":"Multimed. Tools Appl."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Chen, S., Zhang, J., Zhan, R., Zhu, R., and Wang, W. (2022). Few Shot Object Detection for SAR Images via Feature Enhancement and Dynamic Relationship Modeling. Remote. Sens., 14.","DOI":"10.3390\/rs14153669"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Li, Y., Wang, L., and Wang, Z. (2022). Single-Shot Object Detection via Feature Enhancement and Channel Attention. Sensors, 22.","DOI":"10.3390\/s22186857"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Mirzadeh, S.I., Farajtabar, M., Li, A., Levine, N., Matsukawa, A., and Ghasemzadeh, H. (2020, January 7\u201312). Improved knowledge distillation via teacher assistant. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i04.5963"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Li, C., Lin, M., Ding, Z., Lin, N., Zhuang, Y., Huang, Y., Ding, X., and Cao, L. (2022). Knowledge Condensation Distillation. arXiv.","DOI":"10.1007\/978-3-031-20083-0_2"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Ferrari, V., Hebert, M., Sminchisescu, C., and Weiss, Y. (2018). CBAM: Convolutional Block Attention Module. Computer Vision\u2014ECCV 2018, Springer International Publishing.","DOI":"10.1007\/978-3-030-01270-0"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1007\/s11263-009-0275-4","article-title":"The pascal visual object classes (voc) challenge","volume":"88","author":"Everingham","year":"2010","journal-title":"Int. J. Comput. Vis."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., and Schiele, B. (2016, January 27\u201330). The cityscapes dataset for semantic urban scene understanding. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.350"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"501","DOI":"10.1109\/TMI.2004.825627","article-title":"Ridge-based vessel segmentation in color images of the retina","volume":"23","author":"Staal","year":"2004","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Zagoruyko, S., and Komodakis, N. (2016). Wide residual networks. arXiv.","DOI":"10.5244\/C.30.87"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.C. (2018, January 18\u201323). Mobilenetv2: Inverted residuals and linear bottlenecks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00474"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"2481","DOI":"10.1109\/TPAMI.2016.2644615","article-title":"Segnet: A deep convolutional encoder-decoder architecture for image segmentation","volume":"39","author":"Badrinarayanan","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Alom, M.Z., Hasan, M., Yakopcic, C., Taha, T.M., and Asari, V.K. (2018). Recurrent residual convolutional neural network based on u-net (r2u-net) for medical image segmentation. arXiv.","DOI":"10.1109\/NAECON.2018.8556686"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/1\/125\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:02:37Z","timestamp":1760119357000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/1\/125"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,7]]},"references-count":45,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,1]]}},"alternative-id":["e25010125"],"URL":"https:\/\/doi.org\/10.3390\/e25010125","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,7]]}}}