{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T16:13:35Z","timestamp":1774628015125,"version":"3.50.1"},"reference-count":49,"publisher":"MDPI AG","issue":"20","license":[{"start":{"date-parts":[[2022,10,17]],"date-time":"2022-10-17T00:00:00Z","timestamp":1665964800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"the Natural Science Foundation of Shanxi Province","award":["20210302124257"],"award-info":[{"award-number":["20210302124257"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>For remote sensing image scene classification tasks, the classification accuracy of the small-scale deep neural network tends to be low and fails to achieve accuracy in real-world application scenarios. However, although large deep neural networks can improve the classification accuracy of remote sensing image scenes to some extent, the corresponding deep neural networks also have more parameters and cannot be used on existing embedded devices. The main reason for this is that there are a large number of redundant parameters in large deep networks, which directly leads to the difficulty of application on embedded devices and also reduces the classification speed. Considering the contradiction between hardware equipment and classification accuracy requirements, we propose a collaborative consistent knowledge distillation method for improving the classification accuracy of remote sensing image scenes on embedded devices, called CKD. In essence, our method addresses two aspects: (1) We design a multi-branch fused redundant feature mapping module, which significantly improves the parameter redundancy problem. (2) To improve the classification accuracy of the deep model on embedded devices, we propose a knowledge distillation method based on mutually supervised learning. Experiments were conducted on two remote sensing image classification datasets, SIRI-WHU and NWPU-RESISC45, and the experimental results showed that our approach significantly reduced the number of redundant parameters in the deep network; the number of parameters decreased from 1.73 M to 0.90 M. In addition, compared to a series of student sub-networks obtained based on the existing different knowledge distillation methods, the performance of the student sub-networks obtained by CKD for remote sensing scene classification was significantly improved on two different datasets, with an average accuracy of 0.943 and 0.916, respectively.<\/jats:p>","DOI":"10.3390\/rs14205186","type":"journal-article","created":{"date-parts":[[2022,10,18]],"date-time":"2022-10-18T00:31:01Z","timestamp":1666053061000},"page":"5186","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":20,"title":["Collaborative Consistent Knowledge Distillation Framework for Remote Sensing Image Scene Classification Network"],"prefix":"10.3390","volume":"14","author":[{"given":"Shiyi","family":"Xing","sequence":"first","affiliation":[{"name":"James Watt School of Engineering, University of Glasgow, Glasgow G12 8QQ, UK"}]},{"given":"Jinsheng","family":"Xing","sequence":"additional","affiliation":[{"name":"School of Mathematics and Computer Science, Shanxi Normal University, Taiyuan 030031, China"}]},{"given":"Jianguo","family":"Ju","sequence":"additional","affiliation":[{"name":"Department of Information Science and Technology, Northwest University, Xi\u2019an 710069, China"}]},{"given":"Qingshan","family":"Hou","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Northeastern University, Shenyang 110169, China"}]},{"given":"Xiurui","family":"Ding","sequence":"additional","affiliation":[{"name":"School of Natural Sciences, The University of Manchester, Manchester M15 4RB, UK"}]}],"member":"1968","published-online":{"date-parts":[[2022,10,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Alam, E., Sufian, A., Das, A.K., Bhattacharya, A., Ali, M.F., and Rahman, M.M.H. (2021). Leveraging Deep Learning for Computer Vision: A Review. Proceedings of the 22nd International Arab Conference on Information Technology, ACIT 2021, Muscat, Oman, 21\u201323 December 2021, IEEE.","DOI":"10.1109\/ACIT53391.2021.9677361"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"105123","DOI":"10.1016\/j.compbiomed.2021.105123","article-title":"Review and classification of AI-enabled COVID-19 CT imaging models based on computer vision tasks","volume":"141","author":"Hassan","year":"2022","journal-title":"Comput. Biol. Med."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"101356","DOI":"10.1016\/j.aei.2021.101356","article-title":"Recognition of pedestrian trajectories and attributes with computer vision and deep learning techniques","volume":"49","author":"Wong","year":"2021","journal-title":"Adv. Eng. Inform."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"681","DOI":"10.1049\/ipr2.12286","article-title":"An image enhancement algorithm of video surveillance scene based on deep learning","volume":"16","author":"Shen","year":"2022","journal-title":"IET Image Process."},{"key":"ref_5","unstructured":"Augenstein, I., and Habernal, I. (2021). Reviewing Natural Language Processing Research. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Tutorial Abstracts, EACL 2021, Online, 19\u201320 April 2021, Association for Computational Linguistics."},{"key":"ref_6","unstructured":"Jiang, H. (2021). Reducing Human Labor Cost in Deep Learning for Natural Language Processing. [Ph.D. Thesis, Georgia Institute of Technology]."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"443","DOI":"10.1016\/j.neucom.2021.05.103","article-title":"An introduction to Deep Learning in Natural Language Processing: Models, techniques, and tools","volume":"470","author":"Lauriola","year":"2022","journal-title":"Neurocomputing"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"484","DOI":"10.1038\/nature16961","article-title":"Mastering the game of Go with deep neural networks and tree search","volume":"529","author":"Silver","year":"2016","journal-title":"Nature"},{"key":"ref_9","unstructured":"Agarwal, R., Schwarzer, M., Castro, P.S., Courville, A.C., and Bellemare, M.G. (2021, January 6\u201314). Deep Reinforcement Learning at the Edge of the Statistical Precipice. Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, Virtual."},{"key":"ref_10","unstructured":"Curi, S., Bogunovic, I., and Krause, A. (2021, January 18\u201324). Combining Pessimism with Optimism for Robust and Efficient Model-Based Deep Reinforcement Learning. Proceedings of the 38th International Conference on Machine Learning, ICML 2021, Virtual Event."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"El Kaid, A., Brazey, D., Barra, V., and Ba\u00efna, K. (2022). Top-Down System for Multi-Person 3D Absolute Pose Estimation from Monocular Videos. Sensors, 22.","DOI":"10.3390\/s22114109"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1109\/TMI.2020.3021254","article-title":"A Novel Multiresolution-Statistical Texture Analysis Architecture: Radiomics-Aided Diagnosis of PDAC Based on Plain CT Images","volume":"40","author":"Qiu","year":"2021","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2919","DOI":"10.1007\/s10346-020-01473-9","article-title":"Landslide susceptibility prediction based on a semi-supervised multiple-layer perceptron model","volume":"17","author":"Huang","year":"2020","journal-title":"Landslides"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"3048","DOI":"10.1109\/TPAMI.2021.3055564","article-title":"Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks","volume":"44","author":"Wang","year":"2022","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1789","DOI":"10.1007\/s11263-021-01453-z","article-title":"Knowledge Distillation: A Survey","volume":"129","author":"Gou","year":"2021","journal-title":"Int. J. Comput. Vis."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Bucilua, C., Caruana, R., and Niculescu-Mizil, A. (2006, January 20\u201323). Model compression. Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Philadelphia, PA, USA.","DOI":"10.1145\/1150402.1150464"},{"key":"ref_17","unstructured":"Ba, J., and Caruana, R. (2014, January 8\u201313). Do Deep Nets Really Need to be Deep?. Proceedings of the Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, Montreal, QC, Canada."},{"key":"ref_18","unstructured":"Urban, G., Geras, K.J., Kahou, S.E., Aslan, \u00d6., Wang, S., Mohamed, A., Philipose, M., Richardson, M., and Caruana, R. Do Deep Convolutional Nets Really Need to be Deep and Convolutional? In Proceedings of the 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, 24\u201326 April 2017."},{"key":"ref_19","unstructured":"Hinton, G.E., Vinyals, O., and Dean, J. (2015). Distilling the Knowledge in a Neural Network. arXiv."},{"key":"ref_20","unstructured":"Agrawal, D., Zhang, P., Abbadi, A.E., and Mokbel, M.F. (2010). Bag-of-visual-words and spatial extensions for land-use classification. Proceedings of the 18th ACM SIGSPATIAL International Symposium on Advances in Geographic Information Systems, ACM-GIS 2010 San Jose, CA, USA 3\u20135 November 2010, ACM."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2689","DOI":"10.1109\/TGRS.2017.2781712","article-title":"Scene Classification Based on the Sparse Homogeneous-Heterogeneous Topic Feature Model","volume":"56","author":"Zhu","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"2335","DOI":"10.1109\/TPAMI.2017.2651061","article-title":"Compositional Model Based Fisher Vector Coding for Image Classification","volume":"39","author":"Liu","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Yao, Y., Liang, H., Li, X., Zhang, J., and He, J. (2017). Sensing Urban Land-Use Patterns by Integrating Google Tensorflow and Scene-Classification Models. arXiv.","DOI":"10.5194\/isprs-archives-XLII-2-W7-981-2017"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2811","DOI":"10.1109\/TGRS.2017.2783902","article-title":"When Deep Learning Meets Metric Learning: Remote Sensing Image Scene Classification via Learning Discriminative CNNs","volume":"56","author":"Cheng","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Gong, X., Xie, Z., Liu, Y., Shi, X., and Zheng, Z. (2018). Deep Salient Feature Based Anti-Noise Transfer Network for Scene Classification of Remote Sensing Imagery. Remote Sens., 10.","DOI":"10.3390\/rs10030410"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"e474","DOI":"10.7717\/peerj-cs.474","article-title":"Knowledge distillation in deep learning and its applications","volume":"7","author":"Alkhulaifi","year":"2021","journal-title":"PeerJ Comput. Sci."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1765","DOI":"10.1109\/TPDS.2020.3047003","article-title":"Parallel blockwise knowledge distillation for deep neural network compression","volume":"32","author":"Blakeney","year":"2020","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"ref_28","unstructured":"Romero, A., Ballas, N., Kahou, S.E., Chassang, A., Gatta, C., and Bengio, Y. (2015, January 7\u20139). FitNets: Hints for Thin Deep Nets. Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Park, W., Kim, D., Lu, Y., and Cho, M. (2019). Relational Knowledge Distillation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019 Long Beach, CA, USA 16\u201320 June 2019, Computer Vision Foundation\/IEEE.","DOI":"10.1109\/CVPR.2019.00409"},{"key":"ref_30","unstructured":"Tian, Y., Krishnan, D., and Isola, P. (2020, January 26\u201330). Contrastive Representation Distillation. Proceedings of the 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia."},{"key":"ref_31","first-page":"18","article-title":"Local Correlation Consistency for Knowledge Distillation","volume":"Volume 12357","author":"Vedaldi","year":"2020","journal-title":"Proceedings of the Computer Vision\u2014ECCV 2020\u201416th European Conference, Glasgow, UK, 23\u201328 August 2020; Proceedings, Part XII"},{"key":"ref_32","first-page":"664","article-title":"Feature Normalized Knowledge Distillation for Image Classification","volume":"Volume 12370","author":"Vedaldi","year":"2020","journal-title":"Proceedings of the Computer Vision\u2014ECCV 2020\u201416th European Conference, Glasgow, UK, 23\u201328 August 2020; Proceedings, Part XXV"},{"key":"ref_33","unstructured":"Du, S., You, S., Li, X., Wu, J., Wang, F., Qian, C., and Zhang, C. (2020, January 6\u201312). Agree to Disagree: Adaptive Ensemble Knowledge Distillation in Gradient Space. Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, Virtual."},{"key":"ref_34","unstructured":"Lan, X., Zhu, X., and Gong, S. (2018, January 3\u20138). Knowledge Distillation by On-the-Fly Native Ensemble. Proceedings of the Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, Montreal, QC, Canada."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Xiang, T., Hospedales, T.M., and Lu, H. (2018). Deep Mutual Learning. Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, 18\u201322 June 2018, Computer Vision Foundation\/IEEE Computer Society.","DOI":"10.1109\/CVPR.2018.00454"},{"key":"ref_36","first-page":"294","article-title":"Knowledge Transfer via Dense Cross-Layer Mutual-Distillation","volume":"Volume 12360","author":"Vedaldi","year":"2020","journal-title":"Proceedings of the Computer Vision\u2014ECCV 2020\u201416th European Conference, Glasgow, UK, 23\u201328 August 2020; Proceedings, Part XV"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Wu, G., and Gong, S. (2021). Peer Collaborative Learning for Online Knowledge Distillation. Proceedings of the 35th AAAI Conference on Artificial Intelligence, AAAI 2021, 33rd Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, 2\u20139 February 2021, AAAI Press.","DOI":"10.1609\/aaai.v35i12.17234"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Long, J., Shelhamer, E., and Darrell, T. (2015). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, 7\u201312 June 2015, IEEE Computer Society.","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"ref_39","unstructured":"Yu, F., and Koltun, V. (2016, January 2\u20134). Multi-Scale Context Aggregation by Dilated Convolutions. Proceedings of the 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., and Wei, Y. (2017). Deformable Convolutional Networks. Proceedings of the IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, 22\u201329 October 2017, IEEE Computer Society.","DOI":"10.1109\/ICCV.2017.89"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Ding, X., Guo, Y., Ding, G., and Han, J. (2019). ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blocks. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea, 27 October\u20132 November 2019, IEEE.","DOI":"10.1109\/ICCV.2019.00200"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Hu, J., Shen, L., and Sun, G. (2018). Squeeze-and-Excitation Networks. Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, 18\u201322 June 2018, Computer Vision Foundation\/IEEE Computer Society.","DOI":"10.1109\/CVPR.2018.00745"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Han, K., Wang, Y., Tian, Q., Guo, J., Xu, C., and Xu, C. (2020). GhostNet: More Features From Cheap Operations. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, 13\u201319 June 2020, Computer Vision Foundation\/IEEE.","DOI":"10.1109\/CVPR42600.2020.00165"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"1865","DOI":"10.1109\/JPROC.2017.2675998","article-title":"Remote sensing image scene classification: Benchmark and state of the art","volume":"105","author":"Cheng","year":"2017","journal-title":"Proc. IEEE"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"2108","DOI":"10.1109\/TGRS.2015.2496185","article-title":"Dirichlet-Derived Multiple Topic Scene Classification Model for High Spatial Resolution Remote Sensing Imagery","volume":"54","author":"Zhao","year":"2016","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_46","unstructured":"Kingma, D.P., and Ba, J. (2015, January 7\u20139). Adam: A Method for Stochastic Optimization. Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, 27\u201330 June 2016, IEEE Computer Society.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., van der Maaten, L., and Weinberger, K.Q. (2017). Densely Connected Convolutional Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, 21\u201326 July 2017, IEEE Computer Society.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Zheng, L., Shen, L., Tian, L., Wang, S., Wang, J., and Tian, Q. (2015). Scalable Person Re-identification: A Benchmark. Proceedings of the 2015 IEEE International Conference on Computer Vision, ICCV 2015, Santiago, Chile, 7\u201313 December 2015, IEEE Computer Society.","DOI":"10.1109\/ICCV.2015.133"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/20\/5186\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:55:38Z","timestamp":1760144138000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/20\/5186"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,17]]},"references-count":49,"journal-issue":{"issue":"20","published-online":{"date-parts":[[2022,10]]}},"alternative-id":["rs14205186"],"URL":"https:\/\/doi.org\/10.3390\/rs14205186","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,10,17]]}}}