{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T21:10:33Z","timestamp":1792789833235,"version":"4.5.0"},"reference-count":44,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2022,5,22]],"date-time":"2022-05-22T00:00:00Z","timestamp":1653177600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61971388"],"award-info":[{"award-number":["61971388"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["U1706218"],"award-info":[{"award-number":["U1706218"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["41576011"],"award-info":[{"award-number":["41576011"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["ZR2018ZB0852"],"award-info":[{"award-number":["ZR2018ZB0852"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Alexander von Humboldt Foundation and the Key Natural Science Foundation of Shandong Province","award":["61971388"],"award-info":[{"award-number":["61971388"]}]},{"name":"Alexander von Humboldt Foundation and the Key Natural Science Foundation of Shandong Province","award":["U1706218"],"award-info":[{"award-number":["U1706218"]}]},{"name":"Alexander von Humboldt Foundation and the Key Natural Science Foundation of Shandong Province","award":["41576011"],"award-info":[{"award-number":["41576011"]}]},{"name":"Alexander von Humboldt Foundation and the Key Natural Science Foundation of Shandong Province","award":["ZR2018ZB0852"],"award-info":[{"award-number":["ZR2018ZB0852"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Remote sensing image (RSI) scene classification aims to identify the semantic categories of remote sensing images based on their contents. Owing to the strong learning capability of deep convolutional neural networks (CNNs), RSI scene classification methods based on CNNs have drawn much attention and achieved remarkable performance. However, such outstanding deep neural networks are usually computationally expensive and time-consuming, making them impossible to apply on resource-constrained edge devices, such as the embedded systems used on drones. To tackle this problem, we introduce a novel pair-wise similarity knowledge distillation method, which could reduce the model complexity while maintaining satisfactory accuracy, to obtain a compact and efficient deep neural network for RSI scene classification. Different from the existing knowledge distillation methods, we design a novel distillation loss to transfer the valuable discriminative information, which could reduce the within-class variations and restrain the between-class similarity, from the cumbersome model to the compact model. This method could obtain the compact student model with higher performance compared with existing knowledge distillation methods in RSI scene classification. To be specific, we distill the probability outputs between sample pairs with the same label and match the probability outputs between the teacher and student models. Experiments on three public benchmark datasets for RSI scene classification, i.e., AID, UCMerced, and NWPU-RESISC datasets, verify that the proposed method could effectively distill the knowledge and result in a higher performance.<\/jats:p>","DOI":"10.3390\/rs14102483","type":"journal-article","created":{"date-parts":[[2022,5,22]],"date-time":"2022-05-22T07:13:57Z","timestamp":1653203637000},"page":"2483","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Pair-Wise Similarity Knowledge Distillation for RSI Scene Classification"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2842-3526","authenticated-orcid":false,"given":"Haoran","family":"Zhao","sequence":"first","affiliation":[{"name":"College of Information Science and Engineering, Haide College and Institute of Advanced Ocean Study, Ocean University of China, Qingdao 266100, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1870-9037","authenticated-orcid":false,"given":"Xin","family":"Sun","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Haide College and Institute of Advanced Ocean Study, Ocean University of China, Qingdao 266100, China"},{"name":"The Department of Aerospace and Geodesy, Technical University of Munich, 80333 M\u00fcnchen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1825-328X","authenticated-orcid":false,"given":"Feng","family":"Gao","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Haide College and Institute of Advanced Ocean Study, Ocean University of China, Qingdao 266100, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Junyu","family":"Dong","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Haide College and Institute of Advanced Ocean Study, Ocean University of China, Qingdao 266100, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,5,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"8775","DOI":"10.1109\/TGRS.2019.2922908","article-title":"A Multi-Level Semantic Scene Interpretation Strategy for Change Interpretation in Remote Sensing Imagery","volume":"57","author":"Ghazouani","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1155","DOI":"10.1109\/TGRS.2011.2165548","article-title":"Very High Resolution Multiangle Urban Classification Analysis","volume":"50","author":"Longbotham","year":"2012","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2811","DOI":"10.1109\/TGRS.2017.2783902","article-title":"When Deep Learning Meets Metric Learning: Remote Sensing Image Scene Classification via Learning Discriminative CNNs","volume":"56","author":"Cheng","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1793","DOI":"10.1109\/TGRS.2015.2488681","article-title":"Scene Classification via a Gradient Boosting Random Convolutional Network Framework","volume":"54","author":"Zhang","year":"2016","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"6530","DOI":"10.1109\/TGRS.2019.2906883","article-title":"Hydra: An Ensemble of Convolutional Neural Networks for Geospatial Land Classification","volume":"57","author":"Minetto","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_6","unstructured":"Han, S., Pool, J., Tran, J., and Dally, W.J. (2015, January 7\u201312). Learning both Weights and Connections for Efficient Neural Network. Proceedings of the Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_7","unstructured":"Novikov, A., Podoprikhin, D., Osokin, A., and Vetrov, D.P. (2015, January 7\u201312). Tensorizing Neural Networks. Proceedings of the Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_8","first-page":"38","article-title":"Distilling the Knowledge in a Neural Network","volume":"14","author":"Hinton","year":"2015","journal-title":"Comput. Sci."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Bucilu\u01ce, C., Caruana, R., and Niculescu-Mizil, A. (2006, January 20\u201323). Model Compression. Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Philadelphia, PA, USA.","DOI":"10.1145\/1150402.1150464"},{"key":"ref_10","first-page":"2654","article-title":"Do Deep Nets Really Need to be Deep?","volume":"27","author":"Lei","year":"2013","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Tian, L., Wang, Z., He, B., He, C., Wang, D., and Li, D. (2021). Knowledge Distillation of Grassmann Manifold Network for Remote Sensing Scene Classification. Remote Sens., 13.","DOI":"10.3390\/rs13224537"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zhang, R., Chen, Z., Zhang, S., Song, F., Zhang, G., Zhou, Q., and Lei, T. (2020). Remote Sensing Image Scene Classification with Noisy Label Distillation. Remote Sens., 12.","DOI":"10.3390\/rs12152376"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Liu, B.Y., Chen, H.X., Huang, Z., Liu, X., and Yang, Y.Z. (2021). ZoomInNet: A Novel Small Object Detector in Drone Images with Cross-Scale Knowledge Distillation. Remote Sens., 13.","DOI":"10.3390\/rs13061198"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Chai, Y., Fu, K., Sun, X., Diao, W., Yan, Z., Feng, Y., and Wang, L. (2020). Compact Cloud Detection with Bidirectional Self-Attention Knowledge Distillation. Remote Sens., 12.","DOI":"10.3390\/rs12172770"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1023\/B:VISI.0000029664.99615.94","article-title":"Distinctive Image Features from Scale-Invariant Keypoints","volume":"60","author":"Lowe","year":"2004","journal-title":"Int. J. Comput. Vis."},{"key":"ref_16","unstructured":"Dalal, N., and Triggs, B. (2005, January 20\u201325). Histograms of oriented gradients for human detection. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR\u201905), San Diego, CA, USA."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Yang, Y., and Newsam, S. (2010, January 2\u20135). Bag-of-visual-words and spatial extensions for land-use classification. Proceedings of the Sigspatial International Conference on Advances in Geographic Information Systems, San Jose, CA, USA.","DOI":"10.1145\/1869790.1869829"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"747","DOI":"10.1109\/LGRS.2015.2513443","article-title":"Bag-of-Visual-Words Scene Classifier with Local and Global Features for High Spatial Resolution Remote Sensing Imagery","volume":"13","author":"Zhu","year":"2016","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"4620","DOI":"10.1109\/JSTARS.2014.2339842","article-title":"Land-Use Scene Classification Using a Concentric Circle-Structured Multiscale Bag-of-Visual-Words Model","volume":"7","author":"Zhao","year":"2014","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2175","DOI":"10.1109\/TGRS.2014.2357078","article-title":"Saliency-guided unsupervised feature learning for scene classification","volume":"53","author":"Zhang","year":"2014","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"5148","DOI":"10.1109\/TGRS.2017.2702596","article-title":"Remote sensing scene classification by unsupervised representation learning","volume":"55","author":"Lu","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"2250","DOI":"10.1109\/TGRS.2016.2640186","article-title":"Unsupervised feature learning for land-use scene recognition","volume":"55","author":"Fan","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1349","DOI":"10.1109\/TGRS.2015.2478379","article-title":"Unsupervised deep feature extraction for remote sensing image classification","volume":"54","author":"Romero","year":"2015","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"504","DOI":"10.1126\/science.1127647","article-title":"Reducing the dimensionality of data with neural networks","volume":"313","author":"Hinton","year":"2006","journal-title":"Science"},{"key":"ref_25","first-page":"1097","article-title":"Imagenet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky","year":"2012","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1155","DOI":"10.1109\/TGRS.2018.2864987","article-title":"Scene classification with recurrent attention of VHR remote sensing images","volume":"57","author":"Wang","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"4558","DOI":"10.1109\/TGRS.2019.2963364","article-title":"Relation Network for Multilabel Aerial Image Classification","volume":"58","author":"Hua","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Chen, G., Zhang, X., Tan, X., Cheng, Y., Dai, F., Zhu, K., Gong, Y., and Wang, Q. (2018). Training Small Networks for Scene Classification of Remote Sensing Images via Knowledge Distillation. Remote Sens., 10.","DOI":"10.3390\/rs10050719"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"ImageNet Large Scale Visual Recognition Challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Yang, J., Shen, X., Xing, J., Tian, X., Li, H., Deng, B., Huang, J., and Hua, X. (2019, January 16\u201320). Quantization Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00748"},{"key":"ref_31","unstructured":"Ba, J., and Caruana, R. (2014, January 8\u201313). Do Deep Nets Really Need to be Deep?. Proceedings of the Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, Montreal, QC, Canada."},{"key":"ref_32","unstructured":"Romero, A., Ballas, N., Kahou, S.E., Chassang, A., and Bengio, Y. (2015, January 7\u20139). FitNets: Hints for Thin Deep Nets. Proceedings of the 3rd International Conference on Learning Representations, ICLR, San Diego, CA, USA."},{"key":"ref_33","unstructured":"Zagoruyko, S., and Komodakis, N. (2017, January 24\u201326). Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer. Proceedings of the 5th International Conference on Learning Representations, ICLR, Toulon, France."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"107659","DOI":"10.1016\/j.patcog.2020.107659","article-title":"Adversarial co-distillation learning for image recognition","volume":"111","author":"Zhang","year":"2021","journal-title":"Pattern Recognit."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Park, W., Kim, D., Lu, Y., and Cho, M. (2019, January 15\u201320). Relational knowledge distillation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00409"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Tung, F., and Mori, G. (2019, January 27\u201328). Similarity-preserving knowledge distillation. Proceedings of the IEEE International Conference on Computer Vision, Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00145"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Peng, B., Jin, X., Li, D., Zhou, S., Wu, Y., Liu, J., Zhang, Z., and Liu, Y. (November, January 27). Correlation Congruence for Knowledge Distillation. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00511"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"107519","DOI":"10.1016\/j.knosys.2021.107519","article-title":"Knowledge distillation via instance-level sequence learning","volume":"233","author":"Zhao","year":"2021","journal-title":"Knowl. Based Syst."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1865","DOI":"10.1109\/JPROC.2017.2675998","article-title":"Remote Sensing Image Scene Classification: Benchmark and State of the Art","volume":"105","author":"Cheng","year":"2017","journal-title":"Proc. IEEE"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"3965","DOI":"10.1109\/TGRS.2017.2685945","article-title":"AID: A Benchmark Data Set for Performance Evaluation of Aerial Scene Classification","volume":"55","author":"Xia","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"2321","DOI":"10.1109\/LGRS.2015.2475299","article-title":"Deep Learning Based Feature Selection for Remote Sensing Scene Classification","volume":"12","author":"Zou","year":"2015","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Chen, L., Wang, D., Gan, Z., Liu, J., Henao, R., and Carin, L. (2021, January 19\u201325). Wasserstein Contrastive Representation Distillation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, Virtual.","DOI":"10.1109\/CVPR46437.2021.01603"},{"key":"ref_44","first-page":"2579","article-title":"Visualizing data using t-SNE","volume":"9","author":"Hinton","year":"2008","journal-title":"J. Mach. Learn. Res."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/10\/2483\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:16:23Z","timestamp":1760138183000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/10\/2483"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,22]]},"references-count":44,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2022,5]]}},"alternative-id":["rs14102483"],"URL":"https:\/\/doi.org\/10.3390\/rs14102483","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,5,22]]}}}