{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,24]],"date-time":"2026-04-24T13:30:32Z","timestamp":1777037432313,"version":"3.51.4"},"reference-count":50,"publisher":"MDPI AG","issue":"22","license":[{"start":{"date-parts":[[2021,11,11]],"date-time":"2021-11-11T00:00:00Z","timestamp":1636588800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"the National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["No. 41371342"],"award-info":[{"award-number":["No. 41371342"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["No. 2016YFC0803000"],"award-info":[{"award-number":["No. 2016YFC0803000"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Due to device limitations, small networks are necessary for some real-world scenarios, such as satellites and micro-robots. Therefore, the development of a network with both good performance and small size is an important area of research. Deep networks can learn well from large amounts of data, while manifold networks have outstanding feature representation at small sizes. In this paper, we propose an approach that exploits the advantages of deep networks and shallow Grassmannian manifold networks. Inspired by knowledge distillation, we use the information learned from convolutional neural networks to guide the training of the manifold networks. Our approach leads to a reduction in model size, which addresses the problem of deploying deep learning on resource-limited embedded devices. Finally, a series of experiments were conducted on four remote sensing scene classification datasets. The method in this paper improved the classification accuracy by 2.31% and 1.73% on the UC Merced Land Use and SIRIWHU datasets, respectively, and the experimental results demonstrate the effectiveness of our approach.<\/jats:p>","DOI":"10.3390\/rs13224537","type":"journal-article","created":{"date-parts":[[2021,11,11]],"date-time":"2021-11-11T23:04:46Z","timestamp":1636671886000},"page":"4537","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Knowledge Distillation of Grassmann Manifold Network for Remote Sensing Scene Classification"],"prefix":"10.3390","volume":"13","author":[{"given":"Ling","family":"Tian","sequence":"first","affiliation":[{"name":"Electronic Information School, Wuhan University, Wuhan 430072, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9885-3463","authenticated-orcid":false,"given":"Zhichao","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Computer Science, Wuhan University, Wuhan 430072, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bokun","family":"He","sequence":"additional","affiliation":[{"name":"Electronic Information School, Wuhan University, Wuhan 430072, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3662-5769","authenticated-orcid":false,"given":"Chu","family":"He","sequence":"additional","affiliation":[{"name":"Electronic Information School, Wuhan University, Wuhan 430072, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dingwen","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Computer Science, Wuhan University, Wuhan 430072, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Deshi","family":"Li","sequence":"additional","affiliation":[{"name":"Electronic Information School, Wuhan University, Wuhan 430072, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,11,11]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Xu, Z., Zhang, W., Zhang, T., and Li, J. (2021). HRCNet: High-resolution context extraction network for semantic segmentation of remote sensing images. Remote Sens., 13.","DOI":"10.3390\/rs13122290"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Ouyang, S., and Li, Y. (2021). Combining deep semantic segmentation network and graph convolutional neural network for semantic segmentation of remote sensing imagery. Remote Sens., 13.","DOI":"10.3390\/rs13010119"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"296","DOI":"10.1016\/j.isprsjprs.2019.11.023","article-title":"Object detection in optical remote sensing images: A survey and a new benchmark","volume":"159","author":"Li","year":"2020","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Yang, K., Xia, G.S., Liu, Z., Du, B., Yang, W., Pelillo, M., and Zhang, L. (2021). Asymmetric Siamese Networks for Semantic Change Detection in Aerial Images. IEEE Trans. Geosci. Remote. Sens., 1\u201318.","DOI":"10.1109\/TGRS.2021.3113912"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Maxwell, A.E., Bester, M.S., Guillen, L.A., Ramezan, C.A., Carpinello, D.J., Fan, Y., Hartley, F.M., Maynard, S.M., and Pyron, J.L. (2020). Semantic Segmentation Deep Learning for Extracting Surface Mine Extents from Historic Topographic Maps. Remote Sens., 12.","DOI":"10.3390\/rs12244145"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Kalajdjieski, J., Zdravevski, E., Corizzo, R., Lameski, P., Kalajdziski, S., Pires, I.M., Garcia, N.M., and Trajkovik, V. (2020). Air pollution prediction with multi-modal data and deep neural networks. Remote Sens., 12.","DOI":"10.3390\/rs12244142"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1521","DOI":"10.1109\/JSTARS.2015.2513898","article-title":"Unsupervised quaternion feature learning for remote sensing image classification","volume":"9","year":"2016","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"747","DOI":"10.1109\/LGRS.2015.2513443","article-title":"Bag-of-visual-words scene classifier with local and global features for high spatial resolution remote sensing imagery","volume":"13","author":"Zhu","year":"2016","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"5525","DOI":"10.1109\/TGRS.2017.2709802","article-title":"Scene classification based on the fully sparse semantic topic model","volume":"55","author":"Zhu","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2889","DOI":"10.1109\/JSTARS.2017.2683799","article-title":"Fusing local and global features for high-resolution scene classification","volume":"10","author":"Bian","year":"2017","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"676","DOI":"10.1109\/LGRS.2014.2357392","article-title":"Bag of lines (BoL) for improved aerial scene representation","volume":"12","author":"Sridharan","year":"2014","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1357","DOI":"10.1109\/LGRS.2015.2402391","article-title":"A comparative study of bag-of-words and bag-of-topics models of EO image patches","volume":"12","author":"Bahmanyar","year":"2015","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1109\/LGRS.2015.2499239","article-title":"Deep learning earth observation classification using ImageNet pretrained networks","volume":"13","author":"Marmanis","year":"2015","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1109\/LGRS.2017.2731997","article-title":"Remote sensing image scene classification using bag of convolutional features","volume":"14","author":"Cheng","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2047","DOI":"10.1007\/s00521-020-05071-7","article-title":"Multi-deep features fusion for high-resolution remote sensing image scene classification","volume":"33","author":"Yuan","year":"2021","journal-title":"Neural Comput. Appl."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Xu, K., Huang, H., and Deng, P. (2021). Remote Sensing Image Scene Classification Based on Global-Local Dual-Branch Structure Model. IEEE Geosci. Remote. Sens. Lett., 1\u20135.","DOI":"10.1109\/LGRS.2021.3075712"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"250","DOI":"10.1016\/j.ins.2020.06.011","article-title":"Two-stream feature aggregation deep neural network for scene classification of remote sensing images","volume":"539","author":"Xu","year":"2020","journal-title":"Inf. Sci."},{"key":"ref_18","unstructured":"Xu, K., Huang, H., Deng, P., and Li, Y. (2021). Deep Feature Aggregation Framework Driven by Graph Convolutional Network for Scene Classification in Remote Sensing. IEEE Trans. Neural Netw. Learn. Syst., 1\u201315."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"6498","DOI":"10.1109\/TIP.2021.3092816","article-title":"Local Semantic Enhanced ConvNet for Aerial Scene Recognition","volume":"30","author":"Bi","year":"2021","journal-title":"IEEE Trans. Image Process."},{"key":"ref_20","first-page":"7844","article-title":"DLA-MatchNet for few-shot remote sensing image scene classification","volume":"9","author":"Li","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Deng, P., Xu, K., and Huang, H. (2021). When CNNs Meet Vision Transformer: A Joint Framework for Remote Sensing Scene Classification. IEEE Geosci. Remote Sens. Lett., 1\u20135.","DOI":"10.1109\/LGRS.2021.3109061"},{"key":"ref_22","unstructured":"Glorot, X., Bordes, A., and Bengio, Y. (2011, January 11\u201313). Deep sparse rectifier neural networks. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, Lauderdale, FL, USA."},{"key":"ref_23","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1097\/01.NNR.0000280659.88760.7c","article-title":"Multidimensional scaling: A brief overview","volume":"57","author":"Mugavin","year":"2008","journal-title":"Nurs. Res."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"2319","DOI":"10.1126\/science.290.5500.2319","article-title":"A global geometric framework for nonlinear dimensionality reduction","volume":"290","author":"Tenenbaum","year":"2000","journal-title":"Science"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"2323","DOI":"10.1126\/science.290.5500.2323","article-title":"Nonlinear dimensionality reduction by locally linear embedding","volume":"290","author":"Roweis","year":"2000","journal-title":"Science"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1373","DOI":"10.1162\/089976603321780317","article-title":"Laplacian eigenmaps for dimensionality reduction and data representation","volume":"15","author":"Belkin","year":"2003","journal-title":"Neural Comput."},{"key":"ref_28","unstructured":"Huang, Z., Wu, J., and Van Gool, L. (2016). Building deep networks on Grassmann manifolds. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Huang, Z., and Van Gool, L. (2017, January 4\u20139). A riemannian network for spd matrix learning. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.","DOI":"10.1609\/aaai.v31i1.10866"},{"key":"ref_30","unstructured":"Chakraborty, R., Bouza, J., Manton, J., and Vemuri, B.C. (2020). Manifoldnet: A deep neural network for manifold-valued data with applications. IEEE Trans. Pattern Anal. Mach. Intell., 1."},{"key":"ref_31","unstructured":"Fr\u00e9chet, M. (1948). Les \u00e9l\u00e9ments al\u00e9atoires de Nature Quelconque Dans un Espace Distanci\u00e9, Annales de l\u2019institut Henri Poincar\u00e9."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Absil, P.A., Mahony, R., and Sepulchre, R. (2009). Optimization Algorithms on Matrix Manifolds, Princeton University Press.","DOI":"10.1515\/9781400830244"},{"key":"ref_33","unstructured":"Ionescu, C., Vantzos, O., and Sminchisescu, C. (2015). Training deep networks with structured layers by matrix backpropagation. arXiv."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Liu, M., Wang, R., Huang, Z., Shan, S., and Chen, X. (2013, January 9\u201313). Partial least squares regression on grassmannian manifold for emotion recognition. Proceedings of the 15th ACM on International Conference on Multimodal Interaction, Sydney, Australia.","DOI":"10.1145\/2522848.2531738"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Liu, M., Wang, R., Li, S., Shan, S., Huang, Z., and Chen, X. (2014, January 12\u201316). Combining multiple kernel methods on riemannian manifold for emotion recognition in the wild. Proceedings of the 16th International Conference on Multimodal Interaction, Istanbul, Turkey.","DOI":"10.1145\/2663204.2666274"},{"key":"ref_36","unstructured":"Hinton, G., Vinyals, O., and Dean, J. (2015). Distilling the knowledge in a neural network. arXiv."},{"key":"ref_37","unstructured":"Ravi, S. (2017). Projectionnet: Learning efficient on-device deep networks using neural projections. arXiv."},{"key":"ref_38","unstructured":"Romero, A., Ballas, N., Kahou, S.E., Chassang, A., Gatta, C., and Bengio, Y. (2014). Fitnets: Hints for thin deep nets. arXiv."},{"key":"ref_39","unstructured":"Zagoruyko, S., and Komodakis, N. (2016). Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. arXiv."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Yim, J., Joo, D., Bae, J., and Kim, J. (2017, January 21\u201326). A gift from knowledge distillation: Fast optimization, network minimization and transfer learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.754"},{"key":"ref_41","unstructured":"Heo, B., Kim, J., Yun, S., Park, H., Kwak, N., and Choi, J.Y. (November, January 27). A Comprehensive Overhaul of Feature Distillation. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Korea."},{"key":"ref_42","unstructured":"Huang, Z., and Wang, N. (2017). Like what you like: Knowledge distill via neuron selectivity transfer. arXiv."},{"key":"ref_43","unstructured":"Kim, J., Park, S., and Kwak, N. (2018). Paraphrasing complex network: Network compression via factor transfer. arXiv."},{"key":"ref_44","unstructured":"Xu, Z., Hsu, Y.C., and Huang, J. (2017). Training shallow and thin networks for acceleration via knowledge distillation with conditional adversarial networks. arXiv."},{"key":"ref_45","unstructured":"Furlanello, T., Lipton, Z., Tschannen, M., Itti, L., and Anandkumar, A. (2018, January 10\u201315). Born again neural networks. Proceedings of the International Conference on Machine Learning, PMLR, Stockholm, Sweden."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Yang, Y., and Newsam, S. (2010, January 2\u20135). Bag-of-visual-words and spatial extensions for land-use classification. Proceedings of the 18th Sigspatial International Conference on Advances in Geographic Information Systems, San Jose, CA, USA.","DOI":"10.1145\/1869790.1869829"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"2108","DOI":"10.1109\/TGRS.2015.2496185","article-title":"Dirichlet-derived multiple topic scene classification model for high spatial resolution remote sensing imagery","volume":"54","author":"Zhao","year":"2015","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"2321","DOI":"10.1109\/LGRS.2015.2475299","article-title":"Deep learning based feature selection for remote sensing scene classification","volume":"12","author":"Zou","year":"2015","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"3965","DOI":"10.1109\/TGRS.2017.2685945","article-title":"AID: A benchmark data set for performance evaluation of aerial scene classification","volume":"55","author":"Xia","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"703002","DOI":"10.3788\/IRLA201847.0703002","article-title":"Geometry deep network image-set recognition method based on Grassmann manifolds","volume":"47","author":"Tianci","year":"2018","journal-title":"Infrared Laser Eng."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/22\/4537\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:28:44Z","timestamp":1760167724000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/22\/4537"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,11]]},"references-count":50,"journal-issue":{"issue":"22","published-online":{"date-parts":[[2021,11]]}},"alternative-id":["rs13224537"],"URL":"https:\/\/doi.org\/10.3390\/rs13224537","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,11,11]]}}}