{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,26]],"date-time":"2026-02-26T15:23:12Z","timestamp":1772119392726,"version":"3.50.1"},"reference-count":56,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2023,1,7]],"date-time":"2023-01-07T00:00:00Z","timestamp":1673049600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004543","name":"China Scholarship Council","doi-asserted-by":"publisher","award":["202106210056"],"award-info":[{"award-number":["202106210056"]}],"id":[{"id":"10.13039\/501100004543","id-type":"DOI","asserted-by":"publisher"}]},{"name":"\u201cBest International Grant for PhD\u201d of Peter the Great, St. Petersburg Polytechnic University","award":["202106210056"],"award-info":[{"award-number":["202106210056"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>CubeSats provide a low-cost, convenient, and effective way of acquiring remote sensing data, and have great potential for remote sensing object detection. Although deep learning-based models have achieved excellent performance in object detection, they suffer from the problem of numerous parameters, making them difficult to deploy on CubeSats with limited memory and computational power. Existing approaches attempt to prune redundant parameters, but this inevitably causes a degradation in detection accuracy. In this paper, the novel Context-aware Dense Feature Distillation (CDFD) is proposed, guiding a small student network to integrate features extracted from multi-teacher networks to train a lightweight and superior detector for onboard remote sensing object detection. Specifically, a Contextual Feature Generation Module (CFGM) is designed to rebuild the non-local relationships between different pixels and transfer them from teacher to student, thus guiding students to extract rich contextual features to assist in remote sensing object detection. In addition, an Adaptive Dense Multi-teacher Distillation (ADMD) strategy is proposed, which performs adaptive weighted loss fusion of students with multiple well-trained teachers, guiding students to integrate the learning of helpful knowledge from multiple teachers. Extensive experiments were conducted on two large-scale remote sensing object detection datasets with various network structures; the results demonstrate that the trained lightweight network achieves auspicious performance. Our approach also shows good generality for existing state-of-the-art remote sensing object detectors. Furthermore, by experimenting on large general object datasets, we demonstrate that our approach is equally practical for general object detection distillation.<\/jats:p>","DOI":"10.3390\/rs15020370","type":"journal-article","created":{"date-parts":[[2023,1,9]],"date-time":"2023-01-09T04:47:08Z","timestamp":1673239628000},"page":"370","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["Learning Lightweight and Superior Detectors with Feature Distillation for Onboard Remote Sensing Object Detection"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1105-5835","authenticated-orcid":false,"given":"Lingyun","family":"Gu","sequence":"first","affiliation":[{"name":"Institute of Electronics and Telecommunications, Peter the Great Saint-Petersburg Polytechnic University, 195251 Saint Petersburg, Russia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7858-1525","authenticated-orcid":false,"given":"Qingyun","family":"Fang","sequence":"additional","affiliation":[{"name":"School of Aerospace Engineering, Tsinghua University, Beijing 100084, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2568-7948","authenticated-orcid":false,"given":"Zhaokui","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Aerospace Engineering, Tsinghua University, Beijing 100084, China"}]},{"given":"Eugene","family":"Popov","sequence":"additional","affiliation":[{"name":"Institute of Electronics and Telecommunications, Peter the Great Saint-Petersburg Polytechnic University, 195251 Saint Petersburg, Russia"}]},{"given":"Ge","family":"Dong","sequence":"additional","affiliation":[{"name":"School of Aerospace Engineering, Tsinghua University, Beijing 100084, China"}]}],"member":"1968","published-online":{"date-parts":[[2023,1,7]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"103952","DOI":"10.1016\/j.engappai.2020.103952","article-title":"CubeSatNet: Ultralight Convolutional Neural Network designed for on-orbit binary image classification on a 1U CubeSat","volume":"96","author":"Maskey","year":"2020","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_2","unstructured":"Lingyun, G., Popov, E., and Ge, D. (2022). Fast Fourier Convolution Based Remote Sensor Image Object Detection for Earth Observation. arXiv."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016, January 11\u201314). Ssd: Single shot multibox detector. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46448-0_2"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27\u201330). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref_5","first-page":"1137","article-title":"Faster r-cnn: Towards real-time object detection with region proposal networks","volume":"28","author":"Ren","year":"2015","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1016\/j.actaastro.2011.12.014","article-title":"A survey and assessment of the capabilities of Cubesats for Earth observation","volume":"74","author":"Selva","year":"2012","journal-title":"Acta Astronaut."},{"key":"ref_7","unstructured":"Manning, J., Langerman, D., Ramesh, B., Gretok, E., Wilson, C., George, A., MacKinnon, J., and Crum, G. (2018). Machine-Learning Space Applications on Smallsat Platforms with Tensorflow, NASA Goddard Space Flight Center."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Arechiga, A.P., Michaels, A.J., and Black, J.T. (2018, January 11). Onboard image processing for small satellites. Proceedings of the NAECON 2018-IEEE National Aerospace and Electronics Conference, Dayton, OH, USA.","DOI":"10.1109\/NAECON.2018.8556744"},{"key":"ref_9","unstructured":"Huq, R., Islam, M., and Siddique, S. (2018, January 18\u201320). AI-OBC: Conceptual Design of a Deep Neural Network based Next Generation Onboard Computing Architecture for Satellite Systems. Proceedings of the 1st China Microsatellite Symposium, Xi\u2019an, China."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Giuffrida, G., Diana, L., de Gioia, F., Benelli, G., Meoni, G., Donati, M., and Fanucci, L. (2020). Cloudscout: A deep neural network for on-board cloud detection on hyperspectral images. Remote Sens., 12.","DOI":"10.3390\/rs12142205"},{"key":"ref_11","unstructured":"Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Wu, J., Leng, C., Wang, Y., Hu, Q., and Cheng, J. (2016, January 27\u201330). Quantized convolutional neural networks for mobile devices. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.521"},{"key":"ref_13","unstructured":"Kim, J., Park, S., and Kwak, N. (2018, January 3\u20138). Paraphrasing complex network: Network compression via factor transfer. Proceedings of the Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, Montr\u00e9al, QC, Canada."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Mirzadeh, S.I., Farajtabar, M., Li, A., Levine, N., Matsukawa, A., and Ghasemzadeh, H. (2020, January 7\u201312). Improved knowledge distillation via teacher assistant. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i04.5963"},{"key":"ref_15","unstructured":"Lee, S., and Song, B.C. (2019). Graph-based knowledge distillation by multi-head attention network. arXiv."},{"key":"ref_16","unstructured":"Tung, F., and Mori, G. (November, January 27). Similarity-preserving knowledge distillation. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Republic of Korea."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"431","DOI":"10.1109\/LGRS.2020.2975541","article-title":"Cross-scale feature fusion for object detection in optical remote sensing images","volume":"18","author":"Cheng","year":"2020","journal-title":"IEEE Geosci. Remote. Sens. Lett."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1839","DOI":"10.1109\/COMST.2020.2990499","article-title":"Cubesat communications: Recent advances and future challenges","volume":"22","author":"Saeed","year":"2020","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"2111","DOI":"10.1029\/2018GL081584","article-title":"Arctic-Boreal lake dynamics revealed using CubeSat imagery","volume":"46","author":"Cooley","year":"2019","journal-title":"Geophys. Res. Lett."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Houborg, R., and McCabe, M.F. (2018). Daily Retrieval of NDVI and LAI at 3 m Resolution via the Fusion of CubeSat, Landsat, and MODIS Data. Remote Sens., 10.","DOI":"10.3390\/rs10060890"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Altena, B., and K\u00e4\u00e4b, A. (2017, January 27\u201329). Glacier ice loss monitored through the Planet cubesat constellation. Proceedings of the 2017 9th International Workshop on the Analysis of Multitemporal Remote Sensing Images (MultiTemp), Bruges, Belgium.","DOI":"10.1109\/Multi-Temp.2017.8035235"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"111534","DOI":"10.1016\/j.rse.2019.111534","article-title":"Using deep learning to map retrogressive thaw slumps in the Beiluhe region (Tibetan Plateau) from CubeSat images","volume":"237","author":"Huang","year":"2020","journal-title":"Remote Sens. Environ."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Ghuffar, S. (2018). DEM generation from multi satellite PlanetScope imagery. Remote Sens., 10.","DOI":"10.3390\/rs10091462"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Toorian, A., Diaz, K., and Lee, S. (2008, January 8\u201311). The cubesat approach to space access. Proceedings of the 2008 IEEE Aerospace Conference, Tampa, FL, USA.","DOI":"10.1109\/AERO.2008.4526293"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23\u201328). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.81"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Girshick, R. (2015, January 7\u201313). Fast r-cnn. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.169"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"296","DOI":"10.1016\/j.isprsjprs.2019.11.023","article-title":"Object detection in optical remote sensing images: A survey and a new benchmark","volume":"159","author":"Li","year":"2020","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Zhang, K., and Shen, H. (2022). Multi-Stage Feature Enhancement Pyramid Network for Detecting Objects in Optical Remote Sensing Images. Remote Sens., 14.","DOI":"10.3390\/rs14030579"},{"key":"ref_29","first-page":"801920","article-title":"Guiding clean features for object detection in remote sensing images","volume":"19","author":"Cheng","year":"2021","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Li, Y., Yao, T., Pan, Y., and Mei, T. (2022). Contextual transformer networks for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell.","DOI":"10.1109\/TPAMI.2022.3164083"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"108786","DOI":"10.1016\/j.patcog.2022.108786","article-title":"Cross-Modality Attentive Feature Fusion for Object Detection in Multispectral Remote Sensing Imagery","volume":"130","author":"Qingyun","year":"2022","journal-title":"Pattern Recognit."},{"key":"ref_32","unstructured":"Hinton, G., Vinyals, O., and Dean, J. (2015). Distilling the knowledge in a neural network. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1789","DOI":"10.1007\/s11263-021-01453-z","article-title":"Knowledge distillation: A survey","volume":"129","author":"Gou","year":"2021","journal-title":"Int. J. Comput. Vis."},{"key":"ref_34","unstructured":"Peng, Z., Li, Z., Zhang, J., Li, Y., Qi, G.J., and Tang, J. (November, January 27). Few-shot image recognition with knowledge transfer. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Republic of Korea."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"2168","DOI":"10.1109\/TVCG.2019.2903943","article-title":"Deepvid: Deep visual interpretation and diagnosis for image classifiers via knowledge distillation","volume":"25","author":"Wang","year":"2019","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"2415","DOI":"10.1109\/TMI.2019.2963882","article-title":"Unpaired multi-modal segmentation via knowledge distillation","volume":"39","author":"Dou","year":"2020","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Hou, Y., Ma, Z., Liu, C., Hui, T.W., and Loy, C.C. (2020, January 14\u201319). Inter-region affinity distillation for road marking segmentation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01250"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Wang, X., Hu, J.F., Lai, J.H., Zhang, J., and Zheng, W.S. (2019, January 16\u201320). Progressive teacher-student learning for early action prediction. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00367"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Wu, M.C., Chiu, C.T., and Wu, K.H. (2019, January 12\u201317). Multi-teacher knowledge distillation for compressed video action recognition on deep neural networks. Proceedings of the ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK.","DOI":"10.1109\/ICASSP.2019.8682450"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Cun, X., and Pun, C.M. (2020, January 23\u201328). Defocus blur detection via depth distillation. Proceedings of the European Conference on Computer Vision, Glasgow, UK.","DOI":"10.1007\/978-3-030-58601-0_44"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Chawla, A., Yin, H., Molchanov, P., and Alvarez, J. (2021, January 20\u201325). Data-free knowledge distillation for object detection. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision, Nashville, TN, USA.","DOI":"10.1109\/WACV48630.2021.00333"},{"key":"ref_42","unstructured":"Zhao, H., Sun, X., Dong, J., Chen, C., and Dong, Z. (2020). Highlight every step: Knowledge distillation via collaborative teaching. IEEE Trans. Cybern."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Doll\u00e1r, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017, January 21\u201326). Feature pyramid networks for object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.106"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"93058","DOI":"10.1109\/ACCESS.2020.2993998","article-title":"An efficient feature pyramid network for object detection in remote sensing imagery","volume":"8","author":"Qingyun","year":"2020","journal-title":"IEEE Access"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Xia, G.S., Bai, X., Ding, J., Zhu, Z., Belongie, S., Luo, J., Datcu, M., Pelillo, M., and Zhang, L. (2018, January 18\u201323). DOTA: A large-scale dataset for object detection in aerial images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00418"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll\u00e1r, P., and Zitnick, C.L. (2014, January 6\u201312). Microsoft coco: Common objects in context. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"ref_47","first-page":"16468","article-title":"Instance-conditional knowledge distillation for object detection","volume":"34","author":"Kang","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Yang, Z., Li, Z., Jiang, X., Gong, Y., Yuan, Z., Zhao, D., and Yuan, C. (2022, January 18\u201324). Focal and global knowledge distillation for detectors. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00460"},{"key":"ref_49","unstructured":"Chen, K., Wang, J., Pang, J., Cao, Y., Xiong, Y., Li, X., Sun, S., Feng, W., Liu, Z., and Xu, J. (2019). MMDetection: Open mmlab detection toolbox and benchmark. arXiv."},{"key":"ref_50","unstructured":"Redmon, J., and Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Goyal, P., Girshick, R., He, K., and Doll\u00e1r, P. (2017, January 22\u201329). Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.324"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Xie, S., Girshick, R., Doll\u00e1r, P., Tu, Z., and He, K. (2017, January 21\u201326). Aggregated residual transformations for deep neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.634"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Yang, Z., Li, Z., Shao, M., Shi, D., Yuan, Z., and Yuan, C. (2022). Masked Generative Distillation. arXiv.","DOI":"10.1007\/978-3-031-20083-0_4"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Wang, T., Yuan, L., Zhang, X., and Feng, J. (2019, January 16\u201320). Distilling object detectors with fine-grained feature imitation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00507"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Dai, X., Jiang, Z., Wu, Z., Bao, Y., Wang, Z., Liu, S., and Zhou, E. (2021, January 20). General instance distillation for object detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00775"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/2\/370\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:02:40Z","timestamp":1760119360000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/2\/370"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,7]]},"references-count":56,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,1]]}},"alternative-id":["rs15020370"],"URL":"https:\/\/doi.org\/10.3390\/rs15020370","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,7]]}}}