{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T14:28:08Z","timestamp":1783607288023,"version":"3.55.0"},"reference-count":53,"publisher":"MDPI AG","issue":"24","license":[{"start":{"date-parts":[[2024,12,16]],"date-time":"2024-12-16T00:00:00Z","timestamp":1734307200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004739","name":"Youth Innovation Promotion Association, CAS","doi-asserted-by":"publisher","award":["2022119"],"award-info":[{"award-number":["2022119"]}],"id":[{"id":"10.13039\/501100004739","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Few-shot object detection has attracted extensive attention due to the abomination of time-consuming or even impractical large-scale data labeling. Current studies attempted to employ prototype-matching approaches for object detection, constructing class prototypes from textual or visual features. However, single visual prototypes exhibit limited generalization in few-shot scenarios, while single textual prototypes lack the spatial details of remote sensing targets. Therefore, to achieve the best of both worlds, we propose a prototype aggregating module to integrate textual and visual prototypes, leveraging both semantics from textual prototypes and spatial details from visual prototypes. In addition, the transferability of multi-modal few-shot detectors from natural scenarios to remote sensing scenarios remains unexplored, and previous training strategies for FSOD do not adequately consider the characteristics of text encoders. To address the issue, we have conducted extensive ablation studies on different feature extractors of the detector and propose an efficient two-stage training strategy, which takes the characteristics of the text feature extractor into account. Experiments on two common few-shot detection benchmarks demonstrate the effectiveness of our proposed method. In four widely used data splits of DIOR, our method significantly outperforms previous state-of-the-art methods by at most 8.7%.<\/jats:p>","DOI":"10.3390\/rs16244693","type":"journal-article","created":{"date-parts":[[2024,12,16]],"date-time":"2024-12-16T10:08:53Z","timestamp":1734343733000},"page":"4693","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Multi-Modal Prototypes for Few-Shot Object Detection in Remote Sensing Images"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-8604-933X","authenticated-orcid":false,"given":"Yanxing","family":"Liu","sequence":"first","affiliation":[{"name":"Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China"},{"name":"Key Laboratory of Technology in Geo-Spatial Information Processing and Application System, Chinese Academy of Sciences, Beijing 100190, China"},{"name":"School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing 100049, China"},{"name":"Key Laboratory of Target Cognition and Application Technology (TCAT), Chinese Academy of Sciences, Beijing 100190, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5041-3300","authenticated-orcid":false,"given":"Zongxu","family":"Pan","sequence":"additional","affiliation":[{"name":"Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China"},{"name":"Key Laboratory of Technology in Geo-Spatial Information Processing and Application System, Chinese Academy of Sciences, Beijing 100190, China"},{"name":"School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing 100049, China"},{"name":"Key Laboratory of Target Cognition and Application Technology (TCAT), Chinese Academy of Sciences, Beijing 100190, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5228-8462","authenticated-orcid":false,"given":"Jianwei","family":"Yang","sequence":"additional","affiliation":[{"name":"Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China"},{"name":"Key Laboratory of Technology in Geo-Spatial Information Processing and Application System, Chinese Academy of Sciences, Beijing 100190, China"},{"name":"School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing 100049, China"},{"name":"Key Laboratory of Target Cognition and Application Technology (TCAT), Chinese Academy of Sciences, Beijing 100190, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peiling","family":"Zhou","sequence":"additional","affiliation":[{"name":"Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China"},{"name":"Key Laboratory of Technology in Geo-Spatial Information Processing and Application System, Chinese Academy of Sciences, Beijing 100190, China"},{"name":"School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing 100049, China"},{"name":"Key Laboratory of Target Cognition and Application Technology (TCAT), Chinese Academy of Sciences, Beijing 100190, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bingchen","family":"Zhang","sequence":"additional","affiliation":[{"name":"Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China"},{"name":"Key Laboratory of Technology in Geo-Spatial Information Processing and Application System, Chinese Academy of Sciences, Beijing 100190, China"},{"name":"School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing 100049, China"},{"name":"Key Laboratory of Target Cognition and Application Technology (TCAT), Chinese Academy of Sciences, Beijing 100190, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,12,16]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Chen, H., Wang, Y., Wang, G., and Qiao, Y. (2018, January 2\u20137). Lstd: A low-shot transfer detector for object detection. Proceedings of theAAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11716"},{"key":"ref_2","unstructured":"Wang, X., Huang, T.E., Darrell, T., Gonzalez, J.E., and Yu, F. (2020). Frustratingly simple few-shot object detection. arXiv."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Sun, B., Li, B., Cai, S., Yuan, Y., and Zhang, C. (2021, January 19\u201325). Fsce: Few-shot object detection via contrastive proposal encoding. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00727"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1016\/j.isprsjprs.2022.12.004","article-title":"Generalized few-shot object detection in remote sensing images","volume":"195","author":"Zhang","year":"2023","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_5","unstructured":"Kang, B., Liu, Z., Wang, X., Yu, F., Feng, J., and Darrell, T. (November, January 27). Few-shot object detection via feature reweighting. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Republic of Korea."},{"key":"ref_6","unstructured":"Yan, X., Chen, Z., Xu, A., Wang, X., Liang, X., and Lin, L. (November, January 27). Meta r-cnn: Towards general solver for instance-level low-shot learning. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Republic of Korea."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Xiao, Y., and Marlet, R. (2020, January 23\u201328). Few-Shot Object Detection and Viewpoint Estimation for Objects in the Wild. Proceedings of the European Conference on Computer Vision (ECCV), Glasgow, UK.","DOI":"10.1007\/978-3-030-58520-4_12"},{"key":"ref_8","first-page":"1","article-title":"Prototype-CNN for few-shot object detection in remote sensing images","volume":"60","author":"Cheng","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Li, L.H., Zhang, P., Zhang, H., Yang, J., Li, C., Zhong, Y., Wang, L., Yuan, L., Zhang, L., and Hwang, J.N. (2022, January 19\u201320). Grounded language-image pre-training. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01069"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Liu, S., Zeng, Z., Ren, T., Li, F., Zhang, H., Yang, J., Li, C., Yang, J., Su, H., and Zhu, J. (2023). Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv.","DOI":"10.1007\/978-3-031-72970-6_3"},{"key":"ref_11","first-page":"1","article-title":"Few-shot object detection in aerial imagery guided by text-modal knowledge","volume":"61","author":"Lu","year":"2023","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_12","first-page":"1137","article-title":"Faster r-cnn: Towards real-time object detection with region proposal networks","volume":"28","author":"Ren","year":"2015","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"He, K., Gkioxari, G., Doll\u00e1r, P., and Girshick, R. (2017, January 22\u201329). Mask r-cnn. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.322"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Redmon, J., and Farhadi, A. (2017, January 21\u201326). YOLO9000: Better, faster, stronger. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.690"},{"key":"ref_15","unstructured":"Redmon, J., and Farhadi, A. (2018). YOLOv3: An Incremental Improvement. arXiv."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. (2020, January 23\u201328). End-to-end object detection with transformers. Proceedings of the European Conference on Computer Vision, Glasgow, UK.","DOI":"10.1007\/978-3-030-58452-8_13"},{"key":"ref_17","unstructured":"Zhu, X., Su, W., Lu, L., Li, B., Wang, X., and Dai, J. (2020). Deformable detr: Deformable transformers for end-to-end object detection. arXiv."},{"key":"ref_18","unstructured":"Zhang, H., Li, F., Liu, S., Zhang, L., Su, H., Zhu, J., Ni, L.M., and Shum, H.Y. (2022). Dino: Detr with improved denoising anchor boxes for end-to-end object detection. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Zareian, A., Rosa, K.D., Hu, D.H., and Chang, S.F. (2021, January 19\u201325). Open-vocabulary object detection using captions. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.01416"},{"key":"ref_20","unstructured":"Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., and Clark, J. (2021, January 18\u201324). Learning transferable visual models from natural language supervision. Proceedings of the International Conference on Machine Learning, PMLR, Virtual."},{"key":"ref_21","unstructured":"Gu, X., Lin, T.Y., Kuo, W., and Cui, Y. (2021). Open-vocabulary object detection via vision and language knowledge distillation. arXiv."},{"key":"ref_22","unstructured":"Xu, Y., Zhang, M., Fu, C., Chen, P., Yang, X., Li, K., and Xu, C. (2024). Multi-modal queried object detection in the wild. Adv. Neural Inf. Process. Syst., 36."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"296","DOI":"10.1016\/j.isprsjprs.2019.11.023","article-title":"Object detection in optical remote sensing images: A survey and a new benchmark","volume":"159","author":"Li","year":"2020","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_24","first-page":"1","article-title":"Rotation-invariant feature learning via convolutional neural network with cyclic polar coordinates convolutional layer","volume":"61","author":"Mei","year":"2023","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/j.isprsjprs.2018.04.003","article-title":"Multi-scale object detection in remote sensing imagery with convolutional neural networks","volume":"145","author":"Deng","year":"2018","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"7405","DOI":"10.1109\/TGRS.2016.2601622","article-title":"Learning rotation-invariant convolutional neural networks for object detection in VHR optical remote sensing images","volume":"54","author":"Cheng","year":"2016","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"7147","DOI":"10.1109\/TGRS.2018.2848901","article-title":"HSF-Net: Multiscale deep feature embedding for ship detection in optical remote sensing imagery","volume":"56","author":"Li","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_28","unstructured":"Yang, X., Yang, J., Yan, J., Zhang, Y., Zhang, T., Guo, Z., Sun, X., and Fu, K. (November, January 27). Scrdet: Towards more robust detection for small, cluttered and rotated objects. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Republic of Korea."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1452","DOI":"10.1109\/TPAMI.2020.2974745","article-title":"Gliding vertex on the horizontal bounding box for multi-oriented object detection","volume":"43","author":"Xu","year":"2020","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Qiao, L., Zhao, Y., Li, Z., Qiu, X., Wu, J., and Zhang, C. (2021, January 11\u201317). Defrcn: Decoupled faster r-cnn for few-shot object detection. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, BC, Canada.","DOI":"10.1109\/ICCV48922.2021.00856"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Fan, Q., Zhuo, W., Tang, C.K., and Tai, Y.W. (2020, January 13\u201319). Few-shot object detection with attention-RPN and multi-relation detector. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00407"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Lyu, Q., and Wang, W. (2023, January 7\u201314). Compositional prototypical networks for few-shot classification. Proceedings of the AAAI Conference on Artificial Intelligence, Washington, DC, USA.","DOI":"10.1609\/aaai.v37i7.26082"},{"key":"ref_33","first-page":"1","article-title":"Few-shot object detection on remote sensing images","volume":"60","author":"Li","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Le Jeune, P., Lebbah, M., Mokraoui, A., and Azzag, H. (2021, January 13\u201316). Experience feedback using representation learning for few-shot object detection on aerial images. Proceedings of the 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA), Virtual.","DOI":"10.1109\/ICMLA52953.2021.00110"},{"key":"ref_35","first-page":"1","article-title":"Two-Branch Attention Adversarial Domain Adaptation Network for Hyperspectral Image Classification","volume":"60","author":"Huang","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Wolf, S., Meier, J., Sommer, L., and Beyerer, J. (2021, January 11\u201317). Double Head Predictor based Few-Shot Object Detection for Aerial Imagery. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, BC, Canada.","DOI":"10.1109\/ICCVW54120.2021.00086"},{"key":"ref_37","first-page":"1","article-title":"Multiscale Object Contrastive Learning-Derived Few-Shot Object Detection in VHR Imagery","volume":"60","author":"Chen","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"102508","DOI":"10.1016\/j.inffus.2024.102508","article-title":"Unified multimodal fusion transformer for few shot object detection for remote sensing images","volume":"111","author":"Azeem","year":"2024","journal-title":"Inf. Fusion"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TGRS.2023.3334492","article-title":"Transformation-invariant network for few-shot object detection in remote-sensing images","volume":"61","author":"Liu","year":"2023","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_40","first-page":"1","article-title":"Few-Shot Object Detection in Remote-Sensing Images via Label-Consistent Classifier and Gradual Regression","volume":"62","author":"Liu","year":"2024","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"695","DOI":"10.1109\/TCSVT.2023.3287201","article-title":"Dynamic spatial focus for efficient compressed video action recognition","volume":"34","author":"Zheng","year":"2023","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"332","DOI":"10.1109\/TCDS.2023.3274214","article-title":"Adadet: An adaptive object detection system based on early-exit neural networks","volume":"16","author":"Yang","year":"2023","journal-title":"IEEE Trans. Cogn. Dev. Syst."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Yang, L., Jiang, H., Cai, R., Wang, Y., Song, S., Huang, G., and Tian, Q. (2021, January 19\u201325). Condensenet v2: Sparse feature reactivation for deep networks. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00357"},{"key":"ref_44","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Goyal, P., Girshick, R., He, K., and Doll\u00e1r, P. (2017, January 22\u201329). Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.324"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Li, F., Zhang, H., Liu, S., Guo, J., Ni, L.M., and Zhang, L. (2022, January 19\u201320). Dn-detr: Accelerate detr training by introducing query denoising. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01325"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1016\/j.isprsjprs.2014.10.002","article-title":"Multi-class geospatial object detection and geographic image classification based on collection of part detectors","volume":"98","author":"Cheng","year":"2014","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1007\/s11263-009-0275-4","article-title":"The pascal visual object classes (voc) challenge","volume":"88","author":"Everingham","year":"2010","journal-title":"Int. J. Comput. Vis."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 11\u201317). Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, BC, Canada.","DOI":"10.1109\/ICCV48922.2021.00986"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Rukhovich, D., Sofiiuk, K., Galeev, D., Barinova, O., and Konushin, A. (2021, January 21\u201322). Iterdet: Iterative scheme for object detection in crowded environments. Proceedings of the Structural, Syntactic, and Statistical Pattern Recognition: Joint IAPR International Workshops, S+ SSPR 2020, Padua, Italy. Proceedings.","DOI":"10.1007\/978-3-030-73973-7_33"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Wang, D., Zhang, J., Xu, M., Liu, L., Wang, D., Gao, E., Han, C., Guo, H., Du, B., and Tao, D. (2024). MTP: Advancing remote sensing foundation model via multi-task pretraining. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.","DOI":"10.1109\/JSTARS.2024.3408154"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Jiang, W., Zhang, J., Wang, D., Zhang, Q., Wang, Z., and Du, B. (2024). LeMeViT: Efficient vision transformer with learnable meta tokens for remote sensing image interpretation. arXiv.","DOI":"10.24963\/ijcai.2024\/103"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Wang, Z., Yang, B., Yue, H., and Ma, Z. (2024, January 20\u201327). Fine-Grained Prototypes Distillation for Few-Shot Object Detection. Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada.","DOI":"10.1609\/aaai.v38i6.28399"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/24\/4693\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:53:14Z","timestamp":1760115194000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/24\/4693"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,16]]},"references-count":53,"journal-issue":{"issue":"24","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["rs16244693"],"URL":"https:\/\/doi.org\/10.3390\/rs16244693","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,16]]}}}