{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,11]],"date-time":"2026-07-11T15:19:11Z","timestamp":1783783151871,"version":"3.55.0"},"reference-count":48,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2024,8,28]],"date-time":"2024-08-28T00:00:00Z","timestamp":1724803200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,8,28]],"date-time":"2024-08-28T00:00:00Z","timestamp":1724803200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001774","name":"University of Sydney","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001774","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Comput Vis"],"published-print":{"date-parts":[[2025,2]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>Few-shot object detection (FSOD) studies how to detect novel objects with few annotated examples effectively. Recently, it has been demonstrated that decent feature embeddings, including the general feature embeddings that are more invariant to visual changes and the specific feature embeddings that are more discriminative for different object classes, are both important for FSOD. However, current methods lack appropriate mechanisms to sensibly cooperate both types of feature embeddings based on their importance to detecting objects of novel classes, which may result in sub-optimal performance. In this paper, to achieve more effective FSOD, we attempt to explicitly encode both general and specific feature embeddings using learnable tensors and apply a Transformer to help better incorporate them in FSOD according to their relations to the input object features. We thus propose a Transformer-based general and specific embedding learning (T-GSEL) method for FSOD. In T-GSEL, learnable tensors are employed in a three-stage pipeline, encoding feature embeddings in general level, intermediate level, and specific level, respectively. In each stage, we apply a Transformer to first model the relations of the corresponding embedding to input object features and then apply the estimated relations to refine the input features. Meanwhile, we further introduce cross-stage connections between embeddings of different stages to make them complement and cooperate with each other, delivering general, intermediate, and specific feature embeddings stage by stage and utilizing them together for feature refinement in FSOD. In practice, a T-GSEL module is easy to inject. Extensive empirical results further show that our proposed T-GSEL method achieves compelling FSOD performance on both PASCAL VOC and MS COCO datasets compared with other state-of-the-art approaches.<\/jats:p>","DOI":"10.1007\/s11263-024-02199-0","type":"journal-article","created":{"date-parts":[[2024,8,28]],"date-time":"2024-08-28T07:02:29Z","timestamp":1724828549000},"page":"968-984","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["Learning General and Specific Embedding with Transformer for Few-Shot Object Detection"],"prefix":"10.1007","volume":"133","author":[{"given":"Xu","family":"Zhang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhe","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jing","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tongliang","family":"Liu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7225-5449","authenticated-orcid":false,"given":"Dacheng","family":"Tao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,8,28]]},"reference":[{"key":"2199_CR1","unstructured":"Ba, J. L., Kiros, J. R., & Hinton, G. E. (2016). Layer normalization. arXiv preprint arXiv:1607.06450"},{"key":"2199_CR2","doi-asserted-by":"crossref","unstructured":"Carion, N., Massa, F., Synnaeve, G., et\u00a0al (2020) End-to-end object detection with transformers. In European conference on computer vision, Springer, pp. 213\u2013229.","DOI":"10.1007\/978-3-030-58452-8_13"},{"key":"2199_CR3","doi-asserted-by":"crossref","unstructured":"Chen, H., Wang, Y., Wang, G., et\u00a0al. (2018). Lstd: A low-shot transfer detector for object detection. In Proceedings of the AAAI conference on artificial intelligence.","DOI":"10.1609\/aaai.v32i1.11716"},{"issue":"1","key":"2199_CR4","doi-asserted-by":"publisher","first-page":"142","DOI":"10.1007\/s11263-020-01370-7","volume":"129","author":"Z Chen","year":"2021","unstructured":"Chen, Z., Zhang, J., & Tao, D. (2021). Recursive context routing for object detection. International Journal of Computer Vision, 129(1), 142\u2013160.","journal-title":"International Journal of Computer Vision"},{"key":"2199_CR5","doi-asserted-by":"crossref","unstructured":"Chen, Z., Zhang, J., & Tao, D. (2022a) Recurrent glimpse-based decoder for detection with transformer. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp. 5260\u20135269.","DOI":"10.1109\/CVPR52688.2022.00519"},{"key":"2199_CR6","unstructured":"Chen, Z., Zhang, J., Xu, Y., et\u00a0al (2022b) Transformer-based context condensation for boosting feature pyramids in object detection. arXiv preprint arXiv:2207.06603"},{"key":"2199_CR7","doi-asserted-by":"crossref","unstructured":"Dai, Z., Cai, B., Lin, Y., et\u00a0al. (2021). Up-detr: Unsupervised pre-training for object detection with transformers. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp. 1601\u20131610.","DOI":"10.1109\/CVPR46437.2021.00165"},{"key":"2199_CR8","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., et\u00a0al. (2009) Imagenet: A large-scale hierarchical image database. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, IEEE, pp. 248\u2013255.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"2199_CR9","doi-asserted-by":"crossref","unstructured":"Deng, J., Guo, J., Xue, N., et\u00a0al. (2019). Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp. 4690\u20134699.","DOI":"10.1109\/CVPR.2019.00482"},{"issue":"2","key":"2199_CR10","doi-asserted-by":"publisher","first-page":"303","DOI":"10.1007\/s11263-009-0275-4","volume":"88","author":"M Everingham","year":"2010","unstructured":"Everingham, M., Van Gool, L., Williams, C. K., et al. (2010). The pascal visual object classes (voc) challenge. International Journal of Computer Vision, 88(2), 303\u2013338.","journal-title":"International Journal of Computer Vision"},{"issue":"1","key":"2199_CR11","doi-asserted-by":"publisher","first-page":"98","DOI":"10.1007\/s11263-014-0733-5","volume":"111","author":"M Everingham","year":"2015","unstructured":"Everingham, M., Eslami, S. A., Van Gool, L., et al. (2015). The pascal visual object classes challenge: A retrospective. International Journal of Computer Vision, 111(1), 98\u2013136.","journal-title":"International Journal of Computer Vision"},{"key":"2199_CR12","doi-asserted-by":"crossref","unstructured":"Fan, Q., Zhuo, W., Tang, C. K.., et\u00a0al (2020). Few-shot object detection with attention-rpn and multi-relation detector. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp. 4013\u20134022.","DOI":"10.1109\/CVPR42600.2020.00407"},{"key":"2199_CR13","unstructured":"Finn, C., Abbeel, P., Levine, S. (2017). Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning, PMLR, pp. 1126\u20131135."},{"key":"2199_CR14","doi-asserted-by":"crossref","unstructured":"Gao, P., Zheng, M., Wang, X., et\u00a0al. (2021). Fast convergence of detr with spatially modulated co-attention. arXiv preprint arXiv:2101.07448","DOI":"10.1109\/ICCV48922.2021.00360"},{"key":"2199_CR15","unstructured":"Garcia, V., & Bruna, J. (2017) Few-shot learning with graph neural networks. arXiv preprint arXiv:1711.04043"},{"key":"2199_CR16","doi-asserted-by":"crossref","unstructured":"Girshick, R., Donahue, J., & Darrell, T., et\u00a0al. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp. 580\u2013587.","DOI":"10.1109\/CVPR.2014.81"},{"key":"2199_CR17","doi-asserted-by":"crossref","unstructured":"Han, G., Ma, J., & Huang, S., et\u00a0al (2022). Few-shot object detection with fully cross-transformer. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp. 5321\u20135330.","DOI":"10.1109\/CVPR52688.2022.00525"},{"key":"2199_CR18","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., et\u00a0al. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp. 770\u2013778.","DOI":"10.1109\/CVPR.2016.90"},{"key":"2199_CR19","doi-asserted-by":"crossref","unstructured":"He, K., Gkioxari, G., Doll\u00e1r, P., et\u00a0al. (2017). Mask r-cnn. In Proceedings of the IEEE\/CVF international conference on computer vision, pp. 2961\u20132969.","DOI":"10.1109\/ICCV.2017.322"},{"key":"2199_CR20","doi-asserted-by":"crossref","unstructured":"Kang, B., Liu, Z., Wang, X., et\u00a0al. (2019). Few-shot object detection via feature reweighting. In Proceedings of the IEEE\/CVF international conference on computer vision, pp. 8420\u20138429.","DOI":"10.1109\/ICCV.2019.00851"},{"key":"2199_CR21","doi-asserted-by":"crossref","unstructured":"Karlinsky, L., Shtok, J., Harary, S., et\u00a0al. (2019). Repmet: Representative-based metric learning for classification and few-shot object detection. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp. 5197\u20135206.","DOI":"10.1109\/CVPR.2019.00534"},{"key":"2199_CR22","doi-asserted-by":"crossref","unstructured":"Kaul, P., Xie, W., Zisserman, A. (2022). Label, verify, correct: A simple few shot object detection method. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp. 14237\u201314247.","DOI":"10.1109\/CVPR52688.2022.01384"},{"key":"2199_CR23","unstructured":"Khodadadeh, S., B\u00f6l\u00f6ni, L., & Shah, M. (2018) Unsupervised meta-learning for few-shot image classification. arXiv preprint arXiv:1811.11819"},{"key":"2199_CR24","doi-asserted-by":"crossref","unstructured":"Li, Y., Zhu, H., Cheng, Y., et\u00a0al. (2021). Few-shot object detection via classification refinement and distractor retreatment. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp. 15395\u201315403.","DOI":"10.1109\/CVPR46437.2021.01514"},{"key":"2199_CR25","doi-asserted-by":"crossref","unstructured":"Lin, T. Y., Maire, M., Belongie, S., et\u00a0al. (2014). Microsoft coco: Common objects in context. In European conference on computer vision. Springer, pp. 740\u2013755.","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"2199_CR26","doi-asserted-by":"crossref","unstructured":"Lin, T. Y., Doll\u00e1r, P., Girshick, R., et\u00a0al. (2017). Feature pyramid networks for object detection. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp. 2117\u20132125.","DOI":"10.1109\/CVPR.2017.106"},{"key":"2199_CR27","unstructured":"Lyle, C., Kwiatkowksa, M., & Gal, Y. (2019) An analysis of the effect of invariance on generalization in neural networks. In International conference on machine learning workshop on understanding and improving generalization in deep learning."},{"key":"2199_CR28","doi-asserted-by":"crossref","unstructured":"Meng, D., Chen, X., Fan, Z., et\u00a0al. (2021). Conditional detr for fast training convergence. In Proceedings of the IEEE\/CVF international conference on computer vision, pp. 3651\u20133660.","DOI":"10.1109\/ICCV48922.2021.00363"},{"key":"2199_CR29","unstructured":"Nichol, A., & Schulman, J. (2018). Reptile: A scalable metalearning algorithm. arXiv preprint arXiv:1803.02999"},{"key":"2199_CR30","doi-asserted-by":"crossref","unstructured":"Qiao, L., Zhao, Y., Li, Z., et\u00a0al. (2021). Defrcn: Decoupled faster r-cnn for few-shot object detection. In Proceedings of the IEEE\/CVF international conference on computer vision, pp. 8681\u20138690.","DOI":"10.1109\/ICCV48922.2021.00856"},{"key":"2199_CR31","unstructured":"Ren, M., Triantafillou, E., Ravi, S., et\u00a0al. (2018). Meta-learning for semi-supervised few-shot classification. arXiv preprint arXiv:1803.00676"},{"issue":"6","key":"2199_CR32","doi-asserted-by":"publisher","first-page":"1137","DOI":"10.1109\/TPAMI.2016.2577031","volume":"39","author":"S Ren","year":"2016","unstructured":"Ren, S., He, K., Girshick, R., et al. (2016). Faster r-cnn: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1137\u20131149.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"2199_CR33","unstructured":"Snell, J., Swersky, K., & Zemel, R. S. (2017) Prototypical networks for few-shot learning. arXiv preprint arXiv:1703.05175"},{"key":"2199_CR34","doi-asserted-by":"crossref","unstructured":"Sun, B., Li, B., Cai, S., et\u00a0al. (2021). Fsce: Few-shot object detection via contrastive proposal encoding. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp. 7352\u20137362.","DOI":"10.1109\/CVPR46437.2021.00727"},{"key":"2199_CR35","doi-asserted-by":"crossref","unstructured":"Sun, Q., Liu, Y., Chua, T. S., et\u00a0al. (2019) Meta-transfer learning for few-shot learning. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp. 403\u2013412.","DOI":"10.1109\/CVPR.2019.00049"},{"key":"2199_CR36","unstructured":"Sun, Y. (2015). Deep learning face representation by joint identification-verification. The Chinese University of Hong Kong (Hong Kong)."},{"key":"2199_CR37","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., et\u00a0al. (2017). Attention is all you need. In Advances in neural information processing systems, pp. 5998\u20136008."},{"key":"2199_CR38","doi-asserted-by":"crossref","unstructured":"Wang, H., Wang, Y., Zhou, Z., et\u00a0al. (2018). Cosface: Large margin cosine loss for deep face recognition. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp. 5265\u20135274.","DOI":"10.1109\/CVPR.2018.00552"},{"key":"2199_CR39","unstructured":"Wang, X., Huang, T. E., Darrell, T., et\u00a0al. (2020). Frustratingly simple few-shot object detection. arXiv preprint arXiv:2003.06957"},{"key":"2199_CR40","unstructured":"Wang, Y., Zhang, X., Yang, T., et\u00a0al (2021) Anchor detr: Query design for transformer-based detector. arXiv preprint arXiv:2109.07107"},{"key":"2199_CR41","doi-asserted-by":"crossref","unstructured":"Wang, Y. X., Ramanan, D., & Hebert, M. (2019) Meta-learning to detect rare objects. In Proceedings of the IEEE\/CVF international conference on computer vision, pp. 9925\u20139934.","DOI":"10.1109\/ICCV.2019.01002"},{"key":"2199_CR42","doi-asserted-by":"crossref","unstructured":"Wu, A., Han, Y., Zhu, L., et\u00a0al. (2021). Universal-prototype enhancing for few-shot object detection. In Proceedings of the IEEE\/CVF international conference on computer vision, pp. 9567\u20139576.","DOI":"10.1109\/ICCV48922.2021.00943"},{"key":"2199_CR43","doi-asserted-by":"crossref","unstructured":"Wu, J., Liu, S., Huang, D., et\u00a0al. (2020a). Multi-scale positive sample refinement for few-shot object detection. In European conference on computer vision, Springer, pp. 456\u2013472.","DOI":"10.1007\/978-3-030-58517-4_27"},{"key":"2199_CR44","doi-asserted-by":"crossref","unstructured":"Wu, X., Sahoo, D., & Hoi, S. (2020b). Meta-rcnn: Meta learning for few-shot object detection. In Proceedings of the 28th ACM international conference on multimedia, pp. 1679\u20131687.","DOI":"10.1145\/3394171.3413832"},{"key":"2199_CR45","doi-asserted-by":"crossref","unstructured":"Xiao, Y., Marlet, R. (2020). Few-shot object detection and viewpoint estimation for objects in the wild. In European conference on computer vision, Springer, pp. 192\u2013210.","DOI":"10.1007\/978-3-030-58520-4_12"},{"key":"2199_CR46","doi-asserted-by":"crossref","unstructured":"Yan, X., Chen, Z., Xu, A., et\u00a0al. (2019). Meta r-cnn: Towards general solver for instance-level low-shot learning. In Proceedings of the IEEE\/CVF international conference on computer vision, pp. 9577\u20139586.","DOI":"10.1109\/ICCV.2019.00967"},{"key":"2199_CR47","doi-asserted-by":"crossref","unstructured":"Zhang, X., Wang, W., Chen, Z., et\u00a0al. (2023). Clamp: Prompt-based contrastive learning for connecting language and animal pose. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp. 23272\u201323281.","DOI":"10.1109\/CVPR52729.2023.02229"},{"key":"2199_CR48","unstructured":"Zhu, X., Su, W., Lu, L., et\u00a0al. (2020). Deformable detr: Deformable transformers for end-to-end object detection. arXiv preprint arXiv:2010.04159"}],"container-title":["International Journal of Computer Vision"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11263-024-02199-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11263-024-02199-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11263-024-02199-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,22]],"date-time":"2025-01-22T06:43:41Z","timestamp":1737528221000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11263-024-02199-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,28]]},"references-count":48,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,2]]}},"alternative-id":["2199"],"URL":"https:\/\/doi.org\/10.1007\/s11263-024-02199-0","relation":{},"ISSN":["0920-5691","1573-1405"],"issn-type":[{"value":"0920-5691","type":"print"},{"value":"1573-1405","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,8,28]]},"assertion":[{"value":"19 September 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 July 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 August 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}