{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,25]],"date-time":"2026-02-25T19:16:18Z","timestamp":1772046978541,"version":"3.50.1"},"reference-count":52,"publisher":"Springer Science and Business Media LLC","issue":"22","license":[{"start":{"date-parts":[[2023,9,18]],"date-time":"2023-09-18T00:00:00Z","timestamp":1694995200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,9,18]],"date-time":"2023-09-18T00:00:00Z","timestamp":1694995200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100004543","name":"China Scholarship Council","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100004543","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Appl Intell"],"published-print":{"date-parts":[[2023,11]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Few-shot learning aims at recognizing novel visual categories from very few labelled examples. Different from the existing few-shot classification methods that are mainly based on metric learning or meta-learning, in this work we focus on improving the representation capacity of feature extractors. For this purpose, we propose a new two-stage dual selective knowledge transfer (DSKT) framework, to guide models towards better optimization. Specifically, we first exploit an improved multi-task learning approach to train a feature extractor with robust representation capability as a teacher model. Then, we design an effective dual selective knowledge distillation method, which enables the student model to selectively learn knowledge from the teacher model and current samples, thereby improving the student model\u2019s ability to generalize on unseen classes. Extensive experimental results show that our DSKT achieves competitive performances on four well-known few-shot classification benchmarks.<\/jats:p>","DOI":"10.1007\/s10489-023-04994-7","type":"journal-article","created":{"date-parts":[[2023,9,18]],"date-time":"2023-09-18T05:01:35Z","timestamp":1695013295000},"page":"27779-27789","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Dual selective knowledge transfer for few-shot classification"],"prefix":"10.1007","volume":"53","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6035-7787","authenticated-orcid":false,"given":"Kai","family":"He","sequence":"first","affiliation":[]},{"given":"Nan","family":"Pu","sequence":"additional","affiliation":[]},{"given":"Mingrui","family":"Lao","sequence":"additional","affiliation":[]},{"given":"Erwin M.","family":"Bakker","sequence":"additional","affiliation":[]},{"given":"Michael S.","family":"Lew","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,9,18]]},"reference":[{"key":"4994_CR1","doi-asserted-by":"publisher","unstructured":"Afrasiyabi A, Lalonde J, Gagn\u00e9 C (2020) Associative alignment for few-shot image classification. In: ECCV, pp 18\u201335. https:\/\/doi.org\/10.1007\/978-3-030-58558-7_2","DOI":"10.1007\/978-3-030-58558-7_2"},{"key":"4994_CR2","doi-asserted-by":"publisher","unstructured":"Afrasiyabi A, Lalonde J, Gagn\u00e9 C (2021) Mixture-based feature space learning for few-shot image classification. In: ICCV, pp 9021\u20139031. https:\/\/doi.org\/10.1109\/ICCV48922.2021.00891","DOI":"10.1109\/ICCV48922.2021.00891"},{"key":"4994_CR3","unstructured":"Asano YM, Rupprecht C, Vedaldi A (2020) A critical analysis of self-supervision, or what we can learn from a single image. In: ICLR"},{"key":"4994_CR4","doi-asserted-by":"publisher","unstructured":"Baik S, Choi J, Kim H, Cho D, Min J, Lee KM (2021) Meta-learning with task-adaptive loss function for few-shot learning. In: ICCV, pp 9445\u20139454. https:\/\/doi.org\/10.1109\/ICCV48922.2021.00933","DOI":"10.1109\/ICCV48922.2021.00933"},{"key":"4994_CR5","doi-asserted-by":"publisher","unstructured":"Bansal T, Jha R, Munkhdalai T, McCallum A (2020) Self-supervised meta-learning for few-shot natural language classification tasks. In: EMNLP, pp 522\u2013534. https:\/\/doi.org\/10.18653\/v1\/2020.emnlp-main.38","DOI":"10.18653\/v1\/2020.emnlp-main.38"},{"key":"4994_CR6","unstructured":"Chen W, Liu Y, Kira Z, Wang YF, Huang J (2019) A closer look at few-shot classification. In: ICLR"},{"key":"4994_CR7","doi-asserted-by":"publisher","unstructured":"Dvornik N, Mairal J, Schmid C (2019) Diversity with cooperation: Ensemble methods for few-shot classification. In: ICCV, pp 3722\u20133730. https:\/\/doi.org\/10.1109\/ICCV.2019.00382","DOI":"10.1109\/ICCV.2019.00382"},{"key":"4994_CR8","unstructured":"Finn C, Abbeel P, Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In: ICML, pp 1126\u20131135"},{"key":"4994_CR9","unstructured":"Furlanello T, Lipton ZC, Tschannen M, Itti L, Anandkumar A (2018) Born-again neural networks. In: ICML, pp 1602\u20131611"},{"key":"4994_CR10","doi-asserted-by":"publisher","unstructured":"Gan C, Gong B, Liu K, Su H, Guibas LJ (2018) Geometry guided convolutional neural networks for self-supervised video representation learning. In: CVPR, pp 5589\u20135597. https:\/\/doi.org\/10.1109\/CVPR.2018.00586","DOI":"10.1109\/CVPR.2018.00586"},{"key":"4994_CR11","doi-asserted-by":"publisher","unstructured":"Gan C, Zhao H, Chen P, Cox DD, Torralba A (2019) Self-supervised moving vehicle tracking with stereo sound. In: ICCV, pp 7052\u20137061. https:\/\/doi.org\/10.1109\/ICCV.2019.00715","DOI":"10.1109\/ICCV.2019.00715"},{"issue":"9","key":"4994_CR12","doi-asserted-by":"publisher","first-page":"2495","DOI":"10.1007\/s13042-022-01539-1","volume":"13","author":"F Gao","year":"2022","unstructured":"Gao F, Cai L, Yang Z, Song S, Wu C (2022) Multi-distance metric network for few-shot learning. International Journal of Machine Learning and Cybernetics 13(9):2495\u20132506. https:\/\/doi.org\/10.1007\/s13042-022-01539-1","journal-title":"International Journal of Machine Learning and Cybernetics"},{"key":"4994_CR13","unstructured":"Gao Y, Fei N, Liu G, Lu Z, Xiang T (2021) Contrastive prototype learning with augmented embeddings for few-shot learning. In: UAI, pp 140\u2013150"},{"key":"4994_CR14","doi-asserted-by":"publisher","unstructured":"Gidaris S, Komodakis N (2018) Dynamic few-shot visual learning without forgetting. In: CVPR, pp 4367\u20134375. https:\/\/doi.org\/10.1109\/CVPR.2018.00459","DOI":"10.1109\/CVPR.2018.00459"},{"key":"4994_CR15","doi-asserted-by":"publisher","unstructured":"He J, Hong R, Liu X, Xu M, Sun Q (2022) Revisiting local descriptor for improved few-shot classification. ACM Trans Multim Comput Commun Appl 18(2s):127:1\u2013127:23. https:\/\/doi.org\/10.1145\/3511917","DOI":"10.1145\/3511917"},{"key":"4994_CR16","doi-asserted-by":"publisher","unstructured":"He J, Kortylewski A, Yuille AL (2023) CORL: compositional representation learning for few-shot classification. In: WACV, pp 3879\u20133888. https:\/\/doi.org\/10.1109\/WACV56688.2023.00388","DOI":"10.1109\/WACV56688.2023.00388"},{"key":"4994_CR17","doi-asserted-by":"publisher","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: CVPR, pp 770\u2013778. https:\/\/doi.org\/10.1109\/CVPR.2016.90","DOI":"10.1109\/CVPR.2016.90"},{"key":"4994_CR18","doi-asserted-by":"publisher","unstructured":"Hinton GE, Vinyals O, Dean J (2015) Distilling the knowledge in a neural network. arXiv https:\/\/doi.org\/10.48550\/arXiv:1503.02531","DOI":"10.48550\/arXiv:1503.02531"},{"key":"4994_CR19","unstructured":"Hou R, Chang H, Ma B, Shan S, Chen X (2019) Cross attention network for few-shot classification. In: NeurIPS, pp 4005\u20134016"},{"key":"4994_CR20","doi-asserted-by":"publisher","unstructured":"Jiao L, Zhang F, Liu F, Yang S, Li L, Feng Z, Qu R (2019) A survey of deep learning-based object detection. IEEE Access 7:128,837\u2013128,868. https:\/\/doi.org\/10.1109\/ACCESS.2019.2939201","DOI":"10.1109\/ACCESS.2019.2939201"},{"key":"4994_CR21","doi-asserted-by":"publisher","unstructured":"Lee K, Maji S, Ravichandran A, Soatto S (2019) Meta-learning with differentiable convex optimization. In: CVPR, pp 10,657\u201310,665. https:\/\/doi.org\/10.1109\/CVPR.2019.01091","DOI":"10.1109\/CVPR.2019.01091"},{"key":"4994_CR22","unstructured":"Leng Z, Tan M, Liu C, Cubuk ED, Shi J, Cheng S, Anguelov D (2022) Polyloss: A polynomial expansion perspective of classification loss functions. In: ICLR"},{"key":"4994_CR23","doi-asserted-by":"publisher","unstructured":"Li W, Wang L, Xu J, Huo J, Gao Y, Luo J (2019) Revisiting local descriptor based image-to-class measure for few-shot learning. In: CVPR, pp 7260\u20137268. https:\/\/doi.org\/10.1109\/CVPR.2019.00743","DOI":"10.1109\/CVPR.2019.00743"},{"key":"4994_CR24","doi-asserted-by":"publisher","first-page":"327","DOI":"10.1016\/j.neucom.2021.06.090","volume":"459","author":"JY Lim","year":"2021","unstructured":"Lim JY, Lim K, Ooi SY, Lee C (2021) Efficient-prototypicalnet with self knowledge distillation for few-shot learning. Neurocomputing 459:327\u2013337. https:\/\/doi.org\/10.1016\/j.neucom.2021.06.090","journal-title":"Neurocomputing"},{"key":"4994_CR25","doi-asserted-by":"crossref","unstructured":"Lin H, Han G, Ma J, Huang S, Lin X, Chang SF (2023) Supervised masked knowledge distillation for few-shot transformers. In: CVPR, pp 19,649\u201319,659","DOI":"10.1109\/CVPR52729.2023.01882"},{"key":"4994_CR26","doi-asserted-by":"publisher","unstructured":"Liu B, Cao Y, Lin Y, Li Q, Zhang Z, Long M, Hu H (2020) Negative margin matters: Understanding margin in few-shot classification. In: ECCV, pp 438\u2013455. https:\/\/doi.org\/10.1007\/978-3-030-58548-8_26","DOI":"10.1007\/978-3-030-58548-8_26"},{"key":"4994_CR27","doi-asserted-by":"publisher","unstructured":"Liu S, Wang Y (2021) Few-shot learning with online self-distillation. In: ICCVW, pp 1067\u20131070. https:\/\/doi.org\/10.1109\/ICCVW54120.2021.00124","DOI":"10.1109\/ICCVW54120.2021.00124"},{"key":"4994_CR28","doi-asserted-by":"crossref","unstructured":"Ma R, Fang P, Drummond T, Harandi M (2022) Adaptive poincar\u00e9 point to set distance for few-shot classification. In: AAAI, pp 1926\u20131934","DOI":"10.1609\/aaai.v36i2.20087"},{"key":"4994_CR29","unstructured":"Mishra N, Rohaninejad M, Chen X, Abbeel P (2018) A simple neural attentive meta-learner. In: ICLR"},{"key":"4994_CR30","doi-asserted-by":"publisher","unstructured":"Nichol A, Achiam J, Schulman J (2018) On first-order meta-learning algorithms. https:\/\/doi.org\/10.48550\/arXiv.1803.02999","DOI":"10.48550\/arXiv.1803.02999"},{"key":"4994_CR31","unstructured":"Oreshkin BN, L\u00f3pez PR, Lacoste A (2018) TADAM: task dependent adaptive metric for improved few-shot learning. In: NeurIPS, pp 719\u2013729"},{"key":"4994_CR32","unstructured":"Rajasegaran J, Khan S, Hayat M, Khan FS, Shah M (2021) Self-supervised knowledge distillation for few-shot learning. In: BMVC, p 179"},{"key":"4994_CR33","unstructured":"Ravi S, Larochelle H (2017) Optimization as a model for few-shot learning. In: ICLR"},{"key":"4994_CR34","doi-asserted-by":"publisher","unstructured":"Ravichandran A, Bhotika R, Soatto S (2019) Few-shot learning with embedded class models and shot-free meta training. In: ICCV, pp 331\u2013339. https:\/\/doi.org\/10.1109\/ICCV.2019.00042","DOI":"10.1109\/ICCV.2019.00042"},{"key":"4994_CR35","unstructured":"Ren M, Triantafillou E, Ravi S, Snell J, Swersky K, Tenenbaum JB, Larochelle H, Zemel RS (2018) Meta-learning for semi-supervised few-shot classification. In: ICLR"},{"key":"4994_CR36","doi-asserted-by":"crossref","unstructured":"Shen Z, Liu Z, Qin J, Savvides M, Cheng K (2021) Partial is better than all: Revisiting fine-tuning strategy for few-shot learning. In: AAAI, pp 9594\u20139602","DOI":"10.1609\/aaai.v35i11.17155"},{"key":"4994_CR37","doi-asserted-by":"publisher","unstructured":"Simon C, Koniusz P, Nock R, Harandi M (2020) Adaptive subspaces for few-shot learning. In: CVPR, pp 4135\u20134144. https:\/\/doi.org\/10.1109\/CVPR42600.2020.00419","DOI":"10.1109\/CVPR42600.2020.00419"},{"key":"4994_CR38","unstructured":"Snell J, Swersky K, Zemel RS (2017) Prototypical networks for few-shot learning. In: NeurIPS, pp 4077\u20134087"},{"key":"4994_CR39","doi-asserted-by":"publisher","unstructured":"Sun Q, Liu Y, Chua T, Schiele B (2019) Meta-transfer learning for few-shot learning. In: CVPR, pp 403\u2013412. https:\/\/doi.org\/10.1109\/CVPR.2019.00049","DOI":"10.1109\/CVPR.2019.00049"},{"key":"4994_CR40","doi-asserted-by":"publisher","unstructured":"Sung F, Yang Y, Zhang L, Xiang T, Torr PHS, Hospedales TM (2018) Learning to compare: Relation network for few-shot learning. In: CVPR, pp 1199\u20131208. https:\/\/doi.org\/10.1109\/CVPR.2018.00131","DOI":"10.1109\/CVPR.2018.00131"},{"key":"4994_CR41","doi-asserted-by":"publisher","unstructured":"Tian Y, Wang Y, Krishnan D, Tenenbaum JB, Isola P (2020) Rethinking few-shot image classification: A good embedding is all you need? In: ECCV, pp 266\u2013282. https:\/\/doi.org\/10.1007\/978-3-030-58568-6_16","DOI":"10.1007\/978-3-030-58568-6_16"},{"key":"4994_CR42","unstructured":"Vinyals O, Blundell C, Lillicrap T, Kavukcuoglu K, Wierstra D (2016) Matching networks for one shot learning. In: NeurIPS, pp 3630\u20133638"},{"key":"4994_CR43","unstructured":"Wah C, Branson S, Welinder P, Perona P, Belongie S (2011) The caltech-ucsd birds-200-2011 dataset"},{"key":"4994_CR44","doi-asserted-by":"publisher","unstructured":"Wertheimer D, Tang L, Hariharan B (2021) Few-shot classification with feature map reconstruction networks. In: CVPR, pp 8012\u20138021. https:\/\/doi.org\/10.1109\/CVPR46437.2021.00792","DOI":"10.1109\/CVPR46437.2021.00792"},{"key":"4994_CR45","doi-asserted-by":"publisher","unstructured":"Ye H, Hu H, Zhan D, Sha F (2018) Learning embedding adaptation for few-shot learning. arXiv:arXiv1812.03664. https:\/\/doi.org\/10.48550\/arXiv.1812.03664","DOI":"10.48550\/arXiv.1812.03664"},{"key":"4994_CR46","doi-asserted-by":"publisher","unstructured":"Ye H, Hu H, Zhan D, Sha F (2020) Few-shot learning via embedding adaptation with set-to-set functions. In: CVPR, pp 8805\u20138814. https:\/\/doi.org\/10.1109\/CVPR42600.2020.00883","DOI":"10.1109\/CVPR42600.2020.00883"},{"key":"4994_CR47","unstructured":"Yoon SW, Seo J, Moon J (2019) Tapnet: Neural network augmented with task-adaptive projection for few-shot learning. In: ICML, pp 7115\u20137123"},{"key":"4994_CR48","doi-asserted-by":"publisher","unstructured":"Zhang C, Cai Y, Lin G, Shen C (2020) Deepemd: Few-shot image classification with differentiable earth mover\u2019s distance and structured classifiers. In: CVPR, pp 12,200\u201312,210. https:\/\/doi.org\/10.1109\/CVPR42600.2020.01222","DOI":"10.1109\/CVPR42600.2020.01222"},{"key":"4994_CR49","doi-asserted-by":"publisher","unstructured":"Zhang C, Ding H, Lin G, Li R, Wang C, Shen C (2021) Meta navigator: Search for a good adaptation policy for few-shot learning. In: ICCV, pp 9415\u20139424. https:\/\/doi.org\/10.1109\/ICCV48922.2021.00930","DOI":"10.1109\/ICCV48922.2021.00930"},{"key":"4994_CR50","doi-asserted-by":"publisher","unstructured":"Zhao B, Cui Q, Song R, Qiu Y, Liang J (2022) Decoupled knowledge distillation. In: CVPR, pp 11,943\u201311,952. https:\/\/doi.org\/10.1109\/CVPR52688.2022.01165","DOI":"10.1109\/CVPR52688.2022.01165"},{"key":"4994_CR51","doi-asserted-by":"publisher","unstructured":"Zhou F, Wu B, Li Z (2018) Deep meta-learning: Learning to learn in the concept space. arXiv:1802.03596. https:\/\/doi.org\/10.48550\/arXiv.1802.03596","DOI":"10.48550\/arXiv.1802.03596"},{"key":"4994_CR52","doi-asserted-by":"publisher","unstructured":"Zhou Z, Qiu X, Xie J, Wu J, Zhang C (2021) Binocular mutual learning for improving few-shot classification. In: ICCV, pp 8382\u20138391. https:\/\/doi.org\/10.1109\/ICCV48922.2021.00829","DOI":"10.1109\/ICCV48922.2021.00829"}],"container-title":["Applied Intelligence"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10489-023-04994-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10489-023-04994-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10489-023-04994-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,25]],"date-time":"2023-10-25T15:26:57Z","timestamp":1698247617000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10489-023-04994-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,18]]},"references-count":52,"journal-issue":{"issue":"22","published-print":{"date-parts":[[2023,11]]}},"alternative-id":["4994"],"URL":"https:\/\/doi.org\/10.1007\/s10489-023-04994-7","relation":{},"ISSN":["0924-669X","1573-7497"],"issn-type":[{"value":"0924-669X","type":"print"},{"value":"1573-7497","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,18]]},"assertion":[{"value":"31 August 2023","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 September 2023","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that there is no conflict of interests regarding the publication of this paper.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}