{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:54:06Z","timestamp":1760144046433,"version":"build-2065373602"},"reference-count":28,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2024,3,18]],"date-time":"2024-03-18T00:00:00Z","timestamp":1710720000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003819","name":"Natural Science Foundation of Hubei Province","doi-asserted-by":"publisher","award":["2023AFC010","K2023056"],"award-info":[{"award-number":["2023AFC010","K2023056"]}],"id":[{"id":"10.13039\/501100003819","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Science Foundation of Wuhan Institute of Technology","award":["2023AFC010","K2023056"],"award-info":[{"award-number":["2023AFC010","K2023056"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>With the development of deep learning and sensors and sensor collection methods, computer vision inspection technology has developed rapidly. The deep-learning-based classification algorithm requires the acquisition of a model with superior generalization capabilities through the utilization of a substantial quantity of training samples. However, due to issues such as privacy, annotation costs, and sensor-captured images, how to make full use of limited samples has become a major challenge for practical training and deployment. Furthermore, when simulating models and transferring them to actual image scenarios, discrepancies often arise between the common training sets and the target domain (domain offset). Currently, meta-learning offers a promising solution for few-shot learning problems. However, the quantity of supporting set data on the target domain remains limited, leading to limited cross-domain learning effectiveness. To address this challenge, we have developed a self-distillation and mixing (SDM) method utilizing a Teacher\u2013Student framework. This method effectively transfers knowledge from the source domain to the target domain by applying self-distillation techniques and mixed data augmentation, learning better image representations from relatively abundant datasets, and achieving fine-tuning in the target domain. In comparison with nine classical models, the experimental results demonstrate that the SDM method excels in terms of training time and accuracy. Furthermore, SDM effectively transfers knowledge from the source domain to the target domain, even with a limited number of target domain samples.<\/jats:p>","DOI":"10.3390\/s24061939","type":"journal-article","created":{"date-parts":[[2024,3,18]],"date-time":"2024-03-18T06:58:39Z","timestamp":1710745119000},"page":"1939","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Research on a Cross-Domain Few-Shot Adaptive Classification Algorithm Based on Knowledge Distillation Technology"],"prefix":"10.3390","volume":"24","author":[{"given":"Jiuyang","family":"Gao","sequence":"first","affiliation":[{"name":"Hubei Provincial Engineering Technology Research Center of Green Chemical Equipment, School of Mechanical and Electrical Engineering, Wuhan Institute of Technology, Wuhan 430205, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Siyu","family":"Li","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430205, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wenfeng","family":"Xia","sequence":"additional","affiliation":[{"name":"Hubei Provincial Engineering Technology Research Center of Green Chemical Equipment, School of Mechanical and Electrical Engineering, Wuhan Institute of Technology, Wuhan 430205, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2604-3503","authenticated-orcid":false,"given":"Jiuyang","family":"Yu","sequence":"additional","affiliation":[{"name":"Hubei Provincial Engineering Technology Research Center of Green Chemical Equipment, School of Mechanical and Electrical Engineering, Wuhan Institute of Technology, Wuhan 430205, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-9541-835X","authenticated-orcid":false,"given":"Yaonan","family":"Dai","sequence":"additional","affiliation":[{"name":"Hubei Provincial Engineering Technology Research Center of Green Chemical Equipment, School of Mechanical and Electrical Engineering, Wuhan Institute of Technology, Wuhan 430205, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,3,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"8048","DOI":"10.1007\/s11227-022-04992-5","article-title":"Deep ensemble transfer learning-based framework for mammographic image classification","volume":"79","author":"Oza","year":"2023","journal-title":"J. Supercomput."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"114887","DOI":"10.1016\/j.microrel.2022.114887","article-title":"Small sample classification based on data enhancement and its application in flip chip defection","volume":"141","author":"Sha","year":"2023","journal-title":"Microelectron. Reliab."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2711","DOI":"10.1002\/sim.9744","article-title":"A comparison of bias-adjusted generalized estimating equations for sparse binary data in small-sample longitudinal studies","volume":"42","author":"Gosho","year":"2023","journal-title":"Stat. Med."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"17273","DOI":"10.1021\/acs.analchem.3c03177","article-title":"Retention Time Prediction through Learning from a Small Training Data Set with a Pretrained Graph Neural Network","volume":"95","author":"Kwon","year":"2023","journal-title":"Anal. Chem."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"109201","DOI":"10.1016\/j.anucene.2022.109201","article-title":"Pre-trained network-based transfer learning: A small-sample machine learning approach to nuclear power plant classification problem","volume":"175","author":"Zhong","year":"2022","journal-title":"Ann. Nucl. Energy"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"105738","DOI":"10.1016\/j.compfluid.2022.105738","article-title":"A general deep transfer learning framework for predicting the flow field of airfoils with small data","volume":"251","author":"Wang","year":"2023","journal-title":"Comput. Fluids"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"109493","DOI":"10.1016\/j.knosys.2022.109493","article-title":"Cross-domain fault diagnosis of bearing using improved semi-supervised meta-learning towards interference of out-of-distribution samples","volume":"252","author":"Lin","year":"2022","journal-title":"Knowl.-Based Syst."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"108623","DOI":"10.1016\/j.knosys.2022.108623","article-title":"Cross-domain few-shot classification based on lightweight Res2Net and flexible GNN","volume":"247","author":"Chen","year":"2022","journal-title":"Knowl.-Based Syst."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Guo, Y., Codella, N.C., Karlinsky, L., Codella, J.V., Smith, J.R., Saenko, K., Rosing, T., and Feris, R. (2020, January 23\u201328). A broader study of cross-domain few-shot learning. Proceedings of the Computer Vision\u2013ECCV 2020: 16th European Conference, Glasgow, UK. Proceedings, Part XXVII 16.","DOI":"10.1007\/978-3-030-58583-9_8"},{"key":"ref_10","unstructured":"Sukhija, S., Krishnan, N.C., and Singh, G. (2016, January 9\u201315). Supervised Heterogeneous Domain Adaptation via Random Forests. Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI-16), New York, NY, USA."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Motiian, S., Piccirilli, M., Adjeroh, D.A., and Doretto, G. (2017, January 22\u201329). Unified deep supervised domain adaptation and generalization. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.609"},{"key":"ref_12","unstructured":"Motiian, S., Jones, Q., Iranmanesh, S., and Doretto, G. (2017). Few-shot adversarial domain adaptation. arXiv."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Xu, X., Zhou, X., Venkatesan, R., Swaminathan, G., and Majumder, O. (2019, January 15\u201320). d-sne: Domain adaptation using stochastic neighborhood embedding. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00260"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"18277","DOI":"10.1007\/s10489-022-04414-2","article-title":"Center transfer for supervised domain adaptation","volume":"53","author":"Huang","year":"2023","journal-title":"Appl. Intell."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Huang, F., Wang, Z., Huang, X., Qian, Y., Li, Z., and Chen, H. (2023, January 23). Aligning Distillation for Cold-start Item Recommendation. Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR \u201823). Association for Computing Machinery, New York, NY, USA.","DOI":"10.1145\/3539618.3591732"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Xu, M., Zhang, Z., Hu, H., Wang, J., Wang, L., Wei, F., Bai, X., and Liu, Z. (2021, January 11\u201317). End-to-end semi-supervised object detection with soft teacher. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Virtual.","DOI":"10.1109\/ICCV48922.2021.00305"},{"key":"ref_17","unstructured":"Laine, S., and Aila, T. (2016). Temporal Ensembling for Semi-Supervised Learning. arXiv."},{"key":"ref_18","unstructured":"Hinton, G., Vinyals, O., and Dean, J. (2015). Distilling the knowledge in a neural network. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"101065","DOI":"10.1016\/j.iot.2024.101065","article-title":"Intelligent healthcare system for IoMT-integrated sonography: Leveraging multi-scale self-guided attention networks and dynamic self-distillation","volume":"25","author":"Usman","year":"2024","journal-title":"Internet Things"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Saenko, K., Kulis, B., Fritz, M., and Darrell, T. (2010, January 5\u201311). Adapting visual category models to new domains. Proceedings of the Computer Vision\u2013ECCV 2010: 11th European Conference on Computer Vision, Heraklion, Crete, Greece. Proceedings, Part IV 11.","DOI":"10.1007\/978-3-642-15561-1_16"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Venkateswara, H., Eusebio, J., Chakraborty, S., and Panchanathan, S. (2017, January 21\u201326). Deep hashing network for unsupervised domain adaptation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.572"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1145\/3065386","article-title":"Imagenet classification with deep convolutional neural networks","volume":"60","author":"Krizhevsky","year":"2017","journal-title":"Commun. ACM"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Tzeng, E., Hoffman, J., Darrell, T., and Saenko, K. (2015, January 7\u201313). Simultaneous deep transfer across domains and tasks. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.463"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"8619","DOI":"10.1109\/TIP.2021.3118978","article-title":"Supervised domain adaptation: A graph embedding perspective and a rectified experimental protocol","volume":"30","author":"Hedegaard","year":"2021","journal-title":"IEEE Trans. Image Process."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Zhang, H., Cisse, M., Dauphin, Y.N., and Lopez-Paz, D. (2017). mixup: Beyond Empirical Risk Minimization. arXiv.","DOI":"10.1007\/978-1-4899-7687-1_79"},{"key":"ref_26","first-page":"26103","article-title":"A mathematical framework for quantifying transferability in multi-source transfer learning","volume":"34","author":"Tong","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Koniusz, P., Tas, Y., and Porikli, F. (2017, January 21\u201326). Domain adaptation by mixture of alignments of second-or higher-order scatter tensors. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.755"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"Imagenet large scale visual recognition challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/6\/1939\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:15:23Z","timestamp":1760105723000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/6\/1939"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,18]]},"references-count":28,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2024,3]]}},"alternative-id":["s24061939"],"URL":"https:\/\/doi.org\/10.3390\/s24061939","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2024,3,18]]}}}