{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,17]],"date-time":"2025-10-17T20:07:07Z","timestamp":1760731627910,"version":"build-2065373602"},"reference-count":52,"publisher":"MDPI AG","issue":"17","license":[{"start":{"date-parts":[[2024,8,28]],"date-time":"2024-08-28T00:00:00Z","timestamp":1724803200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Natural Science Foundation of China","award":["62476126"],"award-info":[{"award-number":["62476126"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Infrared small-target detection is now commonly used in maritime surveillance, flight guidance, and other fields. However, extracting small targets from complex backgrounds remains a challenging task due to the small-target scale and complex imaging environment. Many studies are based on designing model structures to enhance the precision of target detection, and the number of Params and FLOPs has been significantly augmented. In this work, a knowledge distillation-based detection method (KDD) is proposed to overcome this challenge. KDD employs the small-target labeling information provided by a large-scale teacher model to refine the training process of students, thereby improving the performance and becoming lightweight. Specifically, we added efficient local attention (ELA), which can accurately identify areas of interest while avoiding dimensionality reduction. In addition, we also added the group aggregation bridge (GAB) module to connect low-level and high-level features for the fusion of different feature scales. Furthermore, a feature fusion loss was introduced to enhance the precision of target detection. Extensive evaluations have demonstrated that KDD performs better compared to several methods, achieving extremely low Params and FLOPs, as well as higher FPS.<\/jats:p>","DOI":"10.3390\/rs16173173","type":"journal-article","created":{"date-parts":[[2024,8,28]],"date-time":"2024-08-28T03:57:06Z","timestamp":1724817426000},"page":"3173","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["An Efficient Knowledge Distillation-Based Detection Method for Infrared Small Targets"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0009-0004-2318-0929","authenticated-orcid":false,"given":"Wenjuan","family":"Tang","sequence":"first","affiliation":[{"name":"College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China"},{"name":"HIWING Technology Academy, China Aerospace Science and Industry Corporation Limited (CASIC), Beijing 100074, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4618-7299","authenticated-orcid":false,"given":"Qun","family":"Dai","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5598-5500","authenticated-orcid":false,"given":"Fan","family":"Hao","sequence":"additional","affiliation":[{"name":"School of Integrated Circuits, Beijing University of Posts and Telecommunications, Beijing 100876, China"}]}],"member":"1968","published-online":{"date-parts":[[2024,8,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Li, H., Yang, J., Xu, Y., and Wang, R. (2024). Mitigate Target-level Insensitivity of Infrared Small Target Detection via Posterior Distribution Modeling. arXiv.","DOI":"10.1109\/JSTARS.2024.3429491"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"4250","DOI":"10.1109\/TAES.2023.3238703","article-title":"Attention-Guided Pyramid Context Networks for Detecting Infrared Small Target Under Complex Background","volume":"59","author":"Zhang","year":"2023","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Teutsch, M., and Kr\u00fcger, W. (2010, January 3\u20135). Classification of small boats in infrared images for maritime surveillance. Proceedings of the 2010 International WaterSide Security Conference, Carrara, Italy.","DOI":"10.1109\/WSSC.2010.5730289"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Zhang, M., Zhang, R., Yang, Y., Bai, H., Zhang, J., and Guo, J. (2022, January 19\u201321). ISNet: Shape Matters for Infrared Small Target Detection. Proceedings of the 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00095"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"4996","DOI":"10.1109\/TIP.2013.2281420","article-title":"Infrared Patch-Image Model for Small Target Detection in a Single Image","volume":"22","author":"Gao","year":"2013","journal-title":"IEEE Trans. Image Process."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2145","DOI":"10.1016\/j.patcog.2009.12.023","article-title":"Analysis of new top-hat transformation and the application for infrared dim small target detection","volume":"43","author":"Bai","year":"2010","journal-title":"Pattern Recognit."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Deshpande, S.D., Er, M.H., Venkateswarlu, R., and Chan, P. (1999, January 20\u201322). Max-mean and max-median filters for detection of small targets. Proceedings of the Signal and Data Processing of Small Targets 1999, Denver, CO, USA.","DOI":"10.1117\/12.364049"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1016\/j.infrared.2005.04.006","article-title":"The design of top-hat morphological filter and application to infrared target detection","volume":"48","author":"Zeng","year":"2006","journal-title":"Infrared Phys. Technol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2420","DOI":"10.1109\/78.317863","article-title":"Two-dimensional block diagonal LMS adaptive filtering","volume":"42","author":"Pan","year":"1994","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"574","DOI":"10.1109\/TGRS.2013.2242477","article-title":"A Local Contrast Method for Small Infrared Target Detection","volume":"52","author":"Chen","year":"2014","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2168","DOI":"10.1109\/LGRS.2014.2323236","article-title":"A robust infrared small target detection algorithm based on human visual system","volume":"11","author":"Han","year":"2014","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1016\/j.patcog.2016.04.002","article-title":"Multiscale patch-based contrast measure for small infrared target detection","volume":"58","author":"Wei","year":"2016","journal-title":"Pattern Recognit."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1004","DOI":"10.1109\/TGRS.2019.2942384","article-title":"Infrared Small Target Detection via Low-Rank Tensor Completion With Top-Hat Regularization","volume":"58","author":"Zhu","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"3752","DOI":"10.1109\/JSTARS.2017.2700023","article-title":"Reweighted Infrared Patch-Tensor Model With Both Nonlocal and Local Priors for Single-Frame Small Target Detection","volume":"10","author":"Dai","year":"2017","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zhang, L., Peng, L., Zhang, T., Cao, S., and Peng, Z. (2018). Infrared Small Target Detection via Non-Convex Rank Approximation Minimization Joint l2,1 Norm. Remote Sens., 10.","DOI":"10.3390\/rs10111821"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Lin, Z., Ganesh, A., Wright, J., Wu, L., Chen, M., and Ma, Y. (2009). Fast Convex Optimization Algorithms for Exact Recovery of a Corrupted Low-Rank Matrix, Coordinated Science Laboratory. Coordinated Science Laboratory Report No. UILU-ENG-09-2214, DC-246.","DOI":"10.1109\/CAMSAP.2009.5413299"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"9546","DOI":"10.1109\/TIP.2020.3028457","article-title":"TNLRS: Target-Aware Non-Local Low-Rank Modeling With Saliency Filtering Regularization for Infrared Small Target Detection","volume":"29","author":"Zhu","year":"2020","journal-title":"IEEE Trans. Image Process."},{"key":"ref_18","unstructured":"Wang, S.X. (2022). Image Small Target Detection based on Deep Learning with SNR Controlled Sample Generation. Current Trends in Computer Science and Mechanical Automation Vol. 1, De Gruyter Open Poland."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Wang, H., Zhou, L., and Wang, L. (November, January 27). Miss Detection vs. False Alarm: Adversarial Learning for Small Object Segmentation in Infrared Images. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Repulic of Korea.","DOI":"10.1109\/ICCV.2019.00860"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"9813","DOI":"10.1109\/TGRS.2020.3044958","article-title":"Attentional Local Contrast Networks for Infrared Small Target Detection","volume":"59","author":"Dai","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1745","DOI":"10.1109\/TIP.2022.3199107","article-title":"Dense Nested Attention Network for Infrared Small Target Detection","volume":"32","author":"Li","year":"2023","journal-title":"IEEE Trans. Image Process."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"6616","DOI":"10.1109\/JSTARS.2024.3374054","article-title":"Generative Adversarial Differential Analysis for Infrared Small Target Detection","volume":"17","author":"Ma","year":"2024","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Wang, X., Han, C., Li, J., Nie, T., Li, M., Wang, X., and Huang, L. (2024). Multiscale Feature Extraction U-Net for Infrared Dim- and Small-Target Detection. Remote Sens., 16.","DOI":"10.3390\/rs16040643"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"109788","DOI":"10.1016\/j.patcog.2023.109788","article-title":"Infrared small target segmentation networks: A survey","volume":"143","author":"Kou","year":"2023","journal-title":"Pattern Recognit."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Lan, W., Dang, J., Wang, Y., and Wang, S. (2018, January 5\u20138). Pedestrian Detection Based on YOLO Network Model. Proceedings of the 2018 IEEE International Conference on Mechatronics and Automation (ICMA), Changchun, China.","DOI":"10.1109\/ICMA.2018.8484698"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"15970","DOI":"10.1039\/D3CP00402C","article-title":"Accurate identification and measurement of the precipitate area by two-stage deep neural networks in novel chromium-based alloys","volume":"25","author":"Xia","year":"2023","journal-title":"Phys. Chem. Chem. Phys."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Liu, S., Liu, Z., Li, Y., Liu, W., Ge, C., and Liu, L. (2022, January 25\u201327). Design Compact YOLO based Network for Small Target Detection on Infrared Image. Proceedings of the 2022 China Automation Congress (CAC), Xiamen, China.","DOI":"10.1109\/CAC57257.2022.10054751"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Ciocarlan, A., Le Hegarat-Mascle, S., Lefebvre, S., Woiselle, A., and Barbanson, C. (2024, January 14\u201319). A Contrario Paradigm for Yolo-Based Infrared Small Target Detection. Proceedings of the ICASSP 2024\u20142024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Seoul, Republic of Korea.","DOI":"10.1109\/ICASSP48485.2024.10446505"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1748","DOI":"10.1109\/TSMC.2022.3205365","article-title":"BSC: Belief Shift Clustering","volume":"53","author":"Zhang","year":"2023","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_30","unstructured":"Hinton, G., Vinyals, O., and Dean, J. (2015). Distilling the knowledge in a neural network. arXiv."},{"key":"ref_31","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the Medical Image Computing and Computer-Assisted Intervention\u2013MICCAI 2015: 18th International Conference, Munich, Germany. Proceedings, Part III 18."},{"key":"ref_32","unstructured":"Xu, W., and Wan, Y. (2024). ELA: Efficient Local Attention for Deep Convolutional Neural Networks. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Ruan, J., Xie, M., Gao, J., Liu, T., and Fu, Y. (2023, January 8\u201312). EGE-UNet: An Efficient Group Enhanced UNet for Skin Lesion Segmentation. Proceedings of the Medical Image Computing and Computer Assisted Intervention\u2014MICCAI 2023, Vancouver, BC, Canada.","DOI":"10.1007\/978-3-031-43901-8_46"},{"key":"ref_34","unstructured":"Dong, X., Chen, S., and Pan, S.J. (2017, January 4\u20139). Learning to prune deep neural networks via layer-wise optimal brain surgeon. Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Liu, Z., Li, J., Shen, Z., Huang, G., Yan, S., and Zhang, C. (2017, January 22\u201329). Learning Efficient Convolutional Networks through Network Slimming. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.298"},{"key":"ref_36","unstructured":"Romero, A., Ballas, N., Kahou, S.E., Chassang, A., Gatta, C., and Bengio, Y. (2014). Fitnets: Hints for thin deep nets. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Ahn, S., Hu, S.X., Damianou, A., Lawrence, N.D., and Dai, Z. (2019, January 15\u201320). Variational Information Distillation for Knowledge Transfer. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00938"},{"key":"ref_38","unstructured":"Zoph, B., and Le, Q.V. (2017). Neural Architecture Search with Reinforcement Learning. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Alizadeh Vahid, K., Prabhu, A., Farhadi, A., and Rastegari, M. (2020, January 13\u201319). Butterfly Transform: An Efficient FFT Based Neural Architecture Design. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01204"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Kim, Y.D., Park, E., Yoo, S., Choi, T., Yang, L., and Shin, D. (2015). Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications. arXiv.","DOI":"10.14257\/astl.2016.140.36"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Gusak, J., Kholiavchenko, M., Ponomarev, E., Markeeva, L., Blagoveschensky, P., Cichocki, A., and Oseledets, I. (2019, January 27\u201328). Automated Multi-Stage Compression of Neural Networks. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Republic of Korea.","DOI":"10.1109\/ICCVW.2019.00306"},{"key":"ref_42","unstructured":"Xu, X., Li, M., Tao, C., Shen, T., Cheng, R., Li, J., Xu, C., Tao, D., and Zhou, T. (2024). A Survey on Knowledge Distillation of Large Language Models. arXiv."},{"key":"ref_43","unstructured":"Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., and Bhosale, S. (2023). Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv."},{"key":"ref_44","unstructured":"Zhao, W.X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., and Dong, Z. (2023). A Survey of Large Language Models. arXiv."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Xiang, T., Hospedales, T.M., and Lu, H. (2018, January 18\u201322). Deep Mutual Learning. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00454"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Milletari, F., Navab, N., and Ahmadi, S.A. (2016, January 25\u201328). V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation. Proceedings of the 2016 Fourth International Conference on 3D Vision (3DV), Stanford, CA, USA.","DOI":"10.1109\/3DV.2016.79"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Hu, J., Shen, L., and Sun, G. (2018, January 18\u201322). Squeeze-and-Excitation Networks. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00745"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Hou, Q., Zhou, D., and Feng, J. (2021, January 19\u201325). Coordinate Attention for Efficient Mobile Network Design. Proceedings of the 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.01350"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Dai, Y., Wu, Y., Zhou, F., and Barnard, K. (2021, January 3\u20138). Asymmetric Contextual Modulation for Infrared Small Target Detection. Proceedings of the 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA.","DOI":"10.1109\/WACV48630.2021.00099"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"10027","DOI":"10.1109\/JSTARS.2022.3222758","article-title":"Prior-Guided Data Augmentation for Infrared Small Target Detection","volume":"15","author":"Wang","year":"2022","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_51","first-page":"1","article-title":"Interior Attention-Aware Network for Infrared Small Target Detection","volume":"60","author":"Wang","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_52","first-page":"1","article-title":"LW-IRSTNet: Lightweight Infrared Small Target Segmentation Network and Application Deployment","volume":"61","author":"Kou","year":"2023","journal-title":"IEEE Trans. Geosci. Remote Sens."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/17\/3173\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:43:58Z","timestamp":1760111038000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/17\/3173"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,28]]},"references-count":52,"journal-issue":{"issue":"17","published-online":{"date-parts":[[2024,9]]}},"alternative-id":["rs16173173"],"URL":"https:\/\/doi.org\/10.3390\/rs16173173","relation":{},"ISSN":["2072-4292"],"issn-type":[{"type":"electronic","value":"2072-4292"}],"subject":[],"published":{"date-parts":[[2024,8,28]]}}}