{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T14:49:48Z","timestamp":1783608588124,"version":"3.55.0"},"reference-count":105,"publisher":"Springer Science and Business Media LLC","issue":"10","license":[{"start":{"date-parts":[[2025,7,14]],"date-time":"2025-07-14T00:00:00Z","timestamp":1752451200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,7,14]],"date-time":"2025-07-14T00:00:00Z","timestamp":1752451200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Comput Vis"],"published-print":{"date-parts":[[2025,10]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>Multimodal conversational generative AI has shown impressive capabilities in various vision and language understanding through learning massive text-image data. However, current conversational models still lack knowledge about visual insects since they are often trained on the general knowledge of vision-language data. Meanwhile, understanding insects is a fundamental problem in precision agriculture, helping to promote sustainable development in agriculture. Therefore, this paper proposes a novel multimodal conversational model, <jats:bold>Insect-LLaVA<\/jats:bold>, to promote visual understanding in insect-domain knowledge. In particular, we first introduce a new large-scale <jats:bold>Multimodal Insect Dataset<\/jats:bold> with <jats:bold>Visual Insect Instruction<\/jats:bold> Data that enables the capability of learning the multimodal foundation models. Our proposed dataset enables conversational models to comprehend the visual and semantic features of the insects. Second, we propose a new <jats:bold>Insect-LLaVA<\/jats:bold> model, a new general Large Language and Vision Assistant in Visual Insect Understanding. Then, to enhance the capability of learning insect features, we develop an <jats:bold>Insect Foundation Model<\/jats:bold> by introducing a new micro-feature self-supervised learning with a Patch-wise Relevant Attention mechanism to capture the subtle differences among insect images. We also present Description Consistency loss to improve micro-feature learning via text descriptions. The experimental results evaluated on our new <jats:bold>Visual Insect Question Answering<\/jats:bold> benchmarks illustrate the effective performance of our proposed approach in visual insect understanding and achieve State-of-the-Art performance on standard benchmarks of insect-related tasks Project Page: <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" xlink:href=\"https:\/\/uarkcviu.github.io\/projects\/insectfoundation\" ext-link-type=\"uri\">https:\/\/uarkcviu.github.io\/projects\/insectfoundation<\/jats:ext-link>.<\/jats:p>","DOI":"10.1007\/s11263-025-02521-4","type":"journal-article","created":{"date-parts":[[2025,7,14]],"date-time":"2025-07-14T20:39:36Z","timestamp":1752525576000},"page":"7128-7153","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Insect-Foundation: A Foundation Model and Large Multimodal Dataset for Vision-Language Insect Understanding"],"prefix":"10.1007","volume":"133","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4634-8793","authenticated-orcid":false,"given":"Thanh-Dat","family":"Truong","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hoang-Quan","family":"Nguyen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xuan-Bac","family":"Nguyen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ashley","family":"Dowling","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xin","family":"Li","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Khoa","family":"Luu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,7,14]]},"reference":[{"key":"2521_CR1","unstructured":"Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S. ... others. (2023). Gpt-4 technical report. arXiv preprint arXiv:2303.08774"},{"key":"2521_CR2","first-page":"23716","volume":"35","author":"JB Alayrac","year":"2022","unstructured":"Alayrac, J. B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., \u2026 Borgeaud, S. (2022). Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35, 23716\u201323736.","journal-title":"Advances in neural information processing systems"},{"key":"2521_CR3","doi-asserted-by":"crossref","unstructured":"Alfarisy, A. A., Chen, Q. & Guo, M. (2018). Deep learning based classification for paddy pests & diseases recognition. In Proceedings of 2018 International Conference on Mathematics and Artificial Intelligence, (pp. 21\u201325).","DOI":"10.1145\/3208788.3208795"},{"key":"2521_CR4","doi-asserted-by":"crossref","unstructured":"Alves, A. N., Souza, W. S., & Borges, D. L. (2020). Cotton pests classification in field-based images using deep residual networks. Computers and Electronics in Agriculture, 174, Article 105488.","DOI":"10.1016\/j.compag.2020.105488"},{"key":"2521_CR5","unstructured":"Askell, A., Bai, Y., Chen, A., Drain, D., Ganguli, D., Henighan, T., Jones, A., Joseph, N., Mann, B., DasSarma, N., ... others. (2021). A general language assistant as a laboratory for alignment. arXiv preprint arXiv:2112.00861"},{"key":"2521_CR6","doi-asserted-by":"crossref","unstructured":"Ayan, E., Erbay, H., & Var\u00e7\u0131n, F. (2020). Crop pest classification with a genetic algorithm-based weighted ensemble of deep convolutional neural networks. Computers and Electronics in Agriculture, 179, Article 105809.","DOI":"10.1016\/j.compag.2020.105809"},{"key":"2521_CR7","first-page":"19352","volume":"34","author":"S Badirli","year":"2021","unstructured":"Badirli, S., Akata, Z., Mohler, G., Picard, C., & Dundar, M. M. (2021). Fine-grained zero-shot learning with dna as side information. Advances in Neural Information Processing Systems, 34, 19352\u201319362.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2521_CR8","unstructured":"Bao, H., Dong, L., Piao, S. & Wei, F. (2021). Beit: Bert pre-training of image transformers. arXiv preprint arXiv:2106.08254"},{"key":"2521_CR9","doi-asserted-by":"crossref","unstructured":"Bollis, E., Pedrini, H. & Avila, S. (2020). Weakly supervised learning guided by activation mapping applied to a novel citrus pest benchmark. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops, (pp. 70\u201371).","DOI":"10.1109\/CVPRW50498.2020.00043"},{"key":"2521_CR10","doi-asserted-by":"crossref","unstructured":"Caron, M., Touvron, H., Misra, I., J\u00e9gou, H., Mairal, J., Bojanowski, P. & Joulin, A. (2021). Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE\/CVF International Conference on Computer Vision, (pp. 9650\u20139660).","DOI":"10.1109\/ICCV48922.2021.00951"},{"key":"2521_CR11","unstructured":"Chen, J. & Zhang, A. (2024). FedMBridge: Bridgeable multimodal federated learning. In: Forty-first International Conference on Machine Learning. https:\/\/openreview.net\/forum?id=jrHUbftLd6"},{"key":"2521_CR12","unstructured":"Chen, X., Fan, H., Girshick, R. & He, K. (2020). Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297"},{"key":"2521_CR13","unstructured":"Chen, T., Kornblith, S., Norouzi, M. & Hinton, G. (2020). A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning, (pp. 1597\u20131607). PMLR."},{"key":"2521_CR14","doi-asserted-by":"crossref","unstructured":"Chen, X., Xie, S. & He, K. (2021). An empirical study of training self-supervised vision transformers. in 2021 ieee. In CVF International Conference on Computer Vision (ICCV), (pp. 9620\u20139629).","DOI":"10.1109\/ICCV48922.2021.00950"},{"key":"2521_CR15","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1016\/j.patrec.2022.12.023","volume":"166","author":"Y Chen","year":"2023","unstructured":"Chen, Y., Shen, X., Liu, Y., Tao, Q., & Suykens, J. A. (2023). Jigsaw-vit: Learning jigsaw puzzles in vision transformer. Pattern Recognition Letters, 166, 53\u201360.","journal-title":"Pattern Recognition Letters"},{"key":"2521_CR16","unstructured":"Chiang, W. L., Li, Z., Lin, Z., Sheng, Y., Wu, Z., Zhang, H., Zheng, L., Zhuang, S., Zhuang, Y., Gonzalez, J. E., Stoica, I. & Xing, E. P. (2023). Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90%* ChatGPT Quality. https:\/\/lmsys.org\/blog\/2023-03-30-vicuna\/"},{"key":"2521_CR17","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L. J., Li, K. & Fei-Fei, L. (2009). Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, (pp. 248\u2013255). Ieee","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"2521_CR18","doi-asserted-by":"publisher","first-page":"139","DOI":"10.1016\/j.biosystemseng.2018.02.008","volume":"169","author":"L Deng","year":"2018","unstructured":"Deng, L., Wang, Y., Han, Z., & Yu, R. (2018). Research on insect pest image detection and recognition based on bio-inspired methods. Biosystems Engineering, 169, 139\u2013148.","journal-title":"Biosystems Engineering"},{"key":"2521_CR19","unstructured":"Devlin, J., Chang, M. W., Lee, K. & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805"},{"key":"2521_CR20","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S.... others. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929"},{"key":"2521_CR21","unstructured":"Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A.. ... others. (2024). The llama 3 herd of models. arXiv preprint arXiv:2407.21783"},{"key":"2521_CR22","doi-asserted-by":"crossref","unstructured":"Fan, D. P., Ji, G. P., Sun, G., Cheng, M. M., Shen, J. & Shao, L. (2020). Camouflaged object detection. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, (pp. 2777\u20132787).","DOI":"10.1109\/CVPR42600.2020.00285"},{"key":"2521_CR23","volume-title":"VITRON: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting","author":"H Fei","year":"2024","unstructured":"Fei, H., Wu, S., Zhang, H., Chua, T.-S., & Yan, S. (2024). VITRON: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting. CoRR: Editing."},{"key":"2521_CR24","doi-asserted-by":"crossref","unstructured":"Gan, Z., Li, L., Li, C., Wang, L., Liu, Z., Gao, J., ... others. (2022). Vision-language pre-training: Basics, recent advances, and future trends. Foundations and Trends\u00ae in Computer Graphics and Vision14(3\u20134), 163\u2013352.","DOI":"10.1561\/0600000105"},{"key":"2521_CR25","unstructured":"Gharaee, Z., Gong, Z., Pellegrino, N., Zarubiieva, I., Haurum, J. B., Lowe, S. C., McKeown, J. T. A., Ho, C. Y., McLeod, J., Wei, Y. C., Agda, J., Ratnasingham, S., Steinke, D., Chang, A. X., Taylor, G. W. & Fieguth, P. (2023). A step towards worldwide biodiversity assessment: The BIOSCAN-1M insect dataset. In A. Oh, T. Neumann, A. Globerson, K. Saenko, M. Hardt, S. Levine (Eds.), Advances in Neural Information Processing Systems, vol. 36, pp. 43593\u201343619. Curran Associates, Inc., ???. https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2023\/file\/87dbbdc3a685a97ad28489a1d57c45c1-Paper-Datasets_and_Benchmarks.pdf"},{"key":"2521_CR26","doi-asserted-by":"crossref","unstructured":"Ghosal, D., Majumder, N., Mehrish, A. & Poria, S. (2023). Text-to-audio generation using instruction-tuned llm and latent diffusion model. arXiv preprint arXiv:2304.13731","DOI":"10.1145\/3581783.3612348"},{"key":"2521_CR27","doi-asserted-by":"crossref","unstructured":"He, K., Chen, X., Xie, S., Li, Y., Doll\u00e1r, P. & Girshick, R. (2022). Masked autoencoders are scalable vision learners. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, (pp. 16000\u201316009).","DOI":"10.1109\/CVPR52688.2022.01553"},{"key":"2521_CR28","doi-asserted-by":"crossref","unstructured":"He, K., Fan, H., Wu, Y., Xie, S. & Girshick, R. (2020). Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, (pp. 9729\u20139738).","DOI":"10.1109\/CVPR42600.2020.00975"},{"key":"2521_CR29","doi-asserted-by":"crossref","unstructured":"He, C., Li, K., Zhang, Y., Tang, L., Zhang, Y., Guo, Z. & Li, X. (2023). Camouflaged object detection with feature decomposition and edge reconstruction. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, (pp. 22046\u201322055).","DOI":"10.1109\/CVPR52729.2023.02111"},{"key":"2521_CR30","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S. & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (pp. 770\u2013778).","DOI":"10.1109\/CVPR.2016.90"},{"key":"2521_CR31","first-page":"30726","volume":"36","author":"C He","year":"2024","unstructured":"He, C., Li, K., Zhang, Y., Xu, G., Tang, L., Zhang, Y., Guo, Z., & Li, X. (2024). Weakly-supervised concealed object segmentation with sam-based pseudo labeling and multi-scale feature grouping. Advances in Neural Information Processing Systems, 36, 30726\u201330737.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2521_CR32","unstructured":"Hussain, A. S., Liu, S., Sun, C. & Shan, Y. (2023). M2ugen: Multi-modal music understanding and generation with the power of large language models. arXiv preprint arXiv:2311.11255"},{"key":"2521_CR33","unstructured":"Jia, C., Yang, Y., Xia, Y., Chen, Y. T., Parekh, Z., Pham, H., Le, Q., Sung, Y. H., Li, Z. & Duerig, T. (2021). Scaling up visual and vision-language representation learning with noisy text supervision. In International Conference on Machine Learning, (pp. 4904\u20134916). PMLR."},{"key":"2521_CR34","unstructured":"Krizhevsky, A., Sutskever, I. & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25."},{"key":"2521_CR35","doi-asserted-by":"crossref","unstructured":"Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V. & Zettlemoyer, L. (2019). Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461","DOI":"10.18653\/v1\/2020.acl-main.703"},{"key":"2521_CR36","doi-asserted-by":"crossref","unstructured":"Li, C., Gan, Z., Yang, Z., Yang, J., Li, L., Wang, L., Gao, J., ... others. (2024). Multimodal foundation models: From specialists to general-purpose assistants. Foundations and Trends\u00ae in Computer Graphics and Vision, 16(1-2), 1\u2013214.","DOI":"10.1561\/0600000110"},{"key":"2521_CR37","unstructured":"Li, J., Li, D., Savarese, S. & Hoi, S. (2023). Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International Conference on Machine Learning, (pp. 19730\u201319742). PMLR."},{"key":"2521_CR38","unstructured":"Li, J., Li, D., Xiong, C. & Hoi, S. (2022). Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In International Conference on Machine Learning, (pp. 12888\u201312900). PMLR."},{"key":"2521_CR39","unstructured":"Li, C., Liu, H., Li, L., Zhang, P., Aneja, J., Yang, J., Jin, P., Hu, H., Liu, Z., Lee, Y. J., et al. (2022). Elevater: A benchmark and toolkit for evaluating language-augmented visual models. Advances in Neural Information Processing Systems,35, 9287\u20139301."},{"key":"2521_CR40","doi-asserted-by":"crossref","unstructured":"Lin, T. Y., Doll\u00e1r, P., Girshick, R., He, K., Hariharan, B. & Belongie, S. (2017). Feature pyramid networks for object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (pp. 2117\u20132125).","DOI":"10.1109\/CVPR.2017.106"},{"key":"2521_CR41","doi-asserted-by":"crossref","unstructured":"Lin, T. Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll\u00e1r, P. & Zitnick, C. L. (2014). Microsoft coco: Common objects in context. In Computer Vision\u2013ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, (pp. 740\u2013755). Springer.","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"2521_CR42","doi-asserted-by":"crossref","unstructured":"Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C. Y. & Berg, A. C. (2016). Ssd: Single shot multibox detector. In Computer Vision\u2013ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11\u201314, 2016, Proceedings, Part I 14, (pp. 21\u201337). Springer.","DOI":"10.1007\/978-3-319-46448-0_2"},{"key":"2521_CR43","doi-asserted-by":"crossref","unstructured":"Liu, H., Li, C., Li, Y. & Lee, Y. J. (2024). Improved baselines with visual instruction tuning. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, (pp. 26296\u201326306).","DOI":"10.1109\/CVPR52733.2024.02484"},{"key":"2521_CR44","unstructured":"Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L. & Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692"},{"issue":"1","key":"2521_CR45","doi-asserted-by":"publisher","first-page":"20410","DOI":"10.1038\/srep20410","volume":"6","author":"Z Liu","year":"2016","unstructured":"Liu, Z., Gao, J., Yang, G., Zhang, H., & He, Y. (2016). Localization and classification of paddy field pests using a saliency map and deep convolutional neural network. Scientific reports, 6(1), 20410.","journal-title":"Scientific reports"},{"key":"2521_CR46","first-page":"34892","volume":"36","author":"H Liu","year":"2024","unstructured":"Liu, H., Li, C., Wu, Q., & Lee, Y. J. (2024). Visual instruction tuning. Advances in neural information processing systems, 36, 34892\u201334916.","journal-title":"Advances in neural information processing systems"},{"key":"2521_CR47","doi-asserted-by":"publisher","first-page":"45301","DOI":"10.1109\/ACCESS.2019.2909522","volume":"7","author":"L Liu","year":"2019","unstructured":"Liu, L., Wang, R., Xie, C., Yang, P., Wang, F., Sudirman, S., & Liu, W. (2019). Pestnet: An end-to-end deep learning approach for large-scale multi-class pest detection and classification. Ieee Access, 7, 45301\u201345312.","journal-title":"Ieee Access"},{"key":"2521_CR48","first-page":"28541","volume":"36","author":"C Li","year":"2024","unstructured":"Li, C., Wong, C., Zhang, S., Usuyama, N., Liu, H., Yang, J., Naumann, T., Poon, H., & Gao, J. (2024). Llava-med: Training a large language-and-vision assistant for biomedicine in one day. Advances in Neural Information Processing Systems, 36, 28541\u201328564.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2521_CR49","unstructured":"Loshchilov, I. & Hutter, F. (2016). Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983"},{"key":"2521_CR50","unstructured":"Loshchilov, I. & Hutter, F. (2017). Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101"},{"key":"2521_CR51","unstructured":"Lu, P., Mishra, S., Xia, T., Qiu, L., Chang, K. W., Zhu, S. C., Tafjord, O., Clark, P. & Kalyan, A. (2022). Learn to explain: Multimodal reasoning via thought chains for science question answering. In: The 36th Conference on Neural Information Processing Systems (NeurIPS)"},{"key":"2521_CR52","doi-asserted-by":"crossref","unstructured":"Luo, Z., Zhao, P., Xu, C., Geng, X., Shen, T., Tao, C., Ma, J., Lin, Q. & Jiang, D. (2023). Lexlip: Lexicon-bottlenecked language-image pre-training for large-scale image-text sparse retrieval. In Proceedings of the IEEE\/CVF International Conference on Computer Vision, (pp. 11206\u201311217).","DOI":"10.1109\/ICCV51070.2023.01029"},{"key":"2521_CR53","doi-asserted-by":"crossref","unstructured":"Nanni, L., Maguolo, G., & Pancino, F. (2020). Insect pest image detection and recognition based on bio-inspired methods. Ecological Informatics, 57, Article 101089.","DOI":"10.1016\/j.ecoinf.2020.101089"},{"key":"2521_CR54","unstructured":"Nguyen, X. B., Bisht, A., Churchill, H. & Luu, K. (2022). Two-dimensional quantum material identification via self-attention and soft-labeling in deep learning. arXiv preprint arXiv:2205.15948"},{"key":"2521_CR55","doi-asserted-by":"crossref","unstructured":"Nguyen, X. B., Bui, D. T., Duong, C. N., Bui, T. D. & Luu, K. (2021). Clusformer: A transformer based clustering approach to unsupervised large-scale face and visual landmark recognition. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, (pp. 10847\u201310856).","DOI":"10.1109\/CVPR46437.2021.01070"},{"key":"2521_CR56","doi-asserted-by":"crossref","unstructured":"Nguyen, X. B., Duong, C. N., Li, X., Gauch, S., Seo, H. S. & Luu, K. (2023). Micron-bert: Bert-based facial micro-expression recognition. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, (pp. 1482\u20131492).","DOI":"10.1109\/CVPR52729.2023.00149"},{"key":"2521_CR57","unstructured":"Nguyen, X. B., Duong, C. N., Savvides, M., Roy, K. & Luu, K. (2023). Fairness in visual clustering: A novel transformer clustering approach. arXiv preprint arXiv:2304.07408"},{"key":"2521_CR58","unstructured":"Nguyen, X. B., Li, X., Khan, S. U. & Luu, K. (2023). Brainformer: Modeling mri brain functions to machine vision. arXiv preprint arXiv:2312.00236"},{"key":"2521_CR59","doi-asserted-by":"crossref","unstructured":"Nguyen, H. Q., Truong, T. D., Nguyen, X. B., Dowling, A., Li, X. & Luu, K. (2024). Insect-foundation: A foundation model and large-scale 1m dataset for visual insect understanding. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, (pp. 21945\u201321955).","DOI":"10.1109\/CVPR52733.2024.02072"},{"key":"2521_CR60","doi-asserted-by":"publisher","first-page":"162973","DOI":"10.1109\/ACCESS.2020.3021469","volume":"8","author":"X-B Nguyen","year":"2020","unstructured":"Nguyen, X.-B., Lee, G. S., Kim, S. H., & Yang, H. J. (2020). Self-supervised learning based on spatial awareness for medical image analysis. IEEE Access, 8, 162973\u2013162981. https:\/\/doi.org\/10.1109\/ACCESS.2020.3021469","journal-title":"IEEE Access"},{"key":"2521_CR61","doi-asserted-by":"crossref","unstructured":"Noroozi, M. & Favaro, P. (2016). Unsupervised learning of visual representations by solving jigsaw puzzles. In European Conference on Computer Vision, (pp. 69\u201384). Springer.","DOI":"10.1007\/978-3-319-46466-4_5"},{"key":"2521_CR62","unstructured":"Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A. ... others. (2023). Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193"},{"key":"2521_CR63","unstructured":"Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L.. ... others. (2019). Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems 32."},{"key":"2521_CR64","doi-asserted-by":"crossref","unstructured":"Pham, H., Dai, Z., Ghiasi, G., Kawaguchi, K., Liu, H., Yu, A. W., Yu, J., Chen, Y. T., Luong, M. T., Wu, Y., et al. (2023). Combined scaling for zero-shot transfer learning. Neurocomputing,555, Article 126658.","DOI":"10.1016\/j.neucom.2023.126658"},{"key":"2521_CR65","unstructured":"Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., ... others. (2021). Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, (pp. 8748\u20138763). PMLR."},{"issue":"1","key":"2521_CR66","first-page":"5485","volume":"21","author":"C Raffel","year":"2020","unstructured":"Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2020). Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 21(1), 5485\u20135551.","journal-title":"The Journal of Machine Learning Research"},{"issue":"3","key":"2521_CR67","doi-asserted-by":"publisher","first-page":"355","DOI":"10.1111\/j.1471-8286.2007.01678.x","volume":"7","author":"S Ratnasingham","year":"2007","unstructured":"Ratnasingham, S., & Hebert, P. D. (2007). Bold: The barcode of life data system. Molecular ecology notes, 7(3), 355\u2013364.","journal-title":"Molecular ecology notes"},{"key":"2521_CR68","unstructured":"Redmon, J. & Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767"},{"key":"2521_CR69","unstructured":"Ren, S., He, K., Girshick, R. & Sun, J. (2015). Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems 28."},{"key":"2521_CR70","doi-asserted-by":"crossref","unstructured":"Ren, Z., Huang, Z., Wei, Y., Zhao, Y., Fu, D., Feng, J. & Jin, X. (2024). Pixellm: Pixel reasoning with large multimodal model. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, (pp. 26374\u201326383).","DOI":"10.1109\/CVPR52733.2024.02491"},{"issue":"6","key":"2521_CR71","first-page":"1","volume":"2","author":"R Samanta","year":"2012","unstructured":"Samanta, R., & Ghosh, I. (2012). Tea insect pests classification based on artificial neural networks. International Journal of Computer Engineering Science (IJCES), 2(6), 1\u201313.","journal-title":"International Journal of Computer Engineering Science (IJCES)"},{"key":"2521_CR72","unstructured":"Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., et al. (2022). Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems,35, 25278\u201325294."},{"key":"2521_CR73","unstructured":"Simonyan, K. & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556"},{"key":"2521_CR74","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1146\/annurev-ento-020117-043348","volume":"63","author":"NE Stork","year":"2018","unstructured":"Stork, N. E. (2018). How many species of insects and other terrestrial arthropods are there on earth? Annual review of entomology, 63, 31\u201345.","journal-title":"Annual review of entomology"},{"key":"2521_CR75","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V. & Rabinovich, A. (2015). Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (pp. 1\u20139).","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"2521_CR76","doi-asserted-by":"crossref","unstructured":"Truong, T. D., Bui, Q. H., Duong, C. N., Seo, H. S., Phung, S.L., Li, X. & Luu, K. (2022). Direcformer: A directed attention in transformer approach to robust action recognition. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, (pp. 20030\u201320040).","DOI":"10.1109\/CVPR52688.2022.01940"},{"key":"2521_CR77","doi-asserted-by":"crossref","unstructured":"Truong, T. D., Duong, C. N., Le, N., Phung, S. L., Rainwater, C. & Luu, K. (2021). Bimal: Bijective maximum likelihood approach to domain adaptation in semantic scene segmentation. In Proceedings of the IEEE\/CVF International Conference on Computer Vision, (pp. 8548\u20138557).","DOI":"10.1109\/ICCV48922.2021.00843"},{"key":"2521_CR78","doi-asserted-by":"crossref","unstructured":"Truong, T. D., Le, N., Raj, B., Cothren, J. & Luu, K. (2023). Fredom: Fairness domain adaptation approach to semantic scene understanding. In IEEE\/CVF Computer Vision and Pattern Recognition (CVPR)","DOI":"10.1109\/CVPR52729.2023.01914"},{"key":"2521_CR79","doi-asserted-by":"crossref","unstructured":"Truong, T.-D., Duong, C. N., Quach, K. G., Le, N., Bui, T. D., & Luu, K. (2023). Liaad: Lightweight attentive angular distillation for large-scale age-invariant face recognition. Neurocomputing, 543, Article 126198.","DOI":"10.1016\/j.neucom.2023.03.059"},{"key":"2521_CR80","first-page":"65456","volume":"36","author":"T-D Truong","year":"2024","unstructured":"Truong, T.-D., Nguyen, H.-Q., Raj, B., & Luu, K. (2024). Fairness continual learning approach to semantic scene understanding in open-world environments. Advances in Neural Information Processing Systems, 36, 65456\u201365467.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2521_CR81","doi-asserted-by":"crossref","unstructured":"Van\u00a0Horn, G., Cole, E., Beery, S., Wilber, K., Belongie, S. & Mac\u00a0Aodha, O. (2021). Benchmarking representation learning for natural world image collections. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, (pp. 12884\u201312893).","DOI":"10.1109\/CVPR46437.2021.01269"},{"issue":"1","key":"2521_CR82","doi-asserted-by":"publisher","first-page":"1","DOI":"10.7763\/IJMLC.2014.V4.376","volume":"4","author":"K Venugoban","year":"2014","unstructured":"Venugoban, K., & Ramanan, A. (2014). Image classification of paddy field insect pests using gradient-based features. International Journal of Machine Learning and Computing, 4(1), 1.","journal-title":"International Journal of Machine Learning and Computing"},{"key":"2521_CR83","doi-asserted-by":"crossref","unstructured":"Wang, T., Lin, K., Li, L., Lin, C. C., Yang, Z., Zhang, H., Liu, Z. & Wang, L. (2023). Equivariant similarity for vision-language foundation models. arXiv preprint arXiv:2303.14465","DOI":"10.1109\/ICCV51070.2023.01102"},{"key":"2521_CR84","unstructured":"Wang, P., Yang, A., Men, R., Lin, J., Bai, S., Li, Z., Ma, J., Zhou, C., Zhou, J. & Yang, H. (2022). Ofa: Unifying architectures, tasks, and modalities through a simple sequence-to-sequence learning framework. In International Conference on Machine Learning, (pp. 23318\u201323340). PMLR."},{"key":"2521_CR85","unstructured":"Wang, Z., Yu, J., Yu, A. W., Dai, Z., Tsvetkov, Y. & Cao, Y. (2021). Simvlm: Simple visual language model pretraining with weak supervision. arXiv preprint arXiv:2108.10904"},{"key":"2521_CR86","doi-asserted-by":"publisher","first-page":"102","DOI":"10.1016\/j.knosys.2012.03.014","volume":"33","author":"J Wang","year":"2012","unstructured":"Wang, J., Lin, C., Ji, L., & Liang, A. (2012). A new automatic identification system of insect images at the order level. Knowledge-Based Systems, 33, 102\u2013110.","journal-title":"Knowledge-Based Systems"},{"issue":"5","key":"2521_CR87","doi-asserted-by":"publisher","first-page":"1601","DOI":"10.3390\/s21051601","volume":"21","author":"R Wang","year":"2021","unstructured":"Wang, R., Liu, L., Xie, C., Yang, P., Li, R., & Zhou, M. (2021). Agripest: A large-scale domain-specific benchmark dataset for practical agricultural pest detection in the wild. Sensors, 21(5), 1601.","journal-title":"Sensors"},{"key":"2521_CR88","doi-asserted-by":"crossref","unstructured":"Weng, Y., Han, M., He, H., Chang, X. & Zhuang, B. (2024). Longvlm: Efficient long video understanding via large language models. arXiv preprint arXiv:2404.03384","DOI":"10.1007\/978-3-031-73414-4_26"},{"key":"2521_CR89","unstructured":"Wu, S., Fei, H., Qu, L., Ji, W. & Chua, T.-S. (2024). NExT-GPT: Any-to-any multimodal LLM. In Proceedings of the International Conference on Machine Learning, (pp. 53366\u201353397)."},{"key":"2521_CR90","doi-asserted-by":"crossref","unstructured":"Wu, X., Zhan, C., Lai, Y. K., Cheng, M. M. & Yang, J. (2019). Ip102: A large-scale benchmark dataset for insect pest recognition. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, (pp. 8787\u20138796).","DOI":"10.1109\/CVPR.2019.00899"},{"key":"2521_CR91","doi-asserted-by":"crossref","unstructured":"Xie, Z., Zhang, Z., Cao, Y., Lin, Y., Bao, J., Yao, Z., Dai, Q. & Hu, H. (2022). Simmim: A simple framework for masked image modeling. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, (pp. 9653\u20139663).","DOI":"10.1109\/CVPR52688.2022.00943"},{"key":"2521_CR92","doi-asserted-by":"publisher","first-page":"233","DOI":"10.1016\/j.compag.2018.07.014","volume":"152","author":"C Xie","year":"2018","unstructured":"Xie, C., Wang, R., Zhang, J., Chen, P., Dong, W., Li, R., Chen, T., & Chen, H. (2018). Multi-level learning features for automatic classification of field crop pests. Computers and Electronics in Agriculture, 152, 233\u2013241.","journal-title":"Computers and Electronics in Agriculture"},{"key":"2521_CR93","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1016\/j.compag.2015.10.015","volume":"119","author":"C Xie","year":"2015","unstructured":"Xie, C., Zhang, J., Li, R., Li, J., Hong, P., Xia, J., & Chen, P. (2015). Automatic classification for field crop insects via multiple-task sparse representation and multiple-kernel learning. Computers and Electronics in Agriculture, 119, 123\u2013132.","journal-title":"Computers and Electronics in Agriculture"},{"key":"2521_CR94","unstructured":"Yang, Z., Li, L., Lin, K., Wang, J., Lin, C. C., Liu, Z. & Wang, L. (2023). The dawn of lmms: Preliminary explorations with gpt-4v (ision). arXiv preprint arXiv:2309.17421 9(1), 1."},{"key":"2521_CR95","unstructured":"Ye, W., Zheng, G., Ma, Y., Cao, X., Lai, B., Rehg, J. M. & Zhang, A. (2024). Mm-spubench: Towards better understanding of spurious biases in multimodal llms. arXiv preprint arXiv:2406.17126"},{"key":"2521_CR96","unstructured":"Yu, J., Wang, Z., Vasudevan, V., Yeung, L., Seyedhosseini, M. & Wu, Y. (2022). Coca: Contrastive captioners are image-text foundation models. arXiv preprint arXiv:2205.01917"},{"key":"2521_CR97","doi-asserted-by":"crossref","unstructured":"Yuan, Y., Li, W., Liu, J., Tang, D., Luo, X., Qin, C., Zhang, L. & Zhu, J. (2024). Osprey: Pixel understanding with visual instruction tuning. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, (pp. 28202\u201328211)","DOI":"10.1109\/CVPR52733.2024.02664"},{"key":"2521_CR98","doi-asserted-by":"crossref","unstructured":"Yue, X., Ni, Y., Zhang, K., Zheng, T., Liu, R., Zhang, G., Stevens, S., Jiang, D., Ren, W., Sun, Y., Wei, C., Yu, B., Yuan, R., Sun, R., Yin, M., Zheng, B., Yang, Z., Liu, Y., Huang, W., Sun, H., Su, Y. & Chen, W. (2024). Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, (pp. 9556\u20139567)","DOI":"10.1109\/CVPR52733.2024.00913"},{"key":"2521_CR99","doi-asserted-by":"crossref","unstructured":"Zhai, X., Kolesnikov, A., Houlsby, N. & Beyer, L. (2022). Scaling vision transformers. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, (pp. 12104\u201312113).","DOI":"10.1109\/CVPR52688.2022.01179"},{"key":"2521_CR100","doi-asserted-by":"crossref","unstructured":"Zhai, X., Mustafa, B., Kolesnikov, A. & Beyer, L. (2023). Sigmoid loss for language image pre-training. arXiv preprint arXiv:2303.15343","DOI":"10.1109\/ICCV51070.2023.01100"},{"key":"2521_CR101","doi-asserted-by":"crossref","unstructured":"Zhai, X., Wang, X., Mustafa, B., Steiner, A., Keysers, D., Kolesnikov, A. & Beyer, L. (2022). Lit: Zero-shot transfer with locked-image text tuning. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, (pp. 18123\u201318133).","DOI":"10.1109\/CVPR52688.2022.01759"},{"key":"2521_CR102","doi-asserted-by":"crossref","unstructured":"Zhang, X., Kuang, H., Mou, X., Lyu, H., Wu, K., Chen, S., Luo, J., Huang, X. & Wei, Z. (2024). Somelvlm: A large vision language model for social media processing. arXiv preprint arXiv:2402.13022.","DOI":"10.18653\/v1\/2024.findings-acl.140"},{"key":"2521_CR103","unstructured":"Zhang, T., Li, X., Fei, H., Yuan, H., Wu, S., Ji, S., Loy, C. C. & YAN, S. (2024). OMG-LLaVA: Bridging image-level, object-level, pixel-level reasoning and understanding. In: The Thirty-eighth Annual Conference on Neural Information Processing Systems. https:\/\/openreview.net\/forum?id=WeoNd6PRqS"},{"key":"2521_CR104","doi-asserted-by":"crossref","unstructured":"Zhang, S., Wen, L., Bian, X., Lei, Z. & Li, S. Z. (2018). Single-shot refinement neural network for object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (pp. 4203\u20134212).","DOI":"10.1109\/CVPR.2018.00442"},{"key":"2521_CR105","doi-asserted-by":"crossref","unstructured":"Zhao, Y., Misra, I., Kr\u00e4henb\u00fchl, P. & Girdhar, R.(2023). Learning video representations from large language models. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, (pp. 6586\u20136597).","DOI":"10.1109\/CVPR52729.2023.00637"}],"container-title":["International Journal of Computer Vision"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11263-025-02521-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11263-025-02521-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11263-025-02521-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T08:51:25Z","timestamp":1760086285000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11263-025-02521-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,14]]},"references-count":105,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2025,10]]}},"alternative-id":["2521"],"URL":"https:\/\/doi.org\/10.1007\/s11263-025-02521-4","relation":{},"ISSN":["0920-5691","1573-1405"],"issn-type":[{"value":"0920-5691","type":"print"},{"value":"1573-1405","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,14]]},"assertion":[{"value":"2 October 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 June 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 July 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}