{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,30]],"date-time":"2026-10-30T01:55:25Z","timestamp":1793325325592,"version":"4.5.0"},"reference-count":50,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,11,27]],"date-time":"2023-11-27T00:00:00Z","timestamp":1701043200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Plant Sci."],"abstract":"<jats:p>Diseases pose a significant threat to the citrus industry, and the accurate detection of these diseases represent key factors for their early diagnosis and precise control. Existing diagnostic methods primarily rely on image models trained on vast datasets and limited their applicability due to singular backgrounds. To devise a more accurate, robust, and versatile model for citrus disease classification, this study focused on data diversity, knowledge assistance, and modal fusion. Leaves from healthy plants and plants infected with 10 prevalent diseases (citrus greening, citrus canker, anthracnose, scab, greasy spot, melanose, sooty mold, nitrogen deficiency, magnesium deficiency, and iron deficiency) were used as materials. Initially, three datasets with white, natural, and mixed backgrounds were constructed to analyze their effects on the training accuracy, test generalization ability, and classification balance. This diversification of data significantly improved the model\u2019s adaptability to natural settings. Subsequently, by leveraging agricultural domain knowledge, a structured citrus disease features glossary was developed to enhance the efficiency of data preparation and the credibility of identification results. To address the underutilization of multimodal data in existing models, this study explored semantic embedding methods for disease images and structured descriptive texts. Convolutional networks with different depths (VGG16, ResNet50, MobileNetV2, and ShuffleNetV2) were used to extract the visual features of leaves. Concurrently, TextCNN and fastText were used to extract textual features and semantic relationships. By integrating the complementary nature of the image and text information, a joint learning model for citrus disease features was achieved. ShuffleNetV2 + TextCNN, the optimal multimodal model, achieved a classification accuracy of 98.33% on the mixed dataset, which represented improvements of 9.78% and 21.11% over the single-image and single-text models, respectively. This model also exhibited faster convergence, superior classification balance, and enhanced generalization capability, compared with the other methods. The image-text multimodal feature fusion network proposed in this study, which integrates text and image features with domain knowledge, can identify and classify citrus diseases in scenarios with limited samples and multiple background noise. The proposed model provides a more reliable decision-making basis for the precise application of biological and chemical control strategies for citrus production.<\/jats:p>","DOI":"10.3389\/fpls.2023.1280365","type":"journal-article","created":{"date-parts":[[2023,11,27]],"date-time":"2023-11-27T01:09:21Z","timestamp":1701047361000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":27,"title":["Detection of citrus diseases in complex backgrounds based on image\u2013text multimodal fusion and knowledge assistance"],"prefix":"10.3389","volume":"14","author":[{"given":"Xia","family":"Qiu","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hongwen","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ping","family":"Huang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dan","family":"Zhong","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tao","family":"Guo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Changbin","family":"Pu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zongnan","family":"Li","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yongling","family":"Liu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jin","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Si","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2023,11,27]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compag.2021.106279","article-title":"Tomato plant disease detection using transfer learning with C-GAN synthetic images","volume":"187","author":"Abbas","year":"2021","journal-title":"Comput. Electron. Agric."},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.1016\/j.compag.2019.105091","article-title":"Fine-tuning convolutional neural network with transfer learning for semantic segmentation of ground-level oilseed rape images in a field with high weed pressure","volume":"167","author":"Abdalla","year":"2019","journal-title":"Comput. Electron. Agric."},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.1016\/j.compag.2020.105661","article-title":"Comparison of convolution neural networks for smartphone image based real time classification of citrus leaf disease","volume":"177","author":"Barman","year":"2020","journal-title":"Comput. Electron. Agric."},{"key":"B4","doi-asserted-by":"publisher","first-page":"299","DOI":"10.1080\/08839514.2017.1315516","article-title":"Deep learning for tomato diseases: classification and symptoms visualization","volume":"31","author":"Brahimi","year":"2017","journal-title":"Appl. Artif. Intell."},{"key":"B5","doi-asserted-by":"crossref","first-page":"383","DOI":"10.1109\/ICCCIS51004.2021.9397098","volume-title":"2021 International Conference on Computing, Communication, and Intelligent Systems (ICCCIS)","author":"Chellapandi","year":"2021"},{"key":"B6","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1016\/j.compind.2019.02.003","article-title":"Deep neural networks with transfer learning in millet crop images","volume":"108","author":"Coulibaly","year":"2019","journal-title":"Comput. Ind."},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.1016\/j.compag.2021.106658","article-title":"Assessment of state-of-the-art deep learning based citrus disease detection techniques using annotated optical leaf images","volume":"193","author":"Dananjayan","year":"2022","journal-title":"Comput. Electron. Agric."},{"key":"B8","doi-asserted-by":"publisher","first-page":"7243","DOI":"10.1109\/JIOT.2021.3097379","article-title":"A Deep learning and social ioT approach for plants disease prediction toward a sustainable agriculture","volume":"9","author":"Delnevo","year":"2022","journal-title":"IEEE Internet Things."},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.3389\/fpls.2022.918940","article-title":"A vegetable leaf disease identification model based on image-text cross-modal feature fusion","volume":"13","author":"Feng","year":"2022","journal-title":"Front. Plant Sci."},{"key":"B10","doi-asserted-by":"publisher","first-page":"311","DOI":"10.1016\/j.compag.2018.01.009","article-title":"Deep learning models for plant disease detection and diagnosis","volume":"145","author":"Ferentinos","year":"2018","journal-title":"Comput. Electron. Agric."},{"key":"B11","doi-asserted-by":"publisher","first-page":"780","DOI":"10.1016\/j.eswa.2011.07.073","article-title":"Detecting rottenness caused by Penicillium genus fungi in citrus fruits using machine learning techniques","volume":"39","author":"G\u00f3mez-Sanchis","year":"2012","journal-title":"Expert Syst. Appl."},{"key":"B12","first-page":"1","volume-title":"Deep learning","author":"Goodfellow","year":"2016"},{"key":"B13","doi-asserted-by":"publisher","first-page":"520","DOI":"10.1109\/tcsvt.2019.2892802","article-title":"Fine-grained visual-textual representation learning","volume":"30","author":"He","year":"2020","journal-title":"IEEE Trans. Circuits. Syst. Video. Technol."},{"key":"B14","doi-asserted-by":"crossref","first-page":"770","DOI":"10.1109\/CVPR.2016.90","volume-title":"2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"He","year":"2016"},{"key":"B15","doi-asserted-by":"publisher","first-page":"1568","DOI":"10.1016\/j.asoc.2020.106597","article-title":"Uncertainty quantification for plant disease detection using Bayesian deep learning","volume":"96","author":"Hernandez","year":"2020","journal-title":"Appl. Soft. Comput."},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.3389\/fnbot.2022.881021","article-title":"Multi-scale feature fusion convolutional neural network for indoor small target detection","volume":"16","author":"Huang","year":"2022","journal-title":"Front. Neurorobotics."},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.1016\/j.compag.2021.106184","article-title":"Recognition of rice leaf diseases and wheat leaf diseases based on multi-task deep transfer learning","volume":"186","author":"Jiang","year":"2021","journal-title":"Comput. Electron. Agric."},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1612.03651","article-title":"Fasttext. zip: Compressing text classification models","author":"Joulin","year":"2016","journal-title":"arXiv. preprint.: 1612.03651"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1408.5882","article-title":"Convolutional neural networks for sentence classification","author":"Kim","year":"2014","journal-title":"arXiv. preprint.: 1408.5882"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.3390\/insects13100960","article-title":"Diaphorina citriIntegrated pest management strategies for asian citrus psyllid kuwayama (Hemiptera: psyllidae) and huanglongbing in citrus for sarawak, east Malaysia, borneo","volume":"13","author":"Leong","year":"2022","journal-title":"Insects"},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.3389\/fpls.2022.899754","article-title":"The detection method of potato foliage diseases in complex background based on instance segmentation and semantic segmentation","volume":"13","author":"Li","year":"2022","journal-title":"Front. Plant Sci."},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.1117\/1.oe.59.5.053103","article-title":"Enhanced situation awareness through CNN-based deep multimodal image fusion","volume":"59","author":"Liu","year":"2020","journal-title":"Optical. Eng."},{"key":"B23","doi-asserted-by":"publisher","first-page":"525","DOI":"10.32604\/iasc.2023.038287","article-title":"Lightweight method for plant disease identification using deep learning","volume":"37","author":"Lu","year":"2023","journal-title":"Intelligent. Automation. Soft. Computing."},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.11591\/ijece.v11i2.pp1719-1727","article-title":"Detection of citrus leaf diseases using a deep learning technique","volume":"11","author":"Luaibi","year":"2021","journal-title":"Int. J. Electrical. Comput. Eng. (IJECE)."},{"key":"B25","first-page":"122","volume-title":"15th european conference on computer vision (ECCV)","author":"Ma","year":"2018"},{"key":"B26","doi-asserted-by":"publisher","first-page":"055602","DOI":"10.1088\/1555-6611\/acc6bd","article-title":"Multiclass classifier based on deep learning for detection of citrus disease using fluorescence imaging spectroscopy","volume":"33","author":"Neves","year":"2023","journal-title":"Laser. Phys."},{"key":"B27","doi-asserted-by":"publisher","first-page":"700","DOI":"10.1186\/s12864-019-6007-4","article-title":"Origin and diversification of Xanthomonas citri subsp. citri pathotypes revealed by inclusive phylogenomic, dating, and biogeographic analyses","volume":"20","author":"Patan\u00e9","year":"2019","journal-title":"BMC Genomics"},{"key":"B28","doi-asserted-by":"publisher","first-page":"280","DOI":"10.1016\/j.compag.2018.04.002","article-title":"Deep convolutional neural networks for mobile capture device-based crop disease classification in the wild","volume":"161","author":"Picon","year":"2019","journal-title":"Comput. Electron. Agric."},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.3390\/s23104769","article-title":"Plant disease detection and classification: A systematic literature review","volume":"23","author":"Ramanjot","year":"2023","journal-title":"Sensors. (Basel)."},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.3389\/fpls.2017.01852","article-title":"Deep learning for image-based cassava disease detection","volume":"8","author":"Ramcharan","year":"2017","journal-title":"Front. Plant Sci."},{"key":"B31","doi-asserted-by":"publisher","first-page":"1040","DOI":"10.1016\/j.procs.2018.07.070","article-title":"Tomato crop disease classification using pre-trained deep learning algorithm","volume":"133","author":"Rangarajan","year":"2018","journal-title":"Proc. Comput. Sci."},{"key":"B32","doi-asserted-by":"publisher","DOI":"10.1016\/j.xplc.2020.100138","article-title":"Genomic insights into citrus domestication and its important agronomic traits","volume":"2","author":"Rao","year":"2021","journal-title":"Plant Commun."},{"key":"B33","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1109\/CVPR.2016.13","volume-title":"2016 IEEE conference on Computer Vision and Pattern Recognition (CVPR)","author":"Reed","year":"2016"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.3390\/plants9101319","article-title":"Plant disease classification: a comparative evaluation of convolutional neural networks and deep learning optimizers","volume":"9","author":"Saleem","year":"2020","journal-title":"Plants"},{"key":"B35","doi-asserted-by":"crossref","first-page":"4510","DOI":"10.1109\/CVPR.2018.00474","volume-title":"2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Sandler","year":"2018"},{"key":"B36","doi-asserted-by":"publisher","first-page":"119","DOI":"10.3390\/agriengineering1010009","article-title":"A computational procedure for the recognition and classification of maize leaf diseases out of healthy leaves using convolutional neural networks","volume":"1","author":"Sibiya","year":"2019","journal-title":"AgriEngineering"},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1409.1556","article-title":"Very deep convolutional networks for large-scale image recognition","author":"Simonyan","year":"2014","journal-title":"arXiv. preprint.: 1409.1556"},{"key":"B38","doi-asserted-by":"publisher","DOI":"10.1038\/s41559-023-02162-1","article-title":"Data leakage jeopardizes ecological applications of machine learning","volume":"8","author":"Stock","year":"2023","journal-title":"Nat. Ecol. Evol."},{"key":"B39","doi-asserted-by":"publisher","DOI":"10.3390\/ijms20215256","article-title":"Citrus genetic engineering for disease resistance: past, present and future","volume":"20","author":"Sun","year":"2019","journal-title":"Int. J. Mol. Sci."},{"key":"B40","doi-asserted-by":"publisher","first-page":"927","DOI":"10.1007\/s10489-021-02452-w","article-title":"Citrus disease detection and classification using end-to-end anchor-based deep learning model","volume":"52","author":"Syed-Ab-Rahman","year":"2022","journal-title":"Appl. Intell."},{"key":"B41","doi-asserted-by":"publisher","first-page":"1755","DOI":"10.18280\/ts.380619","article-title":"Early-stage brown spot disease recognition in paddy using image processing and deep learning techniques","volume":"38","author":"Upadhyay","year":"2021","journal-title":"Trait. Signal"},{"key":"B42","doi-asserted-by":"publisher","first-page":"180","DOI":"10.11975\/j.issn.1002-6819.2021.11.020","article-title":"Recognizing crop diseases using bimodal joint representation learning","volume":"37","author":"Wang","year":"2021","journal-title":"Trans. Chin. Soc. Agric. Eng."},{"key":"B43","doi-asserted-by":"publisher","DOI":"10.3389\/fpls.2021.731688","article-title":"A plant disease recognition method based on fusion of images and graph structure text","volume":"12","author":"Wang","year":"2022","journal-title":"Front. Plant Sci."},{"key":"B44","doi-asserted-by":"publisher","DOI":"10.1016\/j.compag.2021.106098","article-title":"Few-shot vegetable disease recognition model based on image text collaborative representation learning","volume":"184","author":"Wang","year":"2021","journal-title":"Comput. Electron. Agric."},{"key":"B45","doi-asserted-by":"publisher","DOI":"10.1093\/bib\/bbab037","article-title":"Machine learning for phytopathology: from the molecular scale towards the network scale","volume":"22","author":"Wang","year":"2021","journal-title":"Briefings Bioinf."},{"key":"B46","doi-asserted-by":"publisher","DOI":"10.3390\/s19143195","article-title":"Citrus pests and diseases recognition model using weakly dense connected convolution network","volume":"19","author":"Xing","year":"2019","journal-title":"Sensors. (Basel)."},{"key":"B47","doi-asserted-by":"publisher","DOI":"10.3389\/fpls.2021.809506","article-title":"\u221aCitrus huanglongbing detection based on multi-modal feature fusion learning","volume":"12","author":"Yang","year":"2021","journal-title":"Front. Plant Sci."},{"key":"B48","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3390\/agriculture12122014","article-title":"Tomato disease classification and identification method based on multimodal fusion deep learning","volume":"12","author":"Zhang","year":"2022","journal-title":"Agriculture"},{"key":"B49","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2020.106128","article-title":"An effective automatic system deployed in agricultural Internet of Things using Multi-Context Fusion Network towards crop disease recognition in the wild","volume":"89","author":"Zhao","year":"2020","journal-title":"Appl. Soft. Computing."},{"key":"B50","doi-asserted-by":"publisher","DOI":"10.1016\/j.compag.2021.106408","article-title":"Crop disease identification and interpretation method based on multimodal deep learning","volume":"189","author":"Zhou","year":"2021","journal-title":"Comput. Electron. Agric."}],"container-title":["Frontiers in Plant Science"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpls.2023.1280365\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,27]],"date-time":"2023-11-27T01:09:33Z","timestamp":1701047373000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpls.2023.1280365\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,27]]},"references-count":50,"alternative-id":["10.3389\/fpls.2023.1280365"],"URL":"https:\/\/doi.org\/10.3389\/fpls.2023.1280365","relation":{},"ISSN":["1664-462X"],"issn-type":[{"value":"1664-462X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,27]]},"article-number":"1280365"}}