{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,13]],"date-time":"2026-03-13T23:57:39Z","timestamp":1773446259649,"version":"3.50.1"},"reference-count":31,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,2,21]],"date-time":"2023-02-21T00:00:00Z","timestamp":1676937600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:p>The Bidirectional Encoder Representations from Transformers (BERT) architecture offers a cutting-edge approach to Natural Language Processing. It involves two steps: 1) pre-training a language model to extract contextualized features and 2) fine-tuning for specific downstream tasks. Although pre-trained language models (PLMs) have been successful in various text-mining applications, challenges remain, particularly in areas with limited labeled data such as plant health hazard detection from individuals' observations. To address this challenge, we propose to combine GAN-BERT, a model that extends the fine-tuning process with unlabeled data through a Generative Adversarial Network (GAN), with ChouBERT, a domain-specific PLM. Our results show that GAN-BERT outperforms traditional fine-tuning in multiple text classification tasks. In this paper, we examine the impact of further pre-training on the GAN-BERT model. We experiment with different hyper parameters to determine the best combination of models and fine-tuning parameters. Our findings suggest that the combination of GAN and ChouBERT can enhance the generalizability of the text classifier but may also lead to increased instability during training. Finally, we provide recommendations to mitigate these instabilities.<\/jats:p>","DOI":"10.3389\/frai.2023.1072329","type":"journal-article","created":{"date-parts":[[2023,2,21]],"date-time":"2023-02-21T06:58:38Z","timestamp":1676962718000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["Improving text mining in plant health domain with GAN and\/or pre-trained language model"],"prefix":"10.3389","volume":"6","author":[{"given":"Shufan","family":"Jiang","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"St\u00e9phane","family":"Cormier","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rafael","family":"Angarita","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Francis","family":"Rousseaux","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2023,2,21]]},"reference":[{"key":"B1","first-page":"281","article-title":"Random search for hyper-parameter optimization","volume":"13","author":"Bergstra","year":"2012","journal-title":"J. Mach. Learn. Res"},{"key":"B2","first-page":"228","article-title":"\u201cMulti-task and generative adversarial learning for robust and sustainable text classification,\u201d","author":"Breazzano","year":"2021","journal-title":"NL4AI@ AI"},{"key":"B3","doi-asserted-by":"publisher","first-page":"2114","DOI":"10.18653\/v1\/2020.acl-main.191","article-title":"\u201cGAN-BERT: generative adversarial learning for robust text classification with a bunch of labeled examples,\u201d","author":"Croce","year":"2020","journal-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics"},{"key":"B4","first-page":"21","article-title":"\u201cClassifying implant-bearing patients via their medical histories: a pre-study on swedish emrs with semi-supervised gan-bert,\u201d","author":"Danielsson","year":"2022","journal-title":"Proceedings of the 13th LREC Conference (LREC2022)"},{"key":"B5","unstructured":"DefourT.\n          EIP-AGRI Brochure Agricultural Knowledge and Innovation Systems. EIP-AGRI-European Commission2018"},{"key":"B6","first-page":"4171","article-title":"\u201cBERT: pre-training of deep bidirectional transformers for language understanding,\u201d","author":"Devlin","year":"2019","journal-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)"},{"key":"B7","first-page":"2672","article-title":"\u201cGenerative adversarial nets,\u201d","author":"Goodfellow","year":"2014","journal-title":"Proceedings of the 27th International Conference on Neural Information Processing Systems \u2013 Volume 2"},{"key":"B8","doi-asserted-by":"publisher","first-page":"515","DOI":"10.1016\/S0308-521X(01)00058-0","article-title":"Advances in application of climate prediction in agriculture","volume":"70","author":"Hammer","year":"2001","journal-title":"Agric Syst"},{"key":"B9","first-page":"6629","article-title":"\u201cGANs trained by a two time-scale update rule converge to a local nash equilibrium,\u201d","author":"Heusel","year":"2017","journal-title":"Proceedings of the 31st International Conference on Neural Information Processing Systems"},{"key":"B10","doi-asserted-by":"crossref","first-page":"653","DOI":"10.1007\/978-3-031-05760-1_40","article-title":"\u201cChouBERT: Pre-training french language model for crowdsensing with tweets in phytosanitary context,\u201d","author":"Jiang","year":"2022","journal-title":"Research Challenges in Information Science: 16th International Conference, RCIS 2022"},{"key":"B11","doi-asserted-by":"publisher","first-page":"e0150015","DOI":"10.1371\/journal.pone.0150015","article-title":"From observation to information: data-driven understanding of on farm yield variation","volume":"11","author":"Jim\u00e9nez","year":"2016","journal-title":"PLoS ONE"},{"key":"B12","doi-asserted-by":"publisher","first-page":"148","DOI":"10.1016\/j.jrurstud.2020.12.009","article-title":"Co-designing a smartphone app for and with farmers: empathising with end-users' values and needs","volume":"82","author":"Kenny","year":"2021","journal-title":"J. Rural Stud"},{"key":"B13","doi-asserted-by":"publisher","first-page":"1234","DOI":"10.1093\/bioinformatics\/btz682","article-title":"BioBERT: a pre-trained biomedical language representation model for biomedical text mining","volume":"36","author":"Lee","year":"2019","journal-title":"Bioinformatics"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1711.05101","article-title":"Decoupled weight decay regularization","author":"Loshchilov","year":"2017","journal-title":"arXiv preprint"},{"key":"B15","doi-asserted-by":"publisher","first-page":"7203","DOI":"10.18653\/v1\/2020.acl-main.645","article-title":"\u201cCamem BERT: A tasty french language model,\u201d","author":"Martin","year":"2020","journal-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-79408-8_15","article-title":"Insect pest management under climate change","author":"Massod","year":"2022","journal-title":"Build. Clim. Resil. Agri. Theor. Pract. Future Prespect."},{"key":"B17","doi-asserted-by":"publisher","first-page":"33","DOI":"10.18653\/v1\/2020.blackboxnlp-1.4","article-title":"\u201cWhat happens to BERT embeddings during fine-tuning?\u201d","author":"Merchant","year":"2020","journal-title":"Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP"},{"key":"B18","author":"Mirza","year":"2014","journal-title":"Conditional generative adversarial nets"},{"key":"B19","article-title":"\u201cOn the stability of fine-tuning BERT: Misconceptions, explanations, and strong baselines,\u201d","author":"Mosbach","year":"2021","journal-title":"9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.5772\/intechopen.101444","article-title":"\u201cClimate change and its consequences in agriculture,\u201d","author":"Mozaffari","year":"2022","journal-title":"The Nature, Causes, Effects and Mitigation of Climate Change on the Environment"},{"key":"B21","doi-asserted-by":"publisher","first-page":"878369","DOI":"10.3389\/fdgth.2022.878369","article-title":"Validating GAN-BioBERT: a methodology for assessing reporting trends in clinical trials","volume":"4","author":"Myszewski","year":"2022","journal-title":"Front. Digit. Health"},{"key":"B22","first-page":"292","article-title":"\u201cInternet of things (iot) and cloud computing for agriculture: an overview,\u201d","author":"Patil","year":"2012","journal-title":"Proceedings of Agro-Informatics and Precision Agriculture (AIPA 2012)"},{"key":"B23","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1016\/j.compag.2018.08.001","article-title":"Computer vision and artificial intelligence in precision agriculture for grain crops: a systematic review","volume":"153","author":"Patr\u00edcio","year":"2018","journal-title":"Comput. Electron. Agric"},{"key":"B24","first-page":"1","article-title":"\u201cAn investigation of social media's roles in knowledge exchange by farmers,\u201d","author":"Phillips","year":"2018","journal-title":"13th European International Farming Systems Association (IFSA) Symposium, Farming systems: facing uncertainties and enhancing opportunities, 1\u20135 July 2018"},{"key":"B25","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1016\/j.jrurstud.2021.08.028","article-title":"#farming365 - exploring farmers' social media use and the (re)presentation of farming lives","volume":"87","author":"Riley","year":"2021","journal-title":"J. Rural Stud"},{"key":"B26","first-page":"2234","article-title":"\u201cImproved techniques for training GANs,\u201d","author":"Salimans","year":"2016","journal-title":"Proceedings of the 30th International Conference on Neural Information Processing Systems"},{"key":"B27","first-page":"1","article-title":"\u201cSemi-supervised annotation of portuguese hate speech across social media domains,\u201d","volume":"11","author":"Santos","year":"2022","journal-title":"11th Symposium on Languages, Applications and Technologies (SLATE 2022), volume 104 of Open Access Series in Informatics (OASIcs)"},{"key":"B28","article-title":"JGAN-BERT, an adversarial learning architecture for paraphrase identification","author":"Ta","year":"2022","journal-title":"CEUR Worksho Proc."},{"key":"B29","doi-asserted-by":"crossref","first-page":"140","DOI":"10.1109\/IE.2015.29","article-title":"\u201cTowards smart farming and sustainable agriculture with drones,\u201d","author":"Tripicchio","year":"2015","journal-title":"2015 International Conference on Intelligent Environments"},{"key":"B30","author":"Wang","year":"2015","journal-title":"Social Sensing: Building Reliable Systems on Unreliable Data, 1st Edn."},{"key":"B31","doi-asserted-by":"publisher","first-page":"588","DOI":"10.1109\/JAS.2017.7510583","article-title":"Generative adversarial networks: introduction and outlook","volume":"4","author":"Wang","year":"2017","journal-title":"IEEE\/CAA J. Autom. Sin"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2023.1072329\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,2,21]],"date-time":"2023-02-21T06:58:55Z","timestamp":1676962735000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2023.1072329\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,21]]},"references-count":31,"alternative-id":["10.3389\/frai.2023.1072329"],"URL":"https:\/\/doi.org\/10.3389\/frai.2023.1072329","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,21]]},"article-number":"1072329"}}