{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,30]],"date-time":"2025-07-30T15:54:23Z","timestamp":1753890863685,"version":"3.41.2"},"reference-count":37,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,11,28]],"date-time":"2024-11-28T00:00:00Z","timestamp":1732752000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Big Data"],"abstract":"<jats:sec><jats:title>Background<\/jats:title><jats:p>Accurate sentiment analysis and intent categorization of tobacco and e-cigarette-related social media content are critical for public health research, yet they necessitate specialized natural language processing approaches.<\/jats:p><\/jats:sec><jats:sec><jats:title>Objective<\/jats:title><jats:p>To compare pre-trained and fine-tuned Flan-T5 models for intent classification and sentiment analysis of tobacco and e-cigarette tweets, demonstrating the effectiveness of pre-training a lightweight large language model for domain specific tasks.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>Three Flan-T5 classification models were developed: (1) tobacco intent, (2) e-cigarette intent, and (3) sentiment analysis. Domain-specific datasets with tobacco and e-cigarette tweets were created using GPT-4 and validated by tobacco control specialists using a rigorous evaluation process. A standardized rubric and consensus mechanism involving domain specialists ensured high-quality datasets. The Flan-T5 Large Language Models were fine-tuned using Low-Rank Adaptation and evaluated against pre-trained baselines on the datasets using accuracy performance metrics. To further assess model generalizability and robustness, the fine-tuned models were evaluated on real-world tweets collected around the COP9 event.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>In every task, fine-tuned models performed much better than pre-trained models. Compared to the pre-trained model's accuracy of 0.33, the fine-tuned model achieved an overall accuracy of 0.91 for tobacco intent classification. The fine-tuned model achieved an accuracy of 0.93 for e-cigarette intent, which is higher than the accuracy of 0.36 for the pre-trained model. The fine-tuned model significantly outperformed the pre-trained model's accuracy of 0.65 in sentiment analysis, achieving an accuracy of 0.94 for sentiments.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p>The effectiveness of lightweight Flan-T5 models in analyzing tweets associated with tobacco and e-cigarette is significantly improved by domain-specific fine-tuning, providing highly accurate instruments for tracking public conversation on tobacco and e-cigarette. The involvement of domain specialists in dataset validation ensured that the generated content accurately represented real-world discussions, thereby enhancing the quality and reliability of the results. Research on tobacco control and the formulation of public policy could be informed by these findings.<\/jats:p><\/jats:sec>","DOI":"10.3389\/fdata.2024.1501154","type":"journal-article","created":{"date-parts":[[2024,11,28]],"date-time":"2024-11-28T06:34:47Z","timestamp":1732775687000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Enhancing sentiment and intent analysis in public health via fine-tuned Large Language Models on tobacco and e-cigarette-related tweets"],"prefix":"10.3389","volume":"7","author":[{"given":"Sherif","family":"Elmitwalli","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"John","family":"Mehegan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Allen","family":"Gallagher","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Raouf","family":"Alebshehy","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2024,11,28]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10115-024-02156-w","article-title":"LLM examiner: automating assessment in informal self-directed e-learning using ChatGPT","volume":"2024","author":"Askarbekuly","year":"2024","journal-title":"Knowl. Inform. Syst"},{"key":"B2","doi-asserted-by":"publisher","first-page":"659","DOI":"10.1007\/s10579-023-09685-w","article-title":"Data augmentation strategies to improve text classification: a use case in smart cities","volume":"58","author":"Bencke","year":"2024","journal-title":"Lang. Resour. Eval"},{"key":"B3","doi-asserted-by":"publisher","first-page":"ooae060","DOI":"10.1093\/jamiaopen\/ooae060","article-title":"Leveraging GPT-4 for identifying cancer phe-notypes in electronic health records: a performance comparison between GPT-4, GPT-3.5-turbo, Flan-T5, Llama-3-8B, and spaCy's rule-based and machine learning-based methods","volume":"7","author":"Bhattarai","year":"2024","journal-title":"JAMIA Open"},{"key":"B4","doi-asserted-by":"publisher","first-page":"102183","DOI":"10.1016\/j.datak.2023.102183","article-title":"framework for investigating the dynamics of user and community sentiments in a social platform","volume":"146","author":"Bonifazi","year":"2023","journal-title":"Data Knowl. Eng"},{"key":"B5","doi-asserted-by":"publisher","first-page":"450","DOI":"10.1109\/TCSS.2019.2956957","article-title":"A survey of sentiment analysis from social media data","volume":"7","author":"Chakraborty","year":"2020","journal-title":"IEEE Trans. Comput. Soc. Syst"},{"key":"B6","first-page":"444","article-title":"\u201cData augmentation for supervised code translation learning,\u201d","author":"Chen","year":"2024","journal-title":"2024 IEEE\/ACM 21st International Conference on Mining Software Repositories (MSR)"},{"key":"B7","first-page":"364","article-title":"\u201cChallenges and opportunities in integrating LLMs into continuous integration\/continuous deployment (CI\/CD) pipelines,\u201d","volume-title":"2024 5th International Seminar on Artificial Intelligence, Networking and Information Technology (AINIT) 2024 Mar 29","author":"Chen","year":"2024"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2210.11416","article-title":"Scaling instruction-finetuned language models","author":"Chung","year":"2022","journal-title":"arXiv preprint arXiv:"},{"key":"B9","first-page":"1","article-title":"Scaling instruction-finetuned language models","volume":"25","author":"Chung","year":"2024","journal-title":"J. Machine Learn. Res."},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1810.04805","article-title":"BERT: pre-training of deep bidirectional transformers for language under-standing","author":"Devlin","year":"2018","journal-title":"arXiv preprint arXiv:"},{"key":"B11","doi-asserted-by":"publisher","first-page":"220","DOI":"10.1038\/s42256-023-00626-4","article-title":"Parameter-efficient fine-tuning of large-scale pre-trained language models","volume":"5","author":"Ding","year":"2023","journal-title":"Nat. Machine Intell"},{"key":"B12","doi-asserted-by":"publisher","first-page":"1357926","DOI":"10.3389\/fdata.2024.1357926","article-title":"Sentiment analysis of COP9-related tweets: a comparative study of pre-trained models and traditional techniques","volume":"7","author":"Elmitwalli","year":"2024","journal-title":"Front. Big Data"},{"key":"B13","doi-asserted-by":"publisher","first-page":"e0298298","DOI":"10.1371\/journal.pone.0298298","article-title":"Topic prediction for tobacco control based on COP9 tweets using machine learning techniques","volume":"19","author":"Elmitwalli","year":"2024","journal-title":"PLoS ONE"},{"key":"B14","doi-asserted-by":"publisher","first-page":"503","DOI":"10.1093\/ntr\/ntad190","article-title":"Oral nicotine gum discussions on Twitter: content analysis","volume":"26","author":"Galimov","year":"2024","journal-title":"Nicot. Tobacco Res"},{"key":"B15","doi-asserted-by":"publisher","first-page":"23627","DOI":"10.1109\/CVPR52733.2024.02230","article-title":"Visual-augmented dynamic semantic prototype for generative zero-shot learning","volume":"2024","author":"Hou","year":"2024","journal-title":"Proc. IEEE\/CVF Conf. Comput. Vis. Pat. Recogn."},{"key":"B16","doi-asserted-by":"publisher","first-page":"4317","DOI":"10.3390\/math11204317","article-title":"Structure-aware low-rank adaptation for parameter-efficient fine-tuning","volume":"11","author":"Hu","year":"2023","journal-title":"Mathematics"},{"key":"B17","doi-asserted-by":"crossref","first-page":"995","DOI":"10.18653\/v1\/2023.semeval-1.137","article-title":"\u201cSaama AI research at SemEval-2023 task 7: exploring the capabilities of Flan-T5 for multi-evidence natural language inference in clinical trial data,\u201d","volume-title":"Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)","author":"Kanakarajan","year":"2023"},{"key":"B18","doi-asserted-by":"publisher","first-page":"31538","DOI":"10.1109\/ACCESS.2024.3369918","article-title":"Text data augmentation techniques for word embeddings in fake news classification","volume":"12","author":"Kapusta","year":"2024","journal-title":"IEEE Access"},{"key":"B19","doi-asserted-by":"publisher","first-page":"20230254","DOI":"10.1098\/rsta.2023.0254","article-title":"Gpt-4 passes the bar exam","volume":"382","author":"Katz","year":"2024","journal-title":"Philos. Trans. Royal Soc. A"},{"key":"B20","doi-asserted-by":"publisher","first-page":"101861","DOI":"10.1016\/j.inffus.2023.101861","article-title":"ChatGPT: Jack of all trades, master of none","volume":"2023","author":"Koco\u0144","year":"2023","journal-title":"Inform. Fusion"},{"key":"B21","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1145\/3647782.3647803","article-title":"\u201cA literature survey on open source large language models,\u201d","volume-title":"Proceedings of the 2024 7th International Conference on Computers in Management and Business","author":"Kukreja","year":"2024"},{"key":"B22","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1007\/s40429-022-00463-2","article-title":"E-cigarette marketing on social media: a scoping review","volume":"10","author":"Lee","year":"2023","journal-title":"Curr. Addict. Rep"},{"key":"B23","first-page":"365","article-title":"\u201cData-efficient fine-tuning for LLM-based recommendation,\u201d","volume-title":"Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval 2024 Jul 10","author":"Lin","year":"2024"},{"key":"B24","doi-asserted-by":"publisher","first-page":"e50150","DOI":"10.2196\/50150","article-title":"A comparison of ChatGPT and fine-tuned Open Pre-Trained Transformers (OPT) against widely used sentiment analysis tools: sentiment analy-sis of COVID-19 survey data","volume":"11","author":"Lossio-Ventura","year":"2024","journal-title":"JMIR Ment. Health"},{"key":"B25","first-page":"15","article-title":"\u201cA review of evaluation metrics in machine learning algorithms,\u201d","volume-title":"Computer Science On-line Conference 2023 Apr 3","author":"Naidu","year":"2023"},{"key":"B26","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/IEMENTech60402.2023.10423400","article-title":"\u201cPerformance analysis of LoRA finetuning llama-2,\u201d","volume-title":"2023 7th International Conference on Electronics, Materials Engineering & Nano-Technology (IEMENTech)","author":"Pathak","year":"2023"},{"key":"B27","doi-asserted-by":"publisher","first-page":"200247","DOI":"10.1016\/j.iswa.2023.200247","article-title":"Efficient intent classifica-tion and entity recognition for university administrative services employing deep learning models","volume":"19","author":"Rizou","year":"2023","journal-title":"Intell. Syst. Appl"},{"key":"B28","doi-asserted-by":"publisher","first-page":"119862","DOI":"10.1016\/j.eswa.2023.119862","article-title":"A review on sentiment analysis from social media platforms","volume":"223","author":"Rodr\u00edguez-Ib\u00e1nez","year":"2023","journal-title":"Expert Syst. Appl"},{"key":"B29","first-page":"39","article-title":"\u201cTowards efficient fine-tuning of pre-trained code models: an experimental study and beyond,\u201d","volume-title":"Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis 2023 Jul 12","author":"Shi","year":"2023"},{"key":"B30","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1145\/3663433.3663456","article-title":"\u201cThe role of generative AI in qualitative research: GPT-4's contributions to a grounded theory analysis,\u201d","volume-title":"Proceedings of the Symposium on Learning, Design and Technology","author":"Sinha","year":"2024"},{"key":"B31","first-page":"821","article-title":"\u201cBridging numerical reasoning and headline generation for enhanced language models,\u201d","volume-title":"Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)","author":"Vaishnavi","year":"2024"},{"key":"B32","doi-asserted-by":"publisher","first-page":"bbad226","DOI":"10.1093\/bib\/bbad226","article-title":"PharmBERT: a domain-specific BERT model for drug labels","volume":"24","author":"ValizadehAslani","year":"2023","journal-title":"Brief. Bioinformat"},{"key":"B33","doi-asserted-by":"publisher","first-page":"75","DOI":"10.3389\/fcomm.2019.00075","article-title":"#Vape: measuring e-cigarette influence on Instagram with deep learning and text analysis","volume":"4","author":"Vassey","year":"2020","journal-title":"Front. Commun"},{"key":"B34","doi-asserted-by":"publisher","first-page":"195","DOI":"10.1007\/s10462-024-10812-4","article-title":"Review of few-shot learning application in CSI human sensing","volume":"57","author":"Wang","year":"2024","journal-title":"Artif. Intell. Rev"},{"key":"B35","doi-asserted-by":"publisher","first-page":"100171","DOI":"10.1016\/j.jjimei.2023.100171","article-title":"PyFin-sentiment: towards a machine-learning-based model for deriving sentiment from financial tweets","volume":"3","author":"Wilksch","year":"2023","journal-title":"Int. J. Inform. Manag. Data Insights"},{"key":"B36","doi-asserted-by":"publisher","first-page":"105316","DOI":"10.1016\/j.autcon.2024.105316","article-title":"Domain-specific language models pre-trained on construction management systems cor-pora","volume":"160","author":"Zhong","year":"2024","journal-title":"Automat. Construct"},{"key":"B37","first-page":"14089","article-title":"\u201cLLaMA-excitor: general instruction tuning via indirect feature interaction,\u201d","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Zou","year":"2024"}],"container-title":["Frontiers in Big Data"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdata.2024.1501154\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,28]],"date-time":"2024-11-28T06:34:55Z","timestamp":1732775695000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdata.2024.1501154\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,28]]},"references-count":37,"alternative-id":["10.3389\/fdata.2024.1501154"],"URL":"https:\/\/doi.org\/10.3389\/fdata.2024.1501154","relation":{},"ISSN":["2624-909X"],"issn-type":[{"type":"electronic","value":"2624-909X"}],"subject":[],"published":{"date-parts":[[2024,11,28]]},"article-number":"1501154"}}