{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,20]],"date-time":"2026-07-20T21:31:50Z","timestamp":1784583110703,"version":"3.55.0"},"reference-count":50,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2025,9,13]],"date-time":"2025-09-13T00:00:00Z","timestamp":1757721600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"\u201cRomanian Hub for Artificial Intelligence-HRIA\u201d, Smart Growth, Digitization and Financial Instruments Program","award":["334906"],"award-info":[{"award-number":["334906"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>The spread of misinformation during the COVID-19 pandemic raised widespread concerns about public health communication and media reliability. In this study, we focus on these issues as they manifested in Romanian-language media and employ Large Language Models (LLMs) to classify misinformation, with a particular focus on super-narratives\u2014broad thematic categories that capture recurring patterns and ideological framings commonly found in pandemic-related fake news, such as anti-vaccination discourse, conspiracy theories, or geopolitical blame. While some of the categories reflect global trends, others are shaped by the Romanian cultural and political context. We introduce a novel dataset of fake news centered on COVID-19 misinformation in the Romanian geopolitical context, comprising both annotated and unannotated articles. We experimented with multiple LLMs using zero-shot, few-shot, supervised, and semi-supervised learning strategies, achieving the best results with an LLaMA 3.1 8B model and semi-supervised learning, which yielded an F1-score of 78.81%. Experimental evaluations compared this approach to traditional Machine Learning classifiers augmented with morphosyntactic features. Results show that semi-supervised learning substantially improved classification results in both binary and multi-class settings. Our findings highlight the effectiveness of semi-supervised adaptation in low-resource, domain-specific contexts, as well as the necessity of enabling real-time misinformation tracking and enhancing transparency through claim-level explainability and fact-based counterarguments.<\/jats:p>","DOI":"10.3390\/info16090796","type":"journal-article","created":{"date-parts":[[2025,9,15]],"date-time":"2025-09-15T11:51:43Z","timestamp":1757937103000},"page":"796","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Detection of Fake News in Romanian: LLM-Based Approaches to COVID-19 Misinformation"],"prefix":"10.3390","volume":"16","author":[{"given":"Alexandru","family":"Dima","sequence":"first","affiliation":[{"name":"Computer Science & Engineering Department, National University of Science and Technology POLITEHNICA Bucharest, 313 Splaiul Independentei, 060042 Bucharest, Romania"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-4855-1533","authenticated-orcid":false,"given":"Ecaterina","family":"Ilis","sequence":"additional","affiliation":[{"name":"Department of British and American Studies, Lucian Blaga University of Sibiu, Bulevardul Victoriei 10, 550024 Sibiu, Romania"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8839-853X","authenticated-orcid":false,"given":"Diana","family":"Florea","sequence":"additional","affiliation":[{"name":"Department of Romance Studies, Lucian Blaga University of Sibiu, Bulevardul Victoriei 10, 550024 Sibiu, Romania"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4815-9227","authenticated-orcid":false,"given":"Mihai","family":"Dascalu","sequence":"additional","affiliation":[{"name":"Computer Science & Engineering Department, National University of Science and Technology POLITEHNICA Bucharest, 313 Splaiul Independentei, 060042 Bucharest, Romania"},{"name":"Academy of Romanian Scientists, Str. Ilfov, Nr. 3, 050044 Bucharest, Romania"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,9,13]]},"reference":[{"key":"ref_1","first-page":"137","article-title":"Defining \u201cfake news\u201d A typology of scholarly definitions","volume":"6","author":"Tandoc","year":"2018","journal-title":"Digit. J."},{"key":"ref_2","unstructured":"Wang, X., Zhang, W., and Rajtmajer, S. (2024). Monolingual and Multilingual Misinformation Detection for Low-Resource Languages: A Comprehensive Survey. arXiv."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3395046","article-title":"A survey of fake news: Fundamental theories, detection methods, and opportunities","volume":"53","author":"Zhou","year":"2020","journal-title":"ACM Comput. Surv."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1109\/MIS.2009.36","article-title":"The unreasonable effectiveness of data","volume":"24","author":"Halevy","year":"2009","journal-title":"IEEE Intell. Syst."},{"key":"ref_5","unstructured":"Al-Onaizan, Y., Bansal, M., and Chen, Y.N. (2024, January 12\u201316). \u201cVorbe\u0219ti Rom\u00e2ne\u0219te?\u201d A Recipe to Train Powerful Romanian LLMs with English Instructions. Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, FL, USA."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Hughes, B., Miller-Idriss, C., Piltch-Loeb, R., Goldberg, B., White, K., Criezis, M., and Savoia, E. (2021). Development of a codebook of online anti-vaccination rhetoric to manage COVID-19 vaccine misinformation. Int. J. Environ. Res. Public Health, 18.","DOI":"10.1101\/2021.03.23.21253727"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"101571","DOI":"10.1016\/j.jksuci.2023.101571","article-title":"A comprehensive survey of fake news in social networks: Attributes, features, and detection approaches","volume":"35","author":"Kondamudi","year":"2023","journal-title":"J. King Saud-Univ. Comput. Inf. Sci."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Tsai, C.M. (2023). Stylometric detection of fake news based on natural language processing using named entity recognition: In-domain and cross-domain analysis. Electronics, 12.","DOI":"10.3390\/electronics12173676"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"103725","DOI":"10.1016\/j.ipm.2024.103725","article-title":"Domain-and category-style clustering for general detection of fake news via contrastive learning","volume":"61","author":"Wu","year":"2024","journal-title":"Inf. Process. Manag."},{"key":"ref_10","unstructured":"Castillo, C., Mendoza, M., and Poblete, B. (April, January 28). Information credibility on twitter. Proceedings of the 20th International Conference on World Wide Web, Hyderabad, India."},{"key":"ref_11","unstructured":"Ruchansky, N., Seo, S., and Liu, Y. (2017, January 6\u201310). Csi: A hybrid deep model for detection of fake news. Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, Singapore."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Reis, J.C., Correia, A., Murai, F., Veloso, A., and Benevenuto, F. (2018, January 27\u201330). Explainable machine learning for detection of fake news. Proceedings of the 10th ACM Conference on Web Science, Amsterdam, The Netherlands.","DOI":"10.1145\/3292522.3326027"},{"key":"ref_13","unstructured":"Ma, J., Gao, W., Mitra, P., Kwon, S., Jansen, B.J., Wong, K.F., and Cha, M. (2016, January 9\u201315). Detecting rumors from microblogs with recurrent neural networks. Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, New York, NY, USA. IJCAI\u201916."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"396","DOI":"10.1016\/j.icte.2021.10.003","article-title":"Detection of fake news using deep learning CNN\u2013RNN based methods","volume":"8","author":"Sastrawan","year":"2022","journal-title":"ICT Express"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"335","DOI":"10.1007\/s41060-021-00302-z","article-title":"Fake news detection based on news content and social contexts: A transformer-based approach","volume":"13","author":"Raza","year":"2022","journal-title":"Int. J. Data Sci. Anal."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., and Zettlemoyer, L. (2019). Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv.","DOI":"10.18653\/v1\/2020.acl-main.703"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"N\u00f8rregaard, J., Horne, B.D., and Adal\u0131, S. (2019, January 11\u201314). NELA-GT-2018: A large multi-labelled news dataset for the study of misinformation in news articles. Proceedings of the International AAAI Conference on Web and Social Media, M\u00fcnich, Germany.","DOI":"10.1609\/icwsm.v13i01.3261"},{"key":"ref_18","unstructured":"Nakamura, K., Levy, S., and Wang, W.Y. (2020, January 13\u201315). Fakeddit: A New Multimodal Benchmark Dataset for Fine-grained Fake News Detection. Proceedings of the Twelfth Language Resources and Evaluation Conference, Marseille, France."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Ma, X., Zhang, Y., Ding, K., Yang, J., Wu, J., and Fan, H. (2024, January 12\u201316). On Fake News Detection with LLM Enhanced Semantics Mining. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Miami, FL, USA.","DOI":"10.18653\/v1\/2024.emnlp-main.31"},{"key":"ref_20","unstructured":"Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., and Bhosale, S. (2023). Llama 2: Open foundation and fine-tuned chat models. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zhou, X., Wu, J., and Zafarani, R. (2020, January 11\u201314). SAFE: Similarity-aware multi-modal detection of fake news. Proceedings of the Pacific-Asia Conference on Knowledge Discovery and Data Mining, Singapore.","DOI":"10.1007\/978-3-030-47436-2_27"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Segura-Bedmar, I., and Alonso-Bartolome, S. (2022). Multimodal detection of fake news. Information, 13.","DOI":"10.3390\/info13060284"},{"key":"ref_23","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). Bert: Bidirectional encoder representations from transformers. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Buzea, M.C., Trausan-Matu, S., and Rebedea, T. (2022). Automatic detection of fake news for romanian online news. Information, 13.","DOI":"10.3390\/info13030151"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Moisi, E.V., Mihalca, B.C., Coman, S.M., Pater, A.M., and Popescu, D.E. (2024). Romanian Fake News Detection Using Machine Learning and Transformer-Based Approaches. Appl. Sci., 14.","DOI":"10.20944\/preprints202411.1843.v1"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1016\/j.jbusres.2020.11.037","article-title":"Fake news, social media and marketing: A systematic review","volume":"124","author":"Sit","year":"2021","journal-title":"J. Bus. Res."},{"key":"ref_27","unstructured":"Wardle, C., and Derakhshan, H. (2017). Information Disorder: Toward An Interdisciplinary Framework for Research and Policymaking, Council of Europe Strasbourg."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"122","DOI":"10.1177\/0267323118760317","article-title":"The disinformation order: Disruptive communication and the decline of democratic institutions","volume":"33","author":"Bennett","year":"2018","journal-title":"Eur. J. Commun."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"102025","DOI":"10.1016\/j.ipm.2019.03.004","article-title":"An overview of online fake news: Characterization, detection, and discussion","volume":"57","author":"Zhang","year":"2020","journal-title":"Inf. Process. Manag."},{"key":"ref_30","first-page":"97","article-title":"Fake news as a two-dimensional phenomenon: A framework and research agenda","volume":"43","author":"Egelhofer","year":"2019","journal-title":"Ann. Int. Commun. Assoc."},{"key":"ref_31","first-page":"157","article-title":"Causes and consequences of mainstream media dissemination of fake news: Literature review and synthesis","volume":"44","author":"Tsfati","year":"2020","journal-title":"Ann. Int. Commun. Assoc."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Terian, S.M. (2025, August 07). What Is Fake News: A New Definition. Available online: https:\/\/revistatransilvania.ro\/wp-content\/uploads\/2021\/12\/Transilvania-11-12.2021-112-120.pdf.","DOI":"10.51391\/trva.2021.11-12.17."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"4944","DOI":"10.1287\/mnsc.2019.3478","article-title":"The implied truth effect: Attaching warnings to a subset of fake news headlines increases perceived accuracy of headlines without warnings","volume":"66","author":"Pennycook","year":"2020","journal-title":"Manag. Sci."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1016\/j.jarmac.2017.07.008","article-title":"Beyond misinformation: Understanding and coping with the \u201cpost-truth\u201d era","volume":"6","author":"Lewandowsky","year":"2017","journal-title":"J. Appl. Res. Mem. Cogn."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1124","DOI":"10.1126\/science.185.4157.1124","article-title":"Judgment under Uncertainty: Heuristics and Biases: Biases in judgments reveal some heuristics of thinking under uncertainty","volume":"185","author":"Tversky","year":"1974","journal-title":"Science"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Stroud, N.J. (2011). Niche News: The Politics of News Choice, Oxford University Press.","DOI":"10.1093\/acprof:oso\/9780199755509.001.0001"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"18888","DOI":"10.1073\/pnas.1908369116","article-title":"Cross-national evidence of a negativity bias in psychophysiological reactions to news","volume":"116","author":"Soroka","year":"2019","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1177\/1529100612451018","article-title":"Misinformation and its correction: Continued influence and successful debiasing","volume":"13","author":"Lewandowsky","year":"2012","journal-title":"Psychol. Sci. Public Interest"},{"key":"ref_39","unstructured":"BuzzSumo Ltd. (2012, June 01). BuzzSumo: Media Mentions in Minutes. Content Ideas for Days. Available online: https:\/\/buzzsumo.com\/."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1094","DOI":"10.1126\/science.aao2998","article-title":"The science of fake news","volume":"359","author":"Lazer","year":"2018","journal-title":"Science"},{"key":"ref_41","unstructured":"Berger, M. (2025, July 29). RapidFuzz\/Levenshtein: Fastest Levenshtein Implementation. Available online: https:\/\/github.com\/rapidfuzz\/Levenshtein."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Dascalu, M., Gutu, G., Ruseti, S., Paraschiv, I.C., Dessus, P., McNamara, D.S., Crossley, S.A., and Trausan-Matu, S. (2017, January 12\u201315). ReaderBench: A multi-lingual framework for analyzing text complexity. Proceedings of the Data Driven Approaches in Digital Education: 12th European Conference on Technology Enhanced Learning, EC-TEL 2017, Tallinn, Estonia. Proceedings 12.","DOI":"10.1007\/978-3-319-66610-5_48"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"881","DOI":"10.1109\/TCSS.2021.3068519","article-title":"WELFake: Word embedding over linguistic features for fake news detection","volume":"8","author":"Verma","year":"2021","journal-title":"IEEE Trans. Comput. Soc. Syst."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"62097","DOI":"10.1109\/ACCESS.2022.3181184","article-title":"A hybrid linguistic and knowledge-based analysis approach for fake news detection on social media","volume":"10","author":"Seddari","year":"2022","journal-title":"IEEE Access"},{"key":"ref_45","unstructured":"Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., and Vaughan, A. (2024). The llama 3 herd of models. arXiv."},{"key":"ref_46","unstructured":"Jiang, A.Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D.S., de las Casas, D., Bressand, F., Lengyel, G., Lample, G., and Saulnier, L. (2023). Mistral 7B. arXiv."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Reimers, N., and Gurevych, I. (2019, January 3\u20137). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, Hong Kong, China.","DOI":"10.18653\/v1\/D19-1410"},{"key":"ref_48","first-page":"596","article-title":"Fixmatch: Simplifying semi-supervised learning with consistency and confidence","volume":"33","author":"Sohn","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_49","unstructured":"Inui, K., Jiang, J., Ng, V., and Wan, X. (2019, January 3\u20137). EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China."},{"key":"ref_50","first-page":"53728","article-title":"Direct preference optimization: Your language model is secretly a reward model","volume":"36","author":"Rafailov","year":"2023","journal-title":"Adv. Neural Inf. Process. Syst."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/16\/9\/796\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:45:08Z","timestamp":1760035508000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/16\/9\/796"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,13]]},"references-count":50,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2025,9]]}},"alternative-id":["info16090796"],"URL":"https:\/\/doi.org\/10.3390\/info16090796","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,9,13]]}}}