{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,20]],"date-time":"2026-04-20T23:46:56Z","timestamp":1776728816543,"version":"3.51.2"},"reference-count":37,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,11,6]],"date-time":"2025-11-06T00:00:00Z","timestamp":1762387200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:p>The rapid growth of social media has resulted in an explosion of online news content, leading to a significant increase in the spread of misleading or false information. While machine learning techniques have been widely applied to detect fake news, the scarcity of labeled datasets remains a critical challenge. Misinformation frequently appears as paired text and images, where a news article or headline is accompanied by a related visuals. In this paper, we introduce a self-learning multimodal model for fake news classification. The model leverages contrastive learning, a robust method for feature extraction that operates without requiring labeled data, and integrates the strengths of Large Language Models (LLMs) to jointly analyze both text and image features. LLMs are excel at this task due to their ability to process diverse linguistic data drawn from extensive training corpora. Our experimental results on a public dataset demonstrate that the proposed model outperforms several state-of-the-art classification approaches, achieving over 85% accuracy, precision, recall, and F1-score. These findings highlight the model's effectiveness in tackling the challenges of multimodal fake news detection.<\/jats:p>","DOI":"10.3389\/frai.2025.1665798","type":"journal-article","created":{"date-parts":[[2025,11,6]],"date-time":"2025-11-06T12:56:41Z","timestamp":1762433801000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["A self-learning multimodal approach for fake news detection"],"prefix":"10.3389","volume":"8","author":[{"given":"Hao","family":"Chen","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yue","family":"Yu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hui","family":"Guo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Baochen","family":"Hu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shu","family":"Hu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jinrong","family":"Hu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Siwei","family":"Lyu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xi","family":"Wu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ching-Sheng","family":"Lin","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xin","family":"Wang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2025,11,6]]},"reference":[{"key":"B1","article-title":"Acm multimedia grand challenge on detecting cheapfakes","author":"Aneja","year":"2022","journal-title":"ArXiv, abs\/2207.14534"},{"key":"B2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3670694","article-title":"A framework for enhancing social media misinformation detection with topical-tactics","volume":"5","author":"Bagozzi","year":"2024","journal-title":"Dig. Threat"},{"key":"B3","doi-asserted-by":"publisher","first-page":"38","DOI":"10.1016\/j.ins.2019.05.035","article-title":"A survey on fake news and rumour detection techniques","volume":"497","author":"Bondielli","year":"2019","journal-title":"Inf. Sci"},{"key":"B4","article-title":"Language models are few-shot learners","author":"Brown","year":"2020","journal-title":"ArXiv, abs\/2005.14165"},{"key":"B5","doi-asserted-by":"crossref","first-page":"557","DOI":"10.1007\/978-981-16-0733-2_39","article-title":"\u201cDeepfake: an overview,\u201d","volume-title":"Proceedings of second international conference on computing, communications, and cyber-security: IC4S 2020","author":"Chadha","year":"2021"},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW59228.2023.00099","article-title":"\u201cHarnessing the power of text-image contrastive models for automatic detection of online misinformation,\u201d","author":"Chen","year":"2023","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops"},{"key":"B7","unstructured":"Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, march 2023\n          \n          6\n          \n            \n              Chiang\n              W.-L.\n            \n            \n              Li\n              Z.\n            \n            \n              Lin\n              Z.\n            \n            \n              Sheng\n              Y.\n            \n            \n              Wu\n              Z.\n            \n            \n              Zhang\n              H.\n            \n          \n          3\n          2023"},{"key":"B8","first-page":"4171","article-title":"\u201cBert: pre-training of deep bidirectional transformers for language understanding,\u201d","author":"Devlin","year":"2019","journal-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.1109\/DSAA49011.2020.00091","article-title":"\u201cMultimodal multi-image fake news detection,\u201d","author":"Giachanou","year":"2020","journal-title":"2020 IEEE 7th International Conference on Data Science and Advanced Analytics (DSAA)"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.1109\/UKRCON.2017.8100379","article-title":"\u201cFake news detection using naive bayes classifier,\u201d","author":"Granik","year":"2017","journal-title":"2017 IEEE First Ukraine Conference on Electrical and Computer Engineering (UKRCON)"},{"key":"B11","article-title":"The future of misinformation detection: new perspectives and trends","author":"Guo","year":"2019","journal-title":"ArXiv, abs\/1909.03654"},{"key":"B12","first-page":"2904","article-title":"\u201cEyes tell all: irregular pupil shapes reveal gan-generated faces,\u201d","volume-title":"ICASSP 2022\u20132022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","author":"Guo","year":""},{"key":"B13","article-title":"Open-eye: an open platform to study human performance on identifying ai-synthesized faces","author":"Guo","year":"","journal-title":"arXiv preprint arXiv:2205.06680"},{"key":"B14","doi-asserted-by":"publisher","first-page":"32574","DOI":"10.1109\/ACCESS.2022.3157297","article-title":"Robust attentive deep neural network for exposing gan-generated faces","volume":"10","author":"Guo","year":"","journal-title":"IEEE Access"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.1109\/ICCWAMTIP51612.2020.9317325","article-title":"\u201cFake news detection using deep recurrent neural networks,\u201d","author":"Jiang","year":"2020","journal-title":"2020 17th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP)"},{"key":"B16","first-page":"994","article-title":"\u201cCafe: catastrophic data leakage in vertical federated learning,\u201d","author":"Jin","year":"2021","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B17","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1016\/j.cogsys.2019.12.005","article-title":"Fndnet \u2013 a deep convolutional neural network for fake news detection","volume":"61","author":"Kaliyar","year":"2020","journal-title":"Cogn. Syst. Res"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00781","article-title":"\u201cMulti-task learning using uncertainty to weigh losses for scene geometry and semantics,\u201d","author":"Kendall","year":"2018","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B19","first-page":"19730","article-title":"\u201cBlip-2: bootstrapping language-image pre-training with frozen image encoders and large language models,\u201d","volume-title":"International Conference on Machine Learning","author":"Li","year":"2023"},{"key":"B20","article-title":"Large language model agent for fake news detection","author":"Li","year":"2024","journal-title":"ArXiv, abs\/2405.01593"},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.1145\/3664647.3681089","article-title":"\u201cFka-owl: advancing multimodal fake news detection through knowledge-augmented lvlms,\u201d","author":"Liu","year":"2024","journal-title":"Proceedings of the 32nd ACM International Conference on Multimedia"},{"key":"B22","doi-asserted-by":"publisher","first-page":"112986","DOI":"10.1016\/j.eswa.2019.112986","article-title":"Fake news, rumor, information pollution in social media and web: a contemporary survey of state-of-the-arts, challenges and opportunities","volume":"153","author":"Meel","year":"2020","journal-title":"Expert Syst. Appl"},{"key":"B23","article-title":"\u201cFakeddit: a new multimodal benchmark dataset for fine-grained fake news detection,\u201d","author":"Nakamura","year":"2020","journal-title":"International Conference on Language Resources and Evaluation"},{"key":"B24","doi-asserted-by":"publisher","first-page":"100007","DOI":"10.1016\/j.jjimei.2020.100007","article-title":"Fake news detection: a hybrid CNN-RNN based deep learning approach","volume":"1","author":"Nasir","year":"2021","journal-title":"Int. J. Inf. Manag. Data Insights"},{"key":"B25","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1007\/s13735-023-00312-6","article-title":"Verite: a robust benchmark for multimodal misinformation detection accounting for unimodal bias","volume":"13","author":"Papadopoulos","year":"2023","journal-title":"Int. J. Multim. Inf. Retr"},{"key":"B26","doi-asserted-by":"publisher","first-page":"108832","DOI":"10.1016\/j.patcog.2022.108832","article-title":"Learning a deep dual-level network for robust deepfake detection","volume":"130","author":"Pu","year":"2022","journal-title":"Patt. Recognit"},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2019.00062","article-title":"\u201cExploiting multi-domain visual information for fake news detection,\u201d","author":"Qi","year":"2019","journal-title":"2019 IEEE International Conference on Data Mining (ICDM)"},{"key":"B28","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3462871","article-title":"\u201cHierarchical multi-modal contextual attention network for fake news detection,\u201d","author":"Qian","year":"2021","journal-title":"Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval"},{"key":"B29","article-title":"Very deep convolutional networks for large-scale image recognition","author":"Simonyan","year":"2014","journal-title":"CoRR, abs\/1409.1556"},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.1109\/BigMM.2019.00-44","article-title":"\u201cSpotfake: a multi-modal framework for fake news detection,\u201d","author":"Singhal","year":"2019","journal-title":"2019 IEEE Fifth International Conference on Multimedia Big Data (BigMM), pages"},{"key":"B31","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.findings-naacl.95","article-title":"Adapting fake news detection to the era of large language models","author":"Su","year":"2023","journal-title":"ArXiv, abs\/2311.04917"},{"key":"B32","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1007\/978-981-19-9304-6_6","author":"Sudhakar","year":"2023","journal-title":"Fake News Detection Approach Based on Logistic Regression in Machine Learning"},{"key":"B33","doi-asserted-by":"publisher","DOI":"10.1109\/CSPA60979.2024.10525308","article-title":"\u201cIntegrating large language models and machine learning for fake news detection,\u201d","author":"Teo","year":"2024","journal-title":"2024 20th IEEE International Colloquium on Signal Processing &Its Applications (CSPA)"},{"key":"B34","article-title":"GAN-generated faces detection: a survey and new perspectives","author":"Wang","year":"2023","journal-title":"arXiv preprint arXiv:2202.07145"},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219903","article-title":"\u201cEann: event adversarial neural networks for multi-modal fake news detection,\u201d","author":"Wang","year":"2018","journal-title":"Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery &Data Mining"},{"key":"B36","article-title":"Minigpt-4: enhancing vision-language understanding with advanced large language models","author":"Zhu","year":"2023","journal-title":"ArXiv, abs\/2304.10592"},{"key":"B37","first-page":"1218","article-title":"\u201cA robustly optimized BERT pre-training approach with post-training,\u201d","author":"Zhuang","year":"2021","journal-title":"Proceedings of the 20th Chinese National Conference on Computational Linguistics"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2025.1665798\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,6]],"date-time":"2025-11-06T12:56:44Z","timestamp":1762433804000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2025.1665798\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,6]]},"references-count":37,"alternative-id":["10.3389\/frai.2025.1665798"],"URL":"https:\/\/doi.org\/10.3389\/frai.2025.1665798","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,6]]},"article-number":"1665798"}}