{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,18]],"date-time":"2026-07-18T16:18:31Z","timestamp":1784391511754,"version":"3.55.0"},"reference-count":64,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2025,6,16]],"date-time":"2025-06-16T00:00:00Z","timestamp":1750032000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"NSF","award":["DMS-2022448"],"award-info":[{"award-number":["DMS-2022448"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computers"],"abstract":"<jats:p>The spread of fake news on social media is complicated by the fact that fake information spreads extremely fast in both textual and visual formats. Traditional approaches to the detection of fake news focus mainly on text and image features, thereby missing valuable information contained within images and texts. In response to this, we propose a multimodal fake news detection method based on BERT, with an extension to text combined with the extracted text from images through Optical Character Recognition (OCR). Here, we consider extending feature analysis with BERT_base_uncased to process inputs for retrieving relevant text from images and determining a confidence score that suggests the probability of the news being authentic. We report extensive experimental results on the ISOT, WELFAKE, TRUTHSEEKER, and ISOT_WELFAKE_TRUTHSEEKER datasets. Our proposed model demonstrates better generalization on the TRUTHSEEKER dataset with an accuracy of 99.97%, achieving substantial improvements over existing methods with an F1-score of 0.98. Experimental results indicate a potential accuracy increment of +3.35% compared to the latest baselines. These results highlight the potential of our approach to serve as a strong resource for automatic fake news detection by effectively integrating both text and visual data streams. Findings suggest that using diverse datasets enhances the resilience of detection systems against misinformation strategies.<\/jats:p>","DOI":"10.3390\/computers14060237","type":"journal-article","created":{"date-parts":[[2025,6,16]],"date-time":"2025-06-16T09:51:22Z","timestamp":1750067482000},"page":"237","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["A BERT-Based Multimodal Framework for Enhanced Fake News Detection Using Text and Image Data Fusion"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-5486-0415","authenticated-orcid":false,"given":"Mohammed","family":"Al-alshaqi","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering and Computer Science, Howard University, Washington, DC 20059, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3638-3464","authenticated-orcid":false,"given":"Danda B.","family":"Rawat","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering and Computer Science, Howard University, Washington, DC 20059, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-4445-2192","authenticated-orcid":false,"given":"Chunmei","family":"Liu","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering and Computer Science, Howard University, Washington, DC 20059, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,6,16]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1007\/s13278-023-01028-5","article-title":"Fake news, disinformation and misinformation in social media: A review","volume":"13","author":"Aimeur","year":"2023","journal-title":"Soc. Netw. Anal. Min."},{"key":"ref_2","unstructured":"Atske, S. (2024, October 16). Social Media and News Fact Sheet. Pew Research Center. Available online: https:\/\/www.pewresearch.org\/journalism\/fact-sheet\/social-media-and-news-fact-sheet\/."},{"key":"ref_3","unstructured":"Konopliov (2024, June 26). Fake News Statistics & Facts (2024)\u2014Redline Digital. Available online: https:\/\/redline.digital\/fake-news-statistics\/."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Zhao, J., Zhao, Z., Shi, L., Kuang, Z., and Liu, Y. (2023). Collaborative mixture-of-experts model for multi-domain fake news detection. Electronics, 12.","DOI":"10.3390\/electronics12163440"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"100531","DOI":"10.1016\/j.cosrev.2022.100531","article-title":"Studying fake news spreading, polarisation dynamics, and manipulation by bots: A tale of networks and language","volume":"47","author":"Ruffo","year":"2023","journal-title":"Comput. Sci. Rev."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"e32","DOI":"10.1017\/dap.2021.20","article-title":"The role of artificial intelligence in disinformation","volume":"3","author":"Bontridder","year":"2021","journal-title":"Data Policy"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Al-Alshaqi, M., and Rawat, D.B. (2023, January 11\u201312). Disinformation Classification Using Transformer-Based Machine Learning. Proceedings of the 2023 6th International Seminar on Research of Information Technology and Intelligent Systems (ISRITI), Batam, Indonesia.","DOI":"10.1109\/ISRITI60336.2023.10467980"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Al-Alshaqi, M., Rawat, D.B., and Liu, C. (2023, January 24). Emotion-Aware Fake News Detection on Social Media with BERT Embeddings. Proceedings of the 2023 International Conference on Modeling & E-Information Research, Artificial Learning and Digital Applications (ICMERALDA), Karawang, Indonesia.","DOI":"10.1109\/ICMERALDA60125.2023.10458214"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"93104","DOI":"10.1109\/ACCESS.2022.3204051","article-title":"Explainable Artificial Intelligence Applications in Cyber Security: State-of-the-Art in Research","volume":"10","author":"Zhang","year":"2022","journal-title":"IEEE Access"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Karlson, N. (2024). Reviving Classical Liberalism Against Populism, Springer Nature.","DOI":"10.1007\/978-3-031-49074-3"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"100395","DOI":"10.1016\/j.crm.2022.100395","article-title":"Lessons from COVID-19 for managing transboundary climate risks and building resilience","volume":"35","author":"Ringsmuth","year":"2022","journal-title":"Clim. Risk Manag."},{"key":"ref_12","first-page":"717","article-title":"Privacy Nicks: How the Law Normalizes Surveillance","volume":"101","author":"Hartzog","year":"2023","journal-title":"SSRN Electron. J."},{"key":"ref_13","unstructured":"Barrett, C.M. (2015). Automated esSay Evaluation and the Computational Paradigm: Machine Scoring Enters the Classroom. [Ph.D. Thesis, University of Rhode Island]."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Al-Alshaqi, M., Rawat, D.B., and Liu, C. (2024). Ensemble Techniques for Robust Fake News Detection: Integrating Transformers, Natural Language Processing, and Machine Learning. Sensors, 24.","DOI":"10.3390\/s24186062"},{"key":"ref_15","unstructured":"S\u00e1ndor, A.V. (2023). The Relationship Between Self-Representation on Social Media and Affective or Anxiety Disorders in the Perspective of the COVID-19 Pandemic. [Ph.D. Thesis, E\u00f6tv\u00f6s Lor\u00e1nd University]."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Giachanou, A., Zhang, G., and Rosso, P. (2020, January 6\u20139). Multimodal Multi-image Fake News Detection. Proceedings of the 2020 IEEE 7th International Conference on Data Science and Advanced Analytics (DSAA), Sydney, Australia.","DOI":"10.1109\/DSAA49011.2020.00091"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Singhal, S., Shah, R.R., Chakraborty, T., Kumaraguru, P., and Satoh, S. (2019, January 11\u201313). SpotFake: A Multi-modal Framework for Fake News Detection. Proceedings of the 2019 IEEE Fifth International Conference on Multimedia Big Data (BigMM), Singapore.","DOI":"10.1109\/BigMM.2019.00-44"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Duc Tuan, N.M., and Quang Nhat Minh, P. (2021, January 2\u20134). Multimodal Fusion with BERT and Attention Mechanism for Fake News Detection. Proceedings of the 2021 RIVF International Conference on Computing and Communication Technologies (RIVF), Hanoi, Vietnam.","DOI":"10.1109\/RIVF51545.2021.9642125"},{"key":"ref_19","first-page":"102642","article-title":"Opinion Paper: \u201cSo what if ChatGPT wrote it?\u201d Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy","volume":"71","author":"Dwivedi","year":"2023","journal-title":"Int. J. Inf. Manag."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1007\/s10844-022-00764-y","article-title":"An image and text-based multimodal model for detecting fake news in OSN\u2019s","volume":"61","author":"Uppada","year":"2023","journal-title":"J. Intell. Inf. Syst."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zhang, T., Wang, D., Chen, H., Zeng, Z., Guo, W., Miao, C., and Cui, L. (2020, January 19\u201324). BDANN: BERT-Based Domain Adaptation Neural Network for Multi-Modal Fake News Detection. Proceedings of the 2020 International Joint Conference on Neural Networks (IJCNN), Glasgow, UK.","DOI":"10.1109\/IJCNN48605.2020.9206973"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Segura-Bedmar, I., and Alonso-Bartolome, S. (2022). Multimodal Fake News Detection. Information, 13.","DOI":"10.3390\/info13060284"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"5587","DOI":"10.1007\/s11042-021-11782-3","article-title":"CB-Fake: A multimodal deep learning framework for automatic fake news detection using capsule neural network and BERT","volume":"81","author":"Palani","year":"2022","journal-title":"Multimed. Tools Appl."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2017","DOI":"10.1162\/jocn_a_01544","article-title":"Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future","volume":"33","author":"Lindsay","year":"2020","journal-title":"J. Cogn. Neurosci."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"110125","DOI":"10.1016\/j.asoc.2023.110125","article-title":"Multimodal fake news detection through data augmentation-based contrastive learning","volume":"136","author":"Hua","year":"2023","journal-title":"Appl. Soft Comput."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"102610","DOI":"10.1016\/j.ipm.2021.102610","article-title":"Detecting fake news by exploring the consistency of multimodal data","volume":"58","author":"Xue","year":"2021","journal-title":"Inf. Process. Manag."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Sojka, P., Kope\u010dek, I., Pala, K., and Hor\u00e1k, A. (2020). Multimodal Fake News Detection with Textual, Visual and Semantic Information. Proceedings of the Text, Speech, and Dialogue, Springer.","DOI":"10.1007\/978-3-030-58323-1"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Jaiswal, R., Singh, U.P., and Singh, K.P. (2021, January 11\u201313). Fake News Detection Using BERT-VGG19 Multimodal Variational Autoencoder. Proceedings of the 2021 IEEE 8th Uttar Pradesh Section International Conference on Electrical, Electronics and Computer Engineering (UPCON), Dehradun, India.","DOI":"10.1109\/UPCON52273.2021.9667614"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"132363","DOI":"10.1109\/ACCESS.2021.3114093","article-title":"Multi-Level Multi-Modal Cross-Attention Network for Fake News Detection","volume":"9","author":"Ying","year":"2021","journal-title":"IEEE Access"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"2391","DOI":"10.1007\/s00530-022-00966-y","article-title":"Combating multimodal fake news on social media: Methods, datasets, and future perspective","volume":"28","author":"Hangloo","year":"2022","journal-title":"Multimed. Syst."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1007\/s13278-023-01065-0","article-title":"FNR: A similarity and transformer-based approach to detect multi-modal fake news in social media","volume":"13","author":"Ghorbanpour","year":"2023","journal-title":"Soc. Netw. Anal. Min."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"13799","DOI":"10.1007\/s11042-022-12290-8","article-title":"An effective strategy for multi-modal fake news detection","volume":"81","author":"Peng","year":"2022","journal-title":"Multimed. Tools Appl."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Qian, S., Wang, J., Hu, J., Fang, Q., and Xu, C. (2021, January 11\u201315). Hierarchical Multi-modal Contextual Attention Network for Fake News Detection. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR \u201921, New York, NY, USA.","DOI":"10.1145\/3404835.3462871"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"14699","DOI":"10.3934\/mbe.2023657","article-title":"Multi-modal transformer for fake news detection","volume":"20","author":"Yang","year":"2023","journal-title":"Math. Biosci. Eng."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Qi, P., Cao, J., Li, X., Liu, H., Sheng, Q., Mi, X., He, Q., Lv, Y., Guo, C., and Yu, Y. (2021, January 20\u201324). Improving Fake News Detection by Using an Entity-enhanced Framework to Fuse Diverse Multimodal Clues. Proceedings of the 29th ACM International Conference on Multimedia, MM \u201921, New York, NY, USA.","DOI":"10.1145\/3474085.3481548"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1007\/s13278-023-01104-w","article-title":"Multimodal fake news detection on social media: A survey of deep learning techniques","volume":"13","author":"Comito","year":"2023","journal-title":"Soc. Netw. Anal. Min."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"46","DOI":"10.1186\/s40537-023-00727-2","article-title":"A survey on deep learning tools dealing with data scarcity: Definitions, challenges, solutions, tips, and applications","volume":"10","author":"Alzubaidi","year":"2023","journal-title":"J. Big Data"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1145\/945546.945549","article-title":"Challenges in information retrieval and language modeling: Report of a workshop held at the center for intelligent information retrieval, University of Massachusetts Amherst, September 2002","volume":"37","author":"Allan","year":"2003","journal-title":"ACM SIGIR Forum"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Mart\u00ednez Hern\u00e1ndez, L.A., Sandoval Orozco, A.L., and Garc\u00eda Villalba, L.J. (2023). Analysis of Digital Information in Storage Devices Using Supervised and Unsupervised Natural Language Processing Techniques. Future Internet, 15.","DOI":"10.3390\/fi15050155"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"13521","DOI":"10.1007\/s10462-023-10466-8","article-title":"Deep learning modelling techniques: Current progress, applications, advantages, and challenges","volume":"56","author":"Ahmed","year":"2023","journal-title":"Artif. Intell. Rev."},{"key":"ref_41","first-page":"102168","article-title":"Setting the future of digital and social media marketing research: Perspectives and research propositions","volume":"59","author":"Dwivedi","year":"2021","journal-title":"Int. J. Inf. Manag."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"5173","DOI":"10.1109\/TASE.2023.3309629","article-title":"DLS-GAN: Generative Adversarial Nets for Defect Location Sensitive Data Augmentation","volume":"21","author":"Li","year":"2024","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"2347","DOI":"10.1109\/COMST.2015.2444095","article-title":"Internet of Things: A Survey on Enabling Technologies, Protocols, and Applications","volume":"17","author":"Guizani","year":"2015","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_44","unstructured":"Kaggle (2024, April 19). Fake-and-Real-News-Dataset. Available online: https:\/\/www.kaggle.com\/datasets\/clmentbisaillon\/fake-and-real-news-dataset."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Callard, F., and Fitzgerald, D. (2015). Rethinking Interdisciplinarity across the Social Sciences and Neurosciences, Palgrave Macmillan London.","DOI":"10.1057\/9781137407962"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"485","DOI":"10.1111\/1748-8583.12400","article-title":"Prediction in HRM research\u2013A gap between rhetoric and reality","volume":"32","author":"Sarstedt","year":"2022","journal-title":"Hum. Resour. Manag. J."},{"key":"ref_47","unstructured":"Roberge, J., and Lebrun, T. (2023). KI-Realit\u00e4ten, Transcript Verlag."},{"key":"ref_48","first-page":"101994","article-title":"Artificial Intelligence (AI): Multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy","volume":"57","author":"Dwivedi","year":"2021","journal-title":"Int. J. Inf. Manag."},{"key":"ref_49","unstructured":"Kaggle (2022, December 07). Fake News Detection Datasets. Available online: https:\/\/www.kaggle.com\/datasets\/emineyetm\/fake-news-detection-datasets."},{"key":"ref_50","unstructured":"Kaggle (2024, February 19). Welfake Dataset for Fake News. Available online: https:\/\/www.kaggle.com\/datasets\/syedsubahani\/wel-fake-news-dataset."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"3376","DOI":"10.1109\/TCSS.2023.3322303","article-title":"The Largest Social Media Ground-Truth Dataset for Real\/Fake Content: TruthSeeker","volume":"11","author":"Dadkhah","year":"2024","journal-title":"IEEE Trans. Comput. Soc. Syst."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1109\/TAFFC.2017.2761757","article-title":"Approaches to Automated Detection of Cyberbullying: A Survey","volume":"11","author":"Salawu","year":"2020","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1007\/s41095-022-0271-y","article-title":"Attention mechanisms in computer vision: A survey","volume":"8","author":"Guo","year":"2022","journal-title":"Comput. Vis. Media"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"12012","DOI":"10.1109\/TKDE.2021.3118815","article-title":"A Comprehensive Survey on Graph Anomaly Detection With Deep Learning","volume":"35","author":"Ma","year":"2023","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Cao, S., and Wang, L. (2021, January 7\u201311). CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Punta Cana, Dominican Republic and Online.","DOI":"10.18653\/v1\/2021.emnlp-main.532"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"1040","DOI":"10.1609\/icwsm.v16i1.19356","article-title":"Overcoming Language Disparity in Online Content Classification with Multimodal Learning","volume":"16","author":"Verma","year":"2022","journal-title":"Proc. Int. Aaai Conf. Web Soc. Media"},{"key":"ref_57","unstructured":"Ferri Borred\u00e0, P. (2024). Deeep Continual Multimodal Multitask Modells for Out-of-Hospital Emergency Medical Call Incidents Triage Support in the Presence of Dataset Shifts. [Ph.D. Thesis, Universitat Polit\u00e8cnica de Val\u00e8ncia]."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"2356","DOI":"10.1109\/COMST.2023.3309864","article-title":"Intrusion Detection in the Automotive Domain: A Comprehensive Review","volume":"25","author":"Lampe","year":"2023","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1177\/0269881119889296","article-title":"Evidence-based guidelines for the pharmacological treatment of schizophrenia: Updated recommendations from the British Association for Psychopharmacology","volume":"34","author":"Barnes","year":"2019","journal-title":"J. Psychopharmacol."},{"key":"ref_60","doi-asserted-by":"crossref","unstructured":"Munger, K. (2024). The YouTube Apparatus, Cambridge University Press.","DOI":"10.1017\/9781009359795"},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"103219","DOI":"10.1016\/j.ipm.2022.103219","article-title":"The state of human-centered NLP technology for fact-checking","volume":"60","author":"Das","year":"2023","journal-title":"Inf. Process. Manag."},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Ras, G., Xie, N., van Gerven, M., and Doran, D. (2022). Explainable Deep Learning: A Field Guide for the Uninitiated. J. Artif. Intell. Res., 73.","DOI":"10.1613\/jair.1.13200"},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"2223","DOI":"10.1093\/rfs\/hhaa009","article-title":"Empirical Asset Pricing via Machine Learning. Technical Report 25398, National Bureau of Economic Research","volume":"33","author":"Gu","year":"2020","journal-title":"Rev. Financ. Stud."},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/j.inffus.2019.12.012","article-title":"Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI","volume":"58","author":"Bennetot","year":"2020","journal-title":"Inf. Fusion"}],"container-title":["Computers"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-431X\/14\/6\/237\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:52:48Z","timestamp":1760032368000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-431X\/14\/6\/237"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,16]]},"references-count":64,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2025,6]]}},"alternative-id":["computers14060237"],"URL":"https:\/\/doi.org\/10.3390\/computers14060237","relation":{},"ISSN":["2073-431X"],"issn-type":[{"value":"2073-431X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,16]]}}}