{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T01:24:41Z","timestamp":1760059481601,"version":"build-2065373602"},"reference-count":52,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T00:00:00Z","timestamp":1750118400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62403412","23KJB520040"],"award-info":[{"award-number":["62403412","23KJB520040"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Natural Science Foundation of the Higher Education Institutions of Jiangsu Province of China","award":["62403412","23KJB520040"],"award-info":[{"award-number":["62403412","23KJB520040"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>Large pre-trained models (PLMs) have provided tremendous opportunities and potentialities for multimodal fake news detection. However, existing multimodal fake news detection methods never manipulate the token-wise hierarchical semantics of news yielded from PLMs and extremely rely on contrastive learning but ignore the symmetry between text and image in terms of the abstract level. This paper proposes a novel multimodal fake news detection method that helps to balance the understanding between text and image via (1) designing a global-token across-attention mechanism to capture the correlations between global text and tokenwise image representations (or tokenwise text and global image representations) obtained from BERT and ViT; (2) proposing a QK-sharing strategy within cross-attention to enforce model symmetry that reduces information redundancy and accelerates fusion without sacrificing representational power; (3) deploying a semantic augmentation module that systematically extracts token-wise multilayered text semantics from stacked BERT blocks via CNN and Bi-LSTM layers, thereby rebalancing abstract-level disparities by symmetrically enriching shallow and deep textual signals. We also prove the effectiveness of our approach by comparing it with four state-of-the-art baselines. All the comparisons were conducted using three widely adopted multimodal fake news datasets. The results show that our approach outperforms the benchmarks by 0.8% in accuracy and 2.2% in F1-score on average across the three datasets, which demonstrates a symmetric, token-centric fusion of fine-grained semantic fusion, thereby driving more robust fake news detection.<\/jats:p>","DOI":"10.3390\/sym17060961","type":"journal-article","created":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T05:52:47Z","timestamp":1750139567000},"page":"961","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["SAFE-GTA: Semantic Augmentation-Based Multimodal Fake News Detection via Global-Token Attention"],"prefix":"10.3390","volume":"17","author":[{"given":"Like","family":"Zhang","sequence":"first","affiliation":[{"name":"The Department of Information Engineering, Yangzhou University, Yangzhou 225012, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5051-5318","authenticated-orcid":false,"given":"Chaowei","family":"Zhang","sequence":"additional","affiliation":[{"name":"The Department of Information Engineering, Yangzhou University, Yangzhou 225012, China"}]},{"given":"Zewei","family":"Zhang","sequence":"additional","affiliation":[{"name":"The Department of Computer Science and Software Engineering, Auburn University, Auburn, AL 36849, USA"}]},{"given":"Yuchao","family":"Huang","sequence":"additional","affiliation":[{"name":"Future Design Laboratory, Zhejiang University, Hangzhou 310027, China"}]}],"member":"1968","published-online":{"date-parts":[[2025,6,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"374","DOI":"10.1126\/science.aau2706","article-title":"Fake news on Twitter during the 2016 US presidential election","volume":"363","author":"Grinberg","year":"2019","journal-title":"Science"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Patwa, P., Sharma, S., Pykl, S., Guptha, V., Kumari, G., Akhtar, M.S., Ekbal, A., Das, A., and Chakraborty, T. (2021). Fighting an infodemic: Covid-19 fake news dataset. Proceedings of the Combating Online Hostile Posts in Regional Languages During Emergency Situation: First International Workshop, CONSTRAINT 2021, Collocated with AAAI 2021, Virtual Event, 8 February 2021. Revised Selected Papers 1, Springer.","DOI":"10.1007\/978-3-030-73696-5_3"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1007\/s00354-023-00211-8","article-title":"A systematic literature review and future perspectives for handling big data analytics in COVID-19 diagnosis","volume":"41","author":"Tenali","year":"2023","journal-title":"New Gener. Comput."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1094","DOI":"10.1126\/science.aao2998","article-title":"The science of fake news","volume":"359","author":"Lazer","year":"2018","journal-title":"Science"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Alonso, M.A., Vilares, D., G\u00f3mez-Rodr\u00edguez, C., and Vilares, J. (2021). Sentiment analysis for fake news detection. Electronics, 10.","DOI":"10.3390\/electronics10111348"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1089\/big.2020.0062","article-title":"Fakenewsnet: A data repository with news content, social context, and spatiotemporal information for studying fake news on social media","volume":"8","author":"Shu","year":"2020","journal-title":"Big data"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Shu, K., Zhou, X., Wang, S., Zafarani, R., and Liu, H. (2019, January 27\u201330). The role of user profiles for fake news detection. Proceedings of the 2019 IEEE\/ACM International Conference on Advances in Social Networks Analysis and Mining, Vancouver, BC, Canada.","DOI":"10.1145\/3341161.3342927"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Nguyen, V.H., Sugiyama, K., Nakov, P., and Kan, M.Y. (2020, January 19\u201323). Fang: Leveraging social context for fake news detection using graph representation. Proceedings of the 29th ACM International Conference on Information & Knowledge Management, Virtual Event.","DOI":"10.1145\/3340531.3412046"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Shu, K., Mahudeswaran, D., Wang, S., and Liu, H. (2020, January 8). Hierarchical propagation networks for fake news detection: Investigation and exploitation. Proceedings of the International AAAI Conference on Web and Social Media, Virtual.","DOI":"10.1609\/icwsm.v14i1.7329"},{"key":"ref_10","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1029","DOI":"10.1007\/s00354-021-00151-1","article-title":"A transformer-based model for evaluation of information relevance in online social-media: A case study of covid-19 media posts","volume":"40","author":"Sharma","year":"2022","journal-title":"New Gener. Comput."},{"key":"ref_12","first-page":"5485","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel","year":"2020","journal-title":"J. Mach. Learn. Res."},{"key":"ref_13","unstructured":"Tian, L., Zhang, X., Wang, Y., and Liu, H. (2020). Early detection of rumours on twitter via stance transfer learning. Proceedings of the Advances in Information Retrieval: 42nd European Conference on IR Research, ECIR 2020, Lisbon, Portugal, 14\u201317 April 2020, Proceedings, Part I 42, Springer."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Lin, H., Yi, P., Ma, J., Jiang, H., Luo, Z., Shi, S., and Liu, R. (2023, January 7\u201314). Zero-shot rumor detection with propagation structure via prompt learning. Proceedings of the AAAI Conference on Artificial Intelligence, Washington, DC, USA.","DOI":"10.1609\/aaai.v37i4.25651"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"11765","DOI":"10.1007\/s11042-020-10183-2","article-title":"FakeBERT: Fake news detection in social media with a BERT-based deep learning approach","volume":"80","author":"Kaliyar","year":"2021","journal-title":"Multimed. Tools Appl."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Zhou, Y., Yang, Y., Ying, Q., Qian, Z., and Zhang, X. (2023, January 10\u201314). Multimodal fake news detection via clip-guided learning. Proceedings of the 2023 IEEE International Conference on Multimedia and Expo (ICME), Brisbane, Australia.","DOI":"10.1109\/ICME55011.2023.00480"},{"key":"ref_17","unstructured":"Yang, C., Zhu, F., Han, J., and Hu, S. (November, January 29). Invariant Meets Specific: A Scalable Harmful Memes Detection Framework. Proceedings of the 31st ACM International Conference on Multimedia, Ottawa, ON, Canada."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"180","DOI":"10.1177\/0002764219878224","article-title":"\u201cFake news\u201d is not simply false information: A concept explication and taxonomy of online content","volume":"65","author":"Molina","year":"2021","journal-title":"Am. Behav. Sci."},{"key":"ref_19","unstructured":"Li, J., Selvaraju, R., Gotmare, A., Joty, S., Xiong, C., and Hoi, S.C.H. (2021, January 6\u201314). Align before fuse: Vision and language representation learning with momentum distillation. Proceedings of the Advances in Neural Information Processing Systems 34 (NeurIPS 2021), Vitrual."},{"key":"ref_20","unstructured":"Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., and Clark, J. (March, January 26). Learning transferable visual models from natural language supervision. Proceedings of the International Conference on Machine Learning, Shenzhen, China."},{"key":"ref_21","unstructured":"Li, J., Li, D., Savarese, S., and Hoi, S. (2023). Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. arXiv."},{"key":"ref_22","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017, January 4\u20139). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems 30 (NIPS 2017), Long Beach, California, USA."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"102653","DOI":"10.1016\/j.ipm.2021.102653","article-title":"When classification accuracy is not enough: Explaining news credibility assessment","volume":"58","author":"Soto","year":"2021","journal-title":"Inf. Process. Manag."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1007\/978-981-16-2937-2_9","article-title":"Fake news detection: Experiments and approaches beyond linguistic features","volume":"Volume 2","author":"Bhatt","year":"2022","journal-title":"Proceedings of the Data Management, Analytics and Innovation: Proceedings of ICDMAI 2021"},{"key":"ref_25","first-page":"1","article-title":"A unified perspective for disinformation detection and truth discovery in social sensing: A survey","volume":"55","author":"Xu","year":"2021","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1016\/j.neucom.2023.02.005","article-title":"Content Based Fake News Detection with machine and deep learning: A systematic review","volume":"530","author":"Capuano","year":"2023","journal-title":"Neurocomputing"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"103116","DOI":"10.1016\/j.ipm.2022.103116","article-title":"Evaluating the generalisability of neural rumour verification models","volume":"60","author":"Kochkina","year":"2023","journal-title":"Inf. Process. Manag."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"2913","DOI":"10.1007\/s11042-022-12668-8","article-title":"Evaluating the effectiveness of publishers\u2019 features in fake news detection on social media","volume":"82","author":"Jarrahi","year":"2023","journal-title":"Multimed. Tools Appl."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"335","DOI":"10.1007\/s41060-021-00302-z","article-title":"Fake news detection based on news content and social contexts: A transformer-based approach","volume":"13","author":"Raza","year":"2022","journal-title":"Int. J. Data Sci. Anal."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"103206","DOI":"10.1016\/j.ipm.2022.103206","article-title":"Preventing profiling for ethical fake news detection","volume":"60","author":"Allein","year":"2023","journal-title":"Inf. Process. Manag."},{"key":"ref_31","unstructured":"Hamdi, T., Slimi, H., Bounhas, I., and Slimani, Y. (2020). A hybrid approach for fake news detection in twitter based on user features and graph embedding. Proceedings of the Distributed Computing and Internet Technology: 16th International Conference, ICDCIT 2020, Bhubaneswar, India, 9\u201312 January 2020, Proceedings 16, Springer."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"102872","DOI":"10.1016\/j.ipm.2022.102872","article-title":"Predictive intelligence in harmful news identification by BERT-based ensemble learning model with text sentiment analysis","volume":"59","author":"Lin","year":"2022","journal-title":"Inf. Process. Manag."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"103354","DOI":"10.1016\/j.ipm.2023.103354","article-title":"Dual emotion based fake news detection: A deep attention-weight update approach","volume":"60","author":"Luvembe","year":"2023","journal-title":"Inf. Process. Manag."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"5328","DOI":"10.1109\/TKDE.2023.3332787","article-title":"Prompt-Learning for Short Text Classification","volume":"36","author":"Zhu","year":"2024","journal-title":"IEEE Trans. Knowl. Data Eng. (TKDE)"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Qian, F., Gong, C., Sharma, K., and Liu, Y. (2018, January 13\u201319). Neural User Response Generator: Fake News Detection with Collective User Intelligence. Proceedings of the 27th International Joint Conference on Artificial Intelligence, Stockholm, Sweden.","DOI":"10.24963\/ijcai.2018\/533"},{"key":"ref_36","first-page":"7178","article-title":"Memory-guided multi-view multi-domain fake news detection","volume":"35","author":"Zhu","year":"2022","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Glazkova, A., Glazkov, M., and Trifonov, T. (2021). g2tmn at Constraint@AAAI2021: Exploiting CT-BERT and Ensembling Learning for COVID-19 Fake News Detection. Combating Online Hostile Posts in Regional Languages During Emergency Situation, Springer International Publishing.","DOI":"10.1007\/978-3-030-73696-5_12"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"110642","DOI":"10.1016\/j.knosys.2023.110642","article-title":"Towards COVID-19 fake news detection using transformer-based models","volume":"274","author":"Alghamdi","year":"2023","journal-title":"Knowl.-Based Syst."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Qi, P., Cao, J., Yang, T., Guo, J., and Li, J. (2019, January 8\u201311). Exploiting multi-domain visual information for fake news detection. Proceedings of the 2019 IEEE International Conference on Data Mining (ICDM), Beijing, China.","DOI":"10.1109\/ICDM.2019.00062"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"598","DOI":"10.1109\/TMM.2016.2617078","article-title":"Novel visual and statistical image features for microblogs news verification","volume":"19","author":"Jin","year":"2016","journal-title":"IEEE Trans. Multimed."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"102610","DOI":"10.1016\/j.ipm.2021.102610","article-title":"Detecting fake news by exploring the consistency of multimodal data","volume":"58","author":"Xue","year":"2021","journal-title":"Inf. Process. Manag."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"103120","DOI":"10.1016\/j.ipm.2022.103120","article-title":"Multimodal fake news detection via progressive fusion networks","volume":"60","author":"Jing","year":"2023","journal-title":"Inf. Process. Manag."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1007\/s13278-023-01104-w","article-title":"Multimodal fake news detection on social media: A survey of deep learning techniques","volume":"13","author":"Comito","year":"2023","journal-title":"Soc. Netw. Anal. Min."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Qi, P., Bu, Y., Cao, J., Ji, W., Shui, R., Xiao, J., Wang, D., and Chua, T.S. (2023, January 7\u201314). Fakesv: A multimodal benchmark with rich social context for fake news detection on short video platforms. Proceedings of the AAAI Conference on Artificial Intelligence, Washington, DC, USA.","DOI":"10.1609\/aaai.v37i12.26689"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1016\/j.patcog.2019.01.006","article-title":"Wider or deeper: Revisiting the resnet model for visual recognition","volume":"90","author":"Wu","year":"2019","journal-title":"Pattern Recognit."},{"key":"ref_46","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Wang, Y., Ma, F., Jin, Z., Yuan, Y., Xun, G., Jha, K., Su, L., and Gao, J. (2018, January 19\u201323). Eann: Event adversarial neural networks for multi-modal fake news detection. Proceedings of the 24th ACM Sigkdd International Conference on Knowledge Discovery & Data Mining, Lodon, UK.","DOI":"10.1145\/3219819.3219903"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Qian, S., Wang, J., Hu, J., Fang, Q., and Xu, C. (2021, January 11\u201315). Hierarchical multi-modal contextual attention network for fake news detection. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual.","DOI":"10.1145\/3404835.3462871"},{"key":"ref_49","unstructured":"Wang, L., Zhang, C., Xu, H., Xu, Y., Xu, X., and Wang, S. (November, January 29). Cross-modal contrastive learning for multimodal fake news detection. Proceedings of the 31st ACM International Conference on Multimedia, Ottawa, ON, Canada."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Chen, Y., Li, D., Zhang, P., Sui, J., Lv, Q., Tun, L., and Shang, L. (2022, January 25\u201329). Cross-modal ambiguity learning for multimodal fake news detection. Proceedings of the ACM Web Conference 2022, Virtual Event, Lyon, France.","DOI":"10.1145\/3485447.3511968"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Singhal, S., Shah, R.R., Chakraborty, T., Kumaraguru, P., and Satoh, S. (2019, January 11\u201313). Spotfake: A multi-modal framework for fake news detection. Proceedings of the 2019 IEEE Fifth International Conference on Multimedia Big Data (BigMM), Singapore.","DOI":"10.1109\/BigMM.2019.00-44"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Singhal, S., Kabra, A., Sharma, M., Shah, R.R., Chakraborty, T., and Kumaraguru, P. (2020, January 7\u201312). Spotfake+: A multimodal framework for fake news detection via transfer learning (student abstract). Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i10.7230"}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/17\/6\/961\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:53:21Z","timestamp":1760032401000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/17\/6\/961"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,17]]},"references-count":52,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2025,6]]}},"alternative-id":["sym17060961"],"URL":"https:\/\/doi.org\/10.3390\/sym17060961","relation":{},"ISSN":["2073-8994"],"issn-type":[{"type":"electronic","value":"2073-8994"}],"subject":[],"published":{"date-parts":[[2025,6,17]]}}}