{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T01:41:50Z","timestamp":1760060510732,"version":"build-2065373602"},"reference-count":33,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2025,9,4]],"date-time":"2025-09-04T00:00:00Z","timestamp":1756944000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Slovak Research and Development Agency","award":["APVV-22-0414"],"award-info":[{"award-number":["APVV-22-0414"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["BDCC"],"abstract":"<jats:p>The main objective of the paper is to verify whether the integration of attention mechanisms could improve the effectiveness of online fake news detection models. The models were training using selected deep learning methods, which were suitable for text processing, such as CNN (Convolutional Neural Network), LSTM (Lon-short Term Memory), BiLSTM (Bidirectional LSTM), GRU (Gated Recurrent Unit), and transformer. The novelty of the paper lies in the addition of attention mechanisms to each of those models, and comparison of their performance across both datasets, LIAR and WELFake. Afterwards, an analysis of resulting changes in terms of the detection performance was carried out. The paper also describes the issue of toxicity in the online space and how it affects society, the toxicity sources, and methods to tackle it. Furthermore, the article provides a description of individual deep learning methods and the principles of attention mechanism. Finally, it was shown that the attention mechanism can increase the accuracy of basic models for fake news detection; however, the differences are insignificant in the case of the LIAR dataset. The reason for this can be found in the dataset itself. In contrast, the addition of attention mechanism to models on the WELFake dataset showed a significant improvement of results, where the average accuracy was 0.967 and average F1-rate was 0.968. These results were better than the results of experiments with the simple transformer. Comparison of the results showed that it makes sense to enrich the basic neural network models with the attention mechanisms, especially with the multi-head attention mechanism. The key finding is that attention mechanisms can enhance fake news detection performance when applied to high-quality, well-balanced datasets.<\/jats:p>","DOI":"10.3390\/bdcc9090230","type":"journal-article","created":{"date-parts":[[2025,9,4]],"date-time":"2025-09-04T10:55:37Z","timestamp":1756983337000},"page":"230","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Analysis of the Effect of Attention Mechanism on the Accuracy of Deep Learning Models for Fake News Detection"],"prefix":"10.3390","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7741-4039","authenticated-orcid":false,"given":"Krist\u00edna","family":"Machov\u00e1","sequence":"first","affiliation":[{"name":"Department of Cybernetics and Artificial Intelligence, Faculty of Electrical Engineering and Informatics, Technical University of Kosice, Letna 9, 042 00 Kosice, Slovakia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7612-9204","authenticated-orcid":false,"given":"Mari\u00e1n","family":"Mach","sequence":"additional","affiliation":[{"name":"Department of Cybernetics and Artificial Intelligence, Faculty of Electrical Engineering and Informatics, Technical University of Kosice, Letna 9, 042 00 Kosice, Slovakia"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-0185-5667","authenticated-orcid":false,"given":"Viliam","family":"Balara","sequence":"additional","affiliation":[{"name":"Department of Cybernetics and Artificial Intelligence, Faculty of Electrical Engineering and Informatics, Technical University of Kosice, Letna 9, 042 00 Kosice, Slovakia"}]}],"member":"1968","published-online":{"date-parts":[[2025,9,4]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1094","DOI":"10.1126\/science.aao2998","article-title":"The science of fake news: Addressing fake news requires a multidisciplinary effort","volume":"359","author":"Lazer","year":"2018","journal-title":"Science"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"102025","DOI":"10.1016\/j.ipm.2019.03.004","article-title":"An overview of online fake news: Characterization, detection, and discussion","volume":"57","author":"Zhang","year":"2020","journal-title":"Inf. Process Manag."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"388","DOI":"10.1016\/j.tics.2021.02.007","article-title":"The Psychology of Fake News","volume":"25","author":"Pennycook","year":"2021","journal-title":"Trends Cogn. Sci."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Siino, M., Di Nuovo, E., Tinnirello, I., and La Cascia, M. (2022). Fake News Spreaders Detection: Sometimes Attention Is Not All You Need. Information, 13.","DOI":"10.3390\/info13090426"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"107600","DOI":"10.1016\/j.asoc.2021.107600","article-title":"Attention-based C-BiLSTM for fake news detection","volume":"110","author":"Trueman","year":"2021","journal-title":"Appl. Soft Comput."},{"key":"ref_6","first-page":"25","article-title":"Attention-Based Deep Learning Models for Detection of Fake News in Social Networks","volume":"15","author":"Ramya","year":"2021","journal-title":"Int. J. Cogn. Inform. Nat. Intell."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1145\/3137597.3137600","article-title":"Fake news detection on social media: A data mining perspective","volume":"19","author":"Shu","year":"2017","journal-title":"ACM SIGKDD Explor. Newsl."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"38370","DOI":"10.1109\/ACCESS.2022.3165226","article-title":"Conspiracy or Not? A Deep Learning Approach to Spot It on Twitter","volume":"10","author":"Galende","year":"2022","journal-title":"IEEE Access"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"11765","DOI":"10.1007\/s11042-020-10183-2","article-title":"FakeBERT: Fake news detection in social media with a BERT-based deep learning approach","volume":"80","author":"Kaliyar","year":"2021","journal-title":"Multimed. Tools Appl."},{"key":"ref_10","first-page":"141","article-title":"ECOL: Early Detection of COVID Lies Using Content, Prior Knowledge and Source Information","volume":"1402","author":"Baris","year":"2021","journal-title":"Commun. Comput. Inf. Sci."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Segura-Bedmar, I., and Alonso-Bartolome, S. (2022). Multimodal Fake News Detection. Information, 13.","DOI":"10.3390\/info13060284"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Yan, Y., Fu, H., and Wu, F. (2024). Multimodal Social Media Fake News Detection Based on 1D-CCNet Attention Mecha-nism. Electronics, 13.","DOI":"10.3390\/electronics13183700"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"129471","DOI":"10.1109\/ACCESS.2021.3112806","article-title":"OPCNN-FAKE: Optimized convolutional neural network for fake news detection","volume":"9","author":"Saleh","year":"2021","journal-title":"IEEE Access"},{"key":"ref_14","first-page":"153","article-title":"Evaluating Deep Learning Approaches for Covid19 Fake News Detection","volume":"1402","author":"Wani","year":"2021","journal-title":"Commun. Comput. Inf. Sci."},{"key":"ref_15","unstructured":"(2025, May 07). LLM for Misinformation Research. Available online: https:\/\/github.com\/ICTMCG\/LLM-for-misinformation-research."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Alghamdi, J., Lin, Y., and Luo, S. (2023). Does Context Matter? Effective Deep Learning Approaches to Curb Fake News Dis-semination on Social Media. Appl. Sci., 13.","DOI":"10.3390\/app13053345"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Mayopu, R.G., Wang, Y.-Y., and Chen, L.-S. (2023). Analyzing Online Fake News Using Latent Semantic Analysis: Case of USA Election Campaign. Big Data Cogn. Comput., 7.","DOI":"10.3390\/bdcc7020081"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Shen, Y., Liu, Q., Guo, N., Yuan, J., and Yang, Y. (2023). Fake News Detection on Social Networks: A Survey. Appl. Sci., 13.","DOI":"10.3390\/app132111877"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Wang, W., and Gang, J. (2018, January 6\u20138). Application of convolutional neural network in natural language processing. Proceedings of the 2018 International Conference on Information Systems and Computer Aided Education (ICISCAE), Changchun, China.","DOI":"10.1109\/ICISCAE.2018.8666928"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Kumaraswamy, B. (2021). Neural networks for data classification. Artificial Intelligence in Data Mining, Elsevier.","DOI":"10.1016\/B978-0-12-820601-0.00011-2"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"132306","DOI":"10.1016\/j.physd.2019.132306","article-title":"Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) network","volume":"404","author":"Sherstinsky","year":"2020","journal-title":"Phys. D"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"73992","DOI":"10.1109\/ACCESS.2020.2988550","article-title":"Sentiment Classification Using a Single-Layered BiLSTM Model","volume":"8","author":"Hameed","year":"2020","journal-title":"IEEE Access"},{"key":"ref_23","first-page":"1597","article-title":"Gate-variants of Gated Recurrent Unit (GRU) neural networks","volume":"17","author":"Dey","year":"2017","journal-title":"Midwest Symp. Circuits Syst."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1007\/s41095-022-0271-y","article-title":"Attention mechanisms in computer vision: A survey","volume":"8","author":"Guo","year":"2022","journal-title":"Comput. Vis. Media"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1016\/j.neucom.2021.03.091","article-title":"A review on the attention mechanism of deep learning","volume":"452","author":"Niu","year":"2021","journal-title":"Neurocomputing"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"2283","DOI":"10.1007\/s11063-021-10730-4","article-title":"Effects of Similarity Score Functions in Attention Mechanisms on the Performance of Neural Question Answering Systems","volume":"54","author":"Shen","year":"2022","journal-title":"Neural Process Lett."},{"key":"ref_27","first-page":"5998","article-title":"Attention is All you Need","volume":"30","author":"Vaswani","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_28","unstructured":"Wang, W.Y. (August, January 30). \u201cLiar, Liar Pants on Fire\u201d: A New Benchmark Dataset for Fake News Detection. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Vancouver, BC, Canada."},{"key":"ref_29","unstructured":"(2025, May 07). LIAR Dataset, Hugging Face. Available online: https:\/\/huggingface.co\/datasets\/ucsbnlp\/liar."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"881","DOI":"10.1109\/TCSS.2021.3068519","article-title":"WELFake: Word Embedding Over Linguistic Features for Fake News Detection","volume":"8","author":"Verma","year":"2021","journal-title":"IEEE Trans. Comput. Soc. Syst."},{"key":"ref_31","unstructured":"(2025, May 07). WelFake dataset, Hugging Face. Available online: https:\/\/huggingface.co\/datasets\/davanstrien\/WELFake."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Hashem-Sweidan, A., El-Bendary, N., Taie, S.A., Idrees, A.M., and Elhariri, E. (2025). Explainable Deep Learning for COVID-19 Vaccine Sentiment in Arabic Tweets Using Multi-Self-Attention BiLSTM with XLNet. Big Data Cogn. Comput., 9.","DOI":"10.3390\/bdcc9020037"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Tsoulos, I.G., and Charilogis, V. (2025). Training Neural Networks with a Procedure Guided by BNF Grammars. Big Data Cogn. Comput., 9.","DOI":"10.3390\/bdcc9010005"}],"container-title":["Big Data and Cognitive Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-2289\/9\/9\/230\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:39:19Z","timestamp":1760035159000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-2289\/9\/9\/230"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,4]]},"references-count":33,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2025,9]]}},"alternative-id":["bdcc9090230"],"URL":"https:\/\/doi.org\/10.3390\/bdcc9090230","relation":{},"ISSN":["2504-2289"],"issn-type":[{"type":"electronic","value":"2504-2289"}],"subject":[],"published":{"date-parts":[[2025,9,4]]}}}