{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T14:42:20Z","timestamp":1774622540923,"version":"3.50.1"},"reference-count":29,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2025,10,30]],"date-time":"2025-10-30T00:00:00Z","timestamp":1761782400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Systems"],"abstract":"<jats:p>This paper proposes a hybrid deep learning model for robust and interpretable sentiment classification of Twitter data. The model integrates Bidirectional Encoder Representations from Transformers (BERT)-based contextual embeddings, a Bidirectional Long Short-Term Memory (BiLSTM) network, and a custom attention mechanism to classify tweets into four sentiment categories: Positive, Negative, Neutral, and Irrelevant. Addressing the challenges of noisy and multilingual social media content, the model incorporates a comprehensive preprocessing pipeline and data augmentation strategies including back-translation and synonym replacement. An ablation study demonstrates that combining BERT with BiLSTM improves the model\u2019s sensitivity to sequence dependencies, while the attention mechanism enhances both classification accuracy and interpretability. Empirical results show that the proposed model outperforms BERT-only and BERT+BiLSTM baselines, achieving F1-scores (F1) above 0.94 across all sentiment classes. Attention weight visualizations further reveal the model\u2019s ability to focus on sentiment-bearing tokens, providing transparency in decision-making. The proposed framework is well-suited for deployment in real-time sentiment monitoring systems and offers a scalable solution for multilingual and multi-class sentiment analysis in dynamic social media environments. We also include a focused characterization of the dataset via an Exploratory Data Analysis in the Methods section.<\/jats:p>","DOI":"10.3390\/systems13110964","type":"journal-article","created":{"date-parts":[[2025,10,30]],"date-time":"2025-10-30T05:08:47Z","timestamp":1761800927000},"page":"964","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["A BERT\u2013LSTM\u2013Attention Framework for Robust Multi-Class Sentiment Analysis on Twitter Data"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-8920-3604","authenticated-orcid":false,"given":"Xinyu","family":"Zhang","sequence":"first","affiliation":[{"name":"Department of Computer Science, Rochester Institute of Technology, Rochester, NY 14623, USA"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-2416-0641","authenticated-orcid":false,"given":"Yang","family":"Liu","sequence":"additional","affiliation":[{"name":"College of Arts & Sciences, University of Miami, Miami, FL 33124, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8040-9281","authenticated-orcid":false,"given":"Tianhui","family":"Zhang","sequence":"additional","affiliation":[{"name":"College of Engineering, Northeastern University, Boston, MA 02115, USA"}]},{"given":"Lingmin","family":"Hou","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Rochester Institute of Technology, Rochester, NY 14623, USA"}]},{"given":"Xianchen","family":"Liu","sequence":"additional","affiliation":[{"name":"Department of Computer Engineering, Florida International University, Miami, FL 33199, USA"}]},{"given":"Zhen","family":"Guo","sequence":"additional","affiliation":[{"name":"Department of Material Engineering, Florida International University, Miami, FL 33199, USA"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-4343-6994","authenticated-orcid":false,"given":"Aliya","family":"Mulati","sequence":"additional","affiliation":[{"name":"Department of Politics and International Relations, Florida International University, Miami, FL 33199, USA"}]}],"member":"1968","published-online":{"date-parts":[[2025,10,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Rahman, M.M., Hossain, T.R., Hasan, M.M., and Hossain, M.J. (2024). RoBERTa-BiLSTM: A Context-Aware Hybrid Model for Sentiment Analysis. arXiv.","DOI":"10.1109\/TETCI.2025.3572150"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Jahin, M.A., Shovon, M.S.H., Mridha, M.F., Islam, M.R., and Watanobe, Y. (2024). A hybrid transformer and attention based recurrent neural network for robust and interpretable sentiment analysis of tweets. Sci. Rep., 14.","DOI":"10.1038\/s41598-024-76079-5"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Chen, Y., Yuan, L., Liu, W., and Sun, M. (2018, January 22\u201326). Twitter Sentiment Analysis via Bi-sense Emoji Embedding and Attention-based LSTM. Proceedings of the 26th ACM International Conference on Multimedia, Seoul, Republic of Korea.","DOI":"10.1145\/3240508.3240533"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Barri\u00e8re, V., and Balahur, A. (2020, January 8\u201313). Improving Sentiment Analysis over Non-English Tweets using Multilingual Transformers and Automatic Translation for Data Augmentation. Proceedings of the 28th International Conference on Computational Linguistics, Barcelona, Spain.","DOI":"10.18653\/v1\/2020.coling-main.23"},{"key":"ref_5","unstructured":"Serrano, S., and Smith, N.A. (August, January 28). Is Attention Interpretable?. Proceedings of the Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Imran, M., Qadir, F., and Castillo, L. (2021). TBCOV: Two Billion Multilingual COVID-19 Tweets with Location and Sentiment Labels. arXiv.","DOI":"10.3390\/data7010008"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Nasrabadi, H.N., and Moattar, M. (2025). Sentiment Analysis of Imbalanced Dataset through Data Augmentation and Generative Annotation using DistilBERT and Low-Rank Fine-Tuning. Res. Sq. Prepr.","DOI":"10.21203\/rs.3.rs-5879286\/v1"},{"key":"ref_8","unstructured":"Rana, P. (2025, September 22). Data Augmentation Techniques for Imbalanced Datasets. Medium. Available online: https:\/\/medium.com."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"30444","DOI":"10.1109\/ACCESS.2025.3541494","article-title":"Sentiment Analysis of Twitter Data Using NLP Models: A Comprehensive Review","volume":"13","author":"Albladi","year":"2025","journal-title":"IEEE Access"},{"key":"ref_10","unstructured":"Zhong, R., Shao, S., and McKeown, K. (2019). Fine-grained Sentiment Analysis with Faithful Attention. arXiv."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Tutek, M., and \u0160najder, J. (2018). Iterative Recursive Attention Model for Interpretable Sequence Classification. arXiv.","DOI":"10.18653\/v1\/W18-5427"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Wu, Z., Nguyen, T.S., and Ong, D.C. (2020). Structured Self-Attention Weights Encode Semantics in Sentiment Analysis. arXiv.","DOI":"10.18653\/v1\/2020.blackboxnlp-1.24"},{"key":"ref_13","unstructured":"Lopardo, G., Precioso, F., and Garreau, D. (2025). Attention Meets Post-hoc Interpretability: A Mathematical Perspective. arXiv."},{"key":"ref_14","first-page":"156","article-title":"Traditional and Deep Learning Approaches for Sentiment Analysis: A Survey","volume":"5","author":"Sharma","year":"2020","journal-title":"Adv. Sci. Technol. Eng. Syst. J. ASTESJ"},{"key":"ref_15","first-page":"431","article-title":"Deep Learning for Sentiment Analysis: A Comprehensive Survey","volume":"28","author":"Zhang","year":"2016","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_16","unstructured":"Tejwani, R. (2014). Sentiment Analysis: A Survey. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1985","DOI":"10.1007\/s10845-023-02096-2","article-title":"HAR2bot: A human-centered augmented reality robot programming method with the awareness of cognitive load","volume":"35","author":"Yang","year":"2024","journal-title":"J. Intell. Manuf."},{"key":"ref_18","unstructured":"Chen, Z., Zhang, J., Ni, J., Li, X., Bian, Y., Islam, M.M., Mondal, A.M., Wei, H., and Luo, D. (2024). Generating In-Distribution Proxy Graphs for Explaining Graph Neural Networks. arXiv."},{"key":"ref_19","unstructured":"Kaggle (2025, January 05). Twitter Entity Sentiment Analysis. Available online: https:\/\/www.kaggle.com\/datasets\/jp797498e\/twitter-entity-sentiment-analysis."},{"key":"ref_20","first-page":"545","article-title":"Sentiment analysis of social media data: Business insights and consumer behavior trends in the USA","volume":"9","author":"Ghosh","year":"2025","journal-title":"Edelweiss Appl. Sci. Technol."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"907","DOI":"10.1109\/TBDATA.2025.3536934","article-title":"AugGPT: Leveraging ChatGPT for Text Data Augmentation","volume":"11","author":"Dai","year":"2025","journal-title":"IEEE Trans. Big Data"},{"key":"ref_22","unstructured":"Confusion Matrix (2025, September 22). Wikipedia. Available online: https:\/\/en.wikipedia.org\/wiki\/Confusion\/_matrix."},{"key":"ref_23","unstructured":"Long Short-Term Memory (2025, September 22). Wikipedia. Available online: https:\/\/en.wikipedia.org\/wiki\/Long\/_short-term\/_memory."},{"key":"ref_24","unstructured":"(2025, September 22). Transformer (Deep Learning Architecture). Available online: https:\/\/en.wikipedia.org\/wiki\/Transformer_(deep_learning_architecture)."},{"key":"ref_25","unstructured":"Smith, L.N. (2018). A disciplined approach to neural network hyper-parameters: Part 1\u2014Learning rate, batch size, momentum, and weight decay. arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Clark, K., Khandelwal, U., Levy, O., and Manning, C.D. (2019, January 1). What Does BERT Look at? An Analysis of BERT\u2019s Attention. Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, Florence, Italy.","DOI":"10.18653\/v1\/W19-4828"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Ribeiro, M.T., Singh, S., and Guestrin, C. (2016, January 13\u201317). Why should I trust you? Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939778"},{"key":"ref_28","first-page":"4765","article-title":"A unified approach to interpreting model predictions","volume":"30","author":"Lundberg","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_29","unstructured":"Sundararajan, M., Taly, A., and Yan, Q. (2017, January 6\u201311). Axiomatic attribution for deep networks. Proceedings of the 34th International Conference on Machine Learning (ICML), Sydney, Australia."}],"container-title":["Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2079-8954\/13\/11\/964\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,30]],"date-time":"2025-10-30T05:48:52Z","timestamp":1761803332000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2079-8954\/13\/11\/964"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,30]]},"references-count":29,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2025,11]]}},"alternative-id":["systems13110964"],"URL":"https:\/\/doi.org\/10.3390\/systems13110964","relation":{},"ISSN":["2079-8954"],"issn-type":[{"value":"2079-8954","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,30]]}}}