{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,15]],"date-time":"2026-04-15T12:13:02Z","timestamp":1776255182620,"version":"3.50.1"},"reference-count":47,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2026,4,15]],"date-time":"2026-04-15T00:00:00Z","timestamp":1776211200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,4,15]],"date-time":"2026-04-15T00:00:00Z","timestamp":1776211200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000923","name":"Australian Research Council","doi-asserted-by":"publisher","award":["DP220103717, LE220100078"],"award-info":[{"award-number":["DP220103717, LE220100078"]}],"id":[{"id":"10.13039\/501100000923","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Data Sci Anal"],"published-print":{"date-parts":[[2026,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    Large pretrained language models (PLMs) provide strong semantic signals for sentiment classification but process text as linear token streams, leaving syntactic relations implicit. This limits robustness on structure-sensitive phenomena such as negation scope, coordination, and clause-level dependencies. Graph neural networks can encode such relations explicitly but lack the broad semantic grounding of large-scale pretraining. Early- and mid-fusion hybrids address this gap but blend modalities through learned parameters, obscuring individual contributions and risking overfitting on smaller datasets. We introduce\n                    <jats:bold>RG-Hybrid<\/jats:bold>\n                    , a dual-branch architecture that combines a fine-tuned RoBERTa encoder with a graph transformer operating over dependency parses. The two branches are trained separately and merged only at the decision stage through parameter-free logit averaging. This design keeps the semantic and structural contributions fully separable, enabling direct attribution of each branch\u2019s role in the final prediction. Evaluated on four sentiment benchmarks spanning document-level and sentence-level corpora, RG-Hybrid achieves consistent and statistically significant improvements over a fine-tuned RoBERTa-only baseline, with relative error rate reductions of up to 17.66%. It outperforms transductive PLM\u2013GNN baselines despite operating fully inductively. Qualitative analysis confirms that explicit dependency structure resolves polarity ambiguities introduced by negation and concessive constructions, while also identifying failure modes on ironic text.\n                  <\/jats:p>","DOI":"10.1007\/s41060-026-01112-x","type":"journal-article","created":{"date-parts":[[2026,4,15]],"date-time":"2026-04-15T11:28:28Z","timestamp":1776252508000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["RG-Hybrid: pure late fusion of RoBERTa and a graph transformer for robust, interpretable sentiment classification"],"prefix":"10.1007","volume":"22","author":[{"given":"Khaled","family":"Alahmadi","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sultan","family":"Alharbi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xianzhi","family":"Wang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2026,4,15]]},"reference":[{"issue":"12","key":"1112_CR1","doi-asserted-by":"publisher","first-page":"7305","DOI":"10.1007\/s10115-024-02214-3","volume":"66","author":"H Bashiri","year":"2024","unstructured":"Bashiri, H., Naderi, H.: Comprehensive review and comparative analysis of transformer models in sentiment analysis. Knowl. Inf. Syst. 66(12), 7305\u20137361 (2024)","journal-title":"Knowl. Inf. Syst."},{"key":"1112_CR2","doi-asserted-by":"publisher","first-page":"2592","DOI":"10.7717\/peerj-cs.2592","volume":"11","author":"M Kumar","year":"2025","unstructured":"Kumar, M., Khan, L., Chang, H.-T.: Evolving techniques in sentiment analysis: a comprehensive review. PeerJ Comput. Sci. 11, 2592 (2025)","journal-title":"PeerJ Comput. Sci."},{"key":"1112_CR3","doi-asserted-by":"crossref","unstructured":"Zhang, W., Deng, Y., Liu, B., Pan, S., Bing, L.: Sentiment analysis in the era of large language models: a reality check. In: Findings of the Association for Computational Linguistics: NAACL 2024, pp. 3881\u20133906. (2024)","DOI":"10.18653\/v1\/2024.findings-naacl.246"},{"issue":"3","key":"1112_CR4","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3439726","volume":"54","author":"S Minaee","year":"2021","unstructured":"Minaee, S., Kalchbrenner, N., Cambria, E., Nikzad, N., Chenaghlu, M., Gao, J.: Deep learning-based text classification: a comprehensive review. ACM Comput. Surv. (CSUR) 54(3), 1\u201340 (2021)","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"1112_CR5","first-page":"34","volume":"8","author":"A Ettinger","year":"2020","unstructured":"Ettinger, A.: What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models. Transact. Assoc. Comput. Ling. 8, 34\u201348 (2020)","journal-title":"Transact. Assoc. Comput. Ling."},{"key":"1112_CR6","doi-asserted-by":"crossref","unstructured":"Ribeiro, M.T., Wu, T., Guestrin, C., Singh, S.: Beyond accuracy: Behavioral testing of NLP models with checklist. arXiv preprint arXiv:2005.04118 (2020)","DOI":"10.18653\/v1\/2020.acl-main.442"},{"key":"1112_CR7","doi-asserted-by":"crossref","unstructured":"Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1 (long and Short Papers), pp. 4171\u20134186. (2019)","DOI":"10.18653\/v1\/N19-1423"},{"key":"1112_CR8","unstructured":"Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., Stoyanov, V.: RoBERTa: a robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692 (2019)"},{"key":"1112_CR9","doi-asserted-by":"publisher","first-page":"373","DOI":"10.1016\/j.neucom.2022.10.071","volume":"518","author":"L Xu","year":"2023","unstructured":"Xu, L., Pang, X., Wu, J., Cai, M., Peng, J.: Learn from structural scope: improving aspect-level sentiment analysis with hybrid graph convolutional networks. Neurocomputing 518, 373\u2013383 (2023)","journal-title":"Neurocomputing"},{"key":"1112_CR10","doi-asserted-by":"crossref","unstructured":"Yang, J., Hu, X., Xiao, G., Shen, Y.: A survey of knowledge enhanced pre-trained language models. ACM Transactions on Asian and Low-Resource Language Information Processing (2024)","DOI":"10.1145\/3631392"},{"key":"1112_CR11","unstructured":"Shirzad, H., Velingker, A., Venkatachalam, B., Sutherland, D.J., Sinop, A.K.: Exphormer: sparse transformers for graphs. In: Proceedings of the International Conference on Machine Learning, pp. 31613\u201331632. PMLR (2023)"},{"key":"1112_CR12","doi-asserted-by":"crossref","unstructured":"Wu, L., Chen, Y., Shen, K., Guo, X., Gao, H., Li, S., Pei, J., Long, B., et al.: Graph neural networks for natural language processing: a survey. Found. Trends\u00ae Mach. Learn. 16(2), 119\u2013328 (2023)","DOI":"10.1561\/2200000096"},{"issue":"8","key":"1112_CR13","doi-asserted-by":"publisher","first-page":"190","DOI":"10.1007\/s10462-024-10808-0","volume":"57","author":"K Wang","year":"2024","unstructured":"Wang, K., Ding, Y., Han, S.C.: Graph neural networks for text classification: a survey. Artif. Intell. Rev. 57(8), 190 (2024)","journal-title":"Artif. Intell. Rev."},{"key":"1112_CR14","doi-asserted-by":"crossref","unstructured":"Li, R., Chen, H., Feng, F., Ma, Z., Wang, X., Hovy, E.: Dual graph convolutional networks for aspect-based sentiment analysis. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (vol. 1: Long Papers), pp. 6319\u20136329. (2021)","DOI":"10.18653\/v1\/2021.acl-long.494"},{"key":"1112_CR15","doi-asserted-by":"crossref","unstructured":"Tian, Y., Chen, G., Song, Y.: Aspect-based sentiment analysis with type-aware graph convolutional networks and layer ensemble. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2910\u20132922. (2021)","DOI":"10.18653\/v1\/2021.naacl-main.231"},{"issue":"6","key":"1112_CR16","doi-asserted-by":"publisher","first-page":"4893","DOI":"10.1007\/s10462-022-10265-7","volume":"56","author":"P Pham","year":"2023","unstructured":"Pham, P., Nguyen, L.T., Pedrycz, W., Vo, B.: Deep learning, graph-based text representation and classification: a survey, perspectives and challenges. Artif. Intell. Rev. 56(6), 4893\u20134927 (2023)","journal-title":"Artif. Intell. Rev."},{"key":"1112_CR17","doi-asserted-by":"crossref","unstructured":"Lin, Y., Meng, Y., Sun, X., Han, Q., Kuang, K., Li, J., Wu, F.: BERTGCN: transductive text classification by combining GNN and BERT. In: Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 1456\u20131462. (2021)","DOI":"10.18653\/v1\/2021.findings-acl.126"},{"issue":"3","key":"1112_CR18","doi-asserted-by":"publisher","first-page":"0318524","DOI":"10.1371\/journal.pone.0318524","volume":"20","author":"J Yan","year":"2025","unstructured":"Yan, J., Pu, P., Jiang, L.: Emotion-RGC net: a novel approach for emotion recognition in social media using RoBERTa and graph neural networks. PLoS ONE 20(3), 0318524 (2025)","journal-title":"PLoS ONE"},{"key":"1112_CR19","doi-asserted-by":"publisher","first-page":"21517","DOI":"10.1109\/ACCESS.2022.3152828","volume":"10","author":"KL Tan","year":"2022","unstructured":"Tan, K.L., Lee, C.P., Anbananthen, K.S.M., Lim, K.M.: RoBERTa-LSTM: a hybrid model for sentiment analysis with transformer and recurrent neural network. IEEE Access 10, 21517\u201321525 (2022)","journal-title":"IEEE Access"},{"key":"1112_CR20","doi-asserted-by":"publisher","first-page":"1292010","DOI":"10.3389\/fnhum.2023.1292010","volume":"17","author":"NA Semary","year":"2023","unstructured":"Semary, N.A., Ahmed, W., Amin, K., P\u0142awiak, P., Hammad, M.: Improving sentiment classification using a RoBERTa-based hybrid model. Front. Hum. Neurosci. 17, 1292010 (2023)","journal-title":"Front. Hum. Neurosci."},{"key":"1112_CR21","doi-asserted-by":"publisher","first-page":"149077","DOI":"10.1109\/ACCESS.2021.3118537","volume":"9","author":"Q Zhu","year":"2021","unstructured":"Zhu, Q., Jiang, X., Ye, R.: Sentiment analysis of review text based on BiGRU-attention and hybrid CNN. IEEE Access 9, 149077\u2013149088 (2021)","journal-title":"IEEE Access"},{"key":"1112_CR22","doi-asserted-by":"publisher","first-page":"84283","DOI":"10.1109\/ACCESS.2023.3296098","volume":"11","author":"LM Pereira","year":"2023","unstructured":"Pereira, L.M., Salazar, A., Vergara, L.: A comparative analysis of early and late fusion for the multimodal two-class problem. IEEE Access 11, 84283\u201384300 (2023)","journal-title":"IEEE Access"},{"key":"1112_CR23","first-page":"17612","volume":"35","author":"VW Liang","year":"2022","unstructured":"Liang, V.W., Zhang, Y., Kwon, Y., Yeung, S., Zou, J.Y.: Mind the gap: Understanding the modality gap in multi-modal contrastive representation learning. Adv. Neural. Inf. Process. Syst. 35, 17612\u201317625 (2022)","journal-title":"Adv. Neural. Inf. Process. Syst."},{"issue":"11","key":"1112_CR24","doi-asserted-by":"publisher","first-page":"11019","DOI":"10.1109\/TKDE.2022.3230975","volume":"35","author":"W Zhang","year":"2023","unstructured":"Zhang, W., Li, X., Deng, Y., Bing, L., Lam, W.: A survey on aspect-based sentiment analysis: tasks, methods, and challenges. IEEE Trans. Knowl. Data Eng. 35(11), 11019\u201311038 (2023)","journal-title":"IEEE Trans. Knowl. Data Eng."},{"issue":"8","key":"1112_CR25","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3548772","volume":"55","author":"T Abdullah","year":"2022","unstructured":"Abdullah, T., Ahmet, A.: Deep learning in sentiment analysis: recent architectures. ACM Comput. Surv. 55(8), 1\u201337 (2022)","journal-title":"ACM Comput. Surv."},{"key":"1112_CR26","doi-asserted-by":"crossref","unstructured":"Abnar, S., Zuidema, W.: Quantifying attention flow in transformers. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4190\u20134197. (2020)","DOI":"10.18653\/v1\/2020.acl-main.385"},{"issue":"3","key":"1112_CR27","doi-asserted-by":"publisher","first-page":"1316","DOI":"10.3390\/app12031316","volume":"12","author":"B AlBadani","year":"2022","unstructured":"AlBadani, B., Shi, R., Dong, J., Al-Sabri, R., Moctard, O.B.: Transformer-based graph convolutional network for sentiment analysis. Appl. Sci. 12(3), 1316 (2022)","journal-title":"Appl. Sci."},{"key":"1112_CR28","first-page":"28877","volume":"34","author":"C Ying","year":"2021","unstructured":"Ying, C., Cai, T., Luo, S., Zheng, S., Ke, G., He, D., Shen, Y., Liu, T.-Y.: Do transformers really perform badly for graph representation? Adv. Neural. Inf. Process. Syst. 34, 28877\u201328888 (2021)","journal-title":"Adv. Neural. Inf. Process. Syst."},{"key":"1112_CR29","unstructured":"Sabour, S., Frosst, N., Hinton, G.E.: Dynamic routing between capsules. Adv. Neural Inf. Proc. Syst. 30, (2017)"},{"key":"1112_CR30","doi-asserted-by":"crossref","unstructured":"Wang, Y., Sun, A., Han, J., Liu, Y., Zhu, X.: Sentiment analysis by capsules. In: Proceedings of the World Wide Web Conference, pp. 1165\u20131174. (2018)","DOI":"10.1145\/3178876.3186015"},{"key":"1112_CR31","doi-asserted-by":"publisher","first-page":"60383","DOI":"10.1109\/ACCESS.2021.3073988","volume":"9","author":"Y Cheng","year":"2021","unstructured":"Cheng, Y., Sun, H., Chen, H., Li, M., Cai, Y., Cai, Z., Huang, J.: Sentiment analysis using multi-head attention capsules with multi-channel CNN and bidirectional GRU. IEEE Access 9, 60383\u201360395 (2021)","journal-title":"IEEE Access"},{"key":"1112_CR32","doi-asserted-by":"publisher","first-page":"831","DOI":"10.7717\/peerj-cs.831","volume":"8","author":"X Jia","year":"2022","unstructured":"Jia, X., Wang, L.: Attention enhanced capsule network for text classification by encoding syntactic dependency trees with graph convolutional neural network. PeerJ Comput. Sci. 8, 831 (2022)","journal-title":"PeerJ Comput. Sci."},{"issue":"120","key":"1112_CR33","first-page":"1","volume":"23","author":"W Fedus","year":"2022","unstructured":"Fedus, W., Zoph, B., Shazeer, N.: Switch transformers: scaling to trillion parameter models with simple and efficient sparsity. J. Mach. Learn. Res. 23(120), 1\u201339 (2022)","journal-title":"J. Mach. Learn. Res."},{"key":"1112_CR34","doi-asserted-by":"crossref","unstructured":"Proskura, P., Zaytsev, A.: Beyond simple averaging: improving NLP ensemble performance with topological-data-analysis-based weighting. In: 2024 IEEE 11th International Conference on Data Science and Advanced Analytics (DSAA), pp. 1\u20138. IEEE (2024)","DOI":"10.1109\/DSAA61799.2024.10722805"},{"key":"1112_CR35","unstructured":"Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017)"},{"key":"1112_CR36","unstructured":"Maas, A., Daly, R.E., Pham, P.T., Huang, D., Ng, A.Y., Potts, C.: Learning word vectors for sentiment analysis. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pp. 142\u2013150. (2011)"},{"key":"1112_CR37","unstructured":"Zhang, X., Zhao, J., LeCun, Y.: Character-level convolutional networks for text classification. Adv. Neural Inf. Proc. Syst. 28, (2015)"},{"key":"1112_CR38","doi-asserted-by":"crossref","unstructured":"Pang, B., Lee, L.: Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In: Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, pp. 115\u2013124. (2005)","DOI":"10.3115\/1219840.1219855"},{"key":"1112_CR39","doi-asserted-by":"crossref","unstructured":"Hu, M., Liu, B.: Mining and summarizing customer reviews. In: Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 168\u2013177. (2004)","DOI":"10.1145\/1014052.1014073"},{"key":"1112_CR40","unstructured":"Liang, X., Wu, L., Li, J., Wang, Y., Meng, Q., Qin, T., Chen, W., Zhang, M., Liu, T.-Y.: R-Drop: regularized dropout for neural networks. In: Advances in Neural Information Processing Systems, vol. 34. (2021)"},{"key":"1112_CR41","unstructured":"Zhang, H., Ciss\u00e9, M., Dauphin, Y.N., Lopez-Paz, D.: mixup: Beyond empirical risk minimization. In: International Conference on Learning Representations (2018)"},{"key":"1112_CR42","first-page":"747","volume":"3","author":"X Bouthillier","year":"2021","unstructured":"Bouthillier, X., Delaunay, P., Bronzi, M., Trofimov, A., Nichyporuk, B., Szeto, J., Mohammadi Sepahvand, N., Raff, E., Madan, K., Voleti, V., et al.: Accounting for variance in machine learning benchmarks. Proc. Mach. Learn. Syst. 3, 747\u2013769 (2021)","journal-title":"Proc. Mach. Learn. Syst."},{"key":"1112_CR43","unstructured":"Dodge, J., Ilharco, G., Schwartz, R., Farhadi, A., Hajishirzi, H., Smith, N.: Fine-tuning pretrained language models: Weight initializations, data orders, and early stopping. arXiv preprint arXiv:2002.06305 (2020)"},{"key":"1112_CR44","doi-asserted-by":"crossref","unstructured":"Zhao, W., Ye, J., Yang, M., Lei, Z., Zhang, S., Zhao, Z.: Investigating capsule networks with dynamic routing for text classification. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 3110\u20133119. (2018)","DOI":"10.18653\/v1\/D18-1350"},{"key":"1112_CR45","doi-asserted-by":"crossref","unstructured":"Conneau, A., Schwenk, H., Barrault, L., Lecun, Y.: Very deep convolutional networks for text classification. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: vol. 1, Long Papers, pp. 1107\u20131116. (2017)","DOI":"10.18653\/v1\/E17-1104"},{"key":"1112_CR46","doi-asserted-by":"crossref","unstructured":"Joulin, A., Grave, E., Bojanowski, P., Mikolov, T.: Bag of tricks for efficient text classification. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: vol. 2, Short Papers, pp. 427\u2013431. (2017)","DOI":"10.18653\/v1\/E17-2068"},{"key":"1112_CR47","doi-asserted-by":"crossref","unstructured":"Adamuthe, A.C., Jagtap, S.: Comparative study of convolutional neural network with word embedding technique for text classification. IJ Intell. Syst. Appl. 8, 56\u201367 (2019)","DOI":"10.5815\/ijisa.2019.08.06"}],"container-title":["International Journal of Data Science and Analytics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s41060-026-01112-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s41060-026-01112-x","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s41060-026-01112-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,15]],"date-time":"2026-04-15T11:28:48Z","timestamp":1776252528000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s41060-026-01112-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,15]]},"references-count":47,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,12]]}},"alternative-id":["1112"],"URL":"https:\/\/doi.org\/10.1007\/s41060-026-01112-x","relation":{},"ISSN":["2364-415X","2364-4168"],"issn-type":[{"value":"2364-415X","type":"print"},{"value":"2364-4168","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,15]]},"assertion":[{"value":"28 December 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 March 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 April 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}],"article-number":"137"}}