{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,11]],"date-time":"2026-06-11T18:06:21Z","timestamp":1781201181347,"version":"3.54.1"},"reference-count":38,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2024,5,24]],"date-time":"2024-05-24T00:00:00Z","timestamp":1716508800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Natural Science Basic Research Program of Shaanxi","award":["2022JQ-675"],"award-info":[{"award-number":["2022JQ-675"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>With the development of deep learning, several graph neural network (GNN)-based approaches have been utilized for text classification. However, GNNs encounter challenges when capturing contextual text information within a document sequence. To address this, a novel text classification model, RB-GAT, is proposed by combining RoBERTa-BiGRU embedding and a multi-head Graph ATtention Network (GAT). First, the pre-trained RoBERTa model is exploited to learn word and text embeddings in different contexts. Second, the Bidirectional Gated Recurrent Unit (BiGRU) is employed to capture long-term dependencies and bidirectional sentence information from the text context. Next, the multi-head graph attention network is applied to analyze this information, which serves as a node feature for the document. Finally, the classification results are generated through a Softmax layer. Experimental results on five benchmark datasets demonstrate that our method can achieve an accuracy of 71.48%, 98.45%, 80.32%, 90.84%, and 95.67% on Ohsumed, R8, MR, 20NG and R52, respectively, which is superior to the existing nine text classification approaches.<\/jats:p>","DOI":"10.3390\/s24113365","type":"journal-article","created":{"date-parts":[[2024,5,24]],"date-time":"2024-05-24T05:36:06Z","timestamp":1716528966000},"page":"3365","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["RB-GAT: A Text Classification Model Based on RoBERTa-BiGRU with Graph ATtention Network"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8762-4091","authenticated-orcid":false,"given":"Shaoqing","family":"Lv","sequence":"first","affiliation":[{"name":"School of Communication and Information Engineering, Xi\u2019an University of Posts and Telecommunications, Xi\u2019an 710121, China"},{"name":"Shaanxi Key Laboratory of Information Communication Network and Security, Xi\u2019an University of Posts and Telecommunications, Xi\u2019an 710121, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jungang","family":"Dong","sequence":"additional","affiliation":[{"name":"School of Communication and Information Engineering, Xi\u2019an University of Posts and Telecommunications, Xi\u2019an 710121, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chichi","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Communication and Information Engineering, Xi\u2019an University of Posts and Telecommunications, Xi\u2019an 710121, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xuanhong","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Communication and Information Engineering, Xi\u2019an University of Posts and Telecommunications, Xi\u2019an 710121, China"},{"name":"Shaanxi Key Laboratory of Information Communication Network and Security, Xi\u2019an University of Posts and Telecommunications, Xi\u2019an 710121, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhiqiang","family":"Bao","sequence":"additional","affiliation":[{"name":"School of Communication and Information Engineering, Xi\u2019an University of Posts and Telecommunications, Xi\u2019an 710121, China"},{"name":"Shaanxi Key Laboratory of Information Communication Network and Security, Xi\u2019an University of Posts and Telecommunications, Xi\u2019an 710121, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,5,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Kowsari, K., Jafari Meimandi, K., Heidarysafa, M., Mendu, S., Barnes, L., and Brown, D. (2019). Text classification algorithms: A survey. Information, 10.","DOI":"10.3390\/info10040150"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"e9","DOI":"10.1002\/spy2.9","article-title":"Detecting opinion spams and fake news using text classification","volume":"1","author":"Ahmed","year":"2018","journal-title":"Secur. Priv."},{"key":"ref_3","unstructured":"Melville, P., Gryc, W., and Lawrence, R.D. (July, January 28). Sentiment analysis of blogs by combining lexical knowledge with text classification. Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Paris, France."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1017\/pan.2020.8","article-title":"Automated text classification of news articles: A practical guide","volume":"29","author":"Boydstun","year":"2021","journal-title":"Political Anal."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Chowdhary, K., and Chowdhary, K. (2020). Natural language processing. Fundam. Artif. Intell., 603\u2013649.","DOI":"10.1007\/978-81-322-3972-7_19"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s41133-020-00032-0","article-title":"A comparative analysis of logistic regression, random forest and KNN models for the text classification","volume":"5","author":"Shah","year":"2020","journal-title":"Augment. Hum. Res."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s11771-019-3978-x","article-title":"Parallel naive Bayes algorithm for large-scale Chinese text classification based on spark","volume":"26","author":"Liu","year":"2019","journal-title":"J. Cent. South Univ."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Kalcheva, N., Karova, M., and Penev, I. (2020, January 24\u201327). Comparison of the accuracy of SVM kemel functions in text classification. Proceedings of the 2020 International Conference on Biomedical Innovations and Applications (BIA), Varna, Bulgaria.","DOI":"10.1109\/BIA50171.2020.9244278"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3439726","article-title":"Deep learning\u2013based text classification: A comprehensive review","volume":"54","author":"Minaee","year":"2021","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Kim, Y. (2014, January 25\u201329). Convolutional Neural Networks for Sentence Classification. Proceedings of the Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.","DOI":"10.3115\/v1\/D14-1181"},{"key":"ref_11","unstructured":"Liu, P., Qiu, X., and Huang, X. (2016, January 9\u201315). Recurrent neural network for text classification with multi-task learning. Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, New York, NY, USA."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1561\/2200000096","article-title":"Graph neural networks for natural language processing: A survey","volume":"16","author":"Wu","year":"2023","journal-title":"Found. Trends\u00ae Mach. Learn."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1016\/j.aiopen.2021.01.001","article-title":"Graph neural networks: A review of methods and applications","volume":"1","author":"Zhou","year":"2020","journal-title":"AI Open"},{"key":"ref_14","first-page":"1","article-title":"Graph neural networks in recommender systems: A survey","volume":"55","author":"Wu","year":"2022","journal-title":"ACM Comput. Surv."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1382","DOI":"10.1016\/j.drudis.2021.02.011","article-title":"Graph neural networks for automated de novo drug design","volume":"26","author":"Xiong","year":"2021","journal-title":"Drug Discov. Today"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"402","DOI":"10.1016\/j.cels.2020.08.016","article-title":"Fast and flexible protein design using deep graph neural networks","volume":"11","author":"Strokach","year":"2020","journal-title":"Cell Syst."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"107096","DOI":"10.1016\/j.compeleceng.2021.107096","article-title":"Multi-level graph neural network for text sentiment analysis","volume":"92","author":"Liao","year":"2021","journal-title":"Comput. Electr. Eng."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Xie, Q., Huang, J., Du, P., Peng, M., and Nie, J.-Y. (2021, January 15). Graph topic neural network for document representation. Proceedings of the Web Conference 2021, Ljubljana, Slovenia.","DOI":"10.1145\/3442381.3450045"},{"key":"ref_19","unstructured":"Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv."},{"key":"ref_20","unstructured":"Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1235","DOI":"10.1162\/neco_a_01199","article-title":"A review of recurrent neural networks: LSTM cells and network architectures","volume":"31","author":"Yu","year":"2019","journal-title":"Neural Comput."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"51522","DOI":"10.1109\/ACCESS.2019.2909919","article-title":"Sentiment analysis of comment texts based on BiLSTM","volume":"7","author":"Xu","year":"2019","journal-title":"IEEE Access"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Tai, K.S., Socher, R., and Manning, C.D. (2015, January 30\u201331). Improved semantic representations from tree-structured long short-term memory networks. Proceedings of the Annual Meeting of the Association for Computational Linguistics, Beijing, China.","DOI":"10.3115\/v1\/P15-1150"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Lei, T., Zhang, Y., Wang, S.I., Dai, H., and Artzi, Y. (2017). Simple recurrent units for highly parallelizable recurrence. arXiv.","DOI":"10.18653\/v1\/D18-1477"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proc. IEEE"},{"key":"ref_26","unstructured":"Grefenstette, E., and Blunsom, P. (2014, January 22\u201327). A convolutional neural network for modelling sentences. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, Baltimore, MD, USA."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Liu, J., Chang, W.-C., Wu, Y., and Yang, Y. (2017, January 7\u201311). Deep learning for extreme multi-label text classification. Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, Tokyo, Japan.","DOI":"10.1145\/3077136.3080834"},{"key":"ref_28","unstructured":"Zhang, X., Zhao, J., and LeCun, Y. (2015, January 7\u201312). Character-level convolutional networks for text classification. Proceedings of the 28th International Conference on Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Yao, L., Mao, C., and Luo, Y. (2019, January 27). Graph convolutional networks for text classification. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.","DOI":"10.1609\/aaai.v33i01.33017370"},{"key":"ref_30","unstructured":"Liu, X., You, X., Zhang, X., Wu, J., and Lv, P. (2020, January 7\u201312). Tensor graph convolutional networks for text classification. Proceedings of the AAAI conference on artificial intelligence, New York, NY, USA."},{"key":"ref_31","unstructured":"Defferrard, M., Bresson, X., and Vandergheynst, P. (2016, January 5\u201310). Convolutional neural networks on graphs with fast localized spectral filtering. Proceedings of the 30th International Conference on Neural Information Processing Systems, Barcelona, Spain."},{"key":"ref_32","unstructured":"Wu, F., Souza, A., Zhang, T., Fifty, C., Yu, T., and Weinberger, K. (2019, January 9\u201315). Simplifying graph convolutional networks. Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA."},{"key":"ref_33","unstructured":"Hu, L., Yang, T., Shi, C., Ji, H., and Li, X. (2019, January 3\u20137). Heterogeneous graph attention networks for semi-supervised short text classification. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Yu, X., Cui, Z., Wu, S., Wen, Z., and Wang, L. (2020, January 5\u201310). Every document owns its structure: Inductive text classification via graph neural networks. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online.","DOI":"10.18653\/v1\/2020.acl-main.31"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Golovin, D., Solnik, B., Moitra, S., Kochanski, G., Karro, J., and Sculley, D. (2017, January 13\u201317). Google vizier: A service for black-box optimization. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, USA.","DOI":"10.1145\/3097983.3098043"},{"key":"ref_36","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Piao, Y., Lee, S., Lee, D., and Kim, S. (2022, January 1). Sparse structure learning via graph neural networks for inductive document classification. Proceedings of the AAAI Conference on Artificial Intelligence, Online.","DOI":"10.1609\/aaai.v36i10.21366"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"124002","DOI":"10.1016\/j.eswa.2024.124002","article-title":"Capturing word positions does help: A multi-element hypergraph gated attention network for document classification","volume":"251","author":"Jin","year":"2024","journal-title":"Expert Syst. Appl."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/11\/3365\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:47:52Z","timestamp":1760107672000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/11\/3365"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,24]]},"references-count":38,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2024,6]]}},"alternative-id":["s24113365"],"URL":"https:\/\/doi.org\/10.3390\/s24113365","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,24]]}}}