{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,3,6]],"date-time":"2025-03-06T05:12:13Z","timestamp":1741237933533,"version":"3.38.0"},"reference-count":30,"publisher":"National Library of Serbia","issue":"1","license":[{"start":{"date-parts":[[2025,1,1]],"date-time":"2025-01-01T00:00:00Z","timestamp":1735689600000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["ComSIS","COMPUT SCI INF SYST","COMPUT SCI INFORM SY","COMPUTER SCI INFORM","COMSIS J"],"published-print":{"date-parts":[[2025]]},"abstract":"<jats:p>Deep learning methods have been widely applied to English text classification tasks in recent years, achieving strong performance. However, current methods face two significant challenges: (1) they struggle to effectively capture longrange contextual structure information within text sequences, and (2) they do not adequately integrate linguistic knowledge into representations for enhancing the performance of classifiers. To this end, a novel multilingual pre-training based multifeature fusion method is proposed for English text classification (MFFMP-ETC). Specifically, MFFMP-ETC consists of the multilingual feature extraction, the multilevel structure learning, and the multi-view representation fusion. MFFMP-ETC utilizes the Multilingual BERT as deep semantic extractor to introduce language information into representation learning, which significantly endows text representations with robustness. Then, MFFMP-ETC integrates Bi-LSTM and TextCNN into multilingual pre-training architecture to capture global and local structure information of English texts, via modelling bidirectional contextual semantic dependencies and multi-granularity local semantic dependencies. Meanwhile, MFFMP-ETC devises the multi-view representation fusion within the invariant semantic learning of representations to aggregate consistent and complementary information among views. MFFMP-ETC synergistically integrates Multilingual BERT?s deep semantic features, Bi-LSTM?s bidirectional context processing, and TextCNN local feature extraction, offering a more comprehensive and effective solution for capturing long-distance dependencies and nuanced contextual information in text classification. Finally, results on three datasets show MFFMP-ETC conducts a new baseline in terms of accuracy, sensitivity, and precision, verifying progressiveness and effectiveness of MFFMP-ETC in the text classification.<\/jats:p>","DOI":"10.2298\/csis240630004z","type":"journal-article","created":{"date-parts":[[2025,1,21]],"date-time":"2025-01-21T08:54:23Z","timestamp":1737449663000},"page":"133-152","source":"Crossref","is-referenced-by-count":0,"title":["Multilingual pretrained based multi-feature fusion model for English text classification"],"prefix":"10.2298","volume":"22","author":[{"given":"Ruijuan","family":"Zhang","sequence":"first","affiliation":[{"name":"School of Foreign Languages, Zhengzhou University of Science and Technology, Zhengzhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1078","reference":[{"key":"ref1","doi-asserted-by":"crossref","unstructured":"Chen, J., Yang, Z., Yang, D.: Mixtext: Linguistically-informed interpolation of hidden space for semi-supervised text classification. arXiv preprint arXiv:2004.12239 (2020)","DOI":"10.18653\/v1\/2020.acl-main.194"},{"key":"ref2","unstructured":"Chen, Y.: Convolutional neural network for sentence classification (2015)"},{"key":"ref3","unstructured":"Defferrard, M., Bresson, X., Vandergheynst, P.: Convolutional neural networks on graphs with fast localized spectral filtering. Advances in neural information processing systems 29 (2016)"},{"key":"ref4","unstructured":"Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)"},{"key":"ref5","doi-asserted-by":"crossref","unstructured":"Gao, J., Li, P., Laghari, A.A., Srivastava, G., Gadekallu, T.R., Abbas, S., Zhang, J.: Incomplete multiview clustering via semidiscrete optimal transport for multimedia data mining in iot. ACM Transactions on Multimedia Computing, Communications and Applications 20(6), 1-20 (2024)","DOI":"10.1145\/3625548"},{"key":"ref6","doi-asserted-by":"crossref","unstructured":"Gao, J., Liu, M., Li, P., Laghari, A.A., Javed, A.R., Victor, N., Gadekallu, T.R.: Deep incomplete multi-view clustering via information bottleneck for pattern mining of data in extremeenvironment iot. IEEE Internet of Things Journal (2023)","DOI":"10.1109\/JIOT.2023.3325272"},{"key":"ref7","doi-asserted-by":"crossref","unstructured":"Gao, J., Liu, M., Li, P., Zhang, J., Chen, Z.: Deep multiview adaptive clustering with semantic invariance. Transactions on Neural Networks and Learning Systems (2023)","DOI":"10.1109\/TNNLS.2023.3265699"},{"key":"ref8","doi-asserted-by":"crossref","unstructured":"Gururangan, S., Dang, T., Card, D., Smith, N.A.: Variational pretraining for semi-supervised text classification. arXiv preprint arXiv:1906.02242 (2019)","DOI":"10.18653\/v1\/P19-1590"},{"key":"ref9","doi-asserted-by":"crossref","unstructured":"Johnson, R., Zhang, T.: Deep pyramid convolutional neural networks for text categorization. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 562-570 (2017)","DOI":"10.18653\/v1\/P17-1052"},{"key":"ref10","doi-asserted-by":"crossref","unstructured":"Joulin, A., Grave, E., Bojanowski, P., Mikolov, T.: Bag of tricks for efficient text classification. arXiv preprint arXiv:1607.01759 (2016)","DOI":"10.18653\/v1\/E17-2068"},{"key":"ref11","doi-asserted-by":"crossref","unstructured":"Li, C., Peng, X., Peng, H., Li, J., Wang, L.: Textgtl: Graph-based transductive learning for semi-supervised text classification via structure-sensitive interpolation. In: IJCAI. pp. 2680- 2686 (2021)","DOI":"10.24963\/ijcai.2021\/369"},{"key":"ref12","doi-asserted-by":"crossref","unstructured":"Li, P., Chen, Z., Yang, L.T., Gao, J., Zhang, Q., Deen, M.J.: An incremental deep convolutional computation model for feature learning on industrial big data. Transactions on Industrial Informatics 15(3), 1341-1349 (2018)","DOI":"10.1109\/TII.2018.2871084"},{"key":"ref13","doi-asserted-by":"crossref","unstructured":"Li, P., Gao, J., Zhang, J., Jin, S., Chen, Z.: Deep reinforcement clustering. Transactions on Multimedia (2022)","DOI":"10.1109\/TMM.2022.3233249"},{"key":"ref14","doi-asserted-by":"crossref","unstructured":"Li, P., Laghari, A.A., Rashid, M., Gao, J., Gadekallu, T.R., Javed, A.R., Yin, S.: A deep multimodal adversarial cycle-consistent network for smart enterprise system. IEEE Transactions on Industrial Informatics 19(1), 693-702 (2022)","DOI":"10.1109\/TII.2022.3197201"},{"key":"ref15","doi-asserted-by":"crossref","unstructured":"Lin, Y., Meng, Y., Sun, X., Han, Q., Kuang, K., Li, J., Wu, F.B.: Transductive text classification by combining gcn and bert. arxiv 2021. arXiv preprint arXiv:2105.05727","DOI":"10.18653\/v1\/2021.findings-acl.126"},{"key":"ref16","doi-asserted-by":"crossref","unstructured":"Liu, C.,Wang, X.: Quality-related english text classification based on recurrent neural network. Journal of Visual Communication and Image Representation 71, 102724 (2020)","DOI":"10.1016\/j.jvcir.2019.102724"},{"key":"ref17","doi-asserted-by":"crossref","unstructured":"Liu, P., Qiu, X., Chen, X., Wu, S., Huang, X.J.: Multi-timescale long short-term memory neural network for modelling sentences and documents. In: conference on empirical methods in natural language processing. pp. 2326-2335 (2015)","DOI":"10.18653\/v1\/D15-1280"},{"key":"ref18","doi-asserted-by":"crossref","unstructured":"Mundra, S., Mittal, N.: Fa-net: fused attention-based network for hindi english code-mixed offensive text classification. Social Network Analysis and Mining 12(1), 100 (2022)","DOI":"10.1007\/s13278-022-00929-1"},{"key":"ref19","doi-asserted-by":"crossref","unstructured":"Peng, H., Li, J., He, Y., Liu, Y., Bao, M.,Wang, L., Song, Y., Yang, Q.: Large-scale hierarchical text classification with recursively regularized deep graph-cnn. In: world wide web conference. pp. 1063-1072 (2018)","DOI":"10.1145\/3178876.3186005"},{"key":"ref20","doi-asserted-by":"crossref","unstructured":"Sachan, D.S., Zaheer, M., Salakhutdinov, R.: Revisiting lstm networks for semi-supervised text classification via mixed objective function. In: Proceedings of the aaai conference on artificial intelligence. vol. 33, pp. 6940-6948 (2019)","DOI":"10.1609\/aaai.v33i01.33016940"},{"key":"ref21","doi-asserted-by":"crossref","unstructured":"Shabestan\u0131, S., Gec\uff0cikli, M.: Machine learning use for english texts\u2019 classification (a minireview). Osmaniye Korkut Ata  \u00dcniversitesi Fen Bilimleri Enstit\u00fcs\u00fc Dergisi 7(1), 414-423 (2024)","DOI":"10.47495\/okufbed.1259868"},{"key":"ref22","unstructured":"Taha, K., Yoo, P.D., Yeun, C., Taha, A.: Text classification: A review, empirical, and experimental evaluation. arXiv preprint arXiv:2401.12982 (2024)"},{"key":"ref23","doi-asserted-by":"crossref","unstructured":"Wang, G., Li, C., Wang, W., Zhang, Y., Shen, D., Zhang, X., Henao, R., Carin, L.: Joint embedding of words and labels for text classification. arXiv preprint arXiv:1805.04174 (2018)","DOI":"10.18653\/v1\/P18-1216"},{"key":"ref24","doi-asserted-by":"crossref","unstructured":"Wang, X., Ji, H., Shi, C., Wang, B., Ye, Y., Cui, P., Yu, P.S.: Heterogeneous graph attention network. In: The world wide web conference. pp. 2022-2032 (2019)","DOI":"10.1145\/3308558.3313562"},{"key":"ref25","doi-asserted-by":"crossref","unstructured":"Wang, Z., Liu, X., Yang, P., Liu, S., Wang, Z.: Cross-lingual text classification with heterogeneous graph neural network. arXiv preprint arXiv:2105.11246 (2021)","DOI":"10.18653\/v1\/2021.acl-short.78"},{"key":"ref26","doi-asserted-by":"crossref","unstructured":"Xie, Q., Huang, J., Du, P., Peng, M., Nie, J.Y.: Inductive topic variational graph auto-encoder for text classification pp. 4218-4227 (2021)","DOI":"10.18653\/v1\/2021.naacl-main.333"},{"key":"ref27","doi-asserted-by":"crossref","unstructured":"Xu, J., Cai, Y., Wu, X., Lei, X., Huang, Q., Leung, H.f., Li, Q.: Incorporating context-relevant concepts into convolutional neural networks for short text classification. Neurocomputing 386, 42-53 (2020)","DOI":"10.1016\/j.neucom.2019.08.080"},{"key":"ref28","doi-asserted-by":"crossref","unstructured":"Yao, L., Mao, C., Luo, Y.: Graph convolutional networks for text classification. In: Proceedings of the AAAI conference on artificial intelligence. vol. 33, pp. 7370-7377 (2019)","DOI":"10.1609\/aaai.v33i01.33017370"},{"key":"ref29","doi-asserted-by":"crossref","unstructured":"Zhang, H., Zhang, J.: Text graph transformer for document classification. In: Conference on empirical methods in natural language processing (EMNLP) (2020)","DOI":"10.18653\/v1\/2020.emnlp-main.668"},{"key":"ref30","unstructured":"Zhang, X., Zhao, J., LeCun, Y.: Character-level convolutional networks for text classification. Advances in neural information processing systems 28 (2015)"}],"container-title":["Computer Science and Information Systems"],"original-title":[],"language":"en","deposited":{"date-parts":[[2025,3,5]],"date-time":"2025-03-05T09:24:18Z","timestamp":1741166658000},"score":1,"resource":{"primary":{"URL":"https:\/\/doiserbia.nb.rs\/Article.aspx?ID=1820-02142500004Z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025]]},"references-count":30,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025]]}},"URL":"https:\/\/doi.org\/10.2298\/csis240630004z","relation":{},"ISSN":["1820-0214","2406-1018"],"issn-type":[{"type":"print","value":"1820-0214"},{"type":"electronic","value":"2406-1018"}],"subject":[],"published":{"date-parts":[[2025]]}}}