{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,16]],"date-time":"2025-10-16T20:52:02Z","timestamp":1760647922160,"version":"3.40.5"},"reference-count":58,"publisher":"World Scientific Pub Co Pte Ltd","issue":"01","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Vietnam J. Comp. Sci."],"published-print":{"date-parts":[[2024,2]]},"abstract":"<jats:p> Sentiment Analysis (SA) has attracted increasing research attention in recent years. Most existing works tackle the SA task by fine-tuning single pre-trained language models combined with specific layers. Despite their effectiveness, the previous studies overlooked the utilization of feature representations from various contextual language models. Ensemble learning techniques have garnered increasing attention within the field of Natural Language Processing (NLP). However, there is still room for improvement in ensemble models for the SA task, particularly in the aspect-level SA task. Furthermore, the utilization of heterogeneous ensembles, which combine various pre-trained transformer-based language models, may prove beneficial in enhancing overall performance by incorporating diverse linguistic representations. This paper introduces two ensemble models that leverage soft voting and feature fusion techniques by combining individual pre-trained transformer-based language models for the SA task. The latest transformer-based models, including PhoBERT, XLM, XLM-Align, InfoXLM, and viBERT_FPT, are employed to integrate knowledge and representations using feature fusion and a soft voting strategy. We conducted extensive experiments on various Vietnamese benchmark datasets, encompassing sentence-level, document-level, and aspect-level SA. The experimental results demonstrate that our approaches outperform most existing methods, achieving new state-of-the-art results with F1-weighted scores of 94.03%, 95.65%, 75.36%, and 76.23% on the UIT_VSFC, Aivivn, UIT_ABSA for the restaurant domain, and UIT_ViSFD datasets, respectively. <\/jats:p>","DOI":"10.1142\/s2196888823500173","type":"journal-article","created":{"date-parts":[[2023,11,18]],"date-time":"2023-11-18T04:37:57Z","timestamp":1700282277000},"page":"137-165","source":"Crossref","is-referenced-by-count":1,"title":["A Study of Vietnamese Sentiment Classification with Ensemble Pre-Trained Language Models"],"prefix":"10.1142","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8340-1405","authenticated-orcid":false,"given":"Dang Van","family":"Thin","sequence":"first","affiliation":[{"name":"University of Information Technology, Vietnam National University Ho Chi Minh City, Quarter 6, Linh Trung Ward, Thu Duc District, Ho Chi Minh City, Vietnam"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3136-0661","authenticated-orcid":false,"given":"Duong Ngoc","family":"Hao","sequence":"additional","affiliation":[{"name":"University of Information Technology, Vietnam National University Ho Chi Minh City, Quarter 6, Linh Trung Ward, Thu Duc District, Ho Chi Minh City, Vietnam"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3931-849X","authenticated-orcid":false,"given":"Ngan Luu-Thuy","family":"Nguyen","sequence":"additional","affiliation":[{"name":"University of Information Technology, Vietnam National University Ho Chi Minh City, Quarter 6, Linh Trung Ward, Thu Duc District, Ho Chi Minh City, Vietnam"}]}],"member":"219","published-online":{"date-parts":[[2023,12,7]]},"reference":[{"key":"S2196888823500173BIB001","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2022.118246"},{"key":"S2196888823500173BIB002","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-02145-9"},{"key":"S2196888823500173BIB003","doi-asserted-by":"publisher","DOI":"10.1016\/B978-0-12-804412-4.00001-2"},{"key":"S2196888823500173BIB004","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2020.2970399"},{"key":"S2196888823500173BIB005","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-022-10144-1"},{"key":"S2196888823500173BIB006","doi-asserted-by":"publisher","DOI":"10.1109\/KSE.2018.8573337"},{"key":"S2196888823500173BIB007","doi-asserted-by":"publisher","DOI":"10.1145\/3446678"},{"key":"S2196888823500173BIB008","doi-asserted-by":"publisher","DOI":"10.1109\/NICS.2018.8606857"},{"key":"S2196888823500173BIB009","doi-asserted-by":"publisher","DOI":"10.1109\/NICS51282.2020.9335912"},{"key":"S2196888823500173BIB010","first-page":"343","volume-title":"Proc. 18th Int. Conf. Intelligent Software Methodologies, Tools, and Techniques (SoMeT)","author":"Cuong N.","year":"2020"},{"key":"S2196888823500173BIB011","doi-asserted-by":"publisher","DOI":"10.1109\/NICS51282.2020.9335899"},{"key":"S2196888823500173BIB012","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-63119-2_61"},{"key":"S2196888823500173BIB013","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2021.12.059"},{"key":"S2196888823500173BIB014","first-page":"4171","volume-title":"Proc. 2019 Conf. North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Devlin J.","year":"2019"},{"key":"S2196888823500173BIB015","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.747"},{"key":"S2196888823500173BIB016","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.acl-long.265"},{"key":"S2196888823500173BIB017","doi-asserted-by":"publisher","DOI":"10.1007\/s11431-020-1647-3"},{"key":"S2196888823500173BIB018","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.naacl-main.280"},{"key":"S2196888823500173BIB019","first-page":"13","volume-title":"Proc. 34th Pacific Asia Conf. Language, Information and Computation","author":"Bui T. V.","year":"2020"},{"key":"S2196888823500173BIB020","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.92"},{"key":"S2196888823500173BIB021","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4614-3223-4_13"},{"key":"S2196888823500173BIB022","doi-asserted-by":"publisher","DOI":"10.1145\/2837185.2837215"},{"key":"S2196888823500173BIB023","first-page":"706","volume-title":"Proc. 32nd Pacific Asia Conf. Language, Information and Computation","author":"Vo H. Q. V.","year":"2018"},{"key":"S2196888823500173BIB024","doi-asserted-by":"publisher","DOI":"10.3233\/JIFS-172053"},{"key":"S2196888823500173BIB025","doi-asserted-by":"publisher","DOI":"10.1109\/ATC.2014.7043403"},{"key":"S2196888823500173BIB026","doi-asserted-by":"publisher","DOI":"10.15625\/1813-9663\/36\/4\/14829"},{"key":"S2196888823500173BIB027","doi-asserted-by":"publisher","DOI":"10.1109\/KSE.2018.8573351"},{"key":"S2196888823500173BIB028","doi-asserted-by":"publisher","DOI":"10.1145\/3589131"},{"key":"S2196888823500173BIB029","doi-asserted-by":"publisher","DOI":"10.1109\/TCSS.2020.3033302"},{"key":"S2196888823500173BIB030","doi-asserted-by":"publisher","DOI":"10.1145\/3503044"},{"key":"S2196888823500173BIB031","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-75417-8_14"},{"key":"S2196888823500173BIB032","doi-asserted-by":"publisher","DOI":"10.1007\/s10479-020-03534-7"},{"key":"S2196888823500173BIB033","first-page":"130","volume-title":"2022 RIVF Int. Conf. Computing and Communication Technologies (RIVF)","author":"Thin D. V.","year":"2022"},{"key":"S2196888823500173BIB034","doi-asserted-by":"publisher","DOI":"10.15625\/1813-9663\/34\/4\/13162"},{"key":"S2196888823500173BIB035","doi-asserted-by":"publisher","DOI":"10.1109\/RIVF48685.2020.9140735"},{"key":"S2196888823500173BIB036","first-page":"169","volume-title":"Int. Conf. Pacific Association for Computational Linguistics","author":"Thin D. V.","year":"2019"},{"issue":"4","key":"S2196888823500173BIB037","first-page":"126","volume":"12","author":"Thin D. V.","year":"2022","journal-title":"Int. J. Mach. Learn. Comput."},{"key":"S2196888823500173BIB038","doi-asserted-by":"publisher","DOI":"10.1109\/CyberneticsCom55287.2022.9865479"},{"key":"S2196888823500173BIB039","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-012-9338-y"},{"key":"S2196888823500173BIB040","doi-asserted-by":"publisher","DOI":"10.1007\/978-981-16-5157-1_30"},{"key":"S2196888823500173BIB041","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2021.107495"},{"key":"S2196888823500173BIB042","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2022.3210182"},{"key":"S2196888823500173BIB043","doi-asserted-by":"publisher","DOI":"10.1080\/01969722.2022.2148510"},{"key":"S2196888823500173BIB044","first-page":"29","volume":"12","author":"Mohammadi A.","year":"2021","journal-title":"Int. J. Nonlinear Anal. Appl."},{"key":"S2196888823500173BIB045","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2021.115819"},{"key":"S2196888823500173BIB046","doi-asserted-by":"publisher","DOI":"10.1145\/3368926.3369675"},{"key":"S2196888823500173BIB047","doi-asserted-by":"publisher","DOI":"10.1109\/KSE53942.2021.9648599"},{"key":"S2196888823500173BIB048","doi-asserted-by":"publisher","DOI":"10.3233\/JIFS-219278"},{"key":"S2196888823500173BIB049","doi-asserted-by":"publisher","DOI":"10.1016\/j.jksuci.2023.01.014"},{"key":"S2196888823500173BIB050","first-page":"647","volume-title":"Proc. Knowledge Science, Engineering and Management","author":"Luc L. P.","year":"2021"},{"key":"S2196888823500173BIB051","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/S16-1002"},{"key":"S2196888823500173BIB052","first-page":"462","volume-title":"Proc. 34th Pacific Asia Conf. Language, Information and Computation","author":"Luu S.","year":"2020"},{"key":"S2196888823500173BIB053","first-page":"420","volume-title":"Proc. 34th Pacific Asia Conf. Language, Information and Computation","author":"Huynh H. D.","year":"2020"},{"key":"S2196888823500173BIB054","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-demos.6"},{"key":"S2196888823500173BIB055","first-page":"28","volume-title":"Proc. 17th Annual Workshop Australasian Language Technology Association","author":"Nguyen D. Q.","year":"2019"},{"key":"S2196888823500173BIB056","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-79463-7_9"},{"key":"S2196888823500173BIB057","doi-asserted-by":"publisher","DOI":"10.1109\/NICS54270.2021.9701518"},{"key":"S2196888823500173BIB058","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.acl-long.243"}],"container-title":["Vietnam Journal of Computer Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.worldscientific.com\/doi\/pdf\/10.1142\/S2196888823500173","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,2,22]],"date-time":"2024-02-22T08:19:58Z","timestamp":1708589998000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.worldscientific.com\/doi\/10.1142\/S2196888823500173"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,7]]},"references-count":58,"journal-issue":{"issue":"01","published-print":{"date-parts":[[2024,2]]}},"alternative-id":["10.1142\/S2196888823500173"],"URL":"https:\/\/doi.org\/10.1142\/s2196888823500173","relation":{},"ISSN":["2196-8888","2196-8896"],"issn-type":[{"type":"print","value":"2196-8888"},{"type":"electronic","value":"2196-8896"}],"subject":[],"published":{"date-parts":[[2023,12,7]]}}}