{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T02:10:06Z","timestamp":1750299006615,"version":"3.41.0"},"reference-count":31,"publisher":"Association for Computing Machinery (ACM)","issue":"5","license":[{"start":{"date-parts":[[2025,4,23]],"date-time":"2025-04-23T00:00:00Z","timestamp":1745366400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Asian Low-Resour. Lang. Inf. Process."],"published-print":{"date-parts":[[2025,5,31]]},"abstract":"<jats:p>The Vietnamese multi-document summarization task faces three key challenges including the long input sequence problem, human-like summary generation, and the scarcity of labeled data. Transformer-based models, enhanced by parallel computation architectures and attention mechanisms, partially mitigate the challenges posed by long input sequences. Additionally, when trained on large-scale text corpora, these models achieve impressive performance in text generation, approaching human-level performance. Furthermore, pre-training with self-supervised learning objectives is an effective strategy to compensate for the scarcity of labeled data. Based on these considerations, this article leverages large corpora of unlabeled Vietnamese text to build a large-scale pseudo-labeled multi-document dataset, which is then used to pre-train a Vietnamese task-specific language model, LatVis. Experimental results demonstrate that, even without fine-tuning, the pre-trained model achieves competitive performance compared to several previous models. After fine-tuning on approximately 300 samples, LatVis obtains notable Rouge Scores of 76.7%, 78.9%, and 73.9% for Rouge-1 F1, and 50.2%, 55.0%, and 46.7% for Rouge-2 F1 on the VMDS, ViMS, VLSP datasets, respectively. To the best of our knowledge, this is the first publicly large-scale task-specific language model pre-trained specifically for the Vietnamese multi-document summarization. This work highlights the potential of task-specific language models for advancing natural language processing for low-resource languages.<\/jats:p>","DOI":"10.1145\/3725848","type":"journal-article","created":{"date-parts":[[2025,3,25]],"date-time":"2025-03-25T15:45:42Z","timestamp":1742917542000},"page":"1-19","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["LatVis: Large-scale Task-specific Language Model for Low-resource Vietnamese Multi-document Summarization"],"prefix":"10.1145","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0740-6380","authenticated-orcid":false,"given":"The Anh","family":"Le","sequence":"first","affiliation":[{"name":"FPT University, Can Tho Campus, Can Tho, Vietnam"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-2240-0451","authenticated-orcid":false,"given":"Hai Son","family":"Le","sequence":"additional","affiliation":[{"name":"Hanoi University of Science and Technology, Hanoi, Vietnam"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,4,23]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"Iz Beltagy Matthew E. Peters and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv:2004.05150. Retrieved from https:\/\/arxiv.org\/abs\/2004.05150"},{"key":"e_1_3_2_3_2","doi-asserted-by":"crossref","unstructured":"Long Doan Linh The Nguyen Nguyen Luong Tran Thai Hoang and Dat Quoc Nguyen. 2021. PhoMT: A High-Quality and Large-Scale Benchmark Dataset for Vietnamese-English Machine Translation. arXiv:2110.12199. Retrieved from https:\/\/arxiv.org\/abs\/2110.12199","DOI":"10.18653\/v1\/2021.emnlp-main.369"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/3366423.3380247"},{"key":"e_1_3_2_5_2","doi-asserted-by":"crossref","unstructured":"Taku Kudo. 2018. Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates. arXiv:1804.10959. Retrieved from https:\/\/arxiv.org\/abs\/1804.10959","DOI":"10.18653\/v1\/P18-1007"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-2012"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.703"},{"key":"e_1_3_2_8_2","first-page":"85","volume-title":"Proceedings of the 3rd Workshop on New Frontiers in Summarization","author":"Li Haoran","year":"2021","unstructured":"Haoran Li, Arash Einolghozati, Srinivasan Iyer, Bhargavi Paranjape, Yashar Mehdad, Sonal Gupta, and Marjan Ghazvininejad. 2021. EASE: Extractive-abstractive summarization with explanations. In Proceedings of the 3rd Workshop on New Frontiers in Summarization. Association for Computational Linguistics, 85\u201395."},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1387"},{"key":"e_1_3_2_10_2","first-page":"1065","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Short Papers)","author":"Liu Yixin","year":"2021","unstructured":"Yixin Liu and Pengfei Liu. 2021. SimCLS: A simple framework for contrastive learning of abstractive summarization. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Short Papers). Association for Computational Linguistics, 1065\u20131072."},{"key":"e_1_3_2_11_2","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics","author":"Liu Yixin","year":"2022","unstructured":"Yixin Liu, Pengfei Liu, Dragomir Radev, and Graham Neubig. 2022. BRIO: Bringing order to abstractive summarization. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2890\u2013290."},{"key":"e_1_3_2_12_2","unstructured":"Chinh Ngo Trieu H. Trinh Long Phan Hieu Tran Tai Dang Hieu Nguyen Minh Nguyen and Minh-Thang Luong. 2022. MTet: Multi-domain Translation for English and Vietnamese. arXiv:2210.05610. Retrieved from https:\/\/arxiv.org\/abs\/2210.05610"},{"key":"e_1_3_2_13_2","doi-asserted-by":"crossref","unstructured":"Dat Quoc Nguyen and Anh Tuan Nguyen. 2020. PhoBERT: Pre-trained language models for Vietnamese. arXiv:2003.00744. Retrieved from https:\/\/arxiv.org\/abs\/2003.00744","DOI":"10.18653\/v1\/2020.findings-emnlp.92"},{"key":"e_1_3_2_14_2","doi-asserted-by":"crossref","unstructured":"Linh The Nguyen and Dat Quoc Nguyen. 2021. PhoNLP: A joint multi-task learning model for Vietnamese part-of-speech tagging named entity recognition and dependency parsing. arXiv:2101.01476. Retrieved from https:\/\/arxiv.org\/pdf\/2101.01476","DOI":"10.18653\/v1\/2021.naacl-demos.1"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.naacl-srw.18"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/RIVF.2008.4586357"},{"key":"e_1_3_2_17_2","doi-asserted-by":"crossref","unstructured":"Matt Post. 2018. A Call for Clarity in Reporting BLEU Scores. arXiv:1804.08771. Retrieved from https:\/\/arxiv.org\/abs\/1804.08771","DOI":"10.18653\/v1\/W18-6319"},{"key":"e_1_3_2_18_2","first-page":"692","volume-title":"Proceedings of the 35th Pacific Asia Conference on Language, Information and Computation","author":"Quoc To Huy","year":"2021","unstructured":"To Huy Quoc, Kiet Van Nguyen, Ngan Luu-Thuy, and Nguyen Anh Gia-Tuan. 2021. Monolingual vs multilingual BERTology for Vietnamese extractive multi-document summarization. In Proceedings of the 35th Pacific Asia Conference on Language, Information and Computation. Association for Computational Lingustics, Shanghai, China, 692\u2013699. Retrieved from https:\/\/aclanthology.org\/2021.paclic-1.73"},{"issue":"140","key":"e_1_3_2_19_2","first-page":"1","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel Colin","year":"2020","unstructured":"Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research 21, 140 (2020), 1\u201367. Retrieved from http:\/\/jmlr.org\/papers\/v21\/20-074.html","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_20_2","doi-asserted-by":"crossref","unstructured":"Rico Sennrich Barry Haddow and Alexandra Birch. 2016. Neural Machine Translation of Rare Words with Subword Units. arXiv:1508.07909. Retrieved from https:\/\/arxiv.org\/abs\/1508.07909","DOI":"10.18653\/v1\/P16-1162"},{"key":"e_1_3_2_21_2","volume-title":"IFSR 2005: Proceedings of the 1st World Congress of the International Federation for Systems","author":"Thanh Le Ha","year":"2005","unstructured":"Le Ha Thanh, Thang Huynh Quyet, and Mai Luong Chi. 2005. A primary study on summarization of documents in Vietnamese. In IFSR 2005: Proceedings of the 1st World Congress of the International Federation for Systems. JAIST."},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2022-10177"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10579-020-09495-4"},{"key":"e_1_3_2_24_2","first-page":"186","volume-title":"Proceedings of the 7th International Conference on Knowledge and Systems Engineering (KSE\u201915)","author":"Ung Van-Giau","year":"2015","unstructured":"Van-Giau Ung, An-Vinh Luong, Nhi-Thao Tran, and Minh-Quoc Nghiem. 2015. Combination of features for Vietnamese news multi-document summarization. In Proceedings of the 7th International Conference on Knowledge and Systems Engineering (KSE\u201915). 186\u2013191."},{"key":"e_1_3_2_25_2","unstructured":"Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N. Gomez Lukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. arXiv:1706.03762. Retrieved from https:\/\/arxiv.org\/abs\/1706.03762"},{"key":"e_1_3_2_26_2","unstructured":"Thanh Vu Dat Quoc Nguyen Dai Quoc Nguyen Mark Dras and Mark Johnson. 2018. VnCoreNLP: A Vietnamese natural language processing toolkit. arXiv:1801.01331. Retrieved from https:\/\/arxiv.org\/abs\/1801.01331"},{"key":"e_1_3_2_27_2","first-page":"5245","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics","author":"Xiao Wen","year":"2022","unstructured":"Wen Xiao, Iz Beltagy, Giuseppe Carenini, and Arman Cohan. 2022. PRIMERA: Pyramid-based masked sentence pre-training for multi-document summarization. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. 5245\u20135263."},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.acl-long.360"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.451"},{"key":"e_1_3_2_30_2","unstructured":"Jingqing Zhang Yao Zhao Mohammad Saleh and Peter J. Liu. 2019. PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization. arXiv:1912.08777. Retrieved from https:\/\/arxiv.org\/abs\/1912.08777"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.552"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1100"}],"container-title":["ACM Transactions on Asian and Low-Resource Language Information Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3725848","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3725848","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:57:04Z","timestamp":1750298224000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3725848"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,23]]},"references-count":31,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2025,5,31]]}},"alternative-id":["10.1145\/3725848"],"URL":"https:\/\/doi.org\/10.1145\/3725848","relation":{},"ISSN":["2375-4699","2375-4702"],"issn-type":[{"type":"print","value":"2375-4699"},{"type":"electronic","value":"2375-4702"}],"subject":[],"published":{"date-parts":[[2025,4,23]]},"assertion":[{"value":"2023-12-11","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-03-17","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-04-23","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}