{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,5]],"date-time":"2026-10-05T18:51:03Z","timestamp":1791226263841,"version":"4.1.0"},"reference-count":38,"publisher":"Public Library of Science (PLoS)","issue":"10","license":[{"start":{"date-parts":[[2026,10,5]],"date-time":"2026-10-05T00:00:00Z","timestamp":1791158400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS One"],"abstract":"<jats:p>This work aims to develop a domain-adapted Bengali text summarization model by training and fine-tuning on general and domain-specific datasets with categories such as state, international, and sports. Flan-T5 and mT5 LLMs were trained on the XLSUM Bengali dataset for text summarization, and they were trained on source domains and fine-tuned on the target domains for domain adaptation. In general-domain summarization on the XLSum Bengali dataset, mT5 achieved stronger overall performance than Flan-T5, with ROUGE-1 and BERTScore values of 0.21 and 0.72, respectively. The Flan-T5-XLSUM model outperformed other LLMs, achieving a ROUGE score of 0.79, a BLEU score of 0.13 and a BERTScore of 0.86. A human evaluation involving 28 participants was conducted to assess summary fluency, adequacy, and factual consistency across domains, confirming the qualitative effectiveness of the proposed models. A custom dataset is developed with manually annotated categories for domain adaptation purposes, contributing to domain-specific Bengali data. The ablation study illustrated that pre-training and fine-tuning on domain-specific data significantly enhanced model performance, with Flan-T5 fine-tuned on sports data. Three explainability methods (LIME, SHAP, and BertViz) revealed that geographically and contextually meaningful tokens strongly influenced the domain-adaptive Flan-T5 Bengali summarization model. The adapter-based Flan-T5 architecture enables effective multi-domain fine-tuning with less than 1% of trainable parameters while preserving model performance.<\/jats:p>","DOI":"10.1371\/journal.pone.0357890","type":"journal-article","created":{"date-parts":[[2026,10,5]],"date-time":"2026-10-05T17:56:39Z","timestamp":1791222999000},"page":"e0357890","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":0,"title":["Domain-adaptive Bengali text summarization: Harnessing transformer-based large language models in closed-set contexts"],"prefix":"10.1371","volume":"21","author":[{"given":"Anusree","family":"Roy","sequence":"first","affiliation":[{"name":"North South University","place":["Dhaka, Bangladesh"],"department":["Electrical and Computer Engineering"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fahrin Hossain","family":"Sunaira","sequence":"additional","affiliation":[{"name":"North South University","place":["Dhaka, Bangladesh"],"department":["Electrical and Computer Engineering"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tateyama","family":"Orpa","sequence":"additional","affiliation":[{"name":"North South University","place":["Dhaka, Bangladesh"],"department":["Electrical and Computer Engineering"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Farasha Shamma","family":"Yussouf","sequence":"additional","affiliation":[{"name":"North South University","place":["Dhaka, Bangladesh"],"department":["Electrical and Computer Engineering"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Intisar Tahmid","family":"Naheen","sequence":"additional","affiliation":[{"name":"North South University","place":["Dhaka, Bangladesh"],"department":["Electrical and Computer Engineering"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5429-2235","authenticated-orcid":true,"given":"Riasat","family":"Khan","sequence":"additional","affiliation":[{"name":"North South University","place":["Dhaka, Bangladesh"],"department":["Electrical and Computer Engineering"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2026,10,5]]},"reference":[{"key":"pone.0357890.ref001","doi-asserted-by":"crossref","unstructured":"Abualigah L, Bashabsheh MQ, Alabool H, Shehab M. Text summarization: a brief review. Recent Advances in NLP: the case of Arabic language. 2020. p. 1\u201315.","DOI":"10.1007\/978-3-030-34614-0_1"},{"key":"pone.0357890.ref002","unstructured":"Islam MS. Research on Bangla language processing in Bangladesh: progress and challenges. In: International Language and Development Conference, 2009. 23\u20135."},{"key":"pone.0357890.ref003","doi-asserted-by":"crossref","unstructured":"Dhar N, Saha G, Bhattacharjee P, Mallick A, Islam MS. Pointer over attention: An improved Bangla text summarization approach using hybrid pointer generator network. In: International Conference on Computer and Information Technology. IEEE; 2021. p. 1\u20135.","DOI":"10.1109\/ICCIT54785.2021.9689852"},{"key":"pone.0357890.ref004","first-page":"153","article-title":"Rank Your Summaries: Enhancing Bengali Text Summarization Via Ranking-Based Approach","volume-title":"Lecture Notes in Networks and Systems","author":"GM Shahariar","year":"2024"},{"key":"pone.0357890.ref005","first-page":"41","article-title":"Bengali Abstractive News Summarization (BANS): A Neural Attention Approach","volume-title":"Advances in Intelligent Systems and Computing","author":"P Bhattacharjee","year":"2020"},{"issue":"1","key":"pone.0357890.ref006","doi-asserted-by":"crossref","DOI":"10.1007\/s42979-022-01446-w","article-title":"Automatic Text Summarization Methods: A Comprehensive Review","volume":"4","author":"G Sharma","year":"2022","journal-title":"SN COMPUT SCI"},{"key":"pone.0357890.ref007","doi-asserted-by":"crossref","unstructured":"Afzal A, Chalumattu R, Matthes F, Mascarell L. AdaptEval: Evaluating Large Language Models on Domain Adaptation for Text Summarization. In: In: Workshop on Customizable NLP: Progress and Challenges in Customizing NLP for a Domain, Application, Group, or Individual. 2024. 76\u201385.","DOI":"10.18653\/v1\/2024.customnlp4u-1.8"},{"key":"pone.0357890.ref008","article-title":"Gradual domain adaptation: Theory and algorithms","volume":"25","author":"Y He","year":"2024","journal-title":"Journal of Machine Learning Research"},{"key":"pone.0357890.ref009","doi-asserted-by":"crossref","unstructured":"Li Y, Miao S, Huang H, Gao Y. Word Matters: What Influences Domain Adaptation in Summarization? In: Annual Meeting of the Association for Computational Linguistics. 2024. 13236\u201349.","DOI":"10.18653\/v1\/2024.acl-long.715"},{"key":"pone.0357890.ref010","doi-asserted-by":"crossref","first-page":"19302","DOI":"10.1109\/ACCESS.2025.3528830","article-title":"Advanced Text Summarization Model Incorporating NLP Techniques and Feature-Based Scoring","volume":"13","author":"E Abdulreda Kadhim","year":"2025","journal-title":"IEEE Access"},{"key":"pone.0357890.ref011","first-page":"133","article-title":"To Adapt or to Fine-Tune: A Case Study on Abstractive Summarization","volume-title":"Lecture Notes in Computer Science","author":"Z Zhao","year":"2022"},{"key":"pone.0357890.ref012","doi-asserted-by":"crossref","unstructured":"Yu T, Liu Z, Fung P. AdaptSum: Towards Low-Resource Domain Adaptation for Abstractive Summarization. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021. 5892\u2013904. https:\/\/doi.org\/10.18653\/v1\/2021.naacl-main.471","DOI":"10.18653\/v1\/2021.naacl-main.471"},{"key":"pone.0357890.ref013","article-title":"Text summarization using large language models: a comparative study of MPT-7B-instruct, Falcon-7b-instruct, and OpenAI Chat-GPT models","author":"L Basyal","year":"2023","journal-title":"arXiv preprint"},{"key":"pone.0357890.ref014","unstructured":"Borah M, Dadure P, Pakray P, et al. Comparative analysis of T5 model for abstractive text summarization on different datasets. In: International Conference on Innovative Computing and Communication. 2022."},{"key":"pone.0357890.ref015","doi-asserted-by":"crossref","unstructured":"Lewis M, Liu Y, Goyal N, Ghazvininejad M, Mohamed A, Levy O, et al. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020. 7871\u201380. https:\/\/doi.org\/10.18653\/v1\/2020.acl-main.703","DOI":"10.18653\/v1\/2020.acl-main.703"},{"key":"pone.0357890.ref016","doi-asserted-by":"crossref","unstructured":"Song H, Su H, Shalyminov I, Cai J, Mansour S. FineSurE: Fine-grained summarization evaluation using LLMs. Annual Meeting of the Association for Computational Linguistics. In: 2024. 906\u201322.","DOI":"10.18653\/v1\/2024.acl-long.51"},{"key":"pone.0357890.ref017","doi-asserted-by":"crossref","unstructured":"Kolagar Z, Zarcone A. Aligning Uncertainty: Leveraging LLMs to Analyze Uncertainty Transfer in Text Summarization. In: Workshop on Uncertainty-Aware NLP. 2024. 41\u201361.","DOI":"10.18653\/v1\/2024.uncertainlp-1.5"},{"key":"pone.0357890.ref018","doi-asserted-by":"crossref","first-page":"91855","DOI":"10.1109\/ACCESS.2022.3197662","article-title":"Bangla-BERT: Transformer-Based Efficient Model for Transfer Learning and Language Understanding","volume":"10","author":"M Kowsher","year":"2022","journal-title":"IEEE Access"},{"key":"pone.0357890.ref019","doi-asserted-by":"crossref","unstructured":"Chowdhury RR, Nayeem MT, Mim TT, Chowdhury MdSR, Jannat T. Unsupervised Abstractive Summarization of Bengali Text Documents. In: Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, 2021. 2612\u20139. https:\/\/doi.org\/10.18653\/v1\/2021.eacl-main.224","DOI":"10.18653\/v1\/2021.eacl-main.224"},{"key":"pone.0357890.ref020","doi-asserted-by":"crossref","unstructured":"Jahan B, Mahtab SS, Faizul Huq Arif M, Emon IS, Milu SA, Julfiker Raju M. An automated Bengali text summarization technique using lexicon-based approach. In: International Conference on Innovation in Computer Science and Engineering, 2021. 363\u201373.","DOI":"10.1007\/978-981-33-4543-0_39"},{"key":"pone.0357890.ref021","unstructured":"MAT R, Islam MS. Evaluating Large Language Models for Summarizing Bangla Texts. In: Widening NLP Workshop. 2024."},{"issue":"5","key":"pone.0357890.ref022","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3722109","article-title":"ConVerSum: A Contrastive Learning-Based Approach for Data-Scarce Solution of Cross-Lingual Summarization Beyond Direct Equivalents","volume":"24","author":"SK Lora","year":"2025","journal-title":"ACM Trans Asian Low-Resour Lang Inf Process"},{"key":"pone.0357890.ref023","doi-asserted-by":"crossref","unstructured":"Parnell J, Unanue IJ, Piccardi M. SumTra: A Differentiable Pipeline for Few-Shot Cross-Lingual Summarization. In: Conference of the North American Chapter of the Association for Computational Linguistics. 2024. 2399\u2013415.","DOI":"10.18653\/v1\/2024.naacl-long.133"},{"key":"pone.0357890.ref024","doi-asserted-by":"crossref","unstructured":"Raihan N, Zampieri M. TigerLLM - A Family of Bangla Large Language Models. In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2025. 887\u201396. https:\/\/doi.org\/10.18653\/v1\/2025.acl-short.69","DOI":"10.18653\/v1\/2025.acl-short.69"},{"issue":"2","key":"pone.0357890.ref025","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1109\/TPAMI.2018.2880750","article-title":"Open Set Domain Adaptation for Image and Action Recognition","volume":"42","author":"P Panareda Busto","year":"2020","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"pone.0357890.ref026","doi-asserted-by":"crossref","unstructured":"Cao Z, Long M, Wang J, Jordan MI. Partial Transfer Learning with Selective Adversarial Networks. In: 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, 2018. 2724\u201332. https:\/\/doi.org\/10.1109\/cvpr.2018.00288","DOI":"10.1109\/CVPR.2018.00288"},{"key":"pone.0357890.ref027","doi-asserted-by":"crossref","unstructured":"You K, Long M, Cao Z, Wang J, Jordan MI. Universal domain adaptation. In: Conference on Computer Vision and Pattern Recognition. 2019. 2720\u20139.","DOI":"10.1109\/CVPR.2019.00283"},{"key":"pone.0357890.ref028","article-title":"Fine-tuning large language models for domain adaptation: exploration of training strategies, scaling, model merging and synergistic capabilities","author":"W Lu","year":"2025","journal-title":"Scientific Reports"},{"key":"pone.0357890.ref029","doi-asserted-by":"crossref","first-page":"4851828","DOI":"10.1155\/2022\/4851828","article-title":"MTL-DAS: Automatic Text Summarization for Domain Adaptation","volume":"2022","author":"J Zhong","year":"2022","journal-title":"Comput Intell Neurosci"},{"key":"pone.0357890.ref030","doi-asserted-by":"crossref","unstructured":"Mao Y, Zhong M, Han J. CiteSum: Citation Text-guided Scientific Extreme Summarization and Domain Adaptation with Limited Supervision. In: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022. 10922\u201335. https:\/\/doi.org\/10.18653\/v1\/2022.emnlp-main.750","DOI":"10.18653\/v1\/2022.emnlp-main.750"},{"key":"pone.0357890.ref031","doi-asserted-by":"crossref","first-page":"181381","DOI":"10.1109\/ACCESS.2019.2958736","article-title":"A Kernelized Unified Framework for Domain Adaptation","volume":"7","author":"RK Sanodiya","year":"2019","journal-title":"IEEE Access"},{"issue":"3","key":"pone.0357890.ref032","doi-asserted-by":"crossref","first-page":"4029","DOI":"10.1109\/TCSS.2023.3347520","article-title":"Online Summarization of Microblog Data: An Aid in Handling Disaster Situations","volume":"11","author":"D Paul","year":"2024","journal-title":"IEEE Trans Comput Soc Syst"},{"key":"pone.0357890.ref033","doi-asserted-by":"crossref","DOI":"10.3390\/math12223576","article-title":"TTG-Text: A Graph-Based Text Representation Framework Enhanced by Typical Testors for Improved Classification","volume":"12","author":"C S\u00e1nchez-Antonio","year":"2024","journal-title":"Mathematics"},{"issue":"10","key":"pone.0357890.ref034","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pone.0222870","article-title":"Forma mentis networks quantify crucial differences in STEM perception between students and experts","volume":"14","author":"M Stella","year":"2019","journal-title":"PLoS One"},{"key":"pone.0357890.ref035","doi-asserted-by":"crossref","unstructured":"Xue L, Constant N, Roberts A, Kale M, Al-Rfou R, Siddhant A, et al. mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer. In: Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021. 483\u201398. https:\/\/aclanthology.org\/2021.naacl-main.41\/","DOI":"10.18653\/v1\/2021.naacl-main.41"},{"key":"pone.0357890.ref036","article-title":"Scaling instruction-finetuned language models","volume":"25","author":"HW Chung","year":"2024","journal-title":"Journal of Machine Learning Research"},{"key":"pone.0357890.ref037","doi-asserted-by":"crossref","unstructured":"Hasan T, Bhattacharjee A, Islam MS, Mubasshir K, Li YF, Kang YB, et al. XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages. In: International Joint Conference on Natural Language Processing, 2021. 4693\u2013703. https:\/\/aclanthology.org\/2021.findings-acl.413\/","DOI":"10.18653\/v1\/2021.findings-acl.413"},{"key":"pone.0357890.ref038","article-title":"ChartSumm: A Comprehensive Benchmark for Automatic Chart Summarization of Long and Short Summaries","author":"R Rahman","year":"2023","journal-title":"Proceedings of the Canadian Conference on Artificial Intelligence"}],"container-title":["PLOS One"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0357890","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,5]],"date-time":"2026-10-05T17:56:49Z","timestamp":1791223009000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0357890"}},"subtitle":[],"editor":[{"given":"Shuai","family":"Liu","sequence":"first","affiliation":[{"name":"Hunan Normal University","place":["CHINA"]}],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026,10,5]]},"references-count":38,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2026,10,5]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0357890","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,5]]}}}