{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,2]],"date-time":"2026-01-02T07:39:26Z","timestamp":1767339566915,"version":"build-2065373602"},"reference-count":117,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2025,4,28]],"date-time":"2025-04-28T00:00:00Z","timestamp":1745798400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["BDCC"],"abstract":"<jats:p>Text summarization is a complex and essential task in natural language processing (NLP) research, focused on extracting the most important information from a document. This study focuses on the Extractive and Abstractive approaches of Bengali Text Summarization (BTS). With the breakthrough advancements in deep learning, summarization is no longer a major challenge for English, given the availability of extensive resources dedicated to this global language. However, the Bengali language remains underexplored. Hence, in this work, a comprehensive review has been conducted on BTS research from 2007 to 2023, analyzing trends, datasets, preprocessing techniques, methodologies, evaluations, and challenges. Leveraging 106 journal and conference papers, this review offers insights into emerging topics and trends in Bengali Abstractive summarization. The review has been augmented with experiments using transformer models from Hugging Face and publicly available datasets to assess the Rouge score accuracy for Abstractive summarization. The extensive literature review conducted in this study reveals that before the advent of transformers, LSTM (Long Short-Term Memory) models were the dominant deep learning approach for text summarization across various languages. For transformers, one of the key datasets utilized was XL-SUM with the MT5 model emerging as the best performer among various contemporary multilingual models. These findings contribute to understanding the contemporary techniques and challenges in BTS. Furthermore, recommendations are made to guide future research endeavors, aiming to provide valuable insights and directions for researchers in this field.<\/jats:p>","DOI":"10.3390\/bdcc9050117","type":"journal-article","created":{"date-parts":[[2025,4,28]],"date-time":"2025-04-28T09:39:47Z","timestamp":1745833187000},"page":"117","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["A Review of the State-of-the-Art Techniques and Analysis of Transformers for Bengali Text Summarization"],"prefix":"10.3390","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3065-2486","authenticated-orcid":false,"given":"MD Iftekharul","family":"Mobin","sequence":"first","affiliation":[{"name":"Department of Computer Science, American International University-Bangladesh, Dhaka 1229, Bangladesh"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-1601-8793","authenticated-orcid":false,"given":"Mahamodul Hasan","family":"Mahadi","sequence":"additional","affiliation":[{"name":"Department of Computer Science, American International University-Bangladesh, Dhaka 1229, Bangladesh"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6572-3451","authenticated-orcid":false,"given":"Al-Sakib Khan","family":"Pathan","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, United International University, Madani Avenue, Dhaka 1212, Bangladesh"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2675-1684","authenticated-orcid":false,"given":"A. F. M. Suaib","family":"Akhter","sequence":"additional","affiliation":[{"name":"Department of Computer Engineering, Sakarya University of Applied Sciences, Serdivan 54050, Sakarya, Turkey"}]}],"member":"1968","published-online":{"date-parts":[[2025,4,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"113679","DOI":"10.1016\/j.eswa.2020.113679","article-title":"Automatic Text Summarization: A Comprehensive Survey","volume":"165","author":"Salama","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1029","DOI":"10.1016\/j.jksuci.2020.05.006","article-title":"Review of Automatic Text Summarization Techniques & Methods","volume":"34","author":"Widyassari","year":"2022","journal-title":"J. King Saud-Univ.-Comput. Inf. Sci."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"5755","DOI":"10.1016\/j.eswa.2013.04.023","article-title":"Assessing Sentence Scoring Techniques for Extractive Text Summarization","volume":"40","author":"Ferreira","year":"2013","journal-title":"Expert Syst. Appl."},{"key":"ref_4","first-page":"251","article-title":"A review on text summarization techniques","volume":"64","author":"Verma","year":"2020","journal-title":"J. Sci. Res."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s10462-016-9475-9","article-title":"Recent automatic text summarization techniques: A survey","volume":"47","author":"Gambhir","year":"2017","journal-title":"Artif. Intell. Rev."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"156043","DOI":"10.1109\/ACCESS.2021.3129786","article-title":"A survey of automatic text summarization: Progress, process and challenges","volume":"9","author":"Mridha","year":"2021","journal-title":"IEEE Access"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"100647","DOI":"10.1016\/j.websem.2021.100647","article-title":"Entity summarization: State of the art and future challenges","volume":"69","author":"Liu","year":"2021","journal-title":"J. Web Semant."},{"key":"ref_8","first-page":"23305","article-title":"Abstractive text summarization employing ontology-based knowledge-aware multi-focus conditional generative adversarial network (OKAM-CGAN) with hybrid pre-processing methodology","volume":"82","author":"Muneera","year":"2022","journal-title":"Multimed. Tools Appl."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Yadav, D., Desai, J., and Yadav, A.K. (2022). Automatic text summarization methods: A comprehensive review. arXiv.","DOI":"10.1109\/ACCESS.2022.3231016"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1007\/s42979-022-01446-w","article-title":"Automatic Text Summarization Methods: A Comprehensive Review","volume":"4","author":"Sharma","year":"2022","journal-title":"SN Comput. Sci."},{"key":"ref_11","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Yenduri, G., Srivastava, G., Maddikunta, P.K.R., Jhaveri, R.H., Wang, W., Vasilakos, A.V., and Gadekallu, T.R. (2023). Generative Pre-trained Transformer: A Comprehensive Review on Enabling Technologies, Potential Applications, Emerging Challenges, and Future Directions. arXiv.","DOI":"10.1109\/ACCESS.2024.3389497"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1016\/j.aiopen.2022.10.001","article-title":"A survey of transformers","volume":"3","author":"Lin","year":"2022","journal-title":"AI Open"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2477","DOI":"10.1007\/s10462-023-10582-5","article-title":"From task to evaluation: An automatic text summarization review","volume":"56","author":"Lu","year":"2023","journal-title":"Artif. Intell. Rev."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Bhandari, M., Gour, P.N., Ashfaq, A., Liu, P., and Neubig, G. Re-evaluating Evaluation in Text Summarization. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP).","DOI":"10.18653\/v1\/2020.emnlp-main.751"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Khilji, A.F.U.R., Sinha, U., Singh, P., Ali, A., and Pakray, P. (2021). Abstractive Text Summarization Approaches with Analysis of Evaluation Techniques. Proceedings of the International Conference on Computational Intelligence in Communications and Business Analytics, Springer International Publishing.","DOI":"10.1007\/978-3-030-75529-4_19"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Mukta, A.P., Mamun, A.A., Basak, C., Nahar, S., and Arif, M.F.H. (2019, January 7\u20139). A phrase-based machine translation from English to Bangla using rule-based approach. Proceedings of the 2019 International Conference on Electrical, Computer and Communication Engineering (ECCE), Cox\u2019s Bazar, Bangladesh.","DOI":"10.1109\/ECACE.2019.8679456"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Hossain, N., and Ahnaf, A. (2021, January 1\u20132). BERT-based Text Simplification Approach to Reduce Linguistic Complexity of Bangla Language. Proceedings of the 2021 International Conference on Intelligent Technology, System and Service for Internet of Everything (ITSS-IoE), Sana\u2019a, Yemen.","DOI":"10.1109\/ITSS-IoE53029.2021.9615303"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Shetu, S.F., Saifuzzaman, M., Parvin, M., Moon, N.N., Yousuf, R., and Sultana, S. (July, January 1\u20133). Identifying the Writing Style of Bangla Language using Natural Language Processing. Proceedings of the 2020 11th International Conference on Computing, Communication and Networking Technologies (ICCCNT), Kharagpur, India.","DOI":"10.1109\/ICCCNT49239.2020.9225670"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"38999","DOI":"10.1109\/ACCESS.2022.3165563","article-title":"Bangla Natural Language Processing: A Comprehensive Analysis of Classical, Machine Learning, and Deep Learning-Based Methods","volume":"10","author":"Sen","year":"2022","journal-title":"IEEE Access"},{"key":"ref_21","first-page":"288","article-title":"Automatic text summarization for Bengali language including grammatical analysis","volume":"8","author":"Sikder","year":"2019","journal-title":"Int. J. Sci. Technol. Res."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"3431","DOI":"10.1007\/s10462-021-10083-3","article-title":"A study on the challenges and opportunities of speech recognition for Bengali language","volume":"55","author":"Mridha","year":"2022","journal-title":"Artif. Intell. Rev."},{"key":"ref_23","unstructured":"Masum, A.K.M., Abujar, S., Talukder, M.A.I., Rabby, A.S.A., and Hossain, S.A. (2019, January 6\u20138). Abstractive method of text summarization with sequence to sequence RNNs. Proceedings of the 2019 10th international conference on computing, communication and networking technologies (ICCCNT), Kanpur, India."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Chowdhury, R.R., Nayeem, M.T., Mim, T.T., Chowdhury, M.S.R., and Jannat, T. (2021). Unsupervised abstractive summarization of bengali text documents. arXiv.","DOI":"10.18653\/v1\/2021.eacl-main.224"},{"key":"ref_25","unstructured":"Mukherjee, A. (2022). Developing Bengali Text Summarization with Transformer Base Model. [Doctoral Dissertation, National College of Ireland]."},{"key":"ref_26","unstructured":"King, B.P. (2015). Practical Natural Language Processing for Low-Resource Languages. [Ph.D. Dissertation, University of Michigan]."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Bhattacharjee, A., Hasan, T., Ahmad, W.U., and Shahriyar, R. (2022). Banglanlg: Benchmarks and resources for evaluating low-resource natural language generation in bangla. arXiv.","DOI":"10.18653\/v1\/2023.findings-eacl.54"},{"key":"ref_28","unstructured":"Alam, F., Hasan, A., Alam, T., Khan, A., Tajrin, J., Khan, N., and Chowdhury, S.A. (2021). A review of Bangla natural language processing tasks and the utility of transformer models. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Karim, M.A. (2013). Technical Challenges and Design Issues in Bangla Language Processing, IGI Global.","DOI":"10.4018\/978-1-4666-3970-6"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Gupta, A., Vavre, A., and Sarawagi, S. (, January June). Training data augmentation for code-mixed translation. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Online.","DOI":"10.18653\/v1\/2021.naacl-main.459"},{"key":"ref_31","unstructured":"Hoque, O.B., Jubair, M.I., Akash, A.A., and Islam, S. (December, January 30). Bdsl36: A dataset for Bangladeshi sign letters recognition. Proceedings of the Asian Conference on Computer Vision, Kyoto, Japan."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Zhang, X., Wei, F., and Zhou, M. (2019). HIBERT: Document level pre-training of hierarchical bidirectional transformers for document summarization. arXiv.","DOI":"10.18653\/v1\/P19-1499"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Sen, O., Fuad, M., Islam, N., Rabbi, J., Hasan, K., Awal, A., Fime, A.A., Fuad, T.H., Sikder, D., and Iftee, D.A.R. (2021). Bangla Natural Language Processing: A Comprehensive Review of Classical Machine Learning and Deep Learning Based Methods. arXiv.","DOI":"10.1109\/ACCESS.2022.3165563"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Hasan, T., Bhattacharjee, A., Samin, K., Hasan, M., Basak, M., Rahman, M.S., and Shahriyar, R. (November, January 16\u201320). Not Low-Resource Anymore: Aligner Ensembling, Batch Filtering, and New Datasets for Bengali-English Machine Translation. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online.","DOI":"10.18653\/v1\/2020.emnlp-main.207"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Uddin, M.N., and Khan, S.A. (2007, January 27\u201329). A study on text summarization techniques and implement few of them for Bangla language. Proceedings of the 2007 10th International Conference on Computer and Information Technology (ICCIT), Dhaka, Bangladesh.","DOI":"10.1109\/ICCITECHN.2007.4579374"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Efat, M.I.A., Ibrahim, M., and Kayesh, H. (2013, January 17\u201318). Automated Bangla text summarization by sentence scoring and ranking. Proceedings of the 2013 International Conference on Informatics, Electronics and Vision (ICIEV), Dhaka, Bangladesh.","DOI":"10.1109\/ICIEV.2013.6572686"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Haque, M.M., Pervin, S., and Begum, Z. (2015, January 21\u201323). Automatic Bengali news documents summarization by introducing sentence frequency and clustering. Proceedings of the 2015 18th International Conference on Computer and Information Technology (ICCIT), Dhaka, Bangladesh.","DOI":"10.1109\/ICCITechn.2015.7488060"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Haque, M.M., Pervin, S., and Begum, Z. (2016, January 22\u201325). Enhancement of keyphrase-based approach of automatic Bangla text summarization. Proceedings of the 2016 IEEE Region 10 Conference (TENCON), Singapore.","DOI":"10.1109\/TENCON.2016.7847955"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Paul, A., Imtiaz, M.T., Latif, A.H., Ahmed, M., Adnan, F.A., Khan, R., Kadery, I., and Rahman, R.M. (2017, January 27\u201329). Bangla news summarization. Proceedings of the Computational Collective Intelligence: 9th International Conference, ICCCI 2017, Nicosia, Cyprus. Proceedings, Part II 9.","DOI":"10.1007\/978-3-319-67077-5_46"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Abujar, S., Hasan, M., Shahin, M., and Hossain, S.A. (2017, January 3\u20135). A heuristic approach of text summarization for Bengali documentation. Proceedings of the 2017 8th International Conference on Computing, Communication and Networking Technologies (ICCCNT), Delhi, India.","DOI":"10.1109\/ICCCNT.2017.8204166"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Akter, S., Asa, A.S., Uddin, M.P., Hossain, M.D., Roy, S.K., and Afjal, M.I. (2017, January 21\u201326). An extractive text summarization technique for Bengali document(s) using K-means clustering algorithm. Proceedings of the 2017 IEEE International Conference on Imaging, Vision & Pattern Recognition (icivpr), Honolulu, HI, USA.","DOI":"10.1109\/ICIVPR.2017.7890883"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Chowdhury, S.R., Sarkar, K., and Dam, S. (2017, January 27\u201329). An approach to generic Bengali text summarization using latent semantic analysis. Proceedings of the 2017 International Conference on Information Technology (ICIT), Singapore.","DOI":"10.1109\/ICIT.2017.12"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"44","DOI":"10.5815\/ijmecs.2018.12.06","article-title":"A rule based extractive text summarization technique for Bangla news documents","volume":"10","author":"Ghosh","year":"2018","journal-title":"Int. J. Mod. Educ. Comput. Sci."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Tumpa, P., Yeasmin, S., Nitu, A., Uddin, M., Afjal, M., and Mamun, M. (2018, January 8\u20139). An improved extractive summarization technique for bengali text(s). Proceedings of the 2018 International Conference on Computer, Communication, Chemical, Material and Electronic Engineering (IC4ME2), Dhaka, Bangladesh.","DOI":"10.1109\/IC4ME2.2018.8465609"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Chandro, P., Arif, M.F.H., Rahman, M.M., Siddik, M.S., Rahman, M.S., and Rahman, M.A. (2018, January 21\u201323). Automated bengali document summarization by collaborating individual word & sentence scoring. Proceedings of the 2018 21st International Conference of Computer and Information Technology (ICCIT), Dhaka, Bangladesh.","DOI":"10.1109\/ICCITECHN.2018.8631926"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Sarkar, A., and Hossen, M.S. (2018, January 21\u201323). Automatic bangla text summarization using term frequency and semantic similarity approach. Proceedings of the 2018 21st International Conference of Computer and Information Technology (ICCIT), Dhaka, Bangladesh.","DOI":"10.1109\/ICCITECHN.2018.8631934"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Talukder, M.A.I., Abujar, S., Masum, A.K.M., Faisal, F., and Hossain, S.A. (2019, January 6\u20138). Bengali abstractive text summarization using sequence to sequence RNNs. Proceedings of the 2019 10th International Conference on Computing, Communication and Networking Technologies (ICCCNT), Kanpur, India.","DOI":"10.1109\/ICCCNT45670.2019.8944839"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Abujar, S., Masum, A.K.M., Mohibullah, M., and Hossain, S.A. (2019, January 6\u20138). An approach for bengali text summarization using word2vector. Proceedings of the 2019 10th International Conference on Computing, Communication and Networking Technologies (ICCCNT), Kanpur, India.","DOI":"10.1109\/ICCCNT45670.2019.8944536"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Masum, A.K.M., Abujar, S., Tusher, R.T.H., Faisal, F., and Hossain, S.A. (2019, January 6\u20138). Sentence similarity measurement for Bengali abstractive text summarization. Proceedings of the 2019 10th International Conference on Computing, Communication and Networking Technologies (ICCCNT), Kanpur, India.","DOI":"10.1109\/ICCCNT45670.2019.8944571"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Rahman, A., Rafiq, F.M., Saha, R., Rafian, R., and Arif, H. (2019, January 7\u20139). Bengali text summarization using TextRank, fuzzy C-Means and aggregate scoring methods. Proceedings of the 2019 IEEE Region 10 Symposium (TENSYMP), Kolkata, India.","DOI":"10.1109\/TENSYMP46218.2019.8971039"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1007\/978-981-15-2043-3_55","article-title":"A bengali text generation approach in context of abstractive text summarization using rnn","volume":"103","author":"Abujar","year":"2020","journal-title":"Innov. Comput. Sci. Eng. Proc. 7th Icicse"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Bhattacharjee, P., Mallick, A., and Saiful Islam, M. Bengali abstractive news summarization (BANS): A neural attention approach. Proceedings of the International Conference on Trends in Computational and Cognitive Engineering: Proceedings of TCCE 2020.","DOI":"10.1007\/978-981-33-4673-4_4"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"539","DOI":"10.1007\/978-981-33-4367-2_51","article-title":"A Bengali Text Summarization Using Encoder-Decoder Based on Social Media Dataset","volume":"Volume 2","author":"Fouzia","year":"2021","journal-title":"Proceedings of the Emerging Technologies in Data Mining and Information Security: Proceedings of IEMIS 2020"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1007\/978-981-33-4367-2_31","article-title":"Abstraction Based Bengali Text Summarization Using Bi-directional Attentive Recurrent Neural Networks","volume":"Volume 2","author":"Islam","year":"2021","journal-title":"Proceedings of the Emerging Technologies in Data Mining and Information Security: Proceedings of IEMIS 2020"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Gupta, A., Chugh, D., and Katarya, R. (2022, January 31). Automated news summarization using transformers. Proceedings of the Sustainable Advanced Computing: Select Proceedings of ICSAC 2021, LNEE, Singapore.","DOI":"10.1007\/978-981-16-9012-9_21"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Ranganathan, J., and Abuka, G. (December, January 28). Text summarization using transformer model. Proceedings of the 2022 Ninth International Conference on Social Networks Analysis, Management and Security (SNAMS), Milan, Italy.","DOI":"10.1109\/SNAMS58071.2022.10062698"},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"18603","DOI":"10.1007\/s00521-023-08687-7","article-title":"An Abstractive Text Summarization Technique Using Transformer Model with Self-Attention Mechanism","volume":"35","author":"Kumar","year":"2023","journal-title":"Neural Comput. Appl."},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Sultana, M., Chakraborty, P., and Choudhury, T. (2022). Bengali abstractive news summarization using Seq2Seq learning with attention. Proceedings of the Cyber Intelligence and Information Retrieval: Proceedings of CIIR 2021, Springer.","DOI":"10.1007\/978-981-16-4284-5_24"},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Hoque Barsha, F.A., and Uddin, M.N. (2023, January 21\u201323). Comparative Analysis of BanglaT5 and Pointer Generator Network for Bengali Abstractive Story Summarization. Proceedings of the 2023 International Conference on Information and Communication Technology for Sustainable Development (ICICT4SD), Dhaka, Bangladesh.","DOI":"10.1109\/ICICT4SD59951.2023.10303633"},{"key":"ref_60","unstructured":"Vlachos, A., and Augenstein, I. (2023, January 2\u20136). BanglaNLG and BanglaT5: Benchmarks and Resources for Evaluating Low-Resource Natural Language Generation in Bangla. Proceedings of the Findings of the Association for Computational Linguistics: EACL 2023, Dubrovnik, Croatia."},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Landro, N., Gallo, I., La Grassa, R., and Federici, E. (2022). Two New Datasets for Italian-Language Abstractive Text Summarization. Information, 13.","DOI":"10.3390\/info13050228"},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Phan, L., Tran, H., Nguyen, H., and Trinh, T.H. (2022). Vit5: Pretrained Text-to-Text Transformer for Vietnamese Language Generation. arXiv.","DOI":"10.18653\/v1\/2022.naacl-srw.18"},{"key":"ref_63","unstructured":"Khandelwal, U., Clark, K., Jurafsky, D., and Kaiser, L. (2019). Sample Efficient Text Summarization using a Single Pre-trained Transformer. arXiv."},{"key":"ref_64","doi-asserted-by":"crossref","unstructured":"Guan, W., Smetannikov, I., and Tianxing, M. (2020, January 27\u201329). Survey on automatic text summarization and transformer models applicability. Proceedings of the 2020 1st International Conference on Control, Robotics and Intelligent System, Xiamen, China.","DOI":"10.1145\/3437802.3437832"},{"key":"ref_65","unstructured":"Zhao, Y., Khalman, M., Joshi, R., Narayan, S., Saleh, M., and Liu, P.J. (2022). Calibrating sequence likelihood improves conditional language generation. arXiv."},{"key":"ref_66","doi-asserted-by":"crossref","first-page":"4111","DOI":"10.1007\/s11063-022-11031-0","article-title":"A Step-by-Step Gradient Penalty with Similarity Calculation for Text Summary Generation","volume":"55","author":"Zhao","year":"2023","journal-title":"Neural Process. Lett."},{"key":"ref_67","doi-asserted-by":"crossref","unstructured":"Tang, T., Li, J., Chen, Z., Hu, Y., Yu, Z., Dai, W., and Wen, J.R. (2022). TextBox 2.0: A Text Generation Library with Pre-trained Language Models. arXiv.","DOI":"10.18653\/v1\/2022.emnlp-demos.42"},{"key":"ref_68","doi-asserted-by":"crossref","unstructured":"La Quatra, M., and Cagliero, L. (2023). BART-IT: An Efficient Sequence-to-Sequence Model for Italian Text Summarization. Future Internet, 15.","DOI":"10.3390\/fi15010015"},{"key":"ref_69","first-page":"1","article-title":"Attention is all you need","volume":"30","author":"Vaswani","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_70","doi-asserted-by":"crossref","unstructured":"Dhar, N., Saha, G., Bhattacharjee, P., Mallick, A., and Islam, M.S. (2021, January 18\u201320). Pointer over attention: An improved bangla text summarization approach using hybrid pointer generator network. Proceedings of the 2021 24th International Conference on Computer and Information Technology (ICCIT), Dhaka, Bangladesh.","DOI":"10.1109\/ICCIT54785.2021.9689852"},{"key":"ref_71","doi-asserted-by":"crossref","first-page":"103227","DOI":"10.1016\/j.ipm.2022.103227","article-title":"Arabic Abstractive Text Summarization using RNN-based and Transformer-based Architectures","volume":"60","author":"Kurdy","year":"2023","journal-title":"Inf. Process. Manag."},{"key":"ref_72","doi-asserted-by":"crossref","unstructured":"N\u00fa\u00f1ez-Robinson, D., Talavera-Montalto, J., and Ugarte, W. (2022, January 21\u201323). A Comparative Analysis on the Summarization of Legal Texts Using Transformer Models. Proceedings of the International Conference on Advanced Research in Technologies, Information, Innovation and Sustainability, Cham, Switzerland.","DOI":"10.1007\/978-3-031-20319-0_28"},{"key":"ref_73","doi-asserted-by":"crossref","unstructured":"Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., and Zettlemoyer, L. (2020, January 5\u201310). BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Online.","DOI":"10.18653\/v1\/2020.acl-main.703"},{"key":"ref_74","doi-asserted-by":"crossref","unstructured":"Nallapati, R., Zhou, B., Gulcehre, C., and Xiang, B. (2016). Abstractive Text Summarization using Sequence-to-Sequence RNNs and Beyond. arXiv.","DOI":"10.18653\/v1\/K16-1028"},{"key":"ref_75","doi-asserted-by":"crossref","unstructured":"Farahani, M., Gharachorloo, M., and Manthouri, M. (2021, January 3\u20134). Leveraging ParsBERT and Pretrained mT5 for Persian Abstractive Text Summarization. Proceedings of the 2021 26th International Computer Conference, Computer Society of Iran (CSICC), Tehran, Iran.","DOI":"10.1109\/CSICC52343.2021.9420563"},{"key":"ref_76","doi-asserted-by":"crossref","unstructured":"Reda, A., Salah, N., Adel, J., Ehab, M., Ahmed, I., Magdy, M., Khoriba, G., and Mohamed, E.H. (2022, January 8\u20139). A Hybrid Arabic Text Summarization Approach Based on Transformers. Proceedings of the 2022 2nd International Mobile, Intelligent, and Ubiquitous Computing Conference (MIUCC), Cairo, Egypt.","DOI":"10.1109\/MIUCC55081.2022.9781694"},{"key":"ref_77","unstructured":"Borah, M.P., Dadure, P., and Pakray, P. (2022). Comparative Analysis of T5 Model for Abstractive Text Summarization on Different Datasets, preprint."},{"key":"ref_78","doi-asserted-by":"crossref","unstructured":"Hossen, M.A., Govindaiah, A., Sultana, S., and Bhuiyan, A.A. (2018, January 25\u201329). Bengali sign language recognition using deep convolutional neural network. Proceedings of the 2018 Joint 7th International Conference on Informatics, Electronics & Vision (ICIEV) and 2018 2nd International Conference on Imaging, Vision & Pattern Recognition (icIVPR), Kitakyushu, Japan.","DOI":"10.1109\/ICIEV.2018.8640962"},{"key":"ref_79","doi-asserted-by":"crossref","unstructured":"Xue, L., Constant, N., Roberts, A., Kale, M., Al-Rfou, R., Siddhant, A., Barua, A., and Raffel, C. (2020). mT5: A massively multilingual pre-trained text-to-text transformer. arXiv.","DOI":"10.18653\/v1\/2021.naacl-main.41"},{"key":"ref_80","doi-asserted-by":"crossref","unstructured":"Fuad, A., and Al-Yahya, M. (2022). Cross-Lingual Transfer Learning for Arabic Task-Oriented Dialogue Systems Using Multilingual Transformer Model mT5. Mathematics, 10.","DOI":"10.3390\/math10050746"},{"key":"ref_81","doi-asserted-by":"crossref","unstructured":"Zhang, J., Luan, H., Sun, M., Zhai, F., Xu, J., Zhang, M., and Liu, Y. (2018). Improving the transformer translation model with document-level context. arXiv.","DOI":"10.18653\/v1\/D18-1049"},{"key":"ref_82","doi-asserted-by":"crossref","unstructured":"Abadi, V.N.M., and Ghasemian, F. (2023). Enhancing Persian Text Summarization Using the mT5 Transformer Model: A Three-Phased Fine-Tuning Approach and Reinforcement Learning. Sci. Rep., 15.","DOI":"10.21203\/rs.3.rs-3682780\/v1"},{"key":"ref_83","doi-asserted-by":"crossref","unstructured":"Goyal, N., Du, J., Ott, M., Anantharaman, G., and Conneau, A. (2021). Larger-scale transformers for multilingual masked language modeling. arXiv.","DOI":"10.18653\/v1\/2021.repl4nlp-1.4"},{"key":"ref_84","unstructured":"Ganesan, K. (2018). Rouge 2.0: Updated and improved measures for evaluation of summarization tasks. arXiv."},{"key":"ref_85","doi-asserted-by":"crossref","first-page":"101375","DOI":"10.1016\/j.mex.2021.101375","article-title":"From Arksey and O\u2019Malley and Beyond: Customizations to enhance a team-based, mixed approach to scoping review methodology","volume":"8","author":"Westphaln","year":"2021","journal-title":"MethodsX"},{"key":"ref_86","doi-asserted-by":"crossref","first-page":"467","DOI":"10.7326\/M18-0850","article-title":"PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation","volume":"169","author":"Tricco","year":"2018","journal-title":"Ann. Intern. Med."},{"key":"ref_87","doi-asserted-by":"crossref","first-page":"5897","DOI":"10.1007\/s10462-021-09964-4","article-title":"Study of automatic text summarization approaches in different languages","volume":"54","author":"Kumar","year":"2021","journal-title":"Artif. Intell. Rev."},{"key":"ref_88","doi-asserted-by":"crossref","unstructured":"Hasan, T., Bhattacharjee, A., Islam, M.S., Mubasshir, K., Li, Y.F., Kang, Y.B., Rahman, M.S., and Shahriyar, R. XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages. Proceedings of the Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021.","DOI":"10.18653\/v1\/2021.findings-acl.413"},{"key":"ref_89","doi-asserted-by":"crossref","unstructured":"Grusky, M., Naaman, M., and Artzi, Y. (2018). Newsroom: A dataset of 1.3 million summaries with diverse extractive strategies. arXiv.","DOI":"10.18653\/v1\/N18-1065"},{"key":"ref_90","doi-asserted-by":"crossref","unstructured":"Fabbri, A.R., Li, I., She, T., Li, S., and Radev, D.R. (2019). Multi-news: A large-scale multi-document summarization dataset and abstractive hierarchical model. arXiv.","DOI":"10.18653\/v1\/P19-1102"},{"key":"ref_91","unstructured":"Liu, P.J., Saleh, M., Pot, E., Goodrich, B., Sepassi, R., Kaiser, L., and Shazeer, N. (2018). Generating Wikipedia by Summarizing Long Sequences. arXiv."},{"key":"ref_92","doi-asserted-by":"crossref","unstructured":"Ladhak, F., Durmus, E., Cardie, C., and McKeown, K. (2020). WikiLingua: A New Benchmark Dataset for Cross-Lingual Abstractive Summarization. arXiv.","DOI":"10.18653\/v1\/2020.findings-emnlp.360"},{"key":"ref_93","doi-asserted-by":"crossref","unstructured":"Ghalandari, D.G., Hokamp, C., Pham, N.T., Glover, J., and Ifrim, G. (2020). A Large-scale Multi-document Summarization Dataset from the Wikipedia Current Events Portal. arXiv.","DOI":"10.18653\/v1\/2020.acl-main.120"},{"key":"ref_94","unstructured":"Wojciech, K., Nazneen, R., Divyansh, A., Caiming, X., and Dragomir, R. (2021). Booksum: A Collection of Datasets for Long-form Narrative Summarization. arXiv."},{"key":"ref_95","doi-asserted-by":"crossref","unstructured":"Zhang, R., and Tetreault, J. (2019). This Email Could Save Your Life: Introducing the Task of Email Subject Line Generation. arXiv.","DOI":"10.18653\/v1\/P19-1043"},{"key":"ref_96","unstructured":"Yasunaga, M., Kasai, J., Zhang, R., Fabbri, A.R., Li, I., Friedman, D., and Radev, D.R. (February, January 27). Scisummnet: A Large Annotated Corpus and Content-Impact Models for Scientific Paper Summarization with Citation Networks. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA."},{"key":"ref_97","doi-asserted-by":"crossref","unstructured":"Cohan, A., Dernoncourt, F., Kim, D.S., Bui, T., Kim, S., Chang, W., and Goharian, N. (2018). A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents. arXiv.","DOI":"10.18653\/v1\/N18-2097"},{"key":"ref_98","doi-asserted-by":"crossref","unstructured":"Chowdhury, T., and Chakraborty, T. (2019, January 3\u20135). CQASumm: Building References for Community Question Answering Summarization Corpora. Proceedings of the ACM India Joint International Conference on Data Science and Management of Data, Kolkata, India.","DOI":"10.1145\/3297001.3297004"},{"key":"ref_99","unstructured":"Sayali, K., Sheide, C., Wan, Z., Fei, S., and Eugene, I. (2020). AquaMUSE: Automatically Generating Datasets for Query-based Multi-Document Summarization. arXiv."},{"key":"ref_100","unstructured":"Zhang, J., Liu, Y., Wei, P., Zhao, X., Shou, L., Palangi, H., Gao, J., Shen, Y., and Dolan, B. (2024, April 23). PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization. Available online: https:\/\/huggingface.co\/google\/pegasus-xsum."},{"key":"ref_101","unstructured":"Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., and Stoyanov, V. (2024, April 23). BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. Available online: https:\/\/huggingface.co\/facebook\/bart-large-xsum."},{"key":"ref_102","unstructured":"Moreno, L.Q. (2024, April 23). BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. Available online: https:\/\/huggingface.co\/morenolq\/bart-base-xsum."},{"key":"ref_103","unstructured":"Passali, T., Gidiotis, A., Chatzikyriakidis, E., and Tsoumakas, G. (2023, April 05). Financial Summarization Pegasus. Available online: https:\/\/huggingface.co\/human-centered-summarization\/financial-summarization-pegasus."},{"key":"ref_104","unstructured":"Kapre, R. (2024, May 05). T5 Large Fine-tuned XSum. Available online: https:\/\/huggingface.co\/sysresearch101\/t5-large-finetuned-xsum."},{"key":"ref_105","unstructured":"Szemraj, P. (2024, January 18). Long T5 TGlobal Base 16384 Booksum V11 Big Patent V2. Available online: https:\/\/huggingface.co\/pszemraj\/long-t5-tglobal-base-16384-booksum-V11-big_patent-V2."},{"key":"ref_106","unstructured":"Broad, N. (2024, May 05). LongT5 Base Global MediaSum. Available online: https:\/\/huggingface.co\/nbroad\/longt5-base-global-mediasum."},{"key":"ref_107","unstructured":"Nimmagadda, S.D., and Paleti, K. (2024, January 18). Pegasus Samsum. Available online: https:\/\/huggingface.co\/dhyutin\/pegasus-samsum."},{"key":"ref_108","unstructured":"Schmid, P. (2024, January 18). DistilBART CNN 12-6 Samsum. Available online: https:\/\/huggingface.co\/philschmid\/distilbart-cnn-12-6-samsum."},{"key":"ref_109","unstructured":"Szemeraj, P. (2022, November 30). Long T5 TGlobal Base 16384 Book Summary. Available online: https:\/\/huggingface.co\/pszemraj\/long-t5-tglobal-base-16384-book-summary."},{"key":"ref_110","unstructured":"Szemeraj, P. (2022, November 30). Long T5 TGlobal Large PubMed 3k Book Sum 16384 WIP. Available online: https:\/\/huggingface.co\/pszemraj\/long-t5-tglobal-large-pubmed-3k-booksum-16384-WIP."},{"key":"ref_111","unstructured":"Szemraj, P. (2022, November 30). Long T5 TGlobal Large PubMed 3k Book Sum 16384 WIP13. Available online: https:\/\/huggingface.co\/pszemraj\/long-t5-tglobal-large-pubmed-3k-booksum-16384-WIP13."},{"key":"ref_112","unstructured":"Szemraj, P. (2022, November 30). LED Base Book Summary. Available online: https:\/\/huggingface.co\/pszemraj\/led-base-book-summary."},{"key":"ref_113","unstructured":"Zaheer, M., Guruganesh, G., Dubey, A., Ainslie, J., Alberti, C., Ontanon, S., Pham, P., Ravula, A., Wang, Q., and Yang, L. (2024, January 18). Big Bird: Transformers for Longer Sequences. Available online: https:\/\/huggingface.co\/google\/bigbird-pegasus-large-arxiv."},{"key":"ref_114","unstructured":"Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., and Funtowicz, M. (2020). NLP: Datasets, metrics, and evaluation methods for natural language processing in 100+ languages. arXiv."},{"key":"ref_115","first-page":"27","article-title":"Hybrid text summarizer for Bangla document","volume":"1","author":"Islam","year":"2020","journal-title":"Int. J. Comput. Vis. Sig. Process"},{"key":"ref_116","doi-asserted-by":"crossref","unstructured":"Islam, M.J., Ahammad, K., and Chowdhury, M.K.H. (2020, January 17\u201319). Summarizing online product reviews in Bengali based on similarity using sequence to sequence RNNs. Proceedings of the 2020 11th International Conference on Electrical and Computer Engineering (ICECE), Dhaka, Bangladesh.","DOI":"10.1109\/ICECE51571.2020.9393048"},{"key":"ref_117","doi-asserted-by":"crossref","first-page":"080","DOI":"10.30574\/wjarr.2022.14.1.0292","article-title":"Bangla text generation system by incorporating attention in sequence-to-sequence model","volume":"14","author":"Banik","year":"2022","journal-title":"World J. Adv. Res. Rev."}],"container-title":["Big Data and Cognitive Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-2289\/9\/5\/117\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:23:30Z","timestamp":1760030610000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-2289\/9\/5\/117"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,28]]},"references-count":117,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2025,5]]}},"alternative-id":["bdcc9050117"],"URL":"https:\/\/doi.org\/10.3390\/bdcc9050117","relation":{},"ISSN":["2504-2289"],"issn-type":[{"type":"electronic","value":"2504-2289"}],"subject":[],"published":{"date-parts":[[2025,4,28]]}}}