{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,4]],"date-time":"2026-06-04T15:57:42Z","timestamp":1780588662637,"version":"3.54.1"},"reference-count":46,"publisher":"Association for Computing Machinery (ACM)","issue":"10","license":[{"start":{"date-parts":[[2024,10,23]],"date-time":"2024-10-23T00:00:00Z","timestamp":1729641600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Asian Low-Resour. Lang. Inf. Process."],"published-print":{"date-parts":[[2024,10,31]]},"abstract":"<jats:p>\n            With the advent of Deep Learning-based Artificial Neural Network models, Natural Language Processing (NLP) has witnessed significant improvements in textual data processing in terms of its efficiency and accuracy. However, the research is mostly restricted to high-resource languages such as English, and low-resource languages still suffer from a lack of available resources in terms of training datasets as well as models with even baseline evaluation results. Considering the limited availability of resources for low-resource languages, we propose a methodology for adapting self-attentive transformer-based architecture models (mBERT, mT5) for low-resource summarization, supplemented by the construction of a new baseline dataset (76.5k article, summary pairs) in a low-resource language, Urdu. Choosing news (a publicly available source) as the application domain has the potential to make the proposed methodology useful for reproducing in other languages with limited resources. Our adapted summarization model\n            <jats:italic>urT5<\/jats:italic>\n            with up to 44.78% reduction in size as compared to\n            <jats:italic>mT5<\/jats:italic>\n            can capture contextual information of the low-resource language effectively with an evaluation score (up to 46.35 ROUGE-1, 77 BERTScore) on par with state-of-the-art models in the high-resource language of English\n            <jats:italic>(PEGASUS: 47.21, BART: 45.14 on XSUM Dataset)<\/jats:italic>\n            . The proposed method provided a baseline approach toward extractive as well as abstractive summarization with competitive evaluation results in a limited resource setup.\n          <\/jats:p>","DOI":"10.1145\/3675780","type":"journal-article","created":{"date-parts":[[2024,7,3]],"date-time":"2024-07-03T11:20:43Z","timestamp":1720005643000},"page":"1-19","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":18,"title":["Low Resource Summarization using Pre-trained Language Models"],"prefix":"10.1145","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-2938-3590","authenticated-orcid":false,"given":"Mubashir","family":"Munaf","sequence":"first","affiliation":[{"name":"National University of Sciences and Technology, Islamabad, Pakistan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9583-5585","authenticated-orcid":false,"given":"Hammad","family":"Afzal","sequence":"additional","affiliation":[{"name":"National University of Sciences and Technology, Islamabad, Pakistan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9377-6945","authenticated-orcid":false,"given":"Khawir","family":"Mahmood","sequence":"additional","affiliation":[{"name":"National University of Sciences and Technology, Islamabad, Pakistan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5392-5187","authenticated-orcid":false,"given":"Naima","family":"Iltaf","sequence":"additional","affiliation":[{"name":"National University of Sciences and Technology, Islamabad, Pakistan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,10,23]]},"reference":[{"key":"e_1_3_2_2_2","article-title":"Load what you need: Smaller versions of multilingual BERT","author":"Abdaoui Amine","year":"2020","unstructured":"Amine Abdaoui, Camille Pradel, and Gr\u00e9goire Sigel. 2020. Load what you need: Smaller versions of multilingual BERT. arXiv preprint arXiv:2010.05609 (2020).","journal-title":"arXiv preprint arXiv:2010.05609"},{"key":"e_1_3_2_3_2","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-14142-8","volume-title":"Mining Text Data","author":"Aggarwal Charu C.","year":"2015","unstructured":"Charu C. Aggarwal and Charu C. Aggarwal. 2015. Mining Text Data. Springer."},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00288"},{"key":"e_1_3_2_5_2","article-title":"Neural machine translation by jointly learning to align and translate","author":"Bahdanau Dzmitry","year":"2014","unstructured":"Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014).","journal-title":"arXiv preprint arXiv:1409.0473"},{"key":"e_1_3_2_6_2","article-title":"A multitask, multilingual, multimodal evaluation of ChatGPT on reasoning, hallucination, and interactivity","author":"Bang Yejin","year":"2023","unstructured":"Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, et\u00a0al. 2023. A multitask, multilingual, multimodal evaluation of ChatGPT on reasoning, hallucination, and interactivity. arXiv preprint arXiv:2302.04023 (2023).","journal-title":"arXiv preprint arXiv:2302.04023"},{"key":"e_1_3_2_7_2","article-title":"Context aware emotion detection from low resource urdu language using deep neural network","author":"Bashir Muhammad Farrukh","year":"2022","unstructured":"Muhammad Farrukh Bashir, Abdul Rehman Javed, Muhammad Umair Arshad, Thippa Reddy Gadekallu, Waseem Shahzad, and Mirza Omer Beg. 2022. Context aware emotion detection from low resource urdu language using deep neural network. Transactions on Asian and Low-Resource Language Information Processing (2022).","journal-title":"Transactions on Asian and Low-Resource Language Information Processing"},{"key":"e_1_3_2_8_2","article-title":"GPT-neox-20b: An open-source autoregressive language model","author":"Black Sid","year":"2022","unstructured":"Sid Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, et\u00a0al. 2022. GPT-neox-20b: An open-source autoregressive language model. arXiv preprint arXiv:2204.06745 (2022).","journal-title":"arXiv preprint arXiv:2204.06745"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00051"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/1102351.1102363"},{"key":"e_1_3_2_11_2","article-title":"Neural summarization by extracting sentences and words","author":"Cheng Jianpeng","year":"2016","unstructured":"Jianpeng Cheng and Mirella Lapata. 2016. Neural summarization by extracting sentences and words. arXiv preprint arXiv:1603.07252 (2016).","journal-title":"arXiv preprint arXiv:1603.07252"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N16-1012"},{"key":"e_1_3_2_13_2","article-title":"BERT: Pre-training of deep bidirectional transformers for language understanding","author":"Devlin Jacob","year":"2018","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).","journal-title":"arXiv preprint arXiv:1810.04805"},{"key":"e_1_3_2_14_2","article-title":"Ethnologue: Languages of the World. 25th ed","author":"Eberhard D. M.","year":"2022","unstructured":"D. M. Eberhard, G. F. Simons, and C. D. Fennig. 2022. Ethnologue: Languages of the World. 25th ed. SIL International, Dallas.","journal-title":"SIL International, Dallas"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/321510.321519"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.5555\/3241691.3241693"},{"key":"e_1_3_2_17_2","article-title":"Fine-tuned language models for text classification","volume":"194","author":"Howard Jeremy","year":"2018","unstructured":"Jeremy Howard and Sebastian Ruder. 2018. Fine-tuned language models for text classification. arXiv preprint arXiv:1801.06146 194 (2018).","journal-title":"arXiv preprint arXiv:1801.06146"},{"key":"e_1_3_2_18_2","article-title":"Convolutional neural network architectures for matching natural language sentences","volume":"27","author":"Hu Baotian","year":"2014","unstructured":"Baotian Hu, Zhengdong Lu, Hang Li, and Qingcai Chen. 2014. Convolutional neural network architectures for matching natural language sentences. Advances in Neural Information Processing Systems 27 (2014).","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_19_2","first-page":"796","volume-title":"Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC\u201916)","author":"Humayoun Muhammad","year":"2016","unstructured":"Muhammad Humayoun, Rao Muhammad Adeel Nawab, Muhammad Uzair, Saba Aslam, and Omer Farzand. 2016. Urdu summary corpus. In Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC\u201916). 796\u2013800."},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3571730"},{"key":"e_1_3_2_21_2","article-title":"Muril: Multilingual representations for Indian languages","author":"Khanuja Simran","year":"2021","unstructured":"Simran Khanuja, Diksha Bansal, Sarvesh Mehtani, Savya Khosla, Atreyee Dey, Balaji Gopalan, Dilip Kumar Margam, Pooja Aggarwal, Rajiv Teja Nagipogu, Shachi Dave, et\u00a0al. 2021. Muril: Multilingual representations for Indian languages. arXiv preprint arXiv:2103.10730 (2021).","journal-title":"arXiv preprint arXiv:2103.10730"},{"key":"e_1_3_2_22_2","article-title":"Cross-lingual language model pretraining","author":"Lample Guillaume","year":"2019","unstructured":"Guillaume Lample and Alexis Conneau. 2019. Cross-lingual language model pretraining. arXiv preprint arXiv:1901.07291 (2019).","journal-title":"arXiv preprint arXiv:1901.07291"},{"key":"e_1_3_2_23_2","first-page":"74","volume-title":"Text Summarization Branches Out","author":"Lin Chin-Yew","year":"2004","unstructured":"Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text Summarization Branches Out. 74\u201381."},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.5555\/1690219.1690290"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1147\/rd.22.0159"},{"key":"e_1_3_2_26_2","article-title":"Efficient estimation of word representations in vector space","author":"Mikolov Tomas","year":"2013","unstructured":"Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013).","journal-title":"arXiv preprint arXiv:1301.3781"},{"key":"e_1_3_2_27_2","article-title":"Abstractive text summarization using sequence-to-sequence RNNs and beyond","author":"Nallapati Ramesh","year":"2016","unstructured":"Ramesh Nallapati, Bowen Zhou, Caglar Gulcehre, Bing Xiang, et\u00a0al. 2016. Abstractive text summarization using sequence-to-sequence RNNs and beyond. arXiv preprint arXiv:1602.06023 (2016).","journal-title":"arXiv preprint arXiv:1602.06023"},{"key":"e_1_3_2_28_2","article-title":"Don\u2019t give me the details, just the summary! Topic-aware convolutional neural networks for extreme summarization","author":"Narayan Shashi","year":"2018","unstructured":"Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018. Don\u2019t give me the details, just the summary! Topic-aware convolutional neural networks for extreme summarization. arXiv preprint arXiv:1808.08745 (2018).","journal-title":"arXiv preprint arXiv:1808.08745"},{"issue":"6","key":"e_1_3_2_29_2","doi-asserted-by":"crossref","first-page":"102383","DOI":"10.1016\/j.ipm.2020.102383","article-title":"Extractive text summarization models for Urdu language","volume":"57","author":"Nawaz Ali","year":"2020","unstructured":"Ali Nawaz, Maheen Bakhtyar, Junaid Baber, Ihsan Ullah, Waheed Noor, and Abdul Basit. 2020. Extractive text summarization models for Urdu language. Information Processing & Management 57, 6 (2020), 102383.","journal-title":"Information Processing & Management"},{"key":"e_1_3_2_30_2","unstructured":"OpenAI. 2023. GPT-4 Technical Report. arxiv:2303.08774 [cs.CL]"},{"key":"e_1_3_2_31_2","article-title":"Multi-reward reinforced summarization with saliency and entailment","author":"Pasunuru Ramakanth","year":"2018","unstructured":"Ramakanth Pasunuru and Mohit Bansal. 2018. Multi-reward reinforced summarization with saliency and entailment. arXiv preprint arXiv:1804.06451 (2018).","journal-title":"arXiv preprint arXiv:1804.06451"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1162"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-1202"},{"key":"e_1_3_2_34_2","article-title":"How multilingual is multilingual BERT?","author":"Pires Telmo","year":"2019","unstructured":"Telmo Pires, Eva Schlinger, and Dan Garrette. 2019. How multilingual is multilingual BERT? arXiv preprint arXiv:1906.01502 (2019).","journal-title":"arXiv preprint arXiv:1906.01502"},{"key":"e_1_3_2_35_2","unstructured":"Alec Radford Karthik Narasimhan Tim Salimans Ilya Sutskever et\u00a0al. 2018. Improving language understanding by generative pre-training. (2018)."},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.5555\/3455716.3455856"},{"key":"e_1_3_2_37_2","article-title":"A neural attention model for abstractive sentence summarization","author":"Rush Alexander M.","year":"2015","unstructured":"Alexander M. Rush, Sumit Chopra, and Jason Weston. 2015. A neural attention model for abstractive sentence summarization. arXiv preprint arXiv:1509.00685 (2015).","journal-title":"arXiv preprint arXiv:1509.00685"},{"key":"e_1_3_2_38_2","article-title":"Bloom: A 176b-parameter open-access multilingual language model","author":"Scao Teven Le","year":"2022","unstructured":"Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ili\u0107, Daniel Hesslow, Roman Castagn\u00e9, Alexandra Sasha Luccioni, Fran\u00e7ois Yvon, Matthias Gall\u00e9, et\u00a0al. 2022. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100 (2022).","journal-title":"arXiv preprint arXiv:2211.05100"},{"key":"e_1_3_2_39_2","article-title":"Get to the point: Summarization with pointer-generator networks","author":"See Abigail","year":"2017","unstructured":"Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer-generator networks. arXiv preprint arXiv:1704.04368 (2017).","journal-title":"arXiv preprint arXiv:1704.04368"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1108\/eb026526"},{"key":"e_1_3_2_41_2","article-title":"Sequence to sequence learning with neural networks","volume":"27","author":"Sutskever Ilya","year":"2014","unstructured":"Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. Advances in Neural Information Processing Systems 27 (2014).","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_42_2","article-title":"Attention is all you need","volume":"30","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in Neural Information Processing Systems 30 (2017).","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00520"},{"key":"e_1_3_2_44_2","article-title":"mT5: A massively multilingual pre-trained text-to-text transformer","author":"Xue Linting","year":"2020","unstructured":"Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2020. mT5: A massively multilingual pre-trained text-to-text transformer. arXiv preprint arXiv:2010.11934 (2020).","journal-title":"arXiv preprint arXiv:2010.11934"},{"key":"e_1_3_2_45_2","article-title":"Language modeling teaches you more syntax than translation does: Lessons learned through auxiliary task analysis","author":"Zhang Kelly W.","year":"2018","unstructured":"Kelly W. Zhang and Samuel R. Bowman. 2018. Language modeling teaches you more syntax than translation does: Lessons learned through auxiliary task analysis. arXiv preprint arXiv:1809.10040 (2018).","journal-title":"arXiv preprint arXiv:1809.10040"},{"key":"e_1_3_2_46_2","article-title":"Bertscore: Evaluating text generation with bert","author":"Zhang Tianyi","year":"2019","unstructured":"Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675 (2019).","journal-title":"arXiv preprint arXiv:1904.09675"},{"key":"e_1_3_2_47_2","article-title":"Neural document summarization by jointly learning to score and select sentences","author":"Zhou Qingyu","year":"2018","unstructured":"Qingyu Zhou, Nan Yang, Furu Wei, Shaohan Huang, Ming Zhou, and Tiejun Zhao. 2018. Neural document summarization by jointly learning to score and select sentences. arXiv preprint arXiv:1807.02305 (2018).","journal-title":"arXiv preprint arXiv:1807.02305"}],"container-title":["ACM Transactions on Asian and Low-Resource Language Information Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3675780","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3675780","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:04:09Z","timestamp":1750291449000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3675780"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,23]]},"references-count":46,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2024,10,31]]}},"alternative-id":["10.1145\/3675780"],"URL":"https:\/\/doi.org\/10.1145\/3675780","relation":{},"ISSN":["2375-4699","2375-4702"],"issn-type":[{"value":"2375-4699","type":"print"},{"value":"2375-4702","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,23]]},"assertion":[{"value":"2023-08-27","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-06-22","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-10-23","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}