{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,18]],"date-time":"2026-03-18T13:47:32Z","timestamp":1773841652153,"version":"3.50.1"},"reference-count":63,"publisher":"Association for Computing Machinery (ACM)","issue":"10","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Asian Low-Resour. Lang. Inf. Process."],"published-print":{"date-parts":[[2025,10,31]]},"abstract":"<jats:p>Continual learning, alternatively referred to as incremental learning or lifelong learning, represents a learning paradigm enabling an agent to acquire new information without compromising its retention of previously learned knowledge. Continual learning research has resulted in a number of ways to prevent catastrophic forgetting in deep neural networks. Apparently, continuous learning of recurrent models has never been used to solve the image captioning problems in the Hindi language. This research presents a novel approach to employ continual learning in the context of generating image captions in Hindi. Given its widespread usage in South Asia and its official status in India, Hindi ranks as the third most spoken language worldwide. We look at the continual learning of Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) based models for image captioning in a systematic way. In continuous image captioning problems, we incorporate an attention-based method that explicitly addresses the transient aspect of vocabulary. Using MS-COCO datasets, we employ our methods to solve the incremental image captioning problem. Our findings show that the proposed method can learn five captioning tasks in consecutive order without forgetting the one it has already learned.<\/jats:p>","DOI":"10.1145\/3749641","type":"journal-article","created":{"date-parts":[[2025,7,21]],"date-time":"2025-07-21T11:20:07Z","timestamp":1753096807000},"page":"1-19","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Continual Learning for Image Captioning in Hindi"],"prefix":"10.1145","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4639-5506","authenticated-orcid":false,"given":"Santosh Kumar","family":"Mishra","sequence":"first","affiliation":[{"name":"Computer Science and Engineering, Rajiv Gandhi Institute of Petroleum Technology, Jais, Amethi","place":["Amethi, India"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5458-9381","authenticated-orcid":false,"given":"Sriparna","family":"Saha","sequence":"additional","affiliation":[{"name":"Department of computer science and engineering, Indian Institute of Technology Patna","place":["Patna, India"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5319-5508","authenticated-orcid":false,"given":"Pushpak","family":"Bhattacharyya","sequence":"additional","affiliation":[{"name":"Computer Science and Engineering, IIT Bombay","place":["Mumbai, India"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,10,10]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01219-9_9"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00636"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01258-8_15"},{"key":"e_1_3_1_5_2","unstructured":"Kyunghyun Cho Bart van Merri\u00ebnboer \u00c7a\u011flar Gu\u0307l\u00e7ehre Dzmitry Bahdanau Fethi Bougares Holger Schwenk and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder\u2013decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). 1724\u20131734."},{"key":"e_1_3_1_6_2","first-page":"1","volume-title":"Proceedings of the 2013 International Joint Conference on Neural Networks (IJCNN)","author":"Coop Robert","year":"2013","unstructured":"Robert Coop and Itamar Arel. 2013. Mitigation of catastrophic forgetting in recurrent neural networks using a Fixed Expansion Layer. In Proceedings of the 2013 International Joint Conference on Neural Networks (IJCNN). IEEE, 1\u20137."},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01059"},{"key":"e_1_3_1_8_2","unstructured":"Matthias De Lange Rahaf Aljundi Marc Masana Sarah Parisot Xu Jia Ale\u0161 Leonardis Gregory Slabaugh and Tinne Tuytelaars. 2021. A continual learning survey: Defying forgetting in classification tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence 44 7 (2021) 3366\u20133385."},{"issue":"7","key":"e_1_3_1_9_2","first-page":"3366","article-title":"A continual learning survey: Defying forgetting in classification tasks","volume":"44","author":"Lange Matthias De","year":"2021","unstructured":"Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ale\u0161 Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. 2021. A continual learning survey: Defying forgetting in classification tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 7 (2021), 3366\u20133385.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/W14-3348"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.01095"},{"key":"e_1_3_1_12_2","doi-asserted-by":"crossref","unstructured":"Rijul Dhir Santosh Kumar Mishra Sriparna Saha and Pushpak Bhattacharyya. 2019. A deep attention based framework for image caption generation in hindi language. Computaci\u00f3n y Sistemas 23 3 (2019) 693\u2013701.","DOI":"10.13053\/cys-23-3-3269"},{"key":"e_1_3_1_13_2","unstructured":"Natalia D\u0131az Rodr\u0131guez Vincenzo Lomonaco David Filliat and Davide Maltoni. 2018. Don\u2019t forget there is more than forgetting: New metrics for Continual Learning. CoRR (2018)."},{"key":"e_1_3_1_14_2","first-page":"18009","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Fang Zhiyuan","year":"2022","unstructured":"Zhiyuan Fang, Jianfeng Wang, Xiaowei Hu, Lin Liang, Zhe Gan, Lijuan Wang, Yezhou Yang, and Zicheng Liu. 2022. Injecting semantic concepts into end-to-end image captioning. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 18009\u201318019."},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00425"},{"key":"e_1_3_1_16_2","first-page":"1176","volume-title":"Proceedings of the 6th International Conference on Neural Information Processing Systems","author":"French Robert M.","year":"1993","unstructured":"Robert M. French. 1993. Catastrophic interference in connectionist networks: Can it be predicted, can it be prevented?. In Proceedings of the 6th International Conference on Neural Information Processing Systems. 1176\u20131177."},{"key":"e_1_3_1_17_2","unstructured":"Jane Gary and Carl Rubino. 2001. Facts about the world\u2019s languages: An encyclopedia of the world\u2019s major languages. HW Wilson NY (2001)."},{"key":"e_1_3_1_18_2","unstructured":"Ian J. Goodfellow Mehdi Mirza Aaron Courville and Yoshua Bengio. 2015. An empirical investigation of catastrophic forgetting in gradient-based neural networks. Stat 1050 (2015) 4."},{"key":"e_1_3_1_19_2","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Gupta Pankaj","year":"2020","unstructured":"Pankaj Gupta, Yatin Chaudhary, Thomas A. Runkler, and Hinrich Sch\u00fctze. 2020. Neural topic modeling with continual lifelong learning. In Proceedings of the International Conference on Machine Learning. Retrieved from https:\/\/api.semanticscholar.org\/CorpusID:219955694"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_1_21_2","unstructured":"Hamed Hemati Marco Schreyer and Damian Borth. 2021. Continual learning for unsupervised anomaly detection in continuous auditing of financial accounting data. arXiv preprint arXiv:2112.13215 (2021)."},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"e_1_3_1_23_2","first-page":"2790","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Houlsby Neil","year":"2019","unstructured":"Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for NLP. In Proceedings of the International Conference on Machine Learning. PMLR, 2790\u20132799."},{"key":"e_1_3_1_24_2","doi-asserted-by":"crossref","unstructured":"Muhammad Awais Hussain Shih-An Huang and Tsung-Han Tsai. 2022. Learning with sharing: An edge-optimized incremental learning method for deep neural networks. IEEE Transactions on Emerging Topics in Computing 11 2 (2022) 461\u2013473.","DOI":"10.1109\/TETC.2022.3210905"},{"key":"e_1_3_1_25_2","doi-asserted-by":"crossref","unstructured":"Jiayi Ji Yiwei Ma Xiaoshuai Sun Yiyi Zhou Yongjian Wu and Rongrong Ji. 2022. Knowing what to learn: a metric-oriented focal mechanism for image captioning. IEEE Transactions on Image Processing 31 (2022) 4321\u20134335.","DOI":"10.1109\/TIP.2022.3183434"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.12283"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298932"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01746"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2017.2773081"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00350"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.02322"},{"key":"e_1_3_1_32_2","first-page":"8653","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Liu Huihui","year":"2021","unstructured":"Huihui Liu, Yiding Yang, and Xinchao Wang. 2021. Overcoming catastrophic forgetting in graph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence. 8653\u20138661."},{"key":"e_1_3_1_33_2","first-page":"11588","volume-title":"Proceedings of the AAAI","author":"Liu Junhao","year":"2020","unstructured":"Junhao Liu, Kai Wang, Chunpu Xu, Zhou Zhao, Ruifeng Xu, Ying Shen, and Min Yang. 2020. Interactive dual generative adversarial networks for image captioning. In Proceedings of the AAAI. 11588\u201311595."},{"key":"e_1_3_1_34_2","unstructured":"Tianlin Liu Lyle Ungar and Jo\u00e3o Sedoc. 2019. Continual learning for sentence representations using conceptors. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies Volume 1 (Long and Short Papers). 3274\u20133279."},{"key":"e_1_3_1_35_2","first-page":"226","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops","author":"Liu Xialei","year":"2020","unstructured":"Xialei Liu, Chenshen Wu, Mikel Menta, Luis Herranz, Bogdan Raducanu, Andrew D. Bagdanov, Shangling Jui, and Joost van de Weijer. 2020. Generative feature replay for class-incremental learning. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops. 226\u2013227."},{"key":"e_1_3_1_36_2","unstructured":"David Lopez-Paz and Marc\u2019Aurelio Ranzato. 2017. Gradient episodic memory for continual learning. Advances in Neural Information Processing Systems 30 (2017)."},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025814"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00810"},{"key":"e_1_3_1_39_2","unstructured":"Junhua Mao Wei Xu Yi Yang Jiang Wang Zhiheng Huang and Alan Yuille. 2014. Deep captioning with multimodal recurrent neural networks (m-rnn). arXiv preprint arXiv:1412.6632 (2014)."},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0079-7421(08)60536-8"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3432246"},{"key":"e_1_3_1_42_2","doi-asserted-by":"crossref","unstructured":"Santosh Kumar Mishra Rijul Dhir Sriparna Saha Pushpak Bhattacharyya and Amit Kumar Singh. 2021. Image captioning in Hindi language using transformer networks. Computers & Electrical Engineering 92 (2021) 107114.","DOI":"10.1016\/j.compeleceng.2021.107114"},{"issue":"3","key":"e_1_3_1_43_2","first-page":"1","article-title":"Efficient channel attention based encoder\u2013decoder approach for image captioning in Hindi","volume":"21","author":"Mishra Santosh Kumar","year":"2021","unstructured":"Santosh Kumar Mishra, Gaurav Rai, Sriparna Saha, and Pushpak Bhattacharyya. 2021. Efficient channel attention based encoder\u2013decoder approach for image captioning in Hindi. Transactions on Asian and Low-Resource Language Information Processing 21, 3 (2021), 1\u201317.","journal-title":"Transactions on Asian and Low-Resource Language Information Processing"},{"key":"e_1_3_1_44_2","first-page":"311","volume-title":"Proceedings of the 40th Annual Meeting on Association for Computational Linguistics","author":"Papineni Kishore","year":"2002","unstructured":"Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics. Association for Computational Linguistics, 311\u2013318."},{"key":"e_1_3_1_45_2","doi-asserted-by":"crossref","unstructured":"German I. Parisi Ronald Kemker Jose L. Part Christopher Kanan and Stefan Wermter. 2019. Continual lifelong learning with neural networks: A review. Neural Networks 113 (2019) 54\u201371.","DOI":"10.1016\/j.neunet.2019.01.012"},{"key":"e_1_3_1_46_2","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1007\/978-3-031-05933-9_9","volume-title":"Proceedings of the Pacific-Asia Conference on Knowledge Discovery and Data Mining","author":"Phan Hoang","year":"2022","unstructured":"Hoang Phan, Anh Phan Tuan, Son Nguyen, Ngo Van Linh, and Khoat Than. 2022. Reducing catastrophic forgetting in neural networks via Gaussian mixture approximation. In Proceedings of the Pacific-Asia Conference on Knowledge Discovery and Data Mining. Springer, 106\u2013117."},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52733.2024.01224"},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.587"},{"key":"e_1_3_1_49_2","first-page":"91","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","author":"Ren Shaoqing","year":"2015","unstructured":"Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster r-cnn: Towards real-time object detection with region proposal networks. In Proceedings of the Advances in Neural Information Processing Systems. 91\u201399."},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00486"},{"key":"e_1_3_1_51_2","first-page":"4548","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Serra Joan","year":"2018","unstructured":"Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. 2018. Overcoming catastrophic forgetting with hard attention to the task. In Proceedings of the International Conference on Machine Learning. PMLR, 4548\u20134557."},{"key":"e_1_3_1_52_2","doi-asserted-by":"publisher","DOI":"10.1162\/neco_a_01246"},{"key":"e_1_3_1_53_2","volume-title":"Proceedings of the North American Chapter of the Association for Computational Linguistics","author":"Thompson Brian","year":"2019","unstructured":"Brian Thompson, Jeremy Gwinnup, Huda Khayrallah, Kevin Duh, and Philipp Koehn. 2019. Overcoming catastrophic forgetting during domain adaptation of neural machine translation. In Proceedings of the North American Chapter of the Association for Computational Linguistics. Retrieved from https:\/\/api.semanticscholar.org\/CorpusID:150376583"},{"key":"e_1_3_1_54_2","first-page":"5998","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the Advances in Neural Information Processing Systems. 5998\u20136008."},{"key":"e_1_3_1_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7299087"},{"key":"e_1_3_1_56_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298935"},{"key":"e_1_3_1_57_2","unstructured":"Mitchell Wortsman Vivek Ramanujan Rosanne Liu Aniruddha Kembhavi Mohammad Rastegari Jason Yosinski and Ali Farhadi. 2020. Supermasks in superposition. Advances in Neural Information Processing Systems 33 (2020) 15173\u201315184."},{"key":"e_1_3_1_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01245"},{"key":"e_1_3_1_59_2","first-page":"2048","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Xu Kelvin","year":"2015","unstructured":"Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio. 2015. Show, attend and tell: Neural image caption generation with visual attention. In Proceedings of the International Conference on Machine Learning. 2048\u20132057."},{"key":"e_1_3_1_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/3503161.3548409"},{"key":"e_1_3_1_61_2","first-page":"2361","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","author":"Yang Zhilin","year":"2016","unstructured":"Zhilin Yang, Ye Yuan, Yuexin Wu, William W. Cohen, and Ruslan R. Salakhutdinov. 2016. Review networks for caption generation. In Proceedings of the Advances in Neural Information Processing Systems. 2361\u20132369."},{"key":"e_1_3_1_62_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.503"},{"key":"e_1_3_1_63_2","doi-asserted-by":"publisher","DOI":"10.1109\/LGRS.2019.2897652"},{"key":"e_1_3_1_64_2","first-page":"13041","volume-title":"Proceedings of the AAAI","author":"Zhou Luowei","year":"2020","unstructured":"Luowei Zhou, Hamid Palangi, Lei Zhang, Houdong Hu, Jason J. Corso, and Jianfeng Gao. 2020. Unified vision-language pre-training for image captioning and VQA. In Proceedings of the AAAI. 13041\u201313049."}],"container-title":["ACM Transactions on Asian and Low-Resource Language Information Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3749641","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T07:01:01Z","timestamp":1760079661000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3749641"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,10]]},"references-count":63,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2025,10,31]]}},"alternative-id":["10.1145\/3749641"],"URL":"https:\/\/doi.org\/10.1145\/3749641","relation":{},"ISSN":["2375-4699","2375-4702"],"issn-type":[{"value":"2375-4699","type":"print"},{"value":"2375-4702","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,10]]},"assertion":[{"value":"2022-12-24","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-01-07","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-10-10","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}