{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,20]],"date-time":"2026-04-20T23:45:37Z","timestamp":1776728737484,"version":"3.51.2"},"reference-count":48,"publisher":"Association for Computing Machinery (ACM)","issue":"10","license":[{"start":{"date-parts":[[2023,10,13]],"date-time":"2023-10-13T00:00:00Z","timestamp":1697155200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Young Faculty Research Fellowship program of Visvesvaraya Ph.D. Scheme of Ministry of Electronics and Information Technology, Government of India"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Asian Low-Resour. Lang. Inf. Process."],"published-print":{"date-parts":[[2023,10,31]]},"abstract":"<jats:p>Image captioning frameworks usually employ an encoder-decoder paradigm, with the encoder receiving abstract image feature vectors as input and the decoder for language modeling. Nowadays, most prominent architectures employ features from region proposals derived from object detection modules. In this work, we propose a novel architecture for image captioning. We employ the object detection module integrated with transformer architecture as an encoder and GPT-2 (Generative Pre-trained Transformer) as a decoder. The encoder utilizes the information of the spatial relationships among detected objects. We introduce a unique methodology for image caption generation in Hindi, which is widely spoken in South Asia and India and is the world\u2019s third most spoken language as well as India\u2019s official language. In terms of BLEU scores, the proposed approach\u2019s performance is comparable to those of other baselines, and the results illustrate that the proposed approach outperforms the other baselines. The efficacy of the proposed approach in generating correct captions is further determined by human assessment in terms of adequacy and fluency.<\/jats:p>","DOI":"10.1145\/3622936","type":"journal-article","created":{"date-parts":[[2023,9,11]],"date-time":"2023-09-11T12:23:57Z","timestamp":1694435037000},"page":"1-16","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["GAGPT-2: A Geometric Attention-based GPT-2 Framework for Image Captioning in Hindi"],"prefix":"10.1145","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4639-5506","authenticated-orcid":false,"given":"Santosh Kumar","family":"Mishra","sequence":"first","affiliation":[{"name":"Rajiv Gandhi Institute of Petroleum Technology, Amethi, India"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-2675-9418","authenticated-orcid":false,"given":"Soham","family":"Chakraborty","sequence":"additional","affiliation":[{"name":"Kalinga Institute of Industrial Technology, Bhubaneswar, India"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5458-9381","authenticated-orcid":false,"given":"Sriparna","family":"Saha","sequence":"additional","affiliation":[{"name":"Indian Institute of Technology Patna, India"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5319-5508","authenticated-orcid":false,"given":"Pushpak","family":"Bhattacharyya","sequence":"additional","affiliation":[{"name":"Indian Institute of Technology Bombay"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,10,13]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"crossref","unstructured":"Peter Anderson Xiaodong He Chris Buehler Damien Teney Mark Johnson Stephen Gould and Lei Zhang. 2018. Bottom-up and top-down attention for image captioning and visual question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 6077\u20136086.","DOI":"10.1109\/CVPR.2018.00636"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00636"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW56347.2022.00512"},{"key":"e_1_3_1_5_2","article-title":"Learning phrase representations using RNN encoder-decoder for statistical machine translation","author":"Cho Kyunghyun","year":"2014","unstructured":"Kyunghyun Cho, Bart Van Merri\u00ebnboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014).","journal-title":"arXiv preprint arXiv:1406.1078"},{"key":"e_1_3_1_6_2","doi-asserted-by":"crossref","unstructured":"Marcella Cornia Matteo Stefanini Lorenzo Baraldi and Rita Cucchiara. 2020. Meshed-memory transformer for image captioning. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition . 10578\u201310587.","DOI":"10.1109\/CVPR42600.2020.01059"},{"key":"e_1_3_1_7_2","doi-asserted-by":"crossref","unstructured":"Aditya Deshpande Jyoti Aneja Liwei Wang Alexander G. Schwing and David Forsyth. 2019. Fast diverse and accurate image captioning guided by part-of-speech. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition . 10695\u201310704.","DOI":"10.1109\/CVPR.2019.01095"},{"issue":"3","key":"e_1_3_1_8_2","article-title":"A deep attention based framework for image caption generation in Hindi language","volume":"23","author":"Dhir Rijul","year":"2019","unstructured":"Rijul Dhir, Santosh Kumar Mishra, Sriparna Saha, and Pushpak Bhattacharyya. 2019. A deep attention based framework for image caption generation in Hindi language. Computaci\u00f3n y Sistemas 23, 3 (2019), 693\u2013701.","journal-title":"Computaci\u00f3n y Sistemas"},{"key":"e_1_3_1_9_2","first-page":"1292","volume-title":"Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing","author":"Elliott Desmond","year":"2013","unstructured":"Desmond Elliott and Frank Keller. 2013. Image description using visual dependency representations. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. 1292\u20131302."},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.5555\/1888089.1888092"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01190"},{"key":"e_1_3_1_12_2","doi-asserted-by":"crossref","unstructured":"Yang Feng Lin Ma Wei Liu and Jiebo Luo. 2019. Unsupervised image captioning. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition . 4125\u20134134.","DOI":"10.1109\/CVPR.2019.00425"},{"key":"e_1_3_1_13_2","unstructured":"J. Garry and C. R. G. Rubino. 2001. Facts about the World\u2019s Languages: An Encyclopedia of the World\u2019s Major Languages Past and Present . H. W. Wilson EBSCO New York."},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICMCIS52405.2021.9486395"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10593-2_35"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01309"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.5555\/2566972.2566993"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/WACV56688.2023.00118"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00378"},{"key":"e_1_3_1_21_2","doi-asserted-by":"crossref","unstructured":"Wenhao Jiang Lin Ma Xinpeng Chen Hanwang Zhang and Wei Liu. 2018. Learning to guide decoding for image captioning. In Proceedings of the AAAI Conference on Artificial Intelligence Vol. 32 6959\u20136966.","DOI":"10.1609\/aaai.v32i1.12283"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298932"},{"key":"e_1_3_1_23_2","doi-asserted-by":"crossref","unstructured":"Girish Kulkarni Visruth Premraj Sagnik Dhar Siming Li Yejin Choi Alexander C. Berg and Tamara L. Berg. 2011. Baby talk: Understanding and generating image descriptions. IEEE Transactions on Pattern Analysis and Machine Intelligence 35 12 (2011) 2891\u20132903.","DOI":"10.1109\/TPAMI.2012.162"},{"key":"e_1_3_1_24_2","first-page":"220","volume-title":"Proceedings of the 15th Conference on Computational Natural Language Learning","author":"Li Siming","year":"2011","unstructured":"Siming Li, Girish Kulkarni, Tamara L. Berg, Alexander C. Berg, and Yejin Choi. 2011. Composing simple image descriptions using web-scale n-grams. In Proceedings of the 15th Conference on Computational Natural Language Learning. Association for Computational Linguistics, 220\u2013228."},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01746"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00350"},{"key":"e_1_3_1_27_2","first-page":"11588","volume-title":"AAAI","author":"Liu Junhao","year":"2020","unstructured":"Junhao Liu, Kai Wang, Chunpu Xu, Zhou Zhao, Ruifeng Xu, Ying Shen, and Min Yang. 2020. Interactive dual generative adversarial networks for image captioning. In AAAI. 11588\u201311595."},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025814"},{"key":"e_1_3_1_29_2","article-title":"Deep captioning with multimodal recurrent neural networks (M-RNN)","author":"Mao Junhua","year":"2014","unstructured":"Junhua Mao, Wei Xu, Yi Yang, Jiang Wang, Zhiheng Huang, and Alan Yuille. 2014. Deep captioning with multimodal recurrent neural networks (M-RNN). arXiv preprint arXiv:1412.6632 (2014).","journal-title":"arXiv preprint arXiv:1412.6632"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/3432246"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.compeleceng.2021.107114"},{"issue":"3","key":"e_1_3_1_32_2","first-page":"1","article-title":"Efficient channel attention based encoder\u2013decoder approach for image captioning in Hindi","volume":"21","author":"Mishra Santosh Kumar","year":"2021","unstructured":"Santosh Kumar Mishra, Gaurav Rai, Sriparna Saha, and Pushpak Bhattacharyya. 2021. Efficient channel attention based encoder\u2013decoder approach for image captioning in Hindi. Transactions on Asian and Low-resource Language Information Processing 21, 3 (2021), 1\u201317.","journal-title":"Transactions on Asian and Low-resource Language Information Processing"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/DSAA.2017.59"},{"key":"e_1_3_1_34_2","first-page":"311","volume-title":"Proceedings of the 40th Annual Meeting on Association for Computational Linguistics","author":"Papineni Kishore","year":"2002","unstructured":"Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics. Association for Computational Linguistics, 311\u2013318."},{"issue":"8","key":"e_1_3_1_35_2","first-page":"9","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford Alec","year":"2019","unstructured":"Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 (2019), 9.","journal-title":"OpenAI Blog"},{"key":"e_1_3_1_36_2","first-page":"91","volume-title":"Advances in Neural Information Processing Systems","author":"Ren Shaoqing","year":"2015","unstructured":"Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster R-CNN: Towards real-time object detection with region proposal networks. In Advances in Neural Information Processing Systems. 91\u201399."},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.131"},{"key":"e_1_3_1_38_2","article-title":"On multimodal microblog summarization","author":"Saini Naveen","year":"2021","unstructured":"Naveen Saini, Sriparna Saha, Pushpak Bhattacharyya, Shubhankar Mrinal, and Santosh Kumar Mishra. 2021. On multimodal microblog summarization. IEEE Transactions on Computational Social Systems 9, 5 (2021), 1317\u20131329.","journal-title":"IEEE Transactions on Computational Social Systems"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00486"},{"key":"e_1_3_1_40_2","first-page":"5998","volume-title":"Advances in Neural Information Processing Systems","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems. 5998\u20136008."},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298935"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2016.2587640"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.3390\/s18020646"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01749"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01245"},{"key":"e_1_3_1_46_2","first-page":"2048","volume-title":"International Conference on Machine Learning","author":"Xu Kelvin","year":"2015","unstructured":"Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio. 2015. Show, attend and tell: Neural image caption generation with visual attention. In International Conference on Machine Learning. 2048\u20132057."},{"key":"e_1_3_1_47_2","first-page":"2361","volume-title":"Advances in Neural Information Processing Systems","author":"Yang Zhilin","year":"2016","unstructured":"Zhilin Yang, Ye Yuan, Yuexin Wu, William W. Cohen, and Ruslan R. Salakhutdinov. 2016. Review networks for caption generation. In Advances in Neural Information Processing Systems. 2361\u20132369."},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.503"},{"key":"e_1_3_1_49_2","first-page":"13041","volume-title":"AAAI","author":"Zhou Luowei","year":"2020","unstructured":"Luowei Zhou, Hamid Palangi, Lei Zhang, Houdong Hu, Jason J. Corso, and Jianfeng Gao. 2020. Unified vision-language pre-training for image captioning and VQA. In AAAI. 13041\u201313049."}],"container-title":["ACM Transactions on Asian and Low-Resource Language Information Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3622936","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3622936","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:57:28Z","timestamp":1750298248000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3622936"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,13]]},"references-count":48,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2023,10,31]]}},"alternative-id":["10.1145\/3622936"],"URL":"https:\/\/doi.org\/10.1145\/3622936","relation":{},"ISSN":["2375-4699","2375-4702"],"issn-type":[{"value":"2375-4699","type":"print"},{"value":"2375-4702","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,10,13]]},"assertion":[{"value":"2022-08-21","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-08-21","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-10-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}