{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T14:10:15Z","timestamp":1755785415462,"version":"3.44.0"},"reference-count":57,"publisher":"Association for Computing Machinery (ACM)","issue":"8","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Asian Low-Resour. Lang. Inf. Process."],"published-print":{"date-parts":[[2025,8,31]]},"abstract":"<jats:p>\n            This article proposes a meta-learning-based, few-shot image captioning framework based on an ensemble of object-relational transformer models and a self-distillation strategy. Unlike traditional approaches, the proposed framework can train models using non-paired images and captions. In each iteration, multiple base models are trained on distinct data samples, forming an ensemble that generates pseudo-captions with confidence-based weighting. An efficient pseudo-feature generation method based on gradient descent enables learning from non-paired captions. These pseudo-features and pseudo-captions are then used to train the base models in subsequent iterations. The obtained results show that using just 1% of the paired image-caption training data significantly improves performance and generates meaningful captions. The proposed framework demonstrates the potential to reduce reliance on large, paired datasets while maintaining high-quality image captioning performance.\n            <jats:xref ref-type=\"fn\">\n              <jats:sup>1<\/jats:sup>\n            <\/jats:xref>\n          <\/jats:p>\n          <jats:p\/>","DOI":"10.1145\/3749642","type":"journal-article","created":{"date-parts":[[2025,7,21]],"date-time":"2025-07-21T11:20:07Z","timestamp":1753096807000},"page":"1-15","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Hi-MetaCap: Configuring Object Relational Transformer in Meta-Learning Environment for Image Captioning in Hindi"],"prefix":"10.1145","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4639-5506","authenticated-orcid":false,"given":"Santosh Kumar","family":"Mishra","sequence":"first","affiliation":[{"name":"Computer Science and Engineering, Rajiv Gandhi Institute of Petroleum Technology, Jais, Amethi, India","place":["Amethi, India"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-2675-9418","authenticated-orcid":false,"given":"Soham","family":"Chakraborty","sequence":"additional","affiliation":[{"name":"Kalinga Institute of Industrial Technology","place":["Bhubaneswar, India"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5458-9381","authenticated-orcid":false,"given":"Sriparna","family":"Saha","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, IIT Patna","place":["Patna, India"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5319-5508","authenticated-orcid":false,"given":"Pushpak","family":"Bhattacharyya","sequence":"additional","affiliation":[{"name":"Computer Science and Engineering, IIT Bombay","place":["Mumbai, India"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,8,21]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00636"},{"key":"e_1_3_2_3_2","first-page":"65","volume-title":"Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and\/or Summarization","author":"Banerjee Satanjeev","year":"2005","unstructured":"Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and\/or Summarization. 65\u201372."},{"key":"e_1_3_2_4_2","unstructured":"Wenhu Chen Aurelien Lucchi and Thomas Hofmann. 2016. A semi-supervised framework for image captioning. arXiv preprint arXiv:1611.05321 (2016)."},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","unstructured":"Kyunghyun Cho Bart van Merri\u00ebnboer Caglar Gulcehre Dzmitry Bahdanau Fethi Bougares Holger Schwenk and Yoshua Bengio. 2014. Learning phrase representations using RNN encoderdecoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) Alessandro Moschitti Bo Pang and Walter Daelemans (Eds.). Association for Computational Linguistics Doha Qatar 1724\u20131734. DOI:10.3115\/v1\/D14-1179","DOI":"10.3115\/v1\/D14-1179"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01059"},{"key":"e_1_3_2_7_2","doi-asserted-by":"crossref","unstructured":"Rijul Dhir Santosh Kumar Mishra Sriparna Saha and Pushpak Bhattacharyya. 2019. A deep attention based framework for image caption generation in hindi language. Computaci\u00f3n y Sistemas 23 3 (2019) 693\u2013701.","DOI":"10.13053\/cys-23-3-3269"},{"key":"e_1_3_2_8_2","unstructured":"John Duchi Elad Hazan and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research 12 (2011) 2121\u20132159."},{"key":"e_1_3_2_9_2","doi-asserted-by":"crossref","first-page":"1292","DOI":"10.18653\/v1\/D13-1128","volume-title":"Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing","author":"Elliott Desmond","year":"2013","unstructured":"Desmond Elliott and Frank Keller. 2013. Image description using visual dependency representations. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. 1292\u20131302."},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-15561-1_2"},{"key":"e_1_3_2_11_2","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Feng Yang","year":"2019","unstructured":"Yang Feng, Lin Ma, Wei Liu, and Jiebo Luo. 2019. Unsupervised image captioning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"e_1_3_2_12_2","doi-asserted-by":"crossref","first-page":"24575","DOI":"10.1109\/CVPR52729.2023.02354","article-title":"StyleAdv: Meta style adversarial training for cross-domain few-shot learning","author":"Fu Yu","year":"2023","unstructured":"Yu Fu, Yu Xie, Yanwei Fu, and Yugang Jiang. 2023. StyleAdv: Meta style adversarial training for cross-domain few-shot learning. In Proceedings of the 2023 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 24575\u201324584. Retrieved from https:\/\/api.semanticscholar.org\/CorpusID:257038175","journal-title":"Proceedings of the 2023 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)"},{"key":"e_1_3_2_13_2","first-page":"235","volume-title":"Proceedings of the International Conference on Artificial Intelligence and Statistics","author":"Gao Xiand","year":"2018","unstructured":"Xiand Gao, Xiaobo Li, and Shuzhong Zhang. 2018. Online learning with non-convex losses and non-stationary regret. In Proceedings of the International Conference on Artificial Intelligence and Statistics. PMLR, 235\u2013243."},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/SMC52423.2021.9658586"},{"key":"e_1_3_2_15_2","first-page":"529","volume-title":"Proceedings of the European Conference on Computer Vision","author":"Gong Yunchao","year":"2014","unstructured":"Yunchao Gong, Liwei Wang, Micah Hodosh, Julia Hockenmaier, and Svetlana Lazebnik. 2014. Improving image-sentence embeddings using large weakly annotated photo collections. In Proceedings of the European Conference on Computer Vision. Springer, 529\u2013545."},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.01042"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_2_18_2","unstructured":"Simao Herdade Armin Kappeler Kofi Boakye and Joao Soares. 2019. Image captioning: Transforming objects into words. Advances in Neural Information Processing Systems 32 (2019)."},{"key":"e_1_3_2_19_2","unstructured":"Geoffrey Hinton Oriol Vinyals and Jeff Dean. 2015. Distilling the knowledge in a neural network. Stat 1050 (2015) 9."},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.11768"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00473"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.12283"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00898"},{"key":"e_1_3_2_24_2","doi-asserted-by":"crossref","unstructured":"Dong-Jin Kim Jinsoo Choi Tae-Hyun Oh and In So Kweon. 2019. Image captioning with very scarce supervised data: Adversarial semi-supervised learning approach. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Association for Computational Linguistics 2012\u20132023.","DOI":"10.18653\/v1\/D19-1208"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2011.5995466"},{"key":"e_1_3_2_26_2","first-page":"11039","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Kuo Chia-Wen","year":"2023","unstructured":"Chia-Wen Kuo and Zsolt Kira. 2023. Haav: Hierarchical aggregation of augmented views for image captioning. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 11039\u201311049."},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-020-01316-z"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00751"},{"key":"e_1_3_2_29_2","first-page":"12888","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Li Junnan","year":"2022","unstructured":"Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In Proceedings of the International Conference on Machine Learning. PMLR, 12888\u201312900."},{"key":"e_1_3_2_30_2","first-page":"220","volume-title":"Proceedings of the 15th Conference on Computational Natural Language Learning","author":"Li Siming","year":"2011","unstructured":"Siming Li, Girish Kulkarni, Tamara L. Berg, Alexander C. Berg, and Yejin Choi. 2011. Composing simple image descriptions using web-scale n-grams. In Proceedings of the 15th Conference on Computational Natural Language Learning. Association for Computational Linguistics, 220\u2013228."},{"key":"e_1_3_2_31_2","unstructured":"Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out Association for Computational Linguistics. Barcelona Spain 74\u201381. Retrieved from https:\/\/aclanthology.org\/W04-1013\/"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.02237"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025814"},{"key":"e_1_3_2_35_2","unstructured":"Junhua Mao Wei Xu Yi Yang Jiang Wang Zhiheng Huang and Alan L. Yuille. 2015. Deep captioning with multimodal recurrent neural networks (m\u2011RNN). In International Conference on Learning Representations (ICLR)."},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/3622936"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/3432246"},{"key":"e_1_3_2_38_2","doi-asserted-by":"crossref","unstructured":"Santosh Kumar Mishra Rijul Dhir Sriparna Saha Pushpak Bhattacharyya and Amit Kumar Singh. 2021. Image captioning in Hindi language using transformer networks. Comput. Electr. Eng. 92 (2021) 107114.","DOI":"10.1016\/j.compeleceng.2021.107114"},{"issue":"2","key":"e_1_3_2_39_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3558391","article-title":"An object localization-based dense image captioning framework in Hindi","volume":"22","author":"Mishra Santosh Kumar","year":"2022","unstructured":"Santosh Kumar Mishra, Harshit, Sriparna Saha, and Pushpak Bhattacharyya. 2022. An object localization-based dense image captioning framework in Hindi. ACM Transactions on Asian and Low-Resource Language Information Processing 22, 2 (2022), 1\u201315.","journal-title":"ACM Transactions on Asian and Low-Resource Language Information Processing"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/SMC52423.2021.9658859"},{"issue":"3","key":"e_1_3_2_41_2","first-page":"1","article-title":"Efficient channel attention based encoder\u2013decoder approach for image captioning in Hindi","volume":"21","author":"Mishra Santosh Kumar","year":"2021","unstructured":"Santosh Kumar Mishra, Gaurav Rai, Sriparna Saha, and Pushpak Bhattacharyya. 2021. Efficient channel attention based encoder\u2013decoder approach for image captioning in Hindi. Transactions on Asian and Low-Resource Language Information Processing 21, 3 (2021), 1\u201317.","journal-title":"Transactions on Asian and Low-Resource Language Information Processing"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3573891"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01098"},{"key":"e_1_3_2_44_2","first-page":"311","volume-title":"Proceedings of the 40th Annual Meeting on Association for Computational Linguistics","author":"Papineni Kishore","year":"2002","unstructured":"Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics. Association for Computational Linguistics, 311\u2013318."},{"key":"e_1_3_2_45_2","first-page":"8748","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Radford Alec","year":"2021","unstructured":"Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et\u00a0al. 2021. Learning transferable visual models from natural language supervision. In Proceedings of the International Conference on Machine Learning. PMLR, 8748\u20138763."},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-015-0816-y"},{"issue":"5","key":"e_1_3_2_47_2","doi-asserted-by":"crossref","first-page":"1317","DOI":"10.1109\/TCSS.2021.3110819","article-title":"On multimodal microblog summarization","volume":"9","author":"Saini Naveen","year":"2021","unstructured":"Naveen Saini, Sriparna Saha, Pushpak Bhattacharyya, Shubhankar Mrinal, and Santosh Kumar Mishra. 2021. On multimodal microblog summarization. IEEE Transactions on Computational Social Systems 9, 5 (2021), 1317\u20131329.","journal-title":"IEEE Transactions on Computational Social Systems"},{"key":"e_1_3_2_48_2","volume-title":"Proceedings of the International Conference on Neural Information Processing Systems","author":"Tarvainen Antti","year":"2017","unstructured":"Antti Tarvainen and Harri Valpola. 2017. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In Proceedings of the International Conference on Neural Information Processing Systems."},{"key":"e_1_3_2_49_2","doi-asserted-by":"crossref","unstructured":"Pinzhuo Tian and Shaorong Xie. 2023. An adversarial meta-training framework for cross-domain few-shot learning. IEEE Transactions on Multimedia 25 (2023) 6881\u20136891.","DOI":"10.1109\/TMM.2022.3215310"},{"key":"e_1_3_2_50_2","first-page":"5998","volume-title":"Proceedings of the International Conference on Neural Information Processing Systems","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the International Conference on Neural Information Processing Systems. 5998\u20136008."},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7299087"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298935"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01155"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.2307\/2332510"},{"key":"e_1_3_2_55_2","first-page":"2048","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Xu Kelvin","year":"2015","unstructured":"Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio. 2015. Show, attend and tell: Neural image caption generation with visual attention. In Proceedings of the International Conference on Machine Learning. 2048\u20132057."},{"key":"e_1_3_2_56_2","doi-asserted-by":"crossref","unstructured":"Qiaoqiao Yang Zihao Ni and Peng Ren. 2022. Meta captioning: A meta learning based remote sensing image captioning framework. ISPRS Journal of Photogrammetry and Remote Sensing 186 (2022) 190\u2013200.","DOI":"10.1016\/j.isprsjprs.2022.02.001"},{"key":"e_1_3_2_57_2","first-page":"2361","volume-title":"Proceedings of the International Conference on Neural Information Processing Systems","author":"Yang Zhilin","year":"2016","unstructured":"Zhilin Yang, Ye Yuan, Yuexin Wu, William W. Cohen, and Ruslan R. Salakhutdinov. 2016. Review networks for caption generation. In Proceedings of the International Conference on Neural Information Processing Systems. 2361\u20132369."},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01264-9_42"}],"container-title":["ACM Transactions on Asian and Low-Resource Language Information Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3749642","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T13:28:09Z","timestamp":1755782889000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3749642"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,21]]},"references-count":57,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2025,8,31]]}},"alternative-id":["10.1145\/3749642"],"URL":"https:\/\/doi.org\/10.1145\/3749642","relation":{},"ISSN":["2375-4699","2375-4702"],"issn-type":[{"type":"print","value":"2375-4699"},{"type":"electronic","value":"2375-4702"}],"subject":[],"published":{"date-parts":[[2025,8,21]]},"assertion":[{"value":"2023-03-10","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-12-08","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-08-21","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}