{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T18:19:13Z","timestamp":1783621153670,"version":"3.55.0"},"reference-count":44,"publisher":"Public Library of Science (PLoS)","issue":"7","license":[{"start":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T00:00:00Z","timestamp":1783555200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS One"],"abstract":"<jats:p>Automated image captioning remains a challenge, as it enables machines to generate context-aware textual descriptions of visual content. Traditional deep learning approaches often rely on lexical overlap and fail to capture semantic relationships among objects, leading to captions that lack contextual richness. This study proposes an encoder\u2013decoder framework that integrates YOLOv5 with a generative transformer to generate descriptive image captions. The proposed model was evaluated against two baselines: CNN-LSTM (M1) and a BERT-based transformer model (M2). M1 achieves BLEU-1 (0.45) and ROUGE-L (0.42) but demonstrates limited semantic understanding with METEOR (0.18) and SPICE (0.07). M2 improves with higher METEOR (0.24) and CIDEr (0.62), although its BLEU scores remain low. The proposed model achieves the highest CIDEr (1.10) and SPICE (0.25), reflecting superior semantic understanding and better capture of object relationships. Despite a lower BLEU (0.40), it significantly outperforms traditional methods in caption quality. To further validate these results, we conducted an expert-based evaluation to assess semantic accuracy, visual grounding, and caption usefulness. The proposed model achieved 93% accuracy in expert evaluations across 500 images, indicating strong contextual alignment with human interpretation. Additionally, we employed exploratory data analysis to examine and visualize the text captions, aiming to gain a deeper understanding of the optimal caption.<\/jats:p>","DOI":"10.1371\/journal.pone.0353466","type":"journal-article","created":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T17:42:32Z","timestamp":1783618952000},"page":"e0353466","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":0,"title":["Contextual image caption creation using object positional embedding and generative models"],"prefix":"10.1371","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-8785-360X","authenticated-orcid":true,"given":"Muhammad","family":"Danyal","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Muhammad","family":"Roman","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6291-2641","authenticated-orcid":true,"given":"Abdul","family":"Shahid","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5766-5172","authenticated-orcid":true,"given":"Muhammad","family":"Yahya","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2026,7,9]]},"reference":[{"issue":"6","key":"pone.0353466.ref001","doi-asserted-by":"crossref","first-page":"3685","DOI":"10.1109\/TCSVT.2021.3107035","article-title":"Region-aware image captioning via interaction learning","volume":"32","author":"AA Liu","year":"2021","journal-title":"IEEE Trans Circuits Syst Video Technol"},{"key":"pone.0353466.ref002","doi-asserted-by":"crossref","first-page":"106838","DOI":"10.1109\/ACCESS.2019.2932731","article-title":"An Improved Faster R-CNN for Small Object Detection","volume":"7","author":"C Cao","year":"2019","journal-title":"IEEE Access"},{"key":"pone.0353466.ref003","doi-asserted-by":"crossref","unstructured":"Lin TY, Doll\u00e1r P, Girshick R, He K, Hariharan B, Belongie S. Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. p. 2117\u201325.","DOI":"10.1109\/CVPR.2017.106"},{"key":"pone.0353466.ref004","doi-asserted-by":"crossref","unstructured":"Xie X, Cheng G, Wang J, Yao X, Han J. Oriented R-CNN for object detection. In: Proceedings of the IEEE\/CVF international conference on computer vision. 2021. p. 3520\u20139.","DOI":"10.1109\/ICCV48922.2021.00350"},{"key":"pone.0353466.ref005","doi-asserted-by":"crossref","unstructured":"Ko\u00e7i J, Topal AO, Ali M. Threat object detection in x-ray images using SSD, r-FCN and faster r-CNN. In: 2020 International Conference on Computing, Networking, Telecommunications & Engineering Sciences Applications (CoNTESA). 2020. p. 10\u20135.","DOI":"10.1109\/CoNTESA50436.2020.9302863"},{"key":"pone.0353466.ref006","doi-asserted-by":"crossref","unstructured":"Zhao G, Ge W, Yu Y. GraphFPN: Graph feature pyramid network for object detection. In: Proceedings of the IEEE\/CVF International Conference on Computer Vision. 2021. p. 2763\u201372.","DOI":"10.1109\/ICCV48922.2021.00276"},{"issue":"1","key":"pone.0353466.ref007","doi-asserted-by":"crossref","first-page":"17","DOI":"10.21608\/njccs.2022.280047","article-title":"Mask R-CNN models","volume":"3","author":"E Hassan","year":"2022","journal-title":"Nile Journal of Commun Comput Sci"},{"key":"pone.0353466.ref008","article-title":"Im2text: Describing images using 1 million captioned photographs","volume":"24","author":"V Ordonez","year":"2011","journal-title":"Adv Neural Inform Process Syst"},{"key":"pone.0353466.ref009","doi-asserted-by":"crossref","first-page":"853","DOI":"10.1613\/jair.3994","article-title":"Framing image description as a ranking task: Data, models and evaluation metrics","volume":"47","author":"M Hodosh","year":"2013","journal-title":"J Artif Intell Res"},{"key":"pone.0353466.ref010","doi-asserted-by":"crossref","unstructured":"Mason R, Charniak E. Nonparametric Method for Data-driven Image Captioning. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 2014. p. 592\u20138.","DOI":"10.3115\/v1\/P14-2097"},{"key":"pone.0353466.ref011","doi-asserted-by":"crossref","unstructured":"Gupta A, Verma Y, Jawahar C. Choosing linguistics over vision to describe images. In: Proceedings of the AAAI conference on artificial intelligence. vol. 26. 2012. p. 606\u201312.","DOI":"10.1609\/aaai.v26i1.8205"},{"key":"pone.0353466.ref012","doi-asserted-by":"crossref","first-page":"351","DOI":"10.1162\/tacl_a_00188","article-title":"Treetalk: Composition and compression of trees for image descriptions","volume":"2","author":"P Kuznetsova","year":"2014","journal-title":"Trans Assoc Comput Linguist"},{"issue":"12","key":"pone.0353466.ref013","doi-asserted-by":"crossref","first-page":"2891","DOI":"10.1109\/TPAMI.2012.162","article-title":"Babytalk: understanding and generating simple image descriptions","volume":"35","author":"G Kulkarni","year":"2013","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"pone.0353466.ref014","unstructured":"Mitchell M, Dodge J, Goyal A, Yamaguchi K, Stratos K, Han X. Midge: Generating image descriptions from computer vision detections. In: Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics. 2012. p. 747\u201356."},{"key":"pone.0353466.ref015","doi-asserted-by":"crossref","unstructured":"Fang H, Gupta S, Iandola F, Srivastava RK, Deng L, Dollar P, et al. From captions to visual concepts and back. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2015. p. 1473\u201382.","DOI":"10.1109\/CVPR.2015.7298754"},{"issue":"1","key":"pone.0353466.ref016","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1007\/s44196-023-00233-6","article-title":"Multiscale Feature Extraction and Fusion of Image and Text in VQA","volume":"16","author":"S Lu","year":"2023","journal-title":"Int J Comput Intell Syst"},{"key":"pone.0353466.ref017","doi-asserted-by":"crossref","unstructured":"Fan Z, Wei Z, Wang S, Wang R, Li Z, Shan H, et al. Tcic: Theme concepts learning cross language and vision for image captioning. arXiv preprint arXiv:210610936. 2021.","DOI":"10.24963\/ijcai.2021\/91"},{"key":"pone.0353466.ref018","doi-asserted-by":"crossref","first-page":"31397","DOI":"10.1109\/ACCESS.2022.3159980","article-title":"Investigating maps of science using contextual proximity of citations based on deep contextualized word representation","volume":"10","author":"M Roman","year":"2022","journal-title":"IEEE Access"},{"key":"pone.0353466.ref019","doi-asserted-by":"crossref","unstructured":"Song Z, Zhou X. Exploring explicit and implicit visual relationships for image captioning. In: 2021 IEEE International Conference on Multimedia and Expo (ICME). IEEE; 2021. p. 1\u20136.","DOI":"10.1109\/ICME51207.2021.9428310"},{"key":"pone.0353466.ref020","doi-asserted-by":"crossref","first-page":"114044","DOI":"10.1109\/ACCESS.2023.3320038","article-title":"Exploiting Contextual Word Embedding for Identification of Important Citations: Incorporating Section-Wise Citation Counts and Metadata Features","volume":"11","author":"A Iqbal","year":"2023","journal-title":"IEEE Access"},{"key":"pone.0353466.ref021","first-page":"274","article-title":"Image Captioning using CNN and LSTM.","volume-title":"4th Smart cities symposium (SCS 2021)","author":"G Sairam","year":"2021"},{"issue":"3","key":"pone.0353466.ref022","doi-asserted-by":"crossref","DOI":"10.35940\/ijitee.C8383.0110321","article-title":"Image caption generator","volume":"10","author":"MJ Panicker","year":"2021","journal-title":"Int J Innov Technol Explor Eng"},{"issue":"2","key":"pone.0353466.ref023","doi-asserted-by":"crossref","first-page":"102178","DOI":"10.1016\/j.ipm.2019.102178","article-title":"Image caption generation with dual attention mechanism","volume":"57","author":"M Liu","year":"2020","journal-title":"Inform Process Manag"},{"key":"pone.0353466.ref024","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1016\/j.patrec.2019.03.021","article-title":"Image caption generation with high-level image features","volume":"123","author":"S Ding","year":"2019","journal-title":"Pattern Recognit Lett"},{"key":"pone.0353466.ref025","unstructured":"Liu W, Chen S, Guo L, Zhu X, Liu J. Cptr: Full transformer network for image captioning. arXiv preprint arXiv:210110804. 2021."},{"key":"pone.0353466.ref026","doi-asserted-by":"crossref","unstructured":"Dong X, Long C, Xu W, Xiao C. Dual graph convolutional networks with transformer and curriculum learning for image captioning. In: Proceedings of the 29th ACM International Conference on Multimedia. 2021. p. 2615\u201324.","DOI":"10.1145\/3474085.3475439"},{"key":"pone.0353466.ref027","doi-asserted-by":"crossref","unstructured":"Yang X, Liu Y, Wang X. ReFormer: The Relational Transformer for Image Captioning. In: Proceedings of the 30th ACM International Conference on Multimedia. 2022. p. 5398\u2013406.","DOI":"10.1145\/3503161.3548409"},{"key":"pone.0353466.ref028","doi-asserted-by":"crossref","unstructured":"Yang X, Tang K, Zhang H, Cai J. Auto-encoding scene graphs for image captioning. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 2019. 10685\u201394.","DOI":"10.1109\/CVPR.2019.01094"},{"key":"pone.0353466.ref029","doi-asserted-by":"crossref","unstructured":"Chen S, Jin Q, Wang P, Wu Q. Say as you wish: Fine-grained control of image caption generation with abstract scene graphs. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition. 2020. 9962\u201371.","DOI":"10.1109\/CVPR42600.2020.00998"},{"key":"pone.0353466.ref030","doi-asserted-by":"crossref","unstructured":"Li Y, Yao T, Pan Y, Chao H, Mei T. Pointing novel objects in image captioning. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition. 2019. p. 12497\u2013506.","DOI":"10.1109\/CVPR.2019.01278"},{"issue":"2","key":"pone.0353466.ref031","doi-asserted-by":"crossref","first-page":"710","DOI":"10.1109\/TPAMI.2019.2909864","article-title":"Context-Aware Visual Policy Network for Fine-Grained Image Captioning","volume":"44","author":"Z-J Zha","year":"2022","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"pone.0353466.ref032","doi-asserted-by":"crossref","unstructured":"Xu G, Niu S, Tan M, Luo Y, Du Q, Wu Q. Towards accurate text-based image captioning with content diversity exploration. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 2021. p. 12637\u201346.","DOI":"10.1109\/CVPR46437.2021.01245"},{"key":"pone.0353466.ref033","doi-asserted-by":"crossref","first-page":"45219","DOI":"10.1109\/ACCESS.2022.3169781","article-title":"Image Captioning Model Using Part-of-Speech Guidance Module for Description With Diverse Vocabulary","volume":"10","author":"JW Bae","year":"2022","journal-title":"IEEE Access"},{"key":"pone.0353466.ref034","doi-asserted-by":"crossref","unstructured":"Caesar H, Uijlings J, Ferrari V. COCO-Stuff: Thing and Stuff Classes in Context. In: 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 2018. p. 1209\u201318.","DOI":"10.1109\/CVPR.2018.00132"},{"issue":"7","key":"pone.0353466.ref035","doi-asserted-by":"crossref","first-page":"677","DOI":"10.3390\/machines11070677","article-title":"YOLO-v1 to YOLO-v8, the rise of YOLO and its complementary nature toward digital manufacturing and industrial defect detection","volume":"11","author":"M Hussain","year":"2023","journal-title":"Machines"},{"key":"pone.0353466.ref036","doi-asserted-by":"crossref","unstructured":"Chauhan R, Ghanshala KK, Joshi R. Convolutional neural network (CNN) for image detection and recognition. In: 2018 first international conference on secure cyber computing and communication (ICSCCC). IEEE; 2018. p. 278\u2013282.","DOI":"10.1109\/ICSCCC.2018.8703316"},{"issue":"9","key":"pone.0353466.ref037","doi-asserted-by":"crossref","DOI":"10.1056\/AIcs2400245","article-title":"GPT-4 Performance, Nondeterminism, and Drift in Genetic Literature Review","volume":"1","author":"SJ Aronson","year":"2024","journal-title":"NEJM AI"},{"key":"pone.0353466.ref038","doi-asserted-by":"crossref","unstructured":"Parmar H, Rai M, Murari UK. A Novel Image Caption Generation Based on CNN and RNN. In: 2024 International Conference on Advances in Computing Research on Science Engineering and Technology (ACROSET). IEEE; 2024. p. 1\u20138.","DOI":"10.1109\/ACROSET62108.2024.10743848"},{"key":"pone.0353466.ref039","doi-asserted-by":"crossref","unstructured":"Yates A, Nogueira R, Lin J. Pretrained Transformers for Text Ranking: BERT and Beyond. In: Proceedings of the 14th ACM International Conference on Web Search and Data Mining. 2021. p. 1154\u20136.","DOI":"10.1145\/3437963.3441667"},{"key":"pone.0353466.ref040","doi-asserted-by":"crossref","unstructured":"Papineni K, Roukos S, Ward T, Zhu WJ. Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th annual meeting of the Association for Computational Linguistics. 2002. p. 311\u20138.","DOI":"10.3115\/1073083.1073135"},{"key":"pone.0353466.ref041","unstructured":"Lin CY. Rouge: A package for automatic evaluation of summaries. Text summarization branches out. 2004. p. 74\u201381."},{"key":"pone.0353466.ref042","unstructured":"Banerjee S, Lavie A. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In: Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and\/or Summarization. 2005. p. 65\u201372."},{"key":"pone.0353466.ref043","doi-asserted-by":"crossref","unstructured":"Vedantam R, Zitnick CL, Parikh D. Cider: Consensus-based image description evaluation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2015. p. 4566\u201375.","DOI":"10.1109\/CVPR.2015.7299087"},{"key":"pone.0353466.ref044","doi-asserted-by":"crossref","unstructured":"Anderson P, Fernando B, Johnson M, Gould S. SPICE: Semantic Propositional Image Caption Evaluation. In: Computer Vision\u2013ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part V 14. Springer; 2016. p. 382\u201398.","DOI":"10.1007\/978-3-319-46454-1_24"}],"container-title":["PLOS One"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0353466","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T17:42:39Z","timestamp":1783618959000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0353466"}},"subtitle":[],"editor":[{"given":"Muhammad","family":"Bilal","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026,7,9]]},"references-count":44,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2026,7,9]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0353466","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,9]]}}}