{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T11:41:50Z","timestamp":1783597310569,"version":"3.55.0"},"reference-count":29,"publisher":"Public Library of Science (PLoS)","issue":"5","license":[{"start":{"date-parts":[[2025,5,8]],"date-time":"2025-05-08T00:00:00Z","timestamp":1746662400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS One"],"abstract":"<jats:p>Identifying anatomical landmarks in endoscopic video frames is essential for the early diagnosis of gastrointestinal diseases. However, this task remains challenging due to variability in visual characteristics across different regions and the limited availability of annotated data. In this study, we propose a novel self-supervised learning (SSL) framework that integrates three complementary pretext task, colorization, jigsaw puzzle solving, and patch prediction, to enhance feature learning from unlabeled endoscopic images. By leveraging these tasks, our model extracts rich, meaningful representations, improving the downstream classification of Z-line, esophageal, and antrum\/pylorus regions. To further enhance feature extraction and model interpretability, we incorporate attention mechanisms, transformer-based architectures, and Grad-CAM visualization. The integration of attention layers and transformers strengthens the model\u2019s ability to learn discriminative and generalizable features, while Grad-CAM improves explainability by highlighting critical decision-making regions. These enhancements make our approach more suitable for clinical deployment, ensuring both high accuracy and interpretability. We evaluate our proposed framework on a comprehensive dataset, demonstrating substantial improvements in classification accuracy, precision, recall, and F1-score compared to conventional models trained without SSL. Specifically, our combined model achieves a classification accuracy of 98%, with high precision and recall across all classes, as reflected in ROC curves and confusion matrices. These results underscore the effectiveness of pretext-task-based SSL, attention mechanism, and transformers for anatomical landmark identification in endoscopic video frames. Our work introduces a scalable and interpretable methodology for improving endoscopic image classification, reducing reliance on large annotated datasets while enhancing model performance in real-world clinical applications. Future research will explore validation on diverse datasets, real-time diagnostic integration, and scalability to further advance medical image analysis using SSL.<\/jats:p>","DOI":"10.1371\/journal.pone.0322028","type":"journal-article","created":{"date-parts":[[2025,5,8]],"date-time":"2025-05-08T20:03:42Z","timestamp":1746734622000},"page":"e0322028","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":5,"title":["Self-supervised learning framework for efficient classification of endoscopic images using pretext tasks"],"prefix":"10.1371","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5905-4017","authenticated-orcid":true,"given":"Shima Ayyoubi","family":"Nezhad","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7381-2606","authenticated-orcid":true,"given":"Golnaz","family":"Tajeddin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Toktam","family":"Khatibi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Masoudreza","family":"Sohrabi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2025,5,8]]},"reference":[{"issue":"2","key":"pone.0322028.ref001","doi-asserted-by":"crossref","first-page":"15","DOI":"10.3390\/technologies12020015","article-title":"A review of machine learning and deep learning for object detection, semantic segmentation, and human action recognition in machine and robotic vision","volume":"12","author":"N Manakitsa","year":"2024","journal-title":"Technologies"},{"key":"pone.0322028.ref002","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1109\/ICIP42928.2021.9506132","article-title":"Image enhanced rotation prediction for self-supervised learning","author":"S Yamaguchi","year":"2021","journal-title":"2021 IEEE International Conference on Image Processing (ICIP)"},{"key":"pone.0322028.ref003","doi-asserted-by":"crossref","first-page":"107090","DOI":"10.1016\/j.knosys.2021.107090","article-title":"Review on self-supervised image recognition using deep neural networks","volume":"224","author":"K Ohri","year":"2021","journal-title":"Knowledge-Based Systems"},{"issue":"2","key":"pone.0322028.ref004","doi-asserted-by":"crossref","first-page":"540","DOI":"10.1109\/TBME.2024.3462502","article-title":"S2P-Matching: Self-supervised patch-based matching using transformer for capsule endoscopic images stitching","volume":"72","author":"F Lu","year":"2025","journal-title":"IEEE Transactions on Biomedical Engineering"},{"key":"pone.0322028.ref005","article-title":"Domain-adaptive pre-training of self-supervised foundation models for medical image classification in gastrointestinal endoscopy","author":"M Roth","year":"2024","journal-title":"arXiv preprint arXiv"},{"issue":"1","key":"pone.0322028.ref006","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1109\/TMI.2024.3429471","article-title":"Multi-label generalized zero shot chest X-ray classification by combining image-text information with feature disentanglement","volume":"44","author":"D Mahapatra","year":"2025","journal-title":"IEEE Trans Med Imaging"},{"key":"pone.0322028.ref007","doi-asserted-by":"crossref","first-page":"108613","DOI":"10.1016\/j.cmpb.2025.108613","article-title":"Semi-supervised strong-teacher consistency learning for few-shot cardiac MRI image segmentation","volume":"261","author":"Y Qiu","year":"2025","journal-title":"Comput Methods Programs Biomed"},{"key":"pone.0322028.ref008","first-page":"1","article-title":"Covid-19 classification using deep learning in chest X-ray images","author":"Z Karhan","year":"2020","journal-title":"2020 Medical Technologies Congress (TIPTEKNO)"},{"key":"pone.0322028.ref009","doi-asserted-by":"crossref","first-page":"100196","DOI":"10.1016\/j.health.2023.100196","article-title":"A novel machine learning-based feature extraction method for classifying intracranial hemorrhage computed tomography images","volume":"3","author":"S Gudadhe","year":"2023","journal-title":"Healthcare Analytics"},{"key":"pone.0322028.ref010","doi-asserted-by":"crossref","first-page":"5137904","DOI":"10.1155\/2018\/5137904","article-title":"Machine learning in ultrasound computer-aided diagnostic systems: A survey","volume":"2018","author":"Q Huang","year":"2018","journal-title":"Biomed Res Int"},{"key":"pone.0322028.ref011","article-title":"An overview of melanoma detection in dermoscopy images using image processing and machine learning.","author":"NK Mishra","year":"2016","journal-title":"arXiv preprint arXiv"},{"issue":"45","key":"pone.0322028.ref012","doi-asserted-by":"crossref","first-page":"5057","DOI":"10.3748\/wjg.v24.i45.5057","article-title":"Methodology to develop machine learning algorithms to improve performance in gastrointestinal endoscopy","volume":"24","author":"T De Lange","year":"2018","journal-title":"World J Gastroenterol"},{"issue":"2","key":"pone.0322028.ref013","doi-asserted-by":"crossref","first-page":"470","DOI":"10.7150\/jca.102759","article-title":"Evaluation of band selection for spectrum-aided visual enhancer (SAVE) for esophageal cancer detection","volume":"16","author":"Y-C Chen","year":"2025","journal-title":"Journal of Cancer"},{"key":"pone.0322028.ref014","doi-asserted-by":"crossref","first-page":"1423405","DOI":"10.3389\/fonc.2024.1423405","article-title":"Computer-aided endoscopic diagnostic system modified with hyperspectral imaging for the classification of esophageal neoplasms","volume":"14","author":"Y-K Wang","year":"2024","journal-title":"Front Oncol"},{"issue":"1","key":"pone.0322028.ref015","doi-asserted-by":"crossref","first-page":"22243","DOI":"10.1038\/s41598-024-73387-8","article-title":"Spectrum aided vision enhancer enhances mucosal visualization by hyperspectral imaging in capsule endoscopy","volume":"14","author":"Y-P Wang","year":"2024","journal-title":"Sci Rep"},{"key":"pone.0322028.ref016","doi-asserted-by":"crossref","first-page":"571","DOI":"10.1007\/978-3-030-87237-3_55","article-title":"Contrastive learning based stain normalization across multiple tumor in histopathology","volume":"2021","author":"J Ke","year":"2021","journal-title":"Medical Image Computing and Computer Assisted Intervention \u2013 MICCAI 2021"},{"issue":"11","key":"pone.0322028.ref017","doi-asserted-by":"crossref","first-page":"7199","DOI":"10.1002\/mp.15172","article-title":"TN-USMA Net: Triple normalization-based gastrointestinal stromal tumors classification on multicenter EUS images with ultrasound-specific pretraining and meta attention","volume":"48","author":"C Liu","year":"2021","journal-title":"Med Phys"},{"issue":"1","key":"pone.0322028.ref018","doi-asserted-by":"crossref","first-page":"94","DOI":"10.1136\/gutjnl-2017-314547","article-title":"Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model","volume":"68","author":"MF Byrne","year":"2019","journal-title":"Gut"},{"key":"pone.0322028.ref019","first-page":"69","article-title":"Unsupervised learning of visual representations by solving jigsaw puzzles.","volume-title":"European conference on computer vision","author":"M Noroozi","year":"2016"},{"key":"pone.0322028.ref020","first-page":"2794","article-title":"Unsupervised learning of visual representations using videos","author":"X Wang","year":"2015","journal-title":"Proceedings of the IEEE International Conference on Computer Vision"},{"key":"pone.0322028.ref021","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1145\/3474906.3474912","article-title":"Helicobacter pylori infection classification based on convolutional neural network and self-supervised learning","author":"G-Z Jian","year":"2021","journal-title":"Proceedings of the 5th International Conference on Graphics and Signal Processing"},{"issue":"1","key":"pone.0322028.ref022","doi-asserted-by":"crossref","first-page":"6100","DOI":"10.1038\/s41598-024-53955-8","article-title":"Improving image classification of gastrointestinal endoscopy using curriculum self-supervised learning","volume":"14","author":"H Guo","year":"2024","journal-title":"Scientific Reports"},{"key":"pone.0322028.ref023","first-page":"1","article-title":"A preliminary analysis of self-supervision for wireless capsule endoscopy","author":"A Vats","year":"2021","journal-title":"2021 9th European Workshop on Visual Information Processing (EUVIP)"},{"key":"pone.0322028.ref024","doi-asserted-by":"crossref","first-page":"105631","DOI":"10.1016\/j.compbiomed.2022.105631","article-title":"Time-based self-supervised learning for Wireless Capsule Endoscopy.","volume":"146","author":"Pascual G","year":"2022","journal-title":"Computers in Biology and Medicine."},{"key":"pone.0322028.ref025","first-page":"649","article-title":"Colorful image colorization,in Computer Vision\u2013ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11\u201314, 2016, Proceedings, Part III 14","author":"R Zhang","year":"2016"},{"key":"pone.0322028.ref026","first-page":"2536","article-title":"Context encoders: Feature learning by inpainting.","author":"D Pathak","year":"2016","journal-title":"Proceedings of the IEEE conference on computer vision and pattern recognition"},{"key":"pone.0322028.ref027","first-page":"1422","article-title":"Unsupervised visual representation learning by context prediction","author":"C Doersch","year":"2015","journal-title":"Proceedings of the IEEE international conference on computer vision"},{"key":"pone.0322028.ref028","doi-asserted-by":"crossref","first-page":"336","DOI":"10.1007\/s11263-019-01228-7","article-title":"Grad-CAM: visual explanations from deep networks via gradient-based localization","volume":"128","author":"RR Selvaraju","year":"2020","journal-title":"International Journal of Computer Vision"},{"key":"pone.0322028.ref029","article-title":"A unified approach to interpreting model predictions.","volume-title":"Advances in Neural Information Processing Systems","author":"SM Lundberg","year":"2017"}],"container-title":["PLOS One"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0322028","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,8]],"date-time":"2025-05-08T20:03:55Z","timestamp":1746734635000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0322028"}},"subtitle":[],"editor":[{"given":"Lei","family":"Chu","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2025,5,8]]},"references-count":29,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2025,5,8]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0322028","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,8]]}}}