{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,8]],"date-time":"2026-01-08T09:55:29Z","timestamp":1767866129900,"version":"3.49.0"},"reference-count":57,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,8,9]],"date-time":"2024-08-09T00:00:00Z","timestamp":1723161600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Robot. AI"],"abstract":"<jats:p>Colonoscopy is a reliable diagnostic method to detect colorectal polyps early on and prevent colorectal cancer. The current examination techniques face a significant challenge of high missed rates, resulting in numerous undetected polyps and irregularities. Automated and real-time segmentation methods can help endoscopists to segment the shape and location of polyps from colonoscopy images in order to facilitate clinician\u2019s timely diagnosis and interventions. Different parameters like shapes, small sizes of polyps, and their close resemblance to surrounding tissues make this task challenging. Furthermore, high-definition image quality and reliance on the operator make real-time and accurate endoscopic image segmentation more challenging. Deep learning models utilized for segmenting polyps, designed to capture diverse patterns, are becoming progressively complex. This complexity poses challenges for real-time medical operations. In clinical settings, utilizing automated methods requires the development of accurate, lightweight models with minimal latency, ensuring seamless integration with endoscopic hardware devices. To address these challenges, in this study a novel lightweight and more generalized Enhanced Nanonet model, an improved version of Nanonet using NanonetB for real-time and precise colonoscopy image segmentation, is proposed. The proposed model enhances the performance of Nanonet using Nanonet B on the overall prediction scheme by applying data augmentation, Conditional Random Field (CRF), and Test-Time Augmentation (TTA). Six publicly available datasets are utilized to perform thorough evaluations, assess generalizability, and validate the improvements: Kvasir-SEG, Endotect Challenge 2020, Kvasir-instrument, CVC-ClinicDB, CVC-ColonDB, and CVC-300. Through extensive experimentation, using the Kvasir-SEG dataset, our model achieves a mIoU score of 0.8188 and a Dice coefficient of 0.8060 with only 132,049 parameters and employing minimal computational resources. A thorough cross-dataset evaluation was performed to assess the generalization capability of the proposed Enhanced Nanonet model across various publicly available polyp datasets for potential real-world applications. The result of this study shows that using CRF (Conditional Random Fields) and TTA (Test-Time Augmentation) enhances performance within the same dataset and also across diverse datasets with a model size of just 132,049 parameters. Also, the proposed method indicates improved results in detecting smaller and sessile polyps (flats) that are significant contributors to the high miss rates.<\/jats:p>","DOI":"10.3389\/frobt.2024.1387491","type":"journal-article","created":{"date-parts":[[2024,8,9]],"date-time":"2024-08-09T04:11:38Z","timestamp":1723176698000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Enhanced accuracy with Segmentation of Colorectal Polyp using NanoNetB, and Conditional Random Field Test-Time Augmentation"],"prefix":"10.3389","volume":"11","author":[{"given":"Muhammad Sajjad","family":"Hussain","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Umer","family":"Asgher","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sajid","family":"Nisar","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Vladimir","family":"Socha","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Arslan","family":"Shaukat","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jinhui","family":"Wang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tian","family":"Feng","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rehan Zafar","family":"Paracha","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Muhammad Ali","family":"Khan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2024,8,9]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"16178","DOI":"10.3748\/WJG.V20.I43.16178","article-title":"Management of malignant colon polyps: current status and controversies","volume":"20","author":"Aarons","year":"2014","journal-title":"World J. Gastroenterology WJG"},{"key":"B2","unstructured":"TensorFlow: A system for large-scale machine learning\n          \n          \n            \n              Abadi\n              M.\n            \n            \n              Barham\n              P.\n            \n            \n              Chen\n              J.\n            \n            \n              Chen\n              Z.\n            \n            \n              Davis\n              A.\n            \n            \n              Dean\n              J.\n            \n          \n          \n          2016"},{"key":"B3","doi-asserted-by":"publisher","first-page":"1612","DOI":"10.1109\/TGRS.2018.2867679","article-title":"Conditional Random Field and Deep Feature Learning for hyperspectral Image Classification","volume":"57","author":"Alam","year":"2019","journal-title":"IEEE Trans. Geoscience Remote Sens."},{"key":"B4","unstructured":"Recurrent Residual Convolutional Neural Network based on U-Net (R2U-Net) for Medical Image Segmentation\n          \n          \n            \n              Alom\n              M. Z.\n            \n            \n              Hasan\n              M.\n            \n            \n              Yakopcic\n              C.\n            \n            \n              Taha\n              T. M.\n            \n            \n              Arasi\n              V. K.\n            \n          \n          \n          2018"},{"key":"B5","doi-asserted-by":"publisher","first-page":"346","DOI":"10.1007\/978-3-540-93860-6_70","article-title":"Texture-based polyp detection in colonoscopy","author":"Ameling","year":"2009","journal-title":"Inf. aktuell"},{"key":"B6","doi-asserted-by":"publisher","first-page":"245","DOI":"10.3390\/INFO12060245","article-title":"Deep Learning Models for Colorectal Polyps","volume":"12","author":"Bardhi","year":"2021","journal-title":"Inf. 2021"},{"key":"B7","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1016\/J.COMPMEDIMAG.2015.02.007","article-title":"WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians","volume":"43","author":"Bernal","year":"2015","journal-title":"Comput. Med. imaging Graph."},{"key":"B8","doi-asserted-by":"publisher","first-page":"3166","DOI":"10.1016\/J.PATCOG.2012.03.002","article-title":"Towards automatic polyp detection with a polyp appearance model","volume":"45","author":"Bernal","year":"2012","journal-title":"Pattern Recognit."},{"key":"B9","unstructured":"Comparative evaluation of instrument segmentation and tracking methods in minimally invasive surgery\n          \n          \n            \n              Bodenstedt\n              S.\n            \n            \n              Allan\n              M.\n            \n            \n              Agustinos\n              A.\n            \n            \n              Du\n              X.\n            \n            \n              Garcia-Peraza-Herrera\n              L.\n            \n            \n              Kenngott\n              H.\n            \n          \n          \n          2018"},{"key":"B10","doi-asserted-by":"publisher","first-page":"394","DOI":"10.3322\/CAAC.21492","article-title":"Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries","volume":"68","author":"Bray","year":"2018","journal-title":"CA a cancer J. Clin."},{"key":"B11","doi-asserted-by":"publisher","first-page":"834","DOI":"10.1109\/TPAMI.2017.2699184","article-title":"DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and fully Connected CRFs","volume":"40","author":"Chen","year":"2018","journal-title":"IEEE Trans. Pattern Analysis Mach. Intell."},{"key":"B12","first-page":"248","article-title":"ImageNet: A large-scale hierarchical image database","author":"Deng","year":"2010"},{"key":"B13","doi-asserted-by":"publisher","first-page":"291","DOI":"10.1136\/GUTJNL-2016-312712","article-title":"Effectiveness of screening colonoscopy in reducing the risk of death from right and left colon cancer: a large community-based study","volume":"67","author":"Doubeni","year":"2018","journal-title":"Gut"},{"key":"B57","first-page":"10008","article-title":"The importance of skip connections in biomedical image segmentation","volume-title":"Deep learning and data labeling for medical applications. DLMIA LABELS 2016. Lecture Notes in Computer Science","author":"Drozdzal","year":"2016"},{"key":"B14","first-page":"770","article-title":"Deep residual learning for image recognition","author":"He","year":"2016"},{"key":"B15","doi-asserted-by":"publisher","first-page":"263","DOI":"10.1007\/978-3-030-68793-9_18","article-title":"The EndoTect 2020 Challenge: Evaluation and Comparison of Classification, Segmentation and Inference Time for Endoscopy","author":"Hicks","year":"2021","journal-title":"Lect. Notes Comput. Sci. Incl. Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinforma. 12668 LNCS"},{"key":"B16","doi-asserted-by":"publisher","first-page":"815","DOI":"10.1109\/tpami.2018.2815688","article-title":"Deeply supervised salient object detection with short connections","volume":"41","author":"Hou","year":"2016","journal-title":"IEEE Trans. Pattern Analysis Mach. Intell."},{"key":"B17","first-page":"7132","article-title":"Squeeze-and-Excitation Networks","author":"Hu","year":"2018"},{"key":"B18","unstructured":"HarDNet-MSEG: A Simple Encoder-Decoder Polyp Segmentation Neural Network that Achieves over 0.9 Mean Dice and 86 FPS\n          \n          \n            \n              Huang\n              C.-H.\n            \n            \n              Wu\n              H.-Y.\n            \n            \n              Lin\n              Y.-L.\n            \n          \n          \n          2021"},{"key":"B19","first-page":"1055","article-title":"UNet 3+: A Full-Scale Connected UNet for Medical Image Segmentation","author":"Huang","year":"2020"},{"key":"B20","unstructured":"SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0\n          \n          \n            \n              Iandola\n              F. N.\n            \n            \n              Han\n              S.\n            \n            \n              Moskewicz\n              M. W.\n            \n            \n              Ashraf\n              K.\n            \n            \n              Dally\n              W. J.\n            \n            \n              Keutzer\n              K.\n            \n          \n          \n          2016"},{"key":"B21","doi-asserted-by":"publisher","first-page":"74","DOI":"10.1016\/J.NEUNET.2019.08.025","article-title":"MultiResUNet : Rethinking the U-Net architecture for multimodal biomedical image segmentation","volume":"121","author":"Ibtehaz","year":"2020","journal-title":"Neural Netw."},{"key":"B22","doi-asserted-by":"publisher","first-page":"5086","DOI":"10.3748\/WJG.V23.I28.5086","article-title":"Colorectal cancer screening: An updated review of the available options","volume":"23","author":"Issa","year":"2017","journal-title":"World J. gastroenterology"},{"key":"B23","doi-asserted-by":"publisher","first-page":"1893","DOI":"10.1158\/1055-9965.EPI-10-0437","article-title":"Global patterns of cancer incidence and mortality rates and trends","volume":"19","author":"Jemal","year":"2010","journal-title":"Cancer Epidemiol. biomarkers Prev."},{"key":"B24","doi-asserted-by":"publisher","first-page":"218","DOI":"10.1007\/978-3-030-67835-7_19","article-title":"Kvasir-Instrument: Diagnostic and therapeutic tool segmentation dataset in gastrointestinal endoscopy","author":"Jha","year":"","journal-title":"Lect. Notes Comput. Sci."},{"key":"B25","unstructured":"TransNetR: Transformer-based Residual Network for Polyp Segmentation with Multi-Center Out-of-Distribution Testing\n          \n          \n            \n              Jha\n              D.\n            \n            \n              Tomar\n              N. K.\n            \n            \n              Sharma\n              V.\n            \n            \n              Bagci\n              U.\n            \n          \n          \n          2023"},{"key":"B26","first-page":"558","article-title":"DoubleU-Net: A deep convolutional neural network for medical image segmentation","author":"Jha","year":""},{"key":"B27","doi-asserted-by":"publisher","first-page":"2029","DOI":"10.1109\/JBHI.2021.3049304","article-title":"A Comprehensive Study on Colorectal Polyp Segmentation With ResUNet++, Conditional Random Field and Test-Time Augmentation","volume":"25","author":"Jha","year":"","journal-title":"IEEE J. Biomed. health Inf."},{"key":"B28","doi-asserted-by":"publisher","first-page":"451","DOI":"10.1007\/978-3-030-37734-2_37","article-title":"Kvasir-SEG: A Segmented Polyp Dataset","author":"Jha","year":"","journal-title":"Lect. Notes Comput. Sci."},{"key":"B29","first-page":"225","article-title":"ResUNet++: An Advanced Architecture for Medical Image Segmentation","author":"Jha","year":""},{"key":"B30","first-page":"37","article-title":"NanoNet: Real-Time Polyp Segmentation in Video Capsule Endoscopy and Colonoscopy","author":"Jha","year":""},{"key":"B31","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1007\/978-3-030-66519-7_4","article-title":"Keras and TensorFlow: A Hands-On Experience","author":"Joseph","year":"2021","journal-title":"EAI\/Springer Innovations Commun. Comput."},{"key":"B32","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1109\/TITB.2003.813794","article-title":"Computer-aided tumor detection in endoscopic video using color wavelet features","volume":"7","author":"Karkanis","year":"2003","journal-title":"IEEE Trans. Inf. Technol. Biomed."},{"key":"B33","unstructured":"Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications\n          \n          \n            \n              Kim\n              Y. D.\n            \n            \n              Park\n              E.\n            \n            \n              Yoo\n              S.\n            \n            \n              Choi\n              T.\n            \n            \n              Yang\n              L.\n            \n            \n              Shin\n              D.\n            \n          \n          \n          2015"},{"key":"B34","doi-asserted-by":"publisher","first-page":"e7468","DOI":"10.1097\/MD.0000000000007468","article-title":"Risk factors of missed colorectal lesions after colonoscopy","volume":"96","author":"Lee","year":"2017","journal-title":"Medicine"},{"key":"B35","doi-asserted-by":"publisher","first-page":"8379","DOI":"10.1038\/S41598-020-65387-1","article-title":"Real-time detection of colon polyps during colonoscopy using deep learning: systematic validation with four independent datasets","volume":"10","author":"Lee","year":"2020","journal-title":"Sci. Rep."},{"key":"B36","unstructured":"Pyramid Attention Network for Semantic Segmentation\n          \n          \n            \n              Li\n              H.\n            \n            \n              Xiong\n              P.\n            \n            \n              An\n              J.\n            \n            \n              Wang\n              L.\n            \n          \n          \n          2018"},{"key":"B37","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1016\/J.MEDIA.2017.07.005","article-title":"A survey on deep learning in medical image analysis","volume":"42","author":"Litjens","year":"2017","journal-title":"Med. image Anal."},{"key":"B38","first-page":"565","article-title":"V-Net: Fully convolutional neural networks for volumetric medical image segmentation","author":"Milletari","year":"2016"},{"key":"B39","doi-asserted-by":"publisher","first-page":"5068","DOI":"10.1038\/S41598-020-61808-3","article-title":"Test-time augmentation for deep learning-based cell segmentation on microscopy images","volume":"10","author":"Moshkov","year":"2020","journal-title":"Sci. Rep."},{"key":"B40","unstructured":"Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation\n          \n          \n            \n              Powers\n              D. M. W.\n            \n          \n          \n          2020"},{"key":"B41","doi-asserted-by":"publisher","first-page":"107404","DOI":"10.1016\/j.patcog.2020.107404","article-title":"U2-Net: Going deeper with nested U-structure for salient object detection","volume":"106","author":"Qin","year":"2020","journal-title":"Pattern Recognit."},{"key":"B42","doi-asserted-by":"publisher","first-page":"234","DOI":"10.1007\/978-3-319-24574-4_28","article-title":"U-net: Convolutional networks for biomedical image segmentation","volume":"9351","author":"Ronneberger","year":"2015","journal-title":"Lect. Notes Comput. Sci."},{"key":"B43","doi-asserted-by":"publisher","first-page":"917","DOI":"10.1007\/S00138-017-0864-0","article-title":"Bright spot regions segmentation and classification for specular highlights detection in colonoscopy videos","volume":"28","author":"S\u00e1nchez","year":"2017","journal-title":"Mach. Vis. Appl."},{"key":"B44","first-page":"4510","article-title":"MobileNetV2: Inverted Residuals and Linear Bottlenecks","author":"Sandler","year":"2018"},{"key":"B45","doi-asserted-by":"publisher","first-page":"306977","DOI":"10.1101\/306977","article-title":"Continuous Dice Coefficient: a Method for Evaluating Probabilistic Segmentations","author":"Shamir","year":"2018","journal-title":"bioRxiv"},{"key":"B46","doi-asserted-by":"publisher","first-page":"2252","DOI":"10.1109\/JBHI.2021.3138024","article-title":"MSRF-Net: A Multi-Scale Residual Fusion Network for Biomedical Image Segmentation","volume":"26","author":"Srivastava","year":"2021","journal-title":"IEEE J. Biomed. Health Inf."},{"key":"B47","doi-asserted-by":"publisher","first-page":"797","DOI":"10.1007\/978-3-030-59719-1_77","article-title":"SAUNet: Shape Attentive U-Net for Interpretable Medical Image Segmentation","author":"Sun","year":"2020","journal-title":"Lect. Notes Comput. Sci."},{"key":"B48","doi-asserted-by":"crossref","DOI":"10.1109\/ICSCC.2019.8843652","article-title":"Vanishing Gradient Mitigation with Deep Learning Neural Network Optimization","author":"Tan","year":"2019"},{"key":"B49","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1586\/17474124.2015.960397","article-title":"Colonoscopy: is it as effective as an advanced diagnostic tool for colorectal cancer screening?","volume":"9","author":"Uraoka","year":"2015","journal-title":"Expert Rev. gastroenterology hepatology"},{"key":"B50","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1007\/978-3-031-16443-9_3","article-title":"UNeXt: MLP-Based Rapid Medical Image Segmentation Network","author":"Valanarasu","year":"2022","journal-title":"Lect. Notes Comput. Sci."},{"key":"B51","doi-asserted-by":"publisher","first-page":"1562","DOI":"10.1109\/TMI.2018.2791721","article-title":"Interactive Medical Image Segmentation Using Deep Learning With Image-Specific Fine Tuning","volume":"37","author":"Wang","year":"2018","journal-title":"Ieee Trans. Med. Imaging"},{"key":"B52","first-page":"1860","article-title":"LEDNet: A Lightweight Encoder-Decoder Network for Real-Time Semantic Segmentation","author":"Wang","year":"2019"},{"key":"B53","doi-asserted-by":"publisher","first-page":"10927","DOI":"10.3748\/WJG.V20.I31.10927","article-title":"Risk factors associated with missed colorectal flat adenoma: A multicenter retrospective tandem colonoscopy study","volume":"20","author":"Xiang","year":"2014","journal-title":"World J. Gastroenterology WJG"},{"key":"B54","doi-asserted-by":"publisher","first-page":"14465","DOI":"10.1038\/s41598-019-50567-5","article-title":"Development of a real-time endoscopic image diagnosis support system using deep learning technology in colonoscopy","volume":"9","author":"Yamada","year":"2019","journal-title":"Sci. Rep. 2019"},{"key":"B55","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1007\/978-3-030-00889-5_1","article-title":"UNet++: A Nested U-Net Architecture for Medical Image Segmentation","volume":"11045","author":"Zhou","year":"2018","journal-title":"Lect. Notes Comput. Sci."},{"key":"B56","doi-asserted-by":"publisher","first-page":"660","DOI":"10.1053\/J.GASTRO.2019.05.011","article-title":"Right-Sided Location Not Associated With Missed Colorectal Adenomas in an Individual-Level Reanalysis of Tandem Colonoscopy Studies","volume":"157","author":"Zimmermann-Fraedrich","year":"2019","journal-title":"Gastroenterology"}],"container-title":["Frontiers in Robotics and AI"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2024.1387491\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,9]],"date-time":"2024-08-09T04:11:48Z","timestamp":1723176708000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2024.1387491\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,9]]},"references-count":57,"alternative-id":["10.3389\/frobt.2024.1387491"],"URL":"https:\/\/doi.org\/10.3389\/frobt.2024.1387491","relation":{},"ISSN":["2296-9144"],"issn-type":[{"value":"2296-9144","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,8,9]]},"article-number":"1387491"}}