{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,20]],"date-time":"2026-05-20T18:04:45Z","timestamp":1779300285687,"version":"3.51.4"},"reference-count":46,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,7,25]],"date-time":"2023-07-25T00:00:00Z","timestamp":1690243200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Robot. AI"],"abstract":"<jats:p>Optical colonoscopy is the gold standard procedure to detect colorectal cancer, the fourth most common cancer in the United Kingdom. Up to 22%\u201328% of polyps can be missed during the procedure that is associated with interval cancer. A vision-based autonomous soft endorobot for colonoscopy can drastically improve the accuracy of the procedure by inspecting the colon more systematically with reduced discomfort. A three-dimensional understanding of the environment is essential for robot navigation and can also improve the adenoma detection rate. Monocular depth estimation with deep learning methods has progressed substantially, but collecting ground-truth depth maps remains a challenge as no 3D camera can be fitted to a standard colonoscope. This work addresses this issue by using a self-supervised monocular depth estimation model that directly learns depth from video sequences with view synthesis. In addition, our model accommodates wide field-of-view cameras typically used in colonoscopy and specific challenges such as deformable surfaces, specular lighting, non-Lambertian surfaces, and high occlusion. We performed qualitative analysis on a synthetic data set, a quantitative examination of the colonoscopy training model, and real colonoscopy videos in near real-time.<\/jats:p>","DOI":"10.3389\/frobt.2023.1212525","type":"journal-article","created":{"date-parts":[[2023,7,25]],"date-time":"2023-07-25T17:12:21Z","timestamp":1690305141000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["Self-supervised monocular depth estimation for high field of view colonoscopy cameras"],"prefix":"10.3389","volume":"10","author":[{"given":"Alwyn","family":"Mathew","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ludovic","family":"Magerand","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Emanuele","family":"Trucco","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Luigi","family":"Manfredi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2023,7,25]]},"reference":[{"key":"B1","volume-title":"EndoMapper dataset of complete calibrated endoscopy","author":"Azagra","year":"2022"},{"key":"B2","first-page":"774","article-title":"Deep multi-view stereo for dense 3d reconstruction from monocular endoscopic video","author":"Bae","year":"2020"},{"key":"B3","doi-asserted-by":"publisher","first-page":"1","DOI":"10.7326\/0003-4819-150-1-200901060-00306","article-title":"Association of colonoscopy and death from colorectal cancer","volume":"150","author":"Baxter","year":"2009","journal-title":"Ann. Intern. Med."},{"key":"B4","doi-asserted-by":"publisher","first-page":"1718","DOI":"10.1109\/lra.2017.2678540","article-title":"A novel robotic meshworm with segment-bending anchoring for colonoscopy","volume":"2","author":"Bernth","year":"2017","journal-title":"IEEE Robotics Automation Lett."},{"key":"B5","doi-asserted-by":"publisher","first-page":"2548","DOI":"10.1007\/s11263-021-01484-6","article-title":"Unsupervised scale-consistent depth learning from video","volume":"129","author":"Bian","year":"2021","journal-title":"Int. J. Comput. Vis."},{"key":"B6","doi-asserted-by":"publisher","first-page":"394","DOI":"10.3322\/caac.21492","article-title":"Global cancer statistics 2018: Globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries","volume":"68","author":"Bray","year":"2018","journal-title":"CA a cancer J. Clin."},{"key":"B7","first-page":"119","article-title":"Depth estimation for colonoscopy images with self-supervised learning from videos","author":"Cheng","year":"2021"},{"key":"B8","doi-asserted-by":"publisher","first-page":"1648","DOI":"10.3390\/jcm9061648","article-title":"Frontiers of robotic colonoscopy: A comprehensive review of robotic colonoscopes and technologies","volume":"9","author":"Ciuti","year":"2020","journal-title":"J. Clin. Med."},{"key":"B9","doi-asserted-by":"crossref","DOI":"10.1586\/17434440.2013.868303","volume-title":"3d imaging techniques for improved colonoscopy","author":"Durr","year":"2014"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.5555\/2969033.2969091","article-title":"Depth map prediction from a single image using a multi-scale deep network","volume":"27","author":"Eigen","year":"2014","journal-title":"Adv. neural Inf. Process. Syst."},{"key":"B11","doi-asserted-by":"publisher","first-page":"545","DOI":"10.1109\/tro.2019.2949466","article-title":"Novel optimization-based design and surgical evaluation of a treaded robotic capsule colonoscope","volume":"36","author":"Formosa","year":"2019","journal-title":"IEEE Trans. Robotics"},{"key":"B12","doi-asserted-by":"publisher","first-page":"3451","DOI":"10.1109\/tmi.2020.2994221","article-title":"Detecting deficient coverage in colonoscopies","volume":"39","author":"Freedman","year":"2020","journal-title":"IEEE Trans. Med. Imaging"},{"key":"B13","first-page":"51","article-title":"Lens distortion for close-range photogrammetry","volume":"52","author":"Fryer","year":"1986","journal-title":"Photogrammetric Eng. remote Sens."},{"key":"B14","doi-asserted-by":"crossref","DOI":"10.1007\/3-540-45053-X_29","article-title":"A unifying theory for central panoramic systems and practical implications","author":"Geyer","year":"2000"},{"key":"B15","first-page":"6602","article-title":"Unsupervised monocular depth estimation with left-right consistency","author":"Godard","year":"2017"},{"key":"B16","first-page":"3827","article-title":"Digging into self-supervised monocular depth estimation","author":"Godard","year":"2019"},{"key":"B17","first-page":"770","article-title":"Deep residual learning for image recognition","author":"He","year":"2016"},{"key":"B18","doi-asserted-by":"publisher","first-page":"2691","DOI":"10.3390\/s21082691","article-title":"Unsupervised monocular depth estimation for colonoscope system using feedback network","volume":"21","author":"Hwang","year":"2021","journal-title":"Sensors"},{"key":"B19","first-page":"611","article-title":"Towards automated colonoscopy diagnosis: Binary polyp size estimation via unsupervised depth learning","author":"Itoh","year":"2018"},{"key":"B20","article-title":"Spatial transformer networks","volume":"28","author":"Jaderberg","year":"2015","journal-title":"Adv. neural Inf. Process. Syst."},{"key":"B21","doi-asserted-by":"publisher","first-page":"102478","DOI":"10.1016\/j.mechatronics.2020.102478","article-title":"A novel robotic colonoscopy system integrating feeding and steering mechanisms with self-propelled paddling locomotion: A pilot study","volume":"73","author":"Kang","year":"2021","journal-title":"Mechatronics"},{"key":"B22","doi-asserted-by":"publisher","first-page":"1335","DOI":"10.1109\/tpami.2006.153","article-title":"A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses","volume":"28","author":"Kannala","year":"2006","journal-title":"IEEE Trans. pattern analysis Mach. Intell."},{"key":"B23","doi-asserted-by":"publisher","first-page":"137","DOI":"10.1109\/lra.2015.2502921","article-title":"An enhanced unified camera model","volume":"1","author":"Khomutenko","year":"2015","journal-title":"IEEE Robotics Automation Lett."},{"key":"B24","volume-title":"Adam: A method for stochastic optimization","author":"Kingma","year":"2014"},{"key":"B25","doi-asserted-by":"publisher","first-page":"957","DOI":"10.1136\/gutjnl-2013-304880","article-title":"Postcolonoscopy colorectal cancers are preventable: A population-based study","volume":"63","author":"le Clercq","year":"2014","journal-title":"Gut"},{"key":"B26","doi-asserted-by":"publisher","first-page":"8379","DOI":"10.1038\/s41598-020-65387-1","article-title":"Real-time detection of colon polyps during colonoscopy using deep learning: Systematic validation with four independent datasets","volume":"10","author":"Lee","year":"2020","journal-title":"Sci. Rep."},{"key":"B27","first-page":"2294","article-title":"Hr-depth: High resolution self-supervised monocular depth estimation","author":"Lyu","year":"2021"},{"key":"B28","first-page":"387","article-title":"Ldpolypvideo benchmark: A large-scale colonoscopy video dataset of diverse polyps","author":"Ma","year":"2021"},{"key":"B29","doi-asserted-by":"publisher","first-page":"2572","DOI":"10.1109\/tmi.2018.2842767","article-title":"Unsupervised reverse domain adaptation for synthetic medical images via adversarial training","volume":"37","author":"Mahmood","year":"2018","journal-title":"IEEE Trans. Med. imaging"},{"key":"B30","doi-asserted-by":"publisher","first-page":"230","DOI":"10.1016\/j.media.2018.06.005","article-title":"Deep learning and conditional random fields-based depth estimation and topographical reconstruction from conventional endoscopy","volume":"48","author":"Mahmood","year":"2018","journal-title":"Med. image Anal."},{"key":"B31","doi-asserted-by":"publisher","first-page":"11109","DOI":"10.1038\/s41598-019-47320-3","article-title":"A soft pneumatic inchworm double balloon (spid) for colonoscopy","volume":"9","author":"Manfredi","year":"2019","journal-title":"Sci. Rep."},{"key":"B32","volume-title":"Endorobotics: Design, R&D and future trends","author":"Manfredi","year":"2022"},{"key":"B33","doi-asserted-by":"publisher","first-page":"705454","DOI":"10.3389\/frobt.2021.705454","article-title":"Endorobots for colonoscopy: Design challenges and available technologies","volume":"8","author":"Manfredi","year":"2021","journal-title":"Front. Robotics AI"},{"key":"B34","volume-title":"Depth reconstruction and computer-aided polyp detection in optical colonoscopy video frames","author":"Nadeem","year":"2016"},{"key":"B35","article-title":"Automatic differentiation in pytorch","volume-title":"NIPS-W","author":"Paszke","year":"2017"},{"key":"B36","doi-asserted-by":"publisher","first-page":"485","DOI":"10.1136\/gut.2010.229534","article-title":"Pancolonic chromoendoscopy with indigo carmine versus standard colonoscopy for detection of neoplastic lesions: A randomised two-centre trial","volume":"60","author":"Pohl","year":"2011","journal-title":"Gut"},{"key":"B37","doi-asserted-by":"publisher","first-page":"1167","DOI":"10.1007\/s11548-019-01962-w","article-title":"Implicit domain adaptation with conditional generative adversarial networks for depth prediction in endoscopy","volume":"14","author":"Rau","year":"2019","journal-title":"Int. J. Comput. assisted radiology Surg."},{"key":"B38","first-page":"45","article-title":"A flexible technique for accurate omnidirectional camera calibration and structure from motion","author":"Scaramuzza","year":"2006"},{"key":"B39","first-page":"7159","article-title":"Self-supervised learning for monocular depth estimation on minimally invasive surgery scenes","author":"Shao","year":"2021"},{"key":"B40","doi-asserted-by":"publisher","first-page":"102338","DOI":"10.1016\/j.media.2021.102338","article-title":"Self-supervised monocular depth and ego-motion estimation in endoscopy: Appearance flow to the rescue","volume":"77","author":"Shao","year":"2022","journal-title":"Med. image Anal."},{"key":"B41","first-page":"552","article-title":"The double sphere camera model","author":"Usenko","year":"2018"},{"key":"B42","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR.2018.00216","article-title":"Learning depth from monocular videos using direct methods","author":"Wang","year":"2018"},{"key":"B43","doi-asserted-by":"publisher","first-page":"600","DOI":"10.1109\/tip.2003.819861","article-title":"Image quality assessment: From error visibility to structural similarity","volume":"13","author":"Wang","year":"2004","journal-title":"IEEE Trans. image Process."},{"key":"B44","first-page":"442","article-title":"Can we consider central catadioptric cameras and fisheye cameras within a unified imaging model","author":"Ying","year":"2004"},{"key":"B45","volume-title":"Unsupervised learning of monocular depth estimation with bundle adjustment, super-resolution and clip loss","author":"Zhou","year":"2018"},{"key":"B46","first-page":"1851","article-title":"Unsupervised learning of depth and ego-motion from video","author":"Zhou","year":"2017"}],"container-title":["Frontiers in Robotics and AI"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2023.1212525\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,7,25]],"date-time":"2023-07-25T17:12:30Z","timestamp":1690305150000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2023.1212525\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,25]]},"references-count":46,"alternative-id":["10.3389\/frobt.2023.1212525"],"URL":"https:\/\/doi.org\/10.3389\/frobt.2023.1212525","relation":{},"ISSN":["2296-9144"],"issn-type":[{"value":"2296-9144","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,25]]},"article-number":"1212525"}}