{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,20]],"date-time":"2026-05-20T18:04:48Z","timestamp":1779300288583,"version":"3.51.4"},"reference-count":30,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2021,3,30]],"date-time":"2021-03-30T00:00:00Z","timestamp":1617062400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,3,30]],"date-time":"2021-03-30T00:00:00Z","timestamp":1617062400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100002920","name":"Research Grants Council, University Grants Committee","doi-asserted-by":"publisher","award":["17205919; 17206818; 17202317"],"award-info":[{"award-number":["17205919; 17206818; 17202317"]}],"id":[{"id":"10.13039\/501100002920","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J CARS"],"published-print":{"date-parts":[[2021,5]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Purpose<\/jats:title><jats:p>Surgical annotation promotes effective communication between medical personnel during surgical procedures. However, existing approaches to 2D annotations are mostly static with respect to a display. In this work, we propose a method to achieve 3D annotations that anchor rigidly and stably to target structures upon camera movement in a transnasal endoscopic surgery setting.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>This is accomplished through intra-operative endoscope tracking and monocular depth estimation. A virtual endoscopic environment is utilized to train a supervised depth estimation network. An adversarial network transfers the style from the real endoscopic view to a synthetic-like view for input into the depth estimation network, wherein framewise depth can be obtained in real time.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>(1) Accuracy: Framewise depth was predicted from images captured from within a nasal airway phantom and compared with ground truth, achieving a SSIM value of 0.8310 \u00b1 0.0655. (2) Stability: mean absolute error (MAE) between reference and predicted depth of a target point was 1.1330 \u00b1 0.9957 mm.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p>Both the accuracy and stability evaluations demonstrated the feasibility and practicality of our proposed method for achieving 3D annotations.<\/jats:p><\/jats:sec>","DOI":"10.1007\/s11548-021-02346-9","type":"journal-article","created":{"date-parts":[[2021,3,30]],"date-time":"2021-03-30T16:03:27Z","timestamp":1617120207000},"page":"731-739","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Real-to-virtual domain transfer-based depth estimation for real-time 3D annotation in transnasal surgery: a study of annotation accuracy and stability"],"prefix":"10.1007","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3370-7884","authenticated-orcid":false,"given":"Hon-Sing","family":"Tong","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yui-Lun","family":"Ng","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhiyu","family":"Liu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Justin D. L.","family":"Ho","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Po-Ling","family":"Chan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9480-4637","authenticated-orcid":false,"given":"Jason Y. K.","family":"Chan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1879-9730","authenticated-orcid":false,"given":"Ka-Wai","family":"Kwok","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,3,30]]},"reference":[{"issue":"8","key":"2346_CR1","doi-asserted-by":"publisher","first-page":"2754","DOI":"10.1245\/s10434-013-2894-9","volume":"20","author":"S Treter","year":"2013","unstructured":"Treter S, Perrier N, Sosa JA, Roman S (2013) Telementoring: a multi-institutional experience with the introduction of a novel surgical approach for adrenalectomy. Ann Surg Oncol 20(8):2754\u20132758","journal-title":"Ann Surg Oncol"},{"issue":"10","key":"2346_CR2","doi-asserted-by":"publisher","first-page":"2156","DOI":"10.1007\/s10439-012-0578-4","volume":"40","author":"KW Kwok","year":"2012","unstructured":"Kwok KW, Sun LW, Mylonas GP, James DR, Orihuela-Espina F, Yang GZ (2012) Collaborative gaze channelling for improved cooperation during robotic assisted surgery. Ann Biomed Eng 40(10):2156\u20132167","journal-title":"Ann Biomed Eng"},{"issue":"5","key":"2346_CR3","doi-asserted-by":"publisher","first-page":"148","DOI":"10.4253\/wjge.v6.i5.148","volume":"6","author":"EM Bogen","year":"2014","unstructured":"Bogen EM, Augestad KM, Patel HR, Lindsetmo RO (2014) Telementoring in education of laparoscopic surgeons: an emerging technology. World J Gastrointestinal Endosc 6(5):148","journal-title":"World J Gastrointestinal Endosc"},{"issue":"1","key":"2346_CR4","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1016\/j.compmedimag.2009.07.007","volume":"34","author":"SL Lee","year":"2010","unstructured":"Lee SL, Lerotic M, Vitiello V, Giannarou S, Kwok KW, Visentini-Scarzanella M, Yang GZ (2010) From medical images to minimally invasive intervention: computer assistance for robotic surgery. Comput Med Imaging Graph 34(1):33\u201345","journal-title":"Comput Med Imaging Graph"},{"key":"2346_CR5","doi-asserted-by":"publisher","first-page":"66","DOI":"10.1016\/j.media.2017.01.007","volume":"37","author":"S Bernhardt","year":"2017","unstructured":"Bernhardt S, Nicolau SA, Soler L, Doignon C (2017) The status of augmented reality in laparoscopic surgery as of 2016. Med Image Anal 37:66\u201390","journal-title":"Med Image Anal"},{"key":"2346_CR6","doi-asserted-by":"crossref","unstructured":"V\u00e1vra P, Roman J, Zon\u010da P, Ihn\u00e1t P, N\u011bmec M, Kumar J, Habib N, El-Gendi A (2017) Recent development of augmented reality in surgery: a review. J Healthcare Eng 1\u20139","DOI":"10.1155\/2017\/4574172"},{"key":"2346_CR7","doi-asserted-by":"crossref","unstructured":"Stoyanov D, Scarzanella MV, Pratt P, Yang GZ (2010) Real-time stereo reconstruction in robotically assisted minimally invasive surgery. In: International conference on medical image computing and computer-assisted intervention. Springer","DOI":"10.1007\/978-3-642-15705-9_34"},{"issue":"4","key":"2346_CR8","doi-asserted-by":"publisher","first-page":"963","DOI":"10.1109\/TMI.2011.2176500","volume":"31","author":"DJ Mirota","year":"2011","unstructured":"Mirota DJ, Wang H, Taylor RH, Ishii M, Gallia GL, Hager GD (2011) A system for video-based navigation for endoscopic endonasal skull base surgery. IEEE Trans Med Imaging 31(4):963\u2013976","journal-title":"IEEE Trans Med Imaging"},{"issue":"5","key":"2346_CR9","doi-asserted-by":"publisher","first-page":"1147","DOI":"10.1109\/TRO.2015.2463671","volume":"31","author":"R Mur-Artal","year":"2015","unstructured":"Mur-Artal R, Montiel JMM, Tardos JD (2015) ORB-SLAM: a versatile and accurate monocular SLAM system. IEEE Trans Rob 31(5):1147\u20131163","journal-title":"IEEE Trans Rob"},{"key":"2346_CR10","doi-asserted-by":"crossref","unstructured":"Mahmoud N, Cirauqui I, Hostettler A, Doignon C, Soler L, Marescaux J, Montiel J (2016) ORBSLAM-based endoscope tracking and 3D reconstruction. International workshop on computer-assisted and robotic endoscopy. Springer","DOI":"10.1007\/978-3-319-54057-3_7"},{"key":"2346_CR11","doi-asserted-by":"crossref","unstructured":"Ma R, Wang R, Pizer S, Rosenman J, McGill SK, Frahm JM (2019) Real-time 3D reconstruction of colonoscopic surfaces for determining missing regions. In: International conference on medical image computing and computer-assisted intervention. Springer","DOI":"10.1007\/978-3-030-32254-0_64"},{"key":"2346_CR12","doi-asserted-by":"crossref","unstructured":"Fu H, Gong M, Wang C, Batmanghelich K, Tao D (2018) Deep ordinal regression network for monocular depth estimation. In: Proceedings of the IEEE conference on computer vision and pattern recognition","DOI":"10.1109\/CVPR.2018.00214"},{"key":"2346_CR13","unstructured":"Alhashim I, Wonka P (2018) High quality monocular depth estimation via transfer learning. arXiv preprint arXiv:1812.11941"},{"issue":"5","key":"2346_CR14","doi-asserted-by":"publisher","first-page":"1438","DOI":"10.1109\/TMI.2019.2950936","volume":"39","author":"X Liu","year":"2019","unstructured":"Liu X, Sinha A, Ishii M, Hager GD, Reiter A, Taylor RH, Unberath M (2019) Dense depth estimation in monocular endoscopy with self-supervised learning methods. IEEE Trans Med Imaging 39(5):1438\u20131447","journal-title":"IEEE Trans Med Imaging"},{"key":"2346_CR15","doi-asserted-by":"crossref","unstructured":"Reiter A, L\u00e9onard S, Sinha A, Ishii M, Taylor RH, Hager GD (2016) Endoscopic-CT: learning-based photometric reconstruction for endoscopic sinus surgery. In: Medical imaging 2016: image processing. International Society for Optics and Photonics","DOI":"10.1117\/12.2216296"},{"key":"2346_CR16","doi-asserted-by":"crossref","unstructured":"Isola P, Zhu JY, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition","DOI":"10.1109\/CVPR.2017.632"},{"issue":"7","key":"2346_CR17","doi-asserted-by":"publisher","first-page":"1167","DOI":"10.1007\/s11548-019-01962-w","volume":"14","author":"A Rau","year":"2019","unstructured":"Rau A, Edwards PE, Ahmad OF, Riordan P, Janatka M, Lovat LB, Stoyanov D (2019) Implicit domain adaptation with conditional generative adversarial networks for depth prediction in endoscopy. Int J Comput Assist Radiol Surg 14(7):1167\u20131176","journal-title":"Int J Comput Assist Radiol Surg"},{"key":"2346_CR18","doi-asserted-by":"crossref","unstructured":"Atapour-Abarghouei A, Breckon TP (2018) Real-time monocular depth estimation using synthetic data with domain adaptation via image style transfer. In: Proceedings of the IEEE conference on computer vision and pattern recognition","DOI":"10.1109\/CVPR.2018.00296"},{"key":"2346_CR19","doi-asserted-by":"crossref","unstructured":"Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE international conference on computer vision","DOI":"10.1109\/ICCV.2017.244"},{"issue":"1","key":"2346_CR20","doi-asserted-by":"publisher","first-page":"258","DOI":"10.1109\/LRA.2019.2955941","volume":"5","author":"C Zhao","year":"2019","unstructured":"Zhao C, Shen M, Sun L, Yang GZ (2019) Generative localization with uncertainty estimation through video-CT data for bronchoscopic biopsy. IEEE Robot Autom Lett 5(1):258\u2013265","journal-title":"IEEE Robot Autom Lett"},{"issue":"12","key":"2346_CR21","doi-asserted-by":"publisher","first-page":"2572","DOI":"10.1109\/TMI.2018.2842767","volume":"37","author":"F Mahmood","year":"2018","unstructured":"Mahmood F, Chen R, Durr NJ (2018) Unsupervised reverse domain adaptation for synthetic medical images via adversarial training. IEEE Trans Med Imaging 37(12):2572\u20132581","journal-title":"IEEE Trans Med Imaging"},{"key":"2346_CR22","unstructured":"Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Advances in neural information processing systems"},{"issue":"3","key":"2346_CR23","doi-asserted-by":"publisher","first-page":"345","DOI":"10.1109\/70.34770","volume":"5","author":"RY Tsai","year":"1989","unstructured":"Tsai RY, Lenz RK (1989) A new technique for fully autonomous and efficient 3 D robotics hand\/eye calibration. IEEE Trans Robot Autom 5(3):345\u2013358","journal-title":"IEEE Trans Robot Autom"},{"key":"2346_CR24","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition","DOI":"10.1109\/CVPR.2016.90"},{"key":"2346_CR25","unstructured":"Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980"},{"key":"2346_CR26","doi-asserted-by":"crossref","unstructured":"Wang L, Shen X, Zhang J, Wang O, Lin Z, Hsieh CY, Kong S, Lu H (2018) DeepLens: shallow depth of field from a single image. arXiv preprint arXiv:1810.08100","DOI":"10.1145\/3272127.3275013"},{"key":"2346_CR27","doi-asserted-by":"publisher","first-page":"698","DOI":"10.1109\/TPAMI.1987.4767965","volume":"5","author":"KS Arun","year":"1987","unstructured":"Arun KS, Huang TS, Blostein SD (1987) Least-squares fitting of two 3-D point sets. IEEE Trans Pattern Anal Mach Intell 5:698\u2013700","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"2346_CR28","doi-asserted-by":"crossref","unstructured":"Luo X, Huang JB, Szeliski R, Matzen K, Kopf J (2020) Consistent video depth estimation. arXiv preprint arXiv:2004.15021","DOI":"10.1145\/3386569.3392377"},{"issue":"4","key":"2346_CR29","doi-asserted-by":"publisher","first-page":"600","DOI":"10.1109\/TIP.2003.819861","volume":"13","author":"Z Wang","year":"2004","unstructured":"Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600\u2013612","journal-title":"IEEE Trans Image Process"},{"key":"2346_CR30","doi-asserted-by":"crossref","unstructured":"Nadeem S, Kaufman A (2016) Computer-aided detection of polyps in optical colonoscopy images. In: Medical imaging 2016: computer-aided diagnosis. International Society for Optics and Photonics","DOI":"10.1117\/12.2216996"}],"container-title":["International Journal of Computer Assisted Radiology and Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-021-02346-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11548-021-02346-9\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-021-02346-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,22]],"date-time":"2022-12-22T22:44:52Z","timestamp":1671749092000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11548-021-02346-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,30]]},"references-count":30,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2021,5]]}},"alternative-id":["2346"],"URL":"https:\/\/doi.org\/10.1007\/s11548-021-02346-9","relation":{},"ISSN":["1861-6410","1861-6429"],"issn-type":[{"value":"1861-6410","type":"print"},{"value":"1861-6429","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,3,30]]},"assertion":[{"value":"19 January 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 March 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"30 March 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}