{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,8]],"date-time":"2026-09-08T13:13:13Z","timestamp":1788873193890,"version":"build-2803163510"},"reference-count":34,"publisher":"Springer Science and Business Media LLC","issue":"7","license":[{"start":{"date-parts":[[2026,5,26]],"date-time":"2026-05-26T00:00:00Z","timestamp":1779753600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,5,26]],"date-time":"2026-05-26T00:00:00Z","timestamp":1779753600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J CARS"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Purpose<\/jats:title>\n                    <jats:p>Natural orifice surgeries minimize the need for incisions and reduce recovery time compared to open surgery; however, they require higher expertise due to visualization and orientation challenges. To enable reliable scene understanding for surgeon guidance and automation, we propose a perception pipeline that generates semantically informed 3D reconstructions.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We bring learning-based segmentation, depth estimation, and 3D reconstruction modules together. Through segmentation, we delineate tumor and prostate lobe borders, and through depth estimation and real-time SLAM-based reconstruction, we generate dense 3D point clouds from monocular videos. By propagating 2D labels into 3D space, we create real-time segmented maps of the surgical scenes. Additionally, we use registration with robot poses to solve the scale ambiguity of mapping from monocular images and allow the use of semantically informed real-time reconstructions in robotic surgeries.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>We achieve sub-millimeter reconstruction accuracy based on average one-sided Chamfer distances, average pose registration RMSE of 0.9 mm, and an estimated scale within 2% of ground truth. Compared to offline Structure-from-Motion baseline, the proposed SLAM-based approach improves processing time while maintaining or improving reconstruction accuracy. Qualitative evaluations show robustness in challenging scenarios, including submerged prostate experiments and cadaver airway explorations.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>We present a modular perception pipeline, integrating semantic segmentation with real-time monocular SLAM for natural orifice surgeries. This pipeline offers a promising solution for scene understanding that can facilitate automation or surgeon guidance. Due to its plug-and-play design and demonstrated generalizability across anatomies and experimental conditions, this framework provides a scalable foundation for future clinical translation.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1007\/s11548-026-03717-w","type":"journal-article","created":{"date-parts":[[2026,5,26]],"date-time":"2026-05-26T17:13:26Z","timestamp":1779815606000},"page":"1465-1474","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Perseus: perception with semantic endoscopic understanding and SLAM"],"prefix":"10.1007","volume":"21","author":[{"given":"Ayberk","family":"Acar","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fangjie","family":"Li","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Susheela Sharma","family":"Stern","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lidia","family":"Al-Zogbi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hao","family":"Li","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kanyifeechukwu Jane","family":"Oguine","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dilara","family":"Isik","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Brendan","family":"Burkhart","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jesse F.","family":"d\u2019Almeida","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"suffix":"III","given":"Robert J.","family":"Webster","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ipek","family":"Oguz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jie Ying","family":"Wu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,5,26]]},"reference":[{"issue":"8","key":"3717_CR1","first-page":"1179","volume":"20","author":"A Malik","year":"2006","unstructured":"Malik A, Mellinger J, Hazey J, Dunkin B, MacFadyen B Jr (2006) Endoluminal and transluminal surgery: current status and future possibilities. Surg Endosc Other Interv Tech 20(8):1179\u20131192","journal-title":"Surg Endosc Other Interv Tech"},{"issue":"2","key":"3717_CR2","doi-asserted-by":"publisher","first-page":"315","DOI":"10.1016\/j.giec.2008.01.015","volume":"18","author":"L Swanstrom","year":"2008","unstructured":"Swanstrom L, Zheng B (2008) Spatial orientation and off-axis challenges for notes. Gastrointest Endosc Clin N Am 18(2):315\u2013324","journal-title":"Gastrointest Endosc Clin N Am"},{"key":"3717_CR3","doi-asserted-by":"crossref","unstructured":"Acar A, Smith M, Al-Zogbi L, Watts T, Li F, Li H, Yilmaz N, Scheikl PM, d\u2019Almeida JF, Sharma S, et al (2025) From monocular vision to autonomous action: guiding tumor resection via 3d reconstruction. In: 2025 IEEE\/RSJ International conference on intelligent robots and systems (IROS). IEEE, pp 21714\u201321720","DOI":"10.1109\/IROS60139.2025.11246413"},{"key":"3717_CR4","doi-asserted-by":"publisher","first-page":"302","DOI":"10.1053\/j.semtcvs.2011.01.007","volume":"22","author":"KI Makris","year":"2010","unstructured":"Makris KI, Rieder E, Swanstrom LL (2010) Natural orifice trans-luminal endoscopic surgery (notes) in thoracic surgery. Semin Thorac Cardiovasc Surg 22:302\u2013309","journal-title":"Semin Thorac Cardiovasc Surg"},{"issue":"6","key":"3717_CR5","doi-asserted-by":"publisher","first-page":"324","DOI":"10.1038\/nrurol.2014.96","volume":"11","author":"MD Tyson","year":"2014","unstructured":"Tyson MD, Humphreys MR (2014) Urological applications of natural orifice transluminal endoscopic surgery. Nat Rev Urol 11(6):324\u2013332","journal-title":"Nat Rev Urol"},{"issue":"12","key":"3717_CR6","doi-asserted-by":"publisher","first-page":"1278","DOI":"10.1164\/rccm.200210-1181SO","volume":"169","author":"A Ernst","year":"2004","unstructured":"Ernst A, Feller-Kopman D, Becker HD, Mehta AC (2004) Central airway obstruction. Am J Respir Crit Care Med 169(12):1278\u20131297","journal-title":"Am J Respir Crit Care Med"},{"issue":"3","key":"3717_CR7","doi-asserted-by":"publisher","first-page":"189","DOI":"10.4103\/2229-5151.164995","volume":"5","author":"DL Stahl","year":"2015","unstructured":"Stahl DL, Richard KM, Papadimos TJ (2015) Complications of bronchoscopy: A concise synopsis. Int J Crit Illn Inj Sci 5(3):189\u2013195","journal-title":"Int J Crit Illn Inj Sci"},{"issue":"1","key":"3717_CR8","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1007\/s10439-019-02325-x","volume":"48","author":"JB Gafford","year":"2020","unstructured":"Gafford JB, Webster S, Dillon N, Blum E, Hendrick R, Maldonado F, Gillaspie EA, Rickman OB, Herrell SD, Webster RJ III (2020) A concentric tube robot system for rigid bronchoscopy: a feasibility study on central airway obstruction removal. Ann Biomed Eng 48(1):181\u2013191","journal-title":"Ann Biomed Eng"},{"issue":"Suppl 9","key":"3717_CR9","first-page":"3","volume":"7","author":"CG Roehrborn","year":"2005","unstructured":"Roehrborn CG (2005) Benign prostatic hyperplasia: an overview. Reviews in urology 7(Suppl 9):3","journal-title":"Reviews in urology"},{"issue":"03n04","key":"3717_CR10","doi-asserted-by":"publisher","first-page":"2550008","DOI":"10.1142\/S2424905X25500084","volume":"10","author":"ME Smith","year":"2025","unstructured":"Smith ME, Yilmaz N, Watts T, Scheikl PM, Ge J, Deguet A, Kuntz A, Krieger A (2025) Autonomous vision-guided resection of central airway obstruction. J Med Robot Res 10(03n04):2550008","journal-title":"J Med Robot Res"},{"issue":"11","key":"3717_CR11","doi-asserted-by":"publisher","first-page":"643","DOI":"10.1038\/s41585-020-0375-z","volume":"17","author":"MJ Connor","year":"2020","unstructured":"Connor MJ, Dasgupta P, Ahmed HU, Raza A (2020) Autonomous surgery in the era of robotic urology: friend or foe of the future surgeon? Nat Rev Urol 17(11):643\u2013649","journal-title":"Nat Rev Urol"},{"issue":"7","key":"3717_CR12","doi-asserted-by":"publisher","first-page":"1259","DOI":"10.1007\/s11548-024-03171-6","volume":"19","author":"T Teufel","year":"2024","unstructured":"Teufel T, Shu H, Soberanis-Mukul RD, Mangulabnan JE, Sahu M, Vedula SS, Ishii M, Hager G, Taylor RH, Unberath M (2024) Oneslam to map them all: a generalized approach to slam for monocular endoscopic imaging based on tracking any point. Int J Comput Assist Radiol Surg 19(7):1259\u20131266","journal-title":"Int J Comput Assist Radiol Surg"},{"key":"3717_CR13","doi-asserted-by":"crossref","unstructured":"Liu X, Li Z, Ishii M, Hager GD, Taylor RH, Unberath M (2022) Sage: slam with appearance and geometry prior for endoscopy. In: 2022 international conference on robotics and automation (ICRA). IEEE, pp 5587\u20135593","DOI":"10.1109\/ICRA46639.2022.9812257"},{"key":"3717_CR14","doi-asserted-by":"crossref","unstructured":"Oliva-Maza L, Steidle F, Klodmann J, Strobl K, Miernik A, Triebel R (2025) Kidneydepth: A synthetic kidney dataset for metric depth estimation in ureteroscopy. In: International Conference on medical image computing and computer-assisted intervention. Springer, pp 331\u2013340","DOI":"10.1007\/978-3-032-05114-1_32"},{"key":"3717_CR15","doi-asserted-by":"publisher","first-page":"23772","DOI":"10.1109\/ACCESS.2021.3050617","volume":"9","author":"Y Liu","year":"2021","unstructured":"Liu Y, Miura J (2021) Rds-slam: Real-time dynamic slam using semantic segmentation methods. Ieee Access 9:23772\u201323785","journal-title":"Ieee Access"},{"key":"3717_CR16","doi-asserted-by":"crossref","unstructured":"Tateno K, Tombari F, Navab N (2015) Real-time and scalable incremental segmentation on dense slam. In: IEEE\/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 4465\u20134472","DOI":"10.1109\/IROS.2015.7354011"},{"issue":"3","key":"3717_CR17","doi-asserted-by":"publisher","first-page":"614","DOI":"10.3390\/sym14030614","volume":"14","author":"H Wu","year":"2022","unstructured":"Wu H, Zhao J, Xu K, Zhang Y, Xu R, Wang A, Iwahori Y (2022) Semantic slam based on deep learning in endocavity environment. Symmetry 14(3):614","journal-title":"Symmetry"},{"key":"3717_CR18","doi-asserted-by":"crossref","unstructured":"Lin S, Miao AJ, Lu J, Yu S, Chiu Z-Y, Richter F, Yip MC (2023) Semantic-super: a semantic-aware surgical perception framework for endoscopic tissue identification, reconstruction, and tracking. In: 2023 IEEE international conference on robotics and automation (ICRA). IEEE, pp 4739\u20134746","DOI":"10.1109\/ICRA48891.2023.10160746"},{"key":"3717_CR19","doi-asserted-by":"crossref","unstructured":"Li H, Wang J, Kumar N, d\u2019Almeida J, Lu D, Acar A, Han J, Yang Q, Ertop TE, Wu JY, et al (2025) Automated segmentation of central airway obstruction from endoscopic video stream with deep learning. In: Medical imaging 2025: image-guided procedures, robotic interventions, and modeling, vol. 13408. SPIE, pp 113\u2013119","DOI":"10.1117\/12.3047413"},{"key":"3717_CR20","doi-asserted-by":"publisher","DOI":"10.1016\/j.urolvj.2022.100184","volume":"16","author":"P Saba","year":"2022","unstructured":"Saba P, Shepard L, Gopal N, Setia S, Jain R, Quarrier S, Miller N, Krambeck A, Humphreys M, Ghazi A (2022) Design and validation of a non-biohazardous simulation model for holmium laser enucleation of the prostate (holep). Urology Video Journal 16:100184","journal-title":"Urology Video Journal"},{"key":"3717_CR21","doi-asserted-by":"crossref","unstructured":"Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 234\u2013241","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"3717_CR22","unstructured":"Ravi N, Gabeur V, Hu Y-T, Hu R, Ryali C, Ma T, Khedr H, R\u00e4dle R, Rolland C, Gustafson L, et al (2025) Sam 2: Segment anything in images and videos. In: International conference on learning representations, vol 2025. pp 28085\u201328128"},{"key":"3717_CR23","doi-asserted-by":"crossref","unstructured":"Li H, Liu H, Hu D, Wang J, Oguz I (2024) Promise: Prompt-driven 3d medical image segmentation using pretrained image foundation models. In: 2024 IEEE international symposium on biomedical imaging (ISBI). IEEE, pp 1\u20135","DOI":"10.1109\/ISBI56570.2024.10635207"},{"key":"3717_CR24","doi-asserted-by":"crossref","unstructured":"Oguine K, Li H, Isik D, Acar A, d\u2019Almeida J, Sharma S, Branscombe L, Petrin J, Wu JY, Webster\u00a0III RJ, et al (2026) Efficient labeling: is prompt-based annotation a viable alternative to manual labeling in endoscopy video segmentation? In: Medical imaging 2026: image processing, vol 13925. SPIE, pp 192\u2013198","DOI":"10.1117\/12.3086587"},{"key":"3717_CR25","doi-asserted-by":"crossref","unstructured":"Li H, Lu D, d\u2019Almeida J, Isik D, Aghdam EK, DiSanto N, Acar A, Sharma S, Wu JY, Webster\u00a0III RJ, et al (2026) Monocular absolute depth estimation from endoscopy via domain-invariant feature learning and latent consistency. In: Medical imaging 2026: image processing, vol 13925. SPIE, pp 85\u201391","DOI":"10.1117\/12.3086626"},{"key":"3717_CR26","doi-asserted-by":"crossref","unstructured":"Lu D, Li H, Pierre C, Kavoussi N, Oguz I (2025) Kidney endoscopy video to preoperative ct alignment for depth estimation. Medical Imaging 2025: Image-guided procedures, robotic interventions, and modeling, vol 13408. SPIE, pp 106\u2013112","DOI":"10.1117\/12.3047484"},{"issue":"5","key":"3717_CR27","doi-asserted-by":"publisher","first-page":"230521","DOI":"10.1148\/ryai.230521","volume":"6","author":"H Li","year":"2024","unstructured":"Li H, Liu H, Busch H, Grimm R, Huisman H, Tong A, Winkel D, Penzkofer T, Shabunin I, Choi MH et al (2024) Deep learning-based unsupervised domain adaptation via a unified model for prostate lesion detection using multisite biparametric mri datasets. Radiol Artif Intell 6(5):230521","journal-title":"Radiol Artif Intell"},{"key":"3717_CR28","doi-asserted-by":"publisher","first-page":"21875","DOI":"10.52202\/079017-0688","volume":"37","author":"L Yang","year":"2024","unstructured":"Yang L, Kang B, Huang Z, Zhao Z, Xu X, Feng J, Zhao H (2024) Depth anything v2. Adv Neural Inf Process Syst 37:21875\u201321911","journal-title":"Adv Neural Inf Process Syst"},{"key":"3717_CR29","first-page":"16558","volume":"34","author":"Z Teed","year":"2021","unstructured":"Teed Z, Deng J (2021) Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras. Adv Neural Inf Process Syst 34:16558\u201316569","journal-title":"Adv Neural Inf Process Syst"},{"issue":"5","key":"3717_CR30","doi-asserted-by":"publisher","first-page":"717","DOI":"10.1109\/70.326576","volume":"10","author":"FC Park","year":"1994","unstructured":"Park FC, Martin BJ (1994) Robot sensor calibration: solving ax= xb on the euclidean group. IEEE Trans Robot Autom 10(5):717\u2013721","journal-title":"IEEE Trans Robot Autom"},{"issue":"04","key":"3717_CR31","doi-asserted-by":"publisher","first-page":"376","DOI":"10.1109\/34.88573","volume":"13","author":"S Umeyama","year":"1991","unstructured":"Umeyama S (1991) Least-squares estimation of transformation parameters between two point patterns. IEEE Transactions on Pattern Analysis & Machine Intelligence 13(04):376\u2013380","journal-title":"IEEE Transactions on Pattern Analysis & Machine Intelligence"},{"key":"3717_CR32","unstructured":"Tyszkiewicz M, Fua P, Trulls E (2020) Disk: Learning local features with policy gradient. Adv Neural Inf Process Syst 33:14254\u201314265"},{"key":"3717_CR33","doi-asserted-by":"crossref","unstructured":"Lindenberger P, Sarlin P-E, Pollefeys M (2023) Lightglue: local feature matching at light speed. In: Proceedings of the IEEE\/CVF international conference on computer vision. pp 17627\u201317638","DOI":"10.1109\/ICCV51070.2023.01616"},{"key":"3717_CR34","doi-asserted-by":"crossref","unstructured":"Sarlin P-E, Cadena C, Siegwart R, Dymczyk M (2019) From coarse to fine: Robust hierarchical localization at large scale. CVPR","DOI":"10.1109\/CVPR.2019.01300"}],"container-title":["International Journal of Computer Assisted Radiology and Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-026-03717-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11548-026-03717-w","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-026-03717-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,8]],"date-time":"2026-09-08T13:03:33Z","timestamp":1788872613000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11548-026-03717-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,26]]},"references-count":34,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2026,7]]}},"alternative-id":["3717"],"URL":"https:\/\/doi.org\/10.1007\/s11548-026-03717-w","relation":{},"ISSN":["1861-6429"],"issn-type":[{"value":"1861-6429","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,26]]},"assertion":[{"value":"23 February 2026","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 May 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 May 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"No relevant financial or non-financial interests to disclose.","order":1,"name":"Ethics","label":"Conflict of interest","group":{"name":"EthicsHeading","label":"Declarations"}}]}}