{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T17:28:24Z","timestamp":1777742904855,"version":"3.51.4"},"reference-count":23,"publisher":"Springer Science and Business Media LLC","issue":"6","license":[{"start":{"date-parts":[[2025,5,8]],"date-time":"2025-05-08T00:00:00Z","timestamp":1746662400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,5,8]],"date-time":"2025-05-08T00:00:00Z","timestamp":1746662400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"NCT Proof of Concept (PoC)-project NAIV"},{"DOI":"10.13039\/501100002347","name":"Bundesministerium f\u00fcr Bildung und Forschung","doi-asserted-by":"publisher","award":["02K23A112"],"award-info":[{"award-number":["02K23A112"]}],"id":[{"id":"10.13039\/501100002347","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J CARS"],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:sec>\n            <jats:title>Purpose:<\/jats:title>\n            <jats:p>Markerless navigation in minimally invasive surgery is still an unsolved challenge. Many proposed navigation systems for minimally invasive surgeries rely on stereoscopic images, while in clinical practice oftentimes monocular endoscopes are used. Combined with the lack of automatic video-based navigation systems for prostatectomies, this paper explores methods to tackle both research gaps at the same time for robot-assisted prostatectomies.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Methods:<\/jats:title>\n            <jats:p>In order to realize a semi-automatic augmented reality overlay for navigated prostatectomy, the camera pose w.r.t. the prostate needs to be estimated. We developed a method where visual cues are drawn on top of the organ after an initial manual alignment, simultaneously creating matching landmarks on the 2D and 3D data. Starting from this key frame, the cues are then tracked in the endoscopic video. Both PnPRansac and differentiable rendering are then explored to perform 2D\u20133D registration for each frame.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Results:<\/jats:title>\n            <jats:p>We performed experiments on synthetic and in vivo data. On synthetic data differentiable rendering can achieve a median target registration error of 6.11 mm. Both PnPRansac and differentiable rendering are feasible methods for 2D\u20133D registration.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Conclusion:<\/jats:title>\n            <jats:p>We demonstrated a video-based markerless augmented reality overlay for navigated prostatectomy, using visual cues as an anchor.<\/jats:p>\n          <\/jats:sec>","DOI":"10.1007\/s11548-025-03374-5","type":"journal-article","created":{"date-parts":[[2025,5,8]],"date-time":"2025-05-08T19:55:04Z","timestamp":1746734104000},"page":"1265-1272","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["An augmented reality overlay for navigated prostatectomy using fiducial-free 2D\u20133D registration"],"prefix":"10.1007","volume":"20","author":[{"given":"Johannes","family":"Bender","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jeremy","family":"Kwe","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Benedikt","family":"Hoeh","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Katharina","family":"Boehm","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ivan","family":"Platzek","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Angelika","family":"Borkowetz","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Stefanie","family":"Speidel","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Micha","family":"Pfeiffer","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2025,5,8]]},"reference":[{"issue":"12","key":"3374_CR1","doi-asserted-by":"publisher","first-page":"1841","DOI":"10.1089\/end.2010.0724","volume":"25","author":"T Simpfend\u00f6rfer","year":"2011","unstructured":"Simpfend\u00f6rfer T, Baumhauer M, M\u00fcller M et al (2011) Augmented reality visualization during laparoscopic radical prostatectomy. J Endourol 25(12):1841\u20131845. https:\/\/doi.org\/10.1089\/end.2010.0724","journal-title":"J Endourol"},{"issue":"8","key":"3374_CR2","doi-asserted-by":"publisher","first-page":"1877","DOI":"10.1109\/TMI.2018.2810778","volume":"37","author":"G Samei","year":"2018","unstructured":"Samei G, Goksel O, Lobo J et al (2018) Real-time FEM-based registration of 3-D to 2.5-D transrectal ultrasound images. IEEE Trans Med Imaging 37(8):1877\u20131886. https:\/\/doi.org\/10.1109\/TMI.2018.2810778","journal-title":"IEEE Trans Med Imaging"},{"issue":"7","key":"3374_CR3","doi-asserted-by":"publisher","first-page":"1181","DOI":"10.1007\/s11548-021-02419-9","volume":"16","author":"M Kalia","year":"2021","unstructured":"Kalia M, Avinash A, Navab N et al (2021) Preclinical evaluation of a markerless, real-time, augmented reality guidance system for robot-assisted radical prostatectomy. Int J Comput Assist Radiol Surg 16(7):1181\u20131188. https:\/\/doi.org\/10.1007\/s11548-021-02419-9","journal-title":"Int J Comput Assist Radiol Surg"},{"issue":"7","key":"3374_CR4","doi-asserted-by":"publisher","first-page":"1081","DOI":"10.1111\/j.1464-410X.2012.11692.x","volume":"111","author":"S Thompson","year":"2013","unstructured":"Thompson S, Penney G, Billia M et al (2013) Design and evaluation of an image-guidance system for robot-assisted radical prostatectomy. BJU Int 111(7):1081\u20131090. https:\/\/doi.org\/10.1111\/j.1464-410X.2012.11692.x","journal-title":"BJU Int"},{"issue":"1","key":"3374_CR5","doi-asserted-by":"publisher","first-page":"92","DOI":"10.23736\/S0393-2249.18.03250-2","volume":"71","author":"Z Kratiras","year":"2019","unstructured":"Kratiras Z, Gavazzi A, Belba A et al (2019) Phase I study of a new tablet-based image guided surgical system in robot-assisted radical prostatectomy. Minerva Urol Nefrol 71(1):92. https:\/\/doi.org\/10.23736\/S0393-2249.18.03250-2","journal-title":"Minerva Urol Nefrol"},{"issue":"8","key":"3374_CR6","doi-asserted-by":"publisher","DOI":"10.1117\/1.JBO.21.8.086008","volume":"21","author":"MNV Oosterom","year":"2016","unstructured":"Oosterom MNV, Engelen MA, Berg NS et al (2016) Navigation of a robot-integrated fluorescence laparoscope in preoperative SPECT\/CT and intraoperative freehand SPECT imaging data: a phantom study. J Biomed Opt 21(8):086008. https:\/\/doi.org\/10.1117\/1.JBO.21.8.086008","journal-title":"J Biomed Opt"},{"issue":"5","key":"3374_CR7","doi-asserted-by":"publisher","first-page":"834","DOI":"10.1111\/bju.14549","volume":"123","author":"F Porpiglia","year":"2019","unstructured":"Porpiglia F, Checcucci E, Amparore D et al (2019) Augmented-reality robot-assisted radical prostatectomy using hyper-accuracy three-dimensional reconstruction (HA3D\u2122) technology: a radiological and pathological study. BJU Int 123(5):834\u2013845. https:\/\/doi.org\/10.1111\/bju.14549","journal-title":"BJU Int"},{"issue":"4","key":"3374_CR8","doi-asserted-by":"publisher","first-page":"505","DOI":"10.1016\/j.eururo.2019.03.037","volume":"76","author":"F Porpiglia","year":"2019","unstructured":"Porpiglia F, Checcucci E, Amparore D et al (2019) Three-dimensional elastic augmented-reality robot-assisted radical prostatectomy using hyperaccuracy three-dimensional reconstruction technology: a step further in the identification of capsular involvement. Eur Urol 76(4):505\u2013514. https:\/\/doi.org\/10.1016\/j.eururo.2019.03.037","journal-title":"Eur Urol"},{"issue":"6","key":"3374_CR9","doi-asserted-by":"publisher","first-page":"625","DOI":"10.1089\/end.2013.0749","volume":"28","author":"O Ukimura","year":"2014","unstructured":"Ukimura O, Aron M, Nakamoto M et al (2014) Three-dimensional surgical navigation model with TilePro display during robot-assisted radical prostatectomy. J Endourol 28(6):625\u2013630. https:\/\/doi.org\/10.1089\/end.2013.0749","journal-title":"J Endourol"},{"issue":"9","key":"3374_CR10","doi-asserted-by":"publisher","first-page":"1435","DOI":"10.1007\/s11548-021-02432-y","volume":"16","author":"L Tanzi","year":"2021","unstructured":"Tanzi L, Piazzolla P, Porpiglia F et al (2021) Real-time deep learning semantic segmentation during intra-operative surgery for 3D augmented reality assistance. Int J Comput Assist Radiol Surg 16(9):1435\u20131445. https:\/\/doi.org\/10.1007\/s11548-021-02432-y","journal-title":"Int J Comput Assist Radiol Surg"},{"issue":"3","key":"3374_CR11","doi-asserted-by":"publisher","DOI":"10.1002\/rcs.2387","volume":"18","author":"E Padovan","year":"2022","unstructured":"Padovan E, Marullo G, Tanzi L et al (2022) A deep learning framework for real-time 3D model registration in robot-assisted laparoscopic surgery. Int J Med Robot Comput Assist Surg MRCAS 18(3):e2387. https:\/\/doi.org\/10.1002\/rcs.2387","journal-title":"Int J Med Robot Comput Assist Surg MRCAS"},{"key":"3374_CR12","doi-asserted-by":"publisher","DOI":"10.1007\/s11548-021-02518-7","author":"B Koo","year":"2021","unstructured":"Koo B, Robu MR, Allam M et al (2021) Automatic, global registration in laparoscopic liver surgery. Int J Comput Assist Radiol Surg (IJCARS). https:\/\/doi.org\/10.1007\/s11548-021-02518-7","journal-title":"Int J Comput Assist Radiol Surg (IJCARS)"},{"key":"3374_CR13","doi-asserted-by":"crossref","unstructured":"Fehrentz M, Azampour MF, Dorent R, et\u00a0al (2024) Intraoperative registration by cross-modal inverse neural rendering. In: Proceedings of medical image computing and computer assisted intervention\u2013MICCAI 2024, vol LNCS 15006. Springer","DOI":"10.1007\/978-3-031-72089-5_30"},{"issue":"7","key":"3374_CR14","doi-asserted-by":"publisher","first-page":"1177","DOI":"10.1007\/s11548-020-02151-w","volume":"15","author":"T Fran\u00e7ois","year":"2020","unstructured":"Fran\u00e7ois T, Calvet L, Madad Zadeh S et al (2020) Detecting the occluding contours of the uterus to automatise augmented laparoscopy: score, loss, dataset, evaluation and user study. Int J Comput Assist Radiol Surg 15(7):1177\u20131186. https:\/\/doi.org\/10.1007\/s11548-020-02151-w","journal-title":"Int J Comput Assist Radiol Surg"},{"issue":"6","key":"3374_CR15","doi-asserted-by":"publisher","first-page":"1712","DOI":"10.1007\/s10439-020-02479-z","volume":"48","author":"Y Espinel","year":"2020","unstructured":"Espinel Y, \u00d6zg\u00fcr E, Calvet L et al (2020) Combining visual cues with interactions for 3D\u20132D registration in liver laparoscopy. Ann Biomed Eng 48(6):1712\u20131727. https:\/\/doi.org\/10.1007\/s10439-020-02479-z","journal-title":"Ann Biomed Eng"},{"key":"3374_CR16","first-page":"326","volume-title":"Medical image computing and computer assisted intervention\u2013MICCAI 2017","author":"B Koo","year":"2017","unstructured":"Koo B, \u00d6zg\u00fcr E, Le Roy B et al (2017) Deformable registration of a preoperative 3D liver volume to a laparoscopy image using contour and shading cues. In: Descoteaux M, Maier-Hein L, Franz A et al (eds) Medical image computing and computer assisted intervention\u2013MICCAI 2017. Springer, Cham, pp 326\u2013334"},{"key":"3374_CR17","doi-asserted-by":"publisher","unstructured":"Karaev N, Rocco I, Graham B, et\u00a0al (2025) CoTracker: It is better to track together. In: Leonardis A, Ricci E, Roth S, et\u00a0al (eds) Computer vision\u2013ECCV 2024. Springer, Cham, pp 18\u201335. https:\/\/doi.org\/10.1007\/978-3-031-73033-7_2","DOI":"10.1007\/978-3-031-73033-7_2"},{"key":"3374_CR18","unstructured":"Ravi N, Reizenstein J, Novotny D, et\u00a0al (2020) Accelerating 3d deep learning with pytorch3d. arXiv:2007.08501"},{"key":"3374_CR19","unstructured":"Kingma DP, Ba J (2017) Adam: a method for stochastic optimization. arXiv:1412.6980"},{"key":"3374_CR20","doi-asserted-by":"publisher","unstructured":"Zhou Y, Barnes C, Lu J, et\u00a0al (2019) On the continuity of rotation representations in neural networks. In: 2019 IEEE\/CVF conference on computer vision and pattern recognition (CVPR), pp 5738\u20135746, ISSN 2575\u20137075. https:\/\/doi.org\/10.1109\/CVPR.2019.00589, https:\/\/ieeexplore.ieee.org\/document\/8953486\/?arnumber=8953486","DOI":"10.1109\/CVPR.2019.00589"},{"key":"3374_CR21","doi-asserted-by":"publisher","first-page":"679","DOI":"10.5194\/isprs-archives-XLIII-B2-2022-679-2022","volume":"43","author":"M Hardner","year":"2022","unstructured":"Hardner M, Docea R, Schneider D (2022) Guided calibration of medical stereo endoscopes. Int Arch Photogramm Remote Sens Spat Inf Sci 43:679\u2013686. https:\/\/doi.org\/10.5194\/isprs-archives-XLIII-B2-2022-679-2022","journal-title":"Int Arch Photogramm Remote Sens Spat Inf Sci"},{"issue":"9","key":"3374_CR22","doi-asserted-by":"publisher","first-page":"1323","DOI":"10.1016\/j.mri.2012.05.001","volume":"30","author":"A Fedorov","year":"2012","unstructured":"Fedorov A, Beichel R, Kalpathy-Cramer J et al (2012) 3D Slicer as an image computing platform for the quantitative imaging network. Magn Reson Imaging 30(9):1323\u20131341. https:\/\/doi.org\/10.1016\/j.mri.2012.05.001","journal-title":"Magn Reson Imaging"},{"key":"3374_CR23","doi-asserted-by":"publisher","unstructured":"Akiba T, Sano S, Yanase T, et\u00a0al (2019) Optuna: a next-generation hyperparameter optimization framework. In: Proceedings of the 25th ACM SIGKDD International conference on knowledge discovery & data mining. association for computing machinery, KDD \u201919, New York, NY, USA, pp 2623\u20132631. https:\/\/doi.org\/10.1145\/3292500.3330701","DOI":"10.1145\/3292500.3330701"}],"container-title":["International Journal of Computer Assisted Radiology and Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-025-03374-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11548-025-03374-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-025-03374-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,14]],"date-time":"2025-06-14T11:34:25Z","timestamp":1749900865000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11548-025-03374-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,8]]},"references-count":23,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2025,6]]}},"alternative-id":["3374"],"URL":"https:\/\/doi.org\/10.1007\/s11548-025-03374-5","relation":{},"ISSN":["1861-6429"],"issn-type":[{"value":"1861-6429","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,8]]},"assertion":[{"value":"17 February 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 March 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 May 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"The data used was authorized by the local ethics committee of TUD Dresden University of Technology (approval code: BO-EK-134032022).","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}}]}}