{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,4]],"date-time":"2026-08-04T20:06:08Z","timestamp":1785873968919,"version":"3.56.0"},"reference-count":30,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2024,12,20]],"date-time":"2024-12-20T00:00:00Z","timestamp":1734652800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,12,20]],"date-time":"2024-12-20T00:00:00Z","timestamp":1734652800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J CARS"],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:sec>\n            <jats:title>Purpose<\/jats:title>\n            <jats:p>In this feasibility study, we aimed to create a dedicated pulmonary augmented reality (AR) workflow to enable a semi-automated intraoperative overlay of the pulmonary anatomy during video-assisted thoracoscopic surgery (VATS) or robot-assisted thoracoscopic surgery (RATS).<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Methods<\/jats:title>\n            <jats:p>Initially, the stereoscopic cameras were calibrated to obtain the intrinsic camera parameters. Intraoperatively, stereoscopic images were recorded and a 3D point cloud was generated from these images. By manually selecting the bifurcation key points, the 3D segmentation (from the diagnostic CT scan) was registered onto the intraoperative 3D point cloud.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Results<\/jats:title>\n            <jats:p>Image reprojection errors were 0.34 and 0.22 pixels for the VATS and RATS cameras, respectively. We created disparity maps and point clouds for all eight patients. Time for creation of the 3D AR overlay was 5\u00a0min. Validation of the point clouds was performed, resulting in a median absolute error of 0.20\u00a0mm [IQR 0.10\u20130.54]. We were able to visualize the AR overlay and identify the arterial bifurcations adequately for five patients. In addition to creating AR overlays of the visible or invisible structures intraoperatively, we successfully visualized branch labels and altered the transparency of the overlays.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Conclusion<\/jats:title>\n            <jats:p>An algorithm was developed transforming the operative field into a 3D point cloud surface. This allowed for an accurate registration and visualization of preoperative 3D models. Using this system, surgeons can navigate through the patient's anatomy intraoperatively, especially during crucial moments, by visualizing otherwise invisible structures. This proposed registration method lays the groundwork for automated intraoperative AR navigation during minimally invasive pulmonary resections.<\/jats:p>\n          <\/jats:sec>","DOI":"10.1007\/s11548-024-03308-7","type":"journal-article","created":{"date-parts":[[2024,12,20]],"date-time":"2024-12-20T19:07:53Z","timestamp":1734721673000},"page":"787-795","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["A novel 3D image registration technique for augmented reality vision in minimally invasive thoracoscopic pulmonary segmentectomy"],"prefix":"10.1007","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8412-2851","authenticated-orcid":false,"given":"J. J.","family":"Peek","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8368-3542","authenticated-orcid":false,"given":"X.","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9196-3923","authenticated-orcid":false,"given":"K.","family":"Hildebrandt","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8179-3924","authenticated-orcid":false,"given":"S. A.","family":"Max","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6118-2341","authenticated-orcid":false,"given":"A. H.","family":"Sadeghi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6200-1998","authenticated-orcid":false,"given":"A. J. J. C.","family":"Bogers","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2647-5509","authenticated-orcid":false,"given":"E. A. F.","family":"Mahtab","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,12,20]]},"reference":[{"issue":"1","key":"3308_CR1","first-page":"17","volume":"73","author":"RL Siegel","year":"2023","unstructured":"Siegel RL, Miller KD, Wagle NS, Jemal A (2023) Cancer statistics, 2023. CA: A Cancer J Clin 73(1):17\u201348","journal-title":"CA: A Cancer J Clin"},{"issue":"10335","key":"3308_CR2","doi-asserted-by":"publisher","first-page":"1607","DOI":"10.1016\/S0140-6736(21)02333-3","volume":"399","author":"H Saji","year":"2022","unstructured":"Saji H, Okada M, Tsuboi M, Nakajima R, Suzuki K, Aokage K, Aoki T, Okami J, Yoshino I, Ito H, Okumura N, Yamaguchi M, Ikeda N, Wakabayashi M, Nakamura K, Fukuda H, Nakamura S, Mitsudomi T, Watanabe SI, Asamura H, West Japan Oncology Group and Japan Clinical Oncology Group (2022) Segmentectomy versus lobectomy in small-sized peripheral non-small-cell lung cancer (JCOG0802\/WJOG4607L): a multicentre, open-label, phase 3, randomised, controlled, non-inferiority trial. Lancet 399(10335):1607\u20131617","journal-title":"Lancet"},{"key":"3308_CR3","doi-asserted-by":"publisher","DOI":"10.1093\/ejcts\/ezad224","author":"A Brunelli","year":"2023","unstructured":"Brunelli A, Decaluwe H, Gonzalez M, Gossot D, Petersen RH, Augustin F, Assouad J, Baste JM, Batirel H, Falcoz PE, Almanzar SF, Furak J, Gomez-Hernandez MT, de Antonio DG, Hansen H, Jimenez M, Koryllos A, Meacci E, Opitz I, Pages PB, Piwkowski C, Ruffini E, Schneiter D, Stupnik T, Szanto Z, Thomas P, Toker A, Tosi D, Veronesi G (2023) European society of thoracic surgeons expert consensus recommendations on technical standards of segmentectomy for primary lung cancer. Eur J Cardio-Thorac Surg. https:\/\/doi.org\/10.1093\/ejcts\/ezad224","journal-title":"Eur J Cardio-Thorac Surg"},{"key":"3308_CR4","doi-asserted-by":"publisher","first-page":"371","DOI":"10.1109\/TMI.2020.3027442","volume":"401","author":"T Collins","year":"2021","unstructured":"Collins T, Pizarro D, Gasparini S, Bourdel N, Chauvet P, Canis M, Calvet L, Bartoli A (2021) Augmented reality guided laparoscopic surgery of the uterus. IEEE Trans Med Imaging 401:371\u2013380","journal-title":"IEEE Trans Med Imaging"},{"key":"3308_CR5","doi-asserted-by":"publisher","first-page":"1285","DOI":"10.1007\/s11548-019-01974-6","volume":"148","author":"X Zhang","year":"2019","unstructured":"Zhang X, Wang J, Wang T, Ji X, Shen Y, Sun Z, Zhang X (2019) A markerless automatic deformable registration framework for augmented reality navigation of laparoscopy partial nephrectomy. Int J Comput Assist Radiol Surg 148:1285\u20131294","journal-title":"Int J Comput Assist Radiol Surg"},{"key":"3308_CR6","doi-asserted-by":"publisher","first-page":"1260","DOI":"10.1016\/j.euf.2020.08.004","volume":"76","author":"R Schiavina","year":"2021","unstructured":"Schiavina R, Bianchi L, Lodi S, Cercenelli L, Chessa F, Bortolani B, Gaudiano C, Casablanca C, Droghetti M, Porreca A, Romagnoli D, Golfieri R, Giunchi F, Fiorentino M, Marcelli E, Diciotti S, Brunocilla E (2021) Real-time augmented reality three-dimensional guided robotic radical prostatectomy: preliminary experience and evaluation of the impact on surgical planning. Eur Urol Focus 76:1260\u20131267","journal-title":"Eur Urol Focus"},{"key":"3308_CR7","doi-asserted-by":"publisher","first-page":"20TR01","DOI":"10.1088\/1361-6560\/ab843e","volume":"6520","author":"Y Fu","year":"2020","unstructured":"Fu Y, Lei Y, Wang T, Curran WJ, Liu T, Yang X (2020) Deep learning in medical image registration: a review. Phys Med Biol 6520:20TR01","journal-title":"Phys Med Biol"},{"key":"3308_CR8","doi-asserted-by":"publisher","first-page":"1330","DOI":"10.1109\/34.888718","volume":"2211","author":"Z Zhang","year":"2000","unstructured":"Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 2211:1330\u20131334. https:\/\/doi.org\/10.1109\/34.888718","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"3308_CR9","unstructured":"OpenCV. Camera Calibration and 3D Reconstruction. Open Source Computer Vision; [updated 17\u201310\u20132024; cited 2024 21\u201310\u20132024]; Available from: https:\/\/docs.opencv.org\/4.10.0\/d9\/d0c\/group__calib3d.html#ga617b1685d4059c6040827800e72ad2b6."},{"key":"3308_CR10","doi-asserted-by":"publisher","first-page":"309","DOI":"10.1016\/j.xjtc.2021.03.016","volume":"7","author":"AH Sadeghi","year":"2021","unstructured":"Sadeghi AH, Maat A, Taverne Y, Cornelissen R, Dingemans AC, Bogers A, Mahtab EAF (2021) Virtual reality and artificial intelligence for 3-dimensional planning of lung segmentectomies. JTCVS Tech 7:309\u2013321","journal-title":"JTCVS Tech"},{"key":"3308_CR11","unstructured":"van Rossum G, Drake FL.(2009) Python 3 reference manual. Scotts valley, CA: CreateSpace."},{"key":"3308_CR12","first-page":"120","volume":"2511","author":"G Bradski","year":"2000","unstructured":"Bradski G (2000) The opencv library. Dr Dobb\u2019s J 2511:120\u2013125","journal-title":"Dr Dobb's J"},{"key":"3308_CR13","unstructured":"Hirschm\u00fcller H, editor. Accurate and efficient stereo processing by semi-global matching and mutual information. Computer vision and pattern recognition; (2005)."},{"key":"3308_CR14","unstructured":"Computer Vision Toolbox version (R2023b). (2023): The Mathworks Inc.; 2023; Available from: https:\/\/nl.mathworks.com\/help\/vision\/."},{"key":"3308_CR15","doi-asserted-by":"crossref","unstructured":"Kikinis R, Pieper SD, Vosburgh KG. (2014). 3D Slicer: A platform for subject-specific image analysis, visualization, and clinical support. In: Jolesz FA, editor. Intraoperative imaging and image-guided therapy. New York, NY: Springer New York; p. 277\u201389.","DOI":"10.1007\/978-1-4614-7657-3_19"},{"key":"3308_CR16","doi-asserted-by":"publisher","first-page":"239","DOI":"10.1109\/34.121791","volume":"14","author":"P Besl","year":"1992","unstructured":"Besl P, McKay HD (1992) A method for registration of 3-D shapes. IEEE Trans Pattern Anal Mach Intell 14:239\u2013256. https:\/\/doi.org\/10.1109\/34.121791","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"3308_CR17","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1109\/robot.1991.132043","volume":"10","author":"Y Chen","year":"1992","unstructured":"Chen Y, Medioni G (1992) Object modeling by registration of multiple range images. Image Vision Comput 10:145\u2013155. https:\/\/doi.org\/10.1109\/robot.1991.132043","journal-title":"Image Vision Comput"},{"key":"3308_CR18","doi-asserted-by":"publisher","first-page":"646","DOI":"10.1177\/15533506241290412","volume":"316","author":"M-CJ Doornbos","year":"2024","unstructured":"Doornbos M-CJ, Peek JJ, Maat APWM, Ruurda JP, De Backer P, Cornelissen BMW, Mahtab EAF, Sadeghi AH, Kluin J (2024) Augmented reality implementation in minimally invasive surgery for future application in pulmonary surgery: a systematic review. Surg Innov 316:646\u2013658. https:\/\/doi.org\/10.1177\/15533506241290412","journal-title":"Surg Innov"},{"key":"3308_CR19","doi-asserted-by":"publisher","first-page":"121","DOI":"10.1016\/j.xjtc.2024.04.011","volume":"26","author":"AH Sadeghi","year":"2024","unstructured":"Sadeghi AH, Mank Q, Tuzcu AS, Hofman J, Siregar S, Maat A, Mottrie A, Kluin J, De Backer P (2024) Artificial intelligence\u2013assisted augmented reality robotic lung surgery: Navigating the future of thoracic surgery. JTCVS Tech 26:121\u2013125. https:\/\/doi.org\/10.1016\/j.xjtc.2024.04.011","journal-title":"JTCVS Tech"},{"key":"3308_CR20","doi-asserted-by":"publisher","first-page":"404","DOI":"10.1007\/978-3-319-46720-7_47","volume-title":"Medical image computing and computer-assisted intervention \u2013 MICCAI 2016: 19th international conference, Athens, Greece, October 17-21, 2016, Proceedings, Part I","author":"T Collins","year":"2016","unstructured":"Collins T, Bartoli A, Bourdel N, Canis M (2016) Robust, real-time, dense and deformable 3D organ tracking in laparoscopic videos. In: Ourselin S, Joskowicz L, Sabuncu MR, Unal G, Wells W (eds) Medical image computing and computer-assisted intervention \u2013 MICCAI 2016: 19th international conference, Athens, Greece, October 17-21, 2016, Proceedings, Part I. Springer International Publishing, Cham, pp 404\u2013412. https:\/\/doi.org\/10.1007\/978-3-319-46720-7_47"},{"key":"3308_CR21","doi-asserted-by":"publisher","first-page":"101140","DOI":"10.1016\/j.gmod.2022.101140","volume":"121","author":"L Zhou","year":"2022","unstructured":"Zhou L, Sun G, Li Y, Li W, Zhiyong S (2022) Point cloud denoising review: from classical to deep learning-based approaches. Gr Models 121:101140. https:\/\/doi.org\/10.1016\/j.gmod.2022.101140","journal-title":"Gr Models"},{"key":"3308_CR22","doi-asserted-by":"publisher","DOI":"10.1016\/j.eururo.2023.02.024","author":"P De Backer","year":"2023","unstructured":"De Backer P, Van Praet C, Simoens J, Peraire Lores M, Creemers H, Mestdagh K, Allaeys C, Vermijs S, Piazza P, Mottaran A, Bravi CA, Paciotti M, Sarchi L, Farinha R, Puliatti S, Cisternino F, Ferraguti F, Debbaut C, De Naeyer G, Decaestecker K, Mottrie A (2023) Improving augmented reality through deep learning: real-time instrument delineation in robotic renal surgery. Eur Urol. https:\/\/doi.org\/10.1016\/j.eururo.2023.02.024","journal-title":"Eur Urol"},{"key":"3308_CR23","doi-asserted-by":"publisher","first-page":"197","DOI":"10.1007\/s11548-015-1276-0","volume":"112","author":"V Penza","year":"2016","unstructured":"Penza V, Ortiz J, Mattos LS, Forgione A, De Momi E (2016) Dense soft tissue 3D reconstruction refined with super-pixel segmentation for robotic abdominal surgery. Int J Comput Assist Radiol Surg 112:197\u2013206. https:\/\/doi.org\/10.1007\/s11548-015-1276-0","journal-title":"Int J Comput Assist Radiol Surg"},{"key":"3308_CR24","doi-asserted-by":"publisher","first-page":"106077","DOI":"10.1016\/j.cmpb.2021.106077","volume":"205","author":"L Xi","year":"2021","unstructured":"Xi L, Zhao Y, Chen L, Gao QH, Tang W, Wan TR, Xue T (2021) Recovering dense 3D point clouds from single endoscopic image. Comput Methods Prog Biomed 205:106077. https:\/\/doi.org\/10.1016\/j.cmpb.2021.106077","journal-title":"Comput Methods Prog Biomed"},{"key":"3308_CR25","doi-asserted-by":"crossref","unstructured":"Pluim JPW, Muenzing SEA, Eppenhof KAJ, Murphy K, editors. (2016) The truth is hard to make: Validation of medical image registration. In 2016 23rd international conference on pattern recognition (ICPR); 4\u20138 Dec. 2016.","DOI":"10.1109\/ICPR.2016.7899978"},{"key":"3308_CR26","doi-asserted-by":"publisher","first-page":"2530","DOI":"10.1109\/TGRS.2019.2952086","volume":"584","author":"S Chen","year":"2020","unstructured":"Chen S, Nan L, Xia R, Zhao J, Wonka P (2020) PLADE: a plane-based descriptor for point cloud registration with small overlap. IEEE Trans Geosci Remote Sens 584:2530\u20132540. https:\/\/doi.org\/10.1109\/TGRS.2019.2952086","journal-title":"IEEE Trans Geosci Remote Sens"},{"issue":"3","key":"3308_CR27","doi-asserted-by":"publisher","first-page":"1629","DOI":"10.3390\/app13031629","volume":"13","author":"S Malhotra","year":"2023","unstructured":"Malhotra S, Halabi O, Dakua SP, Padhan J, Paul S, Palliyali W (2023) Augmented reality in surgical navigation: a review of evaluation and validation metrics. Appl Sci 13(3):1629. https:\/\/doi.org\/10.3390\/app13031629","journal-title":"Appl Sci"},{"key":"3308_CR28","doi-asserted-by":"crossref","unstructured":"Newcombe RA, Izadi S, Hilliges O, Molyneaux D, Kim D, Davison AJ, Kohi P, Shotton J, Hodges S, Fitzgibbon A, editors. (2011) KinectFusion: real-time dense surface mapping and tracking. In 2011 10th IEEE international symposium on mixed and augmented reality; 26\u201329","DOI":"10.1109\/ISMAR.2011.6162880"},{"issue":"3","key":"3308_CR29","doi-asserted-by":"publisher","first-page":"302","DOI":"10.1016\/j.media.2010.12.002","volume":"15","author":"R Richa","year":"2011","unstructured":"Richa R, B\u00f3 APL, Poignet P (2011) Towards robust 3D visual tracking for motion compensation in beating heart surgery. Med Image Anal 15(3):302\u2013315. https:\/\/doi.org\/10.1016\/j.media.2010.12.002","journal-title":"Med Image Anal"},{"key":"3308_CR30","doi-asserted-by":"publisher","DOI":"10.1159\/000513335","author":"O Ivashchenko","year":"2021","unstructured":"Ivashchenko O, Smit J, Ter Beek L, Rijkhorst E-J, Nijkamp J, Ruers T, Kuhlmann K (2021) Clinical implementation of in-house developed MR-based patient-specific 3D models of liver anatomy. Eur Surg Res. https:\/\/doi.org\/10.1159\/000513335","journal-title":"Eur Surg Res"}],"container-title":["International Journal of Computer Assisted Radiology and Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-024-03308-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11548-024-03308-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-024-03308-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,4,27]],"date-time":"2025-04-27T10:03:01Z","timestamp":1745748181000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11548-024-03308-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,20]]},"references-count":30,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2025,4]]}},"alternative-id":["3308"],"URL":"https:\/\/doi.org\/10.1007\/s11548-024-03308-7","relation":{},"ISSN":["1861-6429"],"issn-type":[{"value":"1861-6429","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,20]]},"assertion":[{"value":"19 August 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 December 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 December 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"None.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}]}}