{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,2]],"date-time":"2026-07-02T23:31:22Z","timestamp":1783035082182,"version":"3.54.6"},"reference-count":25,"publisher":"Springer Science and Business Media LLC","issue":"6","license":[{"start":{"date-parts":[[2024,3,8]],"date-time":"2024-03-08T00:00:00Z","timestamp":1709856000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,3,8]],"date-time":"2024-03-08T00:00:00Z","timestamp":1709856000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J CARS"],"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Purpose<\/jats:title>\n                <jats:p>Depth estimation in robotic surgery is vital in 3D reconstruction, surgical navigation and augmented reality visualization. Although the foundation model exhibits outstanding performance in many vision tasks, including depth estimation (e.g., DINOv2), recent works observed its limitations in medical and surgical domain-specific applications. This work presents a low-ranked adaptation (LoRA) of the foundation model for surgical depth estimation.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>We design a foundation model-based depth estimation method, referred to as Surgical-DINO, a low-rank adaptation of the DINOv2 for depth estimation in endoscopic surgery. We build LoRA layers and integrate them into DINO to adapt with surgery-specific domain knowledge instead of conventional fine-tuning. During training, we freeze the DINO image encoder, which shows excellent visual representation capacity, and only optimize the LoRA layers and depth decoder to integrate features from the surgical scene.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>Our model is extensively validated on a MICCAI challenge dataset of SCARED, which is collected from da Vinci Xi endoscope surgery. We empirically show that Surgical-DINO significantly outperforms all the state-of-the-art models in endoscopic depth estimation tasks. The analysis with ablation studies has shown evidence of the remarkable effect of our LoRA layers and adaptation.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusion<\/jats:title>\n                <jats:p>Surgical-DINO shed some light on the successful adaptation of the foundation models into the surgical domain for depth estimation. There is clear evidence in the results that zero-shot prediction on pre-trained weights in computer vision datasets or naive fine-tuning is not sufficient to use the foundation model in the surgical domain directly.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1007\/s11548-024-03083-5","type":"journal-article","created":{"date-parts":[[2024,3,8]],"date-time":"2024-03-08T20:02:14Z","timestamp":1709928134000},"page":"1013-1020","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":56,"title":["Surgical-DINO: adapter learning of foundation models for depth estimation in endoscopic surgery"],"prefix":"10.1007","volume":"19","author":[{"given":"Beilei","family":"Cui","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mobarakol","family":"Islam","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Long","family":"Bai","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6488-1551","authenticated-orcid":false,"given":"Hongliang","family":"Ren","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,3,8]]},"reference":[{"key":"3083_CR1","first-page":"13","volume-title":"International conference on medical image computing and computer-assisted intervention","author":"R Zha","year":"2023","unstructured":"Zha R, Cheng X, Li H, Harandi M, Ge Z (2023) Endosurf: neural surface reconstruction of deformable tissues with stereo endoscope videos. International conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp 13\u201323"},{"issue":"5","key":"3083_CR2","doi-asserted-by":"publisher","first-page":"1438","DOI":"10.1109\/TMI.2019.2950936","volume":"39","author":"X Liu","year":"2019","unstructured":"Liu X, Sinha A, Ishii M, Hager GD, Reiter A, Taylor RH, Unberath M (2019) Dense depth estimation in monocular endoscopy with self-supervised learning methods. IEEE Trans Med Imaging 39(5):1438\u20131447","journal-title":"IEEE Trans Med Imaging"},{"issue":"7","key":"3083_CR3","doi-asserted-by":"publisher","first-page":"2236","DOI":"10.1109\/TUFFC.2022.3171676","volume":"69","author":"X Wei","year":"2022","unstructured":"Wei X, Wang Y, Ge L, Peng B, He Q, Wang R, Huang L, Xu Y, Luo J (2022) Unsupervised convolutional neural network for motion estimation in ultrasound elastography. IEEE Trans Ultrason Ferroelectr Freq Control 69(7):2236\u20132247","journal-title":"IEEE Trans Ultrason Ferroelectr Freq Control"},{"key":"3083_CR4","first-page":"431","volume-title":"International conference on medical image computing and computer-assisted intervention","author":"Y Wang","year":"2022","unstructured":"Wang Y, Long Y, Fan SH, Dou Q (2022) Neural rendering for stereo 3d reconstruction of deformable tissues in robotic surgery. International conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp 431\u2013441"},{"key":"3083_CR5","doi-asserted-by":"crossref","unstructured":"Kirillov A, Mintun E, Ravi N, Mao H, Rolland C, Gustafson L, Xiao T, Whitehead S, Berg AC, Lo W-Y, Doll\u00e1r P, Girshick R (2023) Segment anything. arXiv preprint arXiv:2304.02643","DOI":"10.1109\/ICCV51070.2023.00371"},{"key":"3083_CR6","unstructured":"Oquab M, Darcet T, Moutakanni T, Vo H, Szafraniec M, Khalidov V, Fernandez P, Haziza D, Massa F, El-Nouby A, Assran M, Ballas N, Galuba W, Howes R, Huang P-Y, Li S-W, Misra I, Rabbat M, Sharma V, Synnaeve G, Xu H, Jegou H, Mairal J, Labatut P, Joulin A, Bojanowski P (2023) Dinov2: learning robust visual features without supervision. arXiv preprint arXiv:2304.07193"},{"key":"3083_CR7","doi-asserted-by":"crossref","unstructured":"Wang A, Islam M, Xu M, Zhang Y, Ren H (2023) Sam meets robotic surgery: an empirical study on generalization, robustness and adaptation. arXiv preprint arXiv:2308.07156","DOI":"10.1007\/978-3-031-47401-9_23"},{"key":"3083_CR8","doi-asserted-by":"crossref","unstructured":"Chen T, Zhu L, Ding C, Cao R, Zhang S, Wang Y, Li Z, Sun L, Mao P, Zang Y (2023) Sam fails to segment anything?\u2013sam-adapter: adapting sam in underperformed scenes: Camouflage, shadow, and more. arXiv preprint arXiv:2304.09148","DOI":"10.1109\/ICCVW60793.2023.00361"},{"key":"3083_CR9","first-page":"156","volume-title":"MICCAI workshop on domain adaptation and representation transfer","author":"Q Wu","year":"2023","unstructured":"Wu Q, Zhang Y, Elbatel M (2023) Self-prompting large vision models for few-shot medical image segmentation. MICCAI workshop on domain adaptation and representation transfer. Springer, Berlin, pp 156\u2013167"},{"key":"3083_CR10","unstructured":"Hu EJ, yelong shen Wallis P, Allen-Zhu Z, Li Y, Wang S, Wang L, Chen W (2022) LoRA: low-rank adaptation of large language models. In: International conference on learning representations"},{"issue":"1","key":"3083_CR11","first-page":"5485","volume":"21","author":"C Raffel","year":"2020","unstructured":"Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou Y, Li W, Liu PJ (2020) Exploring the limits of transfer learning with a unified text-to-text transformer. J Mach Learn Res 21(1):5485\u20135551","journal-title":"J Mach Learn Res"},{"key":"3083_CR12","unstructured":"Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S, Uszkoreit J, Houlsby N (2021) An image is worth 16x16 words: transformers for image recognition at scale. In: International conference on learning representations"},{"key":"3083_CR13","doi-asserted-by":"crossref","unstructured":"Zhang K, Liu D (2023) Customized segment anything model for medical image segmentation. arXiv preprint arXiv:2304.13785","DOI":"10.2139\/ssrn.4495221"},{"key":"3083_CR14","unstructured":"Bhat SF, Alhashim I, Wonka P (2021) Adabins: depth estimation using adaptive bins. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp 4009\u20134018"},{"key":"3083_CR15","doi-asserted-by":"crossref","unstructured":"Li Z, Snavely N (2018) Megadepth: learning single-view depth prediction from internet photos. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2041\u20132050","DOI":"10.1109\/CVPR.2018.00218"},{"key":"3083_CR16","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2021.102338","volume":"77","author":"S Shao","year":"2022","unstructured":"Shao S, Pei Z, Chen W, Zhu W, Wu X, Sun D, Zhang B (2022) Self-supervised monocular depth and ego-motion estimation in endoscopy: appearance flow to the rescue. Med Image Anal 77:102338","journal-title":"Med Image Anal"},{"issue":"4","key":"3083_CR17","doi-asserted-by":"publisher","first-page":"7225","DOI":"10.1109\/LRA.2021.3095528","volume":"6","author":"D Recasens","year":"2021","unstructured":"Recasens D, Lamarca J, F\u00e1cil JM, Montiel J, Civera J (2021) Endo-depth-and-motion: reconstruction and tracking in endoscopic videos using depth networks and photometric constraints. IEEE Robot Autom Lett 6(4):7225\u20137232","journal-title":"IEEE Robot Autom Lett"},{"key":"3083_CR18","unstructured":"Loshchilov I, Hutter F (2019) Decoupled weight decay regularization. In: International conference on learning representations"},{"key":"3083_CR19","doi-asserted-by":"crossref","unstructured":"Zhou T, Brown M, Snavely N, Lowe DG (2017) Unsupervised learning of depth and ego-motion from video. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1851\u20131858","DOI":"10.1109\/CVPR.2017.700"},{"key":"3083_CR20","doi-asserted-by":"crossref","unstructured":"Fang Z, Chen X, Chen Y, Gool LV (2020) Towards good practice for CNN-based monocular depth estimation. In: Proceedings of the IEEE winter conference on applications of computer vision, pp 1091\u20131100","DOI":"10.1109\/WACV45572.2020.9093334"},{"key":"3083_CR21","doi-asserted-by":"crossref","unstructured":"Spencer J, Bowden R, Hadfield S (2020) Defeat-net: general monocular depth via simultaneous unsupervised representation learning. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp 14402\u201314413","DOI":"10.1109\/CVPR42600.2020.01441"},{"key":"3083_CR22","unstructured":"Bian J, Li Z, Wang N, Zhan H, Shen C, Cheng M-M, Reid I (2019) Unsupervised scale-consistent depth and ego-motion learning from monocular video. Adv Neural Inf Process Syst 32"},{"key":"3083_CR23","doi-asserted-by":"crossref","unstructured":"Godard C, Mac\u00a0Aodha O, Firman M, Brostow GJ (2019) Digging into self-supervised monocular depth estimation. In: Proceedings of the IEEE international conference on computer vision, pp 3828\u20133838","DOI":"10.1109\/ICCV.2019.00393"},{"key":"3083_CR24","doi-asserted-by":"crossref","unstructured":"Ozyoruk KB, Gokceler GI, Bobrow TL, Coskun G, Incetan K, Almalioglu Y, Mahmood F, Curto E, Perdigoto L, Oliveira M, Sahin H, Araujo H, Alexandrino H, Durr NJ, Gibert HB, Mehmet T (2021) Endoslam dataset and an unsupervised monocular visual odometry and depth estimation approach for endoscopic videos. Med Image Anal 71:102058","DOI":"10.1016\/j.media.2021.102058"},{"key":"3083_CR25","doi-asserted-by":"crossref","unstructured":"Silberman N, Hoiem D, Kohli P, Fergus R (2012) Indoor segmentation and support inference from RGBD images. In: Computer vision\u2013ECCV 2012: 12th European conference on computer vision, Florence, Italy, October 7\u201313, 2012, Proceedings, Part V 12, pp 746\u2013760. Springer","DOI":"10.1007\/978-3-642-33715-4_54"}],"container-title":["International Journal of Computer Assisted Radiology and Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-024-03083-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11548-024-03083-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-024-03083-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,14]],"date-time":"2024-06-14T11:23:33Z","timestamp":1718364213000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11548-024-03083-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,8]]},"references-count":25,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2024,6]]}},"alternative-id":["3083"],"URL":"https:\/\/doi.org\/10.1007\/s11548-024-03083-5","relation":{},"ISSN":["1861-6429"],"issn-type":[{"value":"1861-6429","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,8]]},"assertion":[{"value":"12 January 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 February 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 March 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"This article does not contain any studies with human participants or animals performed by any of the authors.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}},{"value":"This article does not contain patient data.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Informed consent"}}]}}