{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,19]],"date-time":"2026-03-19T05:16:52Z","timestamp":1773897412604,"version":"3.50.1"},"reference-count":35,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2021,5,1]],"date-time":"2021-05-01T00:00:00Z","timestamp":1619827200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,5,12]],"date-time":"2021-05-12T00:00:00Z","timestamp":1620777600000},"content-version":"vor","delay-in-days":11,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100002347","name":"Bundesministerium f\u00fcr Bildung und Forschung","doi-asserted-by":"publisher","award":["16 SV 8019"],"award-info":[{"award-number":["16 SV 8019"]}],"id":[{"id":"10.13039\/501100002347","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J CARS"],"published-print":{"date-parts":[[2021,5]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>\n                           <jats:bold>Purpose<\/jats:bold>\n                        <\/jats:title>\n                <jats:p>Segmentation of surgical instruments in endoscopic video streams is essential for automated surgical scene understanding and process modeling. However, relying on fully supervised deep learning for this task is challenging because manual annotation occupies valuable time of the clinical experts.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>\n                           <jats:bold>Methods<\/jats:bold>\n                        <\/jats:title>\n                <jats:p>We introduce a teacher\u2013student learning approach that learns jointly from annotated simulation data and unlabeled real data to tackle the challenges in <jats:italic>simulation-to-real<\/jats:italic> unsupervised domain adaptation for endoscopic image segmentation.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>\n                           <jats:bold>Results<\/jats:bold>\n                        <\/jats:title>\n                <jats:p>Empirical results on three datasets highlight the effectiveness of the proposed framework over current approaches for the endoscopic instrument segmentation task. Additionally, we provide analysis of major factors affecting the performance on all datasets to highlight the strengths and failure modes of our approach.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>\n                           <jats:bold>Conclusions<\/jats:bold>\n                        <\/jats:title>\n                <jats:p>We show that our proposed approach can successfully exploit the unlabeled real endoscopic video frames and improve generalization performance over pure simulation-based training and the previous state-of-the-art. This takes us one step closer to effective segmentation of surgical instrument in the annotation scarce setting.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1007\/s11548-021-02383-4","type":"journal-article","created":{"date-parts":[[2021,5,12]],"date-time":"2021-05-12T17:02:54Z","timestamp":1620838974000},"page":"849-859","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":22,"title":["Simulation-to-real domain adaptation with teacher\u2013student learning for endoscopic instrument segmentation"],"prefix":"10.1007","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2158-823X","authenticated-orcid":false,"given":"Manish","family":"Sahu","sequence":"first","affiliation":[]},{"given":"Anirban","family":"Mukhopadhyay","sequence":"additional","affiliation":[]},{"given":"Stefan","family":"Zachow","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,5,12]]},"reference":[{"key":"2383_CR1","unstructured":"(2015) Endovis sub-challenge: instrument segmentation and tracking. https:\/\/endovissub-instrument.grand-challenge.org\/. Accessed 28 October 2020"},{"issue":"1","key":"2383_CR2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41598-019-56847-4","volume":"10","author":"S Ali","year":"2020","unstructured":"Ali S, Zhou F, Braden B, Bailey A, Yang S, Cheng G, Zhang P, Li X, Kayser M, Soberanis-Mukul RD, Albarqouni S, Wang X, Wang C, Watanabe S, Oksuz I, Ning Q, Yang S, Khan MA, Gao XW, Realdon S, Loshchenov M, Schnabel JA, East JE, Wagnieres G, Loschenov VB, Grisan E, Daul C, Blondel W, Rittscher J (2020) An objective comparison of detection and segmentation algorithms for artefacts in clinical endoscopy. Sci Rep 10(1):1\u201315","journal-title":"Sci Rep"},{"issue":"5","key":"2383_CR3","doi-asserted-by":"publisher","first-page":"1204","DOI":"10.1109\/TMI.2018.2794439","volume":"37","author":"M Allan","year":"2018","unstructured":"Allan M, Ourselin S, Hawkes DJ, Kelly JD, Stoyanov D (2018) 3-D pose estimation of articulated instruments in robotic minimally invasive surgery. IEEE Trans Med Imaging 37(5):1204\u20131213","journal-title":"IEEE Trans Med Imaging"},{"issue":"12","key":"2383_CR4","doi-asserted-by":"publisher","first-page":"e1006613","DOI":"10.1371\/journal.pcbi.1006613","volume":"14","author":"N Baker","year":"2018","unstructured":"Baker N, Lu H, Erlikhman G, Kellman PJ (2018) Deep convolutional networks do not classify based on global object shape. PLoS Comput Biol 14(12):e1006613","journal-title":"PLoS Comput Biol"},{"key":"2383_CR5","unstructured":"Bodenstedt S, Allan M, Agustinos A, Du X, Garcia-Peraza-Herrera L, Kenngott H, Kurmann T, M\u00fcller-Stich B, Ourselin S, Pakhomov D, Sznitman R, Teichmann M, Thoma M, Vercauteren T, Voros S, Wagner M, Wochner P, Maier-Hein L, Stoyanov D, Speidel S (2018) Comparative evaluation of instrument segmentation and tracking methods in minimally invasive surgery. arXiv:1805.02475"},{"issue":"3","key":"2383_CR6","doi-asserted-by":"publisher","first-page":"542","DOI":"10.1109\/TNN.2009.2015974","volume":"20","author":"O Chapelle","year":"2009","unstructured":"Chapelle O, Scholkopf B, Zien A (2009) Semi-supervised learning. IEEE Trans Neural Netw 20(3):542","journal-title":"IEEE Trans Neural Netw"},{"key":"2383_CR7","doi-asserted-by":"crossref","unstructured":"Colleoni E, Edwards P, Stoyanov D (2020) Synthetic and real inputs for tool segmentation in robotic surgery. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 700\u2013710","DOI":"10.1007\/978-3-030-59716-0_67"},{"issue":"5","key":"2383_CR8","doi-asserted-by":"publisher","first-page":"1276","DOI":"10.1109\/TMI.2017.2787672","volume":"37","author":"X Du","year":"2018","unstructured":"Du X, Kurmann T, Chang PL, Allan M, Ourselin S, Sznitman R, Kelly JD, Stoyanov D (2018) Articulated multi-instrument 2-d pose estimation using fully convolutional networks. IEEE Trans Med Imaging 37(5):1276\u20131287","journal-title":"IEEE Trans Med Imaging"},{"key":"2383_CR9","doi-asserted-by":"crossref","unstructured":"Engelhardt S, De Simone R, Full PM, Karck M, Wolf I (2018) Improving surgical training phantoms by hyperrealism: deep unpaired image-to-image translation from real surgeries. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 747\u2013755","DOI":"10.1007\/978-3-030-00928-1_84"},{"key":"2383_CR10","doi-asserted-by":"crossref","unstructured":"Engelhardt S, Sharan L, Karck M, De Simone R, Wolf I (2019) Cross-domain conditional generative adversarial networks for stereoscopic hyperrealism in surgical training. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 155\u2013163","DOI":"10.1007\/978-3-030-32254-0_18"},{"key":"2383_CR11","unstructured":"French G, Mackiewicz M, Fisher M (2018) Self-ensembling for visual domain adaptation. In: International conference on learning representations"},{"key":"2383_CR12","doi-asserted-by":"publisher","unstructured":"Fuentes-Hurtado F, Kadkhodamohammadi A, Flouty E, Barbarisi S, Luengo I, Stoyanov D (2019) Easylabels: weak labels for scene segmentation in laparoscopic videos. Int J Comput Assist Radiol Surg 14:1247\u20131257. https:\/\/doi.org\/10.1007\/s11548-019-02003-2","DOI":"10.1007\/s11548-019-02003-2"},{"key":"2383_CR13","doi-asserted-by":"crossref","unstructured":"Gonz\u00e1lez C, Bravo-S\u00e1nchez L, Arbelaez P (2020) Isinet: an instance-based approach for surgical instrument segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 595\u2013605","DOI":"10.1007\/978-3-030-59716-0_57"},{"key":"2383_CR14","unstructured":"Hoffman J, Wang D, Yu F, Darrell T (2016) FCNs in the wild: pixel-level adversarial and constraint-based adaptation. arXiv:1612.02649"},{"key":"2383_CR15","doi-asserted-by":"crossref","unstructured":"Jin Y, Cheng K, Dou Q, Heng PA (2019) Incorporating temporal prior from motion flow for instrument segmentation in minimally invasive surgery video. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 440\u2013448","DOI":"10.1007\/978-3-030-32254-0_49"},{"key":"2383_CR16","doi-asserted-by":"crossref","unstructured":"Laina I, Rieke N, Rupprecht C, Vizca\u00edno JP, Eslami A, Tombari F, Navab N (2017) Concurrent segmentation and localization for tracking of surgical instruments. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 664\u2013672","DOI":"10.1007\/978-3-319-66185-8_75"},{"key":"2383_CR17","unstructured":"Laine S, Aila T (2017) Temporal ensembling for semi-supervised learning. In: International conference on learning representations"},{"key":"2383_CR18","doi-asserted-by":"crossref","unstructured":"Liu D, Wei Y, Jiang T, Wang Y, Miao R, Shan F, Li Z (2020) Unsupervised surgical instrument segmentation via anchor generation and semantic diffusion. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 657\u2013667","DOI":"10.1007\/978-3-030-59716-0_63"},{"key":"2383_CR19","unstructured":"Luengo I, Flouty E, Giataganas P, Wisanuvej P, Nehme J, Stoyanov D (2018) Surreal: enhancing surgical simulation realism using style transfer. In: British machine vision conference 2018, BMVC 2018, BMVA, pp 1\u201312"},{"issue":"12","key":"2383_CR20","doi-asserted-by":"publisher","first-page":"2572","DOI":"10.1109\/TMI.2018.2842767","volume":"37","author":"F Mahmood","year":"2018","unstructured":"Mahmood F, Chen R, Durr NJ (2018) Unsupervised reverse domain adaptation for synthetic medical images via adversarial training. IEEE Trans Med Imaging 37(12):2572\u20132581","journal-title":"IEEE Trans Med Imaging"},{"key":"2383_CR21","doi-asserted-by":"publisher","first-page":"105834","DOI":"10.1016\/j.cmpb.2020.105834","volume":"200","author":"A Marzullo","year":"2020","unstructured":"Marzullo A, Moccia S, Catellani M, Calimeri F, De Momi E (2020) Towards realistic laparoscopic image generation using image-domain translation. Comput Methods Programs Biomed 200:105834","journal-title":"Comput Methods Programs Biomed"},{"issue":"6","key":"2383_CR22","doi-asserted-by":"publisher","first-page":"214","DOI":"10.1049\/htl.2019.0071","volume":"6","author":"M Oda","year":"2019","unstructured":"Oda M, Tanaka K, Takabatake H, Mori M, Natori H, Mori K (2019) Realistic endoscopic image generation method using virtual-to-real image-domain translation. Healthc Technol Lett 6(6):214\u2013219","journal-title":"Healthc Technol Lett"},{"key":"2383_CR23","doi-asserted-by":"crossref","unstructured":"Pfeiffer M, Funke I, Robu MR, Bodenstedt S, Strenger L, Engelhardt S, Ro\u00df T, Clarkson MJ, Gurusamy K, Davidson BR, Maier-Hein L, Riediger C, Welsch T, Weitz J, Speidel S (2019) Generating large labeled data sets for laparoscopic image processing tasks using unpaired image-to-image translation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 119\u2013127","DOI":"10.1007\/978-3-030-32254-0_14"},{"issue":"7","key":"2383_CR24","doi-asserted-by":"publisher","first-page":"1167","DOI":"10.1007\/s11548-019-01962-w","volume":"14","author":"A Rau","year":"2019","unstructured":"Rau A, Edwards PE, Ahmad OF, Riordan P, Janatka M, Lovat LB, Stoyanov D (2019) Implicit domain adaptation with conditional generative adversarial networks for depth prediction in endoscopy. Int J Comput Assist Radiol Surg 14(7):1167\u20131176","journal-title":"Int J Comput Assist Radiol Surg"},{"issue":"6","key":"2383_CR25","doi-asserted-by":"publisher","first-page":"925","DOI":"10.1007\/s11548-018-1772-0","volume":"13","author":"T Ross","year":"2018","unstructured":"Ross T, Zimmerer D, Vemuri A, Isensee F, Wiesenfarth M, Bodenstedt S, Both F, Kessler P, Wagner M, M\u00fcller B, Kenngott H, Speidel S, Kopp-Schneider A, Maier-Hein K, Maier-Hein L (2018) Exploiting the potential of unlabeled endoscopic video data with self-supervised learning. Int J Comput Assist Radiol Surg 13(6):925\u2013933","journal-title":"Int J Comput Assist Radiol Surg"},{"key":"2383_CR26","unstructured":"Ross T, Reinke A, Full PM, Wagner M, Kenngott H, Apitz M, Hempe H, Mindroc Filimon D, Scholz P, Nuong Tran T, Bruno P, Arbel\u00e1ez P, Bian GB, Bodenstedt S, Lindstr\u00f6m Bolmgren J, Bravo-S\u00e1nchez L, Chen HB, Gonz\u00e1lez C, Guo D, Halvorsen P, Heng PA, Hosgor E, Hou ZG, Isensee F, Jha D, Jiang T, Jin Y, Kirtac K, Kletz S, Leger S, Li Z, Maier-Hein KH, Ni ZL, Riegler MA, Schoeffmann K, Shi R, Speidel S, Stenzel M, Twick I, Wang G, Wang J, Wang L, Wang L, Zhang Y, Zhou YJ, Zhu L, Wiesenfarth M, Kopp-Schneider A, M\u00fcller-Stich BP, Maier-Hein L (2020) Robust medical instrument segmentation challenge 2019. arXiv:2003.10299"},{"key":"2383_CR27","doi-asserted-by":"crossref","unstructured":"Sahu M, Str\u00f6msd\u00f6rfer R, Mukhopadhyay A, Zachow S (2020) Endo-sim2real: consistency learning-based domain adaptation for instrument segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 784\u2013794","DOI":"10.1007\/978-3-030-59716-0_75"},{"key":"2383_CR28","doi-asserted-by":"crossref","unstructured":"Shvets AA, Rakhlin A, Kalinin AA, Iglovikov VI (2018) Automatic instrument segmentation in robot-assisted surgery using deep learning. In: 2018 17th IEEE international conference on machine learning and applications (ICMLA). IEEE, pp 624\u2013628","DOI":"10.1109\/ICMLA.2018.00100"},{"key":"2383_CR29","unstructured":"Tarvainen A, Valpola H (2017) Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results. In: Advances in neural information processing systems, pp 1195\u20131204"},{"key":"2383_CR30","doi-asserted-by":"crossref","unstructured":"Torralba A, Efros AA (2011) Unbiased look at dataset bias. In: CVPR 2011. IEEE, pp 1521\u20131528","DOI":"10.1109\/CVPR.2011.5995347"},{"issue":"1","key":"2383_CR31","doi-asserted-by":"publisher","first-page":"86","DOI":"10.1109\/TMI.2016.2593957","volume":"36","author":"AP Twinanda","year":"2016","unstructured":"Twinanda AP, Shehata S, Mutter D, Marescaux J, De Mathelin M, Padoy N (2016) Endonet: a deep architecture for recognition tasks on laparoscopic videos. IEEE Trans Med Imaging 36(1):86\u201397","journal-title":"IEEE Trans Med Imaging"},{"issue":"1","key":"2383_CR32","doi-asserted-by":"publisher","first-page":"198","DOI":"10.1109\/JPROC.2019.2946993","volume":"108","author":"T Vercauteren","year":"2020","unstructured":"Vercauteren T, Unberath M, Padoy N, Navab N (2020) Cai4cai: the rise of contextual artificial intelligence in computer-assisted interventions. Proc IEEE 108(1):198\u2013214. https:\/\/doi.org\/10.1109\/JPROC.2019.2946993","journal-title":"Proc IEEE"},{"key":"2383_CR33","doi-asserted-by":"publisher","first-page":"135","DOI":"10.1016\/j.neucom.2018.05.083","volume":"312","author":"M Wang","year":"2018","unstructured":"Wang M, Deng W (2018) Deep visual domain adaptation: a survey. Neurocomputing 312:135\u2013153","journal-title":"Neurocomputing"},{"issue":"5","key":"2383_CR34","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3400066","volume":"11","author":"G Wilson","year":"2020","unstructured":"Wilson G, Cook DJ (2020) A survey of unsupervised deep domain adaptation. ACM Trans Intell Syst Technol (TIST) 11(5):1\u201346","journal-title":"ACM Trans Intell Syst Technol (TIST)"},{"key":"2383_CR35","doi-asserted-by":"crossref","unstructured":"Zhang Y, David P, Gong B (2017) Curriculum domain adaptation for semantic segmentation of urban scenes. In: Proceedings of the IEEE international conference on computer vision, pp 2020\u20132030","DOI":"10.1109\/ICCV.2017.223"}],"container-title":["International Journal of Computer Assisted Radiology and Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-021-02383-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11548-021-02383-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-021-02383-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,5,20]],"date-time":"2021-05-20T09:02:45Z","timestamp":1621501365000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11548-021-02383-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,5]]},"references-count":35,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2021,5]]}},"alternative-id":["2383"],"URL":"https:\/\/doi.org\/10.1007\/s11548-021-02383-4","relation":{},"ISSN":["1861-6410","1861-6429"],"issn-type":[{"value":"1861-6410","type":"print"},{"value":"1861-6429","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,5]]},"assertion":[{"value":"18 March 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 April 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 May 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors state no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"This study contains patient data from a publicly available datasets.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Informed consent"}},{"value":"This article does not contain any studies with human participants or animals performed by any of the authors.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}}]}}