{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,14]],"date-time":"2026-07-14T14:52:03Z","timestamp":1784040723087,"version":"3.55.0"},"publisher-location":"Cham","reference-count":22,"publisher":"Springer Nature Switzerland","isbn-type":[{"value":"9783031439957","type":"print"},{"value":"9783031439964","type":"electronic"}],"license":[{"start":{"date-parts":[[2023,1,1]],"date-time":"2023-01-01T00:00:00Z","timestamp":1672531200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"},{"start":{"date-parts":[[2023,1,1]],"date-time":"2023-01-01T00:00:00Z","timestamp":1672531200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Robust semantic segmentation of intraoperative image data could pave the way for automatic surgical scene understanding and autonomous robotic surgery. Geometric domain shifts, however \u2013 although common in real-world open surgeries due to variations in surgical procedures or situs occlusions \u2013 remain a topic largely unaddressed in the field. To address this gap in the literature, we (1) present the first analysis of state-of-the-art (SOA) semantic segmentation networks in the presence of geometric out-of-distribution (OOD) data, and (2) address generalizability with a dedicated augmentation technique termed \u2019Organ Transplantation\u2019 that we adapted from the general computer vision community. According to a comprehensive validation on six different OOD data sets comprising 600 RGB and yperspectral imaging (HSI) cubes from 33 pigs semantically annotated with 19 classes, we demonstrate a large performance drop of SOA organ segmentation networks applied to geometric OOD data. Surprisingly, this holds true not only for conventional RGB data (drop of Dice similarity coefficient (DSC) by 46 %) but also for HSI data (drop by 45 %), despite the latter\u2019s rich information content per pixel. Using our augmentation scheme improves on the SOA DSC by up to 67% (RGB) and 90% (HSI)) and renders performance on par with in-distribution performance on real OOD test data. The simplicity and effectiveness of our augmentation scheme makes it a valuable network-independent tool for addressing geometric domain shifts in semantic scene segmentation of intraoperative data. Our code and pre-trained models are available at <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/github.com\/IMSY-DKFZ\/htc\">https:\/\/github.com\/IMSY-DKFZ\/htc<\/jats:ext-link>.<\/jats:p>","DOI":"10.1007\/978-3-031-43996-4_59","type":"book-chapter","created":{"date-parts":[[2023,9,30]],"date-time":"2023-09-30T23:07:48Z","timestamp":1696115268000},"page":"618-627","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Semantic Segmentation of\u00a0Surgical Hyperspectral Images Under Geometric Domain Shifts"],"prefix":"10.1007","author":[{"given":"Jan","family":"Sellner","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Silvia","family":"Seidlitz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alexander","family":"Studier-Fischer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alessandro","family":"Motta","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Berkin","family":"\u00d6zdemir","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Beat Peter","family":"M\u00fcller-Stich","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Felix","family":"Nickel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lena","family":"Maier-Hein","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,10,1]]},"reference":[{"issue":"2","key":"59_CR1","doi-asserted-by":"publisher","first-page":"46","DOI":"10.3390\/jimaging9020046","volume":"9","author":"K Alomar","year":"2023","unstructured":"Alomar, K., Aysel, H.I., Cai, X.: Data augmentation in classification and segmentation: a survey and new strategies. J. Imaging 9(2), 46 (2023)","journal-title":"J. Imaging"},{"key":"59_CR2","doi-asserted-by":"crossref","unstructured":"Chen, Z., Fu, Y., Chen, K., Jiang, Y.G.: Image block augmentation for one-shot learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, pp. 3379\u20133386 (2019)","DOI":"10.1609\/aaai.v33i01.33013379"},{"issue":"3","key":"59_CR3","doi-asserted-by":"publisher","first-page":"297","DOI":"10.2307\/1932409","volume":"26","author":"LR Dice","year":"1945","unstructured":"Dice, L.R.: Measures of the amount of ecologic association between species. Ecology 26(3), 297\u2013302 (1945)","journal-title":"Ecology"},{"key":"59_CR4","doi-asserted-by":"crossref","unstructured":"Dwibedi, D., Misra, I., Hebert, M.: Cut, Paste and Learn: Surprisingly Easy Synthesis for Instance Detection (2017)","DOI":"10.1109\/ICCV.2017.146"},{"key":"59_CR5","doi-asserted-by":"crossref","unstructured":"Ghiasi, G., et al.: Simple copy-paste is a strong data augmentation method for instance segmentation. In: 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, pp. 2917\u20132927. IEEE (2021)","DOI":"10.1109\/CVPR46437.2021.00294"},{"key":"59_CR6","unstructured":"Izmailov, P., Podoprikhin, D., Garipov, T., Vetrov, D., Wilson, A.G.: Averaging weights leads to wider optima and better generalization. In: Proceedings of the International Conference on Uncertainty in Artificial Intelligence (2018)"},{"key":"59_CR7","series-title":"Lecture Notes in Computer Science","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1007\/978-3-030-32695-1_1","volume-title":"OR 2.0 Context-Aware Operating Theaters and Machine Learning in Clinical Neuroimaging","author":"A Kadkhodamohammadi","year":"2019","unstructured":"Kadkhodamohammadi, A., Luengo, I., Barbarisi, S., Taleb, H., Flouty, E., Stoyanov, D.: Feature aggregation decoder for segmenting laparoscopic scenes. In: Zhou, L., et al. (eds.) OR 2.0\/MLCN -2019. LNCS, vol. 11796, pp. 3\u201311. Springer, Cham (2019). https:\/\/doi.org\/10.1007\/978-3-030-32695-1_1"},{"issue":"5","key":"59_CR8","doi-asserted-by":"publisher","first-page":"397","DOI":"10.1007\/s42979-021-00784-5","volume":"2","author":"MK Kar","year":"2021","unstructured":"Kar, M.K., Nath, M.K., Neog, D.R.: A review on progress in semantic image segmentation and its application to medical images. SN Comput. Sci. 2(5), 397 (2021)","journal-title":"SN Comput. Sci."},{"key":"59_CR9","unstructured":"Kingma, D.P., Ba, J.: Adam: A Method for Stochastic Optimization. arXiv:1412.6980 (2017)"},{"issue":"1","key":"59_CR10","doi-asserted-by":"publisher","first-page":"12575","DOI":"10.1038\/s41598-022-16923-8","volume":"12","author":"D Kitaguchi","year":"2022","unstructured":"Kitaguchi, D., Fujino, T., Takeshita, N., Hasegawa, H., Mori, K., Ito, M.: Limited generalizability of single deep neural network for surgical instrument segmentation in different surgical environments. Sci. Rep. 12(1), 12575 (2022)","journal-title":"Sci. Rep."},{"key":"59_CR11","unstructured":"Maier-Hein, L., et al.: Metrics reloaded: pitfalls and recommendations for image analysis validation (2023)"},{"key":"59_CR12","doi-asserted-by":"crossref","unstructured":"Nikolov, S., et al.: Clinically applicable segmentation of head and neck anatomy for radiotherapy: deep learning algorithm development and validation study. J. Med. Internet Res. 23(7) (2021)","DOI":"10.2196\/26151"},{"key":"59_CR13","series-title":"Lecture Notes in Computer Science","doi-asserted-by":"publisher","first-page":"234","DOI":"10.1007\/978-3-319-24574-4_28","volume-title":"Medical Image Computing and Computer-Assisted Intervention \u2013 MICCAI 2015","author":"O Ronneberger","year":"2015","unstructured":"Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234\u2013241. Springer, Cham (2015). https:\/\/doi.org\/10.1007\/978-3-319-24574-4_28"},{"key":"59_CR14","doi-asserted-by":"publisher","first-page":"20200016","DOI":"10.1515\/cdbme-2020-0016","volume":"6","author":"P Scheikl","year":"2020","unstructured":"Scheikl, P., et al.: Deep learning for semantic segmentation of organs and tissues in laparoscopic surgery. Curr. Dir. Biomed. Eng. 6, 20200016 (2020)","journal-title":"Curr. Dir. Biomed. Eng."},{"key":"59_CR15","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2022.102488","volume":"80","author":"S Seidlitz","year":"2022","unstructured":"Seidlitz, S., et al.: Robust deep learning-based semantic organ segmentation in hyperspectral images. Med. Image Anal. 80, 102488 (2022)","journal-title":"Med. Image Anal."},{"issue":"1","key":"59_CR16","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1186\/s40537-019-0197-0","volume":"6","author":"C Shorten","year":"2019","unstructured":"Shorten, C., Khoshgoftaar, T.M.: A survey on image data augmentation for deep learning. J. Big Data 6(1), 60 (2019)","journal-title":"J. Big Data"},{"key":"59_CR17","doi-asserted-by":"crossref","unstructured":"Singh, K.K., Lee, Y.J.: Hide-and-seek: forcing a network to be meticulous for weakly-supervised object and action localization. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 3544\u20133553 (2017)","DOI":"10.1109\/ICCV.2017.381"},{"key":"59_CR18","unstructured":"Tan, M., Le, Q.V.: EfficientNet: rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105\u20136114 (2019)"},{"key":"59_CR19","series-title":"LNCS","doi-asserted-by":"publisher","first-page":"355","DOI":"10.1007\/978-3-031-16449-1_34","volume-title":"Medical Image Computing and Computer Assisted Intervention","author":"A Wang","year":"2022","unstructured":"Wang, A., Islam, M., Xu, M., Ren, H.: Rethinking surgical instrument segmentation: a background image can be all you need. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds.) MICCAI 2022. LNCS, vol. 13437, pp. 355\u2013364. Springer, Cham (2022). https:\/\/doi.org\/10.1007\/978-3-031-16449-1_34"},{"issue":"1","key":"59_CR20","doi-asserted-by":"publisher","first-page":"2369","DOI":"10.1038\/s41598-021-82017-6","volume":"11","author":"M Wiesenfarth","year":"2021","unstructured":"Wiesenfarth, M., et al.: Methods and open-source toolkit for analyzing and visualizing challenge results. Sci. Rep. 11(1), 2369 (2021)","journal-title":"Sci. Rep."},{"key":"59_CR21","doi-asserted-by":"crossref","unstructured":"Yun, S., Han, D., Chun, S., Oh, S.J., Yoo, Y., Choe, J.: CutMix: regularization strategy to train strong classifiers with localizable features. In: 2019 IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), pp. 6022\u20136031. IEEE (2019)","DOI":"10.1109\/ICCV.2019.00612"},{"key":"59_CR22","doi-asserted-by":"crossref","unstructured":"Zhong, Z., Zheng, L., Kang, G., Li, S., Yang, Y.: Random erasing data augmentation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 07, pp. 13001\u201313008 (2020)","DOI":"10.1609\/aaai.v34i07.7000"}],"updated-by":[{"DOI":"10.1007\/978-3-031-43996-4_71","type":"correction","label":"Correction","source":"publisher","updated":{"date-parts":[[2023,10,1]],"date-time":"2023-10-01T00:00:00Z","timestamp":1696118400000}}],"container-title":["Lecture Notes in Computer Science","Medical Image Computing and Computer Assisted Intervention \u2013 MICCAI 2023"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/978-3-031-43996-4_59","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,4]],"date-time":"2024-07-04T16:11:44Z","timestamp":1720109504000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/978-3-031-43996-4_59"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023]]},"ISBN":["9783031439957","9783031439964"],"references-count":22,"URL":"https:\/\/doi.org\/10.1007\/978-3-031-43996-4_59","relation":{"correction":[{"id-type":"doi","id":"10.1007\/978-3-031-43996-4_71","asserted-by":"object"}]},"ISSN":["0302-9743","1611-3349"],"issn-type":[{"value":"0302-9743","type":"print"},{"value":"1611-3349","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023]]},"assertion":[{"value":"1 October 2023","order":1,"name":"first_online","label":"First Online","group":{"name":"ChapterHistory","label":"Chapter History"}},{"value":"1 October 2023","order":2,"name":"change_date","label":"Change Date","group":{"name":"ChapterHistory","label":"Chapter History"}},{"value":"Correction","order":3,"name":"change_type","label":"Change Type","group":{"name":"ChapterHistory","label":"Chapter History"}},{"value":"A correction has been published.","order":4,"name":"change_details","label":"Change Details","group":{"name":"ChapterHistory","label":"Chapter History"}},{"value":"MICCAI","order":1,"name":"conference_acronym","label":"Conference Acronym","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"International Conference on Medical Image Computing and Computer-Assisted Intervention","order":2,"name":"conference_name","label":"Conference Name","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Vancouver, BC","order":3,"name":"conference_city","label":"Conference City","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Canada","order":4,"name":"conference_country","label":"Conference Country","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"2023","order":5,"name":"conference_year","label":"Conference Year","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"8 October 2023","order":7,"name":"conference_start_date","label":"Conference Start Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"12 October 2023","order":8,"name":"conference_end_date","label":"Conference End Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"miccai2023","order":10,"name":"conference_id","label":"Conference ID","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"https:\/\/conferences.miccai.org\/2023\/en\/","order":11,"name":"conference_url","label":"Conference URL","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Double-blind","order":1,"name":"type","label":"Type","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"CMT","order":2,"name":"conference_management_system","label":"Conference Management System","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"2250","order":3,"name":"number_of_submissions_sent_for_review","label":"Number of Submissions Sent for Review","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"730","order":4,"name":"number_of_full_papers_accepted","label":"Number of Full Papers Accepted","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"0","order":5,"name":"number_of_short_papers_accepted","label":"Number of Short Papers Accepted","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"32% - The value is computed by the equation \"Number of Full Papers Accepted \/ Number of Submissions Sent for Review * 100\" and then rounded to a whole number.","order":6,"name":"acceptance_rate_of_full_papers","label":"Acceptance Rate of Full Papers","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"3","order":7,"name":"average_number_of_reviews_per_paper","label":"Average Number of Reviews per Paper","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"5","order":8,"name":"average_number_of_papers_per_reviewer","label":"Average Number of Papers per Reviewer","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"Yes","order":9,"name":"external_reviewers_involved","label":"External Reviewers Involved","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}}]}}