{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,25]],"date-time":"2026-09-25T11:27:31Z","timestamp":1790335651233,"version":"4.1.0"},"reference-count":20,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,4,12]],"date-time":"2021-04-12T00:00:00Z","timestamp":1618185600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,4,12]],"date-time":"2021-04-12T00:00:00Z","timestamp":1618185600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100010449","name":"Nationales Centrum f\u00fcr Tumorerkrankungen Heidelberg","doi-asserted-by":"publisher","award":["N\/A"],"award-info":[{"award-number":["N\/A"]}],"id":[{"id":"10.13039\/501100010449","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100010449","name":"Nationales Centrum f\u00fcr Tumorerkrankungen Heidelberg","doi-asserted-by":"publisher","award":["N\/A"],"award-info":[{"award-number":["N\/A"]}],"id":[{"id":"10.13039\/501100010449","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100010449","name":"Nationales Centrum f\u00fcr Tumorerkrankungen Heidelberg","doi-asserted-by":"publisher","award":["N\/A"],"award-info":[{"award-number":["N\/A"]}],"id":[{"id":"10.13039\/501100010449","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100010449","name":"Nationales Centrum f\u00fcr Tumorerkrankungen Heidelberg","doi-asserted-by":"publisher","award":["N\/A"],"award-info":[{"award-number":["N\/A"]}],"id":[{"id":"10.13039\/501100010449","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100010449","name":"Nationales Centrum f\u00fcr Tumorerkrankungen Heidelberg","doi-asserted-by":"publisher","award":["N\/A"],"award-info":[{"award-number":["N\/A"]}],"id":[{"id":"10.13039\/501100010449","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100010449","name":"Nationales Centrum f\u00fcr Tumorerkrankungen Heidelberg","doi-asserted-by":"publisher","award":["N\/A"],"award-info":[{"award-number":["N\/A"]}],"id":[{"id":"10.13039\/501100010449","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100006360","name":"Bundesministerium f\u00fcr Wirtschaft und Energie","doi-asserted-by":"publisher","award":["BMWI 01MT17001C"],"award-info":[{"award-number":["BMWI 01MT17001C"]}],"id":[{"id":"10.13039\/501100006360","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100006360","name":"Bundesministerium f\u00fcr Wirtschaft und Energie","doi-asserted-by":"publisher","award":["BMWI 01MT17001C"],"award-info":[{"award-number":["BMWI 01MT17001C"]}],"id":[{"id":"10.13039\/501100006360","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100006360","name":"Bundesministerium f\u00fcr Wirtschaft und Energie","doi-asserted-by":"publisher","award":["BMWI 01MT17001C"],"award-info":[{"award-number":["BMWI 01MT17001C"]}],"id":[{"id":"10.13039\/501100006360","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100006360","name":"Bundesministerium f\u00fcr Wirtschaft und Energie","doi-asserted-by":"publisher","award":["BMWI 01MT17001C"],"award-info":[{"award-number":["BMWI 01MT17001C"]}],"id":[{"id":"10.13039\/501100006360","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100006360","name":"Bundesministerium f\u00fcr Wirtschaft und Energie","doi-asserted-by":"publisher","award":["BMWI 01MT17001C"],"award-info":[{"award-number":["BMWI 01MT17001C"]}],"id":[{"id":"10.13039\/501100006360","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100010477","name":"Intuitive Surgical","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100010477","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Data"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Image-based tracking of medical instruments is an integral part of surgical data science applications. Previous research has addressed the tasks of detecting, segmenting and tracking medical instruments based on laparoscopic video data. However, the proposed methods still tend to fail when applied to challenging images and do not generalize well to data they have not been trained on. This paper introduces the Heidelberg Colorectal (HeiCo) data set - the first publicly available data set enabling comprehensive benchmarking of medical instrument detection and segmentation algorithms with a specific emphasis on method robustness and generalization capabilities. Our data set comprises 30 laparoscopic videos and corresponding sensor data from medical devices in the operating room for three different types of laparoscopic surgery. Annotations include surgical phase labels for all video frames as well as information on instrument presence and corresponding instance-wise segmentation masks for surgical instruments (if any) in more than 10,000 individual frames. The data has successfully been used to organize international competitions within the Endoscopic Vision Challenges 2017 and 2019.<\/jats:p>","DOI":"10.1038\/s41597-021-00882-2","type":"journal-article","created":{"date-parts":[[2021,4,12]],"date-time":"2021-04-12T06:09:07Z","timestamp":1618207747000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":92,"title":["Heidelberg colorectal data set for surgical data science in the sensor operating room"],"prefix":"10.1038","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4910-9368","authenticated-orcid":false,"given":"Lena","family":"Maier-Hein","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9831-9110","authenticated-orcid":false,"given":"Martin","family":"Wagner","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7094-4926","authenticated-orcid":false,"given":"Tobias","family":"Ross","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Annika","family":"Reinke","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sebastian","family":"Bodenstedt","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peter M.","family":"Full","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hellena","family":"Hempe","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Diana","family":"Mindroc-Filimon","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Patrick","family":"Scholz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thuy Nuong","family":"Tran","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0832-0151","authenticated-orcid":false,"given":"Pierangela","family":"Bruno","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anna","family":"Kisilenko","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Benjamin","family":"M\u00fcller","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tornike","family":"Davitashvili","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Manuela","family":"Capek","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Minu D.","family":"Tizabi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Matthias","family":"Eisenmann","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tim J.","family":"Adler","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5332-4856","authenticated-orcid":false,"given":"Janek","family":"Gr\u00f6hl","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Melanie","family":"Schellenberg","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Silvia","family":"Seidlitz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"T. Y. Emmy","family":"Lai","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"B\u00fcnyamin","family":"Pekdemir","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Veith","family":"Roethlingshoefer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fabian","family":"Both","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sebastian","family":"Bittel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marc","family":"Mengler","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lars","family":"M\u00fcndermann","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Martin","family":"Apitz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Annette","family":"Kopp-Schneider","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stefanie","family":"Speidel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6066-8238","authenticated-orcid":false,"given":"Felix","family":"Nickel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0895-4015","authenticated-orcid":false,"given":"Pascal","family":"Probst","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hannes G.","family":"Kenngott","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Beat P.","family":"M\u00fcller-Stich","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,4,12]]},"reference":[{"key":"882_CR1","doi-asserted-by":"publisher","first-page":"691","DOI":"10.1038\/s41551-017-0132-7","volume":"1","author":"L Maier-Hein","year":"2017","unstructured":"Maier-Hein, L. et al. Surgical data science for next-generation interventions. Nat. Biomed. Eng. 1, 691\u2013696, https:\/\/doi.org\/10.1038\/s41551-017-0132-7 (2017).","journal-title":"Nat. Biomed. Eng."},{"key":"882_CR2","doi-asserted-by":"crossref","unstructured":"Islam, M., Li, Y. & Ren, H. Learning where to look while tracking instruments in robot-assisted surgery. in Med. Image Comput. Comput. Assist. Interv., 412\u2013420 (Springer, 2019).","DOI":"10.1007\/978-3-030-32254-0_46"},{"key":"882_CR3","doi-asserted-by":"publisher","first-page":"1217","DOI":"10.1007\/s11548-019-01995-1","volume":"14","author":"I Funke","year":"2019","unstructured":"Funke, I., Mees, S. T., Weitz, J. & Speidel, S. Video-based surgical skill assessment using 3D convolutional neural networks. Int. J. Comput. Assist. Radiol. Surg. 14, 1217\u20131225, https:\/\/doi.org\/10.1007\/s11548-019-01995-1 (2019).","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"882_CR4","unstructured":"Allan, M. et al. 2017 Robotic instrument segmentation challenge. Preprint at https:\/\/arxiv.org\/abs\/1902.06426 (2019)."},{"key":"882_CR5","doi-asserted-by":"publisher","first-page":"925","DOI":"10.1007\/s11548-018-1772-0","volume":"13","author":"T Ross","year":"2018","unstructured":"Ross, T. et al. Exploiting the potential of unlabeled endoscopic video data with self-supervised learning. Int. J. Comput. Assist. Radiol. Surg. 13, 925\u2013933, https:\/\/doi.org\/10.1007\/s11548-018-1772-0 (2018).","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"882_CR6","doi-asserted-by":"publisher","first-page":"101796","DOI":"10.1016\/j.media.2020.101796","volume":"66","author":"L Maier-Hein","year":"2020","unstructured":"Maier-Hein, L. et al. BIAS: Transparent reporting of biomedical image analysis challenges. Med. Image Anal. 66, 101796, https:\/\/doi.org\/10.1016\/j.media.2020.101796 (2020).","journal-title":"Med. Image Anal."},{"key":"882_CR7","doi-asserted-by":"publisher","unstructured":"Ross, T. et al. Comparative validation of multi-instance instrument segmentation in endoscopy: results of the ROBUST-MIS 2019 challenge. Med. Image Anal. 101920, https:\/\/doi.org\/10.1016\/j.media.2020.101920 (2020).","DOI":"10.1016\/j.media.2020.101920"},{"key":"882_CR8","doi-asserted-by":"publisher","first-page":"2025","DOI":"10.1109\/TBME.2016.2647680","volume":"64","author":"N Ahmidi","year":"2017","unstructured":"Ahmidi, N. et al. A dataset and benchmarks for segmentation and recognition of gestures in robotic surgery. IEEE Trans. Biomed. Eng. 64, 2025\u20132041, https:\/\/doi.org\/10.1109\/TBME.2016.2647680 (2017).","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"882_CR9","doi-asserted-by":"publisher","first-page":"86","DOI":"10.1109\/TMI.2016.2593957","volume":"36","author":"AP Twinanda","year":"2017","unstructured":"Twinanda, A. P. et al. EndoNet: a deep architecture for recognition tasks on laparoscopic videos. IEEE Trans. Med. Imaging 36, 86\u201397, https:\/\/doi.org\/10.1109\/TMI.2016.2593957 (2017).","journal-title":"IEEE Trans. Med. Imaging"},{"key":"882_CR10","doi-asserted-by":"publisher","first-page":"495","DOI":"10.1007\/s11548-013-0940-5","volume":"9","author":"F Lalys","year":"2014","unstructured":"Lalys, F. & Jannin, P. Surgical process modelling: a review. Int. J. CARS 9, 495\u2013511, https:\/\/doi.org\/10.1007\/s11548-013-0940-5 (2014).","journal-title":"Int. J. CARS"},{"key":"882_CR11","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P. & Brox, T. U-net: convolutional networks for biomedical image segmentation. In Med. Image Comput. Comput. Assist. Interv., 234\u2013241 (Springer, 2015).","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"882_CR12","unstructured":"Goodfellow, I., Bengio, Y. & Courville, A. Deep learning. (The MIT Press, 2016)."},{"key":"882_CR13","doi-asserted-by":"publisher","DOI":"10.7303\/syn21903917","author":"T Ross","year":"2020","unstructured":"Ross, T. et al. Heidelberg Colorectal (HeiCo) Data Set for Surgical Data Science in the Sensor Operating Room. Synapse https:\/\/doi.org\/10.7303\/syn21903917 (2020)."},{"key":"882_CR14","doi-asserted-by":"publisher","DOI":"10.7303\/syn18779624","author":"T Ross","year":"2019","unstructured":"Ross, T. et al. Endoscopic Vision Challenge, Sub-Challenge - Robust Medical Instrument Segmentation (ROBUST-MIS) Challenge 2019. Synapse https:\/\/doi.org\/10.7303\/syn18779624 (2019)."},{"key":"882_CR15","doi-asserted-by":"publisher","first-page":"297","DOI":"10.2307\/1932409","volume":"26","author":"LR Dice","year":"1945","unstructured":"Dice, L. R. Measures of the amount of ecologic association between species. Ecology 26, 297\u2013302 (1945).","journal-title":"Ecology"},{"key":"882_CR16","doi-asserted-by":"publisher","first-page":"850","DOI":"10.1109\/34.232073","volume":"15","author":"DP Huttenlocher","year":"1993","unstructured":"Huttenlocher, D. P., Klanderman, G. A. & Rucklidge, W. J. Comparing images using the Hausdorff distance. IEEE Trans. Pattern Anal. Mach. Intell. 15, 850\u2013863 (1993).","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"882_CR17","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-018-07619-7","volume":"9","author":"L Maier-Hein","year":"2018","unstructured":"Maier-Hein, L. et al. Why rankings of biomedical image analysis competitions should be interpreted with care. Nat. Commun. 9, 5217, https:\/\/doi.org\/10.1038\/s41467-018-07619-7 (2018).","journal-title":"Nat. Commun."},{"key":"882_CR18","doi-asserted-by":"publisher","first-page":"1391","DOI":"10.1007\/s00330-018-5695-5","volume":"29","author":"L Joskowicz","year":"2019","unstructured":"Joskowicz, L., Cohen, D., Caplan, N. & Sosna, J. Inter-observer variability of manual contour delineation of structures in CT. Eur. Radiol. 29, 1391\u20131399, https:\/\/doi.org\/10.1007\/s00330-018-5695-5 (2019).","journal-title":"Eur. Radiol."},{"key":"882_CR19","doi-asserted-by":"publisher","DOI":"10.7303\/syn21898456","author":"S Bodenstedt","year":"2020","unstructured":"Bodenstedt, S. Heidelberg Colorectal (HeiCo) Data Set. Synapse https:\/\/doi.org\/10.7303\/syn21898456 (2020)."},{"key":"882_CR20","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-021-82017-6","volume":"11","author":"M Wiesenfarth","year":"2021","unstructured":"Wiesenfarth, M. et al. Methods and open-source toolkit for analyzing and visualizing challenge results. Sci. Rep. 11, 2369, https:\/\/doi.org\/10.1038\/s41598-021-82017-6 (2021).","journal-title":"Sci. Rep."}],"container-title":["Scientific Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41597-021-00882-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41597-021-00882-2","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41597-021-00882-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,2]],"date-time":"2022-12-02T20:22:18Z","timestamp":1670012538000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41597-021-00882-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,12]]},"references-count":20,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2021,12]]}},"alternative-id":["882"],"URL":"https:\/\/doi.org\/10.1038\/s41597-021-00882-2","relation":{"references":[{"id-type":"doi","id":"10.7303\/syn21903917","asserted-by":"subject"},{"id-type":"doi","id":"10.7303\/syn18779624","asserted-by":"subject"},{"id-type":"doi","id":"10.7303\/syn21898456","asserted-by":"subject"}]},"ISSN":["2052-4463"],"issn-type":[{"value":"2052-4463","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,12]]},"assertion":[{"value":"8 June 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 February 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 April 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"L.M.-H., T.R., A.R., S.B. and S.Sp. worked with device manufacturer KARL STORZ SE & Co. KG in the joint research project \u201cOP4.1,\u201d funded by the German Federal Ministry of Economic Affairs and Energy (Grant number BMWI 01MT17001C). M.W., B.M. and H.G.K. worked with device manufacturer KARL STORZ SE & Co. KG in the joint research project \u201cInnOPlan,\u201d funded by the German Federal Ministry of Economic Affairs and Energy (grant number BMWI 01MD15002E). F.N. reports receiving travel support for conference participation as well as equipment provided for laparoscopic surgery courses by KARL STORZ SE & Co. KG, Johnson & Johnson, Intuitive Surgical, Cambridge Medical Robotics and Medtronic. V.R, F.B, S.Bit. and M.M. are\/were employees of the company understand.ai, which sponsored the initial labeling of the ROBUST-MIS data set. L.M. is employee of the company KARL STORZ SE & Co. KG, which sponsored a scientific prize for the surgical workflow analysis in the sensorOR challenge. The authors not listed as employees of understand.ai or KARL STORZ SE & Co. KG are not and have never been in conflict of interest or financial ties with the company. All other authors have no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"101"}}