{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,4]],"date-time":"2026-11-04T14:26:30Z","timestamp":1793802390710,"version":"4.6.1"},"reference-count":40,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,4,12]],"date-time":"2024-04-12T00:00:00Z","timestamp":1712880000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,4,12]],"date-time":"2024-04-12T00:00:00Z","timestamp":1712880000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Data"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>In recent years, the landscape of computer-assisted interventions and post-operative surgical video analysis has been dramatically reshaped by deep-learning techniques, resulting in significant advancements in surgeons\u2019 skills, operation room management, and overall surgical outcomes. However, the progression of deep-learning-powered surgical technologies is profoundly reliant on large-scale datasets and annotations. In particular, surgical scene understanding and phase recognition stand as pivotal pillars within the realm of computer-assisted surgery and post-operative assessment of cataract surgery videos. In this context, we present the largest cataract surgery video dataset that addresses diverse requisites for constructing computerized surgical workflow analysis and detecting post-operative irregularities in cataract surgery. We validate the quality of annotations by benchmarking the performance of several state-of-the-art neural network architectures for phase recognition and surgical scene segmentation. Besides, we initiate the research on domain adaptation for instrument segmentation in cataract surgery by evaluating cross-domain instrument segmentation performance in cataract surgery videos. The dataset and annotations are publicly available in Synapse.<\/jats:p>","DOI":"10.1038\/s41597-024-03193-4","type":"journal-article","created":{"date-parts":[[2024,4,12]],"date-time":"2024-04-12T02:02:21Z","timestamp":1712887341000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":35,"title":["Cataract-1K Dataset for Deep-Learning-Assisted Analysis of Cataract Surgery Videos"],"prefix":"10.1038","volume":"11","author":[{"given":"Negin","family":"Ghamsarian","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yosuf","family":"El-Shabrawi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sahar","family":"Nasirihaghighi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Doris","family":"Putzgruber-Adamitsch","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3447-2359","authenticated-orcid":false,"given":"Martin","family":"Zinkernagel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7467-7028","authenticated-orcid":false,"given":"Sebastian","family":"Wolf","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9218-1704","authenticated-orcid":false,"given":"Klaus","family":"Schoeffmann","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Raphael","family":"Sznitman","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,4,12]]},"reference":[{"key":"3193_CR1","doi-asserted-by":"publisher","first-page":"336","DOI":"10.1109\/TMI.2022.3180078","volume":"42","author":"L Ma","year":"2023","unstructured":"Ma, L. et al. Simulation of postoperative facial appearances via geometric deep learning for efficient orthognathic surgical planning. IEEE Transactions on Medical Imaging 42, 336\u2013345, https:\/\/doi.org\/10.1109\/TMI.2022.3180078 (2023).","journal-title":"IEEE Transactions on Medical Imaging"},{"key":"3193_CR2","doi-asserted-by":"crossref","unstructured":"Quon, J. et al. Deep learning for automated delineation of pediatric cerebral arteries on pre-operative brain magnetic resonance imaging. front surg 2020; 7 (2020).","DOI":"10.3389\/fsurg.2020.517375"},{"key":"3193_CR3","doi-asserted-by":"publisher","first-page":"2958","DOI":"10.1109\/JBHI.2021.3054494","volume":"25","author":"D Xiao","year":"2021","unstructured":"Xiao, D. et al. Estimating reference bony shape models for orthognathic surgical planning using 3d point-cloud deep learning. IEEE Journal of Biomedical and Health Informatics 25, 2958\u20132966, https:\/\/doi.org\/10.1109\/JBHI.2021.3054494 (2021).","journal-title":"IEEE Journal of Biomedical and Health Informatics"},{"key":"3193_CR4","doi-asserted-by":"publisher","first-page":"159","DOI":"10.1177\/15485129211034586","volume":"19","author":"E Yanik","year":"2022","unstructured":"Yanik, E. et al. Deep neural networks for the assessment of surgical skills: A systematic review. The Journal of Defense Modeling and Simulation 19, 159\u2013171 (2022).","journal-title":"The Journal of Defense Modeling and Simulation"},{"key":"3193_CR5","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-022-00566-0","volume":"5","author":"K Lam","year":"2022","unstructured":"Lam, K. et al. Machine learning for technical skill assessment in surgery: a systematic review. NPJ digital medicine 5, 24 (2022).","journal-title":"NPJ digital medicine"},{"key":"3193_CR6","doi-asserted-by":"publisher","first-page":"1959","DOI":"10.1007\/s11548-018-1860-1","volume":"13","author":"Z Wang","year":"2018","unstructured":"Wang, Z. & Majewicz Fey, A. Deep learning with convolutional neural network for objective skill evaluation in robot-assisted surgery. International journal of computer assisted radiology and surgery 13, 1959\u20131970 (2018).","journal-title":"International journal of computer assisted radiology and surgery"},{"key":"3193_CR7","doi-asserted-by":"crossref","unstructured":"Wang, Z. & Fey, A. M. Satr-dl: improving surgical skill assessment and task recognition in robot-assisted surgery with deep neural networks. In 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 1793\u20131796 (IEEE, 2018).","DOI":"10.1109\/EMBC.2018.8512575"},{"key":"3193_CR8","doi-asserted-by":"crossref","unstructured":"Soleymani, A. et al. Surgical skill evaluation from robot-assisted surgery recordings. In 2021 International Symposium on Medical Robotics (ISMR), 1\u20136 (IEEE, 2021).","DOI":"10.1109\/ISMR48346.2021.9661527"},{"key":"3193_CR9","doi-asserted-by":"crossref","unstructured":"Aksamentov, I., Twinanda, A. P., Mutter, D., Marescaux, J. & Padoy, N. Deep neural networks predict remaining surgery duration from cholecystectomy videos. In Medical Image Computing and Computer-Assisted Intervention- MICCAI 2017: 20th International Conference, Quebec City, QC, Canada, September 11-13, 2017, Proceedings, Part II 20, 586\u2013593 (Springer, 2017).","DOI":"10.1007\/978-3-319-66185-8_66"},{"key":"3193_CR10","doi-asserted-by":"publisher","first-page":"1069","DOI":"10.1109\/TMI.2018.2878055","volume":"38","author":"AP Twinanda","year":"2018","unstructured":"Twinanda, A. P., Yengera, G., Mutter, D., Marescaux, J. & Padoy, N. Rsdnet: Learning to predict remaining surgery duration from laparoscopic videos without manual annotations. IEEE transactions on medical imaging 38, 1069\u20131078 (2018).","journal-title":"IEEE transactions on medical imaging"},{"key":"3193_CR11","doi-asserted-by":"crossref","unstructured":"Marafioti, A. et al. Catanet: predicting remaining cataract surgery duration. In Medical Image Computing and Computer Assisted Intervention\u2013MICCAI 2021: 24th International Conference, Strasbourg, France, September 27\u2013October 1, 2021, Proceedings, Part IV 24, 426\u2013435 (Springer, 2021).","DOI":"10.1007\/978-3-030-87202-1_41"},{"key":"3193_CR12","unstructured":"Ghamsarian, N. Deep-learning-assisted analysis of cataract surgery videos, (2021)."},{"key":"3193_CR13","doi-asserted-by":"publisher","unstructured":"Ghamsarian, N. Enabling relevance-based exploration of cataract videos. In Proceedings of the 2020 International Conference on Multimedia Retrieval, ICMR\u201920, 378\u2013382, https:\/\/doi.org\/10.1145\/3372278.3391937 (2020).","DOI":"10.1145\/3372278.3391937"},{"key":"3193_CR14","doi-asserted-by":"publisher","first-page":"e489","DOI":"10.1016\/S2214-109X(20)30488-5","volume":"9","author":"MJ Burton","year":"2021","unstructured":"Burton, M. J. et al. The lancet global health commission on global eye health: vision beyond 2020. The Lancet Global Health 9, e489\u2013e551 (2021).","journal-title":"The Lancet Global Health"},{"key":"3193_CR15","doi-asserted-by":"crossref","unstructured":"Ghamsarian, N., Taschwer, M., Putzgruber-Adamitsch, D., Sarny, S. & Schoeffmann, K. Relevance detection in cataract surgery videos by spatio- temporal action localization. In 2020 25th International Conference on Pattern Recognition (ICPR), 10720\u201310727 (2021).","DOI":"10.1109\/ICPR48806.2021.9412525"},{"key":"3193_CR16","doi-asserted-by":"crossref","unstructured":"Ghamsarian, N., Amirpourazarian, H., Timmerer, C., Taschwer, M. & Sch\u00f6ffmann, K. Relevance-based compression of cataract surgery videos using convolutional neural networks. In Proceedings of the 28th ACM International Conference on Multimedia, 3577\u20133585 (2020).","DOI":"10.1145\/3394171.3413658"},{"key":"3193_CR17","doi-asserted-by":"crossref","unstructured":"Ghamsarian, N. et al. Lensid: A cnn-rnn-based framework towards lens irregularity detection in cataract surgery videos. In de Bruijne, M. et al. (eds.) Medical Image Computing and Computer Assisted Intervention \u2013 MICCAI 2021, 76\u201386 (Springer International Publishing, Cham, 2021).","DOI":"10.1007\/978-3-030-87237-3_8"},{"key":"3193_CR18","doi-asserted-by":"publisher","first-page":"e0258390","DOI":"10.1371\/journal.pone.0258390","volume":"16","author":"N Sokolova","year":"2021","unstructured":"Sokolova, N. et al. Automatic detection of pupil reactions in cataract surgery videos. Plos one 16, e0258390 (2021).","journal-title":"Plos one"},{"key":"3193_CR19","doi-asserted-by":"publisher","first-page":"24","DOI":"10.1016\/j.media.2018.11.008","volume":"52","author":"H Al Hajj","year":"2019","unstructured":"Al Hajj, H. et al. Cataracts: Challenge on automatic tool annotation for cataract surgery. Medical image analysis 52, 24\u201341 (2019).","journal-title":"Medical image analysis"},{"key":"3193_CR20","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2021.102053","volume":"71","author":"M Grammatikopoulou","year":"2021","unstructured":"Grammatikopoulou, M. et al. Cadis: Cataract dataset for surgical rgb-image segmentation. Medical Image Analysis 71, 102053 (2021).","journal-title":"Medical Image Analysis"},{"key":"3193_CR21","doi-asserted-by":"crossref","unstructured":"Ghamsarian, N. et al. Recal-net: Joint region-channel-wise calibrated network for semantic segmentation in cataract surgery videos. In Mantoro, T., Lee, M., Ayu, M. A., Wong, K. W. & Hidayanto, A. N. (eds.) Neural Information Processing, 391\u2013402 (Springer International Publishing, Cham, 2021).","DOI":"10.1007\/978-3-030-92238-2_33"},{"key":"3193_CR22","doi-asserted-by":"crossref","unstructured":"Ghamsarian, N., Taschwer, M., Sznitman, R. & Schoeffmann, K. Deeppyramid: Enabling pyramid view and deformable pyramid reception for semantic segmentation in cataract surgery videos. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 276\u2013286 (Springer, 2022).","DOI":"10.1007\/978-3-031-16443-9_27"},{"key":"3193_CR23","doi-asserted-by":"crossref","unstructured":"Ghamsarian, N., Wolf, S., Zinkernagel, M., Schoeffmann, K. & Sznitman, R. Deeppyramid+: medical image segmentation using pyramid view fusion and deformable pyramid reception. International journal of computer assisted radiology and surgery 1\u20139 (2024).","DOI":"10.1007\/s11548-023-03046-2"},{"key":"3193_CR24","unstructured":"Ethikkommission k\u00e4rnten. https:\/\/www.ethikkommission-kaernten.at\/ueber-uns\/kommission"},{"key":"3193_CR25","doi-asserted-by":"crossref","unstructured":"Ghamsarian, N., Taschwer, M. & Schoeffmann, K. Deblurring cataract surgery videos using a multi-scale deconvolutional neural network. In 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), 872\u2013876 (2020).","DOI":"10.1109\/ISBI45749.2020.9098318"},{"key":"3193_CR26","doi-asserted-by":"publisher","first-page":"194","DOI":"10.1038\/sj.eye.6700268","volume":"17","author":"SA Mirza","year":"2003","unstructured":"Mirza, S. A., Alexandridou, A., Marshall, T. & Stavrou, P. Surgically induced miosis during phacoemulsification in patients with diabetes mellitus. Eye 17, 194\u2013199, https:\/\/doi.org\/10.1038\/sj.eye.6700268 (2003).","journal-title":"Eye"},{"key":"3193_CR27","doi-asserted-by":"publisher","DOI":"10.1186\/s12886-020-01731-3","volume":"20","author":"T Oshika","year":"2020","unstructured":"Oshika, T. et al. Prospective assessment of plate-haptic rotationally asymmetric multifocal toric intraocular lens with near addition of +1.5 diopters. BMC Ophthalmology 20, 454, https:\/\/doi.org\/10.1186\/s12886-020-01731-3 (2020).","journal-title":"BMC Ophthalmology"},{"key":"3193_CR28","doi-asserted-by":"publisher","unstructured":"Ghamsarian, N. et al. Predicting postoperative intraocular lens dislocation in cataract surgery via deep learning. IEEE Access 1\u20131, https:\/\/doi.org\/10.1109\/ACCESS.2024.3361042 (2024).","DOI":"10.1109\/ACCESS.2024.3361042"},{"key":"3193_CR29","doi-asserted-by":"publisher","unstructured":"Nasirihaghighi, S., Ghamsarian, N., Stefanics, D., Schoeffmann, K. & Husslein, H. Action recognition in video recordings from gynecologic laparoscopy. In 2023 IEEE 36th International Symposium on Computer-Based Medical Systems (CBMS), 29\u201334, https:\/\/doi.org\/10.1109\/CBMS58004.2023.00187 (2023).","DOI":"10.1109\/CBMS58004.2023.00187"},{"key":"3193_CR30","doi-asserted-by":"crossref","unstructured":"Deng, J. et al. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, 248\u2013255 (Ieee, 2009).","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"3193_CR31","doi-asserted-by":"publisher","DOI":"10.7303\/syn52540135","author":"N Ghamsarian","year":"2024","unstructured":"Ghamsarian, N. et al. Cataract-1k. Synapse https:\/\/doi.org\/10.7303\/syn52540135 (2024)."},{"key":"3193_CR32","doi-asserted-by":"crossref","unstructured":"Ghamsarian, N. et al. Domain adaptation for medical image segmentation using transformation-invariant self-training. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 331\u2013341 (Springer, 2023).","DOI":"10.1007\/978-3-031-43907-0_32"},{"key":"3193_CR33","doi-asserted-by":"publisher","first-page":"1856","DOI":"10.1109\/TMI.2019.2959609","volume":"39","author":"Z Zhou","year":"2020","unstructured":"Zhou, Z., Siddiquee, M. M. R., Tajbakhsh, N. & Liang, J. Unet++: Redesigning skip connections to exploit multiscale features in image segmentation. IEEE Transactions on Medical Imaging 39, 1856\u20131867, https:\/\/doi.org\/10.1109\/TMI.2019.2959609 (2020).","journal-title":"IEEE Transactions on Medical Imaging"},{"key":"3193_CR34","doi-asserted-by":"publisher","first-page":"3008","DOI":"10.1109\/TMI.2020.2983721","volume":"39","author":"S Feng","year":"2020","unstructured":"Feng, S. et al. Cpfnet: Context pyramid fusion network for medical image segmentation. IEEE Transactions on Medical Imaging 39, 3008\u20133018, https:\/\/doi.org\/10.1109\/TMI.2020.2983721 (2020).","journal-title":"IEEE Transactions on Medical Imaging"},{"key":"3193_CR35","doi-asserted-by":"publisher","first-page":"2281","DOI":"10.1109\/TMI.2019.2903562","volume":"38","author":"Z Gu","year":"2019","unstructured":"Gu, Z. et al. Ce-net: Context encoder network for 2d medical image segmentation. IEEE Transactions on Medical Imaging 38, 2281\u20132292, https:\/\/doi.org\/10.1109\/TMI.2019.2903562 (2019).","journal-title":"IEEE Transactions on Medical Imaging"},{"key":"3193_CR36","doi-asserted-by":"publisher","unstructured":"Chen, X., Zhang, R. & Yan, P. Feature fusion encoder decoder network for automatic liver lesion segmentation. In 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), 430\u2013433, https:\/\/doi.org\/10.1109\/ISBI.2019.8759555 (2019).","DOI":"10.1109\/ISBI.2019.8759555"},{"key":"3193_CR37","doi-asserted-by":"publisher","first-page":"540","DOI":"10.1109\/TMI.2018.2867261","volume":"38","author":"AG Roy","year":"2019","unstructured":"Roy, A. G., Navab, N. & Wachinger, C. Recalibrating fully convolutional networks with spatial and channel \u201csqueeze and excitation\u201d blocks. IEEE Transactions on Medical Imaging 38, 540\u2013549 (2019).","journal-title":"IEEE Transactions on Medical Imaging"},{"key":"3193_CR38","doi-asserted-by":"crossref","unstructured":"Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F. & Adam, H. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European conference on computer vision (ECCV), 801\u2013818 (2018).","DOI":"10.1007\/978-3-030-01234-2_49"},{"key":"3193_CR39","doi-asserted-by":"crossref","unstructured":"Xiao, T., Liu, Y., Zhou, B., Jiang, Y. & Sun, J. Unified perceptual parsing for scene understanding. In Proceedings of the European conference on computer vision (ECCV), 418\u2013434 (2018).","DOI":"10.1007\/978-3-030-01228-1_26"},{"key":"3193_CR40","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P. & Brox, T. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention \u2013 MICCAI 2015, 234\u2013241 (2015).","DOI":"10.1007\/978-3-319-24574-4_28"}],"container-title":["Scientific Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41597-024-03193-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41597-024-03193-4","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41597-024-03193-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,12]],"date-time":"2024-04-12T02:04:35Z","timestamp":1712887475000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41597-024-03193-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,12]]},"references-count":40,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["3193"],"URL":"https:\/\/doi.org\/10.1038\/s41597-024-03193-4","relation":{"references":[{"id-type":"doi","id":"10.7303\/syn52540135","asserted-by":"subject"}]},"ISSN":["2052-4463"],"issn-type":[{"value":"2052-4463","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,12]]},"assertion":[{"value":"8 November 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 March 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 April 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"373"}}