{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,26]],"date-time":"2026-10-26T22:56:32Z","timestamp":1793055392126,"version":"4.5.0"},"reference-count":38,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2023,11,7]],"date-time":"2023-11-07T00:00:00Z","timestamp":1699315200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,11,7]],"date-time":"2023-11-07T00:00:00Z","timestamp":1699315200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100010661","name":"Horizon 2020 Framework Programme","doi-asserted-by":"publisher","award":["857533"],"award-info":[{"award-number":["857533"]}],"id":[{"id":"10.13039\/100010661","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J CARS"],"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Purpose<\/jats:title>\n                <jats:p>Computer-assisted surgical systems provide support information to the surgeon, which can improve the execution and overall outcome of the procedure. These systems are based on deep learning models that are trained on complex and challenging-to-annotate data. Generating synthetic data can overcome these limitations, but it is necessary to reduce the domain gap between real and synthetic data.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>We propose a method for image-to-image translation based on a Stable Diffusion model, which generates realistic images starting from synthetic data. Compared to previous works, the proposed method is better suited for clinical application as it requires a much smaller amount of input data and allows finer control over the generation of details by introducing different variants of supporting control networks.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>The proposed method is applied in the context of laparoscopic cholecystectomy, using synthetic and real data from public datasets. It achieves a mean Intersection over Union of 69.76%, significantly improving the baseline results (69.76 vs. 42.21%).<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusions<\/jats:title>\n                <jats:p>The proposed method for translating synthetic images into images with realistic characteristics will enable the training of deep learning methods that can generalize optimally to real-world contexts, thereby improving computer-assisted intervention guidance systems.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1007\/s11548-023-03030-w","type":"journal-article","created":{"date-parts":[[2023,11,7]],"date-time":"2023-11-07T07:01:52Z","timestamp":1699340512000},"page":"531-539","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":29,"title":["Minimal data requirement for realistic endoscopic image generation with Stable Diffusion"],"prefix":"10.1007","volume":"19","author":[{"given":"Joanna","family":"Kaleta","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7300-5975","authenticated-orcid":false,"given":"Diego","family":"Dall\u2019Alba","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Szymon","family":"P\u0142otka","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Przemys\u0142aw","family":"Korzeniowski","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,11,7]]},"reference":[{"key":"3030_CR1","unstructured":"Binkowski M, Sutherland DJ, Arbel M, Gretton A (2018) Demystifying mmd gans. In: International conference on learning representations"},{"key":"3030_CR2","doi-asserted-by":"publisher","first-page":"105382","DOI":"10.1016\/j.compbiomed.2022.105382","volume":"144","author":"Y Chen","year":"2022","unstructured":"Chen Y, Yang X-H, Wei Z, Heidari AA, Zheng N, Li Z, Chen H, Hu H, Zhou Q, Guan Q (2022) Generative adversarial networks in medical image augmentation: a review. Comput Biol Med 144:105382","journal-title":"Comput Biol Med"},{"key":"3030_CR3","first-page":"8780","volume":"34","author":"P Dhariwal","year":"2021","unstructured":"Dhariwal P, Nichol A (2021) Diffusion models beat gans on image synthesis. Adv Neural Inf Process Syst 34:8780\u20138794","journal-title":"Adv Neural Inf Process Syst"},{"issue":"5","key":"3030_CR4","doi-asserted-by":"publisher","first-page":"961","DOI":"10.1007\/s11548-022-02598-z","volume":"17","author":"T Dowrick","year":"2022","unstructured":"Dowrick T, Davidson B, Gurusamy K, Clarkson MJ (2022) Large scale simulation of labeled intraoperative scenes in unity. Int J Comput Assist Radiol Surg 17(5):961\u2013963","journal-title":"Int J Comput Assist Radiol Surg"},{"key":"3030_CR5","unstructured":"Gal R, Alaluf Y, Atzmon Y, Patashnik O, Bermano AH, Chechik G, Cohen-Or D (2022) An image is worth one word: Personalizing text-to-image generation using textual inversion"},{"key":"3030_CR6","unstructured":"Heusel M, Ramsauer H, Unterthiner T, Nessler B, Hochreiter S (2017) Gans trained by a two time-scale update rule converge to a local nash equilibrium. In: Proceedings of the 31st international conference on neural information processing systems, NIPS\u201917, pp 6629\u20136640"},{"key":"3030_CR7","unstructured":"Hong W-Y, Kao C-L, Kuo Y-H, Wang J-R, Chang W-L, Shih C-S (2020) Cholecseg8k: a semantic segmentation dataset for laparoscopic cholecystectomy based on cholec80. arXiv preprint arXiv:2012.12453"},{"key":"3030_CR8","unstructured":"Hu EJ, Shen Y, Wallis P, Allen-Zhu Z, Li Y, Wang S, Wang L, Chen W (2022) LoRA: Low-rank adaptation of large language models. In: International conference on learning representations"},{"key":"3030_CR9","doi-asserted-by":"publisher","first-page":"102846","DOI":"10.1016\/j.media.2023.102846","volume":"88","author":"A Kazerouni","year":"2023","unstructured":"Kazerouni A, Aghdam EK, Heidari M, Azad R, Fayyaz M, Hacihaliloglu I, Merhof D (2023) Diffusion models in medical imaging: a comprehensive survey. Med Image Anal 88:102846","journal-title":"Med Image Anal"},{"key":"3030_CR10","doi-asserted-by":"crossref","unstructured":"Korzeniowski P, P\u0142otka S, Brawura-Biskupski-Samaha R, Sitek A (2022) Virtual reality simulator for fetoscopic spina bifida repair surgery. In: 2022 IEEE\/RSJ international conference on intelligent robots and systems (IROS), pages 401\u2013406. IEEE","DOI":"10.1109\/IROS47612.2022.9981920"},{"key":"3030_CR11","doi-asserted-by":"crossref","unstructured":"Kumari N, Zhang B, Zhang R, Shechtman E, Zhu J-Y (June 2023) Multi-concept customization of text-to-image diffusion. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (CVPR), pp 1931\u20131941","DOI":"10.1109\/CVPR52729.2023.00192"},{"key":"3030_CR12","doi-asserted-by":"crossref","unstructured":"Lee H-Y, Tseng H-Y, Huang J-B, Singh M, Yang M-H (September 2018) Diverse image-to-image translation via disentangled representations. In: Proceedings of the European conference on computer vision (ECCV)","DOI":"10.1007\/978-3-030-01246-5_3"},{"key":"3030_CR13","doi-asserted-by":"crossref","unstructured":"Lin S, Qin F, Li Y, Bly RA, Moe KS, Hannaford B (2020) Lc-gan: Image-to-image translation based on generative adversarial network for endoscopic images. In: 2020 IEEE\/rsj international conference on intelligent robots and systems (IROS), pp 2914\u20132920. IEEE","DOI":"10.1109\/IROS45743.2020.9341556"},{"issue":"2","key":"3030_CR14","doi-asserted-by":"publisher","first-page":"363","DOI":"10.1097\/SLA.0000000000004594","volume":"276","author":"A Madani","year":"2022","unstructured":"Madani A, Namazi B, Altieri MS, Hashimoto DA, Rivera AM, Pucher PH, Navarrete-Welton A, Sankaranarayanan G, Brunt LM, Okrainec A, Alseidi A (2022) Artificial intelligence for intraoperative guidance: using semantic segmentation to identify surgical anatomy during laparoscopic cholecystectomy. Ann Surg 276(2):363\u2013369","journal-title":"Ann Surg"},{"key":"3030_CR15","doi-asserted-by":"publisher","first-page":"102306","DOI":"10.1016\/j.media.2021.102306","volume":"76","author":"L Maier-Hein","year":"2022","unstructured":"Maier-Hein L, Eisenmann M, Sarikaya D, M\u00e4rz K, Collins T, Malpani A, Fallert J, Feussner H, Giannarou S, Mascagni P et al (2022) Surgical data science-from concepts toward clinical translation. Med Image Anal 76:102306","journal-title":"Med Image Anal"},{"key":"3030_CR16","doi-asserted-by":"publisher","first-page":"105834","DOI":"10.1016\/j.cmpb.2020.105834","volume":"200","author":"A Marzullo","year":"2021","unstructured":"Marzullo A, Moccia S, Catellani M, Calimeri F, De Momi E (2021) Towards realistic laparoscopic image generation using image-domain translation. Comput Methods Programs Biomed 200:105834","journal-title":"Comput Methods Programs Biomed"},{"key":"3030_CR17","doi-asserted-by":"crossref","unstructured":"Moghadam PA, Van\u00a0Dalen S, Martin KC, Lennerz J, Yip S, Farahani H, Bashashati A (2023) A morphology focused diffusion probabilistic model for synthesis of histopathology images. In: Proceedings of the IEEE\/CVF winter conference on applications of computer vision, pp 2000\u20132009","DOI":"10.1109\/WACV56688.2023.00204"},{"key":"3030_CR18","doi-asserted-by":"crossref","unstructured":"Nwoye CI, Gonzalez C, Yu T, Mascagni P, Mutter D, Marescaux J, Padoy N (2020) Recognition of instrument-tissue interactions in endoscopic videos via action triplets. In: Medical Image Computing and Computer Assisted Intervention \u2013 MICCAI 2020, pages 364\u2013374","DOI":"10.1007\/978-3-030-59716-0_35"},{"key":"3030_CR19","doi-asserted-by":"publisher","first-page":"102433","DOI":"10.1016\/j.media.2022.102433","volume":"78","author":"CI Nwoye","year":"2022","unstructured":"Nwoye CI, Yu T, Gonzalez C, Seeliger B, Mascagni P, Mutter D, Marescaux J, Padoy N (2022) Rendezvous: attention mechanisms for the recognition of surgical action triplets in endoscopic videos. Med Image Anal 78:102433","journal-title":"Med Image Anal"},{"issue":"6","key":"3030_CR20","doi-asserted-by":"publisher","first-page":"214","DOI":"10.1049\/htl.2019.0071","volume":"6","author":"M Oda","year":"2019","unstructured":"Oda M, Tanaka K, Takabatake H, Mori M, Natori H, Mori K (2019) Realistic endoscopic image generation method using virtual-to-real image-domain translation. Healthcare Technol Lett 6(6):214\u2013219","journal-title":"Healthcare Technol Lett"},{"key":"3030_CR21","doi-asserted-by":"crossref","unstructured":"\u00d6zbey M, Dalmaz O, Dar SU, Bedel HA, \u00d6zturk \u015e, G\u00fcng\u00f6r A, \u00c7ukur T (2023) Unsupervised medical image translation with adversarial diffusion models. IEEE Trans Med Imaging","DOI":"10.1109\/TMI.2023.3290149"},{"key":"3030_CR22","doi-asserted-by":"crossref","unstructured":"Pfeiffer M, Funke I, Robu MR, Bodenstedt S, Strenger L, Engelhardt S, Ro\u00df T, Clarkson MJ, Gurusamy K, Davidson BR, et\u00a0al. (2019) Generating large labeled data sets for laparoscopic image processing tasks using unpaired image-to-image translation. In: Medical Image Computing and Computer Assisted Intervention\u2013MICCAI 2019: 22nd International Conference, pages 119\u2013127. Springer","DOI":"10.1007\/978-3-030-32254-0_14"},{"key":"3030_CR23","doi-asserted-by":"crossref","unstructured":"Rivoir D, Pfeiffer M, Docea R, Kolbinger F, Riediger C, Weitz J, Speidel S (2021) Long-term temporally consistent unpaired video translation from simulated surgical 3d data. In: 2021 IEEE\/CVF international conference on computer vision (ICCV), pp 3323\u20133333","DOI":"10.1109\/ICCV48922.2021.00333"},{"key":"3030_CR24","doi-asserted-by":"crossref","unstructured":"Rombach R, Blattmann A, Lorenz D, Esser P, Ommer B (2022) High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE\/CVF Conference on computer vision and pattern recognition, pp 10684\u201310695","DOI":"10.1109\/CVPR52688.2022.01042"},{"key":"3030_CR25","doi-asserted-by":"crossref","unstructured":"Ruiz N, Li Y, Jampani V, Pritch Y, Rubinstein M, Aberman K (June 2023) Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In: Proceedings of the IEEE\/CVF Conference on computer vision and pattern recognition (CVPR), pp 22500\u201322510","DOI":"10.1109\/CVPR52729.2023.02155"},{"issue":"5","key":"3030_CR26","doi-asserted-by":"publisher","first-page":"849","DOI":"10.1007\/s11548-021-02383-4","volume":"16","author":"M Sahu","year":"2021","unstructured":"Sahu M, Mukhopadhyay A, Zachow S (2021) Simulation-to-real domain adaptation with teacher-student learning for endoscopic instrument segmentation. Int J Comput Assist Radiol Surg 16(5):849\u2013859","journal-title":"Int J Comput Assist Radiol Surg"},{"key":"3030_CR27","doi-asserted-by":"crossref","unstructured":"Sahu M, Str\u00f6msd\u00f6rfer R, Mukhopadhyay A, Zachow S (2020) Endo-sim2real: Consistency learning-based domain adaptation for instrument segmentation. In: International conference on medical image computing and computer-assisted intervention, pp 784\u2013794. Springer","DOI":"10.1007\/978-3-030-59716-0_75"},{"issue":"2","key":"3030_CR28","doi-asserted-by":"publisher","first-page":"560","DOI":"10.1109\/LRA.2022.3227873","volume":"8","author":"PM Scheikl","year":"2022","unstructured":"Scheikl PM, Tagliabue E, Gyenes B, Wagner M, Dall\u2019Alba D, Fiorini P, Mathis-Ullrich F (2022) Sim-to-real transfer for visual reinforcement learning of deformable object manipulation for robot-assisted surgery. IEEE Robot Automation Lett 8(2):560\u2013567","journal-title":"IEEE Robot Automation Lett"},{"issue":"1","key":"3030_CR29","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1109\/JBHI.2021.3099858","volume":"26","author":"L Sharan","year":"2021","unstructured":"Sharan L, Romano G, Koehler S, Kelm H, Karck M, De Simone R, Engelhardt S (2021) Mutually improved endoscopic image synthesis and landmark detection in unpaired image-to-image translation. IEEE J Biomed Health Inform 26(1):127\u2013138","journal-title":"IEEE J Biomed Health Inform"},{"key":"3030_CR30","doi-asserted-by":"crossref","unstructured":"Su Z, Liu W, Yu Z, Hu D, Liao Q, Tian Q, Pietikainen M, Liu L (2021) Pixel difference networks for efficient edge detection. In: Proceedings of the IEEE\/CVF international conference on computer vision, pp 5117\u20135127","DOI":"10.1109\/ICCV48922.2021.00507"},{"key":"3030_CR31","doi-asserted-by":"crossref","unstructured":"Tokuyasu T, Iwashita Y, Matsunobu Y, Kamiyama T, Ishikake M, Sakaguchi S, Ebe K, Tada K, Endo Y, Etoh T et al (2021) Development of an artificial intelligence system using deep learning to indicate anatomical landmarks during laparoscopic cholecystectomy. Surg Endosc 35:1651\u20131658","DOI":"10.1007\/s00464-020-07548-x"},{"key":"3030_CR32","doi-asserted-by":"crossref","unstructured":"Twinanda AP, Shehata S, Mutter D, Marescaux J, De Mathelin M, Padoy N (2016) Endonet: a deep architecture for recognition tasks on laparoscopic videos. IEEE Trans Med Imaging 36(1):86\u201397","DOI":"10.1109\/TMI.2016.2593957"},{"key":"3030_CR33","doi-asserted-by":"crossref","unstructured":"Wang Z, Zhao L, Chen H, Qiu L, Mo Q, Lin S, Xing W, Lu D (2020) Diversified arbitrary style transfer via deep feature perturbation. In: 2020 IEEE\/CVF conference on computer vision and pattern recognition (CVPR), pp 7786\u20137795, Los Alamitos, CA, USA. IEEE Computer Society","DOI":"10.1109\/CVPR42600.2020.00781"},{"key":"3030_CR34","doi-asserted-by":"crossref","unstructured":"Xie S, Tu Z (2015) Holistically-nested edge detection. In: Proceedings of the IEEE international conference on computer vision, pp 1395\u20131403","DOI":"10.1109\/ICCV.2015.164"},{"issue":"4","key":"3030_CR35","doi-asserted-by":"publisher","first-page":"785","DOI":"10.1007\/s11548-022-02811-z","volume":"18","author":"B Zhang","year":"2023","unstructured":"Zhang B, Goel B, Sarhan MH, Goel VK, Abukhalil R, Kalesan B, Stottler N, Petculescu S (2023) Surgical workflow recognition with temporal convolution and transformer for action segmentation. Int J Comput Assist Radiol Surg 18(4):785\u2013794","journal-title":"Int J Comput Assist Radiol Surg"},{"key":"3030_CR36","doi-asserted-by":"crossref","unstructured":"Zhang L, Agrawala M (2023) Adding conditional control to text-to-image diffusion models. arXiv preprint arXiv:2302.05543","DOI":"10.1109\/ICCV51070.2023.00355"},{"key":"3030_CR37","doi-asserted-by":"crossref","unstructured":"Zhang R, Isola P, Efros AA, Shechtman E, Wang O (2018) The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586\u2013595","DOI":"10.1109\/CVPR.2018.00068"},{"key":"3030_CR38","unstructured":"Zhu J-Y, Zhang R, Pathak D, Darrell T, Efros AA, Wang O, Shechtman E (2017) Toward multimodal image-to-image translation. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS\u201917, pp 465\u2013476, Red Hook, NY, USA. Curran Associates Inc"}],"container-title":["International Journal of Computer Assisted Radiology and Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-023-03030-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11548-023-03030-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-023-03030-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,2,21]],"date-time":"2024-02-21T14:19:58Z","timestamp":1708525198000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11548-023-03030-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,7]]},"references-count":38,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2024,3]]}},"alternative-id":["3030"],"URL":"https:\/\/doi.org\/10.1007\/s11548-023-03030-w","relation":{},"ISSN":["1861-6429"],"issn-type":[{"value":"1861-6429","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,7]]},"assertion":[{"value":"16 July 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 October 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 November 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"All authors declare no conflict of interest. The data in this work are publicly available and have been approved and performed in accordance with ethical standards.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}