{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T02:13:39Z","timestamp":1760235219301,"version":"build-2065373602"},"reference-count":106,"publisher":"MDPI AG","issue":"15","license":[{"start":{"date-parts":[[2021,7,30]],"date-time":"2021-07-30T00:00:00Z","timestamp":1627603200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Science Foundation","award":["IIS-2101107"],"award-info":[{"award-number":["IIS-2101107"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Accurate semantic image segmentation from medical imaging can enable intelligent vision-based assistance in robot-assisted minimally invasive surgery. The human body and surgical procedures are highly dynamic. While machine-vision presents a promising approach, sufficiently large training image sets for robust performance are either costly or unavailable. This work examines three novel generative adversarial network (GAN) methods of providing usable synthetic tool images using only surgical background images and a few real tool images. The best of these three novel approaches generates realistic tool textures while preserving local background content by incorporating both a style preservation and a content loss component into the proposed multi-level loss function. The approach is quantitatively evaluated, and results suggest that the synthetically generated training tool images enhance UNet tool segmentation performance. More specifically, with a random set of 100 cadaver and live endoscopic images from the University of Washington Sinus Dataset, the UNet trained with synthetically generated images using the presented method resulted in 35.7% and 30.6% improvement over using purely real images in mean Dice coefficient and Intersection over Union scores, respectively. This study is promising towards the use of more widely available and routine screening endoscopy to preoperatively generate synthetic training tool images for intraoperative UNet tool segmentation.<\/jats:p>","DOI":"10.3390\/s21155163","type":"journal-article","created":{"date-parts":[[2021,8,1]],"date-time":"2021-08-01T21:44:32Z","timestamp":1627854272000},"page":"5163","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["Local Style Preservation in Improved GAN-Driven Synthetic Image Generation for Endoscopic Tool Segmentation"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9122-7264","authenticated-orcid":false,"given":"Yun-Hsuan","family":"Su","sequence":"first","affiliation":[{"name":"Department of Computer Science, Mount Holyoke College, 50 College Street, South Hadley, MA 01075, USA"}]},{"given":"Wenfan","family":"Jiang","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Mount Holyoke College, 50 College Street, South Hadley, MA 01075, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7021-7569","authenticated-orcid":false,"given":"Digesh","family":"Chitrakar","sequence":"additional","affiliation":[{"name":"Department of Engineering, Trinity College, 300 Summit St., Hartford, CT 06106, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0300-9911","authenticated-orcid":false,"given":"Kevin","family":"Huang","sequence":"additional","affiliation":[{"name":"Department of Engineering, Trinity College, 300 Summit St., Hartford, CT 06106, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8909-3343","authenticated-orcid":false,"given":"Haonan","family":"Peng","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Washington, 185 Stevens Way, Paul Allen Center, Seattle, WA 98105, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7370-4920","authenticated-orcid":false,"given":"Blake","family":"Hannaford","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Washington, 185 Stevens Way, Paul Allen Center, Seattle, WA 98105, USA"}]}],"member":"1968","published-online":{"date-parts":[[2021,7,30]]},"reference":[{"key":"ref_1","unstructured":"Delp, S.L., Loan, J.P., Robinson, C.B., Wong, A.Y., and Stulberg, S.D. (1997). Computer-Assisted Surgical System. (5,682,886), U.S. Patent."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Su, Y.H., Lindgren, K., Huang, K., and Hannaford, B. (2020, January 12\u201315). A Comparison of Surgical Cavity 3D Reconstruction Methods. Proceedings of the 2020 IEEE\/SICE International Symposium on System Integration (SII), Honolulu, HI, USA.","DOI":"10.1109\/SII46433.2020.9026289"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Su, Y.H., Huang, K., and Hannaford, B. (2019, January 3\u20135). Multicamera 3d reconstruction of dynamic surgical cavities: Camera grouping and pair sequencing. Proceedings of the 2019 International Symposium on Medical Robotics (ISMR), Atlanta, GA, USA.","DOI":"10.1109\/ISMR.2019.8710190"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Su, Y.H., Huang, K., and Hannaford, B. (2021). Multicamera 3D Viewpoint Adjustment for Robotic Surgery via Deep Reinforcement Learning. J. Med. Robot. Res., 2140003.","DOI":"10.1142\/S2424905X21400031"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"582","DOI":"10.1007\/s10278-019-00227-x","article-title":"Deep learning techniques for medical image segmentation: Achievements and challenges","volume":"32","author":"Hesamian","year":"2019","journal-title":"J. Digit. Imaging"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Colleoni, E., Edwards, P., and Stoyanov, D. (2020). Synthetic and Real Inputs for Tool Segmentation in Robotic Surgery. International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer.","DOI":"10.1007\/978-3-030-59716-0_67"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2027","DOI":"10.3390\/s21062027","article-title":"Computational Methods for Liver Vessel Segmentation in Medical Imaging: A Review","volume":"21","author":"Ciecholewski","year":"2021","journal-title":"Sensors"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"383","DOI":"10.1007\/s12021-018-9377-x","article-title":"Segan: Adversarial network with multi-scale l 1 loss for medical image segmentation","volume":"16","author":"Xue","year":"2018","journal-title":"Neuroinformatics"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"105395","DOI":"10.1016\/j.cmpb.2020.105395","article-title":"DENSE-INception U-net for medical image segmentation","volume":"192","author":"Zhang","year":"2020","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"150","DOI":"10.1016\/j.compbiomed.2019.03.014","article-title":"A novel end-to-end brain tumor segmentation method using improved fully convolutional networks","volume":"108","author":"Li","year":"2019","journal-title":"Comput. Biol. Med."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1016\/j.patcog.2018.11.009","article-title":"Dual-force convolutional neural networks for accurate brain tumor segmentation","volume":"88","author":"Chen","year":"2019","journal-title":"Pattern Recognit."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"218","DOI":"10.1016\/j.bspc.2019.04.024","article-title":"Automatic detection and localization of Focal Cortical Dysplasia lesions in MRI using fully convolutional neural network","volume":"52","author":"Dev","year":"2019","journal-title":"Biomed. Signal Process. Control"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"105685","DOI":"10.1016\/j.asoc.2019.105685","article-title":"A deep supervised approach for ischemic lesion segmentation from multimodal MRI using Fully Convolutional Network","volume":"84","author":"Karthik","year":"2019","journal-title":"Appl. Soft Comput."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2319","DOI":"10.1109\/TMI.2017.2721362","article-title":"Auto-context convolutional neural network (auto-net) for brain extraction in magnetic resonance imaging","volume":"36","author":"Salehi","year":"2017","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1562","DOI":"10.1109\/TMI.2018.2791721","article-title":"Interactive medical image segmentation using deep learning with image-specific fine tuning","volume":"37","author":"Wang","year":"2018","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Zhou, Z., Siddiquee, M.M.R., Tajbakhsh, N., and Liang, J. (2018). Unet++: A nested u-net architecture for medical image segmentation. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Springer.","DOI":"10.1007\/978-3-030-00889-5_1"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"2281","DOI":"10.1109\/TMI.2019.2903562","article-title":"Ce-net: Context encoder network for 2d medical image segmentation","volume":"38","author":"Gu","year":"2019","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"460","DOI":"10.1016\/j.ijleo.2018.08.086","article-title":"Improved U-NET network for pulmonary nodules segmentation","volume":"174","author":"Tong","year":"2018","journal-title":"Optik"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1016\/j.media.2018.05.008","article-title":"\u03a9-net (omega-net): Fully automatic, multi-view cardiac MR detection, orientation, and segmentation with deep neural networks","volume":"48","author":"Vigneault","year":"2018","journal-title":"Med. Image Anal."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"92539","DOI":"10.1109\/ACCESS.2019.2925060","article-title":"LU-NET: An Improved U-Net for ventricular segmentation","volume":"7","author":"Zhang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"83628","DOI":"10.1109\/ACCESS.2019.2923318","article-title":"Automatic whole heart segmentation using a two-stage u-net framework and an adaptive threshold window","volume":"7","author":"Liu","year":"2019","journal-title":"IEEE Access"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1016\/j.cmpb.2018.12.002","article-title":"Automatic quantification of the LV function and mass: A deep learning approach for cardiovascular MRI","volume":"169","author":"Curiale","year":"2019","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2663","DOI":"10.1109\/TMI.2018.2845918","article-title":"H-DenseUNet: Hybrid densely connected UNet for liver and tumor segmentation from CT volumes","volume":"37","author":"Li","year":"2018","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1016\/j.compbiomed.2018.08.018","article-title":"Robust liver vessel extraction using 3D U-Net with variant dice loss function","volume":"101","author":"Huang","year":"2018","journal-title":"Comput. Biol. Med."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"108822","DOI":"10.1016\/j.ejrad.2020.108822","article-title":"A two-step automated quality assessment for liver MR images based on convolutional neural network","volume":"124","author":"Wang","year":"2020","journal-title":"Eur. J. Radiol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"325","DOI":"10.1016\/j.neucom.2019.01.110","article-title":"AdaResU-Net: Multiobjective adaptive convolutional neural network for medical image segmentation","volume":"392","year":"2020","journal-title":"Neurocomputing"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1016\/j.compmedimag.2019.04.006","article-title":"Model-free prostate cancer segmentation from dynamic contrast-enhanced MRI with recurrent convolutional networks: A feasibility study","volume":"75","author":"Lee","year":"2019","journal-title":"Comput. Med. Imaging Graph."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"44247","DOI":"10.1109\/ACCESS.2019.2908991","article-title":"Nas-unet: Neural architecture search for medical image segmentation","volume":"7","author":"Weng","year":"2019","journal-title":"IEEE Access"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1016\/j.media.2019.01.012","article-title":"Attention gated networks: Learning to leverage salient regions in medical images","volume":"53","author":"Schlemper","year":"2019","journal-title":"Med. Image Anal."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.media.2019.02.006","article-title":"OBELISK-Net: Fewer layers to solve 3D multi-organ segmentation with sparse deformable convolutions","volume":"54","author":"Heinrich","year":"2019","journal-title":"Med. Image Anal."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.compmedimag.2018.01.006","article-title":"Multiple supervised residual network for osteosarcoma segmentation in CT images","volume":"63","author":"Zhang","year":"2018","journal-title":"Comput. Med. Imaging Graph."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"105119","DOI":"10.1016\/j.cmpb.2019.105119","article-title":"Fully automated 3D segmentation and separation of multiple cervical vertebrae in CT images using a 2D convolutional neural network","volume":"184","author":"Bae","year":"2020","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1016\/j.neunet.2019.08.025","article-title":"MultiResUNet: Rethinking the U-Net architecture for multimodal biomedical image segmentation","volume":"121","author":"Ibtehaz","year":"2020","journal-title":"Neural Netw."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"106533","DOI":"10.1016\/j.compeleceng.2019.106533","article-title":"An ensemble of fine-tuned fully convolutional neural networks for pleural effusion cell nuclei segmentation","volume":"81","author":"Kablan","year":"2020","journal-title":"Comput. Electr. Eng."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"101612","DOI":"10.1016\/j.media.2019.101612","article-title":"Trophectoderm segmentation in human embryo images via inceptioned U-Net","volume":"62","author":"Rad","year":"2020","journal-title":"Med. Image Anal."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Colonna, A., Scarpa, F., and Ruggeri, A. (2018). Segmentation of corneal nerves using a u-net-based convolutional neural network. Computational Pathology and Ophthalmic Medical Image Analysis, Springer.","DOI":"10.1007\/978-3-030-00949-6_22"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Shen, D., Liu, T., Peters, T.M., Staib, L.H., Essert, C., Zhou, S., Yap, P.T., and Khan, A. (2019). Dual Encoding U-Net for Retinal Vessel Segmentation. Medical Image Computing and Computer Assisted Intervention\u2014MICCAI 2019, Springer International Publishing.","DOI":"10.1007\/978-3-030-32248-9_98"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"142379","DOI":"10.1109\/ACCESS.2019.2944692","article-title":"TPU cloud-based generalized U-Net for eye fundus image segmentation","volume":"7","author":"Corral","year":"2019","journal-title":"IEEE Access"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"57561","DOI":"10.1109\/ACCESS.2019.2914319","article-title":"Simultaneous arteriole and venule segmentation of dual-modal fundus images using a multi-task cascade network","volume":"7","author":"Zhang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1016\/j.knosys.2019.04.025","article-title":"DUNet: A deformable network for retinal vessel segmentation","volume":"178","author":"Jin","year":"2019","journal-title":"Knowl.-Based Syst."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"483","DOI":"10.1007\/s11548-018-01910-0","article-title":"A dataset of laryngeal endoscopic images with comparative study on convolution neural network-based semantic segmentation","volume":"14","author":"Laves","year":"2019","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"101913","DOI":"10.1016\/j.bspc.2020.101913","article-title":"A multi-scale recurrent fully convolution neural network for laryngeal leukoplakia segmentation","volume":"59","author":"Ji","year":"2020","journal-title":"Biomed. Signal Process. Control"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"751","DOI":"10.1089\/end.2007.9827","article-title":"Navigation in endoscopic soft tissue surgery: Perspectives and limitations","volume":"22","author":"Baumhauer","year":"2008","journal-title":"J. Endourol."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Reiter, A., Allen, P.K., and Zhao, T. (2012). Feature classification for tracking articulated surgical tools. International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer.","DOI":"10.1007\/978-3-642-33418-4_73"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"1198","DOI":"10.1007\/s00464-015-4330-7","article-title":"A simple sensor calibration technique for estimating the 3D pose of endoscopic instruments","volume":"30","author":"Lahanas","year":"2016","journal-title":"Surg. Endosc."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1050","DOI":"10.1109\/TBME.2012.2229278","article-title":"Toward detection and localization of instruments in minimally invasive surgery","volume":"60","author":"Allan","year":"2012","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"234","DOI":"10.12720\/joace.2.3.234-241","article-title":"Visual tracking of laparoscopic instruments","volume":"2","author":"Zhou","year":"2014","journal-title":"J. Autom. Control. Eng. Vol"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Allan, M., Chang, P.L., Ourselin, S., Hawkes, D.J., Sridhar, A., Kelly, J., and Stoyanov, D. (2015). Image based surgical instrument pose estimation with multi-class labelling and optical flow. International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer.","DOI":"10.1007\/978-3-319-24553-9_41"},{"key":"ref_49","unstructured":"Lin, S., Qin, F., Bly, R.A., Moe, K.S., and Hannaford, B. (2020, October 28). UW Sinus Surgery Cadaver\/Live Dataset (UW-Sinus-Surgery-C\/L). Available online: https:\/\/digital.lib.washington.edu\/researchworks\/handle\/1773\/45396."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Rieke, N., Tan, D.J., Alsheakhali, M., Tombari, F., di San Filippo, C.A., Belagiannis, V., Eslami, A., and Navab, N. (2015). Surgical tool tracking and pose estimation in retinal microsurgery. International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer.","DOI":"10.1007\/978-3-319-24553-9_33"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Reiter, A., and Allen, P.K. (2010, January 18\u201322). An online learning approach to in-vivo tracking using synergistic features. Proceedings of the 2010 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Taipei, Taiwan.","DOI":"10.1109\/IROS.2010.5650852"},{"key":"ref_52","unstructured":"McKenna, S., Charif, H.N., and Frank, T. (2005, January 23\u201324). Towards video understanding of laparoscopic surgery: Instrument tracking. Proceedings of the Image and Vision Computing, Auckland, New Zealand."},{"key":"ref_53","first-page":"941511","article-title":"Surgical tool detection and tracking in retinal microsurgery","volume":"Volume 9415","author":"Alsheakhali","year":"2015","journal-title":"Medical Imaging 2015: Image-Guided Procedures, Robotic Interventions, and Modeling"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"2603","DOI":"10.1109\/TMI.2015.2450831","article-title":"Detecting surgical tools by modelling local appearance and global shape","volume":"34","author":"Bouget","year":"2015","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Sznitman, R., Becker, C., and Fua, P. (2014). Fast part-based classification for instrument detection in minimally invasive surgery. International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer.","DOI":"10.1007\/978-3-319-10470-6_86"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Wolf, R., Duchateau, J., Cinquin, P., and Voros, S. (2011). 3D tracking of laparoscopic instruments using statistical and geometric modeling. International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer.","DOI":"10.1007\/978-3-642-23623-5_26"},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Kumar, S., Narayanan, M.S., Singhal, P., Corso, J.J., and Krovi, V. (2013, January 17\u201320). Product of tracking experts for visual tracking of surgical tools. Proceedings of the 2013 IEEE International Conference on Automation Science and Engineering (CASE), Madison, WI, USA.","DOI":"10.1109\/CoASE.2013.6654037"},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Qin, F., Li, Y., Su, Y.H., Xu, D., and Hannaford, B. (2019, January 20\u201324). Surgical instrument segmentation for endoscopic vision with data fusion of cnn prediction and kinematic pose. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8794122"},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Gupta, S., Ali, S., Goldsmith, L., Turney, B., and Rittscher, J. (2020, January 3\u20137). Mi-unet: Improved segmentation in ureteroscopy. Proceedings of the 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), Iowa City, IA, USA.","DOI":"10.1109\/ISBI45749.2020.9098608"},{"key":"ref_60","doi-asserted-by":"crossref","unstructured":"Jha, D., Ali, S., Emanuelsen, K., Hicks, S.A., Thambawita, V., Garcia-Ceja, E., Riegler, M.A., de Lange, T., Schmidt, P.T., and Johansen, H.D. (2021). Kvasir-instrument: Diagnostic and therapeutic tool segmentation dataset in gastrointestinal endoscopy. International Conference on Multimedia Modeling, Springer.","DOI":"10.31219\/osf.io\/s5d48"},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"101920","DOI":"10.1016\/j.media.2020.101920","article-title":"Comparative validation of multi-instance instrument segmentation in endoscopy: Results of the ROBUST-MIS 2019 challenge","volume":"70","author":"Reinke","year":"2021","journal-title":"Med. Image Anal."},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"2188","DOI":"10.1109\/LRA.2019.2900854","article-title":"Real-time instrument segmentation in robotic surgery using auxiliary supervised deep adversarial learning","volume":"4","author":"Islam","year":"2019","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"935","DOI":"10.1109\/LRA.2021.3056354","article-title":"Robotic instrument segmentation with image-to-image translation","volume":"6","author":"Colleoni","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s40537-019-0197-0","article-title":"A survey on image data augmentation for deep learning","volume":"6","author":"Shorten","year":"2019","journal-title":"J. Big Data"},{"key":"ref_65","doi-asserted-by":"crossref","unstructured":"Bloice, M.D., Stocker, C., and Holzinger, A. (2017). Augmentor: An image augmentation library for machine learning. arXiv.","DOI":"10.21105\/joss.00432"},{"key":"ref_66","doi-asserted-by":"crossref","unstructured":"Lindgren, K., Kalavakonda, N., Caballero, D.E., Huang, K., and Hannaford, B. (2018, January 1\u20135). Learned hand gesture classification through synthetically generated training samples. Proceedings of the 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8593433"},{"key":"ref_67","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_68","unstructured":"Eaton-Rosen, Z., Bragman, F., Ourselin, S., and Cardoso, M.J. (2018, January 4\u20136). Improving data augmentation for medical image segmentation. Proceedings of the 1st Conference on Medical Imaging with Deep Learning (MIDL), Amsterdam, The Netherlands."},{"key":"ref_69","doi-asserted-by":"crossref","unstructured":"Kikinis, R., Pieper, S.D., and Vosburgh, K.G. (2014). 3D Slicer: A platform for subject-specific image analysis, visualization, and clinical support. Intraoperative Imaging and Image-Guided Therapy, Springer.","DOI":"10.1007\/978-1-4614-7657-3_19"},{"key":"ref_70","doi-asserted-by":"crossref","first-page":"338","DOI":"10.1089\/end.2015.0620","article-title":"Validation of the RobotiX mentor robotic surgery simulator","volume":"30","author":"Whittaker","year":"2016","journal-title":"J. Endourol."},{"key":"ref_71","doi-asserted-by":"crossref","first-page":"2587","DOI":"10.1007\/s00464-012-2237-0","article-title":"The virtual reality simulator dV-Trainer\u00ae is a valid assessment tool for robotic surgical skills","volume":"26","author":"Perrenot","year":"2012","journal-title":"Surg. Endosc."},{"key":"ref_72","doi-asserted-by":"crossref","unstructured":"Munawar, A., Srishankar, N., and Fischer, G.S. (August, January 31). An Open-Source Framework for Rapid Development of Interactive Soft-Body Simulations for Real-Time Training. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France.","DOI":"10.1109\/ICRA40945.2020.9197573"},{"key":"ref_73","first-page":"1095113","article-title":"Evaluation of 3D slicer as a medical virtual reality visualization platform","volume":"Volume 10951","author":"Choueib","year":"2019","journal-title":"Medical Imaging 2019: Image-Guided Procedures, Robotic Interventions, and Modeling"},{"key":"ref_74","doi-asserted-by":"crossref","first-page":"e2017.00081","DOI":"10.4293\/JSLS.2017.00081","article-title":"Head-to-head comparison of three virtual-reality robotic surgery simulators","volume":"22","author":"Hertz","year":"2018","journal-title":"JSLS J. Soc. Laparoendosc. Surg."},{"key":"ref_75","doi-asserted-by":"crossref","first-page":"2572","DOI":"10.1109\/TMI.2018.2842767","article-title":"Unsupervised reverse domain adaptation for synthetic medical images via adversarial training","volume":"37","author":"Mahmood","year":"2018","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_76","doi-asserted-by":"crossref","unstructured":"Lin, S., Qin, F., Li, Y., Bly, R.A., Moe, K.S., and Hannaford, B. (2020). LC-GAN: Image-to-image Translation Based on Generative Adversarial Network for Endoscopic Images. arXiv.","DOI":"10.1109\/IROS45743.2020.9341556"},{"key":"ref_77","doi-asserted-by":"crossref","unstructured":"Su, Y.H., Huang, K., and Hannaford, B. (2018, January 1\u20133). Real-time vision-based surgical tool segmentation with robot kinematics prior. Proceedings of the 2018 International Symposium on Medical Robotics (ISMR), Atlanta, GA, USA.","DOI":"10.1109\/ISMR.2018.8333305"},{"key":"ref_78","doi-asserted-by":"crossref","unstructured":"Su, Y.H., Huang, I., Huang, K., and Hannaford, B. (2018, January 1\u20135). Comparison of 3d surgical tool segmentation procedures with robot kinematics prior. Proceedings of the 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8594428"},{"key":"ref_79","unstructured":"Liu, M.Y., Breuel, T., and Kautz, J. (2017). Unsupervised image-to-image translation networks. arXiv."},{"key":"ref_80","doi-asserted-by":"crossref","first-page":"101552","DOI":"10.1016\/j.media.2019.101552","article-title":"Generative adversarial network in medical imaging: A review","volume":"58","author":"Yi","year":"2019","journal-title":"Med. Image Anal."},{"key":"ref_81","unstructured":"Simard, P.Y., Steinkraus, D., and Platt, J.C. (2020, October 12). Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis. Icdar. Available online: https:\/\/www.researchgate.net\/profile\/John-Platt-2\/publication\/2880624_Best_Practices_for_Convolutional_Neural_Networks\/links\/00b49524c79b1afb07000000\/Best-Practices-for-Convolutional-Neural-Networks.pdf."},{"key":"ref_82","doi-asserted-by":"crossref","unstructured":"Zhang, X., Smith, N., and Webb, A. (2008). Medical imaging. Biomedical Information Technology, Elsevier.","DOI":"10.1016\/B978-012373583-6.50005-0"},{"key":"ref_83","doi-asserted-by":"crossref","first-page":"6311","DOI":"10.3390\/s20216311","article-title":"Gated Dehazing Network via Least Square Adversarial Learning","volume":"20","author":"Ha","year":"2020","journal-title":"Sensors"},{"key":"ref_84","doi-asserted-by":"crossref","first-page":"3627","DOI":"10.1002\/mp.13047","article-title":"Generating synthetic CTs from magnetic resonance images using generative adversarial networks","volume":"45","author":"Emami","year":"2018","journal-title":"Med. Phys."},{"key":"ref_85","doi-asserted-by":"crossref","unstructured":"Mok, T.C., and Chung, A.C. (2018). Learning data augmentation for brain tumor segmentation with coarse-to-fine generative adversarial networks. International MICCAI Brainlesion Workshop, Springer.","DOI":"10.1007\/978-3-030-11723-8_7"},{"key":"ref_86","doi-asserted-by":"crossref","unstructured":"Shin, H.C., Tenenholtz, N.A., Rogers, J.K., Schwarz, C.G., Senjem, M.L., Gunter, J.L., Andriole, K.P., and Michalski, M. (2018). Medical image synthesis for data augmentation and anonymization using generative adversarial networks. International Workshop on Simulation and Synthesis in Medical Imaging, Springer.","DOI":"10.1007\/978-3-030-00536-8_1"},{"key":"ref_87","doi-asserted-by":"crossref","unstructured":"Gu, X., Knutsson, H., Nilsson, M., and Eklund, A. (2019). Generating diffusion MRI scalar maps from T1 weighted images using generative adversarial networks. Scandinavian Conference on Image Analysis, Springer.","DOI":"10.1007\/978-3-030-20205-7_40"},{"key":"ref_88","doi-asserted-by":"crossref","unstructured":"Hu, Y., Gibson, E., Lee, L.L., Xie, W., Barratt, D.C., Vercauteren, T., and Noble, J.A. (2017). Freehand ultrasound image simulation with spatially-conditioned generative adversarial networks. Molecular Imaging, Reconstruction and Analysis of Moving Body Organs, and Stroke Imaging and Treatment, Springer.","DOI":"10.1007\/978-3-319-67564-0_11"},{"key":"ref_89","doi-asserted-by":"crossref","unstructured":"Tom, F., and Sheet, D. (2018, January 4\u20137). Simulating patho-realistic ultrasound images using deep generative networks with adversarial learning. Proceedings of the 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), Washington, DC, USA.","DOI":"10.1109\/ISBI.2018.8363780"},{"key":"ref_90","doi-asserted-by":"crossref","first-page":"53","DOI":"10.3389\/fbioe.2018.00053","article-title":"Exploring the potential of generative adversarial networks for synthesizing radiological images of the spine to be used in in silico trials","volume":"6","author":"Galbusera","year":"2018","journal-title":"Front. Bioeng. Biotechnol."},{"key":"ref_91","doi-asserted-by":"crossref","unstructured":"Mahapatra, D., Bozorgtabar, B., Thiran, J.P., and Reyes, M. (2018). Efficient active learning for image classification and segmentation using a sample selection and conditional generative adversarial network. International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer.","DOI":"10.1007\/978-3-030-00934-2_65"},{"key":"ref_92","doi-asserted-by":"crossref","first-page":"258","DOI":"10.1001\/jamaophthalmol.2018.6156","article-title":"Assessment of deep generative models for high-resolution synthetic retinal image generation of age-related macular degeneration","volume":"137","author":"Burlina","year":"2019","journal-title":"JAMA Ophthalmol."},{"key":"ref_93","doi-asserted-by":"crossref","unstructured":"Jin, D., Xu, Z., Tang, Y., Harrison, A.P., and Mollura, D.J. (2018). CT-realistic lung nodule simulation from 3D conditional generative adversarial networks for robust lung segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer.","DOI":"10.1007\/978-3-030-00934-2_81"},{"key":"ref_94","doi-asserted-by":"crossref","first-page":"6266","DOI":"10.1109\/LRA.2021.3092302","article-title":"Surgical Tool Segmentation using Generative Adversarial Networks with Unpaired Training Data","volume":"6","author":"Zhang","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_95","doi-asserted-by":"crossref","unstructured":"Wang, H., Xiong, H., and Cai, Y. (2020). Image Localized Style Transfer to Design Clothes Based on CNN and Interactive Segmentation. Comput. Intell. Neurosci., 2020.","DOI":"10.1155\/2020\/8894309"},{"key":"ref_96","doi-asserted-by":"crossref","unstructured":"Zhu, J.Y., Park, T., Isola, P., and Efros, A.A. (2017, January 22\u201329). Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.244"},{"key":"ref_97","unstructured":"Su, Y.H., Chitrakar, D., Jiang, W., and Huang, K. (2021, February 10). The Modified UNet Source Code for This Research. Available online: https:\/\/github.com\/MHC-CycleGAN-Research\/Our-UNet-Code."},{"key":"ref_98","unstructured":"Jiang, W., and Su, Y.H. (2020, November 28). The Modified CycleGAN Source Code for This Research. Available online: https:\/\/github.com\/MHC-CycleGAN-Research\/Our-CycleGAN-Code."},{"key":"ref_99","doi-asserted-by":"crossref","first-page":"6639","DOI":"10.1109\/LRA.2020.3009073","article-title":"Towards better surgical instrument segmentation in endoscopic vision: Multi-angle feature aggregation and contour supervision","volume":"5","author":"Qin","year":"2020","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_100","doi-asserted-by":"crossref","unstructured":"Shamir, R.R., Duchin, Y., Kim, J., Sapiro, G., and Harel, N. (2019). Continuous dice coefficient: A method for evaluating probabilistic segmentations. arXiv.","DOI":"10.1101\/306977"},{"key":"ref_101","doi-asserted-by":"crossref","unstructured":"Rezatofighi, H., Tsoi, N., Gwak, J., Sadeghian, A., Reid, I., and Savarese, S. (2019, January 15\u201320). Generalized intersection over union: A metric and a loss for bounding box regression. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00075"},{"key":"ref_102","doi-asserted-by":"crossref","unstructured":"Su, Y.H., Huang, K., and Hannaford, B. (2019, January 3\u20138). Multicamera 3d reconstruction of dynamic surgical cavities: Non-rigid registration and point classification. Proceedings of the 2019 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China.","DOI":"10.1109\/IROS40897.2019.8968489"},{"key":"ref_103","doi-asserted-by":"crossref","unstructured":"Su, Y.H., Huang, K., and Hannaford, B. (2020, January 18\u201320). Multicamera 3d reconstruction of dynamic surgical cavities: Autonomous optimal camera viewpoint adjustment. Proceedings of the 2020 International Symposium on Medical Robotics (ISMR), Atlanta, GA, USA.","DOI":"10.1109\/ISMR48331.2020.9312951"},{"key":"ref_104","doi-asserted-by":"crossref","unstructured":"Huang, K., Chitrakar, D., Mitra, R., Subedi, D., and Su, Y.H. (2020, January 20\u201324). Characterizing limits of vision-based force feedback in simulated surgical tool-tissue interaction. Proceedings of the 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Montreal, QC, Canada.","DOI":"10.1109\/EMBC44109.2020.9176658"},{"key":"ref_105","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1109\/JSEN.2021.3052755","article-title":"Image-Based Force Estimation in Medical Applications: A Review","volume":"21","author":"Nazari","year":"2021","journal-title":"IEEE Sens. J."},{"key":"ref_106","doi-asserted-by":"crossref","unstructured":"Su, Y.H., Sosnovskaya, Y., Hannaford, B., and Huang, K. (2020, January 9\u201311). Securing Robot-assisted Minimally Invasive Surgery through Perception Complementarities. Proceedings of the 2020 Fourth IEEE International Conference on Robotic Computing (IRC), Taichung, Taiwan.","DOI":"10.1109\/IRC.2020.00013"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/15\/5163\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:37:15Z","timestamp":1760164635000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/15\/5163"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,30]]},"references-count":106,"journal-issue":{"issue":"15","published-online":{"date-parts":[[2021,8]]}},"alternative-id":["s21155163"],"URL":"https:\/\/doi.org\/10.3390\/s21155163","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2021,7,30]]}}}