{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,19]],"date-time":"2025-10-19T16:15:20Z","timestamp":1760890520801,"version":"3.37.3"},"reference-count":42,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2023,6,8]],"date-time":"2023-06-08T00:00:00Z","timestamp":1686182400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,6,8]],"date-time":"2023-06-08T00:00:00Z","timestamp":1686182400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100007013","name":"F. Hoffmann-La Roche","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100007013","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Digit Imaging"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Deep neural networks (DNNs) have recently showed remarkable performance in various computer vision tasks, including classification and segmentation of medical images. Deep ensembles (an aggregated prediction of multiple DNNs) were shown to improve a DNN\u2019s performance in various classification tasks. Here we explore how deep ensembles perform in the image segmentation task, in particular, organ segmentations in CT (Computed Tomography) images. Ensembles of V-Nets were trained to segment multiple organs using several in-house and publicly available clinical studies. The ensembles segmentations were tested on images from a different set of studies, and the effects of ensemble size as well as other ensemble parameters were explored for various organs. Compared to single models, Deep Ensembles significantly improved the average segmentation accuracy, especially for those organs where the accuracy was lower. More importantly, Deep Ensembles strongly reduced occasional \u201ccatastrophic\u201d segmentation failures characteristic of single models and variability of the segmentation accuracy from image to image. To quantify this we defined the \u201chigh risk images\u201d: images for which at least one model produced an outlier metric (performed in the lower 5% percentile). These images comprised about 12% of the test images across all organs. Ensembles performed without outliers for 68%\u2013100% of the \u201chigh risk images\u201d depending on the performance metric used.<\/jats:p>","DOI":"10.1007\/s10278-023-00857-2","type":"journal-article","created":{"date-parts":[[2023,6,8]],"date-time":"2023-06-08T20:35:25Z","timestamp":1686256525000},"page":"2060-2074","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Deep Ensembles Are Robust to Occasional Catastrophic Failures of Individual DNNs for Organs Segmentations in CT Images"],"prefix":"10.1007","volume":"36","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6110-3159","authenticated-orcid":false,"given":"Yury","family":"Petrov","sequence":"first","affiliation":[]},{"given":"Bilal","family":"Malik","sequence":"additional","affiliation":[]},{"given":"Jill","family":"Fredrickson","sequence":"additional","affiliation":[]},{"given":"Skander","family":"Jemaa","sequence":"additional","affiliation":[]},{"given":"Richard A. D.","family":"Carano","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,6,8]]},"reference":[{"key":"857_CR1","unstructured":"B.\u00a0Lakshminarayanan, A.\u00a0Pritzel, C.\u00a0Blundell, Simple and scalable predictive uncertainty estimation using deep ensembles, Advances in neural information processing systems 30 (2017)."},{"issue":"2","key":"857_CR2","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1007\/BF00058655","volume":"24","author":"L Breiman","year":"1996","unstructured":"L.\u00a0Breiman, Bagging predictors, Machine learning 24\u00a0(2) (1996) 123\u2013140.","journal-title":"Machine learning"},{"issue":"2","key":"857_CR3","doi-asserted-by":"publisher","first-page":"197","DOI":"10.1007\/BF00116037","volume":"5","author":"RE Schapire","year":"1990","unstructured":"R.\u00a0E. Schapire, The strength of weak learnability, Machine learning 5\u00a0(2) (1990) 197\u2013227.","journal-title":"Machine learning"},{"issue":"1","key":"857_CR4","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","volume":"45","author":"L Breiman","year":"2001","unstructured":"L.\u00a0Breiman, Random forests, Machine learning 45\u00a0(1) (2001) 5\u201332.","journal-title":"Machine learning"},{"key":"857_CR5","doi-asserted-by":"crossref","unstructured":"X.\u00a0Li, B.\u00a0Aldridge, J.\u00a0Rees, R.\u00a0Fisher, Estimating the ground truth from multiple individual segmentations with application to skin lesion segmentation, in: Proc. Medical Image Understanding and Analysis Conference, UK, Vol.\u00a01, 2010, pp. 101\u2013106.","DOI":"10.1109\/ISBI.2011.5872670"},{"key":"857_CR6","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2021.102029","volume":"71","author":"E Hann","year":"2021","unstructured":"E.\u00a0Hann, I.\u00a0A. Popescu, Q.\u00a0Zhang, R.\u00a0A. Gonzales, A.\u00a0Barut\u00e7u, S.\u00a0Neubauer, V.\u00a0M. Ferreira, S.\u00a0K. Piechnik, Deep neural network ensemble for on-the-fly quality control-driven segmentation of cardiac mri t1 mapping, Medical image analysis 71 (2021) 102029.","journal-title":"Medical image analysis"},{"issue":"7","key":"857_CR7","doi-asserted-by":"publisher","first-page":"903","DOI":"10.1109\/TMI.2004.828354","volume":"23","author":"SK Warfield","year":"2004","unstructured":"S.\u00a0K. Warfield, K.\u00a0H. Zou, W.\u00a0M. Wells, Simultaneous truth and performance level estimation (staple): an algorithm for the validation of image segmentation, IEEE transactions on medical imaging 23\u00a0(7) (2004) 903\u2013921.","journal-title":"IEEE transactions on medical imaging"},{"key":"857_CR8","doi-asserted-by":"publisher","first-page":"28","DOI":"10.1016\/j.compmedimag.2016.07.012","volume":"55","author":"J Zilly","year":"2017","unstructured":"J.\u00a0Zilly, J.\u00a0M. Buhmann, D.\u00a0Mahapatra, Glaucoma detection using entropy sampling and ensemble learning for automatic optic cup and disc segmentation, Computerized Medical Imaging and Graphics 55 (2017) 28\u201341.","journal-title":"Computerized Medical Imaging and Graphics"},{"key":"857_CR9","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1016\/j.compmedimag.2018.05.001","volume":"69","author":"JV Manj\u00f3n","year":"2018","unstructured":"J.\u00a0V. Manj\u00f3n, P.\u00a0Coup\u00e9, P.\u00a0Raniga, Y.\u00a0Xia, P.\u00a0Desmond, J.\u00a0Fripp, O.\u00a0Salvado, Mri white matter lesion segmentation using an ensemble of neural networks and overcomplete patch-based voting, Computerized Medical Imaging and Graphics 69 (2018) 43\u201351.","journal-title":"Computerized Medical Imaging and Graphics"},{"key":"857_CR10","doi-asserted-by":"crossref","unstructured":"N.\u00a0Bnouni, I.\u00a0Rekik, M.\u00a0S. Rhim, N.\u00a0E.\u00a0B. Amara, Dynamic multi-scale cnn forest learning for automatic cervical cancer segmentation, in: International Workshop on Machine Learning in Medical Imaging, Springer, 2018, pp. 19\u201327.","DOI":"10.1007\/978-3-030-00919-9_3"},{"key":"857_CR11","doi-asserted-by":"crossref","unstructured":"K.\u00a0Kamnitsas, W.\u00a0Bai, E.\u00a0Ferrante, S.\u00a0McDonagh, M.\u00a0Sinclair, N.\u00a0Pawlowski, M.\u00a0Rajchl, M.\u00a0Lee, B.\u00a0Kainz, D.\u00a0Rueckert, et\u00a0al., Ensembles of multiple models and architectures for robust brain tumour segmentation, in: International MICCAI brainlesion workshop, Springer, 2017, pp. 450\u2013462.","DOI":"10.1007\/978-3-319-75238-9_38"},{"key":"857_CR12","doi-asserted-by":"publisher","DOI":"10.1016\/j.compmedimag.2019.101660","volume":"79","author":"J Dolz","year":"2020","unstructured":"J.\u00a0Dolz, C.\u00a0Desrosiers, L.\u00a0Wang, J.\u00a0Yuan, D.\u00a0Shen, I.\u00a0B. Ayed, Deep cnn ensembles and suggestive annotations for infant brain mri segmentation, Computerized Medical Imaging and Graphics 79 (2020) 101660.","journal-title":"Computerized Medical Imaging and Graphics"},{"key":"857_CR13","doi-asserted-by":"crossref","unstructured":"A.\u00a0E. Kavur, L.\u00a0I. Kuncheva, M.\u00a0A. Selver, Basic ensembles of vanilla-style deep learning models improve liver segmentation from ct images, in: Convolutional Neural Networks for Medical Image Processing Applications, CRC Press, 2020, pp. 52\u201374.","DOI":"10.1201\/9781003215141-3"},{"key":"857_CR14","doi-asserted-by":"crossref","unstructured":"S.\u00a0Reza, J.\u00a0A. Butman, D.\u00a0M. Park, D.\u00a0L. Pham, S.\u00a0Roy, Adaboosted deep ensembles: Getting maximum performance out of small training datasets, in: International Workshop on Machine Learning in Medical Imaging, Springer, 2020, pp. 572\u2013582.","DOI":"10.1007\/978-3-030-59861-7_58"},{"issue":"2","key":"857_CR15","doi-asserted-by":"publisher","first-page":"203","DOI":"10.1038\/s41592-020-01008-z","volume":"18","author":"F Isensee","year":"2021","unstructured":"F.\u00a0Isensee, P.\u00a0F. Jaeger, S.\u00a0A. Kohl, J.\u00a0Petersen, K.\u00a0H. Maier-Hein, nnu-net: a self-configuring method for deep learning-based biomedical image segmentation, Nature methods 18\u00a0(2) (2021) 203\u2013211.","journal-title":"Nature methods"},{"key":"857_CR16","doi-asserted-by":"publisher","unstructured":"B.\u00a0Ghoshal, A.\u00a0Tucker, B.\u00a0Sanghera, W.\u00a0Lup\u00a0Wong, Estimating uncertainty in deep learning for reporting confidence to clinicians in medical image segmentation and diseases detection, Computational Intelligence 37\u00a0(2) (2021) 701\u2013734. https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/coin.12411https:\/\/doi.org\/10.1111\/coin.12411","DOI":"10.1111\/coin.12411"},{"key":"857_CR17","doi-asserted-by":"crossref","unstructured":"A.\u00a0Jungo, M.\u00a0Reyes, Assessing reliability and challenges of uncertainty estimations for medical image segmentation, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, 2019, pp. 48\u201356.","DOI":"10.1007\/978-3-030-32245-8_6"},{"key":"857_CR18","doi-asserted-by":"crossref","unstructured":"Z.\u00a0Mirikharaji, K.\u00a0Abhishek, S.\u00a0Izadi, G.\u00a0Hamarneh, D-lema: Deep learning ensembles from multiple annotations-application to skin lesion segmentation, in: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 1837\u20131846.","DOI":"10.1109\/CVPRW53098.2021.00203"},{"issue":"3","key":"857_CR19","doi-asserted-by":"publisher","first-page":"231","DOI":"10.1017\/S0269888997003123","volume":"12","author":"AJ Sharkey","year":"1997","unstructured":"A.\u00a0J. Sharkey, N.\u00a0E. Sharkey, Combining diverse neural nets, The Knowledge Engineering Review 12\u00a0(3) (1997) 231\u2013247.","journal-title":"The Knowledge Engineering Review"},{"key":"857_CR20","doi-asserted-by":"crossref","unstructured":"E.\u00a0A. Eisenhauer, P.\u00a0Therasse, J.\u00a0Bogaerts, L.\u00a0H. Schwartz, D.\u00a0Sargent, R.\u00a0Ford, J.\u00a0Dancey, S.\u00a0Arbuck, S.\u00a0Gwyther, M.\u00a0Mooney, et\u00a0al., New response evaluation criteria in solid tumours: revised recist guideline (version 1.1), European Journal of Cancer 45\u00a0(2) (2009) 228\u2013247.","DOI":"10.1016\/j.ejca.2008.10.026"},{"key":"857_CR21","doi-asserted-by":"crossref","unstructured":"F.\u00a0Milletari, N.\u00a0Navab, S.-A. Ahmadi, V-net: Fully convolutional neural networks for volumetric medical image segmentation, in: 2016 fourth international conference on 3D vision (3DV), IEEE, 2016, pp. 565\u2013571.","DOI":"10.1109\/3DV.2016.79"},{"issue":"6","key":"857_CR22","doi-asserted-by":"publisher","first-page":"1045","DOI":"10.1007\/s10278-013-9622-7","volume":"26","author":"K Clark","year":"2013","unstructured":"K.\u00a0Clark, B.\u00a0Vendt, K.\u00a0Smith, J.\u00a0Freymann, J.\u00a0Kirby, P.\u00a0Koppel, S.\u00a0Moore, S.\u00a0Phillips, D.\u00a0Maffitt, M.\u00a0Pringle, et\u00a0al., The cancer imaging archive (tcia): maintaining and operating a public information repository, Journal of digital imaging 26\u00a0(6) (2013) 1045\u20131057. https:\/\/doi.org\/10.1007\/s10278-013-9622-7","journal-title":"Journal of digital imaging"},{"key":"857_CR23","doi-asserted-by":"publisher","unstructured":"E.\u00a0Gibson, F.\u00a0Giganti, Y.\u00a0Hu, E.\u00a0Bonmati, S.\u00a0Bandula, K.\u00a0Gurusamy, B.\u00a0Davidson, S.\u00a0P. Pereira, M.\u00a0J. Clarkson, D.\u00a0C. Barratt, Multi-organ Abdominal CT Reference Standard Segmentations, This data set was developed as part of independent research supported by Cancer Research UK (Multidisciplinary C28070\/A19985) and the National Institute for Health Research UCL\/UCL Hospitals Biomedical Research Centre. (Feb. 2018). https:\/\/doi.org\/10.5281\/zenodo.1169361","DOI":"10.5281\/zenodo.1169361"},{"key":"857_CR24","unstructured":"B.\u00a0Rister, K.\u00a0Shivakumar, T.\u00a0Nobashi, D.\u00a0Rubin, Ct-org: Ct volumes with multiple organ segmentations, The Cancer Imaging Archive (2019)."},{"key":"857_CR25","unstructured":"A.\u00a0L. Simpson, M.\u00a0Antonelli, S.\u00a0Bakas, M.\u00a0Bilello, K.\u00a0Farahani, B.\u00a0van Ginneken, A.\u00a0Kopp-Schneider, B.\u00a0A. Landman, G.\u00a0Litjens, B.\u00a0Menze, O.\u00a0Ronneberger, R.\u00a0M. Summers, P.\u00a0Bilic, P.\u00a0F. Christ, R.\u00a0K.\u00a0G. Do, M.\u00a0Gollub, J.\u00a0Golia-Pernicka, S.\u00a0H. Heckers, W.\u00a0R. Jarnagin, M.\u00a0K. McHugo, S.\u00a0Napel, E.\u00a0Vorontsov, L.\u00a0Maier-Hein, M.\u00a0J. Cardoso, A large annotated medical image dataset for the development and evaluation of segmentation algorithms (2019). http:\/\/arxiv.org\/abs\/1902.09063arXiv:1902.09063."},{"issue":"24","key":"857_CR26","doi-asserted-by":"publisher","first-page":"2288","DOI":"10.1056\/NEJMoa1716948","volume":"378","author":"MA Socinski","year":"2018","unstructured":"M.\u00a0A. Socinski, R.\u00a0M. Jotte, F.\u00a0Cappuzzo, F.\u00a0Orlandi, D.\u00a0Stroyakovskiy, N.\u00a0Nogami, D.\u00a0Rodr\u00edguez-Abreu, D.\u00a0Moro-Sibilot, C.\u00a0A. Thomas, F.\u00a0Barlesi, et\u00a0al., Atezolizumab for first-line treatment of metastatic nonsquamous nsclc, New England Journal of Medicine 378\u00a0(24) (2018) 2288\u20132301.","journal-title":"New England Journal of Medicine"},{"issue":"31","key":"857_CR27","doi-asserted-by":"publisher","first-page":"3529","DOI":"10.1200\/JCO.2017.73.3402","volume":"35","author":"U Vitolo","year":"2017","unstructured":"U.\u00a0Vitolo, M.\u00a0Trn\u011bn\u1ef3, D.\u00a0Belada, J.\u00a0M. Burke, A.\u00a0M. Carella, N.\u00a0Chua, P.\u00a0Abrisqueta, J.\u00a0Demeter, I.\u00a0Flinn, X.\u00a0Hong, et\u00a0al., Obinutuzumab or rituximab plus cyclophosphamide, doxorubicin, vincristine, and prednisone in previously untreated diffuse large b-cell lymphoma, J Clin Oncol 35\u00a0(31) (2017) 3529\u20133537.","journal-title":"J Clin Oncol"},{"key":"857_CR28","doi-asserted-by":"crossref","unstructured":"E.\u00a0A. Perez, C.\u00a0Barrios, W.\u00a0Eiermann, M.\u00a0Toi, Y.-H. Im, P.\u00a0Conte, M.\u00a0Martin, T.\u00a0Pienkowski, X.\u00a0Pivot, H.\u00a0A. Burris, et\u00a0al., Trastuzumab emtansine with or without pertuzumab versus trastuzumab plus taxane for human epidermal growth factor receptor 2\u2013positive, advanced breast cancer: primary results from the phase iii marianne study, Journal of Clinical Oncology 35\u00a0(2) (2017) 141.","DOI":"10.1200\/JCO.2016.67.4887"},{"issue":"8","key":"857_CR29","doi-asserted-by":"publisher","first-page":"1351","DOI":"10.1016\/j.jtho.2020.03.028","volume":"15","author":"R Jotte","year":"2020","unstructured":"R.\u00a0Jotte, F.\u00a0Cappuzzo, I.\u00a0Vynnychenko, D.\u00a0Stroyakovskiy, D.\u00a0Rodr\u00edguez-Abreu, M.\u00a0Hussein, R.\u00a0Soo, H.\u00a0J. Conter, T.\u00a0Kozuki, K.-C. Huang, et\u00a0al., Atezolizumab in combination with carboplatin and nab-paclitaxel in advanced squamous nsclc (impower131): results from a randomized phase iii trial, Journal of Thoracic Oncology 15\u00a0(8) (2020) 1351\u20131360.","journal-title":"Journal of Thoracic Oncology"},{"key":"857_CR30","doi-asserted-by":"crossref","unstructured":"L.\u00a0I. Kuncheva, Combining pattern classifiers: methods and algorithms, John Wiley & Sons, 2014.","DOI":"10.1002\/9781118914564"},{"issue":"3","key":"857_CR31","doi-asserted-by":"publisher","first-page":"573","DOI":"10.1109\/TASSP.1986.1164857","volume":"34","author":"I Pitas","year":"1986","unstructured":"I.\u00a0Pitas, A.\u00a0Venetsanopoulos, Nonlinear mean filters in image processing, IEEE transactions on acoustics, speech, and signal processing 34\u00a0(3) (1986) 573\u2013584.","journal-title":"IEEE transactions on acoustics, speech, and signal processing"},{"key":"857_CR32","doi-asserted-by":"crossref","unstructured":"O.\u00a0Ronneberger, P.\u00a0Fischer, T.\u00a0Brox, U-net: Convolutional networks for biomedical image segmentation, in: International Conference on Medical image computing and computer-assisted intervention, Springer, 2015, pp. 234\u2013241.","DOI":"10.1007\/978-3-319-24574-4_28"},{"issue":"8","key":"857_CR33","doi-asserted-by":"publisher","first-page":"1822","DOI":"10.1109\/TMI.2018.2806309","volume":"37","author":"E Gibson","year":"2018","unstructured":"E.\u00a0Gibson, F.\u00a0Giganti, Y.\u00a0Hu, E.\u00a0Bonmati, S.\u00a0Bandula, K.\u00a0Gurusamy, B.\u00a0Davidson, S.\u00a0P. Pereira, M.\u00a0J. Clarkson, D.\u00a0C. Barratt, Automatic multi-organ segmentation on abdominal ct with dense v-networks, IEEE transactions on medical imaging 37\u00a0(8) (2018) 1822\u20131834.","journal-title":"IEEE transactions on medical imaging"},{"key":"857_CR34","unstructured":"K.\u00a0He, X.\u00a0Zhang, S.\u00a0Ren, J.\u00a0Sun, Delving deep into rectifiers: Surpassing human-level performance on imagenet classification (2015). http:\/\/arxiv.org\/abs\/1502.01852http:\/\/arxiv.org\/abs\/1502.01852arXiv:1502.01852."},{"key":"857_CR35","unstructured":"P.\u00a0Izmailov, D.\u00a0Podoprikhin, T.\u00a0Garipov, D.\u00a0Vetrov, A.\u00a0G. Wilson, Averaging weights leads to wider optima and better generalization, arXiv preprint http:\/\/arxiv.org\/abs\/1803.05407arXiv:1803.05407 (2018)."},{"key":"857_CR36","unstructured":"G.\u00a0Hinton, O.\u00a0Vinyals, J.\u00a0Dean, et\u00a0al., Distilling the knowledge in a neural network, arXiv preprint http:\/\/arxiv.org\/abs\/1503.02531arXiv:1503.02531 2\u00a0(7) (2015)."},{"key":"857_CR37","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2021.104497","volume":"134","author":"Y-H Nai","year":"2021","unstructured":"Y.-H. Nai, B.\u00a0W. Teo, N.\u00a0L. Tan, S.\u00a0O\u2019Doherty, M.\u00a0C. Stephenson, Y.\u00a0L. Thian, E.\u00a0Chiong, A.\u00a0Reilhac, Comparison of metrics for the evaluation of medical segmentations using prostate mri dataset, Computers in Biology and Medicine 134 (2021) 104497.","journal-title":"Computers in Biology and Medicine"},{"key":"857_CR38","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2020.101950","volume":"69","author":"AE Kavur","year":"2021","unstructured":"A.\u00a0E. Kavur, N.\u00a0S. Gezer, M.\u00a0Bari\u015f, S.\u00a0Aslan, P.-H. Conze, V.\u00a0Groza, D.\u00a0D. Pham, S.\u00a0Chatterjee, P.\u00a0Ernst, S.\u00a0\u00d6zkan, et\u00a0al., Chaos challenge-combined (ct-mr) healthy abdominal organ segmentation, Medical Image Analysis 69 (2021) 101950.","journal-title":"Medical Image Analysis"},{"key":"857_CR39","doi-asserted-by":"crossref","unstructured":"D.\u00a0York, N.\u00a0M. Evensen, M.\u00a0L. Martinez, J.\u00a0De\u00a0Basabe\u00a0Delgado, Unified equations for the slope, intercept, and standard errors of the best straight line, American journal of physics 72\u00a0(3) (2004) 367\u2013375.","DOI":"10.1119\/1.1632486"},{"key":"857_CR40","unstructured":"S.\u00a0Fort, H.\u00a0Hu, B.\u00a0Lakshminarayanan, Deep ensembles: A loss landscape perspective, arXiv preprint http:\/\/arxiv.org\/abs\/1912.02757arXiv:1912.02757 (2019)."},{"key":"857_CR41","unstructured":"Z.\u00a0Allen-Zhu, Y.\u00a0Li, Towards understanding ensemble, knowledge distillation and self-distillation in deep learning, arXiv preprint http:\/\/arxiv.org\/abs\/2012.09816arXiv:2012.09816 (2020)."},{"key":"857_CR42","unstructured":"T.\u00a0Garipov, P.\u00a0Izmailov, D.\u00a0Podoprikhin, D.\u00a0P. Vetrov, A.\u00a0G. Wilson, Loss surfaces, mode connectivity, and fast ensembling of dnns, Advances in neural information processing systems 31 (2018)."}],"container-title":["Journal of Digital Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10278-023-00857-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10278-023-00857-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10278-023-00857-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,14]],"date-time":"2023-09-14T18:05:46Z","timestamp":1694714746000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10278-023-00857-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,8]]},"references-count":42,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2023,10]]}},"alternative-id":["857"],"URL":"https:\/\/doi.org\/10.1007\/s10278-023-00857-2","relation":{},"ISSN":["1618-727X"],"issn-type":[{"type":"electronic","value":"1618-727X"}],"subject":[],"published":{"date-parts":[[2023,6,8]]},"assertion":[{"value":"25 January 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 May 2023","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 May 2023","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 June 2023","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The studies were performed in accordance with Good Clinical Practice guidelines and followed the tenets of the Declaration of Helsinki.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics Approval"}},{"value":"All study participants provided written informed consent, and protocols were approved by independent ethics committees at each site.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to Participate"}},{"value":"The manuscript contains no identifiable individual data or images that would require consent to publish from any participant.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for Publication"}},{"value":"The authors declare no competing interests.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing Interests"}}]}}