{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,24]],"date-time":"2026-03-24T05:26:19Z","timestamp":1774329979159,"version":"3.50.1"},"reference-count":28,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2024,3,20]],"date-time":"2024-03-20T00:00:00Z","timestamp":1710892800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Heavily imbalanced datasets are common in lesion segmentation. Specifically, the lesions usually comprise less than 5% of the whole image volume when dealing with brain MRI. A common solution when training with a limited dataset is the use of specific loss functions that rebalance the effect of background and foreground voxels. These approaches are usually evaluated running a single cross-validation split without taking into account other possible random aspects that might affect the true improvement of the final metric (i.e., random weight initialisation or random shuffling). Furthermore, the evolution of the effect of the loss on the heavily imbalanced class is usually not analysed during the training phase. In this work, we present an analysis of different common loss metrics during training on public datasets dealing with brain lesion segmentation in heavy imbalanced datasets. In order to limit the effect of hyperparameter tuning and architecture, we chose a 3D Unet architecture due to its ability to provide good performance on different segmentation applications. We evaluated this framework on two public datasets and we observed that weighted losses have a similar performance on average, even though heavily weighting the gradient of the foreground class gives better performance in terms of true positive segmentation.<\/jats:p>","DOI":"10.3390\/s24061981","type":"journal-article","created":{"date-parts":[[2024,3,20]],"date-time":"2024-03-20T09:14:33Z","timestamp":1710926073000},"page":"1981","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["An Analysis of Loss Functions for Heavily Imbalanced Lesion Segmentation"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4417-1704","authenticated-orcid":false,"given":"Mariano","family":"Cabezas","sequence":"first","affiliation":[{"name":"Brain and Mind Centre, The University of Sydney, Camperdown, NSW 2050, Australia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4521-9113","authenticated-orcid":false,"given":"Yago","family":"Diez","sequence":"additional","affiliation":[{"name":"Faculty of Science, Yamagata University, Yamagata 990-8560, Japan"}]}],"member":"1968","published-online":{"date-parts":[[2024,3,20]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"403","DOI":"10.1007\/s12021-016-9301-1","article-title":"Validation of White-Matter Lesion Change Detection Methods on a Novel Publicly Available MRI Image Database","volume":"1","author":"Lesjak","year":"2016","journal-title":"Neuroinformatics"},{"key":"ref_2","first-page":"1","article-title":"Objective evaluation of multiple sclerosis lesion segmentation using a data management and processing infrastructure","volume":"8","author":"Commowick","year":"2018","journal-title":"Nat. Sci. Rep."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"102357","DOI":"10.1016\/j.nicl.2020.102357","article-title":"Comparison and validation of seven white matter hyperintensities segmentation software in elderly patients","volume":"27","author":"Vanderbecq","year":"2020","journal-title":"Neuroimage Clin."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2556","DOI":"10.1109\/TMI.2019.2905770","article-title":"Standardized Assessment of Automatic Segmentation of White Matter Hyperintensities and Results of the WMH Segmentation Challenge","volume":"38","author":"Kuijf","year":"2019","journal-title":"IEEE Trans. Med. Imag."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"820","DOI":"10.1109\/JPROC.2021.3054390","article-title":"A Review of Deep Learning in Medical Imaging: Imaging Traits, Technology Trends, Case Studies with Progress Highlights, and Future Promises","volume":"109","author":"Zhou","year":"2021","journal-title":"Proc. IEEE"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1016\/j.artmed.2018.08.008","article-title":"Deep convolutional neural networks for brain image analysis on magnetic resonance imaging: A review","volume":"95","author":"Bernal","year":"2019","journal-title":"Artif. Intell. Med."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1038\/s41592-020-01008-z","article-title":"nnU-Net: A self-configuring method for deep learning-based biomedical image segmentation","volume":"18","author":"Isensee","year":"2021","journal-title":"Nat. Methods"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1016\/j.neuroimage.2017.04.034","article-title":"Improving automated multiple sclerosis lesion segmentation with a cascaded 3D convolutional neural network approach","volume":"155","author":"Valverde","year":"2017","journal-title":"Neuroimage"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1065","DOI":"10.1109\/TMI.2020.3046692","article-title":"Analyzing Overfitting under Class Imbalance in Neural Networks for Image Segmentation","volume":"40","author":"Li","year":"2021","journal-title":"IEEE Trans. Med. Imag."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"101638","DOI":"10.1016\/j.nicl.2018.101638","article-title":"One-shot domain adaptation in multiple sclerosis lesion segmentation using convolutional neural networks","volume":"21","author":"Valverde","year":"2019","journal-title":"Neuroimage Clin."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Milletari, F., Navab, N., and Ahmadi, S. (2016, January 25\u201328). V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation. Proceedings of the 2016 Fourth International Conference on 3D Vision (3DV), Stanford, CA, USA.","DOI":"10.1109\/3DV.2016.79"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Sudre, C.H., Li, W., Vercauteren, T., Ourselin, S., and Jorge Cardoso, M. (2017, January 14). Generalised Dice Overlap as a Deep Learning Loss Function for Highly Unbalanced Segmentations. Proceedings of the Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support Workshop\u2014MICCAI, Qu\u00e9bec City, QC, Canada.","DOI":"10.1007\/978-3-319-67558-9_28"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Goyal, P., Girshick, R., He, K., and Doll\u00e1r, P. (2017, January 22\u201329). Focal Loss for Dense Object Detection. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.324"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"102035","DOI":"10.1016\/j.media.2021.102035","article-title":"Loss odyssey in medical image segmentation","volume":"71","author":"Ma","year":"2021","journal-title":"Med. Image Anal."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1007\/BF00200801","article-title":"Bayesian neural networks","volume":"61","author":"Kononenko","year":"1989","journal-title":"Biol. Cybern."},{"key":"ref_16","unstructured":"Hern\u00e1ndez-Lobato, J.M., and Adams, R. (2015, January 6\u201311). Probabilistic backpropagation for scalable learning of bayesian neural networks. Proceedings of the International Conference on Machine Learning, PMLR, Lille, France."},{"key":"ref_17","unstructured":"Lakshminarayanan, B., Pritzel, A., and Blundell, C. (2017). Simple and scalable predictive uncertainty estimation using deep ensembles. arXiv."},{"key":"ref_18","unstructured":"Izmailov, P., Vikram, S., Hoffman, M.D., and Wilson, A.G.G. (2021, January 18\u201324). What Are Bayesian Neural Network Posteriors Really Like?. Proceedings of the 38th International Conference on Machine Learning, Online."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"\u00c7i\u00e7ek, \u00d6., Abdulkadir, A., Lienkamp, S.S., Brox, T., and Ronneberger, O. (2016, January 17\u201321). 3D U-Net: Learning dense volumetric segmentation from sparse annotation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Athens, Greece.","DOI":"10.1007\/978-3-319-46723-8_49"},{"key":"ref_20","unstructured":"Cabezas, M., Luo, Y., Kyle, K., Ly, L., Wang, C., and Barnett, M. (2021, January 2). Estimating lesion activity through feature similarity: A dual path Unet approach for the MSSEG2 MICCAI challenge. Proceedings of the MSSEG-2 Challenge Proceedings\u2014MICCAI 2021, Strasbourg, France."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"196","DOI":"10.1038\/s41746-023-00940-6","article-title":"A real-world clinical validation for AI-based MRI monitoring in multiple sclerosis","volume":"6","author":"Barnett","year":"2023","journal-title":"NPJ Digit. Med."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"297","DOI":"10.2307\/1932409","article-title":"Measures of the amount of ecologic association between species","volume":"26","author":"Dice","year":"1945","journal-title":"Ecology"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"102927","DOI":"10.1016\/j.media.2023.102927","article-title":"USE-Evaluator: Performance metrics for medical image segmentation models supervised by uncertain, small or empty reference annotations in neuroimaging","volume":"90","author":"Ostmeier","year":"2023","journal-title":"Med. Image Anal."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2680","DOI":"10.1109\/JBHI.2022.3151741","article-title":"Multiple Sclerosis Lesion Analysis in Brain Magnetic Resonance Images: Techniques and Clinical Applications","volume":"26","author":"Ma","year":"2022","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_25","unstructured":"Kayhan, O.S., and Gemert, J.C.v. (2020, January 13\u201319). On translation invariance in cnns: Convolutional layers can exploit absolute spatial location. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA."},{"key":"ref_26","unstructured":"Islam, M.A., Kowal, M., Jia, S., Derpanis, K.G., and Bruce, N.D. (2021, January 10\u201317). Global pooling, more than meets the eye: Position information is encoded channel-wise in CNNs. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, QC, Canada."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"8242","DOI":"10.1038\/s41598-020-64803-w","article-title":"Evaluating White Matter Lesion Segmentations with Refined S\u00f8rensen-Dice Analysis","volume":"10","author":"Carass","year":"2020","journal-title":"Sci. Rep."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1451","DOI":"10.1109\/TMI.2006.880587","article-title":"Generalized Overlap Measures for Evaluation and Validation in Medical Image Analysis","volume":"25","author":"Crum","year":"2006","journal-title":"IEEE Trans. Med. Imaging"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/6\/1981\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:16:44Z","timestamp":1760105804000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/6\/1981"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,20]]},"references-count":28,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2024,3]]}},"alternative-id":["s24061981"],"URL":"https:\/\/doi.org\/10.3390\/s24061981","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,20]]}}}