{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,22]],"date-time":"2026-01-22T13:48:14Z","timestamp":1769089694598,"version":"3.49.0"},"reference-count":21,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2021,7,14]],"date-time":"2021-07-14T00:00:00Z","timestamp":1626220800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>Deep learning methods are the de facto solutions to a multitude of medical image analysis tasks. Cardiac MRI segmentation is one such application, which, like many others, requires a large number of annotated data so that a trained network can generalize well. Unfortunately, the process of having a large number of manually curated images by medical experts is both slow and utterly expensive. In this paper, we set out to explore whether expert knowledge is a strict requirement for the creation of annotated data sets on which machine learning can successfully be trained. To do so, we gauged the performance of three segmentation models, namely U-Net, Attention U-Net, and ENet, trained with different loss functions on expert and non-expert ground truth for cardiac cine\u2013MRI segmentation. Evaluation was done with classic segmentation metrics (Dice index and Hausdorff distance) as well as clinical measurements, such as the ventricular ejection fractions and the myocardial mass. The results reveal that generalization performances of a segmentation neural network trained on non-expert ground truth data is, to all practical purposes, as good as that trained on expert ground truth data, particularly when the non-expert receives a decent level of training, highlighting an opportunity for the efficient and cost-effective creation of annotations for cardiac data sets.<\/jats:p>","DOI":"10.3390\/a14070212","type":"journal-article","created":{"date-parts":[[2021,7,14]],"date-time":"2021-07-14T10:13:42Z","timestamp":1626257622000},"page":"212","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Deep Learning Based Cardiac MRI Segmentation: Do We Need Experts?"],"prefix":"10.3390","volume":"14","author":[{"given":"Youssef","family":"Skandarani","sequence":"first","affiliation":[{"name":"Laboratoire ImVIA, University of Bourgogne Franche-Comte, 21000 Dijon, France"},{"name":"CASIS Inc., 21800 Quetigny, France"}]},{"given":"Pierre-Marc","family":"Jodoin","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Sherbrooke, Sherbrooke, QC J1K 2R1, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7970-366X","authenticated-orcid":false,"given":"Alain","family":"Lalande","sequence":"additional","affiliation":[{"name":"Laboratoire ImVIA, University of Bourgogne Franche-Comte, 21000 Dijon, France"},{"name":"Department of Radiology, University Hospital of Dijon, 21000 Dijon, France"}]}],"member":"1968","published-online":{"date-parts":[[2021,7,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1016\/j.media.2017.07.005","article-title":"A survey on deep learning in medical image analysis","volume":"42","author":"Litjens","year":"2017","journal-title":"Med. Image Anal."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2514","DOI":"10.1109\/TMI.2018.2837502","article-title":"Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: Is the problem solved?","volume":"37","author":"Bernard","year":"2018","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"3703","DOI":"10.1109\/TMI.2020.3003240","article-title":"Cardiac Segmentation with Strong Anatomical Guarantees","volume":"39","author":"Painchaud","year":"2020","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Venkataramani, R., Ravishankar, H., and Anamandra, S. (2019, January 8\u201311). Towards Continuous Domain Adaptation for Medical Imaging. Proceedings of the 2019 IEEE 16th International Symposium on Biomedical Imaging, Venice, Italy.","DOI":"10.1109\/ISBI.2019.8759268"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"5217","DOI":"10.1038\/s41467-018-07619-7","article-title":"Why rankings of biomedical image analysis competitions should be interpreted with care","volume":"9","author":"Eisenmann","year":"2018","journal-title":"Nat. Commun."},{"key":"ref_6","unstructured":"(2021, July 14). Dataset List\u2014A List of the Biggest Machine Learning Datasets. Available online: https:\/\/www.datasetlist.com\/."},{"key":"ref_7","unstructured":"(2021, July 14). Amazon Mechanical Turk. Available online: https:\/\/www.mturk.com\/."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"101759","DOI":"10.1016\/j.media.2020.101759","article-title":"Deep learning with noisy labels: Exploring techniques and remedies in medical image analysis","volume":"65","author":"Karimi","year":"2020","journal-title":"Med. Image Anal."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Can, Y.B., Chaitanya, K., Mustafa, B., Koch, L.M., Konukoglu, E., and Baumgartner, C.F. (2018). Learning to Segment Medical Images with Scribble-Supervision Alone. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Springer International Publishing.","DOI":"10.1007\/978-3-030-00889-5_27"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1055\/s-0040-1702009","article-title":"Advancing Medical Imaging Informatics by Deep Learning-Based Domain Adaptation","volume":"29","author":"Choudhary","year":"2020","journal-title":"Yearb. Med. Inf."},{"key":"ref_11","first-page":"2672","article-title":"Generative adversarial nets","volume":"2","author":"Goodfellow","year":"2014","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_12","unstructured":"Skandarani, Y., Painchaud, N., Jodoin, P.M., and Lalande, A. (2020). On the effectiveness of GAN generated cardiac MRIs for segmentation. arXiv."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1437","DOI":"10.1007\/s11548-020-02223-x","article-title":"Fast interactive medical image segmentation with weakly supervised deep learning method","volume":"15","author":"Girum","year":"2020","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Sun, C., Shrivastava, A., Singh, S., and Gupta, A. (2017, January 22\u201329). Revisiting Unreasonable Effectiveness of Data in Deep Learning Era. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.97"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_16","unstructured":"Oktay, O., Schlemper, J., Folgoc, L.L., Lee, M.J., Heinrich, M., Misawa, K., Mori, K., McDonagh, S.G., Hammerla, N., and Kainz, B. (2018). Attention U-Net: Learning Where to Look for the Pancreas. arXiv."},{"key":"ref_17","unstructured":"Paszke, A., Chaurasia, A., Kim, S., and Culurciello, E. (2016). Enet: A deep neural network architecture for real-time semantic segmentation. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Ghosh, A., Kumar, H., and Sastry, P. (2017, January 4\u201310). Robust loss functions under label noise for deep neural networks. Proceedings of the AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.","DOI":"10.1609\/aaai.v31i1.10894"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Campello, V.M., Gkontra, P., Izquierdo, C., Mart\u00edn-Isla, C., Sojoudi, A., Full, P.M., Maier-Hein, K., Zhang, Y., He, Z., and Ma, J. (2021). Multi-Centre, Multi-Vendor and Multi-Disease Cardiac Segmentation: The M&Ms Challenge. IEEE Trans. Med. Imaging.","DOI":"10.1109\/TMI.2021.3090082"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1117\/1.JMI.5.3.034002","article-title":"Large-scale medical image annotation with crowd-powered algorithms","volume":"5","author":"Heim","year":"2018","journal-title":"J. Med. Imaging"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1007\/s11548-016-1445-9","article-title":"Crowdsourcing for error detection in cortical surface delineations","volume":"12","author":"Ganz","year":"2016","journal-title":"Int. J. Comput. Assist. Radiol. Surg."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/14\/7\/212\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:30:37Z","timestamp":1760164237000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/14\/7\/212"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,14]]},"references-count":21,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2021,7]]}},"alternative-id":["a14070212"],"URL":"https:\/\/doi.org\/10.3390\/a14070212","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,7,14]]}}}