{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T02:05:19Z","timestamp":1760234719495,"version":"build-2065373602"},"reference-count":51,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2021,6,9]],"date-time":"2021-06-09T00:00:00Z","timestamp":1623196800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Data"],"abstract":"<jats:p>We propose a framework using contrastive learning as a pre-training task to perform image classification in the presence of noisy labels. Recent strategies, such as pseudo-labeling, sample selection with Gaussian Mixture models, and weighted supervised contrastive learning have, been combined into a fine-tuning phase following the pre-training. In this paper, we provide an extensive empirical study showing that a preliminary contrastive learning step brings a significant gain in performance when using different loss functions: non robust, robust, and early-learning regularized. Our experiments performed on standard benchmarks and real-world datasets demonstrate that: (i) the contrastive pre-training increases the robustness of any loss function to noisy labels and (ii) the additional fine-tuning phase can further improve accuracy, but at the cost of additional complexity.<\/jats:p>","DOI":"10.3390\/data6060061","type":"journal-article","created":{"date-parts":[[2021,6,9]],"date-time":"2021-06-09T14:16:04Z","timestamp":1623248164000},"page":"61","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["A Framework Using Contrastive Learning for Classification with Noisy Labels"],"prefix":"10.3390","volume":"6","author":[{"given":"Madalina","family":"Ciortan","sequence":"first","affiliation":[{"name":"R&D Department, EURA NOVA, 1435 Mont-Saint-Guibert, Belgium"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7976-1034","authenticated-orcid":false,"given":"Romain","family":"Dupuis","sequence":"additional","affiliation":[{"name":"R&D Department, EURA NOVA, 1435 Mont-Saint-Guibert, Belgium"}]},{"given":"Thomas","family":"Peel","sequence":"additional","affiliation":[{"name":"R&D Department, EURA NOVA, 1435 Mont-Saint-Guibert, Belgium"}]}],"member":"1968","published-online":{"date-parts":[[2021,6,9]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Mahajan, D., Girshick, R., Ramanathan, V., He, K., Paluri, M., Li, Y., Bharambe, A., and van der Maaten, L. (2018, January 8\u201314). Exploring the limits of weakly supervised pretraining. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01216-8_12"},{"key":"ref_2","unstructured":"Zhang, C., Bengio, S., Hardt, M., Recht, B., and Vinyals, O. (2016). Understanding deep learning requires rethinking generalization. arXiv."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Patrini, G., Rozza, A., Krishna Menon, A., Nock, R., and Qu, L. (2017, January 21\u201326). Making deep neural networks robust to label noise: A loss correction approach. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.240"},{"key":"ref_4","unstructured":"Goldberger, J., and Ben-Reuven, E. (2020, June 15). Training Deep Neural-Networks Using a Noise Adaptation Layer. ICLR. Available online: https:\/\/openreview.net\/forum?id=H12GRgcxg."},{"key":"ref_5","unstructured":"Xia, X., Liu, T., Wang, N., Han, B., Gong, C., Niu, G., and Sugiyama, M. (2019, January 8\u201314). Are anchor points really indispensable in label-noise learning?. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_6","unstructured":"Hendrycks, D., Mazeika, M., Wilson, D., and Gimpel, K. (2018, January 3\u20138). Using trusted data to train deep networks on labels corrupted by severe noise. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_7","unstructured":"Jiang, L., Zhou, Z., Leung, T., Li, L.J., and Fei-Fei, L. (2018, January 10\u201315). Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels. Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_8","unstructured":"Han, B., Yao, Q., Yu, X., Niu, G., Xu, M., Hu, W., Tsang, I., and Sugiyama, M. (2018, January 3\u20138). Co-teaching: Robust training of deep neural networks with extremely noisy labels. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_9","unstructured":"Li, J., Socher, R., and Hoi, S.C. (May, January 26). DivideMix: Learning with Noisy Labels as Semi-supervised Learning. Proceedings of the International Conference on Learning Representations, Virtual Event."},{"key":"ref_10","first-page":"8778","article-title":"Generalized cross entropy loss for training deep neural networks with noisy labels","volume":"31","author":"Zhang","year":"2018","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_11","unstructured":"Wang, Y., Ma, X., Chen, Z., Luo, Y., Yi, J., and Bailey, J. (November, January 27). Symmetric cross entropy for robust learning with noisy labels. Proceedings of the IEEE International Conference on Computer Vision, Seoul, Korea."},{"key":"ref_12","unstructured":"Ma, X., Huang, H., Wang, Y., Romano, S., Erfani, S., and Bailey, J. (2020, January 13\u201318). Normalized loss functions for deep learning with noisy labels. Proceedings of the 37th International Conference on Machine Learning, Virtual Event."},{"key":"ref_13","unstructured":"Liu, S., Niles-Weed, J., Razavian, N., and Fernandez-Granda, C. (2020, January 6\u201312). Early-Learning Regularization Prevents Memorization of Noisy Labels. Proceedings of the Advances in Neural Information Processing Systems, Online."},{"key":"ref_14","unstructured":"Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. (2020). A Simple Framework for Contrastive Learning of Visual Representations. arXiv."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R. (2020, January 14\u201319). Momentum contrast for unsupervised visual representation learning. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Virtual Event.","DOI":"10.1109\/CVPR42600.2020.00975"},{"key":"ref_16","unstructured":"Song, H., Kim, M., Park, D., and Lee, J.G. (2020). Learning from noisy labels with deep neural networks: A survey. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"193907","DOI":"10.1109\/ACCESS.2020.3031549","article-title":"Contrastive representation learning: A framework and review","volume":"8","author":"Healy","year":"2020","journal-title":"IEEE Access"},{"key":"ref_18","unstructured":"Arazo, E., Ortego, D., Albert, P., O\u2019Connor, N., and McGuinness, K. (2019, January 9\u201315). Unsupervised label noise modeling and loss correction. Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA."},{"key":"ref_19","unstructured":"Song, H., Kim, M., and Lee, J.G. (2019, January 9\u201315). Selfie: Refurbishing unclean samples for robust deep learning. Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA."},{"key":"ref_20","unstructured":"Arpit, D., Jastrzebski, S., Ballas, N., Krueger, D., Bengio, E., Kanwal, M.S., Maharaj, T., Fischer, A., Courville, A., and Bengio, Y. (2017, January 6\u201311). A closer look at memorization in deep networks. Proceedings of the International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_21","unstructured":"Nguyen, D.T., Mummadi, C.K., Ngo, T.P.N., Nguyen, T.H.P., Beggel, L., and Brox, T. (2019, January 6\u20139). SELF: Learning to Filter Noisy Labels with Self-Ensembling. Proceedings of the International Conference on Learning Representations, New Orleans, LA, USA."},{"key":"ref_22","unstructured":"Wang, Z., Jiang, J., Han, B., Feng, L., An, B., Niu, G., and Long, G. (2020). SemiNLL: A Framework of Noisy-Label Learning by Semi-Supervised Learning. arXiv."},{"key":"ref_23","unstructured":"Berthelot, D., Carlini, N., Goodfellow, I., Papernot, N., Oliver, A., and Raffel, C.A. (2019, January 8\u201314). Mixmatch: A holistic approach to semi-supervised learning. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Ortego, D., Arazo, E., Albert, P., O\u2019Connor, N.E., and McGuinness, K. (2020). Multi-Objective Interpolation Training for Robustness to Label Noise. arXiv.","DOI":"10.1109\/CVPR46437.2021.00654"},{"key":"ref_25","unstructured":"Zhang, H., and Yao, Q. (2020). Decoupling Representation and Classifier for Noisy Label Learning. arXiv."},{"key":"ref_26","unstructured":"Li, J., Xiong, C., and Hoi, S.C. (2020). MoPro: Webly Supervised Learning with Momentum Prototypes. arXiv."},{"key":"ref_27","unstructured":"Henaff, O. (2020, January 13\u201318). Data-efficient image recognition with contrastive predictive coding. Proceedings of the International Conference on Machine Learning, Virtual Event."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Misra, I., and Maaten, L.V.D. (2020, January 14\u201319). Self-supervised learning of pretext-invariant representations. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Virtual Event.","DOI":"10.1109\/CVPR42600.2020.00674"},{"key":"ref_29","unstructured":"Kalantidis, Y., Sariyildiz, M.B., Pion, N., Weinzaepfel, P., and Larlus, D. (2020). Hard negative mixing for contrastive learning. arXiv."},{"key":"ref_30","first-page":"21271","article-title":"Bootstrap Your Own Latent\u2014A New Approach to Self-Supervised Learning","volume":"Volume 33","author":"Larochelle","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref_31","unstructured":"Chen, X., Fan, H., Girshick, R., and He, K. (2020). Improved baselines with momentum contrastive learning. arXiv."},{"key":"ref_32","unstructured":"Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., and Joulin, A. (2020, January 6\u201312). Unsupervised Learning of Visual Features by Contrasting Cluster Assignments. Proceedings of the Thirty-Fourth Conference on Neural Information Processing Systems (NeurIPS), Online."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Ghosh, A., Kumar, H., and Sastry, P. (2017, January 4\u20139). Robust loss functions under label noise for deep neural networks. Proceedings of the AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.","DOI":"10.1609\/aaai.v31i1.10894"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1016\/j.patrec.2021.01.010","article-title":"Deep learning for real-time semantic segmentation: Application in ultrasound imaging","volume":"144","author":"Ouahabi","year":"2021","journal-title":"Pattern Recognit. Lett."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Zadeh, S.G., and Schmid, M. (2020). Bias in cross-entropy-based training of deep survival networks. IEEE Trans. Pattern Anal. Mach. Intell.","DOI":"10.1109\/TPAMI.2020.2979450"},{"key":"ref_36","unstructured":"Falcon, W., and Cho, K. (2020). A framework for contrastive self-supervised learning and designing a new approach. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 26\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Wu, Z., Xiong, Y., Yu, S.X., and Lin, D. (2018, January 19\u201321). Unsupervised feature learning via non-parametric instance discrimination. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00393"},{"key":"ref_39","unstructured":"Khosla, P., Teterwak, P., Wang, C., Sarna, A., Tian, Y., Isola, P., Maschinot, A., Liu, C., and Krishnan, D. (2020, January 6\u201312). Supervised Contrastive Learning. Proceedings of the Advances in Neural Information Processing Systems, Online."},{"key":"ref_40","unstructured":"Krizhevsky, A. (2009). Learning Multiple Layers of Features from Tiny Images. [Master\u2019s Thesis, University of Toronto]."},{"key":"ref_41","unstructured":"Li, W., Wang, L., Li, W., Agustsson, E., and Van Gool, L. (2017). Webvision database: Visual learning and understanding from web data. arXiv."},{"key":"ref_42","unstructured":"Xiao, T., Xia, T., Yang, Y., Huang, C., and Wang, X. (2015, January 8\u201310). Learning from massive noisy labeled data for image classification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA."},{"key":"ref_43","unstructured":"Wallach, H., Larochelle, H., Beygelzimer, A., d\u2019Alch\u2019e-Buc, F., Fox, E., and Garnett, R. (2019). PyTorch: An Imperative Style, High-Performance Deep Learning Library. Advances in Neural Information Processing Systems 32, Curran Associates, Inc."},{"key":"ref_44","unstructured":"Song, H., Mitsuo, N., Uchida, S., and Suehiro, D. (2020). No Regret Sample Selection with Noisy Labels. arXiv."},{"key":"ref_45","unstructured":"Yu, X., Han, B., Yao, J., Niu, G., Tsang, I., and Sugiyama, M. (2019, January 9\u201315). How does disagreement help generalization against label corruption?. Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Wei, H., Feng, L., Chen, X., and An, B. (2020). Combating noisy labels by agreement: A joint training method with co-regularization. arXiv.","DOI":"10.1109\/CVPR42600.2020.01374"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"64270","DOI":"10.1109\/ACCESS.2018.2877890","article-title":"Benchmark analysis of representative deep neural network architectures","volume":"6","author":"Bianco","year":"2018","journal-title":"IEEE Access"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Kamabattula, S.R., Devarajan, V., Namazi, B., and Sankaranarayanan, G. (2020). Identifying Training Stop Point with Noisy Labeled Data. arXiv.","DOI":"10.1109\/CSCI51800.2020.00084"},{"key":"ref_49","unstructured":"Kornblith, S., Norouzi, M., Lee, H., and Hinton, G. (2019, January 9\u201315). Similarity of neural network representations revisited. Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA."},{"key":"ref_50","unstructured":"Mitrovic, J., McWilliams, B., and Rey, M. (2020, January 6\u201314). Less can be more in contrastive learning. Proceedings of the \u201cI Can\u2019t Believe It\u2019s Not Better!\u201d NeurIPS 2020 Workshop, Virtual Event."},{"key":"ref_51","unstructured":"Zhang, H., Ciss\u00e9, M., Dauphin, Y., and Lopez-Paz, D. (May, January 30). mixup: Beyond Empirical Risk Minimization. Proceedings of the International Conference on Learning Representations (ICLR), Vancouver, BC, Canada."}],"container-title":["Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2306-5729\/6\/6\/61\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:12:32Z","timestamp":1760163152000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2306-5729\/6\/6\/61"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,9]]},"references-count":51,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2021,6]]}},"alternative-id":["data6060061"],"URL":"https:\/\/doi.org\/10.3390\/data6060061","relation":{},"ISSN":["2306-5729"],"issn-type":[{"type":"electronic","value":"2306-5729"}],"subject":[],"published":{"date-parts":[[2021,6,9]]}}}