{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,22]],"date-time":"2026-06-22T15:28:04Z","timestamp":1782142084997,"version":"3.54.5"},"reference-count":51,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2022,12,1]],"date-time":"2022-12-01T00:00:00Z","timestamp":1669852800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Imperial College London Open Access Fund"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>Self-supervised learning approaches have seen success transferring between similar medical imaging datasets, however there has been no large scale attempt to compare the transferability of self-supervised models against each other on medical images. In this study, we compare the generalisability of seven self-supervised models, two of which were trained in-domain, against supervised baselines across eight different medical datasets. We find that ImageNet pretrained self-supervised models are more generalisable than their supervised counterparts, scoring up to 10% better on medical classification tasks. The two in-domain pretrained models outperformed other models by over 20% on in-domain tasks, however they suffered significant loss of accuracy on all other tasks. Our investigation of the feature representations suggests that this trend may be due to the models learning to focus too heavily on specific areas.<\/jats:p>","DOI":"10.3390\/jimaging8120320","type":"journal-article","created":{"date-parts":[[2022,12,1]],"date-time":"2022-12-01T02:24:46Z","timestamp":1669861486000},"page":"320","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["How Well Do Self-Supervised Models Transfer to Medical Imaging?"],"prefix":"10.3390","volume":"8","author":[{"given":"Jonah","family":"Anton","sequence":"first","affiliation":[{"name":"Department of Computing, Imperial College London, London SW7 2AZ, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8411-814X","authenticated-orcid":false,"given":"Liam","family":"Castelli","sequence":"additional","affiliation":[{"name":"Department of Computing, Imperial College London, London SW7 2AZ, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mun Fai","family":"Chan","sequence":"additional","affiliation":[{"name":"Department of Computing, Imperial College London, London SW7 2AZ, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mathilde","family":"Outters","sequence":"additional","affiliation":[{"name":"Department of Computing, Imperial College London, London SW7 2AZ, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wan Hee","family":"Tang","sequence":"additional","affiliation":[{"name":"Department of Computing, Imperial College London, London SW7 2AZ, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Venus","family":"Cheung","sequence":"additional","affiliation":[{"name":"Department of Computing, Imperial College London, London SW7 2AZ, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0707-8428","authenticated-orcid":false,"given":"Pancham","family":"Shukla","sequence":"additional","affiliation":[{"name":"Department of Computing, Imperial College London, London SW7 2AZ, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rahee","family":"Walambe","sequence":"additional","affiliation":[{"name":"Symbiosis Institute of Technology, Symbiosis International University, Pune 412115, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2653-3780","authenticated-orcid":false,"given":"Ketan","family":"Kotecha","sequence":"additional","affiliation":[{"name":"Symbiosis Centre for Applied Artificial Intelligence, Symbiosis International University, Pune 412115, India"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,12,1]]},"reference":[{"key":"ref_1","first-page":"84","article-title":"ImageNet Classification with Deep Convolutional Neural Networks","volume":"25","author":"Krizhevsky","year":"2012","journal-title":"Neural Inf. Process. Syst."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., van der Maaten, L., and Weinberger, K.Q. (2017, January 21\u201326). Densely Connected Convolutional Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Girshick, R.B. (2015, January 7\u201313). Fast R-CNN. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.169"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S.K., Girshick, R.B., and Farhadi, A. (2016, January 27\u201330). You Only Look Once: Unified, Real-Time Object Detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. (2020, January 23\u201328). End-to-End Object Detection with Transformers. Proceedings of the European Conference on Computer Vision, Glasgow, UK.","DOI":"10.1007\/978-3-030-58452-8_13"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Caron, M., Touvron, H., Misra, I., J\u00e9gou, H., Mairal, J., Bojanowski, P., and Joulin, A. (2021, January 11\u201317). Emerging Properties in Self-Supervised Vision Transformers. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, Canada.","DOI":"10.1109\/ICCV48922.2021.00951"},{"key":"ref_8","unstructured":"Jain, J., Singh, A., Orlov, N., Huang, Z., Li, J., Walton, S., and Shi, H. (2021). SeMask: Semantically Masked Transformers for Semantic Segmentation. arXiv."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Yuan, Z., Yan, Y., Sonka, M., and Yang, T. (2021, January 11\u201317). Large-scale Robust Deep AUC Maximization: A New Surrogate Loss and Empirical Studies on Medical Image Classification. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, Canada.","DOI":"10.1109\/ICCV48922.2021.00303"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"101971","DOI":"10.1016\/j.media.2021.101971","article-title":"Applications of Deep Learning in Fundus Images: A Review","volume":"69","author":"Li","year":"2021","journal-title":"Med. Image Anal."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1038\/nature21056","article-title":"Dermatologist-level classification of skin cancer with deep neural networks","volume":"542","author":"Esteva","year":"2017","journal-title":"Nature"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Sun, C., Shrivastava, A., Singh, S., and Gupta, A. (2017, January 22\u201329). Revisiting Unreasonable Effectiveness of Data in Deep Learning Era. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.97"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1097\/RTI.0000000000000387","article-title":"Deep Learning Applications in Chest Radiography and Computed Tomography: Current State of the Art","volume":"34","author":"Lee","year":"2019","journal-title":"J. Thorac. Imaging"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"e1045","DOI":"10.7717\/peerj-cs.1045","article-title":"Self-supervised learning methods and applications in medical imaging analysis: A survey","volume":"8","author":"Shurrab","year":"2022","journal-title":"Peer J Computer Sci."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"4037","DOI":"10.1109\/TPAMI.2020.2992393","article-title":"Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey","volume":"43","author":"Jing","year":"2020","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_16","unstructured":"Sowrirajan, H., Yang, J., Ng, A.Y., and Rajpurkar, P. (2021, January 7\u20139). MoCo Pretraining Improves Representation and Transferability of Chest X-ray Models. Proceedings of the 4th Conference on Medical Imaging with Deep Learning, Lubeck, Germany."},{"key":"ref_17","unstructured":"Sriram, A., Muckley, M.J., Sinha, K., Shamout, F., Pineau, J., Geras, K.J., Azour, L., Aphinyanaphongs, Y., Yakubova, N., and Moore, W. (2021). COVID-19 Prognosis via Self-Supervised Representation Learning and Multi-Image Prediction. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20\u201321). ImageNet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_19","unstructured":"Raghu, M., Zhang, C., Kleinberg, J.M., and Bengio, S. (2019, January 8\u201314). Transfusion: Understanding Transfer Learning with Applications to Medical Imaging. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Ericsson, L., Gouk, H., and Hospedales, T.M. (2021, January 20\u201325). How Well Do Self-Supervised Models Transfer?. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00537"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Li, Y., and Shen, L. (2017). Skin Lesion Analysis Towards Melanoma Detection Using Deep Learning Network. Sensors, 18.","DOI":"10.3390\/s18020556"},{"key":"ref_22","unstructured":"Chen, T., Kornblith, S., Norouzi, M., and Hinton, G.E. (2020, January 12\u201318). A Simple Framework for Contrastive Learning of Visual Representations. Proceedings of the International Conference on Machine Learning, Vienna, Austria."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Misra, I., and van der Maaten, L. (2020, January 13\u201319). Self-Supervised Learning of Pretext-Invariant Representations. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00674"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R.B. (2020, January 13\u201319). Momentum Contrast for Unsupervised Visual Representation Learning. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00975"},{"key":"ref_25","first-page":"21271","article-title":"Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning","volume":"33","author":"Grill","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_26","first-page":"9912","article-title":"Unsupervised Learning of Visual Features by Contrasting Cluster Assignments","volume":"33","author":"Caron","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1109\/MSP.2021.3134634","article-title":"Self-Supervised Representation Learning: Introduction, Advances and Challenges","volume":"39","author":"Ericsson","year":"2021","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_28","first-page":"54","article-title":"How Transferable Are Self-supervised Features in Medical Image Classification Tasks?","volume":"158","author":"Truong","year":"2021","journal-title":"Proc. Mach. Learn. Res."},{"key":"ref_29","unstructured":"Chaves, L., Bissoto, A., Valle, E., and Avila, S. (2021). An Evaluation of Self-Supervised Pre-Training for Skin-Lesion Analysis. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Azizi, S., Mustafa, B., Ryan, F., Beaver, Z., Freyberg, J., Deaton, J., Loh, A., Karthikesalingam, A., Kornblith, S., and Chen, T. (2021, January 10\u201317). Big Self-Supervised Models Advance Medical Image Classification. Proceedings of the 2021 IEEE\/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.00346"},{"key":"ref_31","unstructured":"Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R.L., and Shpanskaya, K.S. (February, January 27). CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA."},{"key":"ref_32","first-page":"475","article-title":"Two public chest X-ray datasets for computer-aided screening of pulmonary diseases","volume":"4","author":"Jaeger","year":"2014","journal-title":"Quant. Imaging Med. Surg."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41597-019-0322-0","article-title":"MIMIC-CXR: A large publicly available database of labeled chest radiographs","volume":"6","author":"Johnson","year":"2019","journal-title":"Sci. Data"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Navarro, F., Watanabe, C., Shit, S., Sekuboyina, A., Peeken, J.C., Combs, S.E., and Menze, B.H. (2022, January 11\u201315). Self-Supervised Pretext Tasks in Model Robustness & Generalizability: A Revisit from Medical Imaging Perspective. Proceedings of the 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Glasgow, UK.","DOI":"10.1109\/EMBC48229.2022.9870911"},{"key":"ref_35","unstructured":"Zhao, N., Wu, Z., Lau, R.W.H., and Lin, S. (2020). What makes instance discrimination good for transfer learning?. arXiv."},{"key":"ref_36","unstructured":"Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., and Antiga, L. (2019, January 8\u201314). PyTorch: An Imperative Style, High-Performance Deep Learning Library. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_37","unstructured":"Chen, X., Fan, H., Girshick, R., and He, K. (2020). Improved Baselines with Momentum Contrastive Learning. arXiv."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., and Summers, R.M. (2017, January 21\u201326). ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.369"},{"key":"ref_39","unstructured":"Fu, H., Li, F., Orlando, J.I., Bogunovi\u0107, H., Sun, X., Liao, J., Xu, Y., Zhang, S., and Zhang, X. (2022, April 14). PALM: PAthoLogic Myopia Challenge. Available online: https:\/\/ieee-dataport.org\/documents\/palm-pathologic-myopia-challenge."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"122","DOI":"10.1016\/j.media.2019.05.010","article-title":"BACH: Grand challenge on breast cancer histology images","volume":"56","author":"Aresta","year":"2019","journal-title":"Med. Image Anal."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"E65","DOI":"10.1148\/radiol.2020200905","article-title":"Using Artificial Intelligence to Detect COVID-19 and Community-acquired Pneumonia Based on Pulmonary CT: Evaluation of the Diagnostic Accuracy","volume":"296","author":"Li","year":"2020","journal-title":"Radiology"},{"key":"ref_42","unstructured":"Kwok, S. Multiclass Classification of Breast Cancer in Whole-Slide Images. Proceedings of the Image Analysis and Recognition."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Guo, Y., Codella, N.C.F., Karlinsky, L., Smith, J.R., Rosing, T., and Feris, R.S. (2019). A New Benchmark for Evaluation of Cross-Domain Few-Shot Learning. arXiv.","DOI":"10.1007\/978-3-030-58583-9_8"},{"key":"ref_44","unstructured":"Snell, J., Swersky, K., and Zemel, R.S. (2017, January 4\u20139). Prototypical Networks for Few-shot Learning. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_45","unstructured":"Selvaraju, R.R., Das, A., Vedantam, R., Cogswell, M., Parikh, D., and Batra, D. (29, January 22). Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization. Proceedings of the IEEE international Conference on Computer Vision, Venice, Italy."},{"key":"ref_46","unstructured":"Ulyanov, D., Vedaldi, A., and Lempitsky, V.S. (2018, January 18\u201323). Deep Image Prior. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Zhang, R., Isola, P., Efros, A.A., Shechtman, E., and Wang, O. (2018, January 18\u201323). The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00068"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Ericsson, L., Gouk, H., and Hospedales, T.M. (2021). Why Do Self-Supervised Models Transfer? Investigating the Impact of Invariance on Downstream Tasks. arXiv.","DOI":"10.1109\/CVPR46437.2021.00537"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"3521","DOI":"10.1073\/pnas.1611835114","article-title":"Overcoming catastrophic forgetting in neural networks","volume":"114","author":"Kirkpatrick","year":"2016","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1109\/TIP.2003.819861","article-title":"Image quality assessment: From error visibility to structural similarity","volume":"13","author":"Wang","year":"2004","journal-title":"IEEE Trans. Image Process."},{"key":"ref_51","unstructured":"Zagoruyko, S., and Komodakis, N. (2016). Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer. arXiv."}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/8\/12\/320\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:30:53Z","timestamp":1760146253000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/8\/12\/320"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,1]]},"references-count":51,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["jimaging8120320"],"URL":"https:\/\/doi.org\/10.3390\/jimaging8120320","relation":{},"ISSN":["2313-433X"],"issn-type":[{"value":"2313-433X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,1]]}}}