{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,4]],"date-time":"2026-06-04T21:29:49Z","timestamp":1780608589186,"version":"3.54.1"},"reference-count":34,"publisher":"MDPI AG","issue":"21","license":[{"start":{"date-parts":[[2022,11,1]],"date-time":"2022-11-01T00:00:00Z","timestamp":1667260800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Duke University Bass Connections, Climate+, and Data+ programs"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Transfer learning has been shown to be an effective method for achieving high-performance models when applying deep learning to remote sensing data. Recent research has demonstrated that representations learned through self-supervision transfer better than representations learned on supervised classification tasks. However, little research has focused explicitly on applying self-supervised encoders to remote sensing tasks. Using three diverse remote sensing datasets, we compared the performance of encoders pre-trained through both supervision and self-supervision on ImageNet, then fine-tuned on a final remote sensing task. Furthermore, we explored whether performance benefited from further pre-training on remote sensing data. Our experiments used SwAV due to its comparably lower computational requirements, as this method would prove most easily replicable by practitioners. We show that an encoder pre-trained on ImageNet using self-supervision transfers better than one pre-trained using supervision on three diverse remote sensing applications. Moreover, self-supervision on the target data alone as a pre-training step seldom boosts performance beyond this transferred encoder. We attribute this inefficacy to the lower diversity and size of remote sensing datasets, compared to ImageNet. In conclusion, we recommend that researchers use self-supervised representations for transfer learning on remote sensing data and that future research should focus on ways to increase performance further using self-supervision.<\/jats:p>","DOI":"10.3390\/rs14215500","type":"journal-article","created":{"date-parts":[[2022,11,2]],"date-time":"2022-11-02T08:15:12Z","timestamp":1667376912000},"page":"5500","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Self-Supervised Encoders Are Better Transfer Learners in Remote Sensing Applications"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8680-1527","authenticated-orcid":false,"given":"Zachary D.","family":"Calhoun","sequence":"first","affiliation":[{"name":"Department of Civil and Environmental Engineering, Duke University, 121 Hudson Hall, Science Dr, Durham, NC 27708, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2249-9255","authenticated-orcid":false,"given":"Saad","family":"Lahrichi","sequence":"additional","affiliation":[{"name":"Division of Natural and Applied Sciences, Duke Kunshan University, No. 8 Duke Ave., Kunshan 215316, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7113-9208","authenticated-orcid":false,"given":"Simiao","family":"Ren","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, Duke University, Durham, NC 27708, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7851-4920","authenticated-orcid":false,"given":"Jordan M.","family":"Malof","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Montana, Missoula, MT 59812, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9847-0243","authenticated-orcid":false,"given":"Kyle","family":"Bradbury","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, Duke University, Durham, NC 27708, USA"},{"name":"Nicholas Institute for Energy, Environment & Sustainability, Duke University, Durham, NC 27708, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,11,1]]},"reference":[{"key":"ref_1","first-page":"63","article-title":"Generalizing from a Few Examples: A Survey on Few-shot Learning","volume":"53","author":"Wang","year":"2020","journal-title":"ACM Comput. Surv."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Zhang, P., Bai, Y., Wang, D., Bai, B., and Li, Y. (2021). Few-Shot Classification of Aerial Scene Images via Meta-Learning. Remote Sens., 13.","DOI":"10.20944\/preprints202010.0033.v1"},{"key":"ref_3","first-page":"6214","article-title":"Low-Shot Learning for the Semantic Segmentation of Remote Sensing Imagery","volume":"56","author":"Kemker","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_4","unstructured":"Miller, E., Matsakis, N., and Viola, P. (2000, January 13\u201315). Learning from one example through shared densities on transforms. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2000 (Cat. No.PR00662), Hilton Head Island, SC, USA."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1186\/s40537-019-0197-0","article-title":"A survey on Image Data Augmentation for Deep Learning","volume":"6","author":"Shorten","year":"2019","journal-title":"J. Big Data"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Kong, F., Huang, B., Bradbury, K., and Malof, J. (2020, January 1\u20135). The Synthinel-1 dataset: A collection of high resolution synthetic overhead imagery for building segmentation. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision, Snowmass Village, CO, USA.","DOI":"10.1109\/WACV45572.2020.9093339"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Xie, M., Jean, N., Burke, M., Lobell, D., and Ermon, S. (2016, January 12\u201317). Transfer Learning from Deep Features for Remote Sensing and Poverty Mapping. Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.","DOI":"10.1609\/aaai.v30i1.9906"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Pires de Lima, R., and Marfurt, K. (2020). Convolutional Neural Network for Remote-Sensing Scene Classification: Transfer Learning Analysis. Remote Sens., 12.","DOI":"10.3390\/rs12010086"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"ImageNet Large Scale Visual Recognition Challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Ericsson, L., Gouk, H., and Hospedales, T.M. (2021, January 20\u201325). How Well Do Self-Supervised Models Transfer?. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00537"},{"key":"ref_11","first-page":"1399","article-title":"Do we Still Need Imagenet Pre-Training in Remote Sensing Scene Classification?","volume":"43B3","year":"2022","journal-title":"ISPRS\u2014Int. Arch. Photogramm. Remote. Sens. Spat. Inf. Sci."},{"key":"ref_12","unstructured":"Gidaris, S., Singh, P., and Komodakis, N. (2022, January 25\u201329). Unsupervised Representation Learning by Predicting Image Rotations. Proceedings of the International Conference on Learning Representations, Virtual."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Leibe, B., Matas, J., Sebe, N., and Welling, M. (2016). Colorful Image Colorization. Computer Vision\u2014ECCV 2016, Springer International Publishing. Lecture Notes in Computer Science.","DOI":"10.1007\/978-3-319-46478-7"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Pathak, D., Kr\u00e4henb\u00fchl, P., Donahue, J., Darrell, T., and Efros, A.A. (2016, January 27\u201330). Context Encoders: Feature Learning by Inpainting. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.278"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Leibe, B., Matas, J., Sebe, N., and Welling, M. (2016). Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles. Computer Vision\u2014ECCV 2016, Springer International Publishing. Lecture Notes in Computer Science.","DOI":"10.1007\/978-3-319-46487-9"},{"key":"ref_16","unstructured":"Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. (2020, January 13\u201318). A Simple Framework for Contrastive Learning of Visual Representations. Proceedings of the 37th International Conference on Machine Learning, PMLR 2020, Virtual."},{"key":"ref_17","unstructured":"Chen, T., Kornblith, S., Swersky, K., Norouzi, M., and Hinton, G. (2020, January 6\u201312). Big self-supervised models are strong semi-supervised learners. Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS\u201920, Vancouver, BC, Canada."},{"key":"ref_18","first-page":"21271","article-title":"Bootstrap Your Own Latent\u2014A New Approach to Self-Supervised Learning","volume":"Volume 33","author":"Grill","year":"2020","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"},{"key":"ref_19","unstructured":"Jean, N., Wang, S., Samar, A., Azzari, G., Lobell, D., and Ermon, S. (February, January 27). Tile2Vec: Unsupervised Representation Learning for Spatially Distributed Data. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Ayush, K., Uzkent, B., Meng, C., Tanmay, K., Burke, M., Lobell, D., and Ermon, S. (2021, January 10\u201317). Geography-Aware Self-Supervised Learning. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, BC, Canada.","DOI":"10.1109\/ICCV48922.2021.01002"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Ma\u00f1as, O., Lacoste, A., Gir\u00f3-i Nieto, X., Vazquez, D., and Rodr\u00edguez, P. (2021, January 10\u201317). Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Data. Proceedings of the 2021 IEEE\/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.00928"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Reed, C.J., Yue, X., Nrusimha, A., Ebrahimi, S., Vijaykumar, V., Mao, R., Li, B., Zhang, S., Guillory, D., and Metzger, S. (2022, January 4\u20138). Self-Supervised Pretraining Improves Self-Supervised Pretraining. Proceedings of the 2022 IEEE\/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA.","DOI":"10.1109\/WACV51458.2022.00112"},{"key":"ref_23","first-page":"9912","article-title":"Unsupervised Learning of Visual Features by Contrasting Cluster Assignments","volume":"Volume 33","author":"Caron","year":"2020","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Navab, N., Hornegger, J., Wells, W.M., and Frangi, A.F. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer-Assisted Intervention\u2014MICCAI 2015, Springer International Publishing. Lecture Notes in Computer Science.","DOI":"10.1007\/978-3-319-24571-3"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Huang, B., Lu, K., Audeberr, N., Khalel, A., Tarabalka, Y., Malof, J., Boulch, A., Le Saux, B., Collins, L., and Bradbury, K. (2018, January 22\u201327). Large-Scale Semantic Classification: Outcome of the First Year of Inria Aerial Image Labeling Benchmark. Proceedings of the IGARSS 2018\u20142018 IEEE International Geoscience and Remote Sensing Symposium, Valencia, Spain.","DOI":"10.1109\/IGARSS.2018.8518525"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Milletari, F., Navab, N., and Ahmadi, S.A. (2016, January 25\u201328). V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation. Proceedings of the 2016 Fourth International Conference on 3D Vision (3DV), Stanford, CA, USA.","DOI":"10.1109\/3DV.2016.79"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Maggiori, E., Tarabalka, Y., Charpiat, G., and Alliez, P. (2017, January 23\u201328). Can semantic labeling methods generalize to any city? The inria aerial image labeling benchmark. Proceedings of the 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Fort Worth, TX, USA.","DOI":"10.1109\/IGARSS.2017.8127684"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"160106","DOI":"10.1038\/sdata.2016.106","article-title":"Distributed solar photovoltaic array location and extent dataset for remote sensing object identification","volume":"3","author":"Bradbury","year":"2016","journal-title":"Sci. Data"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Aung, H.L., Uzkent, B., Burke, M., Lobell, D., and Ermon, S. (2020, January 14\u201319). Farm Parcel Delineation Using Spatio-temporal Convolutional Networks. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, WA, USA.","DOI":"10.1109\/CVPRW50498.2020.00046"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Newell, A., and Deng, J. (2020, January 14\u201319). How Useful Is Self-Supervised Pretraining for Visual Tasks?. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00737"},{"key":"ref_32","unstructured":"Huh, M., Agrawal, P., and Efros, A.A. (2016). What makes ImageNet good for transfer learning?. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Caron, M., Touvron, H., Misra, I., J\u00e9gou, H., Mairal, J., Bojanowski, P., and Joulin, A. (2021, January 10\u201317). Emerging Properties in Self-Supervised Vision Transformers. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.00951"},{"key":"ref_34","unstructured":"Ridnik, T., Ben-Baruch, E., Noy, A., and Zelnik, L. (2021). ImageNet-21K Pretraining for the Masses. arXiv."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/21\/5500\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:07:11Z","timestamp":1760144831000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/21\/5500"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,1]]},"references-count":34,"journal-issue":{"issue":"21","published-online":{"date-parts":[[2022,11]]}},"alternative-id":["rs14215500"],"URL":"https:\/\/doi.org\/10.3390\/rs14215500","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,1]]}}}