{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T13:55:52Z","timestamp":1770990952168,"version":"3.50.1"},"reference-count":55,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2024,3,10]],"date-time":"2024-03-10T00:00:00Z","timestamp":1710028800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Innovative Research Group Project of the National Science Foundation of China","award":["51821003"],"award-info":[{"award-number":["51821003"]}]},{"name":"Innovative Research Group Project of the National Science Foundation of China","award":["201701D121065"],"award-info":[{"award-number":["201701D121065"]}]},{"name":"Innovative Research Group Project of the National Science Foundation of China","award":["20210302124329"],"award-info":[{"award-number":["20210302124329"]}]},{"name":"National Science Foundation of Shanxi Province","award":["51821003"],"award-info":[{"award-number":["51821003"]}]},{"name":"National Science Foundation of Shanxi Province","award":["201701D121065"],"award-info":[{"award-number":["201701D121065"]}]},{"name":"National Science Foundation of Shanxi Province","award":["20210302124329"],"award-info":[{"award-number":["20210302124329"]}]},{"name":"Fundamental Research Program of Shanxi Province","award":["51821003"],"award-info":[{"award-number":["51821003"]}]},{"name":"Fundamental Research Program of Shanxi Province","award":["201701D121065"],"award-info":[{"award-number":["201701D121065"]}]},{"name":"Fundamental Research Program of Shanxi Province","award":["20210302124329"],"award-info":[{"award-number":["20210302124329"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>In the realm of Earth observation and remote sensing data analysis, the advancement of hyperspectral imaging (HSI) classification technology is of paramount importance. Nevertheless, the intricate nature of hyperspectral data, coupled with the scarcity of labeled data, presents significant challenges in this domain. To mitigate these issues, we introduce a self-supervised learning algorithm predicated on a spectral transformer for HSI classification under conditions of limited labeled data, with the objective of enhancing the efficacy of HSI classification. The S3L algorithm operates in two distinct phases: pretraining and fine-tuning. During the pretraining phase, the algorithm learns the spatial representation of HSI from unlabeled data, utilizing a masking mechanism and a spectral transformer, thereby augmenting the sequence dependence of spectral features. Subsequently, in the fine-tuning phase, labeled data is employed to refine the pretrained weights, thereby improving the precision of HSI classification. Within the comprehensive encoder\u2013decoder framework, we propose a novel spectral transformer module specifically engineered to synergize spatial feature extraction with spectral domain analysis. This innovative module adeptly navigates the complex interplay among various spectral bands, capturing both global and sequential spectral dependencies. Uniquely, it incorporates a gated recurrent unit (GRU) layer within the encoder to enhance its ability to process spectral sequences. Our experimental evaluations across several public datasets reveal that our proposed method, distinguished by its spectral transformer, achieves superior classification performance, particularly in scenarios with limited labeled samples, outperforming existing state-of-the-art approaches.<\/jats:p>","DOI":"10.3390\/rs16060970","type":"journal-article","created":{"date-parts":[[2024,3,11]],"date-time":"2024-03-11T08:56:41Z","timestamp":1710147401000},"page":"970","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":20,"title":["S3L: Spectrum Transformer for Self-Supervised Learning in Hyperspectral Image Classification"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-0685-6121","authenticated-orcid":false,"given":"Hufeng","family":"Guo","sequence":"first","affiliation":[{"name":"State Key Laboratory of Dynamic Measurement Technology, School of Instrument and Electronics, North University of China, Taiyuan 030051, China"},{"name":"Department of Transportation Information Engineering, Henan College of Transportation, Zhengzhou 451460, China"}]},{"given":"Wenyi","family":"Liu","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Dynamic Measurement Technology, School of Instrument and Electronics, North University of China, Taiyuan 030051, China"}]}],"member":"1968","published-online":{"date-parts":[[2024,3,10]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1109\/MGRS.2016.2616418","article-title":"Advanced spectral classifiers for hyperspectral images: A review","volume":"5","author":"Ghamisi","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1109\/MGRS.2018.2854840","article-title":"New frontiers in spectral-spatial hyperspectral image classification: The latest advances based on mathematical morphology, Markov random fields, segmentation, sparse representation, and deep learning","volume":"6","author":"Ghamisi","year":"2018","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"3854635","DOI":"10.1155\/2022\/3854635","article-title":"Hyperspectral image classification: Potentials, challenges, and future directions","volume":"2022","author":"Datta","year":"2022","journal-title":"Comput. Intell. Neurosci."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1109\/MGRS.2023.3269979","article-title":"Taking Artificial Intelligence Into Space Through Objective Selection of Hyperspectral Earth Observation Applications: To bring the \u201cbrain\u201d close to the \u201ceyes\u201d of satellite missions","volume":"11","author":"Wijata","year":"2023","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1109\/JSTARS.2013.2262926","article-title":"A kernel-based feature selection method for SVM with RBF kernel for hyperspectral image classification","volume":"7","author":"Kuo","year":"2013","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1778","DOI":"10.1109\/TGRS.2004.831865","article-title":"Classification of hyperspectral remote sensing images with support vector machines","volume":"42","author":"Melgani","year":"2004","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1082","DOI":"10.1109\/JSTARS.2018.2809781","article-title":"Cascaded random forest for hyperspectral image classification","volume":"11","author":"Zhang","year":"2018","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1279","DOI":"10.1109\/TGRS.2009.2031812","article-title":"A nonparametric feature extraction and its application to nearest neighbor classification for hyperspectral image data","volume":"48","author":"Yang","year":"2009","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"7140","DOI":"10.1109\/TGRS.2017.2743102","article-title":"PCA-based edge-preserving features for hyperspectral image classification","volume":"55","author":"Kang","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"3516","DOI":"10.1109\/TGRS.2017.2675902","article-title":"Learning to diversify deep belief networks for hyperspectral image classification","volume":"55","author":"Zhong","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"3585","DOI":"10.1109\/TCSVT.2023.3237328","article-title":"SACF-Net: Skip-Attention Based Correspondence Filtering Network for Point Cloud Registration","volume":"33","author":"Wu","year":"2023","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Qing, Y., and Liu, W. (2021). Hyperspectral image classification based on multi-scale residual network with attention mechanism. Remote Sens., 13.","DOI":"10.3390\/rs13030335"},{"key":"ref_13","first-page":"5514215","article-title":"Local similarity-based spatial\u2013spectral fusion hyperspectral image classification with deep CNN and Gabor filtering","volume":"60","author":"Bhatti","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_14","first-page":"5512914","article-title":"Adaptive spatial pyramid constraint for hyperspectral image classification with limited training samples","volume":"60","author":"Yue","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"11709","DOI":"10.1109\/TCYB.2021.3070577","article-title":"A spectral-spatial-dependent global learning framework for insufficient and imbalanced hyperspectral image classification","volume":"52","author":"Zhu","year":"2021","journal-title":"IEEE Trans. Cybern."},{"key":"ref_16","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017, January 4\u20139). Attention is all you need. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_17","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"5522214","DOI":"10.1109\/TGRS.2022.3221534","article-title":"Spectral\u2013spatial feature tokenization transformer for hyperspectral image classification","volume":"60","author":"Sun","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_19","first-page":"5528715","article-title":"Hyperspectral image transformer classification networks","volume":"60","author":"Yang","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Tao, Q., and Yin, Y. (2024). A Lightweight Man-Overboard Detection and Tracking Model Using Aerial Images for Maritime Search and Rescue. Remote Sens., 16.","DOI":"10.3390\/rs16010165"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Wang, J., Wang, X., Guan, J., Zhang, L., Zhang, F., and Chang, T. (2024). STPF-Net: Short-Term Precipitation Forecast Based on a Recurrent Neural Network. Remote Sens., 16.","DOI":"10.3390\/rs16010052"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"7606","DOI":"10.1109\/TGRS.2014.2315209","article-title":"Manifold-Based Sparse Representation for Hyperspectral Image Classification","volume":"52","author":"Tang","year":"2014","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"3235","DOI":"10.1109\/TGRS.2015.2514161","article-title":"Nonlinear Multiple Kernel Learning with Multiple-Structure-Element Extended Morphological Profiles for Hyperspectral Image Classification","volume":"54","author":"Gu","year":"2016","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2112","DOI":"10.1109\/TGRS.2008.916629","article-title":"Supervised Classification of Remotely Sensed Imagery Using a Modified k-NN Technique","volume":"46","author":"Samaniego","year":"2008","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_25","unstructured":"Ren, Y., Zhang, Y., Wei, W., and Li, L. (2014, January 8\u20139). A spectral-spatial hyperspectral data classification approach using random forest with label constraints. Proceedings of the 2014 IEEE Workshop on Electronics, Computer and Applications, Ottawa, ON, Canada."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Banki, M.H., and Shirazi, A.A.B. (2010, January 26\u201328). New kernel function for hyperspectral image classification. Proceedings of the 2010 the 2nd International Conference on Computer and Automation Engineering (ICCAE), Singapore.","DOI":"10.1109\/ICCAE.2010.5451241"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Hsu, P.H., and Cheng, Y.Y. (August, January 28). Hyperspectral Image Classification via Joint Sparse Representation. Proceedings of the IGARSS 2019\u20142019 IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan.","DOI":"10.1109\/IGARSS.2019.8898445"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"401","DOI":"10.1080\/01431161.2013.869633","article-title":"Exploring ELM-based spatial\u2013spectral classification of hyperspectral images","volume":"35","author":"Dora","year":"2014","journal-title":"Int. J. Remote Sens."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"641","DOI":"10.1109\/LGRS.2016.2532380","article-title":"Hyperspectral Image Classification With Robust Sparse Representation","volume":"13","author":"Li","year":"2016","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"5525921","DOI":"10.1109\/TGRS.2022.3160513","article-title":"Evolving block-based convolutional neural network for hyperspectral image classification","volume":"60","author":"Lu","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_31","first-page":"4017605","article-title":"Robust matching for SAR and optical images using multiscale convolutional gradient features","volume":"19","author":"Zhou","year":"2021","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1348","DOI":"10.1109\/TCCN.2022.3176640","article-title":"Signal modulation classification based on the transformer network","volume":"8","author":"Cai","year":"2022","journal-title":"IEEE Trans. Cogn. Commun. Netw."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"4340","DOI":"10.1109\/TGRS.2020.3016820","article-title":"More diverse means better: Multimodal deep learning meets remote-sensing imagery classification","volume":"59","author":"Hong","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_34","first-page":"5517010","article-title":"Convolutional neural networks for multimodal remote sensing data classification","volume":"60","author":"Wu","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_35","first-page":"6005705","article-title":"Dual-channel convolution network with image-based global learning framework for hyperspectral image classification","volume":"19","author":"Yu","year":"2021","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Qing, Y., Huang, Q., Feng, L., Qi, Y., and Liu, W. (2022). Multiscale Feature Fusion Network Incorporating 3D Self-Attention for Hyperspectral Image Classification. Remote Sens., 14.","DOI":"10.3390\/rs14030742"},{"key":"ref_37","first-page":"5514715","article-title":"Spectral\u2013spatial transformer network for hyperspectral image classification: A factorized architecture search framework","volume":"60","author":"Zhong","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"5537415","DOI":"10.1109\/TGRS.2022.3203476","article-title":"Spatial\u2013spectral transformer with cross-attention for hyperspectral image classification","volume":"60","author":"Peng","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Qing, Y., Liu, W., Feng, L., and Gao, W. (2021). Improved transformer net for hyperspectral image classification. Remote Sens., 13.","DOI":"10.3390\/rs13112216"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1626","DOI":"10.1109\/TMM.2023.3284591","article-title":"Self-Supervised Intra-Modal and Cross-Modal Contrastive Learning for Point Cloud Understanding","volume":"26","author":"Wu","year":"2024","journal-title":"IEEE Trans. Multimed."},{"key":"ref_41","unstructured":"Teng, J., Huang, W., and He, H. (2022, January 28\u201330). Can pretext-based self-supervised learning be boosted by downstream data? A theoretical analysis. Proceedings of the International Conference on Artificial Intelligence and Statistics, Virtual."},{"key":"ref_42","unstructured":"Bansal, Y., Kaplun, G., and Barak, B. (2020). For self-supervised learning, rationality implies generalization, provably. arXiv."},{"key":"ref_43","unstructured":"Huang, W., Yi, M., Zhao, X., and Jiang, Z. (2021). Towards the generalization of contrastive self-supervised learning. arXiv."},{"key":"ref_44","unstructured":"Pandey, P., Pai, A., Bhatt, N., Das, P., Makharia, G., and Prathosh, A.P. (2021). Contrastive semi-supervised learning for 2D medical image segmentation. arXiv."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Park, S., Han, S., Kim, S., Kim, D., Park, S., Hong, S., and Cha, M. (2021, January 20\u201325). Improving Unsupervised Image Clustering With Robust Learning. Proceedings of the 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.01210"},{"key":"ref_46","unstructured":"Jenni, S., Meishvili, G., and Favaro, P. (2020). Computer Vision\u2014ECCV 2020, Proceedings of the 16th European Conference, Glasgow, UK, 23\u201328 August 2020, Springer."},{"key":"ref_47","first-page":"5679","article-title":"Self-supervised co-training for video representation learning","volume":"33","author":"Han","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_48","unstructured":"Xie, S., Gu, J., Guo, D., Qi, C.R., Guibas, L., and Litany, O. (2020). Computer Vision\u2014ECCV 2020, Proceedings of the 16th European Conference, Glasgow, UK, 23\u201328 August 2020, Springer. Proceedings, Part III 16."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"118","DOI":"10.1109\/MGRS.2019.2911100","article-title":"Hyperspectral Band Selection: A Review","volume":"7","author":"Sun","year":"2019","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1109\/LGRS.2019.2918719","article-title":"HybridSN: Exploring 3-D\u20132-D CNN feature hierarchy for hyperspectral image classification","volume":"17","author":"Roy","year":"2019","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_51","first-page":"5516417","article-title":"A semisupervised Siamese network for hyperspectral image classification","volume":"60","author":"Jia","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Wang, W., Dou, S., Jiang, Z., and Sun, L. (2018). A fast dense spectral\u2013spatial convolution network framework for hyperspectral images classification. Remote Sens., 10.","DOI":"10.3390\/rs10071068"},{"key":"ref_53","first-page":"5501618","article-title":"Deep Cross-Domain Few-Shot Learning for Hyperspectral Image Classification","volume":"60","author":"Li","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Hu, X., Li, T., Zhou, T., Liu, Y., and Peng, Y. (2021). Contrastive learning based on transformer for hyperspectral image classification. Appl. Sci., 11.","DOI":"10.3390\/app11188670"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Li, R., Zheng, S., Duan, C., Yang, Y., and Wang, X. (2020). Classification of Hyperspectral Image Based on Double-Branch Dual-Attention Mechanism Network. Remote Sens., 12.","DOI":"10.20944\/preprints201912.0059.v2"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/6\/970\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:11:34Z","timestamp":1760105494000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/6\/970"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,10]]},"references-count":55,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2024,3]]}},"alternative-id":["rs16060970"],"URL":"https:\/\/doi.org\/10.3390\/rs16060970","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,10]]}}}