{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,18]],"date-time":"2026-04-18T15:23:10Z","timestamp":1776525790338,"version":"3.51.2"},"reference-count":50,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2021,1,30]],"date-time":"2021-01-30T00:00:00Z","timestamp":1611964800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61971164"],"award-info":[{"award-number":["61971164"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Recently, a great many deep convolutional neural network (CNN)-based methods have been proposed for hyperspectral image (HSI) classification. Although the proposed CNN-based methods have the advantages of spatial feature extraction, they are difficult to handle the sequential data with and CNNs are not good at modeling the long-range dependencies. However, the spectra of HSI are a kind of sequential data, and HSI usually contains hundreds of bands. Therefore, it is difficult for CNNs to handle HSI processing well. On the other hand, the Transformer model, which is based on an attention mechanism, has proved its advantages in processing sequential data. To address the issue of capturing relationships of sequential spectra in HSI in a long distance, in this study, Transformer is investigated for HSI classification. Specifically, in this study, a new classification framework titled spatial-spectral Transformer (SST) is proposed for HSI classification. In the proposed SST, a well-designed CNN is used to extract the spatial features, and a modified Transformer (a Transformer with dense connection, i.e., DenseTransformer) is proposed to capture sequential spectra relationships, and multilayer perceptron is used to finish the final classification task. Furthermore, dynamic feature augmentation, which aims to alleviate the overfitting problem and therefore generalize the model well, is proposed and added to the SST (SST-FA). In addition, to address the issue of limited training samples in HSI classification, transfer learning is combined with SST, and another classification framework titled transferring-SST (T-SST) is proposed. At last, to mitigate the overfitting problem and improve the classification accuracy, label smoothing is introduced for the T-SST-based classification framework (T-SST-L). The proposed SST, SST-FA, T-SST, and T-SST-L are tested on three widely used hyperspectral datasets. The obtained results reveal that the proposed models provide competitive results compared to the state-of-the-art methods, which shows that the concept of Transformer opens a new window for HSI classification.<\/jats:p>","DOI":"10.3390\/rs13030498","type":"journal-article","created":{"date-parts":[[2021,1,30]],"date-time":"2021-01-30T21:15:51Z","timestamp":1612041351000},"page":"498","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":408,"title":["Spatial-Spectral Transformer for Hyperspectral Image Classification"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0455-4230","authenticated-orcid":false,"given":"Xin","family":"He","sequence":"first","affiliation":[{"name":"School of Electronics and Information Engineering, Harbin Institute of Technology, Harbin 150001, China"}]},{"given":"Yushi","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Electronics and Information Engineering, Harbin Institute of Technology, Harbin 150001, China"}]},{"given":"Zhouhan","family":"Lin","sequence":"additional","affiliation":[{"name":"Shanghai Jiao Tong University, Shanghai 201100, China"}]}],"member":"1968","published-online":{"date-parts":[[2021,1,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1109\/MGRS.2013.2244672","article-title":"Hyperspectral remote sensing data analysis and future challenges","volume":"1","author":"Plaza","year":"2013","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"8290","DOI":"10.1109\/TGRS.2019.2920112","article-title":"Hyperspectral image classification via compressive sensing","volume":"57","author":"Della","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1246","DOI":"10.1109\/TGRS.2003.813206","article-title":"Preprocessing EO-1 Hyperion hyperspectral data to support the application of agricultural indexes","volume":"41","author":"Datt","year":"2003","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1007\/s10658-011-9878-z","article-title":"Recent advances in sensing plant diseases for precision crop protection","volume":"133","author":"Mahlein","year":"2012","journal-title":"Eur. J. Plant Pathol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2843","DOI":"10.1109\/TGRS.2013.2266672","article-title":"Consistency of measurements of wavelength position from hyperspectral imagery: Use of the ferric iron crystal field absorption at \u223c900 nm as an indicator of mineralogy","volume":"52","author":"Murphy","year":"2014","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"010901","DOI":"10.1117\/1.JBO.19.1.010901","article-title":"Medical hyperspectral imaging: A review","volume":"19","author":"Lu","year":"2014","journal-title":"J. Biomed. Opt."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1016\/j.rse.2015.05.023","article-title":"Measuring freshwater aquatic ecosystems: The need for a hyperspectral global mapping satellite mission","volume":"167","author":"Hestir","year":"2015","journal-title":"Remote Sens. Environ."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1109\/MSP.2013.2279179","article-title":"Advances in Hyperspectral image classification: Earth monitoring with statistical learning methods","volume":"31","author":"Tuia","year":"2014","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1109\/MGRS.2018.2854840","article-title":"New frontiers in spectral-spatial hyperspectral image classification: The latest advances based on mathematical morphology, Markov random fields, segmentation, sparse representation, and deep learning","volume":"6","author":"Ghamisi","year":"2018","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1109\/MGRS.2020.2979764","article-title":"Feature extraction for hyperspectral imagery: The evolution from shallow to deep: Overview and toolbox","volume":"8","author":"Rasti","year":"2020","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1109\/MGRS.2019.2912563","article-title":"Deep learning for classification of hyperspectral data: A comparative review","volume":"7","author":"Audebert","year":"2019","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1778","DOI":"10.1109\/TGRS.2004.831865","article-title":"Classification of hyperspectral remote sensing images with support vector machines","volume":"42","author":"Melgani","year":"2004","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_13","unstructured":"Gualtieri, J., and Chettri, S. (2000, January 24\u201328). Support vector machines for classification of hyperspectral data. Proceedings of the IGARSS, Honolulu, HI, USA."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1109\/MGRS.2017.2762087","article-title":"Advances in hyperspectral image and signal processing: A comprehensive overview of the state of the art","volume":"5","author":"Ghamisi","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"3804","DOI":"10.1109\/TGRS.2008.922034","article-title":"Spectral and spatial classification of hyperspectral data using SVMs and morphological profiles","volume":"46","author":"Fauvel","year":"2008","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"480","DOI":"10.1109\/TGRS.2004.842478","article-title":"Classification of hyperspectral data from urban areas based on extended morphological profiles","volume":"43","author":"Benediktsson","year":"2005","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"542","DOI":"10.1109\/LGRS.2010.2091253","article-title":"Classification of hyperspectral images by using extended morphological attribute profiles and independent component analysis","volume":"8","author":"Mura","year":"2011","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1803","DOI":"10.1109\/TGRS.2017.2768479","article-title":"Extinction profiles fusion for hyperspectral images classification","volume":"56","author":"Fang","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"4073","DOI":"10.1109\/JSTARS.2016.2517204","article-title":"Spectral-spatial classification of hyperspectral image based on deep auto-encoder","volume":"9","author":"Ma","year":"2016","journal-title":"IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens."},{"key":"ref_20","first-page":"3516","article-title":"Learning to diversify deep belief networks for hyperspectral image classification","volume":"55","author":"Zhong","year":"2017","journal-title":"IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"6232","DOI":"10.1109\/TGRS.2016.2584107","article-title":"Deep feature extraction and classification of hyperspectral images based on convolutional neural networks","volume":"54","author":"Chen","year":"2016","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"3599","DOI":"10.1109\/TGRS.2018.2886022","article-title":"A CNN with multiscale convolution and diversified metric for hyperspectral image classification","volume":"57","author":"Gong","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"3639","DOI":"10.1109\/TGRS.2016.2636241","article-title":"Deep recurrent neural networks for hyperspectral image classification","volume":"55","author":"Mou","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"4141","DOI":"10.1109\/JSTARS.2018.2844873","article-title":"Spatial sequential recurrent neural networks for hyperspectral image classification","volume":"11","author":"Zhang","year":"2018","journal-title":"IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"5329","DOI":"10.1109\/TGRS.2019.2899057","article-title":"Classification of hyperspectral images based on multiclass spatial\u2013spectral generative adversarial networks","volume":"57","author":"Feng","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"5046","DOI":"10.1109\/TGRS.2018.2805286","article-title":"Generative adversarial networks for hyperspectral image classification","volume":"56","author":"Zhu","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Zhu, K., Chen, Y., Ghamisi, P., Jia, X., and Benediktsson, J.A. (2019). Deep convolutional capsule network for hyperspectral image spectral and spectral-spatial classification. Remote Sens., 11.","DOI":"10.3390\/rs11030223"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"2145","DOI":"10.1109\/TGRS.2018.2871782","article-title":"Capsule networks for hyperspectral image classification","volume":"57","author":"Paoletti","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"258619","DOI":"10.1155\/2015\/258619","article-title":"Deep convolutional neural networks for hyperspectral image classification","volume":"2015","author":"Hu","year":"2015","journal-title":"J. Sens."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"844","DOI":"10.1109\/TGRS.2016.2616355","article-title":"Hyperspectral image classification using deep pixel-pair features","volume":"55","author":"Li","year":"2016","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"8065","DOI":"10.1109\/TGRS.2019.2918080","article-title":"Visual attention-driven hyperspectral image classification","volume":"57","author":"Haut","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Li, Y., Zhang, H., and Shen, Q. (2017). Spectral-spatial classification of hyperspectral imagery with 3D convolutional neural network. Remote Sens., 9.","DOI":"10.3390\/rs9010067"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"He, M., Li, B., and Chen, H. (2017, January 17\u201320). Multi-scale 3D deep convolutional neural network for hyperspectral image classification. Proceedings of the IEEE International Conference on Image Processing, Beijing, China.","DOI":"10.1109\/ICIP.2017.8297014"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"847","DOI":"10.1109\/TGRS.2017.2755542","article-title":"Spectral\u2013spatial residual network for hyperspectral image classification: A 3D deep learning framework","volume":"56","author":"Zhong","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"521","DOI":"10.1162\/tacl_a_00115","article-title":"Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies","volume":"4","author":"Tal","year":"2016","journal-title":"Trans. Assoc. Comput. Linguist."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"5384","DOI":"10.1109\/TGRS.2019.2899129","article-title":"Cascaded recurrent neural networks for hyperspectral image classification","volume":"57","author":"Hang","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Tang, G., Mathias, M., Rio, A., and Sennrich, R. (2018). Why self-attention? A targeted evaluation of neural machine translation architectures. arXiv, Available online: https:\/\/arxiv.org\/ pdf\/1808.08946v3.pdf.","DOI":"10.18653\/v1\/D18-1458"},{"key":"ref_38","unstructured":"Lin, Z., Feng, M., Santos, C., Yu, M., Xiang, B., Zhou, B., and Bengio, Y. (2017). A structured self-attentive sentence embedding. arXiv, Available online: https:\/\/arxiv.org\/ pdf\/1703.03130.pdf."},{"key":"ref_39","first-page":"5998","article-title":"Attention is all you need","volume":"30","author":"Vaswani","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_40","first-page":"249","article-title":"Understanding the difficulty of training deep feedforward neural networks","volume":"9","author":"Glorot","year":"2010","journal-title":"J. Mach. Learn. Res."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"2798","DOI":"10.1109\/TGRS.2017.2783886","article-title":"Pretraining for hyperspectral convolutional neural network classification","volume":"56","author":"Windrim","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_42","unstructured":"Hendrycks, D., and Gimpel, K. (2020). Gaussian error linear units (GELUs). arXiv, Available online: https:\/\/arxiv.org\/ pdf\/1606.08415.pdf."},{"key":"ref_43","unstructured":"Ba, J., Kiros, J., and Hinton, G. (2016). Layer normalization. arXiv, Available online: https:\/\/arxiv.org\/ pdf\/1607.06450v1.pdf."},{"key":"ref_44","unstructured":"Yosinski, J., Clune, J., Bengio, Y., and Lipson, H. (2019, January 8\u201314). How transferable are features in deep neural networks?. Proceedings of the Advance in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Oquab, M., Bottou, L., Laptev, I., and Sivic, J. (2014, January 20\u201323). Learning and transferring mid-level image representations using convolutional neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.222"},{"key":"ref_46","unstructured":"Hinton, G., Vinyals, O., and Dean, J. (2015). Distilling the knowledge in a neural network. arXiv, Available online: https:\/\/arxiv.org\/abs\/1503.02531."},{"key":"ref_47","unstructured":"Muller, R., Kornblith, S., and Hinton, G. (2019, January 8\u201314). When does label smoothing help?. Proceedings of the NeurIPS, Vancouver, BC, Canada."},{"key":"ref_48","unstructured":"Kingma, D.P., and Ba, J. (2017). Adam: A method for stochastic optimization. arXiv, Available online: https:\/\/arxiv.org\/abs\/1412.6980."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"2355","DOI":"10.1109\/LGRS.2017.2764915","article-title":"Hyperspectral images classification with gabor filtering and convolutional neural network","volume":"14","author":"Chen","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/3\/498\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:17:42Z","timestamp":1760159862000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/3\/498"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,1,30]]},"references-count":50,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2021,2]]}},"alternative-id":["rs13030498"],"URL":"https:\/\/doi.org\/10.3390\/rs13030498","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,1,30]]}}}