{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,13]],"date-time":"2026-03-13T04:02:49Z","timestamp":1773374569807,"version":"3.50.1"},"reference-count":56,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2021,3,26]],"date-time":"2021-03-26T00:00:00Z","timestamp":1616716800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100014219","name":"National Science Fund for Distinguished Young Scholars","doi-asserted-by":"publisher","award":["61925112"],"award-info":[{"award-number":["61925112"]}],"id":[{"id":"10.13039\/501100014219","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Recently, many convolutional networks have been built to fuse a low spatial resolution (LR) hyperspectral image (HSI) and a high spatial resolution (HR) multispectral image (MSI) to obtain HR HSIs. However, most deep learning-based methods are supervised methods, which require sufficient HR HSIs for supervised training. Collecting plenty of HR HSIs is laborious and time-consuming. In this paper, a self-supervised spectral-spatial residual network (SSRN) is proposed to alleviate dependence on a mass of HR HSIs. In SSRN, the fusion of HR MSIs and LR HSIs is considered a pixel-wise spectral mapping problem. Firstly, this paper assumes that the spectral mapping between HR MSIs and HR HSIs can be approximated by the spectral mapping between LR MSIs (derived from HR MSIs) and LR HSIs. Secondly, the spectral mapping between LR MSIs and LR HSIs is explored by SSRN. Finally, a self-supervised fine-tuning strategy is proposed to transfer the learned spectral mapping to generate HR HSIs. SSRN does not require HR HSIs as the supervised information in training. Simulated and real hyperspectral databases are utilized to verify the performance of SSRN.<\/jats:p>","DOI":"10.3390\/rs13071260","type":"journal-article","created":{"date-parts":[[2021,3,26]],"date-time":"2021-03-26T13:17:53Z","timestamp":1616764673000},"page":"1260","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":34,"title":["Hyperspectral Image Super-Resolution with Self-Supervised Spectral-Spatial Residual Network"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3878-7681","authenticated-orcid":false,"given":"Wenjing","family":"Chen","sequence":"first","affiliation":[{"name":"Key Laboratory of Spectral Imaging Technology CAS, Xi\u2019an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi\u2019an 710119, China"},{"name":"University of Chinese Academy of Sciences, Beijing 100049, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8398-6324","authenticated-orcid":false,"given":"Xiangtao","family":"Zheng","sequence":"additional","affiliation":[{"name":"Key Laboratory of Spectral Imaging Technology CAS, Xi\u2019an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi\u2019an 710119, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7037-5188","authenticated-orcid":false,"given":"Xiaoqiang","family":"Lu","sequence":"additional","affiliation":[{"name":"Key Laboratory of Spectral Imaging Technology CAS, Xi\u2019an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi\u2019an 710119, China"}]}],"member":"1968","published-online":{"date-parts":[[2021,3,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"116","DOI":"10.1109\/TIP.2019.2928895","article-title":"Super-Resolution for Hyperspectral and Multispectral Image Fusion Accounting for Seasonal Spectral Variability","volume":"29","author":"Borsoi","year":"2020","journal-title":"IEEE Trans. Image Process."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"3232","DOI":"10.1109\/TGRS.2019.2951160","article-title":"Spectral-Spatial Attention Network for Hyperspectral Image Classification","volume":"58","author":"Sun","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Zhao, X., Li, W., Zhang, M., Tao, R., and Ma, P. (2020). Adaptive Iterated Shrinkage Thresholding-Based Lp-Norm Sparse Representation for Hyperspectral Imagery Target Detection. Remote Sens., 12.","DOI":"10.3390\/rs12233991"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Ren, X., Lu, L., and Chanussot, J. (2020). Toward Super-Resolution Image Construction Based on Joint Tensor Decomposition. Remote Sens., 12.","DOI":"10.3390\/rs12162535"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1363","DOI":"10.1109\/TGRS.2016.2623626","article-title":"Multispectral and Hyperspectral Image Fusion Based on Group Spectral Embedding and Low-Rank Factorization","volume":"55","author":"Zhang","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Chen, W., and Lu, X. (2020). Unregistered Hyperspectral and Multispectral Image Fusion with Synchronous Nonnegative Matrix Factorization. Chinese Conference on Pattern Recognition and Computer Vision, Proceedings of the Third Chinese Conference, PRCV 2020, Nanjing, China, 16\u201318 October 2020, Springer.","DOI":"10.1007\/978-3-030-60633-6_50"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1240","DOI":"10.1109\/JSTARS.2019.2901752","article-title":"An Efficient Residual Learning Neural Network for Hyperspectral Image Superresolution","volume":"12","author":"Liu","year":"2019","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1109\/MGRS.2015.2440094","article-title":"Hyperspectral Pansharpening: A Review","volume":"3","author":"Loncan","year":"2015","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1109\/MGRS.2016.2637824","article-title":"Hyperspectral and Multispectral Data Fusion: A comparative review of the recent literature","volume":"5","author":"Yokoya","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"5625","DOI":"10.1109\/TIP.2018.2855418","article-title":"Self-Similarity Constrained Sparse Representation for Hyperspectral Image Super-Resolution","volume":"27","author":"Han","year":"2018","journal-title":"IEEE Trans. Image Process."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Feng, X., He, L., Cheng, Q., Long, X., and Yuan, Y. (2020). Hyperspectral and Multispectral Remote Sensing Image Fusion Based on Endmember Spatial Information. Remote Sens., 12.","DOI":"10.3390\/rs12061009"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"3373","DOI":"10.1109\/TGRS.2014.2375320","article-title":"A Convex Formulation for Hyperspectral Image Superresolution via Subspace-Based Regularization","volume":"53","author":"Almeida","year":"2015","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"4109","DOI":"10.1109\/TIP.2015.2458572","article-title":"Fast Fusion of Multi-Band Images Based on Solving a Sylvester Equation","volume":"24","author":"Wei","year":"2015","journal-title":"IEEE Trans. Image Process."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"5997","DOI":"10.1109\/TGRS.2017.2718728","article-title":"Hyperspectral and Multispectral Image Fusion Based on Local Low Rank and Coupled Spectral Unmixing","volume":"55","author":"Zhou","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1030","DOI":"10.1109\/JSTARS.2017.2785411","article-title":"Spatial-Spectral-Graph-Regularized Low-Rank Tensor Decomposition for Multispectral and Hyperspectral Image Fusion","volume":"11","author":"Zhang","year":"2018","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"4469","DOI":"10.1109\/TCYB.2019.2951572","article-title":"Nonlocal Sparse Tensor Factorization for Semiblind Hyperspectral and Multispectral Image Fusion","volume":"50","author":"Dian","year":"2020","journal-title":"IEEE Trans. Cybern."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1016\/j.inffus.2020.04.006","article-title":"Pan-GAN: An unsupervised pan-sharpening method for remote sensing image fusion","volume":"62","author":"Ma","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Zhang, L., Nie, J., Wei, W., Zhang, Y., Liao, S., and Shao, L. (2020, January 14\u201319). Unsupervised Adaptation Learning for Hyperspectral Imagery Super-Resolution. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00314"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Zhang, X., Huang, W., Wang, Q., and Li, X. (2020). SSR-NET: Spatial-Spectral Reconstruction Network for Hyperspectral and Multispectral Image Fusion. IEEE Trans. Geosci. Remote Sens., 1\u201313.","DOI":"10.1109\/TGRS.2020.3038405"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Xie, Q., Zhou, M., Zhao, Q., Meng, D., Zuo, W., and Xu, Z. (2019, January 16\u201320). Multispectral and Hyperspectral Image Fusion by MS\/HS Fusion Net. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00168"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"5345","DOI":"10.1109\/TNNLS.2018.2798162","article-title":"Deep Hyperspectral Image Sharpening","volume":"29","author":"Dian","year":"2018","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"6055","DOI":"10.1109\/TGRS.2019.2904108","article-title":"Hyperspectral Image Super-Resolution Using Deep Feature Matrix Factorization","volume":"57","author":"Xie","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1795","DOI":"10.1109\/LGRS.2017.2736020","article-title":"Boosting the Accuracy of Multispectral Image Pansharpening by Learning a Deep Residual Network","volume":"14","author":"Wei","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Wang, W., Zeng, W., Huang, Y., Ding, X., and Paisley, J. (2019, January 16\u201320). Deep Blind Hyperspectral Image Fusion. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Long Beach, CA, USA.","DOI":"10.1109\/ICCV.2019.00425"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"8011","DOI":"10.1109\/TGRS.2019.2917759","article-title":"DDLPS: Detail-Based Deep Laplacian Pansharpening for Hyperspectral Imagery","volume":"57","author":"Li","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1963","DOI":"10.1109\/JSTARS.2017.2655112","article-title":"Hyperspectral Image Superresolution by Transfer Learning","volume":"10","author":"Yuan","year":"2017","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Sidorov, O., and Hardeberg, J.Y. (2019, January 16\u201320). Deep Hyperspectral Prior: Single-Image Denoising, Inpainting, Super-Resolution. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Long Beach, CA, USA.","DOI":"10.1109\/ICCVW.2019.00477"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Qu, Y., Qi, H., and Kwan, C. (2018, January 18\u201323). Unsupervised Sparse Dirichlet-Net for Hyperspectral Image Super-Resolution. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00266"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1016\/j.neucom.2017.08.019","article-title":"Super-resolution of hyperspectral image via superpixel-based sparse representation","volume":"273","author":"Fang","year":"2018","journal-title":"Neurocomputing"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Akhtar, N., Shafait, F., and Mian, A. (2014). Sparse Spatio-spectral Representation for Hyperspectral Image Super-resolution. European Conference on Computer Vision, Proceedings of the Computer Vision\u2014ECCV 2014, Zurich, Switzerland, 6\u201312 September 2014, Springer International Publishing.","DOI":"10.1007\/978-3-319-10584-0_5"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"3658","DOI":"10.1109\/TGRS.2014.2381272","article-title":"Hyperspectral and Multispectral Image Fusion Based on a Sparse Representation","volume":"53","author":"Wei","year":"2015","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Wei, Q., Dobigeon, N., and Tourneret, J. (2014, January 4\u20139). Bayesian fusion of hyperspectral and multispectral images. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Florence, Italy.","DOI":"10.1109\/ICASSP.2014.6854186"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"455","DOI":"10.1109\/TGRS.2004.837324","article-title":"Hyperspectral resolution enhancement using high-resolution multispectral imagery with arbitrary response functions","volume":"43","author":"Eismann","year":"2005","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1117","DOI":"10.1109\/JSTSP.2015.2407855","article-title":"Bayesian Fusion of Multi-Band Images","volume":"9","author":"Wei","year":"2015","journal-title":"IEEE J. Sel. Top. Signal Process."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"2942","DOI":"10.1109\/TIP.2018.2814210","article-title":"A MAP-Based Approach for Hyperspectral Imagery Super-Resolution","volume":"27","author":"Irmak","year":"2018","journal-title":"IEEE Trans. Image Process."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"528","DOI":"10.1109\/TGRS.2011.2161320","article-title":"Coupled Nonnegative Matrix Factorization Unmixing for Hyperspectral and Multispectral Data Fusion","volume":"50","author":"Yokoya","year":"2012","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1652","DOI":"10.1109\/TGRS.2017.2766080","article-title":"A Convex Optimization-Based Coupled Nonnegative Matrix Factorization Algorithm for Hyperspectral and Multispectral Data Fusion","volume":"56","author":"Lin","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"4118","DOI":"10.1109\/TIP.2018.2836307","article-title":"Fusing Hyperspectral and Multispectral Images via Coupled Sparse Tensor Factorization","volume":"27","author":"Li","year":"2018","journal-title":"IEEE Trans. Image Process."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Li, J., Cui, R., Li, B., Song, R., Li, Y., and Du, Q. (2019). Hyperspectral Image Super-Resolution with 1D-2D Attentional Convolutional Neural Network. Remote Sens., 11.","DOI":"10.3390\/rs11232859"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Fu, Y., Zhang, T., Zheng, Y., Zhang, D., and Huang, H. (2019, January 16\u201320). Hyperspectral Image Super-Resolution with Optimized RGB Guidance. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01193"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"7565","DOI":"10.1109\/TIP.2020.3004261","article-title":"FusionNet: An Unsupervised Convolutional Variational Network for Hyperspectral and Multispectral Image Fusion","volume":"29","author":"Wang","year":"2020","journal-title":"IEEE Trans. Image Process."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1037","DOI":"10.1109\/LGRS.2014.2376034","article-title":"A New Pan-Sharpening Method with Deep Neural Networks","volume":"12","author":"Huang","year":"2015","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Li, J., Wu, C., Song, R., Xie, W., Ge, C., Li, B., and Li, Y. (2020). Hybrid 2-D-3-D Deep Residual Attentional Network with Structure Tensor Constraints for Spectral Super-Resolution of RGB Images. IEEE Trans. Geosci. Remote Sens., 1\u201315.","DOI":"10.1109\/TGRS.2020.3004934"},{"key":"ref_44","unstructured":"Fu, Y., Zhang, T., Zheng, Y., Zhang, D., and Huang, H. (2020). Joint Camera Spectral Response Selection and Hyperspectral Image Recovery. IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1109\/TPAMI.2018.2873729","article-title":"Hyperspectral Recovery from RGB Images using Gaussian Processes","volume":"42","author":"Akhtar","year":"2020","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1070","DOI":"10.1109\/TCI.2020.3000320","article-title":"Reconstruction of Hyperspectral Data from RGB Images with Prior Category Information","volume":"6","author":"Yan","year":"2020","journal-title":"IEEE Trans. Comput. Imaging"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"3173","DOI":"10.1109\/TGRS.2018.2794326","article-title":"Hyperspectral Image Classification with Deep Feature Fusion Network","volume":"56","author":"Song","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1109\/TGRS.2019.2931801","article-title":"Remote Sensing Scene Classification by Gated Bidirectional Network","volume":"58","author":"Sun","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"2810","DOI":"10.1109\/TIP.2021.3055613","article-title":"A Supervised Segmentation Network for Hyperspectral Image Classification","volume":"30","author":"Sun","year":"2021","journal-title":"IEEE Trans. Image Process."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"3007","DOI":"10.1109\/TGRS.2019.2946751","article-title":"Subspace Clustering Constrained Sparse NMF for Hyperspectral Unmixing","volume":"58","author":"Lu","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Du, X., Zheng, X., Lu, X., and Doudkin, A.A. (2021). Multisource Remote Sensing Data Classification with Graph Fusion Network. IEEE Trans. Geosci. Remote Sens., 1\u201311.","DOI":"10.1109\/TGRS.2020.3047130"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"5135","DOI":"10.1109\/TIP.2019.2916734","article-title":"Hyperspectral Image Super-Resolution via Subspace-Based Low Tensor Multi-Rank Regularization","volume":"28","author":"Dian","year":"2019","journal-title":"IEEE Trans. Image Process."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Wang, X., Girshick, R., Gupta, A., and He, K. (2018, January 18\u201323). Non-local Neural Networks. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00813"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"2241","DOI":"10.1109\/TIP.2010.2046811","article-title":"Generalized Assorted Pixel Camera: Postcapture Control of Resolution, Dynamic Range, and Spectrum","volume":"19","author":"Yasuma","year":"2010","journal-title":"IEEE Trans. Image Process."},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Akhtar, N., Shafait, F., and Mian, A. (2015, January 7\u201312). Bayesian sparse representation for hyperspectral image super resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298986"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/7\/1260\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:41:28Z","timestamp":1760161288000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/7\/1260"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,26]]},"references-count":56,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2021,4]]}},"alternative-id":["rs13071260"],"URL":"https:\/\/doi.org\/10.3390\/rs13071260","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,3,26]]}}}