{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,16]],"date-time":"2026-05-16T16:19:39Z","timestamp":1778948379273,"version":"3.51.4"},"reference-count":36,"publisher":"MDPI AG","issue":"17","license":[{"start":{"date-parts":[[2022,8,29]],"date-time":"2022-08-29T00:00:00Z","timestamp":1661731200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Natural Science Basic Research Program of ShaanXi","award":["2022JQ-647"],"award-info":[{"award-number":["2022JQ-647"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>High-resolution (HR) multispectral (MS) images contain sharper detail and structure compared to the ground truth high-resolution hyperspectral (HS) images. In this paper, we propose a novel supervised learning method, which considers pansharpening as the spectral super-resolution of high-resolution multispectral images and generates high-resolution hyperspectral images. The proposed method learns the spectral mapping between high-resolution multispectral images and the ground truth high-resolution hyperspectral images. To consider the spectral correlation between bands, we build a three-dimensional (3D) convolution neural network (CNN). The network consists of three parts using an encoder\u2013decoder framework: spatial\/spectral feature extraction from high-resolution multispectral images\/low-resolution (LR) hyperspectral images, feature transform, and image reconstruction to generate the results. In the image reconstruction network, we design the spatial\u2013spectral fusion (SSF) blocks to reuse the extracted spatial and spectral features in the reconstructed feature layer. Then, we develop the discrepancy-based deep hybrid gradient (DDHG) losses with the spatial\u2013spectral gradient (SSG) loss and deep gradient transfer (DGT) loss. The spatial\u2013spectral gradient loss and deep gradient transfer loss are developed to preserve the spatial and spectral gradients from the ground truth high-resolution hyperspectral images and high-resolution multispectral images. To overcome the spectral and spatial discrepancy between two images, we design a spectral downsampling (SD) network and a gradient consistency estimation (GCE) network for hybrid gradient losses. In the experiments, it is seen that the proposed method outperforms the state-of-the-art methods in the subjective and objective experiments in terms of the structure and spectral preservation of high-resolution hyperspectral images.<\/jats:p>","DOI":"10.3390\/rs14174250","type":"journal-article","created":{"date-parts":[[2022,8,30]],"date-time":"2022-08-30T01:37:55Z","timestamp":1661823475000},"page":"4250","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Deep Pansharpening via 3D Spectral Super-Resolution Network and Discrepancy-Based Gradient Transfer"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5481-1082","authenticated-orcid":false,"given":"Haonan","family":"Su","sequence":"first","affiliation":[{"name":"Shaanxi Key Laboratory for Network Computing and Security Technology, Department of Computer Science and Engineering, Xi\u2019an University of Technology, No. 5 South Jinhua Road, Xi\u2019an 710048, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3742-4029","authenticated-orcid":false,"given":"Haiyan","family":"Jin","sequence":"additional","affiliation":[{"name":"Shaanxi Key Laboratory for Network Computing and Security Technology, Department of Computer Science and Engineering, Xi\u2019an University of Technology, No. 5 South Jinhua Road, Xi\u2019an 710048, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ce","family":"Sun","sequence":"additional","affiliation":[{"name":"Xi\u2019an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi\u2019an 710119, China"},{"name":"Key Laboratory of Space Precision Measurement Technology, Chinese Academy of Sciences, Xi\u2019an 710119, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,8,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"174","DOI":"10.1016\/j.isprsjprs.2018.05.022","article-title":"Detecting Newly Grown Tree Leaves from Unmanned-Aerial-Vehicle Images using Hyperspectral Target Detection Techniques","volume":"142","author":"Lin","year":"2018","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1205","DOI":"10.1109\/TGRS.2018.2865197","article-title":"Hyperspectral Image Denoising Employing a Spatial\u2013Spectral Deep Residual Convolutional Neural Network","volume":"57","author":"Yuan","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"6808","DOI":"10.1109\/TGRS.2019.2908756","article-title":"Unsupervised Spatial\u2013Spectral Feature Learning by 3D-Convolutional Autoencoder for Hyperspectral Classification","volume":"57","author":"Mei","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"246","DOI":"10.1109\/TCSVT.2020.2975566","article-title":"Multiscale Densely-Connected Fusion Networks for Hyperspectral Images Classification","volume":"31","author":"Xie","year":"2021","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_5","first-page":"561","article-title":"A generalized component substitution technique for spatial enhancement of multispectral images using a higher resolution data set","volume":"58","author":"Shettigara","year":"1992","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1109\/TCSVT.2020.2975936","article-title":"Hyperspectral Computational Imaging via Collaborative Tucker3 Tensor Decomposition","volume":"31","author":"Xu","year":"2021","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"930","DOI":"10.1109\/LGRS.2013.2281996","article-title":"Contrast and Error-Based Fusion Schemes for Multispectral Image Pansharpening","volume":"11","author":"Vivone","year":"2014","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"591","DOI":"10.14358\/PERS.72.5.591","article-title":"MTF-tailored multiscale fusion of high-resolution MS and pan imagery","volume":"72","author":"Aiazzi","year":"2006","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Choi, J., Park, H., and Seo, D. (2019). Pansharpening Using Guided Filtering to Improve the Spatial Clarity of VHR Satellite Imagery. Remote Sens., 11.","DOI":"10.3390\/rs11060633"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1847","DOI":"10.1109\/TGRS.2008.917131","article-title":"Bayesian Data Fusion for Adaptable Image Pansharpening","volume":"46","author":"Fasbender","year":"2008","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Chen, C., Li, Y., Liu, W., and Huang, J. (2014, January 23\u201328). Image Fusion with Local Spectral Consistency and Dynamic Gradient Sparsity. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Washington, DC, USA.","DOI":"10.1109\/CVPR.2014.347"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1109\/TIP.2018.2866954","article-title":"High-Quality Bayesian Pansharpening","volume":"28","author":"Wang","year":"2019","journal-title":"IEEE Trans. Image Process."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Masi, G., Cozzolino, D., Verdoliva, L., and Scarpa, G. (2016). Pansharpening by Convolutional Neural Networks. Remote Sens., 8.","DOI":"10.3390\/rs8070594"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1656","DOI":"10.1109\/JSTARS.2018.2805923","article-title":"Remote Sensing Image Fusion with Deep Convolutional Neural Network","volume":"11","author":"Shao","year":"2018","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"8011","DOI":"10.1109\/TGRS.2019.2917759","article-title":"DDLPS: Detail-Based Deep Laplacian Pansharpening for Hyperspectral Imagery","volume":"57","author":"Li","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"5345","DOI":"10.1109\/TNNLS.2018.2798162","article-title":"Deep Hyperspectral Image Sharpening","volume":"29","author":"Dian","year":"2018","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1529","DOI":"10.1109\/TNNLS.2019.2920857","article-title":"Hyperspectral Pansharpening with Deep Priors","volume":"31","author":"Xie","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1708","DOI":"10.1109\/TCSVT.2021.3078559","article-title":"Hyperspectral Image Super-Resolution via Deep Prior Regularization with Parameter Estimation","volume":"32","author":"Wang","year":"2021","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"4295","DOI":"10.1109\/JSTARS.2020.3008047","article-title":"Pansharpening via Unsupervised Convolutional Neural Networks","volume":"13","author":"Luo","year":"2020","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Zhou, C., Zhang, J., Liu, J., Zhang, C., Fei, R., and Xu, S. (2020). PercepPan: Towards Unsupervised Pan-Sharpening Based on Perceptual Loss. Remote Sens., 12.","DOI":"10.3390\/rs12142318"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"3486","DOI":"10.1109\/TGRS.2020.3010441","article-title":"Rethinking CNN-Based Pansharpening: Guided Colorization of Panchromatic Images via GANs","volume":"59","author":"Ozcelik","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_22","unstructured":"Yasuma, F., Mitsunaga, T., Iso, D., and Nayar, S. (2022, July 20). Generalized Assorted Pixel Camera: Post-Capture Control of Resolution, Dynamic Range and Spectrum. Available online: http:\/\/citeseerx.ist.psu.edu\/viewdoc\/download?doi=10.1.1.360.7873&rep=rep1&type=pdf."},{"key":"ref_23","unstructured":"Yokoya, N., and Iwasaki, A. (2016). Airborne Hyperspectral Data over Chikusei, Space Application Laboratory, University of Tokyo. Technical Report SAL-2016-05-27."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1145\/584091.584093","article-title":"A mathematical theory of communication","volume":"5","author":"Shannon","year":"2001","journal-title":"ACM SIGMOBILE Mob. Comput. Commun. Rev."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Navab, N., Hornegger, J., Wells, W.M., and Frangi, A.F. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. MICCAI 2015, Proceedings of the Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, 5\u20139 October 2015, Springer International Publishing.","DOI":"10.1007\/978-3-319-24571-3"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"383","DOI":"10.1007\/s12021-018-9377-x","article-title":"SegAN: Adversarial Network with Multi-scale L1 Loss for Medical Image Segmentation","volume":"16","author":"Xue","year":"2018","journal-title":"Neuroinformatics"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"528","DOI":"10.1109\/TGRS.2011.2161320","article-title":"Coupled Nonnegative Matrix Factorization Unmixing for Hyperspectral and Multispectral Data Fusion","volume":"50","author":"Yokoya","year":"2012","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1632","DOI":"10.1109\/LSP.2016.2608858","article-title":"R-FUSE: Robust Fast Fusion of Multiband Images Based on Solving a Sylvester Equation","volume":"23","author":"Wei","year":"2016","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"6995","DOI":"10.1109\/TGRS.2020.3031366","article-title":"Detail Injection-Based Deep Convolutional Neural Networks for Pansharpening","volume":"59","author":"Deng","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Xie, Q., Zhou, M., Zhao, Q., Meng, D., Zuo, W., and Xu, Z. (2019, January 15\u201320). Multispectral and Hyperspectral Image Fusion by MS\/HS Fusion Net. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00168"},{"key":"ref_32","first-page":"691","article-title":"Fusion of satellite images of different spatial resolution: Assessing the quality of resulting images","volume":"63","author":"Wald","year":"1997","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1109\/TIP.2003.819861","article-title":"Image quality assessment: From error visibility to structural similarity","volume":"13","author":"Zhou","year":"2004","journal-title":"IEEE Trans. Image Process."},{"key":"ref_34","unstructured":"Yuhas, R.H., and Goetz, A.B.J. (1992, January 1\u20135). Discrimination among semi-arid landscape endmembers using the spectral angle mapper (SAM) algorithm. Proceedings of the Summaries 3rd Annual JPL Airborne Geoscience Workshop, Pasadena, CA, USA."},{"key":"ref_35","first-page":"49","article-title":"Fusion of high spatial and spectral resolution images: The ARSIS concept and its implementation","volume":"66","author":"Ranchin","year":"2000","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"662","DOI":"10.1109\/LGRS.2009.2022650","article-title":"Hypercomplex Quality Assessment of Multi\/Hyperspectral Images","volume":"6","author":"Garzelli","year":"2009","journal-title":"IEEE Geosci. Remote Sens. Lett."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/17\/4250\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:19:21Z","timestamp":1760141961000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/17\/4250"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,29]]},"references-count":36,"journal-issue":{"issue":"17","published-online":{"date-parts":[[2022,9]]}},"alternative-id":["rs14174250"],"URL":"https:\/\/doi.org\/10.3390\/rs14174250","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,8,29]]}}}