{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T02:26:07Z","timestamp":1760235967379,"version":"build-2065373602"},"reference-count":36,"publisher":"MDPI AG","issue":"20","license":[{"start":{"date-parts":[[2021,10,11]],"date-time":"2021-10-11T00:00:00Z","timestamp":1633910400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["U20B2065, 61972206, 62020106004"],"award-info":[{"award-number":["U20B2065, 61972206, 62020106004"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100010014","name":"Six Talent Peaks Project in Jiangsu Province","doi-asserted-by":"publisher","award":["RJFW-015"],"award-info":[{"award-number":["RJFW-015"]}],"id":[{"id":"10.13039\/501100010014","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Pansharpening aims to fuse the abundant spectral information of multispectral (MS) images and the spatial details of panchromatic (PAN) images, yielding a high-spatial-resolution MS (HRMS) image. Traditional methods only focus on the linear model, ignoring the fact that degradation process is a nonlinear inverse problem. Due to convolutional neural networks (CNNs) having an extraordinary effect in overcoming the shortcomings of traditional linear models, they have been adapted for pansharpening in the past few years. However, most existing CNN-based methods cannot take full advantage of the structural information of images. To address this problem, a new pansharpening method combining a spatial structure enhancement operator with a CNN architecture is employed in this study. The proposed method uses the Sobel operator as an edge-detection operator to extract abundant high-frequency information from the input PAN and MS images, hence obtaining the abundant spatial features of the images. Moreover, we utilize the CNN to acquire the spatial feature maps, preserving the information in both the spatial and spectral domains. Simulated experiments and real-data experiments demonstrated that our method had excellent performance in both quantitative and visual evaluation.<\/jats:p>","DOI":"10.3390\/rs13204062","type":"journal-article","created":{"date-parts":[[2021,10,11]],"date-time":"2021-10-11T21:45:32Z","timestamp":1633988732000},"page":"4062","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Convolutional Neural Network for Pansharpening with Spatial Structure Enhancement Operator"],"prefix":"10.3390","volume":"13","author":[{"given":"Weiwei","family":"Huang","sequence":"first","affiliation":[{"name":"Engineering Research Center of Digital Forensics, Ministry of Education, Nanjing University of Information Science and Technology, Nanjing 210044, China"}]},{"given":"Yan","family":"Zhang","sequence":"additional","affiliation":[{"name":"Engineering Research Center of Digital Forensics, Ministry of Education, Nanjing University of Information Science and Technology, Nanjing 210044, China"}]},{"given":"Jianwei","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Mathematics & Statistics, Nanjing University of Information Science and Technology, Nanjing 210044, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4408-3800","authenticated-orcid":false,"given":"Yuhui","family":"Zheng","sequence":"additional","affiliation":[{"name":"Engineering Research Center of Digital Forensics, Ministry of Education, Nanjing University of Information Science and Technology, Nanjing 210044, China"}]}],"member":"1968","published-online":{"date-parts":[[2021,10,11]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"108060","DOI":"10.1016\/j.sigpro.2021.108060","article-title":"TSLRLN: Tensor subspace low-rank learning with non-local prior for hyperspectral image mixed denoising","volume":"184","author":"He","year":"2021","journal-title":"Signal Process."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Sun, L., He, C., Zheng, Y., and Tang, S. (2020). SLRL4D: Joint restoration of subspace low-rank learning and non-local 4-D transform filtering for hyperspectral image. Remote Sens., 12.","DOI":"10.3390\/rs12182979"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"He, W., Yao, Q., Li, C., Yokoya, N., Zhao, Q., Zhang, H., and Zhang, L. (2020). Non-local meets global: An integrated paradigm for hyperspectral image restoration. IEEE Trans. Pattern Anal. Mach. Intell., 1.","DOI":"10.1109\/TPAMI.2020.3027563"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1109\/MGRS.2015.2440094","article-title":"Hyperspectral pansharpening: A review","volume":"3","author":"Loncan","year":"2015","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"102","DOI":"10.1016\/j.inffus.2018.05.006","article-title":"Review of the pansharpening methods for remote sensing images based on the idea of meta-analysis: Practical discussion and challenges","volume":"46","author":"Meng","year":"2019","journal-title":"Inf. Fusion"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"3230","DOI":"10.1109\/TGRS.2007.901007","article-title":"Improving component substitution pansharpening through multivariate regression of MS+Pan data","volume":"45","author":"Aiazzi","year":"2007","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_7","first-page":"1","article-title":"Hyperspectral image denoising with total variation regularization and nonlocal low-rank tensor decomposition","volume":"99","author":"Zhang","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1016\/0034-4257(87)90088-5","article-title":"Color enhancement of highly correlated images. ii. channel ratio and \u201cchromaticity\u201d transformation techniques","volume":"22","author":"Gillespie","year":"1987","journal-title":"Remote Sens. Environ."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1109\/LGRS.2007.909934","article-title":"Indusion: Fusion of Multispectral and Panchromatic Images Using the Induction Scaling Technique","volume":"5","author":"Khan","year":"2008","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"591","DOI":"10.14358\/PERS.72.5.591","article-title":"MTF-tailored multiscale fusion of high-resolution MS and pan imagery","volume":"72","author":"Aiazzi","year":"2006","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"3461","DOI":"10.1080\/014311600750037499","article-title":"Smoothing filter-based intensity modulation: A spectral preserve image fusion technique for improving spatial details","volume":"21","author":"Liu","year":"2000","journal-title":"Int. J. Remote Sens."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Shi, G., Luo, F., Tang, Y., and Li, Y. (2021). Dimensionality reduction of hyperspectral image based on local constrained manifold structure collaborative preserving embedding. Remote Sens., 13.","DOI":"10.3390\/rs13071363"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Duan, Y., Huang, H., and Wang, T. (2021). Semisupervised Feature Extraction of Hyperspectral Image Using Nonlinear Geodesic Sparse Hypergraphs. IEEE Trans. Geosci. Remote Sens., 1\u201315.","DOI":"10.1109\/TGRS.2021.3110855"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1693","DOI":"10.1109\/TSMCB.2012.2198810","article-title":"Adjustable model-based fusion method for multispectral and panchromatic images","volume":"42","author":"Zhang","year":"2012","journal-title":"IEEE Trans Syst. Man Cybern. Part. B Cybern."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1109\/TPAMI.2015.2439281","article-title":"Image Super-Resolution Using Deep Convolutional Networks","volume":"38","author":"Dong","year":"2016","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"594","DOI":"10.3390\/rs8070594","article-title":"Pansharpening by convolutional neural networks","volume":"8","author":"Masi","year":"2016","journal-title":"Remote Sens."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1188","DOI":"10.1109\/JSTARS.2019.2898574","article-title":"Pansharpening via detail injection based convolutional neural networks","volume":"12","author":"He","year":"2019","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"Lecun","year":"1998","journal-title":"Proc. IEEE"},{"key":"ref_19","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep residual learning for image recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1795","DOI":"10.1109\/LGRS.2017.2736020","article-title":"Boosting the Accuracy of Multispectral Image Pansharpening by Learning a Deep Residual Network","volume":"14","author":"Wei","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1109\/TPAMI.1981.4767056","article-title":"The Max Roberts Operator is a Hueckel-Type Edge Detector","volume":"3","author":"Rosenfeld","year":"1981","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"358","DOI":"10.1109\/4.996","article-title":"Design of an image edge detection filter using the Sobel operator","volume":"23","author":"Kanopoulos","year":"1988","journal-title":"IEEE J. Solid State Circuits"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Dong, W., and Shisheng, Z. (2008, January 12\u201314). Color Image Recognition Method Based on the Prewitt Operator. Proceedings of the 2008 International Conference on Computer Science and Software Engineering, Wuhan, China.","DOI":"10.1109\/CSSE.2008.567"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"886","DOI":"10.1109\/TPAMI.2007.1027","article-title":"Laplacian Operator-Based Edge Detectors","volume":"29","author":"Wang","year":"2007","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"679","DOI":"10.1109\/TPAMI.1986.4767851","article-title":"A Computational Approach To Edge Detection","volume":"8","author":"Canny","year":"1986","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1007\/s11263-006-6852-x","article-title":"A variational model for p+ xs image fusion","volume":"69","author":"Ballester","year":"2006","journal-title":"Int. J. Comput. Vis."},{"key":"ref_28","unstructured":"Hadsell, R., Chopra, S., and Lecun, Y. (2006, January 17\u201322). Dimensionality Reduction by Learning an Invariant Mapping. Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR\u201906), New York, NY, USA."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"400","DOI":"10.1214\/aoms\/1177729586","article-title":"A stochastic approximation method","volume":"22","author":"Robbins","year":"1951","journal-title":"Ann. Math. Stat."},{"key":"ref_30","unstructured":"Yuhas, R.H., Goetz, A.F., and Boardman, J.W. (1992, January 1\u20135). Discrimination among semi-arid landscape endmembers using the spectral angle mapper (sam) algorithm. Proceedings of the 3rd Annual JPL Airborne Geoscience Workshop, Pasadena, CA, USA."},{"key":"ref_31","unstructured":"Wald, L. (2002). Data Fusion: Definitions and Architectures: Fusion of Images of Different Spatial Resolutions, Les Presses de l\u2019E\u0301cole des Mines."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"743","DOI":"10.1080\/014311698215973","article-title":"A wavelet transform method to merge Landsat TM and SPOT panchromatic data","volume":"19","author":"Zhou","year":"1998","journal-title":"Int. J. Remote. Sens."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1109\/97.995823","article-title":"A universal image quality index","volume":"9","author":"Wang","year":"2002","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"193","DOI":"10.14358\/PERS.74.2.193","article-title":"Multispectral and panchromatic data fusion assessment without reference","volume":"74","author":"Alparone","year":"2008","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"978","DOI":"10.1109\/JSTARS.2018.2794888","article-title":"A multiscale and multidepth convolutional neural network for remote sensing imagery pan-sharpening","volume":"11","author":"Yuan","year":"2018","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"2300","DOI":"10.1109\/TGRS.2002.803623","article-title":"Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis","volume":"40","author":"Aiazzi","year":"2002","journal-title":"IEEE Trans. Geosci. Remote Sens."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/20\/4062\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:11:44Z","timestamp":1760166704000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/20\/4062"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,11]]},"references-count":36,"journal-issue":{"issue":"20","published-online":{"date-parts":[[2021,10]]}},"alternative-id":["rs13204062"],"URL":"https:\/\/doi.org\/10.3390\/rs13204062","relation":{},"ISSN":["2072-4292"],"issn-type":[{"type":"electronic","value":"2072-4292"}],"subject":[],"published":{"date-parts":[[2021,10,11]]}}}