{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,21]],"date-time":"2026-02-21T06:32:08Z","timestamp":1771655528625,"version":"3.50.1"},"reference-count":36,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2021,6,21]],"date-time":"2021-06-21T00:00:00Z","timestamp":1624233600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61972060"],"award-info":[{"award-number":["61972060"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["U1713213"],"award-info":[{"award-number":["U1713213"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62027827"],"award-info":[{"award-number":["62027827"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2019YFE0110800"],"award-info":[{"award-number":["2019YFE0110800"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012669","name":"Natural Science Foundation Project of Chongqing, Chongqing Science and Technology Commission","doi-asserted-by":"publisher","award":["cstc2020jcyj-zdxmX0025"],"award-info":[{"award-number":["cstc2020jcyj-zdxmX0025"]}],"id":[{"id":"10.13039\/501100012669","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012669","name":"Natural Science Foundation Project of Chongqing, Chongqing Science and Technology Commission","doi-asserted-by":"publisher","award":["cstc2019cxcyljrc-td0270"],"award-info":[{"award-number":["cstc2019cxcyljrc-td0270"]}],"id":[{"id":"10.13039\/501100012669","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Deep learning has been widely used in various computer vision tasks. As a result, researchers have begun to explore the application of deep learning for pansharpening and have achieved remarkable results. However, most current pansharpening methods focus only on the mapping relationship between images and the lack overall structure enhancement, and do not fully and completely research optimization goals and fusion rules. Therefore, for these problems, we propose a pansharpening generative adversarial network with multilevel structure enhancement and a multistream fusion architecture. This method first uses multilevel gradient operators to obtain the structural information of the high-resolution panchromatic image. Then, it combines the spectral features with multilevel gradient information and inputs them into two subnetworks of the generator for fusion training. We design a comprehensive optimization goal for the generator, which not only minimizes the gap between the fused image and the real image but also considers the adversarial loss between the generator and the discriminator and the multilevel structure loss between the fused image and the panchromatic image. It is worth mentioning that we comprehensively consider the spectral information and the multilevel structure as the input of the discriminator, which makes it easier for the discriminator to distinguish real and fake images. Experiments show that our proposed method is superior to state-of-the-art methods in both the subjective visual and objective assessments of fused images, especially in road and building areas.<\/jats:p>","DOI":"10.3390\/rs13122423","type":"journal-article","created":{"date-parts":[[2021,6,21]],"date-time":"2021-06-21T13:29:58Z","timestamp":1624282198000},"page":"2423","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["A Pansharpening Generative Adversarial Network with Multilevel Structure Enhancement and a Multistream Fusion Architecture"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7966-7227","authenticated-orcid":false,"given":"Liping","family":"Zhang","sequence":"first","affiliation":[{"name":"Chongqing Key Laboratory of Image Cognition, Chognqing University of Posts and Telecommunications, Chongqing 400065, China"},{"name":"College of Software Engineering, Chognqing University of Posts and Telecommunications, Chongqing 400065, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9033-8245","authenticated-orcid":false,"given":"Weisheng","family":"Li","sequence":"additional","affiliation":[{"name":"Chongqing Key Laboratory of Image Cognition, Chognqing University of Posts and Telecommunications, Chongqing 400065, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6649-4176","authenticated-orcid":false,"given":"Hefeng","family":"Huang","sequence":"additional","affiliation":[{"name":"College of Software Engineering, Chognqing University of Posts and Telecommunications, Chongqing 400065, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7482-6417","authenticated-orcid":false,"given":"Dajiang","family":"Lei","sequence":"additional","affiliation":[{"name":"Chongqing Key Laboratory of Image Cognition, Chognqing University of Posts and Telecommunications, Chongqing 400065, China"}]}],"member":"1968","published-online":{"date-parts":[[2021,6,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1109\/MGRS.2015.2440094","article-title":"Hyperspectral pansharpening: A review","volume":"3","author":"Loncan","year":"2015","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2987","DOI":"10.1109\/TGRS.2014.2367129","article-title":"Marker-controlled watershed-based segmentation of multiresolution remote sensing images","volume":"53","author":"Gaetano","year":"2014","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2235","DOI":"10.1109\/TGRS.2015.2497966","article-title":"Spatial-Hessian-feature-guided variational model for pan-sharpening","volume":"54","author":"Liu","year":"2015","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_4","first-page":"561","article-title":"A generalized component substitution technique for spatial enhancement of multispectral images using a higher resolution data set","volume":"58","author":"Shettigara","year":"1992","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_5","first-page":"247","article-title":"Remote sensing image fusion method based on PCA and NSCT transform","volume":"38","author":"Feng","year":"2017","journal-title":"J. Graphics"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"746","DOI":"10.1109\/LGRS.2010.2046715","article-title":"An adaptive IHS pan-sharpening method","volume":"7","author":"Rahmani","year":"2010","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1109\/TGRS.2007.907604","article-title":"Optimal MMSE pan sharpening of very high resolution multispectral images","volume":"46","author":"Garzelli","year":"2007","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2464","DOI":"10.1109\/78.157290","article-title":"The discrete wavelet transform: Wedding the a trous and Mallat algorithms","volume":"40","author":"Shensa","year":"1992","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2300","DOI":"10.1109\/TGRS.2002.803623","article-title":"Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis","volume":"40","author":"Aiazzi","year":"2002","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"591","DOI":"10.14358\/PERS.72.5.591","article-title":"MTF-tailored multiscale fusion of high-resolution MS and Pan imagery","volume":"72","author":"Aiazzi","year":"2006","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_11","first-page":"58","article-title":"A fusion method of satellite remote sensing image based on IHS transform and curvelet transform","volume":"44","author":"Zheng","year":"2016","journal-title":"J. South China Univ. Technol."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1170","DOI":"10.1080\/2150704X.2016.1222098","article-title":"Pan-sharpening with structural consistency and \u21131\/2 gradient prior","volume":"7","author":"Zeng","year":"2016","journal-title":"Remote Sens. Lett."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"738","DOI":"10.1109\/TGRS.2010.2067219","article-title":"A new pan-sharpening method using a compressed sensing technique","volume":"49","author":"Li","year":"2010","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"158","DOI":"10.1016\/j.inffus.2017.10.007","article-title":"Deep learning for pixel-level image fusion: Recent advances and future prospects","volume":"42","author":"Liu","year":"2018","journal-title":"Inf. Fusion"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Masi, G., Cozzolino, D., Verdoliva, L., and Scarpa, G. (2016). Pansharpening by convolutional neural networks. Remote Sens., 8.","DOI":"10.3390\/rs8070594"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1109\/TPAMI.2015.2439281","article-title":"Image super-resolution using deep convolutional networks","volume":"38","author":"Dong","year":"2015","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Yang, J., Fu, X., Hu, Y., Huang, Y., Ding, X., and Paisley, J. (2017, January 22\u201329). PanNet: A deep network architecture for pan-sharpening. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.193"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Liu, X., Wang, Y., and Liu, Q. (2018, January 7\u201310). PSGAN: A generative adversarial network for remote sensing image pan-sharpening. Proceedings of the 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece.","DOI":"10.1109\/ICIP.2018.8451049"},{"key":"ref_20","unstructured":"Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014). Generative adversarial nets. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"18347","DOI":"10.1007\/s00521-020-04973-w","article-title":"A generative adversarial network with structural enhancement and spectral supplement for pan-sharpening","volume":"32","author":"Zhang","year":"2020","journal-title":"Neural Comput. Appl."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Xiang, Z., Xiao, L., Liu, P., and Zhang, Y. (August, January 28). A Multi-Scale Densely Deep Learning Method for Pansharpening. Proceedings of the IGARSS 2019\u20142019 IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan.","DOI":"10.1109\/IGARSS.2019.8898095"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21\u201326). Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"27163","DOI":"10.1109\/ACCESS.2020.2971502","article-title":"A remote-sensing image pan-sharpening method based on multi-scale channel attention residual network","volume":"8","author":"Li","year":"2020","journal-title":"IEEE Access"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1016\/j.inffus.2020.04.006","article-title":"Pan-GAN: An unsupervised pan-sharpening method for remote sensing image fusion","volume":"62","author":"Ma","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Wang, J., Shao, Z., Huang, X., Lu, T., Zhang, R., and Ma, J. (2021). Pan-sharpening via High-pass Modification Convolutional Neural Network. arXiv.","DOI":"10.1109\/ICIP42928.2021.9506568"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Chen, C., Li, Y., Liu, W., and Huang, J. (2014, January 23\u201328). Image fusion with local spectral consistency and dynamic gradient sparsity. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.347"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1109\/TIP.2018.2866954","article-title":"High-Quality Bayesian Pansharpening","volume":"28","author":"Wang","year":"2019","journal-title":"IEEE Trans. Image Process."},{"key":"ref_29","first-page":"3","article-title":"Rectifier nonlinearities improve neural network acoustic models","volume":"30","author":"Maas","year":"2013","journal-title":"Proc. Icml"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, 5\u20139 October 2015, Springer.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"2565","DOI":"10.1109\/TGRS.2014.2361734","article-title":"A critical comparison among pansharpening algorithms","volume":"53","author":"Vivone","year":"2014","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_32","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_33","unstructured":"Wald, L. (2002). Data Fusion: Definitions and Architectures\u2014Fusion of Images of Different Spatial Resolutions, Les Presses des Mines."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"662","DOI":"10.1109\/LGRS.2009.2022650","article-title":"Hypercomplex quality assessment of multi\/hyperspectral images","volume":"6","author":"Garzelli","year":"2009","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"743","DOI":"10.1080\/014311698215973","article-title":"A wavelet transform method to merge Landsat TM and SPOT panchromatic data","volume":"19","author":"Zhou","year":"1998","journal-title":"Int. J. Remote Sens."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"193","DOI":"10.14358\/PERS.74.2.193","article-title":"Multispectral and panchromatic data fusion assessment without reference","volume":"74","author":"Alparone","year":"2008","journal-title":"Photogramm. Eng. Remote Sens."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/12\/2423\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:20:17Z","timestamp":1760163617000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/12\/2423"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,21]]},"references-count":36,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2021,6]]}},"alternative-id":["rs13122423"],"URL":"https:\/\/doi.org\/10.3390\/rs13122423","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,6,21]]}}}