{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,9]],"date-time":"2026-01-09T19:49:12Z","timestamp":1767988152664,"version":"3.49.0"},"reference-count":44,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2024,1,31]],"date-time":"2024-01-31T00:00:00Z","timestamp":1706659200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61971315"],"award-info":[{"award-number":["61971315"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Joint super-resolution and pansharpening (JSP) brings new insight into the spatial improvement of multispectral images. How to efficiently balance the spatial and spectral qualities in JSP is important for deep learning-based approaches. To address this problem, we propose a unified interpretable deep network for JSP, named UIJSP-Net. First, we formulate the JSP problem as an optimization problem in a specially designed physical model based on the relationship among the JSP result, the multispectral image, and the panchromatic image. In particular, two deep priors are utilized to describe latent distributions of different variables, which can improve the accuracy of the physical model. Furthermore, we adopt the alternating direction method of multipliers to solve the above optimization problem, where a series of iterative steps are generated. Finally, we design UIJSP-Net by unfolding these iterative steps into multiple corresponding stages in a unified network. Because UIJSP-Net has clear physical meanings, the spatial resolution of multispectral images can be efficiently improved while the spectral information can be kept as well. Extensive experimental results are carried out on both simulated and real datasets to demonstrate the superiority of UIJSP-Net over other state-of-the-art methods from qualitative and quantitative aspects.<\/jats:p>","DOI":"10.3390\/rs16030540","type":"journal-article","created":{"date-parts":[[2024,1,31]],"date-time":"2024-01-31T09:56:34Z","timestamp":1706694994000},"page":"540","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Unified Interpretable Deep Network for Joint Super-Resolution and Pansharpening"],"prefix":"10.3390","volume":"16","author":[{"given":"Dian","family":"Yu","sequence":"first","affiliation":[{"name":"Electronic Information School, Wuhan University, Wuhan 430072, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wei","family":"Zhang","sequence":"additional","affiliation":[{"name":"Electronic Information School, Wuhan University, Wuhan 430072, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mingzhu","family":"Xu","sequence":"additional","affiliation":[{"name":"Electronic Information School, Wuhan University, Wuhan 430072, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xin","family":"Tian","sequence":"additional","affiliation":[{"name":"Electronic Information School, Wuhan University, Wuhan 430072, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hao","family":"Jiang","sequence":"additional","affiliation":[{"name":"Electronic Information School, Wuhan University, Wuhan 430072, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,1,31]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"9088","DOI":"10.1109\/TNNLS.2022.3155655","article-title":"A Triple-Double Convolutional Neural Network for Panchromatic Sharpening","volume":"34","author":"Zhang","year":"2023","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"107434","DOI":"10.1016\/j.sigpro.2019.107434","article-title":"Image fusion employing adaptive spectral-spatial gradient sparse regularization in UAV remote sensing","volume":"170","author":"Zhang","year":"2020","journal-title":"Signal Process."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"284","DOI":"10.1109\/JAS.2022.106013","article-title":"Sparse Tensor Prior for Hyperspectral, Multispectral, and Panchromatic Image Fusion","volume":"10","author":"Tian","year":"2023","journal-title":"IEEE\/CAA J. Autom. Sin."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Wang, W., Zhou, Z., Zhang, X., Lv, T., Liu, H., and Liang, L. (2022). DiTBN: Detail Injection-Based Two-Branch Network for Pansharpening of Remote Sensing Images. Remote Sens., 14.","DOI":"10.3390\/rs14236120"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Masi, G., Cozzolino, D., Verdoliva, L., and Scarpa, G. (2016). Pansharpening by convolutional neural networks. Remote Sens., 8.","DOI":"10.3390\/rs8070594"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Yang, J., Fu, X., Hu, Y., Huang, Y., Ding, X., and Paisley, J. (2017, January 22\u201329). PanNet: A deep network architecture for pan-sharpening. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.193"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"978","DOI":"10.1109\/JSTARS.2018.2794888","article-title":"A multiscale and multidepth convolutional neural network for remote sensing imagery pan-sharpening","volume":"11","author":"Yuan","year":"2018","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"158","DOI":"10.1016\/j.inffus.2021.09.002","article-title":"Laplacian pyramid networks: A new approach for multispectral pansharpening","volume":"78","author":"Jin","year":"2022","journal-title":"Inf. Fusion"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Ke, C., Liang, H., Li, D., and Tian, X. (2023, January 4\u201310). High-Frequency Transformer Network Based on Window Cross-Attention for Pansharpening. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Rhodes Island, Greece.","DOI":"10.1109\/ICASSP49357.2023.10096538"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1016\/j.inffus.2020.04.006","article-title":"Pan-GAN: An unsupervised pan-sharpening method for remote sensing image fusion","volume":"62","author":"Ma","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"9","DOI":"10.5194\/isprs-archives-XLIII-B1-2022-9-2022","article-title":"Joint Super-Resolution and Image Restoration for PL\u00c9IADES NEO Imagery","volume":"43","author":"Chouteau","year":"2022","journal-title":"Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1707","DOI":"10.1109\/TMM.2016.2593039","article-title":"Image interpolation based on non-local geometric similarities and directional gradients","volume":"18","author":"Zhu","year":"2016","journal-title":"IEEE Trans. Multimed."},{"key":"ref_13","unstructured":"Yang, J., Wright, J., Huang, T., and Ma, Y. (2008, January 23\u201328). Image super-resolution as sparse representation of raw image patches. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, USA."},{"key":"ref_14","unstructured":"Zhang, K., Gao, X., Tao, D., and Li, X. (2012, January 16\u201321). Multi-scale dictionary for single image super-resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1944846.1944852","article-title":"Image and video upscaling from local self-examples","volume":"30","author":"Freedman","year":"2011","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Dong, C., Loy, C.C., He, K., and Tang, X. (2014, January 6\u201312). Learning a deep convolutional network for image super-resolution. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10593-2_13"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Dai, T., Cai, J., Zhang, Y., Xia, S.T., and Zhang, L. (2019, January 15\u201320). Second-order attention network for single image super-resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01132"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Zhang, K., Gool, L.V., and Timofte, R. (2020, January 14\u201319). Deep unfolding network for image super-resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00328"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Wei, D., Qin, C., Wang, H., Pfister, H., and Fu, Y. (2021, January 11\u201317). Context reasoning attention network for image super-resolution. Proceedings of the IEEE International Conference on Computer Vision, Montreal, BC, Canada.","DOI":"10.1109\/ICCV48922.2021.00424"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1405","DOI":"10.1109\/TPAMI.2022.3163307","article-title":"Bayesian image super-resolution with deep modeling of image statistics","volume":"45","author":"Gao","year":"2022","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_21","first-page":"1","article-title":"Variational PCA fusion for Pan-sharpening very high resolution imagery","volume":"57","author":"Zhou","year":"2014","journal-title":"Sci. China Inf. Sci."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"3230","DOI":"10.1109\/TGRS.2007.901007","article-title":"Improving component substitution pansharpening through multivariate regression of MS + Pan data","volume":"45","author":"Aiazzi","year":"2007","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"746","DOI":"10.1109\/LGRS.2010.2046715","article-title":"An adaptive IHS pan-sharpening method","volume":"7","author":"Rahmani","year":"2010","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"3461","DOI":"10.1080\/014311600750037499","article-title":"Smoothing filter-based intensity modulation: A spectral preserve image fusion technique for improving spatial details","volume":"21","author":"Liu","year":"2000","journal-title":"Int. J. Remote Sens."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"930","DOI":"10.1109\/LGRS.2013.2281996","article-title":"Contrast and error-based fusion schemes for multispectral image pansharpening","volume":"11","author":"Vivone","year":"2013","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Addesso, P., Restaino, R., and Vivone, G. (2021). An Improved Version of the Generalized Laplacian Pyramid Algorithm for Pansharpening. Remote Sens., 13.","DOI":"10.3390\/rs13173386"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"4682","DOI":"10.1109\/TGRS.2017.2697943","article-title":"Intersensor statistical matching for pansharpening: Theoretical issues and practical solutions","volume":"55","author":"Alparone","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1777","DOI":"10.1080\/01431161.2016.1163749","article-title":"Pan-sharpening using a guided filter","volume":"37","author":"Liu","year":"2016","journal-title":"Int. J. Remote Sens."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1007\/s11263-006-6852-x","article-title":"A variational model for P+ XS image fusion","volume":"69","author":"Ballester","year":"2006","journal-title":"Int. J. Comput. Vis."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"318","DOI":"10.1109\/LGRS.2013.2257669","article-title":"A new pansharpening algorithm based on total variation","volume":"11","author":"Palsson","year":"2013","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Fu, X., Lin, Z., Huang, Y., and Ding, X. (2019, January 15\u201320). A variational pan-sharpening with local gradient constraints. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01051"},{"key":"ref_32","first-page":"1","article-title":"Variational pansharpening by exploiting cartoon-texture similarities","volume":"60","author":"Tian","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Xu, H., Le, Z., Huang, J., and Ma, J. (2021). A Cross-Direction and Progressive Network for Pan-Sharpening. Remote Sens., 13.","DOI":"10.3390\/rs13153045"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1795","DOI":"10.1109\/LGRS.2017.2736020","article-title":"Boosting the accuracy of multispectral image pansharpening by learning a deep residual network","volume":"14","author":"Wei","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"5549","DOI":"10.1109\/TGRS.2019.2900419","article-title":"Pan-sharpening using an efficient bidirectional pyramid network","volume":"57","author":"Zhang","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"6995","DOI":"10.1109\/TGRS.2020.3031366","article-title":"Detail injection-based deep convolutional neural networks for pansharpening","volume":"59","author":"Deng","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_37","first-page":"1","article-title":"VP-Net: An interpretable deep network for variational pansharpening","volume":"60","author":"Tian","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"5401915","DOI":"10.1109\/TGRS.2023.3265404","article-title":"A novel spatial fidelity with learnable nonlinear mapping for panchromatic sharpening","volume":"61","author":"Wen","year":"2023","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Zhou, M., Huang, J., Fang, Y., Fu, X., and Liu, A. (2022, January 7\u201314). Pan-sharpening with customized transformer and invertible neural network. Proceedings of the AAAI Conference on Artificial Intelligence, Washington, DC, USA.","DOI":"10.1609\/aaai.v36i3.20267"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1137\/080716542","article-title":"A fast iterative shrinkage-thresholding algorithm for linear inverse problems","volume":"2","author":"Beck","year":"2009","journal-title":"SIAM J. Imaging Sci."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"6360","DOI":"10.1109\/TPAMI.2021.3088914","article-title":"Plug-and-Play Image Restoration With Deep Denoiser Prior","volume":"44","author":"Zhang","year":"2022","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Yang, D., and Sun, J. (2018, January 8\u201314). Proximal dehaze-net: A prior learning-based deep network for single image dehazing. Proceedings of the European Conference on Computer Vision, Munich, Germany.","DOI":"10.1007\/978-3-030-01234-2_43"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"7539","DOI":"10.1109\/JSTARS.2022.3202866","article-title":"A Deep-Shallow Fusion Network With Multidetail Extractor and Spectral Attention for Hyperspectral Pansharpening","volume":"15","author":"Zhuo","year":"2022","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"6316","DOI":"10.1109\/JSTARS.2021.3090252","article-title":"PGMAN: An Unsupervised Generative Multiadversarial Network for Pansharpening","volume":"14","author":"Zhou","year":"2021","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/3\/540\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T13:52:12Z","timestamp":1760104332000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/3\/540"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,31]]},"references-count":44,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2024,2]]}},"alternative-id":["rs16030540"],"URL":"https:\/\/doi.org\/10.3390\/rs16030540","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,1,31]]}}}