{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,29]],"date-time":"2025-12-29T11:47:10Z","timestamp":1767008830232,"version":"build-2065373602"},"reference-count":53,"publisher":"MDPI AG","issue":"15","license":[{"start":{"date-parts":[[2022,8,4]],"date-time":"2022-08-04T00:00:00Z","timestamp":1659571200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Henan Province Science and Technology Breakthrough Project","award":["212102210102","212102210105"],"award-info":[{"award-number":["212102210102","212102210105"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Pansharpening based on deep learning (DL) has shown great advantages. Most convolutional neural network (CNN)-based methods focus on obtaining local features from multispectral (MS) and panchromatic (PAN) images, but ignore the nonlocal dependence on images. Therefore, Transformer-based methods are introduced to obtain long-range information on images. However, the representational capabilities of features extracted by CNN or Transformer alone are weak. To solve this problem, a local and nonlocal feature interaction network (LNFIN) is proposed in this paper for pansharpening. It comprises Transformer and CNN branches. Furthermore, a feature interaction module (FIM) is proposed to fuse different features and return to the two branches to enhance the representational capability of features. Specifically, a CNN branch consisting of multiscale dense modules (MDMs) is proposed for acquiring local features of the image, and a Transformer branch consisting of pansharpening Transformer modules (PTMs) is introduced for acquiring nonlocal features of the image. In addition, inspired by the PTM, a shift pansharpening Transformer module (SPTM) is proposed for the learning of texture features to further enhance the spatial representation of features. The LNFIN outperforms the state-of-the-art method experimentally on three datasets.<\/jats:p>","DOI":"10.3390\/rs14153743","type":"journal-article","created":{"date-parts":[[2022,8,5]],"date-time":"2022-08-05T02:12:39Z","timestamp":1659665559000},"page":"3743","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["A Local and Nonlocal Feature Interaction Network for Pansharpening"],"prefix":"10.3390","volume":"14","author":[{"given":"Junru","family":"Yin","sequence":"first","affiliation":[{"name":"College of Computer and Communication Engineering, Zhengzhou University of Light Industry, Zhengzhou 450000, China"}]},{"given":"Jiantao","family":"Qu","sequence":"additional","affiliation":[{"name":"College of Computer and Communication Engineering, Zhengzhou University of Light Industry, Zhengzhou 450000, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6465-8678","authenticated-orcid":false,"given":"Le","family":"Sun","sequence":"additional","affiliation":[{"name":"School of Computer Science, Nanjing University of Information Science and Technology, Nanjing 210044, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0095-1354","authenticated-orcid":false,"given":"Wei","family":"Huang","sequence":"additional","affiliation":[{"name":"College of Computer and Communication Engineering, Zhengzhou University of Light Industry, Zhengzhou 450000, China"}]},{"given":"Qiqiang","family":"Chen","sequence":"additional","affiliation":[{"name":"College of Computer and Communication Engineering, Zhengzhou University of Light Industry, Zhengzhou 450000, China"}]}],"member":"1968","published-online":{"date-parts":[[2022,8,4]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"3818","DOI":"10.1109\/TNNLS.2019.2944869","article-title":"Non-Peaked Discriminant Analysis for Data Representation","volume":"30","author":"Ye","year":"2019","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1109\/TNNLS.2020.3027588","article-title":"Learning Robust Discriminant Subspace Based on Joint L2,p- and L2,s-Norm Distance Metrics","volume":"33","author":"Fu","year":"2022","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"108060","DOI":"10.1016\/j.sigpro.2021.108060","article-title":"TSLRLN: Tensor subspace low-rank learning with non-local prior for hyperspectral image mixed denoising","volume":"184","author":"He","year":"2021","journal-title":"Signal Process"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TGRS.2022.3231215","article-title":"Spectral-Spatial Feature Tokenization Transformer for Hyperspectral Image Classification","volume":"60","author":"Sun","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_5","unstructured":"Ye, Q., Huang, P., Zhang, Z., Zheng, Y., Fu, L., and Yang, W. (2021). Multiview Learning with Robust Double-Sided Twin SVM. IEEE Trans. Cybern., 1\u201314."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1016\/S1566-2535(01)00036-7","article-title":"A new look at IHS-like image fusion methods","volume":"2","author":"Tu","year":"2001","journal-title":"Inf. Fusion"},{"key":"ref_7","first-page":"339","article-title":"Extracting spectral contrast in Landsat Thematic Mapper image data using selective principal component analysis","volume":"55","author":"Kwarteng","year":"1989","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1109\/TGRS.2010.2051674","article-title":"A new adaptive component-substitution-based satellite image fusion by using partial replacement","volume":"49","author":"Choi","year":"2011","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1016\/j.inffus.2016.03.003","article-title":"A review of remote sensing image fusion methods","volume":"32","author":"Ghassemian","year":"2016","journal-title":"Inf. Fusion"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1109\/TGRS.2007.907604","article-title":"Optimal MMSE Pan Sharpening of Very High Resolution Multispectral Images","volume":"46","author":"Garzelli","year":"2008","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"3230","DOI":"10.1109\/TGRS.2007.901007","article-title":"Improving Component Substitution Pansharpening Through Multivariate Regression of MS +Pan Data","volume":"45","author":"Aiazzi","year":"2007","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"743","DOI":"10.1080\/014311698215973","article-title":"A wavelet transform method to merge Landsat TM and SPOT panchromatic data","volume":"19","author":"Zhou","year":"1998","journal-title":"Int. J. Remote Sens."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2565","DOI":"10.1109\/TGRS.2014.2361734","article-title":"A Critical Comparison Among Pansharpening Algorithms","volume":"53","author":"Vivone","year":"2015","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"3418","DOI":"10.1109\/TIP.2018.2819501","article-title":"Full Scale Regression-Based Injection Coefficients for Panchromatic Sharpening","volume":"27","author":"Vivone","year":"2018","journal-title":"IEEE Trans. Image Process."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1109\/TGRS.2009.2028613","article-title":"Fast and Efficient Panchromatic Sharpening","volume":"48","author":"Lee","year":"2010","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Vivone, G., Alparone, L., Garzelli, A., and Lolli, S. (2019). Fast reproducible pansharpening based on instrument and acquisition modeling: AWLP revisited. Remote Sens., 11.","DOI":"10.3390\/rs11192315"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1007\/s11263-006-6852-x","article-title":"A variational model for P + XS image fusion","volume":"69","author":"Ballester","year":"2006","journal-title":"Int. J. Comput. Vis."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1997","DOI":"10.1109\/TGRS.2014.2351754","article-title":"Pansharpening based on semiblind deconvolution","volume":"53","author":"Vivone","year":"2015","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Liu, Y., and Wang, Z. (2013, January 22\u201323). A practical pan-sharpening method with wavelet transform and sparse representation. Proceedings of the IEEE International Conference on Imaging Systems and Techniques (IST), Beijing, China.","DOI":"10.1109\/IST.2013.6729708"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1170","DOI":"10.1080\/2150704X.2016.1222098","article-title":"Pan-sharpening with structural consistency and \u21131\/2 gradient prior","volume":"7","author":"Zeng","year":"2016","journal-title":"Remote Sens. Lett."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Fu, X., Lin, Z., Huang, Y., and Ding, X. (2019, January 15\u201320). A variational pan-sharpening with local gradient constraints. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01051"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1117","DOI":"10.1109\/TGRS.2018.2864750","article-title":"Convolution Structure Sparse Coding for Fusion of Panchromatic and Multispectral Images","volume":"57","author":"Zhang","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"318","DOI":"10.1109\/LGRS.2013.2257669","article-title":"A new pansharpening algorithm based on total variation","volume":"11","author":"Palsson","year":"2014","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1016\/j.rse.2006.02.014","article-title":"Downscaling cokriging for image sharpening","volume":"102","author":"Atkinson","year":"2006","journal-title":"Remote Sens. Environ."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1016\/j.isprsjprs.2016.02.006","article-title":"Area-to-point regression kriging for pan-sharpening","volume":"114","author":"Wang","year":"2016","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"8599","DOI":"10.1109\/TGRS.2020.3041724","article-title":"Object-based area-to-point regression kriging for pansharpening","volume":"59","author":"Zhang","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"3092","DOI":"10.1109\/JSTARS.2019.2917584","article-title":"HyperPNN: Hyperspectral pansharpening via spectrally predictive convolutional neural networks","volume":"12","author":"He","year":"2019","journal-title":"IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"5898","DOI":"10.1109\/JSTARS.2020.3025040","article-title":"Spectral-fidelity convolutional neural Networks for hyperspectral pansharpening","volume":"13","author":"He","year":"2020","journal-title":"IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1109\/JSTARS.2020.3032472","article-title":"Pansharpening based on joint-guided detail extraction","volume":"14","author":"Yang","year":"2021","journal-title":"IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"4082","DOI":"10.1109\/JSTARS.2019.2939670","article-title":"Superresolution land cover mapping based on pixel-, subpixel-, and superpixel-scale spatial dependence with pansharpening Technique","volume":"12","author":"Wang","year":"2019","journal-title":"IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1188","DOI":"10.1109\/JSTARS.2019.2898574","article-title":"Pansharpening via detail injection based convolutional neural networks","volume":"12","author":"He","year":"2019","journal-title":"IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"10303","DOI":"10.1109\/JSTARS.2021.3117944","article-title":"Pansharpening via subpixel convolutional residual network","volume":"14","author":"Li","year":"2021","journal-title":"IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"11879","DOI":"10.1109\/JSTARS.2021.3126645","article-title":"Attention_FPNet: Two-branch remote sensing image pansharpening network based on attention feature fusion","volume":"14","author":"Zhong","year":"2021","journal-title":"IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"4295","DOI":"10.1109\/JSTARS.2020.3008047","article-title":"Pansharpening via unsupervised convolutional neural networks","volume":"13","author":"Luo","year":"2020","journal-title":"IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1109\/TPAMI.2015.2439281","article-title":"Image super-resolution using deep convolutional networks","volume":"38","author":"Dong","year":"2015","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Masi, G., Cozzolino, D., and Verdoliva, L. (2016). Pansharpening by convolutional neural networks. Remote Sens., 8.","DOI":"10.3390\/rs8070594"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Yang, J., Fu, X., Hu, Y., Huang, Y., Ding, X., and Paisley, J. (2017, January 22\u201329). PanNet: A deep network architecture for pan-sharpening. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.193"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1016\/j.inffus.2020.04.006","article-title":"Pan-GAN: An unsupervised pan-sharpening method for remote sensing image fusion","volume":"62","author":"Ma","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Wang, W., Zhou, Z., and Liu, H. (2021). MSDRN: Pansharpening of Multispectral Images via Multi-Scale Deep Residual Network. Remote Sens., 13.","DOI":"10.3390\/rs13061200"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Wang, Y., Deng, L.J., and Zhang, T.J. (2021, January 20\u201324). SSconv: Explicit Spectral-to-Spatial Convolution for Pansharpening. Proceedings of the 29th ACM International Conference on Multimedia, Chengdu, China.","DOI":"10.1145\/3474085.3475600"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"6995","DOI":"10.1109\/TGRS.2020.3031366","article-title":"Detail Injection-Based Deep Convolutional Neural Networks for Pansharpening","volume":"59","author":"Deng","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_42","first-page":"1","article-title":"Progressive Pan-Sharpening via Cross-Scale Collaboration Networks","volume":"19","author":"Yang","year":"2022","journal-title":"IEEE Geosci. Remote. Sens. Lett."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Zhou, M., Yan, K., Huang, J., Yang, Z., Fu, X., and Zhao, F. (2022, January 19\u201320). Mutual Information-Driven Pan-Sharpening. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00184"},{"key":"ref_44","first-page":"1","article-title":"Effective Pan-Sharpening with Transformer and Invertible Neural Network","volume":"60","author":"Zhou","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Yang, F., Yang, H., Fu, J., Lu, H., and Guo, B. (2020, January 14\u201319). Learning texture transformer network for image super-resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Virtual.","DOI":"10.1109\/CVPR42600.2020.00583"},{"key":"ref_46","unstructured":"Nithin, G.R., Kumar, N., Kakani, R., Venkateswaran, N., Garg, A., and Gupta, U.K. (TechRxiv, 2021). Pansformers: Transformer-Based Self-Attention Network for Pansharpening, TechRxiv, preprint."},{"key":"ref_47","first-page":"1","article-title":"Multistage dual-attention guided fusion network for hyperspectral pansharpening","volume":"60","author":"Guan","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_48","unstructured":"Yuhas, R.H., Goetz, A.F., and Boardman, J.W. (1992, January 1\u20135). Discrimination among semi-arid landscape endmembers using the spectral angle mapper (SAM) algorithm. Proceedings of the Summaries 3rd Annual JPL Airborne Geoscience Workshop, Pasadena, CA, USA."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Khademi, G., and Ghassemian, H. (2017, January 19\u201320). A multi-objective component-substitution-based pansharpening. Proceedings of the 3rd International Conference on Pattern Recognition and Image Analysis (IPRIA), Shahrekord, Iran.","DOI":"10.1109\/PRIA.2017.7983056"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.inffus.2019.07.010","article-title":"Remote sensing image fusion based on two-stream fusion network","volume":"55","author":"Liu","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1109\/97.995823","article-title":"A universal image quality index","volume":"9","author":"Wang","year":"2002","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1109\/LGRS.2004.836784","article-title":"A global quality measurement of pan-sharpened multispectral imagery","volume":"1","author":"Alparone","year":"2004","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"193","DOI":"10.14358\/PERS.74.2.193","article-title":"Multispectral and panchromatic data fusion assessment without reference","volume":"74","author":"Alparone","year":"2008","journal-title":"Photogramm. Eng. Remote Sens."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/15\/3743\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:04:19Z","timestamp":1760141059000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/15\/3743"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,4]]},"references-count":53,"journal-issue":{"issue":"15","published-online":{"date-parts":[[2022,8]]}},"alternative-id":["rs14153743"],"URL":"https:\/\/doi.org\/10.3390\/rs14153743","relation":{},"ISSN":["2072-4292"],"issn-type":[{"type":"electronic","value":"2072-4292"}],"subject":[],"published":{"date-parts":[[2022,8,4]]}}}