{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,26]],"date-time":"2025-11-26T05:08:14Z","timestamp":1764133694585,"version":"build-2065373602"},"reference-count":46,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2024,3,15]],"date-time":"2024-03-15T00:00:00Z","timestamp":1710460800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"the Startup Foundation for Introducing Talent of NUIST","award":["2022r075","61702363"],"award-info":[{"award-number":["2022r075","61702363"]}]},{"name":"the National Natural Science Foundation of China","award":["2022r075","61702363"],"award-info":[{"award-number":["2022r075","61702363"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Despite notable advancements achieved on Hyperspectral (HS) pansharpening tasks through deep learning techniques, previous methods are inherently constrained by convolution or self-attention intrinsic defects, leading to limited performance. In this paper, we proposed an Attention-Interactive Dual-Branch Convolutional Neural Network (AIDB-Net) for HS pansharpening. Our model purely consists of convolutional layers and simultaneously inherits the strengths of both convolution and self-attention, especially the modeling of short- and long-range dependencies. Specially, we first extract, tokenize, and align the hyperspectral image (HSI) and panchromatic image (PAN) by Overlapping Patch Embedding Blocks. Then, we specialize a novel Spectral-Spatial Interactive Attention which is able to globally interact and fuse the cross-modality features. The resultant token-global similarity scores can guide the refinement and renewal of the textural details and spectral characteristics within HSI features. By deeply combined these two paradigms, our AIDB-Net significantly improve the pansharpening performance. Moreover, with the acceleration by the convolution inductive bias, our interactive attention can be trained without large scale dataset and achieves competitive time cost with its counterparts. Compared with the state-of-the-art methods, our AIDB-Net makes 5.2%, 3.1%, and 2.2% improvement on PSNR metric on three public datasets, respectively. Comprehensive experiments quantitatively and qualitatively demonstrate the effectiveness and superiority of our AIDB-Net.<\/jats:p>","DOI":"10.3390\/rs16061044","type":"journal-article","created":{"date-parts":[[2024,3,15]],"date-time":"2024-03-15T12:02:39Z","timestamp":1710504159000},"page":"1044","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["AIDB-Net: An Attention-Interactive Dual-Branch Convolutional Neural Network for Hyperspectral Pansharpening"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2098-9878","authenticated-orcid":false,"given":"Qian","family":"Sun","sequence":"first","affiliation":[{"name":"School of Electronic and Information Engineering, Nanjing University of Information Science and Technology, Nanjing 210044, China"}]},{"given":"Yu","family":"Sun","sequence":"additional","affiliation":[{"name":"School of Software, Nanjing University of Information Science and Technology, Nanjing 210044, China"}]},{"given":"Chengsheng","family":"Pan","sequence":"additional","affiliation":[{"name":"School of Electronic and Information Engineering, Nanjing University of Information Science and Technology, Nanjing 210044, China"}]}],"member":"1968","published-online":{"date-parts":[[2024,3,15]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"5539217","DOI":"10.1109\/TGRS.2022.3208165","article-title":"Multi-Structure KELM With Attention Fusion Strategy for Hyperspectral Image Classification","volume":"60","author":"Sun","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Sun, L., Song, X., Guo, H., Zhao, G., and Wang, J. (2021). Patch-wise semantic segmentation for hyperspectral images via a cubic capsule network with emap features. Remote Sens., 13.","DOI":"10.3390\/rs13173497"},{"key":"ref_3","first-page":"5506617","article-title":"Mixed Noise Removal for Hyperspectral Images Based on Global Tensor Low-Rankness and Nonlocal SVD-Aided Group Sparsity","volume":"61","author":"Sun","year":"2023","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Sun, L., Chen, Y., and Li, B. (2023). SISLU-Net: Spatial Information-Assisted Spectral Information Learning Unmixing Network for Hyperspectral Images. Remote Sens., 15.","DOI":"10.3390\/rs15030817"},{"key":"ref_5","first-page":"5403214","article-title":"A Dual-Path Fusion Network for Pan-Sharpening","volume":"60","author":"Wang","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_6","first-page":"5531715","article-title":"Interactformer: Interactive Transformer and CNN for Hyperspectral Image Super-Resolution","volume":"60","author":"Liu","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1016\/S1566-2535(01)00036-7","article-title":"A new look at IHS-like image fusion methods","volume":"2","author":"Tu","year":"2001","journal-title":"Inf. Fusion"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2565","DOI":"10.1109\/TGRS.2014.2361734","article-title":"A Critical Comparison Among Pansharpening Algorithms","volume":"53","author":"Vivone","year":"2015","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_9","first-page":"339","article-title":"Extracting spectral contrast in Landsat Thematic Mapper image data using selective principal component analysis","volume":"55","author":"Kwarteng","year":"1989","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_10","unstructured":"Laben, C.A., and Brower, B.V. (2000). Process for Enhancing the Spatial Resolution of Multispectral Imagery Using Pan-Sharpening. (6,011,875), US Patent."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"3230","DOI":"10.1109\/TGRS.2007.901007","article-title":"Improving Component Substitution Pansharpening Through Multivariate Regression of MS +Pan Data","volume":"45","author":"Aiazzi","year":"2007","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"3461","DOI":"10.1080\/014311600750037499","article-title":"Smoothing filter-based intensity modulation: A spectral preserve image fusion technique for improving spatial details","volume":"21","author":"Liu","year":"2000","journal-title":"Int. J. Remote Sens."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2300","DOI":"10.1109\/TGRS.2002.803623","article-title":"Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis","volume":"40","author":"Aiazzi","year":"2002","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"591","DOI":"10.14358\/PERS.72.5.591","article-title":"MTF-tailored multiscale fusion of high-resolution MS and Pan imagery","volume":"72","author":"Aiazzi","year":"2006","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2984","DOI":"10.1109\/JSTARS.2015.2420582","article-title":"Processing of Multiresolution Thermal Hyperspectral and Digital Color Data: Outcome of the 2014 IEEE GRSS Data Fusion Contest","volume":"8","author":"Liao","year":"2015","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"528","DOI":"10.1109\/TGRS.2011.2161320","article-title":"Coupled Nonnegative Matrix Factorization Unmixing for Hyperspectral and Multispectral Data Fusion","volume":"50","author":"Yokoya","year":"2012","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"3373","DOI":"10.1109\/TGRS.2014.2375320","article-title":"A Convex Formulation for Hyperspectral Image Superresolution via Subspace-Based Regularization","volume":"53","author":"Almeida","year":"2015","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1117","DOI":"10.1109\/JSTSP.2015.2407855","article-title":"Bayesian Fusion of Multi-Band Images","volume":"9","author":"Wei","year":"2015","journal-title":"IEEE J. Sel. Top. Signal Process."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Masi, G., Cozzolino, D., Verdoliva, L., and Scarpa, G. (2016). Pansharpening by convolutional neural networks. Remote Sens., 8.","DOI":"10.3390\/rs8070594"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Yang, J., Fu, X., Hu, Y., Huang, Y., Ding, X., and Paisley, J. (2017, January 22\u201329). PanNet: A Deep Network Architecture for Pan-Sharpening. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.193"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1795","DOI":"10.1109\/LGRS.2017.2736020","article-title":"Boosting the Accuracy of Multispectral Image Pansharpening by Learning a Deep Residual Network","volume":"14","author":"Wei","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"4957","DOI":"10.1109\/TGRS.2020.3020162","article-title":"PSMD-Net: A Novel Pan-Sharpening Method Based on a Multiscale Dense Network","volume":"59","author":"Peng","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"5520816","DOI":"10.1109\/TGRS.2021.3139292","article-title":"Hyperspectral Pansharpening Based on Improved Deep Image Prior and Residual Reconstruction","volume":"60","author":"Bandara","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"8059","DOI":"10.1109\/TGRS.2020.2986313","article-title":"Hyperspectral Pansharpening Using Deep Prior and Dual Attention Residual Network","volume":"58","author":"Zheng","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"3092","DOI":"10.1109\/JSTARS.2019.2917584","article-title":"HyperPNN: Hyperspectral Pansharpening via Spectrally Predictive Convolutional Neural Networks","volume":"12","author":"He","year":"2019","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_26","first-page":"1","article-title":"A dual-branch detail extraction network for hyperspectral pansharpening","volume":"60","author":"Qu","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"5513315","DOI":"10.1109\/TGRS.2021.3108122","article-title":"Edge-Conditioned Feature Transform Network for Hyperspectral and Multispectral Image Fusion","volume":"60","author":"Zheng","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"5409011","DOI":"10.1109\/TGRS.2022.3168465","article-title":"Vision Transformer for Pansharpening","volume":"60","author":"Meng","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_29","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16 \u00d7 16 words: Transformers for image recognition at scale. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Bandara, W.G.C., and Patel, V.M. (2022, January 18\u201324). HyperTransformer: A Textural and Spectral Feature Fusion Transformer for Pansharpening. Proceedings of the 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00181"},{"key":"ref_31","first-page":"6012305","article-title":"Fusformer: A Transformer-Based Fusion Network for Hyperspectral Image Super-Resolution","volume":"19","author":"Hu","year":"2022","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Zhou, H., Liu, Q., and Wang, Y. (2022, January 18\u201322). PanFormer: A Transformer Based Model for Pan-Sharpening. Proceedings of the 2022 IEEE International Conference on Multimedia and Expo (ICME), Taipei, Taiwan.","DOI":"10.1109\/ICME52920.2022.9859770"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Wu, H., Xiao, B., Codella, N., Liu, M., Dai, X., Yuan, L., and Zhang, L. (2021, January 11\u201317). CvT: Introducing Convolutions to Vision Transformers. Proceedings of the 2021 IEEE\/CVF International Conference on Computer Vision (ICCV), Montreal, BC, Canada.","DOI":"10.1109\/ICCV48922.2021.00009"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Pan, X., Ge, C., Lu, R., Song, S., Chen, G., Huang, Z., and Huang, G. (2022, January 18\u201324). On the Integration of Self-Attention and Convolution. Proceedings of the 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00089"},{"key":"ref_35","unstructured":"Ma, X., Wang, H., Qin, C., Li, K., Zhao, X., Fu, J., and Fu, Y. (2022). A Close Look at Spatial Modeling: From Attention to Convolution. arXiv."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Sidorov, O., and Yngve Hardeberg, J. (2019, January 27\u201328). Deep hyperspectral prior: Single-image denoising, inpainting, super-resolution. Proceedings of the IEEE\/CVF International Conference on Computer Vision Workshops, Seoul, Republic of Korea.","DOI":"10.1109\/ICCVW.2019.00477"},{"key":"ref_37","unstructured":"Hendrycks, D., and Gimpel, K. (2016). Gaussian error linear units (gelus). arXiv."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Yu, W., Luo, M., Zhou, P., Si, C., Zhou, Y., Wang, X., Feng, J., and Yan, S. (2022, January 18\u201324). MetaFormer is Actually What You Need for Vision. Proceedings of the 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01055"},{"key":"ref_39","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"S110","DOI":"10.1016\/j.rse.2007.07.028","article-title":"Recent advances in techniques for hyperspectral image processing","volume":"113","author":"Plaza","year":"2009","journal-title":"Remote Sens. Environ."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1149","DOI":"10.1109\/TGRS.2003.815999","article-title":"Overview of the Earth Observing One (EO-1) mission","volume":"41","author":"Ungar","year":"2003","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_42","unstructured":"Yokoya, N., and Iwasaki, A. (2016). Airborne Hyperspectral Data over Chikusei, Space Application Laboratory, the University of Tokyo. Tech. Rep. SAL-2016-05-27."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Zeng, Y., Huang, W., Liu, M., Zhang, H., and Zou, B. (2010, January 18\u201320). Fusion of satellite images in urban area: Assessing the quality of resulting images. Proceedings of the 2010 18th International Conference on Geoinformatics, Beijing, China.","DOI":"10.1109\/GEOINFORMATICS.2010.5568105"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1109\/MGRS.2015.2440094","article-title":"Hyperspectral Pansharpening: A Review","volume":"3","author":"Loncan","year":"2015","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_45","first-page":"600","article-title":"Image quality assessment: From error visibility to structural similarity","volume":"13","author":"Wang","year":"2004","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TGRS.2022.3230846","article-title":"Swin transformer embedding UNet for remote sensing image semantic segmentation","volume":"60","author":"He","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/6\/1044\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:14:21Z","timestamp":1760105661000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/6\/1044"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,15]]},"references-count":46,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2024,3]]}},"alternative-id":["rs16061044"],"URL":"https:\/\/doi.org\/10.3390\/rs16061044","relation":{},"ISSN":["2072-4292"],"issn-type":[{"type":"electronic","value":"2072-4292"}],"subject":[],"published":{"date-parts":[[2024,3,15]]}}}