{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,27]],"date-time":"2025-10-27T16:21:33Z","timestamp":1761582093704,"version":"build-2065373602"},"reference-count":50,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2021,5,18]],"date-time":"2021-05-18T00:00:00Z","timestamp":1621296000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2018YFB0505001"],"award-info":[{"award-number":["2018YFB0505001"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61702374"],"award-info":[{"award-number":["61702374"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Pansharpening aims at fusing the rich spectral information of multispectral (MS) images and the spatial details of panchromatic (PAN) images to generate a fused image with both high resolutions. In general, the existing pansharpening methods suffer from the problems of spectral distortion and lack of spatial detail information, which might prevent the accuracy computation for ground object identification. To alleviate these problems, we propose a Hybrid Attention mechanism-based Residual Neural Network (HARNN). In the proposed network, we develop an encoder attention module in the feature extraction part to better utilize the spectral and spatial features of MS and PAN images. Furthermore, the fusion attention module is designed to alleviate spectral distortion and improve contour details of the fused image. A series of ablation and contrast experiments are conducted on GF-1 and GF-2 datasets. The fusion results with less distorted pixels and more spatial details demonstrate that HARNN can implement the pansharpening task effectively, which outperforms the state-of-the-art algorithms.<\/jats:p>","DOI":"10.3390\/rs13101962","type":"journal-article","created":{"date-parts":[[2021,5,18]],"date-time":"2021-05-18T12:17:16Z","timestamp":1621340236000},"page":"1962","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["Hybrid Attention Based Residual Network for Pansharpening"],"prefix":"10.3390","volume":"13","author":[{"given":"Qin","family":"Liu","sequence":"first","affiliation":[{"name":"School of Software Engineering, Tongji University, 4800 Caoan Road Jiading District, Shanghai 201804, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6569-2613","authenticated-orcid":false,"given":"Letong","family":"Han","sequence":"additional","affiliation":[{"name":"School of Software Engineering, Tongji University, 4800 Caoan Road Jiading District, Shanghai 201804, China"}]},{"given":"Rui","family":"Tan","sequence":"additional","affiliation":[{"name":"School of Software Engineering, Tongji University, 4800 Caoan Road Jiading District, Shanghai 201804, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0352-9730","authenticated-orcid":false,"given":"Hongfei","family":"Fan","sequence":"additional","affiliation":[{"name":"School of Software Engineering, Tongji University, 4800 Caoan Road Jiading District, Shanghai 201804, China"}]},{"given":"Weiqi","family":"Li","sequence":"additional","affiliation":[{"name":"School of Software Engineering, Tongji University, 4800 Caoan Road Jiading District, Shanghai 201804, China"}]},{"given":"Hongming","family":"Zhu","sequence":"additional","affiliation":[{"name":"School of Software Engineering, Tongji University, 4800 Caoan Road Jiading District, Shanghai 201804, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3755-4870","authenticated-orcid":false,"given":"Bowen","family":"Du","sequence":"additional","affiliation":[{"name":"School of Software Engineering, Tongji University, 4800 Caoan Road Jiading District, Shanghai 201804, China"},{"name":"Department of Computer Science, University of Warwick, Gibbet Hill Road, Coventry CV47 AL, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1612-4844","authenticated-orcid":false,"given":"Sicong","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Geodesy and Geomatics, Tongji University, 1239 Siping Road Yangpu District, Shanghai 200082, China"}]}],"member":"1968","published-online":{"date-parts":[[2021,5,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1109\/JPROC.2018.2806218","article-title":"Modern Small Satellites-Changing the Economics of Space","volume":"106","author":"Sweeting","year":"2018","journal-title":"Proc. IEEE"},{"key":"ref_2","first-page":"657","article-title":"Understanding image fusion","volume":"70","author":"Zhang","year":"2004","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1016\/j.isprsjprs.2020.11.001","article-title":"A review of image fusion techniques for pan-sharpening of high-resolution satellite imagery","volume":"171","author":"Javan","year":"2021","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"102","DOI":"10.1016\/j.inffus.2018.05.006","article-title":"Review of the Pansharpening Methods for Remote Sensing Images Based on the Idea of Meta-analysis: Practical Discussion and Challenges","volume":"46","author":"Meng","year":"2018","journal-title":"Inf. Fusion"},{"key":"ref_5","first-page":"109","article-title":"Review and Performance Comparison of Pansharpening Algorithms for RASAT Images","volume":"18","author":"Kahraman","year":"2018","journal-title":"J. Electr. Electron. Eng."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2565","DOI":"10.1109\/TGRS.2014.2361734","article-title":"A Critical Comparison Among Pansharpening Algorithms","volume":"53","author":"Vivone","year":"2014","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"150","DOI":"10.1016\/j.inffus.2015.06.006","article-title":"Combining the spectral PCA and spatial PCA fusion methods by an optimal filter","volume":"27","author":"Shahdoosti","year":"2016","journal-title":"Inf. Fusion"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"7380","DOI":"10.1109\/TGRS.2014.2311815","article-title":"High-Fidelity Component Substitution Pansharpening by the Fitting of Substitution Data","volume":"52","author":"Xu","year":"2014","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_9","first-page":"249","article-title":"Wavelet based image fusion techniques\u2014An introduction, review and comparison","volume":"62","author":"Dare","year":"2007","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"594","DOI":"10.3390\/rs8070594","article-title":"Pansharpening by Convolutional Neural Networks","volume":"8","author":"Giuseppe","year":"2016","journal-title":"Remote Sens."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Liu, X., Wang, Y., and Liu, Q. (2018). PSGAN: A Generative Adversarial Network for Remote Sensing Image Pan-Sharpening. IEEE Trans. Geosci. Remote Sens.","DOI":"10.1109\/ICIP.2018.8451049"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"5549","DOI":"10.1109\/TGRS.2019.2900419","article-title":"Pan-Sharpening Using an Efficient Bidirectional Pyramid Network","volume":"57","author":"Zhang","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Hallabia, H., Hamam, H., and Hamida, A.B. (2020). An Optimal Use of SCE-UA Method Cooperated with Superpixel Segmentation for Pansharpening. IEEE Geosci. Remote Sens. Lett.","DOI":"10.1109\/LGRS.2020.3004320"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"He, L., Xi, D., Li, J., and Zhu, J. (2020). A Spectral-Aware Convolutional Neural Network for Pansharpening. Appl. Sci., 10.","DOI":"10.3390\/app10175809"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Liu, X., Wang, Y., and Liu, Q. (2018). Remote Sensing Image Fusion Based on Two-Stream Fusion Network, Springer.","DOI":"10.1007\/978-3-319-73603-7_35"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1795","DOI":"10.1109\/LGRS.2017.2736020","article-title":"Boosting the Accuracy of Multispectral Image Pansharpening by Learning a Deep Residual Network","volume":"14","author":"Wei","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_17","first-page":"1","article-title":"Spectral and Spatial Quality assessment of IHS and Wavelet Based Pan-sharpening Techniques for High Resolution Satellite Imagery","volume":"6","author":"Dadrasjavan","year":"2018","journal-title":"Adv. Image Video Process."},{"key":"ref_18","unstructured":"Laben, C.A., and Brower, B.V. (2000). Process for Enhancing the Spatial Resolution of Multispectral Imagery Using Pan-Sharpening. (US6011875A), U.S. Patent."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"985","DOI":"10.1109\/LGRS.2013.2284282","article-title":"An Improved Adaptive Intensity\u2013Hue\u2013Saturation Method for the Fusion of Remote Sensing Images","volume":"11","author":"Leung","year":"2013","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1606","DOI":"10.1109\/LGRS.2016.2597271","article-title":"Nonlinear IHS: A Promising Method for Pan-Sharpening","volume":"13","author":"Ghahremani","year":"2016","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Vivone, G., Alparone, L., Id, A., and Lolli, S. (2019). Fast Reproducible Pansharpening Based on Instrument and Acquisition Modeling: AWLP Revisited. Remote Sens., 11.","DOI":"10.3390\/rs11192315"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1016\/j.isprsjprs.2003.10.002","article-title":"Multi-resolution, object-oriented fuzzy analysis of remote sensing data for GIS-ready information","volume":"58","author":"Benz","year":"2004","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1109\/LGRS.2010.2067192","article-title":"Improved Additive-Wavelet Image Fusion","volume":"8","author":"Kim","year":"2011","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"3674","DOI":"10.1109\/TGRS.2006.881758","article-title":"Estimation of the Number of Decomposition Levels for a Wavelet-Based Multiresolution Multisensor Image Fusion","volume":"44","author":"Pradhan","year":"2006","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"532","DOI":"10.1109\/TCOM.1983.1095851","article-title":"The Laplacian pyramid as a compact image code","volume":"31","author":"Burt","year":"1983","journal-title":"IEEE Trans. Commun."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"4779","DOI":"10.1109\/TGRS.2012.2230332","article-title":"Remote Sensing Image Fusion via Sparse Representations Over Learned Dictionaries","volume":"51","author":"Li","year":"2013","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"180","DOI":"10.1109\/LGRS.2014.2331291","article-title":"A pansharpening method based on the sparse representation of injected details","volume":"12","author":"Vicinanza","year":"2014","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"3658","DOI":"10.1109\/TGRS.2014.2381272","article-title":"Hyperspectral and Multispectral Image Fusion based on a Sparse Representation","volume":"53","author":"Wei","year":"2015","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1016\/j.inffus.2019.02.003","article-title":"Ensemble of CNN for Multi-Focus Image Fusion","volume":"51","author":"Aghagolzadeh","year":"2019","journal-title":"Inf. Fusion"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Prabhakar, K.R., Srikar, V.S., and Babu, R.V. (2017, January 22\u201329). DeepFuse: A deep unsupervised approach for exposure fusion with extreme exposure image pairs. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.505"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Xu, K., Wen, L., Li, G., Bo, L., and Huang, Q. (2019, January 16\u201320). Spatiotemporal CNN for video object segmentation. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00147"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"699","DOI":"10.1016\/j.ins.2010.10.016","article-title":"Fuzzy clustering algorithms for unsupervised change detection in remote sensing images","volume":"181","author":"Ghosh","year":"2011","journal-title":"Inf. Sci."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Ullah, W., Ullah, A., Hussain, T., Khan, Z.A., and Baik, S.W. (2021). An Efficient Anomaly Recognition Framework Using an Attention Residual LSTM in Surveillance Videos. Sensors, 21.","DOI":"10.3390\/s21082811"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"107102","DOI":"10.1016\/j.asoc.2021.107102","article-title":"Efficient Activity Recognition using Lightweight CNN and DS-GRU Network for Surveillance Applications","volume":"103","author":"Ullah","year":"2021","journal-title":"Appl. Soft Comput."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"5443","DOI":"10.1109\/TGRS.2018.2817393","article-title":"Target-Adaptive CNN-Based Pansharpening","volume":"56","author":"Scarpa","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1007\/s11220-016-0135-6","article-title":"Remote Sensing Image Fusion with Convolutional Neural Network","volume":"17","author":"Zhong","year":"2016","journal-title":"Sens. Imaging"},{"key":"ref_37","unstructured":"Chao, D., Chen, C.L., He, K., and Tang, X. (2014). Learning a Deep Convolutional Network for Image Super-Resolution, Springer International Publishing."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1109\/TPAMI.2015.2439281","article-title":"Image Super-Resolution Using Deep Convolutional Networks","volume":"38","author":"Dong","year":"2016","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"541","DOI":"10.1162\/neco.1989.1.4.541","article-title":"Backpropagation Applied to Handwritten Zip Code","volume":"1","author":"Lecun","year":"1989","journal-title":"Neural Comput."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Masi, G., Cozzolino, D., Verdoliva, L., and Scarpa, G. (2017, January 6\u20138). CNN-based pansharpening of multi-resolution remote-sensing images. Proceedings of the Urban Remote Sensing Event, Dubai, United Arab Emirates.","DOI":"10.1109\/JURSE.2017.7924534"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Vitale, S. (28\u20132, January 28). A CNN-based pansharpening method with perceptual loss. Proceedings of the 2019 IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan.","DOI":"10.1109\/IGARSS.2019.8900390"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1656","DOI":"10.1109\/JSTARS.2018.2805923","article-title":"Remote Sensing Image Fusion with Deep Convolutional Neural Network","volume":"11","author":"Shao","year":"2018","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_43","first-page":"2672","article-title":"Generative Adversarial Networks","volume":"3","author":"Goodfellow","year":"2014","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Jian, S. (2016). Identity Mappings in Deep Residual Networks, Springer.","DOI":"10.1007\/978-3-319-46493-0_38"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Rui, X., Cao, Y., Kang, Y., Song, W., and Ba, R. (2020, January 25\u201328). Maskpan: Mask Prior Guided Network for Pansharpening. Proceedings of the 2020 IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates.","DOI":"10.1109\/ICIP40778.2020.9190853"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"978","DOI":"10.1109\/JSTARS.2018.2794888","article-title":"A Multiscale and Multidepth Convolutional Neural Network for Remote Sensing Imagery Pan-Sharpening","volume":"11","author":"Yuan","year":"2018","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_47","unstructured":"Chen, H., Teng, M., Shi, B., Wang, Y., and Huang, T. (2020). Learning to Deblur and Generate High Frame Rate Video with an Event Camera. arXiv."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Chollet, F. (2017, January 21\u201326). Xception: Deep Learning with Depthwise Separable Convolutions. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.195"},{"key":"ref_49","unstructured":"Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv."},{"key":"ref_50","first-page":"691","article-title":"Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images","volume":"63","author":"Wald","year":"1997","journal-title":"Photogramm. Eng. Remote Sens."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/10\/1962\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:03:16Z","timestamp":1760162596000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/10\/1962"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,5,18]]},"references-count":50,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2021,5]]}},"alternative-id":["rs13101962"],"URL":"https:\/\/doi.org\/10.3390\/rs13101962","relation":{},"ISSN":["2072-4292"],"issn-type":[{"type":"electronic","value":"2072-4292"}],"subject":[],"published":{"date-parts":[[2021,5,18]]}}}