{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,17]],"date-time":"2026-01-17T20:10:57Z","timestamp":1768680657517,"version":"3.49.0"},"reference-count":44,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2024,9,26]],"date-time":"2024-09-26T00:00:00Z","timestamp":1727308800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62376214"],"award-info":[{"award-number":["62376214"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["92270117"],"award-info":[{"award-number":["92270117"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["2023-JC-YB-533"],"award-info":[{"award-number":["2023-JC-YB-533"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Natural Science Basic Research Program of Shaanxi","award":["62376214"],"award-info":[{"award-number":["62376214"]}]},{"name":"Natural Science Basic Research Program of Shaanxi","award":["92270117"],"award-info":[{"award-number":["92270117"]}]},{"name":"Natural Science Basic Research Program of Shaanxi","award":["2023-JC-YB-533"],"award-info":[{"award-number":["2023-JC-YB-533"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Pansharpening refers to enhancing the spatial resolution of multispectral images through panchromatic images while preserving their spectral features. However, existing traditional methods or deep learning methods always have certain distortions in the spatial or spectral dimensions. This paper proposes a remote sensing spatial\u2013spectral fusion method based on a multi-scale dual-stream convolutional neural network, which includes feature extraction, feature fusion, and image reconstruction modules for each scale. In terms of feature fusion, we propose a multi cascade module to better fuse image features. We also design a new loss function aim at enhancing the high degree of consistency between fused images and reference images in terms of spatial details and spectral information. To validate its effectiveness, we conduct thorough experimental analyses on two widely used remote sensing datasets: GeoEye-1 and Ikonos. Compared with the nine leading pansharpening techniques, the proposed method demonstrates superior performance in multiple key evaluation metrics.<\/jats:p>","DOI":"10.3390\/rs16193583","type":"journal-article","created":{"date-parts":[[2024,9,26]],"date-time":"2024-09-26T06:55:52Z","timestamp":1727333752000},"page":"3583","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["MDSCNN: Remote Sensing Image Spatial\u2013Spectral Fusion Method via Multi-Scale Dual-Stream Convolutional Neural Network"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3318-8137","authenticated-orcid":false,"given":"Wenqing","family":"Wang","sequence":"first","affiliation":[{"name":"School of Automation and Information Engineering, Xi\u2019an University of Technology, Xi\u2019an 710048, China"},{"name":"Shaanxi Key Laboratory of Complex System Control and Intelligent Information Processing, Xi\u2019an University of Technology, Xi\u2019an 710048, China"}]},{"given":"Fei","family":"Jia","sequence":"additional","affiliation":[{"name":"School of Automation and Information Engineering, Xi\u2019an University of Technology, Xi\u2019an 710048, China"}]},{"given":"Yifei","family":"Yang","sequence":"additional","affiliation":[{"name":"School of Automation and Information Engineering, Xi\u2019an University of Technology, Xi\u2019an 710048, China"}]},{"given":"Kunpeng","family":"Mu","sequence":"additional","affiliation":[{"name":"School of Automation and Information Engineering, Xi\u2019an University of Technology, Xi\u2019an 710048, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6618-1380","authenticated-orcid":false,"given":"Han","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Automation and Information Engineering, Xi\u2019an University of Technology, Xi\u2019an 710048, China"},{"name":"Shaanxi Key Laboratory of Complex System Control and Intelligent Information Processing, Xi\u2019an University of Technology, Xi\u2019an 710048, China"}]}],"member":"1968","published-online":{"date-parts":[[2024,9,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Tsvetkovskaya, I., Tekutieva, N., Prokofeva, E., and Vostrikov, A. (2020, January 11\u201313). Methods of Obtaining Geospatial Data Using Satellite Communications and Their Processing Using Convolutional Neural Networks. Proceedings of the 2020 Moscow Workshop on Electronic and Networking Technologies (MWENT), IEEE, Moscow, Russia.","DOI":"10.1109\/MWENT47943.2020.9067413"},{"key":"ref_2","first-page":"103303","article-title":"MF-SRCDNet: Multi-feature fusion super-resolution building change detection framework for multi-sensor high-resolution remote sensing imagery","volume":"119","author":"Li","year":"2023","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Wang, Y., Li, S., Teng, F., Lin, Y., Wang, M., and Cai, H. (2022). Improved mask R-CNN for rural building roof type recognition from uav high-resolution images: A case study in hunan province, China. Remote Sens., 14.","DOI":"10.3390\/rs14020265"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"e33208","DOI":"10.1016\/j.heliyon.2024.e33208","article-title":"Hyperspectral Imaging and Its Applications: A Review","volume":"10","author":"Bhargava","year":"2024","journal-title":"Heliyon"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"4259","DOI":"10.1109\/TGRS.2018.2890404","article-title":"A novel approach to the unsupervised update of land-cover maps by classification of time series of multispectral images","volume":"57","author":"Paris","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1016\/j.neucom.2020.05.082","article-title":"Discriminant sub-dictionary learning with adaptive multiscale superpixel representation for hyperspectral image classification","volume":"409","author":"Tu","year":"2020","journal-title":"Neurocomputing"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1109\/MGRS.2015.2440094","article-title":"Hyperspectral pansharpening: A review","volume":"3","author":"Loncan","year":"2015","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"985","DOI":"10.1109\/LGRS.2013.2284282","article-title":"An improved adaptive intensity\u2013hue\u2013saturation method for the fusion of remote sensing images","volume":"11","author":"Leung","year":"2013","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1606","DOI":"10.1109\/LGRS.2016.2597271","article-title":"Nonlinear IHS: A promising method for pan-sharpening","volume":"13","author":"Ghahremani","year":"2016","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Khan, S.S., Ran, Q., Khan, M., and Ji, Z. (2019, January 11\u201313). Pan-sharpening framework based on laplacian sharpening with Brovey. Proceedings of the 2019 IEEE International Conference on Signal, Information and Data Processing (ICSIDP), IEEE, Chongqing, China.","DOI":"10.1109\/ICSIDP47821.2019.9173129"},{"key":"ref_11","first-page":"339","article-title":"Extracting spectral contrast in Landsat Thematic Mapper image data using selective principal component analysis","volume":"55","author":"Kwarteng","year":"1989","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"3230","DOI":"10.1109\/TGRS.2007.901007","article-title":"Improving component substitution pansharpening through multivariate regression of MS + Pan data","volume":"45","author":"Aiazzi","year":"2007","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1109\/TGRS.2007.907604","article-title":"Optimal MMSE pan sharpening of very high resolution multispectral images","volume":"46","author":"Garzelli","year":"2007","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Nason, G.P., and Silverman, B.W. (1995). The stationary wavelet transform and some statistical applications. Wavelets and Statistics, Springer.","DOI":"10.1007\/978-1-4612-2544-7_17"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2464","DOI":"10.1109\/78.157290","article-title":"The discrete wavelet transform: Wedding the a trous and Mallat algorithms","volume":"40","author":"Shensa","year":"1992","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Burt, P.J., and Adelson, E.H. (1987). The Laplacian pyramid as a compact image code. Readings in Computer Vision, Elsevier.","DOI":"10.1016\/B978-0-08-051581-6.50065-9"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1289","DOI":"10.1109\/TIT.2006.871582","article-title":"Compressed sensing","volume":"52","author":"Donoho","year":"2006","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"738","DOI":"10.1109\/TGRS.2010.2067219","article-title":"A new pan-sharpening method using a compressed sensing technique","volume":"49","author":"Li","year":"2010","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"318","DOI":"10.1109\/LGRS.2013.2257669","article-title":"A new pansharpening algorithm based on total variation","volume":"11","author":"Palsson","year":"2013","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1269","DOI":"10.1109\/LGRS.2018.2836951","article-title":"A new variational model in texture space for pansharpening","volume":"15","author":"Lotfi","year":"2018","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"3647","DOI":"10.1109\/TNNLS.2017.2736011","article-title":"Learning low-rank decomposition for pan-sharpening with spatial-spectral offsets","volume":"29","author":"Yang","year":"2017","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"4793","DOI":"10.1109\/JSTARS.2014.2347072","article-title":"Pansharpening based on low-rank and sparse decomposition","volume":"7","author":"Rong","year":"2014","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1109\/TPAMI.2015.2439281","article-title":"Image super-resolution using deep convolutional networks","volume":"38","author":"Dong","year":"2015","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Masi, G., Cozzolino, D., Verdoliva, L., and Scarpa, G. (2016). Pansharpening by convolutional neural networks. Remote Sens., 8.","DOI":"10.3390\/rs8070594"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s11220-016-0135-6","article-title":"Remote sensing image fusion with convolutional neural network","volume":"17","author":"Zhong","year":"2016","journal-title":"Sens. Imaging"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.inffus.2019.07.010","article-title":"Remote sensing image fusion based on two-stream fusion network","volume":"55","author":"Liu","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1188","DOI":"10.1109\/JSTARS.2019.2898574","article-title":"Pansharpening via detail injection based convolutional neural networks","volume":"12","author":"He","year":"2019","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_28","first-page":"1","article-title":"Spatial-spectral dual back-projection network for pansharpening","volume":"61","author":"Zhang","year":"2023","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_29","first-page":"5001505","article-title":"Cross-interaction kernel attention network for pansharpening","volume":"20","author":"Zhang","year":"2023","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Bandara, W.G.C., and Patel, V.M. (2022, January 18\u201324). Hypertransformer: A textural and spectral feature fusion transformer for pansharpening. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00181"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1016\/j.inffus.2022.10.010","article-title":"P2Sharpen: A progressive pansharpening network with deep spectral transformation","volume":"91","author":"Zhang","year":"2023","journal-title":"Inf. Fusion"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Shang, Y., Liu, J., Zhang, J., and Wu, Z. (2024). MFT-GAN: A Multiscale Feature-guided Transformer Network for Unsupervised Hyperspectral Pansharpening. IEEE Trans. Geosci. Remote Sens.","DOI":"10.1109\/TGRS.2024.3402058"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Lim, B., Son, S., Kim, H., Nah, S., and Mu Lee, K. (2017, January 21\u201326). Enhanced deep residual networks for single image super-resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA.","DOI":"10.1109\/CVPRW.2017.151"},{"key":"ref_34","first-page":"691","article-title":"Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images","volume":"63","author":"Wald","year":"1997","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s12518-016-0179-2","article-title":"Evaluation of pan-sharpening methods for spatial and spectral quality","volume":"9","author":"Pushparaj","year":"2017","journal-title":"Appl. Geomat."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Yang, Y., Wan, W., Huang, S., Lin, P., and Que, Y. (2017). A novel pan-sharpening framework based on matting model and multiscale transform. Remote Sens., 9.","DOI":"10.3390\/rs9040391"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1672","DOI":"10.1109\/TGRS.2006.869923","article-title":"A new intensity-hue-saturation fusion approach to image fusion with a tradeoff parameter","volume":"44","author":"Choi","year":"2006","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"743","DOI":"10.1080\/014311698215973","article-title":"A wavelet transform method to merge Landsat TM and SPOT panchromatic data","volume":"19","author":"Zhou","year":"1998","journal-title":"Int. J. Remote Sens."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1109\/97.995823","article-title":"A universal image quality index","volume":"9","author":"Wang","year":"2002","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1109\/TIP.2003.819861","article-title":"Image quality assessment: From error visibility to structural similarity","volume":"13","author":"Wang","year":"2004","journal-title":"IEEE Trans. Image Process."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"193","DOI":"10.14358\/PERS.74.2.193","article-title":"Multispectral and panchromatic data fusion assessment without reference","volume":"74","author":"Alparone","year":"2008","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"591","DOI":"10.14358\/PERS.72.5.591","article-title":"MTF-tailored multiscale fusion of high-resolution MS and Pan imagery","volume":"72","author":"Aiazzi","year":"2006","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"6421","DOI":"10.1109\/TGRS.2019.2906073","article-title":"Robust band-dependent spatial-detail approaches for panchromatic sharpening","volume":"57","author":"Vivone","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Vivone, G., Alparone, L., Garzelli, A., and Lolli, S. (2019). Fast reproducible pansharpening based on instrument and acquisition modeling: AWLP revisited. Remote Sens., 11.","DOI":"10.3390\/rs11192315"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/19\/3583\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:03:40Z","timestamp":1760112220000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/19\/3583"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,26]]},"references-count":44,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2024,10]]}},"alternative-id":["rs16193583"],"URL":"https:\/\/doi.org\/10.3390\/rs16193583","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,9,26]]}}}