{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T02:43:02Z","timestamp":1760150582782,"version":"build-2065373602"},"reference-count":55,"publisher":"MDPI AG","issue":"24","license":[{"start":{"date-parts":[[2023,12,8]],"date-time":"2023-12-08T00:00:00Z","timestamp":1701993600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Natural Science Foundation of China","award":["U1711266","41925007","XJ2020025"],"award-info":[{"award-number":["U1711266","41925007","XJ2020025"]}]},{"name":"Hong Kong Scholars Program","award":["U1711266","41925007","XJ2020025"],"award-info":[{"award-number":["U1711266","41925007","XJ2020025"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Due to technological limitations and budget constraints, spatiotemporal image fusion uses the complementarity of high temporal\u2013low spatial resolution (HTLS) and high spatial\u2013low temporal resolution (HSLT) data to obtain high temporal and spatial resolution (HTHS) fusion data, which can effectively satisfy the demand for HTHS data. However, some existing spatiotemporal image fusion models ignore the large difference in spatial resolution, which yields worse results for spatial information under the same conditions. Based on the flexible spatiotemporal data fusion (FSDAF) framework, this paper proposes a multilevel single-image super-resolution (SISR) method to solve this issue under the large difference in spatial resolution. The following are the advantages of the proposed method. First, multilevel super-resolution (SR) can effectively avoid the limitation of a single SR method for a large spatial resolution difference. In addition, the issue of noise accumulation caused by multilevel SR can be alleviated by learning-based SR (the cross-scale internal graph neural network (IGNN)) and then interpolation-based SR (the thin plate spline (TPS)). Finally, we add the reference information to the super-resolution, which can effectively control the noise generation. This method has been subjected to comprehensive experimentation using two authentic datasets, affirming that our proposed method surpasses the current state-of-the-art spatiotemporal image fusion methodologies in terms of performance and effectiveness.<\/jats:p>","DOI":"10.3390\/rs15245675","type":"journal-article","created":{"date-parts":[[2023,12,11]],"date-time":"2023-12-11T13:18:21Z","timestamp":1702300701000},"page":"5675","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["MSISR-STF: Spatiotemporal Fusion via Multilevel Single-Image Super-Resolution"],"prefix":"10.3390","volume":"15","author":[{"given":"Xiongwei","family":"Zheng","sequence":"first","affiliation":[{"name":"School of Computer Science, China University of Geoscience, Wuhan 430074, China"},{"name":"China Geological Survey, Beijing 100037, China"}]},{"given":"Ruyi","family":"Feng","sequence":"additional","affiliation":[{"name":"School of Computer Science, China University of Geoscience, Wuhan 430074, China"},{"name":"Hubei Key Laboratory of Intelligent Geo-Information Processing, China University of Geosciences, Wuhan 430074, China"}]},{"given":"Junqing","family":"Fan","sequence":"additional","affiliation":[{"name":"School of Computer Science, China University of Geoscience, Wuhan 430074, China"},{"name":"Hubei Key Laboratory of Intelligent Geo-Information Processing, China University of Geosciences, Wuhan 430074, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3882-1616","authenticated-orcid":false,"given":"Wei","family":"Han","sequence":"additional","affiliation":[{"name":"School of Computer Science, China University of Geoscience, Wuhan 430074, China"},{"name":"Hubei Key Laboratory of Intelligent Geo-Information Processing, China University of Geosciences, Wuhan 430074, China"}]},{"given":"Shengnan","family":"Yu","sequence":"additional","affiliation":[{"name":"School of Computer Science, China University of Geoscience, Wuhan 430074, China"},{"name":"Hubei Key Laboratory of Intelligent Geo-Information Processing, China University of Geosciences, Wuhan 430074, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9896-1656","authenticated-orcid":false,"given":"Jia","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Computer Science, China University of Geoscience, Wuhan 430074, China"},{"name":"Hubei Key Laboratory of Intelligent Geo-Information Processing, China University of Geosciences, Wuhan 430074, China"}]}],"member":"1968","published-online":{"date-parts":[[2023,12,8]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2914","DOI":"10.1016\/j.rse.2008.02.010","article-title":"North American forest disturbance mapped from a decadal Landsat record","volume":"112","author":"Masek","year":"2008","journal-title":"Remote Sens. Environ."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Brezini, S., and Deville, Y. (2023). Hyperspectral and Multispectral Image Fusion with Automated Extraction of Image-Based Endmember Bundles and Sparsity-Based Unmixing to Deal with Spectral Variability. Sensors, 23.","DOI":"10.3390\/s23042341"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"527","DOI":"10.1016\/j.rse.2014.10.018","article-title":"Mapping land cover in complex Mediterranean landscapes using Landsat: Improved classification accuracies from integrating multi-seasonal and synthetic imagery","volume":"156","author":"Senf","year":"2015","journal-title":"Remote Sens. Environ."},{"key":"ref_4","first-page":"650","article-title":"Completion of the 1990s National Land Cover Data Set for the conterminous United States from Landsat Thematic Mapper data and ancillary data sources","volume":"67","author":"Vogelmann","year":"2001","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1016\/j.future.2013.12.018","article-title":"Modeling and simulation for natural disaster contingency planning driven by high-resolution remote sensing images","volume":"37","author":"Dou","year":"2014","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1109\/MGRS.2015.2434351","article-title":"Fusing Landsat and MODIS data for vegetation monitoring","volume":"3","author":"Gao","year":"2015","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/S0034-4257(02)00084-6","article-title":"An overview of MODIS Land data processing and product status","volume":"83","author":"Justice","year":"2002","journal-title":"Remote Sens. Environ."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Zhu, X., Cai, F., Tian, J., and Williams, T. (2018). Spatiotemporal fusion of multisource remote sensing data: Literature survey, taxonomy, principles, applications, and future directions. Remote Sens., 10.","DOI":"10.3390\/rs10040527"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2207","DOI":"10.1109\/TGRS.2006.872081","article-title":"On the blending of the Landsat and MODIS surface reflectance: Predicting daily Landsat surface reflectance","volume":"44","author":"Gao","year":"2006","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1613","DOI":"10.1016\/j.rse.2009.03.007","article-title":"A new data fusion model for high spatial-and temporal-resolution mapping of forest disturbance based on Landsat and MODIS","volume":"113","author":"Hilker","year":"2009","journal-title":"Remote Sens. Environ."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2610","DOI":"10.1016\/j.rse.2010.05.032","article-title":"An enhanced spatial and temporal adaptive reflectance fusion model for complex heterogeneous regions","volume":"114","author":"Zhu","year":"2010","journal-title":"Remote Sens. Environ."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"6346","DOI":"10.3390\/rs5126346","article-title":"An improved image fusion approach based on enhanced spatial and temporal the adaptive reflectance fusion model","volume":"5","author":"Fu","year":"2013","journal-title":"Remote Sens."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"706","DOI":"10.1080\/01431161.2016.1271471","article-title":"Improving spatiotemporal reflectance fusion using image inpainting and steering kernel regression techniques","volume":"38","author":"Wu","year":"2017","journal-title":"Int. J. Remote Sens."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1016\/j.rse.2017.10.046","article-title":"Spatio-temporal fusion for daily Sentinel-2 images","volume":"204","author":"Wang","year":"2018","journal-title":"Remote Sens. Environ."},{"key":"ref_15","first-page":"1","article-title":"A Flexible Object-Level Processing Strategy to Enhance the Weight Function-Based Spatiotemporal Fusion Method","volume":"60","author":"Guo","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"629","DOI":"10.1109\/TGRS.2020.2994260","article-title":"Superpixel-Based Reweighted Low-Rank and Total Variation Sparse Unmixing for Hyperspectral Remote Sensing Imagery","volume":"59","author":"Li","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1212","DOI":"10.1109\/36.763276","article-title":"Unmixing-based multisensor multiresolution image fusion","volume":"37","author":"Zhukov","year":"1999","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"453","DOI":"10.1109\/LGRS.2008.919685","article-title":"Unmixing-based Landsat TM and MERIS FR data fusion","volume":"5","author":"Clevers","year":"2008","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"063507","DOI":"10.1117\/1.JRS.6.063507","article-title":"Use of MODIS and Landsat time series data to generate high-resolution temporal synthetic Landsat data using a spatial and temporal reflectance fusion model","volume":"6","author":"Wu","year":"2012","journal-title":"J. Appl. Remote Sens."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"5346","DOI":"10.3390\/rs5105346","article-title":"An enhanced spatial and temporal data fusion model for fusing Landsat and MODIS surface reflectance to generate high temporal Landsat-like data","volume":"5","author":"Zhang","year":"2013","journal-title":"Remote Sens."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"24002","DOI":"10.3390\/s150924002","article-title":"Generating daily synthetic Landsat imagery by combining Landsat and MODIS data","volume":"15","author":"Wu","year":"2015","journal-title":"Sensors"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"374","DOI":"10.1016\/j.rse.2016.07.028","article-title":"Land cover change detection by integrating object-based data blending model of Landsat and MODIS","volume":"184","author":"Lu","year":"2016","journal-title":"Remote Sens. Environ."},{"key":"ref_23","first-page":"1","article-title":"Unmixing-Based Spatiotemporal Image Fusion Accounting for Complex Land Cover Changes","volume":"60","author":"Jiang","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_24","first-page":"1","article-title":"Generalized Linear Spectral Mixing Model for Spatial\u2013Temporal\u2013Spectral Fusion","volume":"60","author":"Zhou","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1016\/j.rse.2013.03.021","article-title":"Blending multi-resolution satellite sea surface temperature (SST) products using Bayesian maximum entropy method","volume":"135","author":"Li","year":"2013","journal-title":"Remote Sens. Environ."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"561","DOI":"10.1080\/2150704X.2013.769283","article-title":"Unified fusion of remote-sensing imagery: Generating simultaneously high-resolution synthetic spatial\u2013temporal\u2013spectral earth observations","volume":"4","author":"Huang","year":"2013","journal-title":"Remote Sens. Lett."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"7135","DOI":"10.1109\/TGRS.2016.2596290","article-title":"An integrated framework for the spatio-temporal-spectral fusion of remote sensing images","volume":"54","author":"Shen","year":"2016","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Liao, L., Song, J., Wang, J., Xiao, Z., and Wang, J. (2016). Bayesian method for building frequent Landsat-like NDVI datasets by integrating MODIS and Landsat NDVI. Remote Sens., 8.","DOI":"10.3390\/rs8060452"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Xue, J., Leung, Y., and Fung, T. (2017). A Bayesian data fusion approach to spatio-temporal fusion of remotely sensed images. Remote Sens., 9.","DOI":"10.3390\/rs9121310"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"221:1","DOI":"10.1145\/3510414","article-title":"A Survey on Active Deep Learning: From Model Driven to Data Driven","volume":"54","author":"Liu","year":"2022","journal-title":"ACM Comput. Surv."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"3707","DOI":"10.1109\/TGRS.2012.2186638","article-title":"Spatiotemporal reflectance fusion via sparse representation","volume":"50","author":"Huang","year":"2012","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1883","DOI":"10.1109\/TGRS.2012.2213095","article-title":"Spatiotemporal satellite image fusion through one-pair image learning","volume":"51","author":"Song","year":"2012","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"209199","DOI":"10.1109\/ACCESS.2020.3011258","article-title":"Spatiotemporal Remote-Sensing Image Fusion With Patch-Group Compressed Sensing","volume":"8","author":"Li","year":"2020","journal-title":"IEEE Access"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"2107","DOI":"10.1109\/TGRS.2017.2775103","article-title":"Improving satellite estimates of the fraction of absorbed photosynthetically active radiation through data integration: Methodology and validation","volume":"56","author":"Tao","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Wei, J., Wang, L., Liu, P., and Song, W. (2016). Spatiotemporal fusion of remote sensing images with structural sparsity and semi-coupled dictionary learning. Remote Sens., 9.","DOI":"10.3390\/rs9010021"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"821","DOI":"10.1109\/JSTARS.2018.2797894","article-title":"Spatiotemporal satellite image fusion using deep convolutional neural networks","volume":"11","author":"Song","year":"2018","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TGRS.2022.3230439","article-title":"A Flexible Reference-Insensitive Spatiotemporal Fusion Model for Remote Sensing Images Using Conditional Generative Adversarial Network","volume":"60","author":"Tan","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TGRS.2022.3215431","article-title":"RFSDAF: A New Spatiotemporal Fusion Method Robust to Registration Errors","volume":"60","author":"Hou","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1362","DOI":"10.1109\/LGRS.2015.2402644","article-title":"Spatial and temporal image fusion via regularized spatial unmixing","volume":"12","author":"Xu","year":"2015","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1016\/j.rse.2014.09.012","article-title":"A comparison of STARFM and an unmixing-based algorithm for Landsat and MODIS data fusion","volume":"156","author":"Gevaert","year":"2015","journal-title":"Remote Sens. Environ."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1016\/j.rse.2015.11.016","article-title":"A flexible spatiotemporal method for fusing satellite images with different resolutions","volume":"172","author":"Zhu","year":"2016","journal-title":"Remote Sens. Environ."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Xie, D., Zhang, J., Zhu, X., Pan, Y., Liu, H., Yuan, Z., and Yun, Y. (2016). An improved STARFM with help of an unmixing-based method to generate high spatial and temporal resolution remote sensing data in complex heterogeneous regions. Sensors, 16.","DOI":"10.3390\/s16020207"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1016\/j.rse.2017.05.011","article-title":"Generating a series of fine spatial and temporal resolution land cover maps by fusing coarse spatial resolution remotely sensed images and fine spatial resolution land cover maps","volume":"196","author":"Li","year":"2017","journal-title":"Remote Sens. Environ."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"5851","DOI":"10.1109\/TGRS.2020.3023432","article-title":"CycleGAN-STF: Spatiotemporal Fusion via CycleGAN-Based Image Generation","volume":"59","author":"Chen","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"6723","DOI":"10.1109\/JSTARS.2023.3296468","article-title":"A Rigorously-Incremental Spatiotemporal Data Fusion Method for Fusing Remote Sensing Images","volume":"16","author":"Jing","year":"2023","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1109\/TPAMI.2015.2439281","article-title":"Image super-resolution using deep convolutional networks","volume":"38","author":"Dong","year":"2015","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Lim, B., Son, S., Kim, H., Nah, S., and Mu Lee, K. (2017, January 21\u201326). Enhanced deep residual networks for single image super-resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA.","DOI":"10.1109\/CVPRW.2017.151"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"4747","DOI":"10.1109\/TNNLS.2019.2957527","article-title":"Hyperspectral Images Super-Resolution via Learning High-Order Coupled Tensor Ring Representation","volume":"11","author":"Xu","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"3945","DOI":"10.1109\/JSTARS.2023.3252585","article-title":"Gradient Prior Dilated Convolution Network for Remote Sensing Image Super-Resolution","volume":"16","author":"Liu","year":"2023","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1109\/TPAMI.2004.1261081","article-title":"Fundamental limits of reconstruction-based superresolution algorithms under local translation","volume":"26","author":"Lin","year":"2004","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"567","DOI":"10.1109\/34.24792","article-title":"Principal warps: Thin-plate splines and the decomposition of deformations","volume":"31","author":"Bookstein","year":"1989","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_52","first-page":"3499","article-title":"Cross-scale internal graph neural network for image super-resolution","volume":"33","author":"Zhou","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Huang, X., and Belongie, S. (2017, January 22\u201329). Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.167"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1109\/TIP.2003.819861","article-title":"Image quality assessment: From error visibility to structural similarity","volume":"13","author":"Wang","year":"2004","journal-title":"IEEE Trans. Image Process."},{"key":"ref_55","unstructured":"Yuhas, R.H., Goetz, A.F., and Boardman, J.W. (1992, January 1\u20135). Discrimination among semi-arid landscape endmembers using the spectral angle mapper (SAM) algorithm. Proceedings of the Summaries of the Third Annual JPL Airborne Geoscience Workshop, Pasadena, CA, USA."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/24\/5675\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:35:42Z","timestamp":1760132142000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/24\/5675"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,8]]},"references-count":55,"journal-issue":{"issue":"24","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["rs15245675"],"URL":"https:\/\/doi.org\/10.3390\/rs15245675","relation":{},"ISSN":["2072-4292"],"issn-type":[{"type":"electronic","value":"2072-4292"}],"subject":[],"published":{"date-parts":[[2023,12,8]]}}}