{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,31]],"date-time":"2026-03-31T14:15:12Z","timestamp":1774966512171,"version":"3.50.1"},"reference-count":53,"publisher":"MDPI AG","issue":"24","license":[{"start":{"date-parts":[[2019,12,5]],"date-time":"2019-12-05T00:00:00Z","timestamp":1575504000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["CNS-1739705"],"award-info":[{"award-number":["CNS-1739705"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"name":"EarthCube program","award":["AGS-1740693"],"award-info":[{"award-number":["AGS-1740693"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["No. 41722109"],"award-info":[{"award-number":["No. 41722109"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["No. 91738302"],"award-info":[{"award-number":["No. 91738302"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003819","name":"Natural Science Foundation of Hubei Province","doi-asserted-by":"publisher","award":["No. 2018CFA053"],"award-info":[{"award-number":["No. 2018CFA053"]}],"id":[{"id":"10.13039\/501100003819","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Wuhan Yellow Crane Talents (Science) Program","award":["2016"],"award-info":[{"award-number":["2016"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Earth observation data with high spatiotemporal resolution are critical for dynamic monitoring and prediction in geoscience applications, however, due to some technique and budget limitations, it is not easy to acquire satellite images with both high spatial and high temporal resolutions. Spatiotemporal image fusion techniques provide a feasible and economical solution for generating dense-time data with high spatial resolution, pushing the limits of current satellite observation systems. Among existing various fusion algorithms, deeplearningbased models reveal a promising prospect with higher accuracy and robustness. This paper refined and improved the existing deep convolutional spatiotemporal fusion network (DCSTFN) to further boost model prediction accuracy and enhance image quality. The contributions of this paper are twofold. First, the fusion result is improved considerably with brand-new network architecture and a novel compound loss function. Experiments conducted in two different areas demonstrate these improvements by comparing them with existing algorithms. The enhanced DCSTFN model shows superior performance with higher accuracy, vision quality, and robustness. Second, the advantages and disadvantages of existing deeplearningbased spatiotemporal fusion models are comparatively discussed and a network design guide for spatiotemporal fusion is provided as a reference for future research. Those comparisons and guidelines are summarized based on numbers of actual experiments and have promising potentials to be applied for other image sources with customized spatiotemporal fusion networks.<\/jats:p>","DOI":"10.3390\/rs11242898","type":"journal-article","created":{"date-parts":[[2019,12,5]],"date-time":"2019-12-05T03:16:36Z","timestamp":1575515796000},"page":"2898","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":132,"title":["An Enhanced Deep Convolutional Model for Spatiotemporal Image Fusion"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0168-4216","authenticated-orcid":false,"given":"Zhenyu","family":"Tan","sequence":"first","affiliation":[{"name":"State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS), Wuhan University, Wuhan 430079, China"},{"name":"Center for Spatial Information Science and Systems (CSISS), George Mason University, Fairfax, VA 22030, USA"}]},{"given":"Liping","family":"Di","sequence":"additional","affiliation":[{"name":"Center for Spatial Information Science and Systems (CSISS), George Mason University, Fairfax, VA 22030, USA"}]},{"given":"Mingda","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9684-0204","authenticated-orcid":false,"given":"Liying","family":"Guo","sequence":"additional","affiliation":[{"name":"Center for Spatial Information Science and Systems (CSISS), George Mason University, Fairfax, VA 22030, USA"}]},{"given":"Meiling","family":"Gao","sequence":"additional","affiliation":[{"name":"School of Resource and Environmental Sciences, Wuhan University, Wuhan 430079, China"}]}],"member":"1968","published-online":{"date-parts":[[2019,12,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Xiaolin, Z., Fangyi, C., Jiaqi, T., and Trecia, W. (2018). Spatiotemporal Fusion of Multisource Remote Sensing Data: Literature Survey, Taxonomy, Principles, Applications, and Future Directions. Remote Sens., 10.","DOI":"10.3390\/rs10040527"},{"key":"ref_2","first-page":"132","article-title":"Multitemporal fusion of Landsat\/TM and ENVISAT\/MERIS for crop monitoring","volume":"23","author":"Alonso","year":"2013","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1016\/j.rse.2011.10.014","article-title":"Evaluation of Landsat and MODIS data fusion products for analysis of dryland forest phenology","volume":"117","author":"Walker","year":"2012","journal-title":"Remote Sens. Environ."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1775","DOI":"10.1080\/01431160110075802","article-title":"Using a time series of satellite imagery to detect land use and land cover changes in the Atlanta, Georgia metropolitan area","volume":"23","author":"Yang","year":"2002","journal-title":"Int. J. Remote Sens."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1798","DOI":"10.3390\/rs70201798","article-title":"Comparison of Spatiotemporal Fusion Models: A Review","volume":"7","author":"Chen","year":"2015","journal-title":"Remote Sens."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"7135","DOI":"10.1109\/TGRS.2016.2596290","article-title":"An Integrated Framework for the Spatio\u2013Temporal\u2013Spectral Fusion of Remote Sensing Images","volume":"54","author":"Shen","year":"2016","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2207","DOI":"10.1109\/TGRS.2006.872081","article-title":"On the blending of the Landsat and MODIS surface reflectance: Predicting daily Landsat surface reflectance","volume":"44","author":"Gao","year":"2006","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1988","DOI":"10.1016\/j.rse.2009.05.011","article-title":"Generation of dense time series synthetic Landsat data through data blending with MODIS using a spatial and temporal adaptive reflectance fusion model","volume":"113","author":"Hilker","year":"2009","journal-title":"Remote Sens. Environ."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1016\/j.inffus.2011.08.001","article-title":"Multisensor data fusion: A review of the state-of-the-art","volume":"14","author":"Khaleghi","year":"2013","journal-title":"Inf. Fusion"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Belgiu, M., and Stein, A. (2019). Spatiotemporal Image Fusion in Remote Sensing. Remote Sens., 11.","DOI":"10.3390\/rs11070818"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1016\/j.rse.2014.02.001","article-title":"Landsat-8: Science and product vision for terrestrial global change research","volume":"145","author":"Roy","year":"2014","journal-title":"Remote Sens. Environ."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1228","DOI":"10.1109\/36.701075","article-title":"The Moderate Resolution Imaging Spectroradiometer (MODIS): Land remote sensing for global change research","volume":"36","author":"Justice","year":"1998","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Tan, Z., Yue, P., Di, L., and Tang, J. (2018). Deriving High Spatiotemporal Remote Sensing Images Using Deep Convolutional Network. Remote Sens., 10.","DOI":"10.3390\/rs10071066"},{"key":"ref_14","first-page":"278","article-title":"The assessment of multi-sensor image fusion using wavelet transforms for mapping the Brazilian Savanna","volume":"8","author":"Clevers","year":"2006","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1016\/j.inffus.2016.03.003","article-title":"A review of remote sensing image fusion methods","volume":"32","author":"Ghassemian","year":"2016","journal-title":"Inf. Fusion"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1613","DOI":"10.1016\/j.rse.2009.03.007","article-title":"A new data fusion model for high spatial- and temporal-resolution mapping of forest disturbance based on Landsat and MODIS","volume":"113","author":"Hilker","year":"2009","journal-title":"Remote Sens. Environ."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1016\/j.rse.2015.11.016","article-title":"A flexible spatiotemporal method for fusing satellite images with different resolutions","volume":"172","author":"Zhu","year":"2016","journal-title":"Remote Sens. Environ."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Lu, L., Huang, Y., Di, L., and Hang, D. (2017). A New Spatial Attraction Model for Improving Subpixel Land Cover Classification. Remote Sens., 9.","DOI":"10.3390\/rs9040360"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"561","DOI":"10.1080\/2150704X.2013.769283","article-title":"Unified fusion of remote-sensing imagery: Generating simultaneously high-resolution synthetic spatial\u2013temporal\u2013spectral earth observations","volume":"4","author":"Huang","year":"2013","journal-title":"Remote Sens. Lett."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Xue, J., Leung, Y., and Fung, T. (2017). A Bayesian Data Fusion Approach to Spatio-Temporal Fusion of Remotely Sensed Images. Remote Sens., 9.","DOI":"10.3390\/rs9121310"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.agrformet.2013.11.001","article-title":"Mapping daily evapotranspiration at field scales over rainfed and irrigated agricultural areas using remote sensing data fusion","volume":"186","author":"Cammalleri","year":"2014","journal-title":"Agric. For. Meteorol."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"3707","DOI":"10.1109\/TGRS.2012.2186638","article-title":"Spatiotemporal Reflectance Fusion via Sparse Representation","volume":"50","author":"Huang","year":"2012","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"821","DOI":"10.1109\/JSTARS.2018.2797894","article-title":"Spatiotemporal Satellite Image Fusion Using Deep Convolutional Neural Networks","volume":"11","author":"Song","year":"2018","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1883","DOI":"10.1109\/TGRS.2012.2213095","article-title":"Spatiotemporal Satellite Image Fusion Through One-Pair Image Learning","volume":"51","author":"Song","year":"2013","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/j.neunet.2014.09.003","article-title":"Deep learning in neural networks: An overview","volume":"61","author":"Schmidhuber","year":"2015","journal-title":"Neural Netw."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Ducournau, A., and Fablet, R. (2016, January 4). Deep learning for ocean remote sensing: An application of convolutional neural networks for super-resolution on satellite-derived SST data. Proceedings of the 2016 9th IAPR Workshop on Pattern Recogniton in Remote Sensing (PRRS), Cancun, Mexico.","DOI":"10.1109\/PRRS.2016.7867019"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1016\/j.inffus.2016.12.001","article-title":"Multi-focus image fusion with a deep convolutional neural network","volume":"36","author":"Liu","year":"2017","journal-title":"Inf. Fusion"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Masi, G., Cozzolino, D., Verdoliva, L., and Scarpa, G. (2016). Pansharpening by Convolutional Neural Networks. Remote Sens., 8.","DOI":"10.3390\/rs8070594"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1795","DOI":"10.1109\/LGRS.2017.2736020","article-title":"Boosting the Accuracy of Multispectral Image Pansharpening by Learning a Deep Residual Network","volume":"14","author":"Wei","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"639","DOI":"10.1109\/LGRS.2017.2668299","article-title":"Multispectral and Hyperspectral Image Fusion Using a 3-D-Convolutional Neural Network","volume":"14","author":"Palsson","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Scarpa, G., Gargiulo, M., Mazza, A., and Gaetano, R. (2018). A CNN-Based Fusion Method for Feature Extraction from Sentinel Data. Remote Sens., 10.","DOI":"10.3390\/rs10020236"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Liu, X., Deng, C., Chanussot, J., Hong, D., and Zhao, B. (2019). StfNet: A Two-Stream Convolutional Neural Network for Spatiotemporal Image Fusion. IEEE Trans. Geosci. Remote. Sens., 1\u201313.","DOI":"10.1109\/TGRS.2019.2907310"},{"key":"ref_34","first-page":"47","article-title":"Loss Functions for Neural Networks for Image Processing","volume":"3","author":"Zhao","year":"2015","journal-title":"ArXiv"},{"key":"ref_35","unstructured":"Dumoulin, V., and Visin, F. (2016). A Guide to Convolution Arithmetic for Deep Learning. arXiv."},{"key":"ref_36","unstructured":"Wu, B., Duan, H., Liu, Z., and Sun, G. (2017). SRPGAN: Perceptual Generative Adversarial Network for Single Image Super Resolution. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.neucom.2016.12.038","article-title":"A survey of deep neural network architectures and their applications","volume":"234","author":"Liu","year":"2017","journal-title":"Neurocomputing"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1109\/TIP.2003.819861","article-title":"Image quality assessment: From error visibility to structural similarity","volume":"13","author":"Wang","year":"2004","journal-title":"IEEE Trans. Image Process."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"3112","DOI":"10.1016\/j.rse.2008.03.009","article-title":"Multi-temporal MODIS\u2013Landsat data fusion for relative radiometric normalization, gap filling, and prediction of Landsat data","volume":"112","author":"Roy","year":"2008","journal-title":"Remote Sens. Environ."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Odena, A., Dumoulin, V., and Olah, C. (2016). Deconvolution and Checkerboard Artifacts. Distill.","DOI":"10.23915\/distill.00003"},{"key":"ref_41","unstructured":"Vermote, E. (2015). MOD09A1 MODIS\/Terra Surface Reflectance 8-Day L3 Global 500m SIN Grid V006. NASA EOSDIS Land Process. DAAC, 10."},{"key":"ref_42","unstructured":"Paszke, A., Gross, S., Chintala, S., and Chanan, G. (2019, December 04). Pytorch: Tensors and Dynamic Neural Networks in Python with Strong GPU Acceleration. Available online: https:\/\/github.com\/pytorch\/pytorch."},{"key":"ref_43","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A Method for Stochastic Optimization. arXiv."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1016\/j.aqpro.2015.02.019","article-title":"A Review of Quality Metrics for Fused Image","volume":"4","author":"Jagalingam","year":"2015","journal-title":"Aquat. Procedia"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Wang, Q., Yu, D., and Shen, Y. (2009, January 5\u20137). An overview of image fusion metrics. Proceedings of the 2009 IEEE Instrumentation and Measurement Technology Conference, Singapore.","DOI":"10.1109\/IMTC.2009.5168582"},{"key":"ref_46","unstructured":"Yuhas, R.H., Goetz, A.F., and Boardman, J.W. (1992, January 1\u20135). Discrimination among semi-arid landscape endmembers using the spectral angle mapper (SAM) algorithm. Proceedings of the Summaries of the Third Annual JPL Airborne Geoscience Workshop, Pasadena, CA, USA."},{"key":"ref_47","first-page":"49","article-title":"Fusion of high spatial and spectral resolution images: The ARSIS concept and its implementation","volume":"66","author":"Ranchin","year":"2000","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_48","unstructured":"Ioffe, S., and Szegedy, C. (2015). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. arXiv."},{"key":"ref_49","unstructured":"Vedaldi, V.L.D.U.A. (2016). Instance Normalization: The Missing Ingredient for Fast Stylization. arXiv."},{"key":"ref_50","unstructured":"Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014, January 8\u201313). Generative adversarial nets. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Sajjadi, M.S., Sch\u00f6lkopf, B., and Hirsch, M. (2017, January 22\u201329). Enhancenet: Single image super-resolution through automated texture synthesis. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.481"},{"key":"ref_52","unstructured":"Shridhar, K., Laumann, F., and Liwicki, M. (2019). A comprehensive guide to bayesian convolutional neural network with variational inference. arXiv."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1186\/s40537-016-0043-6","article-title":"A survey of transfer learning","volume":"3","author":"Weiss","year":"2016","journal-title":"J. Big Data"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/11\/24\/2898\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:40:21Z","timestamp":1760190021000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/11\/24\/2898"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,12,5]]},"references-count":53,"journal-issue":{"issue":"24","published-online":{"date-parts":[[2019,12]]}},"alternative-id":["rs11242898"],"URL":"https:\/\/doi.org\/10.3390\/rs11242898","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,12,5]]}}}