{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,28]],"date-time":"2026-03-28T02:06:39Z","timestamp":1774663599486,"version":"3.50.1"},"reference-count":49,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2018,7,5]],"date-time":"2018-07-05T00:00:00Z","timestamp":1530748800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["CNS-1739705"],"award-info":[{"award-number":["CNS-1739705"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Major State Research Development Program of China","award":["No. 2017YFB0503704"],"award-info":[{"award-number":["No. 2017YFB0503704"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["No. 41722109"],"award-info":[{"award-number":["No. 41722109"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Due to technical and budget limitations, there are inevitably some trade-offs in the design of remote sensing instruments, making it difficult to acquire high spatiotemporal resolution remote sensing images simultaneously. To address this problem, this paper proposes a new data fusion model named the deep convolutional spatiotemporal fusion network (DCSTFN), which makes full use of a convolutional neural network (CNN) to derive high spatiotemporal resolution images from remotely sensed images with high temporal but low spatial resolution (HTLS) and low temporal but high spatial resolution (LTHS). The DCSTFN model is composed of three major parts: the expansion of the HTLS images, the extraction of high frequency components from LTHS images, and the fusion of extracted features. The inputs of the proposed network include a pair of HTLS and LTHS reference images from a single day and another HTLS image on the prediction date. Convolution is used to extract key features from inputs, and deconvolution is employed to expand the size of HTLS images. The features extracted from HTLS and LTHS images are then fused with the aid of an equation that accounts for temporal ground coverage changes. The output image on the prediction day has the spatial resolution of LTHS and temporal resolution of HTLS. Overall, the DCSTFN model establishes a complex but direct non-linear mapping between the inputs and the output. Experiments with MODerate Resolution Imaging Spectroradiometer (MODIS) and Landsat Operational Land Imager (OLI) images show that the proposed CNN-based approach not only achieves state-of-the-art accuracy, but is also more robust than conventional spatiotemporal fusion algorithms. In addition, DCSTFN is a faster and less time-consuming method to perform the data fusion with the trained network, and can potentially be applied to the bulk processing of archived data.<\/jats:p>","DOI":"10.3390\/rs10071066","type":"journal-article","created":{"date-parts":[[2018,7,5]],"date-time":"2018-07-05T10:52:44Z","timestamp":1530787964000},"page":"1066","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":142,"title":["Deriving High Spatiotemporal Remote Sensing Images Using Deep Convolutional Network"],"prefix":"10.3390","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0168-4216","authenticated-orcid":false,"given":"Zhenyu","family":"Tan","sequence":"first","affiliation":[{"name":"State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS), Wuhan University, Wuhan 430079, China"},{"name":"Center for Spatial Information Science and Systems (CSISS), George Mason University, Fairfax, VA 22030, USA"}]},{"given":"Peng","family":"Yue","sequence":"additional","affiliation":[{"name":"School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, China"},{"name":"Hubei Province Engineering Center for Intelligent Geoprocessing (HPECIG), Wuhan 430079, China"},{"name":"Collaborative Innovation Center of Geospatial Technology (INNOGST), Wuhan 430079, China"}]},{"given":"Liping","family":"Di","sequence":"additional","affiliation":[{"name":"Center for Spatial Information Science and Systems (CSISS), George Mason University, Fairfax, VA 22030, USA"}]},{"given":"Junmei","family":"Tang","sequence":"additional","affiliation":[{"name":"Center for Spatial Information Science and Systems (CSISS), George Mason University, Fairfax, VA 22030, USA"}]}],"member":"1968","published-online":{"date-parts":[[2018,7,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1016\/j.isprsjprs.2015.10.004","article-title":"Remote sensing platforms and sensors: A survey","volume":"115","author":"Toth","year":"2016","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"415","DOI":"10.1109\/JSTARS.2010.2089575","article-title":"Earth Observation Sensor Web: An Overview","volume":"3","author":"Di","year":"2010","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_3","unstructured":"Di, L. (2007, January 13\u201314). Geospatial sensor web and self-adaptive Earth predictive systems (SEPS). Proceedings of the Earth Science Technology Office (ESTO)\/Advanced Information System Technology (AIST) Sensor Web Principal Investigator (PI) Meeting, San Diego, CA, USA."},{"key":"ref_4","first-page":"83","article-title":"Criteria of selecting satellite data for studying land resources","volume":"15","author":"Alavipanah","year":"2010","journal-title":"Desert"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2610","DOI":"10.1016\/j.rse.2010.05.032","article-title":"An enhanced spatial and temporal adaptive reflectance fusion model for complex heterogeneous regions","volume":"114","author":"Zhu","year":"2010","journal-title":"Remote Sens. Environ."},{"key":"ref_6","first-page":"1","article-title":"Heterogeneous Spatial Data: Fusion, Modeling, and Analysis for GIS Applications","volume":"8","author":"Spagnuolo","year":"2016","journal-title":"Synth. Lect. Vis. Comput. Comput. Gr. Anim. Comput. Photogr. Imag."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1016\/j.rse.2014.02.001","article-title":"Landsat-8: Science and product vision for terrestrial global change research","volume":"145","author":"Roy","year":"2014","journal-title":"Remote Sens. Environ."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1228","DOI":"10.1109\/36.701075","article-title":"The Moderate Resolution Imaging Spectroradiometer (MODIS): Land remote sensing for global change research","volume":"36","author":"Justice","year":"1998","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"929","DOI":"10.14358\/PERS.79.10.929","article-title":"Web-service-based Monitoring and Analysis of Global Agricultural Drought","volume":"79","author":"Deng","year":"2013","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Yang, Z., Di, L., Yu, G., and Chen, Z. (2011, January 24\u201329). Vegetation condition indices for crop vegetation condition monitoring. Proceedings of the 2011 IEEE International Geoscience and Remote Sensing Symposium, Vancouver, BC, Canada.","DOI":"10.1109\/IGARSS.2011.6049984"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1007\/s41651-017-0003-5","article-title":"Delineation of Groundwater Potential Zones in River Basins Using Geospatial Tools\u2014An Example from Southern Western Ghats, Kerala, India","volume":"1","author":"Nair","year":"2017","journal-title":"J. Geovisualiz. Spat. Anal."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2207","DOI":"10.1109\/TGRS.2006.872081","article-title":"On the blending of the Landsat and MODIS surface reflectance: predicting daily Landsat surface reflectance","volume":"44","author":"Gao","year":"2006","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1988","DOI":"10.1016\/j.rse.2009.05.011","article-title":"Generation of dense time series synthetic Landsat data through data blending with MODIS using a spatial and temporal adaptive reflectance fusion model","volume":"113","author":"Hilker","year":"2009","journal-title":"Remote Sens. Environ."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1016\/j.rse.2013.02.007","article-title":"Assessing the accuracy of blending Landsat\u2013MODIS surface reflectances in two landscapes with contrasting spatial and temporal dynamics: A framework for algorithm selection","volume":"133","author":"Emelyanova","year":"2013","journal-title":"Remote Sens. Environ."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1016\/j.rse.2017.05.011","article-title":"Generating a series of fine spatial and temporal resolution land cover maps by fusing coarse spatial resolution remotely sensed images and fine spatial resolution land cover maps","volume":"196","author":"Li","year":"2017","journal-title":"Remote Sens. Environ."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1798","DOI":"10.3390\/rs70201798","article-title":"Comparison of Spatiotemporal Fusion Models: A Review","volume":"7","author":"Chen","year":"2015","journal-title":"Remote Sens."},{"key":"ref_17","first-page":"278","article-title":"The assessment of multi-sensor image fusion using wavelet transforms for mapping the Brazilian Savanna","volume":"8","author":"Clevers","year":"2006","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1613","DOI":"10.1016\/j.rse.2009.03.007","article-title":"A new data fusion model for high spatial- and temporal-resolution mapping of forest disturbance based on Landsat and MODIS","volume":"113","author":"Hilker","year":"2009","journal-title":"Remote Sens. Environ."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"4367","DOI":"10.1080\/01431161.2013.777488","article-title":"A spatial and temporal reflectance fusion model considering sensor observation differences","volume":"34","author":"Shen","year":"2013","journal-title":"Int. J. Remote Sens."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"453","DOI":"10.1109\/LGRS.2008.919685","article-title":"Unmixing-Based Landsat TM and MERIS FR Data Fusion","volume":"5","author":"Clevers","year":"2008","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1016\/j.rse.2015.11.016","article-title":"A flexible spatiotemporal method for fusing satellite images with different resolutions","volume":"172","author":"Zhu","year":"2016","journal-title":"Remote Sens. Environ."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Lu, L., Huang, Y., Di, L., and Hang, D. (2017). A New Spatial Attraction Model for Improving Subpixel Land Cover Classification. Remote Sens., 9.","DOI":"10.3390\/rs9040360"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"3707","DOI":"10.1109\/TGRS.2012.2186638","article-title":"Spatiotemporal Reflectance Fusion via Sparse Representation","volume":"50","author":"Huang","year":"2012","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1016\/j.inffus.2016.12.001","article-title":"Multi-focus image fusion with a deep convolutional neural network","volume":"36","author":"Liu","year":"2017","journal-title":"Inf. Fusion"},{"key":"ref_25","unstructured":"Blum, R.S., and Liu, Z. (2005). Multi-Sensor Image Fusion and Its Applications, CRC Press."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/j.neunet.2014.09.003","article-title":"Deep learning in neural networks: An overview","volume":"61","author":"Schmidhuber","year":"2015","journal-title":"Neural Netw."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.neucom.2016.12.038","article-title":"A survey of deep neural network architectures and their applications","volume":"234","author":"Liu","year":"2017","journal-title":"Neurocomputing"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Kim, Y. (arXiv, 2014). Convolutional neural networks for sentence classification, arXiv.","DOI":"10.3115\/v1\/D14-1181"},{"key":"ref_30","first-page":"1097","article-title":"ImageNet Classification with Deep Convolutional Neural Networks","volume":"Volume 25","author":"Pereira","year":"2012","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref_31","first-page":"1995","article-title":"Convolutional networks for images, speech, and time series","volume":"3361","author":"LeCun","year":"1995","journal-title":"Handb. Brain Theory Neural Netw."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Zeiler, M.D., Taylor, G.W., and Fergus, R. (2011, January 6\u201313). Adaptive deconvolutional networks for mid and high level feature learning. Proceedings of the 2011 International Conference on Computer Vision, Barcelona, Spain.","DOI":"10.1109\/ICCV.2011.6126474"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Fleet, D., Pajdla, T., Schiele, B., and Tuytelaars, T. (2014). Visualizing and Understanding Convolutional Networks. Computer Vision\u2014ECCV 2014, Springer International Publishing.","DOI":"10.1007\/978-3-319-10599-4"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"640","DOI":"10.1109\/TPAMI.2016.2572683","article-title":"Fully Convolutional Networks for Semantic Segmentation","volume":"39","author":"Shelhamer","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_35","unstructured":"Radford, A., Metz, L., and Chintala, S. (arXiv, 2015). Unsupervised representation learning with deep convolutional generative adversarial networks, arXiv."},{"key":"ref_36","unstructured":"Dumoulin, V., and Visin, F. (arXiv, 2016). A guide to convolution arithmetic for deep learning, arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1109\/TPAMI.2015.2439281","article-title":"Image Super-Resolution Using Deep Convolutional Networks","volume":"38","author":"Dong","year":"2016","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Masi, G., Cozzolino, D., Verdoliva, L., and Scarpa, G. (2016). Pansharpening by Convolutional Neural Networks. Remote Sens., 8.","DOI":"10.3390\/rs8070594"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1795","DOI":"10.1109\/LGRS.2017.2736020","article-title":"Boosting the Accuracy of Multispectral Image Pansharpening by Learning a Deep Residual Network","volume":"14","author":"Wei","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1253","DOI":"10.1109\/LGRS.2017.2704625","article-title":"Deep Fusion of Remote Sensing Data for Accurate Classification","volume":"14","author":"Chen","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_41","unstructured":"Nair, V., and Hinton, G.E. (2010, January 21\u201324). Rectified Linear Units Improve Restricted Boltzmann Machines. Proceedings of the 27th International Conference on International Conference on Machine Learning (ICML\u201910), Madison, WI, USA."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1109\/MGRS.2017.2762307","article-title":"Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources","volume":"5","author":"Zhu","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_43","unstructured":"Chollet, F. (2018, June 29). Keras. Available online: https:\/\/github.com\/keras-team\/keras."},{"key":"ref_44","first-page":"265","article-title":"TensorFlow: A System for Large-Scale Machine Learning","volume":"16","author":"Abadi","year":"2016","journal-title":"OSDI"},{"key":"ref_45","unstructured":"Kingma, D.P., and Ba, J. (arXiv, 2014). Adam: A method for stochastic optimization, arXiv."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1016\/j.jhydrol.2009.08.003","article-title":"Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling","volume":"377","author":"Gupta","year":"2009","journal-title":"J. Hydrol."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1109\/TIP.2003.819861","article-title":"Image quality assessment: from error visibility to structural similarity","volume":"13","author":"Wang","year":"2004","journal-title":"IEEE Trans. Image Process."},{"key":"ref_48","unstructured":"Ioffe, S., and Szegedy, C. (arXiv, 2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift, arXiv."},{"key":"ref_49","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/10\/7\/1066\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:11:24Z","timestamp":1760195484000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/10\/7\/1066"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,7,5]]},"references-count":49,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2018,7]]}},"alternative-id":["rs10071066"],"URL":"https:\/\/doi.org\/10.3390\/rs10071066","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,7,5]]}}}