{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,3]],"date-time":"2026-04-03T15:32:12Z","timestamp":1775230332583,"version":"3.50.1"},"reference-count":42,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2022,9,20]],"date-time":"2022-09-20T00:00:00Z","timestamp":1663632000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Natural Science Foundation of China","award":["41875122"],"award-info":[{"award-number":["41875122"]}]},{"name":"National Natural Science Foundation of China","award":["U1811464"],"award-info":[{"award-number":["U1811464"]}]},{"name":"National Natural Science Foundation of China","award":["2017YFA0604300"],"award-info":[{"award-number":["2017YFA0604300"]}]},{"name":"National Natural Science Foundation of China","award":["2017YFA0604400"],"award-info":[{"award-number":["2017YFA0604400"]}]},{"name":"National Natural Science Foundation of China","award":["2018XBYJRC004"],"award-info":[{"award-number":["2018XBYJRC004"]}]},{"name":"National Natural Science Foundation of China","award":["2017TQ04Z359"],"award-info":[{"award-number":["2017TQ04Z359"]}]},{"name":"National Key R&amp;D Program of China","award":["41875122"],"award-info":[{"award-number":["41875122"]}]},{"name":"National Key R&amp;D Program of China","award":["U1811464"],"award-info":[{"award-number":["U1811464"]}]},{"name":"National Key R&amp;D Program of China","award":["2017YFA0604300"],"award-info":[{"award-number":["2017YFA0604300"]}]},{"name":"National Key R&amp;D Program of China","award":["2017YFA0604400"],"award-info":[{"award-number":["2017YFA0604400"]}]},{"name":"National Key R&amp;D Program of China","award":["2018XBYJRC004"],"award-info":[{"award-number":["2018XBYJRC004"]}]},{"name":"National Key R&amp;D Program of China","award":["2017TQ04Z359"],"award-info":[{"award-number":["2017TQ04Z359"]}]},{"name":"Western Talents","award":["41875122"],"award-info":[{"award-number":["41875122"]}]},{"name":"Western Talents","award":["U1811464"],"award-info":[{"award-number":["U1811464"]}]},{"name":"Western Talents","award":["2017YFA0604300"],"award-info":[{"award-number":["2017YFA0604300"]}]},{"name":"Western Talents","award":["2017YFA0604400"],"award-info":[{"award-number":["2017YFA0604400"]}]},{"name":"Western Talents","award":["2018XBYJRC004"],"award-info":[{"award-number":["2018XBYJRC004"]}]},{"name":"Western Talents","award":["2017TQ04Z359"],"award-info":[{"award-number":["2017TQ04Z359"]}]},{"name":"Guangdong Top Young Talents","award":["41875122"],"award-info":[{"award-number":["41875122"]}]},{"name":"Guangdong Top Young Talents","award":["U1811464"],"award-info":[{"award-number":["U1811464"]}]},{"name":"Guangdong Top Young Talents","award":["2017YFA0604300"],"award-info":[{"award-number":["2017YFA0604300"]}]},{"name":"Guangdong Top Young Talents","award":["2017YFA0604400"],"award-info":[{"award-number":["2017YFA0604400"]}]},{"name":"Guangdong Top Young Talents","award":["2018XBYJRC004"],"award-info":[{"award-number":["2018XBYJRC004"]}]},{"name":"Guangdong Top Young Talents","award":["2017TQ04Z359"],"award-info":[{"award-number":["2017TQ04Z359"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Sensors onboard satellite platforms with short revisiting periods acquire frequent earth observation data. One limitation to the utility of satellite-based data is missing information in the time series of images due to cloud contamination and sensor malfunction. Most studies on gap-filling and cloud removal process individual images, and existing multi-temporal image restoration methods still have problems in dealing with images that have large areas with frequent cloud contamination. Considering these issues, we proposed a deep learning-based method named content-sequence-texture generation (CSTG) network to generate gap-filled time series of images. The method uses deep neural networks to restore remote sensing images with missing information by accounting for image contents, textures and temporal sequences. We designed a content generation network to preliminarily fill in the missing parts and a sequence-texture generation network to optimize the gap-filling outputs. We used time series of Moderate-resolution Imaging Spectroradiometer (MODIS) data in different regions, which include various surface characteristics in North America, Europe and Asia to train and test the proposed model. Compared to the reference images, the CSTG achieved structural similarity (SSIM) of 0.953 and mean absolute errors (MAE) of 0.016 on average for the restored time series of images in artificial experiments. The developed method could restore time series of images with detailed texture and generally performed better than the other comparative methods, especially with large or overlapped missing areas in time series. Our study provides an available method to gap-fill time series of remote sensing images and highlights the power of the deep learning methods in reconstructing remote sensing images.<\/jats:p>","DOI":"10.3390\/rs14194692","type":"journal-article","created":{"date-parts":[[2022,9,21]],"date-time":"2022-09-21T00:08:09Z","timestamp":1663718889000},"page":"4692","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":24,"title":["Gap-Filling and Missing Information Recovery for Time Series of MODIS Data Using Deep Learning-Based Methods"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1066-9088","authenticated-orcid":false,"given":"Yidan","family":"Wang","sequence":"first","affiliation":[{"name":"School of Geography and Planning, Sun Yat-sen University, Guangzhou 510275, China"}]},{"given":"Xuewen","family":"Zhou","sequence":"additional","affiliation":[{"name":"School of Atmospheric Sciences, Sun Yat-sen University, Zhuhai 519082, China"}]},{"given":"Zurui","family":"Ao","sequence":"additional","affiliation":[{"name":"Beidou Research Institute, Faculty of Engineering, South China Normal University, Foshan 528000, China"}]},{"given":"Kun","family":"Xiao","sequence":"additional","affiliation":[{"name":"School of Geography and Planning, Sun Yat-sen University, Guangzhou 510275, China"}]},{"given":"Chenxi","family":"Yan","sequence":"additional","affiliation":[{"name":"School of Geography and Planning, Sun Yat-sen University, Guangzhou 510275, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1146-4874","authenticated-orcid":false,"given":"Qinchuan","family":"Xin","sequence":"additional","affiliation":[{"name":"School of Geography and Planning, Sun Yat-sen University, Guangzhou 510275, China"}]}],"member":"1968","published-online":{"date-parts":[[2022,9,20]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.rse.2017.11.001","article-title":"Land cover classification and wetland inundation mapping using modis","volume":"204","author":"Georgakakos","year":"2018","journal-title":"Remote Sens. Environ."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Vali, A., Comai, S., and Matteucci, M. (2020). Deep learning for land use and land cover classification based on hyperspectral and multispectral earth observation data: A review. Remote Sens., 12.","DOI":"10.3390\/rs12152495"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Li, Z., Xin, Q., Sun, Y., and Cao, M. (2021). A deep learning-based framework for automated extraction of building footprint polygons from very high-resolution aerial imagery. Remote Sens., 13.","DOI":"10.3390\/rs13183630"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"467","DOI":"10.5194\/bg-16-467-2019","article-title":"A simple time-stepping scheme to simulate leaf area index, phenology, and gross primary production across deciduous broadleaf forests in the eastern united states","volume":"16","author":"Xin","year":"2019","journal-title":"Biogeosciences"},{"key":"ref_5","first-page":"102487","article-title":"Development of a global annual land surface phenology dataset for 1982\u20132018 from the avhrr data by implementing multiple phenology retrieving methods","volume":"103","author":"Wu","year":"2021","journal-title":"Int. J. Appl. Earth Obs."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"271","DOI":"10.1007\/s10333-020-00829-0","article-title":"Assessment of paddy fields\u2019 damage caused by cyclone nargis using modis time-series images (2004\u20132013)","volume":"19","author":"Omori","year":"2021","journal-title":"Paddy Water Environ."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"4400115","DOI":"10.1109\/TGRS.2020.3035469","article-title":"Breaking limits of remote sensing by deep learning from simulated data for flood and debris-flow mapping","volume":"60","author":"Yokoya","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Chen, S., Chen, X., Chen, X., Chen, J., Cao, X., Shen, M., Yang, W., and Cui, X. (2018). A novel cloud removal method based on ihot and the cloud trajectories for landsat imagery. Remote Sens., 10.","DOI":"10.3390\/rs10071040"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"224","DOI":"10.1016\/j.isprsjprs.2014.06.011","article-title":"An effective thin cloud removal procedure for visible remote sensing images","volume":"96","author":"Shen","year":"2014","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"675","DOI":"10.1016\/0098-3004(96)00010-6","article-title":"Atmospheric correction of satellite data with haze removal including a haze\/clear transition region","volume":"22","author":"Richter","year":"1996","journal-title":"Comput. Geosci."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1109\/MSP.2013.2273004","article-title":"Image inpainting: Overview and recent advances","volume":"31","author":"Guillemot","year":"2014","journal-title":"IEEE Signal Proc. Mag."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"788","DOI":"10.1049\/iet-cvi.2016.0009","article-title":"Recovering defective landsat 7 enhanced thematic mapper plus images via multiple linear regression model","volume":"10","author":"Sadiq","year":"2016","journal-title":"IET Comput. Vis."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"654","DOI":"10.1016\/j.isprsjprs.2009.06.001","article-title":"Geostatistical interpolation of slc-off landsat etm plus images","volume":"64","author":"Pringle","year":"2009","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"7650","DOI":"10.1080\/01431161.2014.975377","article-title":"Application of geographically weighted regression to fill gaps in slc-off landsat etm plus satellite imagery","volume":"35","author":"Zhang","year":"2014","journal-title":"Int. J. Remote Sens."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1016\/j.isprsjprs.2014.02.015","article-title":"Cloud removal for remotely sensed images by similar pixel replacement guided with a spatio-temporal mrf model","volume":"92","author":"Cheng","year":"2014","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"569","DOI":"10.1137\/080730421","article-title":"A fast algorithm for edge-preserving variational multichannel image restoration","volume":"2","author":"Yang","year":"2009","journal-title":"SIAM J. Imaging Sci."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"7901","DOI":"10.1109\/TGRS.2020.3038878","article-title":"Spatial\u2013spectral radial basis function-based interpolation for landsat etm+ slc-off image gap filling","volume":"59","author":"Wang","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"232","DOI":"10.1109\/TGRS.2012.2197682","article-title":"Cloud removal from multitemporal satellite images using information cloning","volume":"51","author":"Lin","year":"2013","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"11481","DOI":"10.3390\/rs70911481","article-title":"Removal of thin clouds in landsat-8 oli data with independent component analysis","volume":"7","author":"Shen","year":"2015","journal-title":"Remote Sens."},{"key":"ref_20","first-page":"379","article-title":"Combined haze and cirrus removal for multispectral imagery","volume":"13","author":"Makarau","year":"2016","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1016\/S0034-4257(02)00034-2","article-title":"An image transform to characterize and compensate for spatial variations in thin cloud contamination of landsat images","volume":"82","author":"Zhang","year":"2002","journal-title":"Remote Sens. Environ."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"2409","DOI":"10.1109\/TGRS.2011.2173499","article-title":"Quantitative restoration for modis band 6 on aqua","volume":"50","author":"Gladkova","year":"2012","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"234","DOI":"10.1016\/j.rse.2013.04.002","article-title":"Toward near real-time monitoring of forest disturbance by fusion of modis and landsat data","volume":"135","author":"Xin","year":"2013","journal-title":"Remote Sens. Environ."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.isprsjprs.2019.08.017","article-title":"Large-scale modis aod products recovery: Spatial-temporal hybrid fusion considering aerosol variation mitigation","volume":"157","author":"Wang","year":"2019","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"3112","DOI":"10.1016\/j.rse.2008.03.009","article-title":"Multi-temporal modis-landsat data fusion for relative radiometric normalization, gap filling, and prediction of landsat data","volume":"112","author":"Roy","year":"2008","journal-title":"Remote Sens. Environ."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"111901","DOI":"10.1016\/j.rse.2020.111901","article-title":"Multispectral high resolution sensor fusion for smoothing and gap-filling in the cloud","volume":"247","author":"Maneta","year":"2020","journal-title":"Remote Sens. Environ."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"46151","DOI":"10.1109\/ACCESS.2020.2979291","article-title":"Multi-temporal landsat data automatic cloud removal using poisson blending","volume":"8","author":"Hu","year":"2020","journal-title":"IEEE Access"},{"key":"ref_28","first-page":"584","article-title":"Automatic cloud removal from multi-temporal spot images","volume":"205","author":"Tseng","year":"2008","journal-title":"Appl. Math. Comput."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1053","DOI":"10.1016\/j.rse.2010.12.010","article-title":"A simple and effective method for filling gaps in landsat etm plus slc-off images","volume":"115","author":"Chen","year":"2011","journal-title":"Remote Sens. Environ."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"182","DOI":"10.1016\/j.rse.2012.12.012","article-title":"Recovering missing pixels for landsat etm plus slc-off imagery using multi-temporal regression analysis and a regularization method","volume":"131","author":"Zeng","year":"2013","journal-title":"Remote Sens. Environ."},{"key":"ref_31","first-page":"2672","article-title":"Generative adversarial nets","volume":"Volume 2","author":"Goodfellow","year":"2014","journal-title":"Proceedings of the 27th International Conference on Neural Information Processing Systems"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Zhang, H., Xu, T., Li, H., Zhang, S., Wang, X., Huang, X., and Metaxas, D. (2017, January 22\u201329). Stackgan: Text to photo-realistic image synthesis with stacked generative adversarial networks. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.629"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Zhu, J., Park, T., Isola, P., and Efros, A.A. (2017, January 22\u201329). Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.244"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1109\/JSTARS.2019.2954130","article-title":"Thick clouds removal from multitemporal zy-3 satellite images using deep learning","volume":"13","author":"Chen","year":"2019","journal-title":"IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1016\/j.isprsjprs.2020.02.008","article-title":"Thick cloud and cloud shadow removal in multitemporal imagery using progressively spatio-temporal patch group deep learning","volume":"162","author":"Zhang","year":"2020","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"2865","DOI":"10.1109\/TGRS.2019.2956959","article-title":"Thick cloud removal with optical and sar imagery via convolutional-mapping-deconvolutional network","volume":"58","author":"Li","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1016\/j.rse.2014.12.014","article-title":"Improvement and expansion of the fmask algorithm: Cloud, cloud shadow, and snow detection for landsats 4-7, 8, and sentinel 2 images","volume":"159","author":"Zhu","year":"2015","journal-title":"Remote Sens. Environ."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1016\/j.isprsjprs.2019.02.017","article-title":"Deep learning based cloud detection for medium and high resolution remote sensing images of different sensors","volume":"150","author":"Li","year":"2019","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_39","unstructured":"Shi, X., Chen, Z., Wang, H., Yeung, D.Y., Wong, W.K., and Woo, W.C. (2015). Convolutional Lstm Network: A Machine Learning Approach for Precipitation Nowcasting, MIT Press."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Si, C., Chen, W., Wang, W., Wang, L., and Tan, T. (2019, January 15\u201320). An attention enhanced graph convolutional lstm network for skeleton-based action recognition. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00132"},{"key":"ref_41","unstructured":"Kingma, D., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"4274","DOI":"10.1109\/TGRS.2018.2810208","article-title":"Missing data reconstruction in remote sensing image with a unified spatial-temporal-spectral deep convolutional neural network","volume":"56","author":"Zhang","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/19\/4692\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:35:46Z","timestamp":1760142946000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/19\/4692"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,20]]},"references-count":42,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2022,10]]}},"alternative-id":["rs14194692"],"URL":"https:\/\/doi.org\/10.3390\/rs14194692","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,20]]}}}