{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,11]],"date-time":"2026-07-11T15:25:44Z","timestamp":1783783544703,"version":"3.55.0"},"reference-count":59,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2023,2,21]],"date-time":"2023-02-21T00:00:00Z","timestamp":1676937600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"Natural Science Foundation of China","doi-asserted-by":"publisher","award":["42271367"],"award-info":[{"award-number":["42271367"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>In the field of remote sensing, cloud and cloud shadow will result in optical remote sensing image contamination, particularly high cloud cover, which will result in the complete loss of certain ground object information. The presence of thick cloud severely limits the use of optical images in production and scientific research, so it is critical to conduct further research into removing the thick cloud occlusion in optical images to improve the utilization rate of optical images. The state-of-the-art cloud removal methods proposed are largely based on convolutional neural network (CNN). However, due to CNN\u2019s inability to gather global content information, those cloud removal approaches cannot be improved further. Inspired by the transformer and multisource image fusion cloud removal method, we propose a transformer-based cloud removal method (Former-CR), which directly reconstructs cloudless images from SAR images and cloudy optical images. The transformer-based model can efficiently extract and fuse global and local context information in SAR and optical images, generating high-quality cloudless images with higher global consistency. In order to enhance the global structure, local details, and visual effect of the reconstructed image, we design a new loss function to guide the image reconstruction. A comparison with several SAR-based cloud removal methods through qualitative and quantitative experimental evaluation on the SEN12MS-CR dataset demonstrates that our proposed method is effective and superior.<\/jats:p>","DOI":"10.3390\/rs15051196","type":"journal-article","created":{"date-parts":[[2023,2,22]],"date-time":"2023-02-22T01:39:47Z","timestamp":1677029987000},"page":"1196","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":52,"title":["Former-CR: A Transformer-Based Thick Cloud Removal Method with Optical and SAR Imagery"],"prefix":"10.3390","volume":"15","author":[{"given":"Shuning","family":"Han","sequence":"first","affiliation":[{"name":"College of Surveying and Geo-Informatics, Tongji University, Shanghai 200092, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jianmei","family":"Wang","sequence":"additional","affiliation":[{"name":"College of Surveying and Geo-Informatics, Tongji University, Shanghai 200092, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shaoming","family":"Zhang","sequence":"additional","affiliation":[{"name":"College of Surveying and Geo-Informatics, Tongji University, Shanghai 200092, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,2,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2865","DOI":"10.1109\/TGRS.2019.2956959","article-title":"Thick Cloud Removal with Optical and SAR Imagery via Convolutional-Mapping-Deconvolutional Network","volume":"58","author":"Li","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1016\/0034-4257(94)90057-4","article-title":"Kriging in the shadows: Geostatistical interpolation for remote sensing","volume":"49","author":"Rossi","year":"1994","journal-title":"Remote Sens. Environ."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"5644","DOI":"10.1080\/01431161.2012.666363","article-title":"Remote-sensing image analysis and geostatistics","volume":"33","year":"2012","journal-title":"Int. J. Remote Sens."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2363","DOI":"10.1109\/TGRS.2008.2010454","article-title":"A bandelet-based inpainting technique for clouds removal from remotely sensed images","volume":"47","author":"Maalouf","year":"2009","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1163","DOI":"10.1109\/LSP.2017.2703092","article-title":"Missing information reconstruction for single remote sensing images using structure-preserving global optimization","volume":"24","author":"Cheng","year":"2017","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Meng, F., Yang, X., Zhou, C., and Li, Z. (2017). A Sparse Dictionary Learning-Based Adaptive Patch Inpainting Method for Thick Clouds Removal from High-Spatial Resolution Remote Sensing Imagery. Sensors, 17.","DOI":"10.3390\/s17092130"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"6371","DOI":"10.1109\/TGRS.2020.3027819","article-title":"Single Image Cloud Removal Using U-Net and Generative Adversarial Networks","volume":"59","author":"Zheng","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"232","DOI":"10.1109\/TGRS.2012.2197682","article-title":"Cloud removal from multitemporal satellite images using information cloning","volume":"51","author":"Lin","year":"2013","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1255","DOI":"10.1007\/s12524-018-0806-y","article-title":"A Cloud Removal Algorithm to Generate Cloud and Cloud Shadow Free Images Using Information Cloning","volume":"46","author":"Kalkan","year":"2018","journal-title":"J. Indian Soc. Remote Sens."},{"key":"ref_10","unstructured":"Storey, J., Scaramuzza, P., Schmidt, G., and Barsi, J. (2005, January 23\u201327). Landsat 7 scan line corrector-off gap-filled product development. Proceedings of the Pecora 16 Conference on Global Priorities in Land Remote Sensing, Sioux Falls, SD, USA."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Zhang, X., Qin, F., and Qin, Y. (2010, January 29\u201331). Study on the thick cloud removal method based on multi-temporal remote sensing images. Proceedings of the 2010 International Conference on Multimedia Technology, Ningbo, China.","DOI":"10.1109\/ICMULT.2010.5631017"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Du, W., Qin, Z., Fan, J., Gao, M., Wang, F., and Abbasi, B. (2019). An efficient approach to remove thick cloud in VNIR bands of multi-temporal remote sensing images. Remote Sens., 11.","DOI":"10.3390\/rs11111284"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1016\/j.isprsjprs.2018.04.005","article-title":"A two-step framework for reconstructing remotely sensed land surface temperatures contaminated by cloud","volume":"141","author":"Zeng","year":"2018","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Li, Z., Shen, H., Cheng, Q., Li, W., and Zhang, L. (2019). Thick cloud removal in high-resolution satellite images using stepwise radiometric adjustment and residual correction. Remote Sens., 11.","DOI":"10.3390\/rs11161925"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1016\/j.isprsjprs.2014.02.015","article-title":"Cloud removal for remotely sensed images by similar pixel replacement guided with a spatio-temporal mrf model","volume":"92","author":"Cheng","year":"2014","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1109\/TGRS.2012.2237408","article-title":"Patch-based information reconstruction of cloud-contaminated multitemporal images","volume":"52","author":"Lin","year":"2013","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"5963","DOI":"10.1109\/TGRS.2019.2903594","article-title":"A Coarse-to-Fine Framework for Cloud Removal in Remote Sensing Image Sequence","volume":"57","author":"Zhang","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1090","DOI":"10.1109\/LGRS.2018.2829028","article-title":"Two-pass robust component analysis for cloud removal in satellite image sequence","volume":"15","author":"Wen","year":"2018","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"7086","DOI":"10.1109\/TGRS.2014.2307354","article-title":"Recovering quantitative remote sensing products contaminated by thick clouds and shadows using multitemporal dictionary learning","volume":"52","author":"Li","year":"2014","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.isprsjprs.2015.03.009","article-title":"Sparse-based reconstruction of missing information in remote sensing images from spectral\/temporal complementary information","volume":"106","author":"Li","year":"2015","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"3629","DOI":"10.1109\/JSTARS.2016.2533547","article-title":"Patch matching-based multitemporal group sparse representation for the missing information reconstruction of remote-sensing images","volume":"9","author":"Li","year":"2016","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"2998","DOI":"10.1109\/TGRS.2015.2509860","article-title":"Cloud removal based on sparse representation via multitemporal dictionary learning","volume":"54","author":"Xu","year":"2016","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"4274","DOI":"10.1109\/TGRS.2018.2810208","article-title":"Missing data reconstruction in remote sensing image with a unifified spatial-temporal-spectral deep convolutional neural network","volume":"56","author":"Zhang","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1109\/MGRS.2015.2441912","article-title":"Missing information reconstruction of remote sensing data: A technical review","volume":"3","author":"Shen","year":"2015","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"862","DOI":"10.1109\/JSTARS.2019.2898348","article-title":"A spatiotemporal fusion based cloud removal method for remote sensing images with land cover changes","volume":"12","author":"Shen","year":"2019","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"6646","DOI":"10.1080\/01431161.2018.1466068","article-title":"Cloud removal for hyperspectral remotely sensed images based on hyperspectral information fusion","volume":"39","author":"Zhang","year":"2018","journal-title":"Int. J. Remote Sens."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1016\/j.isprsjprs.2018.12.013","article-title":"Cloud removal in remote sensing images using nonnegative matrix factorization and error correction","volume":"148","author":"Li","year":"2019","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_28","first-page":"410","article-title":"Cloud removal of optical image using SAR data for ALOS applications. Experimenting on simulated ALOS data","volume":"29","author":"Hoan","year":"2009","journal-title":"J. Remote Sens. Soc. Japan"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2973","DOI":"10.3390\/rs5062973","article-title":"Removal of optically thick clouds from multi-spectral satellite images using multi-frequency sar data","volume":"5","author":"Eckardt","year":"2013","journal-title":"Remote Sens."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"5","DOI":"10.5194\/isprs-annals-IV-1-5-2018","article-title":"Sar to optical image synthesis for cloud removal with generative adversarial networks","volume":"4","author":"Bermudez","year":"2018","journal-title":"ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Grohnfeldt, C., Schmitt, M., and Zhu, X. (2018, January 22\u201327). A conditional generative adversarial network to fuse sar and multispectral optical data for cloud removal from sentinel-2 images. Proceedings of the IGARSS 2018\u20142018 IEEE International Geoscience and Remote Sensing Symposium, Valencia, Spain.","DOI":"10.1109\/IGARSS.2018.8519215"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Gao, J., Yuan, Q., Li, J., Zhang, H., and Su, X. (2020). Cloud Removal with Fusion of High Resolution Optical and SAR Images Using Generative Adversarial Networks. Remote Sens., 12.","DOI":"10.3390\/rs12010191"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"4105309","DOI":"10.1109\/TGRS.2021.3131035","article-title":"Cloud Removal in Remote Sensing Images Using Generative Adversarial Networks and SAR-to-Optical Image Translation","volume":"60","author":"Darbaghshahi","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"333","DOI":"10.1016\/j.isprsjprs.2020.05.013","article-title":"Cloud removal in sentinel-2 imagery using a deep residual neural network and SAR-optical data fusion","volume":"166","author":"Meraner","year":"2020","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_35","first-page":"5998","article-title":"Attention is all you need","volume":"30","author":"Ashish","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Chen, H.T., Wang, Y.H., Guo, T.Y., Xu, C., Deng, Y.P., Liu, Z.H., Ma, S.W., Xu, C.J., Xu, C., and Gao, W. (2021, January 19\u201325). Pre-Trained Image Processing Transformer. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.01212"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Liang, J., Cao, J., Sun, G., Zhang, K., Van Gool, L., and Timofte, R. (2021, January 11\u201317). Swinir: Image restoration using swin transformer. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, QC, Canada.","DOI":"10.1109\/ICCVW54120.2021.00210"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Wang, Z., Cun, X., Bao, J., Zhou, W., Liu, J., and Li, H. (2022, January 18\u201324). Uformer: A general u-shaped transformer for image restoration. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01716"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Fuentes Reyes, M., Auer, S., Merkle, N., Henry, C., and Schmitt, M. (2019). Sar-to-optical image translation based on conditional generative adversarial networks\u2014Optimization, opportunities and limits. Remote Sens., 11.","DOI":"10.3390\/rs11172067"},{"key":"ref_40","unstructured":"Dong, C., Loy, C.C., He, K., and Tang, X. (2014). European Conference on Computer Vision, Springer."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"3142","DOI":"10.1109\/TIP.2017.2662206","article-title":"Beyond a Gaussian Denoiser: Residual learning of deep CNN for image denoising","volume":"26","author":"Zhang","year":"2017","journal-title":"IEEE Trans. Image Process."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Cavigelli, L., Hager, P., and Benini, L. (2017, January 14\u201319). CAS-CNN: A deep convolutional neural network for image compression artifact suppression. Proceedings of the 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, USA.","DOI":"10.1109\/IJCNN.2017.7965927"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"6360","DOI":"10.1109\/TPAMI.2021.3088914","article-title":"Plug-and-play image restoration with deep denoiser prior","volume":"44","author":"Zhang","year":"2021","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Wang, X., Yu, K., Wu, S., Gu, J., Liu, Y., Dong, C., and Change Loy, C. (2018, January 8\u201314). Esrgan: Enhanced super-resolution generative adversarial networks. Proceedings of the European Conference on Computer Vision (ECCV) Workshops, Munich, Germany.","DOI":"10.1007\/978-3-030-11021-5_5"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Tian, Y., Kong, Y., Zhong, B., and Fu, Y. (2018, January 18\u201323). Residual dense network for image super-resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00262"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"268","DOI":"10.1016\/j.isprsjprs.2022.08.002","article-title":"GLF-CR: SAR-enhanced cloud removal with global\u2013local fusion","volume":"192","author":"Xu","year":"2022","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_47","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., and Houlsby, N. (2021, January 1\u20135). An image is worth 16 \u00d7 16 words: Transformers for image recognition at scale. Proceedings of the International Conference on Learning Representations, Vienna, Austria."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 10\u201317). Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.00986"},{"key":"ref_49","first-page":"30392","article-title":"Early convolutions help transformers see better","volume":"34","author":"Xiao","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Wu, H., Xiao, B., Codella, N., Liu, M., Dai, X., Yuan, L., and Zhang, L. (2021, January 10\u201317). Cvt: Introducing convolutions to vision transformers. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.00009"},{"key":"ref_51","unstructured":"Li, Y., Zhang, K., Cao, J., Timofte, R., and VanGool, L. (2021). LocalViT: Bringing Locality to Vision Transformers. arXiv."},{"key":"ref_52","unstructured":"Hendrycks, D., and Gimpel, K. (2016). Gaussian error linear units (gelus). arXiv."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Hu, J., Shen, L., and Sun, G. (2018, January 18\u201323). Squeeze-and-Excitation Networks. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00745"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Zhang, R., Isola, P., Efros, A.A., Shechtman, E., and Wang, O. (2018, January 18\u201323). The unreasonable effectiveness of deep features as a perceptual metric. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00068"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"5866","DOI":"10.1109\/TGRS.2020.3024744","article-title":"Multisensor data fusion for cloud removal in global and all-season sentinel-2 imagery. IEEE Trans","volume":"59","author":"Ebel","year":"2021","journal-title":"Geosci. Remote Sens."},{"key":"ref_57","unstructured":"Loshchilov, I., and Hutter, F. (2017). Decoupled weight decay regularization. arXiv."},{"key":"ref_58","first-page":"234","article-title":"U-net: Convolutional networks for biomedical image segmentation","volume":"Volume 9351","author":"Ronneberger","year":"2015","journal-title":"Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention"},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Lim, B., Son, S., Kim, H., Nah, S., and Mu Lee, K. (2017, January 21\u201326). Enhanced deep residual networks for single image super-resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA.","DOI":"10.1109\/CVPRW.2017.151"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/5\/1196\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:38:45Z","timestamp":1760121525000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/5\/1196"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,21]]},"references-count":59,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2023,3]]}},"alternative-id":["rs15051196"],"URL":"https:\/\/doi.org\/10.3390\/rs15051196","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,21]]}}}