{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,8]],"date-time":"2026-07-08T22:04:32Z","timestamp":1783548272333,"version":"3.55.0"},"reference-count":52,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2023,3,31]],"date-time":"2023-03-31T00:00:00Z","timestamp":1680220800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["42201063"],"award-info":[{"award-number":["42201063"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["ZDYF2021SHFZ260"],"award-info":[{"award-number":["ZDYF2021SHFZ260"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["322QN345"],"award-info":[{"award-number":["322QN345"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["520QN295"],"award-info":[{"award-number":["520QN295"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Key Research and Development Program of Hainan Province","award":["42201063"],"award-info":[{"award-number":["42201063"]}]},{"name":"Key Research and Development Program of Hainan Province","award":["ZDYF2021SHFZ260"],"award-info":[{"award-number":["ZDYF2021SHFZ260"]}]},{"name":"Key Research and Development Program of Hainan Province","award":["322QN345"],"award-info":[{"award-number":["322QN345"]}]},{"name":"Key Research and Development Program of Hainan Province","award":["520QN295"],"award-info":[{"award-number":["520QN295"]}]},{"name":"Hainan Provincial Natural Science Foundation of China","award":["42201063"],"award-info":[{"award-number":["42201063"]}]},{"name":"Hainan Provincial Natural Science Foundation of China","award":["ZDYF2021SHFZ260"],"award-info":[{"award-number":["ZDYF2021SHFZ260"]}]},{"name":"Hainan Provincial Natural Science Foundation of China","award":["322QN345"],"award-info":[{"award-number":["322QN345"]}]},{"name":"Hainan Provincial Natural Science Foundation of China","award":["520QN295"],"award-info":[{"award-number":["520QN295"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>In the field of SAR-to-optical image synthesis, current methods based on conditional generative adversarial networks (CGANs) have satisfying performance under simple scenarios, but the performance drops severely under complicated scenarios. Considering that SAR images can form a robust time series due to SAR\u2019s all-weather imaging ability, we take advantage of this and extract a temporal correlation from bi-temporal SAR images to guide the translation. To achieve this, we introduce a co-attention mechanism into the CGAN that learns the correlation between optically-available and optically-absent time points, selectively enhances the features of the former time point, and eventually guides the model to a better optical image synthesis on the latter time point. Additionally, we adopt a strategy to balance the weight of optical and SAR features to extract better features from the SAR input. With these strategies, the quality of synthesized images is notably improved in complicated scenarios. The synthesized images can increase the spatial and temporal resolution of optical imagery, greatly improving the availability of data for the applications of crop monitoring, change detection, and visual interpretation.<\/jats:p>","DOI":"10.3390\/rs15071863","type":"journal-article","created":{"date-parts":[[2023,3,31]],"date-time":"2023-03-31T01:37:02Z","timestamp":1680226622000},"page":"1863","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Temporal Co-Attention Guided Conditional Generative Adversarial Network for Optical Image Synthesis"],"prefix":"10.3390","volume":"15","author":[{"given":"Yongchun","family":"Weng","sequence":"first","affiliation":[{"name":"Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China"},{"name":"University of Chinese Academy of Sciences, Beijing 100049, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yong","family":"Ma","sequence":"additional","affiliation":[{"name":"Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fu","family":"Chen","sequence":"additional","affiliation":[{"name":"Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Erping","family":"Shang","sequence":"additional","affiliation":[{"name":"Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wutao","family":"Yao","sequence":"additional","affiliation":[{"name":"Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shuyan","family":"Zhang","sequence":"additional","affiliation":[{"name":"Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jin","family":"Yang","sequence":"additional","affiliation":[{"name":"Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jianbo","family":"Liu","sequence":"additional","affiliation":[{"name":"Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,3,31]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Li, Z., Shen, H., Cheng, Q., Li, W., and Zhang, L. (2019). Thick cloud removal in high-resolution satellite images using stepwise radiometric adjustment and residual correction. Remote Sens., 11.","DOI":"10.3390\/rs11161925"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1016\/j.isprsjprs.2021.02.018","article-title":"Sentinel sar-optical fusion for crop type mapping using deep learning and google earth engine","volume":"175","author":"Adrian","year":"2021","journal-title":"Isprs J. Photogramm. Remote Sens."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1109\/LGRS.2020.2965558","article-title":"Classification of large-scale high-resolution sar images with deep transfer learning","volume":"18","author":"Huang","year":"2021","journal-title":"Ieee Geosci. Remote Sens. Lett."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TGRS.2023.3335484","article-title":"Few-shot class-incremental SAR target recognition based on hierarchical embedding and incremental evolutionary network","volume":"61","author":"Wang","year":"2023","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"7153","DOI":"10.1109\/JSTARS.2022.3199091","article-title":"A target sar image expansion method based on conditional wasserstein deep convolutional GAN for automatic target recognition","volume":"15","author":"Qin","year":"2022","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1016\/j.isprsjprs.2021.07.007","article-title":"A deep translation (gan) based change detection network for optical and sar remote sensing images","volume":"179","author":"Li","year":"2021","journal-title":"Isprs J. Photogramm. Remote Sens."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1917","DOI":"10.1109\/TGRS.2020.3000296","article-title":"Building change detection in VHR SAR images via unsupervised deep transcoding","volume":"59","author":"Saha","year":"2021","journal-title":"IEEE Transactions on Geoscience and Remote Sensing"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Fuentes Reyes, M., Auer, S., Merkle, N., Henry, C., and Schmitt, M. (2019). Sar-to-optical image translation based on conditional generative adversarial networks\u2014Optimization, opportunities and limits. Remote. Sens., 11.","DOI":"10.3390\/rs11172067"},{"key":"ref_9","first-page":"1","article-title":"A comparative analysis of ganbased methods for sar-to-optical image translation","volume":"19","author":"Zhao","year":"2022","journal-title":"EEE Geosci. Remote. Sens. Lett."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Isola, P., Zhu, J.-Y., Zhou, T., and Efros, A.A. (2017, January 21\u201326). Image-to-image translation with conditional adversarial networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.632"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"5","DOI":"10.5194\/isprs-annals-IV-1-5-2018","article-title":"SAR to optical image synthesis for cloud removal with generative adversarial networks","volume":"IV-1","author":"Bermudez","year":"2018","journal-title":"ISPRS Ann. Photogramm. Remote. Sens. Spat. Inf. Sci."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Grohnfeldt, C., Schmitt, M., and Zhu, X. (2018, January 22\u201327). A conditional generative adversarial network to fuse SAR and multispectral optical data for cloud removal from sentinel-2 images. Proceedings of the IGARSS 2018-2018 IEEE International Geoscience and Remote Sensing Symposium, Valencia, Spain.","DOI":"10.1109\/IGARSS.2018.8519215"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ebel, P., Schmitt, M., and Zhu, X.X. (October, January 26). Cloud removal in unpaired Sentinel-2 imagery using cycle-consistent GAN and SAR-optical data fusion. Proceedings of the IGARSS 2020-2020 IEEE International Geoscience and Remote Sensing Symposium, Waikoloa, HI, USA.","DOI":"10.1109\/IGARSS39084.2020.9324060"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Gao, J., Yuan, Q., Li, J., Zhang, H., and Su, X. (2020). Cloud removal with fusion of high resolution optical and sar images using generative adversarial networks. Remote. Sens., 12.","DOI":"10.3390\/rs12010191"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zuo, Z., and Li, Y. (2021, January 11\u201316). A SAR-to-optical image translation method based on, PIX2PIX. Proceedings of the 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, Brussels, Belgium.","DOI":"10.1109\/IGARSS47720.2021.9555111"},{"key":"ref_16","unstructured":"Lei, W. (2020). Polarimetric SAR Image Information Representation and Classification Based on Deep Learning. [Ph.D. Thesis, Wuhan University]."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TGRS.2021.3068532","article-title":"Synthesizing optical and SAR imagery from land cover maps and auxiliary raster data","volume":"60","author":"Baier","year":"2021","journal-title":"IEEE Trans. Geosci. Remote. Sens."},{"key":"ref_18","first-page":"1","article-title":"FG-GAN: A fine-grained generative adversarial network for unsupervised SAR-to-optical image translation","volume":"60","author":"Yang","year":"2022","journal-title":"IEEE Trans. Geosci. Remote. Sens."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"2947","DOI":"10.1109\/TPAMI.2018.2872043","article-title":"On the effectiveness of least squares generative adversarial networks","volume":"41","author":"Mao","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_20","unstructured":"Arjovsky, M., Chintala, S., and Bottou, L. (2017, January 6\u201311). Wasserstein generative adversarial networks. Proceedings of the International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"4217","DOI":"10.1109\/TPAMI.2020.2970919","article-title":"A style-based generator architecture for generative adversarial networks","volume":"43","author":"Karras","year":"2020","journal-title":"EEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Richardson, E., Alaluf, Y., Patashnik, O., Nitzan, Y., Azar, Y., Shapiro, S., and Cohen-Or, D. (2021, January 20\u201325). Encoding in style: A stylegan encoder for image-to-image translation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00232"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Wang, T.-C., Liu, M.-Y., Zhu, J.-Y., Tao, A., Kautz, J., and Catanzaro, B. (2018, January 18\u201323). High-resolution image synthesis and semantic manipulation with conditional GANs. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00917"},{"key":"ref_24","unstructured":"Zhang, H., Goodfellow, I., Metaxas, D., and Odena, A. (2019, January 6\u201315). Self-attention generative adversarial networks. Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"3487","DOI":"10.1109\/TIP.2021.3061286","article-title":"Complementary, heterogeneous and adversarial networks for image-to-image translation","volume":"30","author":"Gao","year":"2021","journal-title":"IEEE Trans. Image Process."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Tang, H., Xu, D., Sebe, N., Wang, Y., Corso, J.J., and Yan, Y. (2019, January 15\u201320). Multi-channel attention selection GAN with cascaded semantic guidance for cross-view image translation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00252"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"141","DOI":"10.5194\/isprs-annals-IV-1-141-2018","article-title":"The sen1-2 dataset for deep learning in SAR-optical data fusion","volume":"IV-1","author":"Schmitt","year":"2018","journal-title":"ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_28","unstructured":"Uziel, R., Ronen, M., and Freifeld, O. (November, January 27). Bayesian adaptive superpixel segmentation. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Republic of Korea."},{"key":"ref_29","first-page":"1","article-title":"Atrous cGAN for SAR to optical image translation","volume":"19","author":"Castro","year":"2020","journal-title":"IEEE Geosci. Remote. Sens. Lett."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1947","DOI":"10.1109\/TPAMI.2018.2856256","article-title":"StackGAN++: Realistic image synthesis with stacked generative adversarial networks","volume":"41","author":"Zhang","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Xie, Y., and Yang, L. (2018, January 18\u201323). Photographic text-to-image synthesis with a hierarchically-nested adversarial network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00649"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Jiang, L., Zhang, C., Huang, M., Liu, C., Shi, J., and Loy, C.C. (2020, January 23\u201328). Tsit: A simple and versatile framework for image-to-image translation. Proceedings of the European Conference on Computer Vision, Glasgow, UK.","DOI":"10.1007\/978-3-030-58580-8_13"},{"key":"ref_33","first-page":"3518","article-title":"ImageBART: Bidirectional context with multinomial diffusion for autoregressive image synthesis","volume":"34","author":"Esser","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_34","first-page":"1","article-title":"Cascaded diffusion models for high fidelity image generation","volume":"23","author":"Ho","year":"2022","journal-title":"J. Mach. Learn. Res."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"108373","DOI":"10.1016\/j.patcog.2021.108373","article-title":"A coarse-to-fine approach for dynamic-to-static image translation","volume":"123","author":"Wang","year":"2022","journal-title":"Pattern Recognit."},{"key":"ref_36","unstructured":"Agustsson, E., Tschannen, M., Mentzer, F., Timofte, R., and Gool, L.V. (November, January 27). Generative adversarial networks for extreme learned image compression. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Republic of Korea."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1219","DOI":"10.1109\/TIP.2020.3043093","article-title":"Learning spatial attention for face super-resolution","volume":"30","author":"Chen","year":"2020","journal-title":"IEEE Trans. Image Process."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Leibe, B., Matas, J., Sebe, N., and Welling, M. (2016). Computer Vision\u2014ECCV 2016, Springer.","DOI":"10.1007\/978-3-319-46454-1"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1220","DOI":"10.1109\/LGRS.2019.2894734","article-title":"Synthesis of multispectral optical images from SAR\/optical multitemporal data using conditional generative adversarial networks","volume":"16","author":"Bermudez","year":"2019","journal-title":"IEEE Geosci. Remote. Sens. Lett."},{"key":"ref_40","unstructured":"Louis, J. (2022, September 21). To Be Updated Sentinel 2 MSI\u2014Level 2A Product Definition. Issue 4.4. Available online: https:\/\/sentinel.esa.int\/documents\/247904\/1848117\/Sentinel-2-Level-2A-Product-Definition-Document.pdf."},{"key":"ref_41","unstructured":"Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., and Antiga, L. (2019). PyTorch: An Imperative Style, High-Performance Deep Learning Library. arXiv."},{"key":"ref_42","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_43","first-page":"6626","article-title":"GANs trained by a two time-scale update rule converge to a local nash equilibrium","volume":"30","author":"Heusel","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_44","first-page":"2234","article-title":"Improved techniques for training GANs","volume":"29","author":"Salimans","year":"2016","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2016, January 27\u201330). Rethinking the inception architecture for computer vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.308"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. (2009, January 20\u201325). Imagenet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_47","unstructured":"Yuanji, W., Jianhua, L., Yi, L., Yao, F., and Qinzhong, J. (2003, January 14\u201317). Image quality evaluation based on image weighted separating block peak signal to noise ratio. Proceedings of the 2003 International Conference on Neural Networks and Signal Processing, Nanjing, China."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1109\/TIP.2003.819861","article-title":"Image quality assessment: From error visibility to structural similarity","volume":"13","author":"Wang","year":"2004","journal-title":"IEEE Trans. Image Process."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"8","DOI":"10.4236\/jcc.2019.73002","article-title":"Image quality assessment through FSIM, SSIM, MSE and PSNR\u2014A comparative study","volume":"7","author":"Sara","year":"2019","journal-title":"J. Comput. Commun."},{"key":"ref_50","unstructured":"Kynk\u00e4\u00e4nniemi, T., Karras, T., Laine, S., Lehtinen, J., and Aila, T. (2019). Improved precision and recall metric for assessing generative models. arXiv."},{"key":"ref_51","unstructured":"Sajjadi, M.S.M., Bachem, O., Lucic, M., Bousquet, O., and Gelly, S. (2018). Assessing Generative Models via Precision and Recall. arXiv."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Esser, P., Rombach, R., and Ommer, B. (2021, January 20\u201325). Taming transformers for high-resolution image synthesis. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.01268"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/7\/1863\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:07:48Z","timestamp":1760123268000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/7\/1863"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,31]]},"references-count":52,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2023,4]]}},"alternative-id":["rs15071863"],"URL":"https:\/\/doi.org\/10.3390\/rs15071863","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,31]]}}}