{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T23:28:44Z","timestamp":1776382124407,"version":"3.51.2"},"reference-count":37,"publisher":"MDPI AG","issue":"17","license":[{"start":{"date-parts":[[2019,9,3]],"date-time":"2019-09-03T00:00:00Z","timestamp":1567468800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Due to its all time capability, synthetic aperture radar (SAR) remote sensing plays an important role in Earth observation. The ability to interpret the data is limited, even for experts, as the human eye is not familiar to the impact of distance-dependent imaging, signal intensities detected in the radar spectrum as well as image characteristics related to speckle or steps of post-processing. This paper is concerned with machine learning for SAR-to-optical image-to-image translation in order to support the interpretation and analysis of original data. A conditional adversarial network is adopted and optimized in order to generate alternative SAR image representations based on the combination of SAR images (starting point) and optical images (reference) for training. Following this strategy, the focus is set on the value of empirical knowledge for initialization, the impact of results on follow-up applications, and the discussion of opportunities\/drawbacks related to this application of deep learning. Case study results are shown for high resolution (SAR: TerraSAR-X, optical: ALOS PRISM) and low resolution (Sentinel-1 and -2) data. The properties of the alternative image representation are evaluated based on feedback from experts in SAR remote sensing and the impact on road extraction as an example for follow-up applications. The results provide the basis to explain fundamental limitations affecting the SAR-to-optical image translation idea but also indicate benefits from alternative SAR image representations.<\/jats:p>","DOI":"10.3390\/rs11172067","type":"journal-article","created":{"date-parts":[[2019,9,4]],"date-time":"2019-09-04T08:28:13Z","timestamp":1567585693000},"page":"2067","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":135,"title":["SAR-to-Optical Image Translation Based on Conditional Generative Adversarial Networks\u2014Optimization, Opportunities and Limits"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6593-5152","authenticated-orcid":false,"given":"Mario","family":"Fuentes Reyes","sequence":"first","affiliation":[{"name":"Remote Sensing Technology Institute, German Aerospace Center (DLR), 82234 Wessling, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9310-2337","authenticated-orcid":false,"given":"Stefan","family":"Auer","sequence":"additional","affiliation":[{"name":"Remote Sensing Technology Institute, German Aerospace Center (DLR), 82234 Wessling, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4177-1066","authenticated-orcid":false,"given":"Nina","family":"Merkle","sequence":"additional","affiliation":[{"name":"Remote Sensing Technology Institute, German Aerospace Center (DLR), 82234 Wessling, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4330-3058","authenticated-orcid":false,"given":"Corentin","family":"Henry","sequence":"additional","affiliation":[{"name":"Remote Sensing Technology Institute, German Aerospace Center (DLR), 82234 Wessling, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0575-2362","authenticated-orcid":false,"given":"Michael","family":"Schmitt","sequence":"additional","affiliation":[{"name":"Signal Processing in Earth Observation, Technical University of Munich (TUM), 80333 Munich, Germany"}]}],"member":"1968","published-online":{"date-parts":[[2019,9,3]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1445","DOI":"10.1109\/TGRS.2009.2029339","article-title":"Ray-Tracing Simulation Techniques for Understanding High-Resolution SAR Images","volume":"48","author":"Auer","year":"2010","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1109\/MGRS.2013.2277512","article-title":"A tutorial on speckle reduction in synthetic aperture radar images","volume":"1","author":"Argenti","year":"2013","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Zhang, R., Isola, P., and Efros, A.A. (2016, January 11\u201314). Colorful Image Colorization. Proceedings of the Computer Vision\u2014ECCV 2016, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46487-9_40"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"4779","DOI":"10.1109\/JSTARS.2017.2723082","article-title":"Simulation-Based Interpretation and Alignment of High-Resolution Optical and SAR Images","volume":"10","author":"Auer","year":"2017","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1811","DOI":"10.1109\/JSTARS.2018.2803212","article-title":"Exploring the potential of conditional adversarial networks for optical and SAR image matching","volume":"11","author":"Merkle","year":"2018","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"5","DOI":"10.5194\/isprs-annals-IV-1-5-2018","article-title":"SAR to optical image synthesis for cloud removal with generative adversarial networks","volume":"IV-1","author":"Bermudez","year":"2018","journal-title":"ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Merkle, N., Fischer, P., Auer, S., and M\u00fcller, R. (2017, January 23\u201328). On the Possibility of Conditional Adversarial Networks for Multi-Sensor Image Matching. Proceedings of the 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Fort Worth, TX, USA.","DOI":"10.1109\/IGARSS.2017.8127535"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Zhu, J., Park, T., Isola, P., and Efros, A.A. (2017, January 22\u201329). Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.244"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1109\/MGRS.2017.2762307","article-title":"Deep learning in remote sensing: A comprehensive review and list of resources","volume":"5","author":"Zhu","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Chierchia, G., Cozzolino, D., Poggi, G., and Verdoliva, L. (2017, January 23\u201328). SAR Image Despeckling Through Convolutional Neural Networks. Proceedings of the 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Fort Worth, TX, USA.","DOI":"10.1109\/IGARSS.2017.8128234"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2080","DOI":"10.1109\/TIP.2007.901238","article-title":"Image denoising by sparse 3-D transform-domain collaborative filtering","volume":"16","author":"Dabov","year":"2007","journal-title":"IEEE Trans. Image Process."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2021","DOI":"10.1109\/TGRS.2014.2352555","article-title":"NL-SAR: A unified nonlocal framework for resolution-preserving (Pol)(In)SAR denoising","volume":"53","author":"Deledalle","year":"2015","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Ao, D., Dumitru, C.O., Schwarz, G., and Datcu, M. (2018). Dialectical GAN for SAR image translation: From Sentinel-1 to TerraSAR-X. Remote Sens., 10.","DOI":"10.20944\/preprints201807.0340.v1"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Isola, P., Zhu, J., Zhou, T., and Efros, A.A. (2017, January 21\u201326). Image-to-Image Translation with Conditional Adversarial Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.632"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Wang, P., and Patel, V.M. (2018, January 23\u201327). Generating High Quality Visible Images from SAR Images Using CNNs. Proceedings of the 2018 IEEE Radar Conference (RadarConf18), Oklahoma City, OK, USA.","DOI":"10.1109\/RADAR.2018.8378622"},{"key":"ref_17","unstructured":"Ley, A., D\u2019Hondt, O., Valade, S., H\u00e4nsch, R., and Hellwich, O. (2018, January 4\u20137). Exploiting GAN-Based SAR to Optical Image Transcoding for Improved Classification via Deep Learning. Proceedings of the EUSAR 2018, Aachen, Germany."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"He, W., and Yokoya, N. (2018). Multi-Temporal Sentinel-1 and -2 Data Fusion for Optical Image Simulation. ISPRS Int. J. Geo-Inf., 7.","DOI":"10.3390\/ijgi7100389"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"784","DOI":"10.1109\/LGRS.2018.2799232","article-title":"Identifying corresponding patches in SAR and optical images with a pseudo-siamese CNN","volume":"15","author":"Hughes","year":"2018","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Hughes, L., Schmitt, M., and Zhu, X. (2018). Mining hard negative samples for SAR-optical image matching using generative adversarial networks. Remote Sens., 10.","DOI":"10.3390\/rs10101552"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"141","DOI":"10.5194\/isprs-annals-IV-1-141-2018","article-title":"The SEN1-2 dataset for deep learning in SAR-optical data fusion","volume":"IV-1","author":"Schmitt","year":"2018","journal-title":"ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Grohnfeldt, C., Schmitt, M., and Zhu, X. (2018, January 22\u201327). A Conditional Generative Adversarial Network to Fuse SAR and Multispectral Optical Data for Cloud Removal from Sentinel-2 Images. Proceedings of the IGARSS 2018\u20142018 IEEE International Geoscience and Remote Sensing Symposium, Valencia, Spain.","DOI":"10.1109\/IGARSS.2018.8519215"},{"key":"ref_23","unstructured":"Mirza, M., and Osindero, S. (arXiv, 2014). Conditional generative adversarial nets, arXiv."},{"key":"ref_24","unstructured":"Ioffe, S., and Szegedy, C. (2015, January 6\u201311). Batch Normalization: Accelerating Deep Network Training By Reducing Internal Covariate Shift. Proceedings of the 32nd International Conference on International Conference on Machine Learning\u2014Volume 37, Lille, France."},{"key":"ref_25","unstructured":"Ulyanov, D., Vedaldi, A., and Lempitsky, V. (arXiv, 2016). Instance normalization: The missing ingredient for fast stylization, arXiv."},{"key":"ref_26","unstructured":"Kingma, D., and Ba, J. (arXiv, 2014). Adam: A method for stochastic optimization, arXiv."},{"key":"ref_27","unstructured":"Glorot, X., and Bengio, Y. (2010, January 13\u201315). Understanding the Difficulty of Training Deep Feedforward Neural Networks. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, Sardinia, Italy."},{"key":"ref_28","unstructured":"Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., and Weinberger, K.Q. (2014). Generative Adversarial Nets. Advances in Neural Information Processing Systems 27, Curran Associates, Inc."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Leibe, B., Matas, J., Sebe, N., and Welling, M. (2016). Perceptual Losses for Real-Time Style Transfer and Super-Resolution, Springer International Publishing. Computer Vision\u2014ECCV 2016.","DOI":"10.1007\/978-3-319-46454-1"},{"key":"ref_30","unstructured":"Lucic, M., Kurach, K., Michalski, M., Bousquet, O., and Gelly, S. Are GANs created equal? A large-scale study. Proceedings of the 32Nd International Conference on Neural Information Processing Systems."},{"key":"ref_31","first-page":"615","article-title":"The TerraSAR-X Satellite","volume":"48","author":"Pitz","year":"2010","journal-title":"TGRS"},{"key":"ref_32","unstructured":"Schneider, M., M\u00fcller, R., Krauss, T., Reinartz, P., H\u00f6rsch, B., and Schmuck, S. (2010, January 5\u20137). Urban Atlas\u2014DLR Processing Chain for Orthorectification of Prism and AVNIR-2 Images and TerraSAR-X as Possible GCP Source. Proceedings of the 3rd ALOS PI Symposium, Buffalo, NY, USA."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Lee, H., Tseng, H., Huang, J., Singh, M., and Yang, M. (2018, January 8\u201314). Diverse Image-to-Image Translation via Disentangled Representations. Proceedings of the European Conference on Computer Vision, Munich, Germany.","DOI":"10.1007\/978-3-030-01246-5_3"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Baier, G., and Zhu, X.X. (2016, January 10\u201315). GPU-based nonlocal filtering for large scale SAR processing. Proceedings of the 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Beijing, China.","DOI":"10.1109\/IGARSS.2016.7730984"},{"key":"ref_35","unstructured":"Chu, C., Zhmoginov, A., and Sandler, M. (arXiv, 2017). CycleGAN, a master of steganography, arXiv."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H. (2018, January 8\u201314). Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01234-2_49"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1867","DOI":"10.1109\/LGRS.2018.2864342","article-title":"Road segmentation in SAR satellite images with deep fully convolutional neural networks","volume":"15","author":"Henry","year":"2018","journal-title":"IEEE Geosci. Remote Sens. Lett."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/11\/17\/2067\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:16:28Z","timestamp":1760188588000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/11\/17\/2067"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,9,3]]},"references-count":37,"journal-issue":{"issue":"17","published-online":{"date-parts":[[2019,9]]}},"alternative-id":["rs11172067"],"URL":"https:\/\/doi.org\/10.3390\/rs11172067","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,9,3]]}}}