{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,14]],"date-time":"2026-04-14T15:49:31Z","timestamp":1776181771136,"version":"3.50.1"},"reference-count":56,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2017,6,10]],"date-time":"2017-06-10T00:00:00Z","timestamp":1497052800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Improving the geo-localization of optical satellite images is an important pre-processing step for many remote sensing tasks like monitoring by image time series or scene analysis after sudden events. These tasks require geo-referenced and precisely co-registered multi-sensor data. Images captured by the high resolution synthetic aperture radar (SAR) satellite TerraSAR-X exhibit an absolute geo-location accuracy within a few decimeters. These images represent therefore a reliable source to improve the geo-location accuracy of optical images, which is in the order of tens of meters. In this paper, a deep learning-based approach for the geo-localization accuracy improvement of optical satellite images through SAR reference data is investigated. Image registration between SAR and optical images requires few, but accurate and reliable matching points. These are derived from a Siamese neural network. The network is trained using TerraSAR-X and PRISM image pairs covering greater urban areas spread over Europe, in order to learn the two-dimensional spatial shifts between optical and SAR image patches. Results confirm that accurate and reliable matching points can be generated with higher matching accuracy and precision with respect to state-of-the-art approaches.<\/jats:p>","DOI":"10.3390\/rs9060586","type":"journal-article","created":{"date-parts":[[2017,6,12]],"date-time":"2017-06-12T10:27:59Z","timestamp":1497263279000},"page":"586","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":113,"title":["Exploiting Deep Matching and SAR Data for the Geo-Localization Accuracy Improvement of Optical Satellite Images"],"prefix":"10.3390","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4177-1066","authenticated-orcid":false,"given":"Nina","family":"Merkle","sequence":"first","affiliation":[{"name":"German Aerospace Center (DLR), Remote Sensing Technology Institute, 82234 Wessling, Germany"}]},{"given":"Wenjie","family":"Luo","sequence":"additional","affiliation":[{"name":"Department of Computer Science University of Toronto, Toronto, ON M5S 3G, Canada"}]},{"given":"Stefan","family":"Auer","sequence":"additional","affiliation":[{"name":"German Aerospace Center (DLR), Remote Sensing Technology Institute, 82234 Wessling, Germany"}]},{"given":"Rupert","family":"M\u00fcller","sequence":"additional","affiliation":[{"name":"German Aerospace Center (DLR), Remote Sensing Technology Institute, 82234 Wessling, Germany"}]},{"given":"Raquel","family":"Urtasun","sequence":"additional","affiliation":[{"name":"Department of Computer Science University of Toronto, Toronto, ON M5S 3G, Canada"}]}],"member":"1968","published-online":{"date-parts":[[2017,6,10]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"606","DOI":"10.1109\/TGRS.2009.2031062","article-title":"The TerraSAR-X Mission and System Design","volume":"48","author":"Werninghaus","year":"2010","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"661","DOI":"10.1109\/TGRS.2010.2060264","article-title":"Imaging Geodesy- Toward Centimeter-Level Ranging Accuracy with TerraSAR-X","volume":"49","author":"Eineder","year":"2011","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_3","unstructured":"Cumming, I., and Wong, F. (2005). Digital Processing of Synthetic Aperture Radar Data: Algorithms and Implementation, Artech House. Number Bd. 1 in Artech House Remote Sensing Library."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1762","DOI":"10.1109\/LGRS.2014.2308353","article-title":"Linear Signatures in Urban SAR Images\u2014Partly Misinterpreted?","volume":"11","author":"Auer","year":"2014","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1016\/j.isprsjprs.2010.10.003","article-title":"Orthorectification of VHR Optical Satellite Data Exploiting the Geometric Accuracy of TerraSAR-X Data","volume":"66","author":"Reinartz","year":"2011","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Merkle, N., M\u00fcller, R., and Reinartz, P. (2015, January 25\u201327). Registration of Optical and SAR Satellite Images based on Geometric Feature Templates. Proceedings of the International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, International Conference on Sensors & Models in Remote Sensing & Photogrammetry, Kish Island, Iran.","DOI":"10.5194\/isprsarchives-XL-1-W5-447-2015"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Perko, R., Raggam, H., Gutjahr, K., and Schardt, M. (2011, January 24\u201329). Using Worldwide Available TerraSAR-X Data to Calibrate the Geo-location Accuracy of Optical Sensors. Proceedings of the 2011 IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2011, Vancouver, BC, Canada.","DOI":"10.1109\/IGARSS.2011.6049732"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Shi, W., Su, F., Wang, R., and Fan, J. (2012, January 22\u201327). A Visual Circle Based Image Registration Algorithm for Optical and SAR Imagery. Proceedings of the 2012 IEEE International Geoscience and Remote Sensing Symposium, Munich, Germany.","DOI":"10.1109\/IGARSS.2012.6351089"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Siddique, M.A., Sarfraz, M.S., Bornemann, D., and Hellwich, O. (2012, January 22\u201327). Automatic Registration of SAR and Optical Images Based on Mutual Information Assisted Monte Carlo. Proceedings of the 2012 IEEE International Geoscience and Remote Sensing Symposium, Munich, Germany.","DOI":"10.1109\/IGARSS.2012.6351159"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"939","DOI":"10.1109\/TGRS.2009.2034842","article-title":"Mutual-Information-Based Registration of TerraSAR-X and Ikonos Imagery in Urban Areas","volume":"48","author":"Suri","year":"2010","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"4050","DOI":"10.1109\/TGRS.2012.2187456","article-title":"Robust Automatic Registration of Multimodal Satellite Images Using CCRE with Partial Volume Interpolation","volume":"50","author":"Hasan","year":"2012","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2104","DOI":"10.1109\/TGRS.2004.835294","article-title":"On the Possibility of Automatic Multisensor Image Registration","volume":"42","author":"Inglada","year":"2004","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/1687-6180-2013-3","article-title":"Multi-Modal Image Matching Based on Local Frequency Information","volume":"2013","author":"Liu","year":"2013","journal-title":"EURASIP J. Adv. Signal Process."},{"key":"ref_14","unstructured":"Li, Q., Qu, G., and Li, Z. (2013, January 14\u201316). Matching Between SAR Images and Optical Images Based on HOG Descriptor. Proceedings of the IET International Radar Conference, Xi\u2019an, China."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"9","DOI":"10.5194\/isprs-annals-III-1-9-2016","article-title":"HOPC: A Novel Similarity Metric Based on Geometric Structural Properties for Multi-modal Remote Sensing Image Matching","volume":"III-1","author":"Ye","year":"2016","journal-title":"ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"585","DOI":"10.14358\/PERS.71.5.585","article-title":"A Robust Technique for Precise Registration of Radar and Optical Satellite Images","volume":"71","author":"Hong","year":"2005","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"320","DOI":"10.1109\/83.366480","article-title":"A Contour-Based Approach to Multisensor Image Registration","volume":"4","author":"Li","year":"1995","journal-title":"IEEE Trans. Image Process."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"569","DOI":"10.1080\/01431160701294687","article-title":"Multisource Data Registration Based on NURBS Description of Contours","volume":"29","author":"Pan","year":"2008","journal-title":"Int. J. Remote Sens."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1016\/S0924-2716(01)00031-4","article-title":"An Improved Model for Automatic Feature-Based Registration of SAR and SPOT Images","volume":"56","author":"Dare","year":"2001","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1016\/j.isprsjprs.2015.01.015","article-title":"A Generic Framework for Image Rectification Using Multiple Types of Feature","volume":"102","author":"Long","year":"2015","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"657","DOI":"10.1109\/LGRS.2012.2216500","article-title":"Registration of Optical and SAR Satellite Images by Exploring the Spatial Relationship of the Improved SIFT","volume":"10","author":"Fan","year":"2013","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"3997","DOI":"10.1080\/01431161.2015.1070321","article-title":"An Automatic Optical and SAR Image Registration Method with Iterative Level Set Segmentation and SIFT","volume":"36","author":"Xu","year":"2015","journal-title":"Int. J. Remote Sens."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"6058","DOI":"10.1109\/TGRS.2015.2431498","article-title":"Automatic Optical-to-SAR Image Registration by Iterative Line Extraction and Voronoi Integrated Spectral Point Matching","volume":"53","author":"Sui","year":"2015","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2277","DOI":"10.1080\/01431161.2015.1030046","article-title":"Automatic and Accurate Registration of VHR Optical and SAR Images Using a Quadtree Structure","volume":"36","author":"Han","year":"2015","journal-title":"Int. J. Remote Sens."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"2094","DOI":"10.1109\/JSTARS.2014.2329330","article-title":"Deep Learning-Based Classification of Hyperspectral Data","volume":"7","author":"Chen","year":"2014","journal-title":"Sel. Top. Appl. Earth Obs. Remote Sens. IEEE J."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Liang, H., and Li, Q. (2016). Hyperspectral Imagery Classification Using Sparse Representations of Convolutional Neural Network Features. Remote Sens., 8.","DOI":"10.3390\/rs8020099"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1279","DOI":"10.1109\/TNNLS.2015.2477537","article-title":"Salient Band Selection for Hyperspectral Image Classification via Manifold Ranking","volume":"27","author":"Wang","year":"2016","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Mnih, V., and Hinton, G.E. (2010, January 5\u201311). Learning to Detect Roads in High-Resolution Aerial Images. Proceedings of the 11th European Conference on Computer Vision, Heraklion, Crete, Greece.","DOI":"10.1007\/978-3-642-15567-3_16"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Matthyus, G., Wang, S., Fidler, S., and Urtasun, R. (2016, January 27\u201330). HD Maps: Fine-grained Road Segmentation by Parsing Ground and Aerial Images. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.393"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"2351","DOI":"10.1109\/LGRS.2015.2478256","article-title":"High-Resolution SAR Image Classification via Deep Convolutional Autoencoders","volume":"12","author":"Geng","year":"2015","journal-title":"Geosci. Remote Sens. Lett. IEEE"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Masi, G., Cozzolino, D., Verdoliva, L., and Scarpa, G. (2016). Pansharpening by Convolutional Neural Networks. Remote Sens., 8.","DOI":"10.3390\/rs8070594"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Luo, W., Schwing, A.G., and Urtasun, R. (2016, January 27\u201330). Efficient Deep Learning for Stereo Matching. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.614"},{"key":"ref_33","first-page":"1","article-title":"Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches","volume":"17","author":"Zbontar","year":"2016","journal-title":"J. Mach. Learn. Res."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Bai, M., Luo, W., Kundu, K., and Urtasun, R. (2016, January 8\u201316). Exploiting Semantic Information and Deep Matching for Optical Flow. Proceedings of the European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46466-4_10"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Weinzaepfel, P., Revaud, J., Harchaoui, Z., and Schmid, C. (2013, January 1\u20138). DeepFlow: Large Displacement OpticalFlow with Deep Matching. Proceedings of the IEEE Intenational Conference on Computer Vision (ICCV), Sydney, Australia.","DOI":"10.1109\/ICCV.2013.175"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Altwaijry, H., Trulls, E., Hays, J., Fua, P., and Belongie, S. (2016, January 27\u201330). Learning to Match Aerial Images with Deep Attentive Architectures. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.385"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Cui, Y., Belongie, S., and Hays, J. (2015, January 7\u201312). Learning Deep Representations for Ground-to-Aerial Geolocalization. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7299135"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Altwaijry, H., Veit, A., and Belongie, S. (2016, January 19\u201322). Learning to Detect and Match Keypoints with Deep Architectures. Proceedings of the British Machine Vision Conference (BMVC), York, UK.","DOI":"10.5244\/C.30.49"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Bromley, J., Guyon, I., LeCun, Y., S\u00e4ckinger, E., and Shah, R. (December1994, January 28). Signature Verification using a \u201cSiamese\u201d Time Delay Neural Network. Proceedings of the Conference on Neural Information Processing Systems (NIPS), Denver, CO, USA.","DOI":"10.1142\/9789812797926_0003"},{"key":"ref_40","first-page":"3371","article-title":"Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion","volume":"11","author":"Vincent","year":"2010","journal-title":"J. Mach. Learn. Res."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Fischer, A., and Igel, C. (2012). An Introduction to Restricted Boltzmann Machines. Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, Proceedings of the 17th Iberoamerican Congress, CIARP 2012, Buenos Aires, Argentina, 3\u20136 September 2012, Springer.","DOI":"10.1007\/978-3-642-33275-3_2"},{"key":"ref_42","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3\u20138). ImageNet Classification with Deep Convolutional Neural Networks. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA."},{"key":"ref_43","unstructured":"Han, X., Leung, T., Jia, Y., Sukthankar, R., and Berg, A.C. (2015, January 7\u201312). MatchNet: Unifying Feature and Metric Learning for Patch-Based Matching. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Zagoruyko, S., and Komodakis, N. (2015, January 7\u201312). Learning to Compare Image Patches via Convolutional Neural Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7299064"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Simo-Serra, E., Trulls, E., Ferraz, L., Kokkinos, I., Fua, P., and Moreno-Noguer, F. (2015, January 11\u201318). Discriminative Learning of Deep Convolutional Feature Point Descriptors. Proceedings of the International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.22"},{"key":"ref_46","unstructured":"Yu, F., and Koltun, V. (2016, January 2\u20134). Multi-Scale Context Aggregation by Dilated Convolutions. Proceedings of the International Conference on Learning Representations (ICCV), San Juan, Puerto Rico."},{"key":"ref_47","unstructured":"Ioffe, S., and Szegedy, C. (2015, January 6\u201311). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. Proceedings of the 32nd International Conference on Machine Learning (ICML-15), Lille, France."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"2661","DOI":"10.1109\/TIP.2009.2029593","article-title":"Iterative Weighted Maximum Likelihood Denoising with Probabilistic Patch-Based Weights","volume":"18","author":"Deledalle","year":"2009","journal-title":"IEEE Trans. Image Process."},{"key":"ref_49","unstructured":"Buades, A., and Coll, B. (2005, January 20\u201326). A Non-Local Algorithm for Image Denoising. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), San Diego, CA, USA."},{"key":"ref_50","unstructured":"Kingma, D., and Ba, J. (2014). Adam: A Method for Stochastic Optimization. arXiv."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"61","DOI":"10.14358\/PERS.78.1.61","article-title":"Automated Georeferencing of Optical Satellite Data with Integrated Sensor Model Improvement","volume":"78","author":"Schneider","year":"2012","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_52","unstructured":"Schneider, M., M\u00fcller, R., Krauss, T., Reinartz, P., H\u00f6rsch, B., and Schmuck, S. (2009, January 9\u201313). Urban Atlas\u2014DLR Processing Chain for Orthorectification of PRISM and AVNIR-2 Images and TerraSAR-X as possible GCP Source. Proceedings of the International Proceedings: 3rd ALOS PI Symposium, Kona, HI, USA."},{"key":"ref_53","unstructured":"Bossard, M., Feranec, J., and Otahel, J. (2000). CORINE Land Cover Technical Guide\u2014Addendum 2000, European Environmental Agency."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2015, January 11\u201318). Delving Deep Into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.123"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Burger, W., and Burge, M.J. (2009). Principles of Digital Image Processing: Core Algorithms, Springer Publishing Company. [1st ed.].","DOI":"10.1007\/978-1-84800-195-4"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Walters-Williams, J., and Li, Y. (2009). Estimation of Mutual Information: A Survey. Rough Sets and Knowledge Technology, Proceedings of the 4th International Conference, RSKT 2009, Gold Coast, Australia, 14\u201316 July 2009, Springer.","DOI":"10.1007\/978-3-642-02962-2_49"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/9\/6\/586\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:38:42Z","timestamp":1760207922000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/9\/6\/586"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,6,10]]},"references-count":56,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2017,6]]}},"alternative-id":["rs9060586"],"URL":"https:\/\/doi.org\/10.3390\/rs9060586","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,6,10]]}}}