{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T15:49:55Z","timestamp":1784130595883,"version":"3.55.0"},"reference-count":38,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2024,1,22]],"date-time":"2024-01-22T00:00:00Z","timestamp":1705881600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["42101458"],"award-info":[{"award-number":["42101458"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["42171456"],"award-info":[{"award-number":["42171456"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["42130112"],"award-info":[{"award-number":["42130112"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["41901285"],"award-info":[{"award-number":["41901285"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["42277478"],"award-info":[{"award-number":["42277478"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Synthetic aperture radar (SAR) and optical images provide highly complementary ground information. The fusion of SAR and optical data can significantly enhance semantic segmentation inference results. However, the fusion methods for multimodal data remains a challenge for current research due to significant disparities in imaging mechanisms from diverse sources. Our goal was to bridge the significant gaps between optical and SAR images by developing a dual-input model that utilizes image-level fusion. To improve most existing state-of-the-art image fusion methods, which often assign equal weights to multiple modalities, we employed the principal component analysis (PCA) transform approach. Subsequently, we performed feature-level fusion on shallow feature maps, which retain rich geometric information. We also incorporated a channel attention module to highlight channels rich in features and suppress irrelevant information. This step is crucial due to the substantial similarity between SAR and optical images in shallow layers such as geometric features. In summary, we propose a generic multimodal fusion strategy that can be attached to most encoding\u2013decoding structures for feature classification tasks, designed with two inputs. One input is the optical image, and the other is the three-band fusion data obtained by combining the PCA component of the optical image with the SAR. Our feature-level fusion method effectively integrates multimodal data. The efficiency of our approach was validated using various public datasets, and the results showed significant improvements when applied to several land cover classification models.<\/jats:p>","DOI":"10.3390\/rs16020431","type":"journal-article","created":{"date-parts":[[2024,1,22]],"date-time":"2024-01-22T11:36:41Z","timestamp":1705923401000},"page":"431","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":28,"title":["Learning SAR-Optical Cross Modal Features for Land Cover Classification"],"prefix":"10.3390","volume":"16","author":[{"given":"Yujun","family":"Quan","sequence":"first","affiliation":[{"name":"School of Surveying and Mapping, PLA Strategic Support Force Information Engineering University, Zhengzhou 450001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rongrong","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Geoscience and Technology, Zhengzhou University, Zhengzhou 450001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jian","family":"Li","sequence":"additional","affiliation":[{"name":"School of Geoscience and Technology, Zhengzhou University, Zhengzhou 450001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Song","family":"Ji","sequence":"additional","affiliation":[{"name":"School of Surveying and Mapping, PLA Strategic Support Force Information Engineering University, Zhengzhou 450001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hengliang","family":"Guo","sequence":"additional","affiliation":[{"name":"School of Geoscience and Technology, Zhengzhou University, Zhengzhou 450001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3332-9668","authenticated-orcid":false,"given":"Anzhu","family":"Yu","sequence":"additional","affiliation":[{"name":"School of Surveying and Mapping, PLA Strategic Support Force Information Engineering University, Zhengzhou 450001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,1,22]]},"reference":[{"key":"ref_1","first-page":"1","article-title":"DML: Differ-Modality Learning for Building Semantic Segmentation","volume":"60","author":"Xia","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Peng, B., Zhang, W., Hu, Y., Chu, Q., and Li, Q. (2022). LRFFNet: Large Receptive Field Feature Fusion Network for Semantic Segmentation of SAR Images in Building Areas. Remote Sens., 14.","DOI":"10.3390\/rs14246291"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2573","DOI":"10.1109\/JSTARS.2023.3250461","article-title":"CroFuseNet: A Semantic Segmentation Network for Urban Impervious Surface Extraction Based on Cross Fusion of Optical and SAR Images","volume":"16","author":"Wu","year":"2023","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1562","DOI":"10.1109\/JSTARS.2022.3144587","article-title":"CFNet: A Cross Fusion Network for Joint Land Cover Classification Using Optical and SAR Images","volume":"15","author":"Kang","year":"2022","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Xu, H., He, M., Rao, Z., and Li, W. (2021, January 19\u201322). Him-Net: A New Neural Network Approach for SAR and Optical Image Template Matching1. Proceedings of the 2021 IEEE International Conference on Image Processing (ICIP), Anchorage, AK, USA.","DOI":"10.1109\/ICIP42928.2021.9506616"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1016\/j.inffus.2020.01.003","article-title":"Pixel level fusion techniques for SAR and optical images: A review","volume":"59","author":"Kulkarni","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1109\/JMASS.2023.3244848","article-title":"Advances and Challenges in Multimodal Remote Sensing Image Registration","volume":"4","author":"Zhu","year":"2023","journal-title":"IEEE J. Miniaturization Air Space Syst."},{"key":"ref_8","first-page":"113","article-title":"Multi-Scale PIIFD for Registration of Multi-Source Remote Sensing Images","volume":"30","author":"Gao","year":"2021","journal-title":"J. Beijing Inst. Technol."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Ye, Y., Zhu, B., Tang, T., Yang, C., Xu, Q., and Zhang, G. (2022). A Robust Multimodal Remote Sensing Image Registration Method and System Using Steerable Filters with First- and Second-order Gradients. arXiv.","DOI":"10.1016\/j.isprsjprs.2022.04.011"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Shakya, A., Biswas, M., and Pal, M. (2023). Fusion and Classification of SAR and Optical Data Using Multi-Image Color Components with Differential Gradients. Remote Sens., 15.","DOI":"10.3390\/rs15010274"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1016\/j.inffus.2005.09.006","article-title":"Pixel- and region-based image fusion with complex wavelets","volume":"8","author":"Lewis","year":"2007","journal-title":"Inf. Fusion"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Gaetano, R., Cozzolino, D., D\u2019Amiano, L., Verdoliva, L., and Poggi, G. (2017, January 23\u201328). Fusion of sar-optical data for land cover monitoring. Proceedings of the 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Fort Worth, TX, USA.","DOI":"10.1109\/IGARSS.2017.8128242"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Zhang, R., Tang, X., You, S., Duan, K., Xiang, H., and Luo, H. (2020). A Novel Feature-Level Fusion Framework Using Optical and SAR Remote Sensing Images for Land Use\/Land Cover (LULC) Classification in Cloudy Mountainous Area. Appl. Sci., 10.","DOI":"10.3390\/app10082928"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1061","DOI":"10.1109\/LGRS.2014.2377722","article-title":"Impacts of Feature Normalization on Optical and SAR Data Fusion for Land Use\/Land Cover Classification","volume":"12","author":"Zhang","year":"2015","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Maggiolo, L., Solarna, D., Moser, G., and Serpico, S.B. (2022, January 17\u201322). Optical-SAR Decision Fusion with Markov Random Fields for High-Resolution Large-Scale Land Cover Mapping. Proceedings of the IGARSS 2022\u20142022 IEEE International Geoscience and Remote Sensing Symposium, Kuala Lumpur, Malaysia.","DOI":"10.1109\/IGARSS46834.2022.9884751"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1011","DOI":"10.1109\/JSTARS.2020.2975252","article-title":"Multimodal Bilinear Fusion Network With Second-Order Attention-Based Channel Selection for Land Cover Classification","volume":"13","author":"Li","year":"2020","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Zhang, D., Gade, M., and Zhang, J. (2022, January 17\u201322). SOF-UNet: SAR and Optical Fusion Unet for Land Cover Classification. Proceedings of the IGARSS 2022\u20142022 IEEE International Geoscience and Remote Sensing Symposium, Kuala Lumpur, Malaysia.","DOI":"10.1109\/IGARSS46834.2022.9884504"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"3829","DOI":"10.1109\/TGRS.2020.3015389","article-title":"Collaborative Attention-Based Heterogeneous Gated Fusion Network for Land Cover Classification","volume":"59","author":"Li","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1016\/j.isprsjprs.2022.02.013","article-title":"DKDFN: Domain Knowledge-Guided deep collaborative fusion network for multimodal unitemporal remote sensing land cover classification","volume":"186","author":"Li","year":"2022","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"060002","DOI":"10.3788\/LOP57.060002","article-title":"Reiview on key technologies of target exploration in underwater optical images","volume":"57","year":"2020","journal-title":"Laster Optoelectron. Prog."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"10835","DOI":"10.1109\/JSTARS.2021.3121405","article-title":"Multireceiver SAS Imagery Based on Monostatic Conversion","volume":"14","author":"Zhang","year":"2021","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Yang, P. (2023). An imaging algorithm for high-resolution imaging sonar system. Multimed. Tools Appl.","DOI":"10.1007\/s11042-023-16757-0"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Luo, D., Li, L., Mu, F., and Gao, L. (2014, January 1\u201314). Fusion of high spatial resolution optical and polarimetric SAR images for urban land cover classification. Proceedings of the 2014 Third International Workshop on Earth Observation and Remote Sensing Applications (EORSA), Changsha, China.","DOI":"10.1109\/EORSA.2014.6927913"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. arXiv.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Zhou, Z., Siddiquee, M.M.R., Tajbakhsh, N., and Liang, J. (2018). UNet++: A Nested U-Net Architecture for Medical Image Segmentation. arXiv.","DOI":"10.1007\/978-3-030-00889-5_1"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"6559","DOI":"10.1109\/JSTARS.2022.3195808","article-title":"Building Extraction From Remote Sensing Images With DoG as Prior Constraint","volume":"15","author":"Quan","year":"2022","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (, January 11\u201317). Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. Proceedings of the 2021 IEEE\/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.00986"},{"key":"ref_28","unstructured":"Simonyan, K., and Zisserman, A. (2015). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_30","first-page":"630","article-title":"Development of a PCA-based land use\/land cover classification utilizing Sentinel-2 time series","volume":"11","author":"Makar","year":"2022","journal-title":"Middle East J. Agric. Res."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Lai, S.H., Lepetit, V., Nishino, K., and Sato, Y. (2017). Computer Vision\u2014ACCV 2016: 13th Asian Conference on Computer Vision, Taipei, Taiwan, 20\u201324 November 2016, Revised Selected Papers, Part I 13, Springer International Publishing.","DOI":"10.1007\/978-3-319-54193-8"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"5537915","DOI":"10.1109\/TGRS.2022.3206208","article-title":"Semi-Active Convolutional Neural Networks for Hyperspectral Image Classification","volume":"60","author":"Yao","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"5530612","DOI":"10.1109\/TGRS.2022.3179737","article-title":"Multibranch Feature Fusion Network With Self- and Cross-Guided Attention for Hyperspectral and LiDAR Classification","volume":"60","author":"Dong","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"5108316","DOI":"10.1109\/TGRS.2023.3306042","article-title":"A Novel Coarse-to-Fine Deep Learning Registration Framework for Multimodal Remote Sensing Images","volume":"61","author":"Quan","year":"2023","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_35","unstructured":"M\u00fcller, R., Kornblith, S., and Hinton, G.E. (2019). When Does Label Smoothing Help?. arXiv."},{"key":"ref_36","first-page":"102638","article-title":"MCANet: A joint semantic segmentation framework of optical and SAR images for land use classification","volume":"106","author":"Li","year":"2022","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_37","first-page":"102896","article-title":"A dual-stream high resolution network: Deep fusion of GF-2 and GF-3 data for land cover classification","volume":"112","author":"Ren","year":"2022","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_38","unstructured":"Sun, Y., Zhao, Y., Wang, Z., and Fan, Y. (2020, January 22). SOLC. Available online: https:\/\/github.com\/yisun98\/SOLC."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/2\/431\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T13:47:24Z","timestamp":1760104044000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/2\/431"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,22]]},"references-count":38,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2024,1]]}},"alternative-id":["rs16020431"],"URL":"https:\/\/doi.org\/10.3390\/rs16020431","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,1,22]]}}}