{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,21]],"date-time":"2026-03-21T19:14:19Z","timestamp":1774120459899,"version":"3.50.1"},"reference-count":50,"publisher":"MDPI AG","issue":"17","license":[{"start":{"date-parts":[[2024,8,23]],"date-time":"2024-08-23T00:00:00Z","timestamp":1724371200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Federal Ministry for Economic Affairs and Climate Action","award":["01MK21004K"],"award-info":[{"award-number":["01MK21004K"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Earth observation missions such as Sentinel and Landsat support the large-scale identification of agricultural crops by providing free radar and multispectral satellite images. The extraction of representative image information as well as the combination of different image sources for improved feature selection still represent a major challenge in the field of remote sensing. In this paper, we propose a novel three-dimensional (3D) deep learning U-Net model to fuse multi-level image features from multispectral and synthetic aperture radar (SAR) time series data for seasonal crop-type mapping at a regional scale. For this purpose, we used a dual-stream U-Net with a 3D squeeze-and-excitation fusion module applied at multiple stages in the network to progressively extract and combine multispectral and SAR image features. Additionally, we introduced a distinctive method for generating patch-based multitemporal multispectral composites by selective image sampling within a 14-day window, prioritizing those with minimal cloud cover. The classification results showed that the proposed network provided the best overall accuracy (94.5%) compared to conventional two-dimensional (2D) and three-dimensional U-Net models (2D: 92.6% and 3D: 94.2%). Our network successfully learned multi-modal dependencies between the multispectral and SAR satellite images, leading to improved field mapping of spectrally similar and heterogeneous classes while mitigating the limitations imposed by persistent cloud coverage. Additionally, the feature representations extracted by the proposed network demonstrated their transferability to a new cropping season, providing a reliable mapping of spatio-temporal crop type patterns.<\/jats:p>","DOI":"10.3390\/rs16173115","type":"journal-article","created":{"date-parts":[[2024,8,23]],"date-time":"2024-08-23T12:53:19Z","timestamp":1724417599000},"page":"3115","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Multi-Stage Feature Fusion of Multispectral and SAR Satellite Images for Seasonal Crop-Type Mapping at Regional Scale Using an Adapted 3D U-Net Model"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8647-3906","authenticated-orcid":false,"given":"Lucas","family":"Wittstruck","sequence":"first","affiliation":[{"name":"Institute of Computer Science, University of Osnabrueck, Wachsbleiche 27, D-49090 Osnabrueck, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4652-1640","authenticated-orcid":false,"given":"Thomas","family":"Jarmer","sequence":"additional","affiliation":[{"name":"Institute of Computer Science, University of Osnabrueck, Wachsbleiche 27, D-49090 Osnabrueck, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2586-3748","authenticated-orcid":false,"given":"Bj\u00f6rn","family":"Waske","sequence":"additional","affiliation":[{"name":"Institute of Computer Science, University of Osnabrueck, Wachsbleiche 27, D-49090 Osnabrueck, Germany"}]}],"member":"1968","published-online":{"date-parts":[[2024,8,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1016\/j.compag.2019.04.026","article-title":"Using NDVI percentiles to monitor real-time crop growth","volume":"162","author":"Li","year":"2019","journal-title":"Comput. Electron. Agric."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"e20180055","DOI":"10.1590\/1678-992x-2018-0055","article-title":"Applying the NDVI from satellite images in delimiting management zones for annual crops","volume":"77","author":"Damian","year":"2020","journal-title":"Sci. Agric."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"511","DOI":"10.1080\/22797254.2018.1455540","article-title":"A rule-based approach for crop identification using multi-temporal and multi-sensor phenological metrics","volume":"51","author":"Ghazaryan","year":"2018","journal-title":"Eur. J. Remote Sens."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Orynbaikyzy, A., Gessner, U., Mack, B., and Conrad, C. (2020). Crop Type Classification Using Fusion of Sentinel-1 and Sentinel-2 Data: Assessing the Impact of Feature Selection, Optical Data Availability, and Parcel Sizes on the Accuracies. Remote Sens., 12.","DOI":"10.3390\/rs12172779"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Asam, S., Gessner, U., Almengor Gonz\u00e1lez, R., Wenzl, M., Kriese, J., and Kuenzer, C. (2022). Mapping Crop Types of Germany by Combining Temporal Statistical Metrics of Sentinel-1 and Sentinel-2 Time Series with LPIS Data. Remote Sens., 14.","DOI":"10.3390\/rs14132981"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"112831","DOI":"10.1016\/j.rse.2021.112795","article-title":"Mapping of crop types and crop sequences with combined time series of Sentinel-1, Sentinel-2 and Landsat 8 data for Germany","volume":"269","author":"Schwieder","year":"2022","journal-title":"Remote Sens. Environ."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Wang, X., Fang, S., Yang, Y., Du, J., and Wu, H. (2023). A New Method for Crop Type Mapping at the Regional Scale Using Multi-Source and Multi-Temporal Sentinel Imagery. Remote Sens., 15.","DOI":"10.3390\/rs15092466"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"3858","DOI":"10.1109\/TGRS.2007.898446","article-title":"Fusion of Support Vector Machines for Classification of Multisensor Data","volume":"45","author":"Waske","year":"2007","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"5279","DOI":"10.3390\/rs6065279","article-title":"Mapping Land Management Regimes in Western Ukraine Using Optical and SAR Data","volume":"6","author":"Stefanski","year":"2014","journal-title":"Remote Sens."},{"key":"ref_10","first-page":"595","article-title":"Combining Sentinel-1 and Sentinel-2 data for improved land use and land cover mapping of monsoon regions","volume":"73","author":"Steinhausen","year":"2018","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Joshi, N., Baumann, M., Ehammer, A., Fensholt, R., Grogan, K., Hostert, P., Jepsen, M., Kuemmerle, T., Meyfroidt, P., and Mitchard, E. (2016). A Review of the Application of Optical and Radar Remote Sensing Data Fusion to Land Use Mapping and Monitoring. Remote Sens., 8.","DOI":"10.3390\/rs8010070"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1080\/01431161.2011.587844","article-title":"Crop classification using multi-configuration SAR data in the North China Plain","volume":"33","author":"Jia","year":"2012","journal-title":"Int. J. Remote Sens."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"8076","DOI":"10.1080\/01431161.2018.1539275","article-title":"Crop classification with WorldView-2 imagery using Support Vector Machine comparing texture analysis approaches and grey relational analysis in Jianan Plain, Taiwan","volume":"40","author":"Wan","year":"2019","journal-title":"Int. J. Remote Sens."},{"key":"ref_14","first-page":"103178","article-title":"Mapping crop type in Northeast China during 2013\u20132021 using automatic sampling and tile-based image classification","volume":"117","author":"Xuan","year":"2023","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"207","DOI":"10.1016\/j.compag.2009.06.004","article-title":"Object- and pixel-based analysis for mapping crops and their agro-environmental associated measures using QuickBird imagery","volume":"68","year":"2009","journal-title":"Comput. Electron. Agric."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2492","DOI":"10.1109\/JSTARS.2013.2253089","article-title":"Optimization of Object-Based Image Analysis With Random Forests for Land Cover Mapping","volume":"6","author":"Stefanski","year":"2013","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1016\/j.rse.2017.10.005","article-title":"Sentinel-2 cropland mapping using pixel-based and object-based time-weighted dynamic time warping analysis","volume":"204","author":"Belgiu","year":"2018","journal-title":"Remote Sens. Environ."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1457","DOI":"10.1109\/TGRS.2008.916089","article-title":"Classifying Multilevel Imagery From SAR and Optical Sensors by Decision Fusion","volume":"46","author":"Waske","year":"2008","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_19","first-page":"53","article-title":"A Progressive Crop Type Classification Using Multitemporal Remote Sensing Data and Phenological Information","volume":"86","author":"Heupel","year":"2018","journal-title":"PFG\u2014J. Photogramm. Remote Sens. Geoinf. Sci."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"430","DOI":"10.1016\/j.rse.2018.11.032","article-title":"Deep learning based multi-temporal crop classification","volume":"221","author":"Zhong","year":"2019","journal-title":"Remote Sens. Environ."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"778","DOI":"10.1109\/LGRS.2017.2681128","article-title":"Deep Learning Classification of Land Cover and Crop Types Using Remote Sensing Data","volume":"14","author":"Kussul","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Gu, L., He, F., and Yang, S. (2019, January 5\u20136). Crop Classification based on Deep Learning in Northeast China using SAR and Optical Imagery. Proceedings of the 2019 SAR in Big Data Era (BIGSARDATA), Beijing, China.","DOI":"10.1109\/BIGSARDATA.2019.8858437"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.isprsjprs.2019.09.016","article-title":"Combining Sentinel-1 and Sentinel-2 Satellite Image Time Series for land cover mapping via a multi-source deep learning architecture","volume":"158","author":"Ienco","year":"2019","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Ofori-Ampofo, S., Pelletier, C., and Lang, S. (2021). Crop Type Mapping from Optical and Radar Time Series Using Attention-Based Deep Learning. Remote Sens., 13.","DOI":"10.3390\/rs13224668"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Weilandt, F., Behling, R., Goncalves, R., Madadi, A., Richter, L., Sanona, T., Spengler, D., and Welsch, J. (2023). Early Crop Classification via Multi-Modal Satellite Data Fusion and Temporal Attention. Remote Sens., 15.","DOI":"10.3390\/rs15030799"},{"key":"ref_26","unstructured":"Sohn, K., Shang, W., and Lee, H. (2014, January 8\u201313). Improved multimodal deep learning with variation of information. Proceedings of the 27th International Conference on Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1016\/j.isprsjprs.2021.02.018","article-title":"Sentinel SAR-optical fusion for crop type mapping using deep learning and Google Earth Engine","volume":"175","author":"Adrian","year":"2021","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Valero, S., Arnaud, L., Planells, M., and Ceschia, E. (2021). Synergy of Sentinel-1 and Sentinel-2 Imagery for Early Seasonal Agricultural Crop Mapping. Remote Sens., 13.","DOI":"10.3390\/rs13234891"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"222","DOI":"10.1016\/j.isprsjprs.2022.11.020","article-title":"Bridging optical and SAR satellite image time series via contrastive feature extraction for crop classification","volume":"195","author":"Yuan","year":"2023","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Xu, D., Li, Z., Feng, H., Wu, F., and Wang, Y. (2024). Multi-Scale Feature Fusion Network with Symmetric Attention for Land Cover Classification Using SAR and Optical Images. Remote Sens., 16.","DOI":"10.3390\/rs16060957"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"272","DOI":"10.1016\/j.isprsjprs.2023.04.008","article-title":"Aligning semantic distribution in fusing optical and SAR images for land use classification","volume":"199","author":"Li","year":"2023","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Hu, J., Shen, L., and Sun, G. (2018, January 18\u201323). Squeeze-and-Excitation Networks. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00745"},{"key":"ref_33","unstructured":"NIBIS Kartenserver (2023). Bodenkarte von Niedersachsen 1:50000 (BK50), Landesamt f\u00fcr Bergbau, Energie und Geologie (LBEG)."},{"key":"ref_34","unstructured":"Deutscher Wetterdienst (2023). Lufttemperatur & Niederschlag: Normalwerte (Zeitraum 1971\u20132000), Deutscher Wetterdienst."},{"key":"ref_35","unstructured":"Lower Saxony Ministry for Food, Agriculture, and Consumer Protection (2021). Die Nieders\u00e4chsische Landwirtschaft in Zahlen 2021: Einschlie\u00dflich Erg\u00e4nzungen und Aktualisierungen Stand Juli 2023, Lower Saxony Ministry for Food, Agriculture, and Consumer Protection."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1016\/j.rse.2017.06.031","article-title":"Google Earth Engine: Planetary-scale geospatial analysis for everyone","volume":"202","author":"Gorelick","year":"2017","journal-title":"Remote Sens. Environ."},{"key":"ref_37","unstructured":"European Court of Auditors (2016). The Land Parcel Identification System. Special Report No 25, 2016, Publications Office of the EU."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1016\/S0034-4257(02)00096-2","article-title":"Overview of the radiometric and biophysical performance of the MODIS vegetation indices","volume":"83","author":"Huete","year":"2002","journal-title":"Remote Sens. Environ."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"324","DOI":"10.1016\/S2095-3119(15)61321-1","article-title":"How do temporal and spectral features matter in crop classification in Heilongjiang Province, China?","volume":"16","author":"Hu","year":"2017","journal-title":"J. Integr. Agric."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"4038","DOI":"10.1109\/JSTARS.2019.2938388","article-title":"Derivation of Tasseled Cap Transformation Coefficients for Sentinel-2 MSI At-Sensor Reflectance Data","volume":"12","author":"Shi","year":"2019","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"653","DOI":"10.1080\/22797254.2019.1701560","article-title":"Multi-temporal RapidEye Tasselled Cap data for land cover classification","volume":"52","author":"Raab","year":"2019","journal-title":"Eur. J. Remote Sens."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Shimizu, K., Murakami, W., Furuichi, T., and Estoque, R.C. (2023). Mapping Land Use\/Land Cover Changes and Forest Disturbances in Vietnam Using a Landsat Temporal Segmentation Algorithm. Remote Sens., 15.","DOI":"10.3390\/rs15030851"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"234","DOI":"10.1007\/978-3-319-24574-4_28","article-title":"U-net: Convolutional networks for biomedical image segmentation","volume":"Volume 9351","author":"Ronneberger","year":"2015","journal-title":"Lecture Notes in Computer Science"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Wei, S., Zhang, H., Wang, C., Wang, Y., and Xu, L. (2019). Multi-Temporal SAR Data Large-Scale Crop Mapping Based on U-Net Model. Remote Sens., 11.","DOI":"10.3390\/rs11010068"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1016\/j.neunet.2019.08.025","article-title":"MultiResUNet: Rethinking the U-Net architecture for multimodal biomedical image segmentation","volume":"121","author":"Ibtehaz","year":"2020","journal-title":"Neural Netw."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1016\/j.rse.2014.02.015","article-title":"Good practices for estimating area and assessing accuracy of land change","volume":"148","author":"Olofsson","year":"2014","journal-title":"Remote Sens. Environ."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"157","DOI":"10.5589\/m11-022","article-title":"Crop classification in rain-fed and irrigated agricultural areas using Landsat TM and ALOS\/PALSAR data","volume":"37","author":"Albizua","year":"2011","journal-title":"Canadian J. Remote Sens."},{"key":"ref_48","unstructured":"Imbery, F., Friedrich, K., Haeseler, S., Koppe, C., Janssen, W., and Bissolli, P. (2018). Vorl\u00e4ufiger R\u00fcckblick auf den Sommer 2018\u2013Eine Bilanz Extremer Wetterereignisse, Deutscher Wetterdienst."},{"key":"ref_49","unstructured":"Krampf, K. (2019). Die Ernte Nach Hitze und Trockenheit\u2014Eine Regionalanalyse des Landwirtschaftlichen Anbaus in Niedersachsen im Sommer 2018, Statistische Monatshefte Niedersachsen."},{"key":"ref_50","unstructured":"Landwirtschaftskammer Niedersachen (2018). Landessortenversuche 2018: Sommerfuttergerste, Landwirtschaftskammer Niedersachen."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/17\/3115\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:42:12Z","timestamp":1760110932000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/17\/3115"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,23]]},"references-count":50,"journal-issue":{"issue":"17","published-online":{"date-parts":[[2024,9]]}},"alternative-id":["rs16173115"],"URL":"https:\/\/doi.org\/10.3390\/rs16173115","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,8,23]]}}}