{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,20]],"date-time":"2026-05-20T08:41:25Z","timestamp":1779266485281,"version":"3.51.4"},"reference-count":53,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2022,12,27]],"date-time":"2022-12-27T00:00:00Z","timestamp":1672099200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001665","name":"ANR TIMES","doi-asserted-by":"publisher","award":["ANR-17-CE23-0015"],"award-info":[{"award-number":["ANR-17-CE23-0015"]}],"id":[{"id":"10.13039\/501100001665","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001665","name":"ANR TIMES","doi-asserted-by":"publisher","award":["CNES, 2019\u20132022"],"award-info":[{"award-number":["CNES, 2019\u20132022"]}],"id":[{"id":"10.13039\/501100001665","id-type":"DOI","asserted-by":"publisher"}]},{"name":"French TOSCA project AIMCEE","award":["ANR-17-CE23-0015"],"award-info":[{"award-number":["ANR-17-CE23-0015"]}]},{"name":"French TOSCA project AIMCEE","award":["CNES, 2019\u20132022"],"award-info":[{"award-number":["CNES, 2019\u20132022"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>In the context of global change, up-to-date land use land cover (LULC) maps is a major challenge to assess pressures on natural areas. These maps also allow us to assess the evolution of land cover and to quantify changes over time (such as urban sprawl), which is essential for having a precise understanding of a given territory. Few studies have combined information from Sentinel-1 and Sentinel-2 imagery, but merging radar and optical imagery has been shown to have several benefits for a range of study cases, such as semantic segmentation or classification. For this study, we used a newly produced dataset, MultiSenGE, which provides a set of multitemporal and multimodal patches over the Grand-Est region in France. To merge these data, we propose a CNN approach based on spatio-temporal and spatio-spectral feature fusion, ConvLSTM+Inception-S1S2. We used a U-Net base model and ConvLSTM extractor for spatio-temporal features and an inception module for the spatio-spectral features extractor. The results show that describing an overrepresented class is preferable to map urban fabrics (UF). Furthermore, the addition of an Inception module on a date allowing the extraction of spatio-spectral features improves the classification results. Spatio-spectro-temporal method (ConvLSTM+Inception-S1S2) achieves higher global weighted F1Score than all other methods tested.<\/jats:p>","DOI":"10.3390\/rs15010151","type":"journal-article","created":{"date-parts":[[2022,12,28]],"date-time":"2022-12-28T05:30:27Z","timestamp":1672205427000},"page":"151","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":22,"title":["Multimodal and Multitemporal Land Use\/Land Cover Semantic Segmentation on Sentinel-1 and Sentinel-2 Imagery: An Application on a MultiSenGE Dataset"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7557-6862","authenticated-orcid":false,"given":"Romain","family":"Wenger","sequence":"first","affiliation":[{"name":"LIVE UMR 7362 CNRS, University of Strasbourg, F-67000 Strasbourg, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Anne","family":"Puissant","sequence":"additional","affiliation":[{"name":"LIVE UMR 7362 CNRS, University of Strasbourg, F-67000 Strasbourg, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3694-4703","authenticated-orcid":false,"given":"Jonathan","family":"Weber","sequence":"additional","affiliation":[{"name":"IRIMAS UR 7499, University of Haute-Alsace, F-68100 Mulhouse, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lhassane","family":"Idoumghar","sequence":"additional","affiliation":[{"name":"IRIMAS UR 7499, University of Haute-Alsace, F-68100 Mulhouse, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4960-7554","authenticated-orcid":false,"given":"Germain","family":"Forestier","sequence":"additional","affiliation":[{"name":"IRIMAS UR 7499, University of Haute-Alsace, F-68100 Mulhouse, France"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,12,27]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Inglada, J., Vincent, A., Arias, M., Tardy, B., Morin, D., and Rodes, I. (2017). Operational High Resolution Land Cover Map Production at the Country Scale Using Satellite Image Time Series. Remote Sens., 9.","DOI":"10.3390\/rs9010095"},{"key":"ref_2","unstructured":"Zanaga, D., Van De Kerchove, R., De Keersmaecker, W., Souverijns, N., Brockmann, C., Quast, R., Wevers, J., Grosu, A., Paccini, A., and Vergnaud, S. (2022, June 10). ESA WorldCover 10 m 2020 v100. Available online: https:\/\/zenodo.org\/record\/5571936."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41597-022-01307-4","article-title":"Dynamic World, Near real-time global 10 m land use land cover mapping","volume":"9","author":"Brown","year":"2022","journal-title":"Sci. Data"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Karra, K., Kontgis, C., Statman-Weil, Z., Mazzariello, J.C., Mathis, M., and Brumby, S.P. (2021, January 11\u201316). Global land use\/land cover with Sentinel 2 and deep learning. Proceedings of the 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, Brussels, Belgium.","DOI":"10.1109\/IGARSS47720.2021.9553499"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"226609","DOI":"10.1109\/ACCESS.2020.3046261","article-title":"Urban Impervious Surface Extraction Based on Multi-Features and Random Forest","volume":"8","author":"Guo","year":"2020","journal-title":"IEEE Access"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"890","DOI":"10.1002\/joc.4747","article-title":"The urban cool island phenomenon in a high-rise high-density city and its mechanisms","volume":"37","author":"Yang","year":"2017","journal-title":"Int. J. Climatol."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"El Mendili, L., Puissant, A., Chougrad, M., and Sebari, I. (2020). Towards a Multi-Temporal Deep Learning Approach for Mapping Urban Fabric Using Sentinel 2 Images. Remote Sens., 12.","DOI":"10.3390\/rs12030423"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2466","DOI":"10.1109\/TGRS.2003.817267","article-title":"A shape-based approach to change detection of lakes using time series remote sensing images","volume":"41","author":"Li","year":"2003","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Karasiak, N., Sheeren, D., Fauvel, M., Willm, J., Dejoux, J.F., and Monteil, C. (2017, January 27\u201329). Mapping tree species of forests in southwest France using Sentinel-2 image time series. Proceedings of the 2017 9th International Workshop on the Analysis of Multitemporal Remote Sensing Images (MultiTemp), Brugge, Belgium.","DOI":"10.1109\/Multi-Temp.2017.8035215"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"16091","DOI":"10.3390\/rs71215820","article-title":"Object-Based Crop Classification with Landsat-MODIS Enhanced Time-Series Data","volume":"7","author":"Li","year":"2015","journal-title":"Remote Sens."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Pratic\u00f2, S., Solano, F., Di Fazio, S., and Modica, G. (2021). Machine Learning Classification of Mediterranean Forest Habitats in Google Earth Engine Based on Seasonal Sentinel-2 Time-Series and Input Image Composition Optimisation. Remote Sens., 13.","DOI":"10.3390\/rs13040586"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zhou, T., Zhao, M., Sun, C., and Pan, J. (2018). Exploring the Impact of Seasonality on Urban Land-Cover Mapping Using Multi-Season Sentinel-1A and GF-1 WFV Images in a Subtropical Monsoon-Climate Region. ISPRS Int. J. Geo-Inf., 7.","DOI":"10.3390\/ijgi7010003"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"166","DOI":"10.1016\/j.isprsjprs.2019.04.015","article-title":"Deep learning in remote sensing applications: A meta-analysis and review","volume":"152","author":"Ma","year":"2019","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Pelletier, C., Webb, G.I., and Petitjean, F. (2019). Temporal convolutional neural network for the classification of satellite image time series. Remote Sens., 11.","DOI":"10.3390\/rs11050523"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zhang, P., Ke, Y., Zhang, Z., Wang, M., Li, P., and Zhang, S. (2018). Urban Land Use and Land Cover Classification Using Novel Deep Learning Models Based on High Spatial Resolution Satellite Imagery. Sensors, 18.","DOI":"10.3390\/s18113717"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"113192","DOI":"10.1016\/j.rse.2022.113192","article-title":"Unsupervised domain adaptation for global urban extraction using sentinel-1 SAR and sentinel-2 MSI data","volume":"280","author":"Hafner","year":"2022","journal-title":"Remote Sens. Environ."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-Net: Convolutional Networks for Biomedical Image Segmentation. Proceedings of the 2015 International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2481","DOI":"10.1109\/TPAMI.2016.2644615","article-title":"SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation","volume":"39","author":"Badrinarayanan","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1983","DOI":"10.1080\/01431161.2022.2054295","article-title":"U-Net feature fusion for multi-class semantic segmentation of urban fabrics from Sentinel-2 imagery: An application on Grand Est Region, France","volume":"43","author":"Wenger","year":"2022","journal-title":"Int. J. Remote Sens."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Ru\u00dfwurm, M., Pelletier, C., Zollner, M., Lef\u00e8vre, S., and K\u00f6rner, M. (2019). Breizhcrops: A time series dataset for crop type mapping. arXiv.","DOI":"10.5194\/isprs-archives-XLIII-B2-2020-1545-2020"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.isprsjprs.2019.09.016","article-title":"Combining Sentinel-1 and Sentinel-2 Satellite Image Time Series for land cover mapping via a multi-source deep learning architecture","volume":"158","author":"Ienco","year":"2019","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1016\/j.inffus.2016.03.003","article-title":"A review of remote sensing image fusion methods","volume":"32","author":"Ghassemian","year":"2016","journal-title":"Inf. Fusion"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Hedayati, P., and Bargiel, D. (2018, January 22\u201327). Fusion of Sentinel-1 and Sentinel-2 Images for Classification of Agricultural Areas Using a Novel Classification Approach. Proceedings of the IGARSS 2018\u20142018 IEEE International Geoscience and Remote Sensing Symposium, Valencia, Spain.","DOI":"10.1109\/IGARSS.2018.8518327"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Inglada, J., Arias, M., Tardy, B., Morin, D., Valero, S., Hagolle, O., Dedieu, G., Sepulcre, G., Bontemps, S., and Defourny, P. (2015, January 26\u201331). Benchmarking of algorithms for crop type land-cover maps using Sentinel-2 image time series. Proceedings of the 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Milan, Italy.","DOI":"10.1109\/IGARSS.2015.7326700"},{"key":"ref_25","unstructured":"Smets, P. (1994). What is Dempster-Shafer\u2019s model. Advances in the Dempster-Shafer Theory of Evidence, John Wiley & Sons, Inc."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"718","DOI":"10.1080\/17445647.2017.1372316","article-title":"Fusion of Sentinel-1A and Sentinel-2A data for land cover mapping: A case study in the lower Magdalena region, Colombia","volume":"13","author":"Clerici","year":"2017","journal-title":"J. Maps"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1016\/j.isprsjprs.2017.11.011","article-title":"Beyond RGB: Very high resolution urban remote sensing with multimodal deep networks","volume":"140","author":"Audebert","year":"2018","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"545","DOI":"10.1109\/TNNLS.2016.2636227","article-title":"A Deep Convolutional Coupling Network for Change Detection Based on Heterogeneous Optical and Radar Images","volume":"29","author":"Liu","year":"2018","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"778","DOI":"10.1109\/LGRS.2017.2681128","article-title":"Deep Learning Classification of Land Cover and Crop Types Using Remote Sensing Data","volume":"14","author":"Kussul","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Pfeuffer, A., Schulz, K., and Dietmayer, K. (2019, January 9\u201312). Semantic segmentation of video sequences with convolutional lstms. Proceedings of the 2019 IEEE Intelligent Vehicles Symposium (IV), Paris, France.","DOI":"10.1109\/IVS.2019.8813852"},{"key":"ref_31","unstructured":"Shi, X., Chen, Z., Wang, H., Yeung, D.Y., Wong, W.K., and Woo, W.C. (2015). Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting, 2015. arXiv."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Wang, D., Yang, Y., and Ning, S. (2018, January 8\u201313). DeepSTCL: A deep spatio-temporal ConvLSTM for travel demand prediction. Proceedings of the 2018 International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, Brazil.","DOI":"10.1109\/IJCNN.2018.8489530"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Jamaluddin, I., Thaipisutikul, T., Chen, Y.N., Chuang, C.H., and Hu, C.L. (2021). MDPrePost-Net: A Spatial-Spectral-Temporal Fully Convolutional Network for Mapping of Mangrove Degradation Affected by Hurricane Irma 2017 Using Sentinel-2 Data. Remote Sens., 13.","DOI":"10.3390\/rs13245042"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"112600","DOI":"10.1016\/j.rse.2021.112600","article-title":"Spatial and temporal deep learning methods for deriving land-use following deforestation: A pan-tropical case study using Landsat time series","volume":"264","author":"Masolele","year":"2021","journal-title":"Remote Sens. Environ."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Chang, Y.L., Tan, T.H., Chen, T.H., Chuah, J.H., Chang, L., Wu, M.C., Tatini, N.B., Ma, S.C., and Alkhaleefah, M. (2022). Spatial-Temporal Neural Network for Rice Field Classification from SAR Images. Remote Sens., 14.","DOI":"10.3390\/rs14081929"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"635","DOI":"10.5194\/isprs-annals-V-3-2022-635-2022","article-title":"Multisenge: A Multimodal and Multitemporal Benchmark Dataset for Land Use\/Land Cover Remote Sensing Applications","volume":"V-3-2022","author":"Wenger","year":"2022","journal-title":"ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_37","unstructured":"Wenger, R., Puissant, A., and Mich\u00e9a, D. Towards an annual Urban Settlement map in France at 10m spatial resolution using a method for massive streams of Sentinel-2, LIVE CNRS UMR7362, Strasbourg, France, to be submitted."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Maxwell, A.E., Warner, T.A., and Guill\u00e9n, L.A. (2021). Accuracy Assessment in Convolutional Neural Network-Based Deep Learning Remote Sensing Studies\u2014Part 2: Recommendations and Best Practices. Remote Sens., 13.","DOI":"10.3390\/rs13132591"},{"key":"ref_39","unstructured":"Yakubovskiy, P. (2022, November 08). Segmentation Models. Available online: https:\/\/github.com\/qubvel\/segmentation_models."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"112467","DOI":"10.1016\/j.rse.2021.112467","article-title":"Learning U-Net without forgetting for near real-time wildfire monitoring by the fusion of SAR and optical time series","volume":"261","author":"Zhang","year":"2021","journal-title":"Remote Sens. Environ."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"198","DOI":"10.1016\/j.isprsjprs.2021.02.011","article-title":"Large-scale rice mapping under different years based on time-series Sentinel-1 images using deep semantic segmentation model","volume":"174","author":"Wei","year":"2021","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"505","DOI":"10.5194\/isprs-annals-V-3-2020-505-2020","article-title":"Semantic Segmentation of Brazilian Savanna Vegetation Using High Spatial Resolution Satellite Data and U-Net","volume":"V-3-2020","author":"Neves","year":"2020","journal-title":"ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1685","DOI":"10.1109\/LGRS.2017.2728698","article-title":"Land Cover Classification via Multitemporal Spatial Data by Deep Recurrent Neural Networks","volume":"14","author":"Ienco","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1109\/MGRS.2017.2762307","article-title":"Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources","volume":"5","author":"Zhu","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"6531","DOI":"10.1109\/JSTARS.2022.3197937","article-title":"Calibrated Focal Loss for Semantic Labeling of High-Resolution Remote Sensing Images","volume":"15","author":"Bai","year":"2022","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_46","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A Method for Stochastic Optimization. arXiv."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"2664","DOI":"10.1080\/01431161.2019.1694725","article-title":"Analysis of various optimizers on deep convolutional neural network model in the application of hyperspectral remote sensing image classification","volume":"41","author":"Bera","year":"2020","journal-title":"Int. J. Remote Sens."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Pires de Lima, R., and Marfurt, K. (2019). Convolutional neural network for remote-sensing scene classification: Transfer learning analysis. Remote Sens., 12.","DOI":"10.3390\/rs12010086"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"64381","DOI":"10.1109\/ACCESS.2021.3075951","article-title":"Improving road semantic segmentation using generative adversarial network","volume":"9","author":"Abdollahi","year":"2021","journal-title":"IEEE Access"},{"key":"ref_50","first-page":"1","article-title":"Global and local contrastive self-supervised learning for semantic segmentation of HR remote sensing images","volume":"60","author":"Li","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Maxwell, A.E., Warner, T.A., and Guill\u00e9n, L.A. (2021). Accuracy Assessment in Convolutional Neural Network-Based Deep Learning Remote Sensing Studies\u2014Part 1: Literature Review. Remote Sens., 13.","DOI":"10.3390\/rs13132450"},{"key":"ref_52","unstructured":"Wenger, R., Puissant, A., Weber, J., Idoumghar, L., and Forestier, G. (2022, June 08). A New Remote Sensing Benchmark Dataset for Machine Learning Applications: MultiSenGE, 2022. ANR-17-CE23-0015. Available online: https:\/\/zenodo.org\/record\/6375466#.Y6q45UwRWUk."},{"key":"ref_53","unstructured":"Iqbal, H. (2022, June 08). HarisIqbal88\/PlotNeuralNet v1.0.0. Available online: https:\/\/zenodo.org\/record\/2526396#.Y6q5CEwRWUk."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/1\/151\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:52:56Z","timestamp":1760147576000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/1\/151"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,27]]},"references-count":53,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,1]]}},"alternative-id":["rs15010151"],"URL":"https:\/\/doi.org\/10.3390\/rs15010151","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,27]]}}}