{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,24]],"date-time":"2026-03-24T05:46:48Z","timestamp":1774331208603,"version":"3.50.1"},"reference-count":33,"publisher":"MDPI AG","issue":"16","license":[{"start":{"date-parts":[[2021,8,19]],"date-time":"2021-08-19T00:00:00Z","timestamp":1629331200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>To improve precipitation estimation accuracy, new methods, which are able to merge different precipitation measurement modalities, are necessary. In this study, we propose a deep learning method to merge rain gauge measurements with a ground-based radar composite and thermal infrared satellite imagery. The proposed convolutional neural network, composed of an encoder\u2013decoder architecture, performs a multiscale analysis of the three input modalities to estimate simultaneously the rainfall probability and the precipitation rate value with a spatial resolution of 2 km. The training of our model and its performance evaluation are carried out on a dataset spanning 5 years from 2015 to 2019 and covering Belgium, the Netherlands, Germany and the North Sea. Our results for instantaneous precipitation detection, instantaneous precipitation rate estimation, and for daily rainfall accumulation estimation show that the best accuracy is obtained for the model combining all three modalities. The ablation study, done to compare every possible combination of the three modalities, shows that the combination of rain gauges measurements with radar data allows for a considerable increase in the accuracy of the precipitation estimation, and the addition of satellite imagery provides precipitation estimates where rain gauge and radar coverage are lacking. We also show that our multi-modal model significantly improves performance compared to the European radar composite product provided by OPERA and the quasi gauge-adjusted radar product RADOLAN provided by the DWD for precipitation rate estimation.<\/jats:p>","DOI":"10.3390\/rs13163278","type":"journal-article","created":{"date-parts":[[2021,8,19]],"date-time":"2021-08-19T09:58:06Z","timestamp":1629367086000},"page":"3278","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":31,"title":["A Deep Learning Multimodal Method for Precipitation Estimation"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4905-5590","authenticated-orcid":false,"given":"Arthur","family":"Moraux","sequence":"first","affiliation":[{"name":"Royal Meteorological Institute of Belgium, Avenue Circulaire 3, 1180 Brussels, Belgium"},{"name":"ETRO Department, Vrije Universiteit Brussel, Pleinlaan 2, 1050 Brussels, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4571-1450","authenticated-orcid":false,"given":"Steven","family":"Dewitte","sequence":"additional","affiliation":[{"name":"Royal Meteorological Institute of Belgium, Avenue Circulaire 3, 1180 Brussels, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0688-8173","authenticated-orcid":false,"given":"Bruno","family":"Cornelis","sequence":"additional","affiliation":[{"name":"ETRO Department, Vrije Universiteit Brussel, Pleinlaan 2, 1050 Brussels, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7290-0428","authenticated-orcid":false,"given":"Adrian","family":"Munteanu","sequence":"additional","affiliation":[{"name":"ETRO Department, Vrije Universiteit Brussel, Pleinlaan 2, 1050 Brussels, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,8,19]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"505","DOI":"10.5194\/hess-20-505-2016","article-title":"Development and verification of a stochastic precipitation nowcasting system for urban hydrology in Belgium","volume":"20","author":"Foresti","year":"2016","journal-title":"Hydrol. Earth Syst. Sci."},{"key":"ref_2","unstructured":"WMO (2018). Guide to Instruments and Methods of Observation: Volume I\u2014Measurement of Meteorological Variables, WMO."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Villarini, G., Mapaka, P.V., Krajewski, W.F., and Moore, R.J. (2008). Rainfall and sampling uncertainties: A rain gauge perspective. J. Geophys. Res., 113.","DOI":"10.1029\/2007JD009214"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1007\/s10712-009-9079-x","article-title":"Review of the Different Sources of Uncertainty in Single Polarization Radar-Based Estimates of Rainfall","volume":"31","author":"Villarini","year":"2010","journal-title":"Surv. Geophys."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1175\/2009BAMS2747.1","article-title":"RADAR-Rainfall Uncertainties","volume":"91","author":"Krajewski","year":"2010","journal-title":"Bull. Am. Meteorol. Soc."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1016\/j.atmosres.2011.08.014","article-title":"Recent advances in classification of observations from dual polarization weather radars","volume":"119","author":"Chandrasekar","year":"2013","journal-title":"Atmos. Res."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"6356","DOI":"10.1029\/2018WR023332","article-title":"A review of radar-rain gauge data merging methods and their potential for urban hydrological applications","volume":"55","author":"Wang","year":"2019","journal-title":"Water Resour. Res."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1223","DOI":"10.1175\/JHM-D-15-0166.1","article-title":"Generation and Verification of Rainfall Estimates from 10-Yr Volumetric Weather Radar Measurements","volume":"17","author":"Goudenhoofdt","year":"2016","journal-title":"J. Hydrometeorol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1088","DOI":"10.1175\/1525-7541(2003)004<1088:SREUCP>2.0.CO;2","article-title":"Satellite rainfall estimation using combined passive microwave and infrared algorithms","volume":"4","author":"Kidd","year":"2003","journal-title":"J. Hydrometeorol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1041","DOI":"10.1002\/joc.635","article-title":"Satellite rainfall climatology: A review","volume":"21","author":"Kidd","year":"2001","journal-title":"Int. J. Climatol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1271","DOI":"10.1175\/JHM-D-16-0176.1","article-title":"Precipitation Identification with Bispectral Satellite Information Using Deep Learning Approaches","volume":"18","author":"Tao","year":"2017","journal-title":"J. Hydrometeorol."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"5801","DOI":"10.5194\/hess-22-5801-2018","article-title":"The PERSIANN family of global satellite precipitation data: A review and evaluation of products","volume":"22","author":"Nguyen","year":"2018","journal-title":"Hydrol. Earth Syst. Sci."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"982","DOI":"10.1109\/TGRS.2019.2942280","article-title":"A machine learning system for precipitation estimation using satellite and ground radar network observations","volume":"58","author":"Chen","year":"2019","journal-title":"IEEE Trans. Geosci. Remote. Sens."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2273","DOI":"10.1175\/JHM-D-19-0110.1","article-title":"PERSIANN-CNN: Precipitation Estimation from Remotely Sensed Information Using Artificial Neural Networks\u2013Convolutional Neural Networks","volume":"20","author":"Sadeghi","year":"2019","journal-title":"J. Hydrometeorol."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Hayatbini, N., Kong, B., Hsu, K.-l., Nguyen, P., Sorooshian, S., Stephens, G., Fowlkes, C., Nemani, R., and Ganguly, S. (2019). Conditional Generative Adversarial Networks (cGANs) for Near Real-Time Precipitation Estimation from Multispectral GOES-16 Satellite Imageries\u2014PERSIANN-cGAN. Remote Sens., 11.","DOI":"10.3390\/rs11192193"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2457","DOI":"10.1175\/1520-0477(1998)079<2457:SBIIPF>2.0.CO;2","article-title":"Satellite-Satellite-based insights into precipitation formation processes in continental and maritime convective clouds","volume":"79","author":"Rosenfeld","year":"1998","journal-title":"Bull. Am. Meteorol. Soc."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"6739","DOI":"10.5194\/acp-8-6739-2008","article-title":"Clouds-aerosols-precipitation satellite analysis tool (CAPSAT)","volume":"8","author":"Lensky","year":"2008","journal-title":"Atmos. Chem. Phys."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Roebeling, R.A., and Holleman, I. (2009). SEVIRI rainfall retrieval and validation using weather radar observations. J. Geophys. Res. Atmos., 114.","DOI":"10.1029\/2009JD012102"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"977","DOI":"10.1175\/BAMS-83-7-Schmetz-2","article-title":"An introduction to Meteosat second generation (MSG)","volume":"83","author":"Schmetz","year":"2002","journal-title":"Bull. Am. Meteorol. Soc."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Moraux, A., Dewitte, S., Cornelis, B., and Munteanu, A. (2019). Deep Learning for Precipitation Estimation from Satellite and Rain Gauges Measurements. Remote Sens., 11.","DOI":"10.3390\/rs11212463"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"195","DOI":"10.5194\/hess-13-195-2009","article-title":"Evaluation of radar-gauge merging methods for quantitative precipitation estimates","volume":"13","author":"Goudenhoofdt","year":"2009","journal-title":"Hydrol. Earth Syst. Sci."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1038\/s41586-019-0912-1","article-title":"Prabhat Deep learning and process understanding for data-driven Earth system science","volume":"566","author":"Reichstein","year":"2019","journal-title":"Nature"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Liu, Y., Minh Nguyen, D., Deligiannis, N., Ding, W., and Munteanu, A. (2017). Hourglass-shapenetwork based semantic segmentation for high resolution aerial imagery. Remote Sens., 9.","DOI":"10.3390\/rs9060522"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Saltikoff, E., Haase, G., Delobbe, L., Gaussiat, N., Martet, M., Idziorek, D., Leijnse, H., Nov\u00e1k, P., Lukach, M., and Stephan, K. (2019). OPERA the radar project. Atmosphere, 10.","DOI":"10.3390\/atmos10060320"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"768","DOI":"10.1016\/j.jhydrol.2019.03.093","article-title":"Long-term analysis of gauge-adjusted radar rainfall accumulations at European scale","volume":"573","author":"Park","year":"2019","journal-title":"J. Hydrol."},{"key":"ref_27","unstructured":"Winterrath, T., Brendel, C., Hafer, M., Jungh\u00e4nel, T., Klameth, A., Lengfeld, K., Walawender, E., Weigl, E., and Becker, A. (2018). RADKLIM Version 2017.002: Reprocessed Quasi Gauge-Adjusted Radar Data, 5-Minute Precipitation Sums (YW), Deutscher Wetterdienst."},{"key":"ref_28","unstructured":"Ioffe, S., and Szegedy, C. (2015, January 6\u201311). Batch normalization: Accelerating deep network training by reducing internal covariate shift. Proceedings of the 32nd International Conference on Machine Learning, Lille, France."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going Deeper with Convolutions. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2016, January 27\u201330). Rethinking the inception architecture for computer vision. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.308"},{"key":"ref_32","unstructured":"Kendall, A., Gal, Y., and Cipolla, R. (2018, January 18\u201323). Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1175\/1520-0469(1948)005<0165:TDORWS>2.0.CO;2","article-title":"The distribution of raindrops with size","volume":"5","author":"Marshall","year":"1948","journal-title":"J. Meteor."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/16\/3278\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:47:05Z","timestamp":1760165225000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/16\/3278"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,19]]},"references-count":33,"journal-issue":{"issue":"16","published-online":{"date-parts":[[2021,8]]}},"alternative-id":["rs13163278"],"URL":"https:\/\/doi.org\/10.3390\/rs13163278","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,8,19]]}}}