{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,30]],"date-time":"2026-04-30T00:43:53Z","timestamp":1777509833055,"version":"3.51.4"},"reference-count":42,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2021,12,22]],"date-time":"2021-12-22T00:00:00Z","timestamp":1640131200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Deep-learning-based radar echo extrapolation methods have achieved remarkable progress in the precipitation nowcasting field. However, they suffer from a common notorious problem\u2014they tend to produce blurry predictions. Although some efforts have been made in recent years, the blurring problem is still under-addressed. In this work, we propose three effective strategies to assist deep-learning-based radar echo extrapolation methods to achieve more realistic and detailed prediction. Specifically, we propose a spatial generative adversarial network (GAN) and a spectrum GAN to improve image fidelity. The spatial and spectrum GANs aim at penalizing the distribution discrepancy between generated and real images from the spatial domain and spectral domain, respectively. In addition, a masked style loss is devised to further enhance the details by transferring the detailed texture of ground truth radar sequences to extrapolated ones. We apply a foreground mask to prevent the background noise from transferring to the outputs. Moreover, we also design a new metric termed the power spectral density score (PSDS) to quantify the perceptual quality from a frequency perspective. The PSDS metric can be applied as a complement to other visual evaluation metrics (e.g., LPIPS) to achieve a comprehensive measurement of image sharpness. We test our approaches with both ConvLSTM baseline and U-Net baseline, and comprehensive ablation experiments on the SEVIR dataset show that the proposed approaches are able to produce much more realistic radar images than baselines. Most notably, our methods can be readily applied to any deep-learning-based spatiotemporal forecasting models to acquire more detailed results.<\/jats:p>","DOI":"10.3390\/rs14010024","type":"journal-article","created":{"date-parts":[[2021,12,23]],"date-time":"2021-12-23T02:02:57Z","timestamp":1640224977000},"page":"24","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":26,"title":["Towards a More Realistic and Detailed Deep-Learning-Based Radar Echo Extrapolation Method"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1256-7023","authenticated-orcid":false,"given":"Yuan","family":"Hu","sequence":"first","affiliation":[{"name":"DAMO Academy, Alibaba Group, Beijing 100102, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lei","family":"Chen","sequence":"additional","affiliation":[{"name":"DAMO Academy, Alibaba Group, Beijing 100102, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhibin","family":"Wang","sequence":"additional","affiliation":[{"name":"DAMO Academy, Alibaba Group, Beijing 100102, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3890-6599","authenticated-orcid":false,"given":"Xiang","family":"Pan","sequence":"additional","affiliation":[{"name":"DAMO Academy, Alibaba Group, Beijing 100102, China"},{"name":"Key Laboratory of Mesoscale Severe Weather\/MOE, School of Atmospheric Sciences, Nanjing University, Nanjing 210033, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hao","family":"Li","sequence":"additional","affiliation":[{"name":"DAMO Academy, Alibaba Group, Beijing 100102, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,12,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2079","DOI":"10.1175\/1520-0477(1998)079<2079:NTASR>2.0.CO;2","article-title":"Nowcasting thunderstorms: A status report","volume":"79","author":"Wilson","year":"1998","journal-title":"Bull. Am. Meteorol. Soc."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"409","DOI":"10.1175\/BAMS-D-11-00263.1","article-title":"Use of NWP for nowcasting convective precipitation: Recent progress and challenges","volume":"95","author":"Sun","year":"2014","journal-title":"Bull. Am. Meteorol. Soc."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"785","DOI":"10.1175\/1520-0426(1993)010<0785:TTITAA>2.0.CO;2","article-title":"TITAN: Thunderstorm identification, tracking, analysis, and nowcasting\u2014A radar-based methodology","volume":"10","author":"Dixon","year":"1993","journal-title":"J. Atmos. Ocean. Technol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1286","DOI":"10.1175\/1520-0450(1995)034<1286:NOMAGO>2.0.CO;2","article-title":"Nowcasting of motion and growth of precipitation with radar over a complex orography","volume":"34","author":"Li","year":"1995","journal-title":"J. Appl. Meteorol. Climatol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1016\/j.atmosres.2018.05.030","article-title":"A radar-based centroid tracking algorithm for severe weather surveillance: Identifying split\/merge processes in convective systems","volume":"213","author":"Rigo","year":"2018","journal-title":"Atmos. Res."},{"key":"ref_6","unstructured":"Lucas, B.D., and Kanade, T. (1981, January 24\u201328). An iterative image registration technique with an application to stereo vision. Proceedings of the 7th International Joint Conference on Artificial Intelligence, Vancouver, BC, Canada."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2631","DOI":"10.5194\/gmd-13-2631-2020","article-title":"RainNet v1. 0: A convolutional neural network for radar-based precipitation nowcasting","volume":"13","author":"Ayzel","year":"2020","journal-title":"Geosci. Model Dev."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1038\/s41586-019-0912-1","article-title":"Deep learning and process understanding for data-driven Earth system science","volume":"566","author":"Reichstein","year":"2019","journal-title":"Nature"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Ayhan, B., Kwan, C., Budavari, B., Kwan, L., Lu, Y., Perez, D., Li, J., Skarlatos, D., and Vlachos, M. (2020). Vegetation detection using deep learning and conventional methods. Remote Sens., 12.","DOI":"10.3390\/rs12152502"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Xu, Y., Wu, L., Xie, Z., and Chen, Z. (2018). Building extraction in very high resolution remote sensing imagery using deep learning and guided filters. Remote Sens., 10.","DOI":"10.3390\/rs10010144"},{"key":"ref_11","unstructured":"S\u00f8nderby, C.K., Espeholt, L., Heek, J., Dehghani, M., Oliver, A., Salimans, T., Agrawal, S., Hickey, J., and Kalchbrenner, N. (2020). Metnet: A neural weather model for precipitation forecasting. arXiv."},{"key":"ref_12","unstructured":"Xingjian, S.H.I., Chen, Z., and Wang, H. (2015, January 7\u201312). Convolutional LSTM network: A machine learning approach for precipitation nowcasting. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_13","unstructured":"Shi, X., Gao, Z., Lausen, L., Wang, H., Yeung, D.Y., Wong, W.K., and Woo, W.C. (2017). Deep learning for precipitation nowcasting: A benchmark and a new model. arXiv."},{"key":"ref_14","unstructured":"Wang, Y., Long, M., Wang, J., Gao, Z., and Yu, P.S. (2017, January 4). Predrnn: Recurrent neural networks for predictive learning using spatiotemporal lstms. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Wang, Y., Zhang, J., Zhu, H., Long, M., Wang, J., and Yu, P.S. (2019, January 15\u201320). Memory in memory: A predictive neural network for learning higher-order non-stationarity from spatiotemporal dynamics. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00937"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"e2020MS002109","DOI":"10.1029\/2020MS002109","article-title":"Improving data-driven global weather prediction using deep convolutional neural networks on a cubed sphere","volume":"2","author":"Weyn","year":"2020","journal-title":"J. Adv. Model. Earth Syst."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"927","DOI":"10.1175\/JTECH-D-19-0146.1","article-title":"A deep learning network for cloud-to-ground lightning nowcasting with multisource data","volume":"37","author":"Zhou","year":"2020","journal-title":"J. Atmos. Ocean. Technol."},{"key":"ref_19","unstructured":"Veillette, M., Samsi, S., and Mattioli, C. (2020, January 6\u201312). SEVIR: A Storm Event Imagery Dataset for Deep Learning Applications in Radar and Satellite Meteorology. Proceedings of the 34th Conference on Neural Information Processing Systems (NeurIPS 2020), Vancouver, BC, Canada."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Tran, Q.K., and Song, S. (2019). Computer vision in precipitation nowcasting: Applying image quality assessment metrics for training deep neural networks. Atmosphere, 10.","DOI":"10.3390\/atmos10050244"},{"key":"ref_21","unstructured":"Mathieu, M., Couprie, C., and LeCun, Y. (2015). Deep multi-scale video prediction beyond mean square error. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Talebi, H., and Milanfar, P. (2018, January 4\u20136). Learned perceptual image enhancement. Proceedings of the 2018 IEEE international conference on computational photography (ICCP), Pittsburgh, PA, USA.","DOI":"10.1109\/ICCPHOT.2018.8368474"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Wu, H., Yao, Z., Wang, J., and Long, M. (2021, January 19\u201325). MotionRNN: A flexible model for video prediction with spacetime-varying motions. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.01518"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"e2021GL095302","DOI":"10.1029\/2021GL095302","article-title":"Improving Nowcasting of Convective Development by Incorporating Polarimetric Radar Variables into a Deep Learning Model","volume":"48","author":"Pan","year":"2021","journal-title":"Geophys. Res. Lett."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"e2020EA001621","DOI":"10.1029\/2020EA001621","article-title":"Application of a Radar Echo Extrapolation-Based Deep Learning Method in Strong Convection Nowcasting","volume":"8","author":"Yin","year":"2021","journal-title":"Earth Space Sci."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1109\/TIP.2003.819861","article-title":"Image quality assessment: From error visibility to structural similarity","volume":"13","author":"Wang","year":"2004","journal-title":"IEEE Trans. Image Process."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1398","DOI":"10.1109\/ACSSC.2003.1292216","article-title":"Multiscale structural similarity for image quality assessment","volume":"Volume 2","author":"Wang","year":"2003","journal-title":"Proceedings of the Thrity-Seventh Asilomar Conference on Signals, Systems & Computers"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Jing, J., Li, Q., and Peng, X. (2019). MLC-LSTM: Exploiting the spatiotemporal correlation between multi-level weather radar echoes for echo sequence extrapolation. Sensors, 19.","DOI":"10.3390\/s19183988"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"601","DOI":"10.1109\/LGRS.2019.2926776","article-title":"A generative adversarial gated recurrent unit model for precipitation nowcasting","volume":"17","author":"Tian","year":"2019","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Ravuri, S., Lenc, K., and Willson, M. (2021). Skillful Precipitation Nowcasting using Deep Generative Models of Radar. arXiv.","DOI":"10.1038\/s41586-021-03854-z"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Zhang, R., Isola, P., Efros, A.A., Shechtman, E., and Wang, O. (2018, January 18\u201322). The unreasonable effectiveness of deep features as a perceptual metric. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00068"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Chen, Y., Li, G., and Jin, C. (2020). SSD-GAN: Measuring the Realness in the Spatial and Spectral Domains. arXiv.","DOI":"10.1609\/aaai.v35i2.16196"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Gatys, L.A., Ecker, A.S., and Bethge, M. (2016, January 27\u201330). Image style transfer using convolutional neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.265"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Johnson, J., Alahi, A., and Fei-Fei, L. (2016). Perceptual losses for real-time style transfer and super-resolution. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-46475-6_43"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1109\/TAU.1967.1161901","article-title":"The use of fast Fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms","volume":"15","author":"Welch","year":"1967","journal-title":"IEEE Trans. Audio Electroacoust."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"2378","DOI":"10.1109\/TIP.2011.2109730","article-title":"FSIM: A feature similarity index for image quality assessment","volume":"20","author":"Zhang","year":"2011","journal-title":"IEEE Trans. Image Process."},{"key":"ref_37","unstructured":"Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., and Keutzer, K. (2016). SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size. arXiv."},{"key":"ref_38","first-page":"1097","article-title":"Imagenet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky","year":"2012","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_39","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Gupta, A., Johnson, J., Alahi, A., and Fei-Fei, L. (2017, January 22\u201329). Characterizing and improving stability in neural style transfer. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.438"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Lai, W.S., Huang, J.B., Wang, O., Shechtman, E., Yumer, E., and Yang, M.H. (2018, January 8\u201314). Learning blind video temporal consistency. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01267-0_11"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Mallya, A., Wang, T.C., Sapra, K., and Liu, M.Y. (2020, January 23\u201328). World-consistent video-to-video synthesis. Proceedings of the Computer Vision\u2014ECCV 2020: 16th European Conference, Glasgow, UK.","DOI":"10.1007\/978-3-030-58598-3_22"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/1\/24\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:51:03Z","timestamp":1760169063000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/1\/24"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,22]]},"references-count":42,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,1]]}},"alternative-id":["rs14010024"],"URL":"https:\/\/doi.org\/10.3390\/rs14010024","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,12,22]]}}}