{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,4]],"date-time":"2026-06-04T14:21:19Z","timestamp":1780582879268,"version":"3.54.1"},"reference-count":54,"publisher":"MDPI AG","issue":"21","license":[{"start":{"date-parts":[[2021,10,25]],"date-time":"2021-10-25T00:00:00Z","timestamp":1635120000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2018YFC1506905"],"award-info":[{"award-number":["2018YFC1506905"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"name":"the Key R&amp;D Program of Jiangsu Province","award":["BE2019052, BE2017076"],"award-info":[{"award-number":["BE2019052, BE2017076"]}]},{"name":"the Key R&amp;D industrialization Program of Suzhou","award":["SGC201733, SGC201854"],"award-info":[{"award-number":["SGC201733, SGC201854"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Precipitation nowcasting by radar echo extrapolation using machine learning algorithms is a field worthy of further study, since rainfall prediction is essential in work and life. Current methods of predicting the radar echo images need further improvement in prediction accuracy as well as in presenting the predicted details of the radar echo images. In this paper, we propose a two-stage spatiotemporal context refinement network (2S-STRef) to predict future pixel-level radar echo maps (deterministic output) more accurately and with more distinct details. The first stage is an efficient and concise spatiotemporal prediction network, which uses the spatiotemporal RNN module embedded in an encoder and decoder structure to give a first-stage prediction. The second stage is a proposed detail refinement net, which can preserve the high-frequency detailed feature of the radar echo images by using the multi-scale feature extraction and fusion residual block. We used a real-world radar echo map dataset of South China to evaluate the proposed 2S-STRef model. The experiments showed that compared with the PredRNN++ and ConvLSTM method, our 2S-STRef model performs better on the precipitation nowcasting, as well as at the image quality evaluating index and the forecasting indices. At a given 45 dBZ echo threshold (heavy precipitation) and with a 2 h lead time, the widely used CSI, HSS, and SSIM indices of the proposed 2S-STRef model are found equal to 0.195, 0.312, and 0.665, respectively. In this case, the proposed model outperforms the OpticalFlow method and PredRNN++ model.<\/jats:p>","DOI":"10.3390\/rs13214285","type":"journal-article","created":{"date-parts":[[2021,10,25]],"date-time":"2021-10-25T21:42:05Z","timestamp":1635198125000},"page":"4285","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["Two-Stage Spatiotemporal Context Refinement Network for Precipitation Nowcasting"],"prefix":"10.3390","volume":"13","author":[{"given":"Dan","family":"Niu","sequence":"first","affiliation":[{"name":"School of Automation, Southeast University, Nanjing 210096, China"},{"name":"Key Laboratory of Measurement and Control of CSE, Ministry of Education Research Laboratory, Nanjing 210096, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Junhao","family":"Huang","sequence":"additional","affiliation":[{"name":"School of Automation, Southeast University, Nanjing 210096, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zengliang","family":"Zang","sequence":"additional","affiliation":[{"name":"Institute of Meteorology and Oceanography, PLA University of Science and Technology, Nanjing 211101, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Liujia","family":"Xu","sequence":"additional","affiliation":[{"name":"School of Automation, Southeast University, Nanjing 210096, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hongshu","family":"Che","sequence":"additional","affiliation":[{"name":"School of Automation, Southeast University, Nanjing 210096, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuanqing","family":"Tang","sequence":"additional","affiliation":[{"name":"School of Automation, Southeast University, Nanjing 210096, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,10,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"248","DOI":"10.1126\/science.1115255","article-title":"Weather forecasting with ensemble methods","volume":"310","author":"Gneiting","year":"2005","journal-title":"Science"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1038\/548379a","article-title":"Machine learning tapped to improve climate forecasts","volume":"548","author":"Jones","year":"2017","journal-title":"Nature"},{"key":"ref_3","unstructured":"Schmid, F., Wang, Y., and Harou, A. (2017). Nowcasting guidelines\u2014A summary. WMO-No. 1198, World Meteorological Organization."},{"key":"ref_4","unstructured":"Shi, X., Gao, Z., Lausen, L., Wang, H., Yeung, D.Y., Wong, W.K., and Woo, W.C. (2017, January 4\u20139). Deep learning for precipitation nowcasting: A benchmark and a new model. Proceedings of the 30th International Conference on Neural Information Processing Systems (NeurIPS), Long Beach, CA, USA."},{"key":"ref_5","unstructured":"Bromberg, C.L., Gazen, C., Hickey, J.J., Burge, J., Barrington, L., and Agrawal, S. (2019, January 8\u201314). Machine learning for precipitation nowcasting from radar images. Proceedings of the Machine Learning and the Physical Sciences Workshop at the 33rd Conference on Neural Information Processing Systems (NeurIPS), Vancouver, BC, Canada."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Qiu, M., Zhao, P., Zhang, K., Huang, J., Shi, X., Wang, X., and Chu, W. (2017, January 18\u201321). A short-term rainfall prediction model using multi-task convolutional neural networks. Proceedings of the IEEE International Conference on Data Mining, New Orleans, LA, USA.","DOI":"10.1109\/ICDM.2017.49"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Bouget, V., B\u00e9r\u00e9ziat, D., Brajard, J., Charantonis, A., and Filoche, A. (2021). Fusion of rain radar images and wind forecasts in a deep learning model applied to rain nowcasting. Remote Sens., 13.","DOI":"10.3390\/rs13020246"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"568","DOI":"10.1038\/s41586-019-1559-7","article-title":"Deep learning for multi-year ENSO forecasts","volume":"573","author":"Ham","year":"2019","journal-title":"Nature"},{"key":"ref_9","unstructured":"Shi, X., Chen, Z., Wang, H., Yeung, D.Y., Wong, W.K., and Woo, W.C. (2015, January 7\u201312). Convolutional LSTM network: A machine learning approach for precipitation nowcasting. Proceedings of the 28th International Conference on Neural Information Processing Systems (NeurIPS), Montreal, QC, Canada."},{"key":"ref_10","unstructured":"Marchuk, G. (2012). Numerical Methods in Weather Prediction, Elsevier."},{"key":"ref_11","first-page":"285","article-title":"Some current problems in numerical weather prediction","volume":"41","author":"Tolstykh","year":"2005","journal-title":"Izv. Atmos. Ocean. Phys."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"409","DOI":"10.1175\/BAMS-D-11-00263.1","article-title":"Use of NWP for nowcasting convective precipitation: Recent progress and challenges","volume":"95","author":"Juanzhen","year":"2014","journal-title":"Bull. Am. Meteorol. Soc."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"250","DOI":"10.1109\/TGE.1979.294654","article-title":"Automatic cell detection and tracking","volume":"17","author":"Crane","year":"1979","journal-title":"IEEE Trans. Geosci. Electron."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1038\/273287a0","article-title":"Three-dimensional storm motion detection by conventional weather radar","volume":"273","author":"Rinehart","year":"1978","journal-title":"Nature"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1016\/j.jhydrol.2003.11.011","article-title":"Development of a precipitation nowcasting algorithm based upon optical flow techniques","volume":"288","author":"Bowler","year":"2004","journal-title":"J. Hydrol."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1007\/s13143-010-1008-x","article-title":"McGill algorithm for precipitation nowcasting by lagrangian extrapolation (MAPLE) applied to the South Korean radar network","volume":"46","author":"Bellon","year":"2010","journal-title":"Asia-Pac. J. Atmos. Sci."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Woo, W.-C., and Wong, W.-K. (2017). Operational application of optical flow techniques to radar-based rainfall nowcasting. Atmosphere, 8.","DOI":"10.3390\/atmos8030048"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2859","DOI":"10.1175\/1520-0493(2002)130<2859:SDOTPO>2.0.CO;2","article-title":"Scale-dependence of the predictability of precipitation from continental radar images. Part I: Description of the methodology","volume":"130","author":"Germann","year":"2002","journal-title":"Mon. Weather Rev."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1175\/1520-0450(2004)043<0074:SDOTPO>2.0.CO;2","article-title":"Scale dependence of the predictability of precipitation from continental radar images. Part II: Probability forecasts","volume":"43","author":"Germann","year":"2004","journal-title":"J. Appl. Meteorol."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1099","DOI":"10.1175\/MWR-D-19-0193.1","article-title":"Improving radar echo Lagrangian extrapolation nowcasting by blending numerical model wind information: Statistical performance of 16 typhoon cases","volume":"148","author":"Chung","year":"2020","journal-title":"Mon. Weather Rev."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1175\/1520-0450(2003)042<0381:ADASSA>2.0.CO;2","article-title":"A dynamic and spatial scaling approach to advection forecasting","volume":"42","author":"Seed","year":"2003","journal-title":"J. Appl. Meteorol."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"124140","DOI":"10.1016\/j.jhydrol.2019.124140","article-title":"Improved rainfall nowcasting using Burgers\u2019 equation","volume":"581","author":"Ryu","year":"2020","journal-title":"J. Hydrol."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"4185","DOI":"10.5194\/gmd-12-4185-2019","article-title":"Pysteps: An open-source Python library for probabilistic precipitation nowcasting (v1. 0)","volume":"12","author":"Pulkkinen","year":"2019","journal-title":"Geosci. Model Dev."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"601","DOI":"10.1109\/LGRS.2019.2926776","article-title":"A generative adversarial gated recurrent unit model for precipitation nowcasting","volume":"17","author":"Tian","year":"2019","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Hern\u00e1ndez, E., Sanchez-Anguix, V., Julian, V., Palanca, J., and Duque, N. (2016, January 18\u201320). Rainfall prediction: A deep learning approach. Proceedings of the International Conference on Hybrid Artificial Intelligence Systems, Seville, Spain.","DOI":"10.1007\/978-3-319-32034-2_13"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1016\/j.energy.2012.01.006","article-title":"Numerical weather prediction (NWP) and hybrid ARMA\/ANN model to predict global radiation","volume":"39","author":"Cyril","year":"2012","journal-title":"Energy"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"2073","DOI":"10.1175\/BAMS-D-16-0123.1","article-title":"Using artificial intelligence to improve real-time decision-making for high-impact weather","volume":"98","author":"McGovern","year":"2017","journal-title":"Bull. Am. Meteorol. Soc."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Wang, B., Lu, J., Yan, Z., Luo, H., Li, T., Zheng, Y., and Zhang, G. (2019, January 4\u20138). Deep uncertainty quantification: A machine learning approach for weather forecasting. Proceedings of the International Conference on Knowledge Discovery and data Mining (SIGKDD) 2019, Anchorage, AK, USA.","DOI":"10.1145\/3292500.3330704"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2498","DOI":"10.12928\/telkomnika.v18i5.14665","article-title":"Prediction of rainfall using improved deep learning with particle swarm optimization","volume":"18","year":"2020","journal-title":"Telkomnika"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"158296","DOI":"10.1109\/ACCESS.2019.2950328","article-title":"Attention-based dual-source spatiotemporal neural network for lightning forecast","volume":"7","author":"Lin","year":"2019","journal-title":"IEEE Access"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Basha, C.Z., Bhavana, N., Bhavya, P., and Sowmya, V. (2020, January 2\u20134). Rainfall prediction using machine learning & deep learning techniques. Proceedings of the International Conference on Electronics and Sustainable Communication Systems (ICESC), Coimbatore, India.","DOI":"10.1109\/ICESC48915.2020.9155896"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1109\/MCI.2009.932254","article-title":"Time series prediction using support vector machines: A survey","volume":"4","author":"Sapankevych","year":"2009","journal-title":"IEEE Comput. Intell. Mag."},{"key":"ref_33","unstructured":"Sutskever, I., Vinyals, O., and Le, Q.V. (2014, January 8\u201313). Sequence to sequence learning with neural networks. Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS), Montreal, QC, Canada."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1016\/j.procs.2018.08.153","article-title":"Single layer & multi-layer long short-term memory (LSTM) model with intermediate variables for weather forecasting","volume":"135","author":"Salman","year":"2018","journal-title":"Procedia Comput. Sci."},{"key":"ref_35","unstructured":"Benjamin, K., Lior, W., and Yehuda, A. (2015, January 7\u201312). A dynamic convolutional layer for short range weather prediction. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA."},{"key":"ref_36","unstructured":"Wang, Y., Long, M., Wang, J., Gao, Z., and Yu, P.S. (2017, January 4\u20139). PredRNN: Recurrent neural networks for predictive learning using spatiotemporal lstms. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_37","unstructured":"Wang, Y., Gao, Z., Long, M., Wang, J., and Philip, S.Y. (2018, January 14\u201316). PredRNN++: Towards a resolution of the deep-in-time dilemma in spatiotemporal predictive learning. Proceedings of the International Conference on Machine Learning (PMLR), Beijing, China."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Geng, Y., Li, Q., Lin, T., Jiang, L., Xu, L., Zheng, D., Yao, W., Lyu, W., and Zhang, Y. (2019, January 4\u20138). Lightnet: A dual spatiotemporal encoder network model for lightning prediction. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA.","DOI":"10.1145\/3292500.3330717"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Zhang, F., Wang, X., Guan, J., Wu, M., and Guo, L. (2021). RN-Net: A deep learning approach to 0\u20132 h rainfall nowcasting based on radar and automatic weather station data. Sensors, 21.","DOI":"10.3390\/s21061981"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Xu, Z., Du, J., Wang, J., Jiang, C., and Ren, Y. (2019, January 20\u201324). Satellite image prediction relying on GAN and LSTM neural networks. Proceedings of the IEEE International Conference on Communications (ICC), Shanghai, China.","DOI":"10.1109\/ICC.2019.8761462"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Ravuri, S., Lenc, K., Willson, M., Kangin, D., Lam, R., Mirowski, P., and Mohamed, S. (2021). Skillful Precipitation nowcasting using deep generative models of radar. arXiv.","DOI":"10.1038\/s41586-021-03854-z"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s13143-019-00127-8","article-title":"Weather forecasting using ensemble of spatial-temporal attention network and multi-layer perceptron","volume":"2020","author":"Li","year":"2020","journal-title":"Asia-Pac. J. Atmos. Sci."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"52774","DOI":"10.1109\/ACCESS.2020.2980977","article-title":"Hybrid deep learning approach for multi-step-ahead daily rainfall prediction using GCM simulations","volume":"8","author":"Khan","year":"2020","journal-title":"IEEE Access"},{"key":"ref_44","unstructured":"Eddy, I., Nikolaus, M., Tonmoy, S., Margret, K., Alexey, D., and Thomas, B. (2017, January 21\u201326). Flownet 2.0: Evolution of optical flow estimation with deep networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"93179","DOI":"10.1109\/ACCESS.2020.2995187","article-title":"MPL-GAN: Toward realistic meteorological predictive learning using conditional GAN","volume":"8","author":"Liu","year":"2020","journal-title":"IEEE Access"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"20200097","DOI":"10.1098\/rsta.2020.0097","article-title":"Can deep learning beat numerical weather prediction?","volume":"379","author":"Schultz","year":"2021","journal-title":"Philos. Trans. R. Soc. A"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"6624","DOI":"10.1002\/wrcr.20536","article-title":"Formulation and evaluation of a scale decomposition-based stochastic precipitation nowcast scheme","volume":"49","author":"Seed","year":"2013","journal-title":"Water Resour. Res."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"334","DOI":"10.1016\/j.neucom.2019.10.076","article-title":"Multi-scale feature fusion residual network for single image super-resolution","volume":"379","author":"Qin","year":"2020","journal-title":"Neurocomputing"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1016\/j.neucom.2019.09.093","article-title":"Deep fractal residual network for fast and accurate single image super resolution","volume":"398","author":"Zhou","year":"2020","journal-title":"Neurocomputing"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Hu, J., Shen, L., and Sun, G. (2018, January 18\u201322). Squeeze-and-excitation networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00745"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1109\/TIP.2003.819861","article-title":"Image quality assessment: From error visibility to structural similarity","volume":"13","author":"Wang","year":"2004","journal-title":"IEEE Trans. Image Process."},{"key":"ref_52","unstructured":"Diederik, K., and Jimmy, B. (2015). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"11","DOI":"10.3390\/atmos8010011","article-title":"Ensemble classification for anomalous propagation echo detection with clustering-based subset-selection method","volume":"8","author":"Hansoo","year":"2017","journal-title":"Atmosphere"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Song, K., Yang, G., Wang, Q., Xu, C., Liu, J., Liu, W., and Zhang, W. (2019, January 8\u201311). Deep learning prediction of incoming rainfalls: An operational service for the city of Beijing, China. Proceedings of the 2019 International Conference on Data Mining Workshops (ICDMW), Beijing, China.","DOI":"10.1109\/ICDMW.2019.00036"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/21\/4285\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:23:01Z","timestamp":1760167381000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/21\/4285"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,25]]},"references-count":54,"journal-issue":{"issue":"21","published-online":{"date-parts":[[2021,11]]}},"alternative-id":["rs13214285"],"URL":"https:\/\/doi.org\/10.3390\/rs13214285","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,25]]}}}