{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,21]],"date-time":"2026-01-21T04:23:31Z","timestamp":1768969411997,"version":"3.49.0"},"reference-count":37,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2022,12,26]],"date-time":"2022-12-26T00:00:00Z","timestamp":1672012800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Key R&amp;D Program of China","award":["2022YFE0106400"],"award-info":[{"award-number":["2022YFE0106400"]}]},{"name":"National Key R&amp;D Program of China","award":["2022QNLM010203-3"],"award-info":[{"award-number":["2022QNLM010203-3"]}]},{"name":"National Key R&amp;D Program of China","award":["41830964"],"award-info":[{"award-number":["41830964"]}]},{"name":"National Key R&amp;D Program of China","award":["ts201712017"],"award-info":[{"award-number":["ts201712017"]}]},{"name":"Special funds of Shandong Province for Qingdao Marine Science and technology National Laboratory","award":["2022YFE0106400"],"award-info":[{"award-number":["2022YFE0106400"]}]},{"name":"Special funds of Shandong Province for Qingdao Marine Science and technology National Laboratory","award":["2022QNLM010203-3"],"award-info":[{"award-number":["2022QNLM010203-3"]}]},{"name":"Special funds of Shandong Province for Qingdao Marine Science and technology National Laboratory","award":["41830964"],"award-info":[{"award-number":["41830964"]}]},{"name":"Special funds of Shandong Province for Qingdao Marine Science and technology National Laboratory","award":["ts201712017"],"award-info":[{"award-number":["ts201712017"]}]},{"name":"National Natural Science Foundation of China","award":["2022YFE0106400"],"award-info":[{"award-number":["2022YFE0106400"]}]},{"name":"National Natural Science Foundation of China","award":["2022QNLM010203-3"],"award-info":[{"award-number":["2022QNLM010203-3"]}]},{"name":"National Natural Science Foundation of China","award":["41830964"],"award-info":[{"award-number":["41830964"]}]},{"name":"National Natural Science Foundation of China","award":["ts201712017"],"award-info":[{"award-number":["ts201712017"]}]},{"name":"Shandong Province\u2019s \u201cTaishan\u201d Scientist Program","award":["2022YFE0106400"],"award-info":[{"award-number":["2022YFE0106400"]}]},{"name":"Shandong Province\u2019s \u201cTaishan\u201d Scientist Program","award":["2022QNLM010203-3"],"award-info":[{"award-number":["2022QNLM010203-3"]}]},{"name":"Shandong Province\u2019s \u201cTaishan\u201d Scientist Program","award":["41830964"],"award-info":[{"award-number":["41830964"]}]},{"name":"Shandong Province\u2019s \u201cTaishan\u201d Scientist Program","award":["ts201712017"],"award-info":[{"award-number":["ts201712017"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Convolution-based recurrent neural networks and convolutional neural networks have been used extensively in spatiotemporal prediction. However, these methods tend to concentrate on fixed-scale spatiotemporal state transitions and disregard the complexity of spatiotemporal motion. Through statistical analysis, we found that the distribution of the spatiotemporal sequence and the variety of spatiotemporal motion state transitions exhibit some regularity. In light of these statistics and observations, we propose the Multi-scale Spatiotemporal Neural Network (MSSTNet), an end-to-end neural network based on 3D convolution. It can be separated into three major child modules: a distribution feature extraction module, a multi-scale motion state capture module, and a feature decoding module. Furthermore, the MSST unit is designed to model multi-scale spatial and temporal information in the multi-scale motion state capture module. We first conduct the experiments on the MovingMNIST dataset, which is the most commonly used dataset in the field of spatiotemporal prediction, MSSTNet can achieve state-of-the-art results for this dataset, and ablation experiments demonstrate that the MSST unit has positive significance for spatiotemporal prediction. In addition, this paper applies the model to valuable precipitation nowcasting, due to efficiently capturing the multi-scale information of distribution and motion, the new MSSTNet model can predict the real-world radar echo more accurately.<\/jats:p>","DOI":"10.3390\/rs15010137","type":"journal-article","created":{"date-parts":[[2022,12,27]],"date-time":"2022-12-27T02:53:11Z","timestamp":1672109591000},"page":"137","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["MSSTNet: A Multi-Scale Spatiotemporal Prediction Neural Network for Precipitation Nowcasting"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9524-0071","authenticated-orcid":false,"given":"Yuankang","family":"Ye","sequence":"first","affiliation":[{"name":"College of Intelligent Systems Science and Engineering, Harbin Engineering University, Harbin 150001, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8417-8428","authenticated-orcid":false,"given":"Feng","family":"Gao","sequence":"additional","affiliation":[{"name":"College of Intelligent Systems Science and Engineering, Harbin Engineering University, Harbin 150001, China"},{"name":"Qingdao Innovation and Development Center, Harbin Engineering University, Qingdao 266400, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wei","family":"Cheng","sequence":"additional","affiliation":[{"name":"Beijing Institute of Applied Meteorology, Beijing 100029, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chang","family":"Liu","sequence":"additional","affiliation":[{"name":"College of Intelligent Systems Science and Engineering, Harbin Engineering University, Harbin 150001, China"},{"name":"Qingdao Hatran Ocean Intelligence Technology Co., Ltd., Qingdao 266400, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6569-9842","authenticated-orcid":false,"given":"Shaoqing","family":"Zhang","sequence":"additional","affiliation":[{"name":"College of Intelligent Systems Science and Engineering, Harbin Engineering University, Harbin 150001, China"},{"name":"Key Laboratory of Physical Oceanography, MOE, Institute for Advanced Ocean Study, Frontiers Science Center for Deep Ocean Multispheres and Earth System (DOMES), Ocean University of China, Qingdao 266100, China"},{"name":"The College of Ocean and Atmosphere, Ocean University of China, Qingdao 266100, China"},{"name":"Ocean Dynamics and Climate Function Lab\/Pilot National Laboratory for Marine Science and Technology (QNLM), Qingdao 266237, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,12,26]]},"reference":[{"key":"ref_1","unstructured":"Shi, X., and Yeung, D.Y. (2018). Machine learning for spatiotemporal sequence forecasting: A survey. arXiv."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Sun, N., Zhou, Z., Li, Q., and Jing, J. (2022). Three-Dimensional Gridded Radar Echo Extrapolation for Convective Storm Nowcasting Based on 3D-ConvLSTM Model. Remote Sens., 14.","DOI":"10.3390\/rs14174256"},{"key":"ref_3","first-page":"22009","article-title":"Sevir: A storm event imagery dataset for deep learning applications in radar and satellite meteorology","volume":"33","author":"Veillette","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"672","DOI":"10.1038\/s41586-021-03854-z","article-title":"Skilful precipitation nowcasting using deep generative models of radar","volume":"597","author":"Ravuri","year":"2021","journal-title":"Nature"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Sun, N., Zhou, Z., Li, Q., and Zhou, X. (2022). Spatiotemporal Prediction of Monthly Sea Subsurface Temperature Fields Using a 3D U-Net-Based Model. Remote Sens., 14.","DOI":"10.3390\/rs14194890"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Mao, K., Gao, F., Zhang, S., and Liu, C. (2022). An Information Spatial-Temporal Extension Algorithm for Shipborne Predictions Based on Deep Neural Networks with Remote Sensing Observations\u2014Part I: Ocean Temperature. Remote Sens., 14.","DOI":"10.3390\/rs14081791"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Hou, S., Li, W., Liu, T., Zhou, S., Guan, J., Qin, R., and Wang, Z. (2022). MIMO: A Unified Spatio-Temporal Model for Multi-Scale Sea Surface Temperature Prediction. Remote Sens., 14.","DOI":"10.3390\/rs14102371"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"568","DOI":"10.1038\/s41586-019-1559-7","article-title":"Deep learning for multi-year ENSO forecasts","volume":"573","author":"Ham","year":"2019","journal-title":"Nature"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_10","first-page":"1","article-title":"Convolutional LSTM network: A machine learning approach for precipitation nowcasting","volume":"28","author":"Shi","year":"2015","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_11","first-page":"1","article-title":"Deep learning for precipitation nowcasting: A benchmark and a new model","volume":"30","author":"Shi","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_12","unstructured":"Wang, Y., Gao, Z., Long, M., Wang, J., and Philip, S.Y. (2018, January 10\u201315). Predrnn++: Towards a resolution of the deep-in-time dilemma in spatiotemporal predictive learning. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_13","first-page":"1","article-title":"Convective precipitation nowcasting using U-Net Model","volume":"60","author":"Han","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_14","unstructured":"Agrawal, S., Barrington, L., Bromberg, C., Burge, J., Gazen, C., and Hickey, J. (2019). Machine learning for precipitation nowcasting from radar images. arXiv."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Wu, H., Yao, Z., Wang, J., and Long, M. (2021, January 20\u201325). MotionRNN: A flexible model for video prediction with spacetime-varying motions. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.01518"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Qiu, Z., Yao, T., and Mei, T. (2017, January 22\u201329). Learning spatio-temporal representation with pseudo-3d residual networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.590"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Xie, S., Sun, C., Huang, J., Tu, Z., and Murphy, K. (2018, January 8\u201314). Rethinking spatiotemporal feature learning: Speed-accuracy trade-offs in video classification. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01267-0_19"},{"key":"ref_19","first-page":"1","article-title":"Predrnn: Recurrent neural networks for predictive learning using spatiotemporal lstms","volume":"30","author":"Wang","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Wang, Y., Zhang, J., Zhu, H., Long, M., Wang, J., and Yu, P.S. (2019, January 16\u201317). Memory in memory: A predictive neural network for learning higher-order non-stationarity from spatiotemporal dynamics. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00937"},{"key":"ref_21","unstructured":"Wang, Y., Jiang, L., Yang, M.H., Li, L.J., Long, M., and Fei-Fei, L. (May, January 30). Eidetic 3D LSTM: A model for video prediction and beyond. Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Tran, D., Bourdev, L., Fergus, R., Torresani, L., and Paluri, M. (2015, January 7\u201313). Learning spatiotemporal features with 3d convolutional networks. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.510"},{"key":"ref_23","unstructured":"Lin, Z., Li, M., Zheng, Z., Cheng, Y., and Yuan, C. (2020, January 7\u201312). Self-attention convlstm for spatiotemporal prediction. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA."},{"key":"ref_24","unstructured":"Yu, W., Lu, Y., Easterbrook, S., and Fidler, S. (2019, January 6\u20139). Efficient and Information-Preserving Future Frame Prediction and Beyond. Proceedings of the International Conference on Learning Representations, New Orleans, LA, USA."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Wang, Y., Wu, H., Zhang, J., Gao, Z., Wang, J., Yu, P., and Long, M. (2022). Predrnn: A recurrent neural network for spatiotemporal predictive learning. IEEE Trans. Pattern Anal. Mach. Intell.","DOI":"10.1109\/TPAMI.2022.3165153"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N., and Liang, J. (2018). Unet++: A nested u-net architecture for medical image segmentation. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Springer.","DOI":"10.1007\/978-3-030-00889-5_1"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Huang, H., Lin, L., Tong, R., Hu, H., Zhang, Q., Iwamoto, Y., Han, X., Chen, Y.W., and Wu, J. (2020, January 4\u20138). Unet 3+: A full-scale connected unet for medical image segmentation. Proceedings of the ICASSP 2020\u20142020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.","DOI":"10.1109\/ICASSP40776.2020.9053405"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Song, K., Yang, G., Wang, Q., Xu, C., Liu, J., Liu, W., Shi, C., Wang, Y., Zhang, G., and Yu, X. (2019, January 8\u201311). Deep learning prediction of incoming rainfalls: An operational service for the city of Beijing China. Proceedings of the 2019 International Conference on Data Mining Workshops (ICDMW), Beijing, China.","DOI":"10.1109\/ICDMW.2019.00036"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"178","DOI":"10.1016\/j.patrec.2021.01.036","article-title":"SmaAt-UNet: Precipitation nowcasting using a small attention-UNet architecture","volume":"145","author":"Trebing","year":"2021","journal-title":"Pattern Recognit. Lett."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Gao, Z., Tan, C., Wu, L., and Li, S.Z. (2022, January 18\u201322). SimVP: Simpler Yet Better Video Prediction. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52688.2022.00317"},{"key":"ref_31","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1109\/TIP.2003.819861","article-title":"Image quality assessment: From error visibility to structural similarity","volume":"13","author":"Wang","year":"2004","journal-title":"IEEE Trans. Image Process."},{"key":"ref_33","unstructured":"Guen, V.L., and Thome, N. (2020, January 13\u201319). Disentangling physical dynamics from unknown factors for unsupervised video prediction. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1177","DOI":"10.1002\/qj.49711247414","article-title":"Analysis methods for numerical weather prediction","volume":"112","author":"Lorenc","year":"1986","journal-title":"Q. J. R. Meteorol. Soc."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"106548","DOI":"10.1016\/j.atmosres.2022.106548","article-title":"Nowcasting extreme rain and extreme wind speed with machine learning techniques applied to different input datasets","volume":"282","author":"Chkeir","year":"2022","journal-title":"Atmos. Res."},{"key":"ref_36","first-page":"8","article-title":"A convolutional recurrent neural network for strong convective rainfall nowcasting using weather radar data in Southeastern Brazil","volume":"3","author":"Caseri","year":"2022","journal-title":"Artif. Intell. Geosci."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"e2021GL095302","DOI":"10.1029\/2021GL095302","article-title":"Improving Nowcasting of Convective Development by Incorporating Polarimetric Radar Variables Into a Deep-Learning Model","volume":"48","author":"Pan","year":"2021","journal-title":"Geophys. Res. Lett."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/1\/137\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:52:05Z","timestamp":1760147525000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/1\/137"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,26]]},"references-count":37,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,1]]}},"alternative-id":["rs15010137"],"URL":"https:\/\/doi.org\/10.3390\/rs15010137","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,26]]}}}