{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,4]],"date-time":"2026-07-04T03:54:35Z","timestamp":1783137275502,"version":"3.54.6"},"reference-count":69,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2019,2,15]],"date-time":"2019-02-15T00:00:00Z","timestamp":1550188800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["SES-1132031"],"award-info":[{"award-number":["SES-1132031"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Recurrent neural networks (RNNs) are nonlinear dynamical models commonly used in the machine learning and dynamical systems literature to represent complex dynamical or sequential relationships between variables. Recently, as deep learning models have become more common, RNNs have been used to forecast increasingly complicated systems. Dynamical spatio-temporal processes represent a class of complex systems that can potentially benefit from these types of models. Although the RNN literature is expansive and highly developed, uncertainty quantification is often ignored. Even when considered, the uncertainty is generally quantified without the use of a rigorous framework, such as a fully Bayesian setting. Here we attempt to quantify uncertainty in a more formal framework while maintaining the forecast accuracy that makes these models appealing, by presenting a Bayesian RNN model for nonlinear spatio-temporal forecasting. Additionally, we make simple modifications to the basic RNN to help accommodate the unique nature of nonlinear spatio-temporal data. The proposed model is applied to a Lorenz simulation and two real-world nonlinear spatio-temporal forecasting applications.<\/jats:p>","DOI":"10.3390\/e21020184","type":"journal-article","created":{"date-parts":[[2019,2,25]],"date-time":"2019-02-25T03:30:08Z","timestamp":1551065408000},"page":"184","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":90,"title":["Bayesian Recurrent Neural Network Models for Forecasting and Quantifying Uncertainty in Spatial-Temporal Data"],"prefix":"10.3390","volume":"21","author":[{"given":"Patrick L.","family":"McDermott","sequence":"first","affiliation":[{"name":"Jupiter Intelligence, Boulder, CO 80302, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christopher K.","family":"Wikle","sequence":"additional","affiliation":[{"name":"Department of Statistics, University of Missouri, Columbia, MO 65211, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2019,2,15]]},"reference":[{"key":"ref_1","unstructured":"Fan, J., and Yao, Q. (2005). Nonlinear Time Series, Springer."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Billings, S.A. (2013). Nonlinear System Identification: NARMAX Methods in the Time, Frequency, and Spatio-Temporal Domains, John Wiley & Sons.","DOI":"10.1002\/9781118535561"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1002\/wics.1341","article-title":"Modern perspectives on statistics for spatio-temporal data","volume":"7","author":"Wikle","year":"2015","journal-title":"Wiley Interdiscip. Rev. Comput. Stat."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"3953","DOI":"10.1175\/1520-0442(2001)013<3953:LLPOPS>2.0.CO;2","article-title":"Long-lead prediction of Pacific SSTs via Bayesian dynamic modeling","volume":"13","author":"Berliner","year":"2000","journal-title":"J. Clim."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"335","DOI":"10.1007\/s13253-013-0141-2","article-title":"Hierarchical Bayesian spatio-temporal Conway\u2013Maxwell Poisson models with dynamic dispersion","volume":"18","author":"Wu","year":"2013","journal-title":"J. Agric. Biol. Environ. Stat."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"236","DOI":"10.1198\/jasa.2009.tm09036","article-title":"Statistical agent-based models for discrete spatio-temporal systems","volume":"105","author":"Hooten","year":"2010","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1007\/s11749-010-0209-z","article-title":"A general science-based framework for dynamical spatio-temporal models","volume":"19","author":"Wikle","year":"2010","journal-title":"Test"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1002\/env.2374","article-title":"A model-based approach for analog spatio-temporal dynamic forecasting","volume":"27","author":"McDermott","year":"2016","journal-title":"Environmetrics"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"e2456","DOI":"10.1002\/env.2456","article-title":"Sparsity in nonlinear dynamic spatiotemporal models using implied advection","volume":"28","author":"Richardson","year":"2017","journal-title":"Environmetrics"},{"key":"ref_10","unstructured":"Cressie, N., and Wikle, C. (2011). Statistics for Spatio-Temporal Data, John Wiley & Sons."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1175\/1520-0442(2000)013<0287:SCBNNA>2.0.CO;2","article-title":"Skill comparisons between neural networks and canonical correlation analysis in predicting the equatorial Pacific sea surface temperatures","volume":"13","author":"Tang","year":"2000","journal-title":"J. Clim."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Dixon, M.F., Polson, N.G., and Sokolov, V.O. (arXiv, 2017). Deep Learning for Spatio-Temporal Modeling: Dynamic Traffic Flows and High Frequency Trading, arXiv.","DOI":"10.1002\/asmb.2399"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"855","DOI":"10.1109\/TPAMI.2008.137","article-title":"A novel connectionist system for unconstrained handwriting recognition","volume":"31","author":"Graves","year":"2009","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Ning, G., Zhang, Z., Huang, C., He, Z., Ren, X., and Wang, H. (arXiv, 2016). Spatially supervised recurrent convolutional neural networks for visual object tracking, arXiv.","DOI":"10.1109\/ISCAS.2017.8050867"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Yildiz, I.B., von Kriegstein, K., and Kiebel, S.J. (2013). From birdsong to human speech recognition: Bayesian inference on a hierarchy of nonlinear dynamical systems. PLoS Comput. Biol., 9.","DOI":"10.1371\/journal.pcbi.1003219"},{"key":"ref_17","unstructured":"Graves, A. (arXiv, 2013). Generating sequences with recurrent neural networks, arXiv."},{"key":"ref_18","unstructured":"Jaeger, H. (2001). The \u201cEcho State\u201d Approach to Analysing and Training Recurrent Neural Networks-with an Erratum Note, German National Research Center for Information Technology GMD Technical Report."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1016\/j.cosrev.2009.03.005","article-title":"Reservoir computing approaches to recurrent neural network training","volume":"3","author":"Jaeger","year":"2009","journal-title":"Comput. Sci. Rev."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1002\/sta4.160","article-title":"An Ensemble Quadratic Echo State Network for Nonlinear Spatio-Temporal Forecasting","volume":"6","author":"McDermott","year":"2017","journal-title":"STAT"},{"key":"ref_21","unstructured":"Van der Westhuizen, J., and Lasenby, J. (arXiv, 2017). Bayesian LSTMs in medicine, arXiv."},{"key":"ref_22","unstructured":"Neal, R.M. (1994). Bayesian Learning for Neural Networks. [Ph.D. Thesis, University of Toronto]."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Chatzis, S.P. (2015). Sparse Bayesian Recurrent Neural Networks. Machine Learning and Knowledge Discovery in Databases, Springer.","DOI":"10.1007\/978-3-319-23525-7_22"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1109\/TNNLS.2015.2499302","article-title":"Bayesian recurrent neural network for language modeling","volume":"27","author":"Chien","year":"2016","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Gan, Z., Li, C., Chen, C., Pu, Y., Su, Q., and Carin, L. (arXiv, 2016). Scalable Bayesian Learning of Recurrent Neural Networks for Language Modeling, arXiv.","DOI":"10.18653\/v1\/P17-1030"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1264","DOI":"10.1080\/01621459.1999.10473879","article-title":"Parameter expansion for data augmentation","volume":"94","author":"Liu","year":"1999","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"532","DOI":"10.1214\/009053607000000569","article-title":"A theoretical comparison of the data augmentation, marginal augmentation and PX-DA algorithms","volume":"36","author":"Hobert","year":"2008","journal-title":"Ann. Stat."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Hobert, J.P. (2011). The data augmentation algorithm: Theory and methodology. Handbook of Markov Chain Monte Carlo, Chapman & Hall\/CRC.","DOI":"10.1201\/b10905-11"},{"key":"ref_29","unstructured":"Chung, J., Gulcehre, C., Cho, K., and Bengio, Y. (2015, January 6\u201311). Gated feedback recurrent neural networks. Proceedings of the International Conference on Machine Learning, Lille, France."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"366","DOI":"10.1007\/BFb0091924","article-title":"Detecting strange attractors in turbulence","volume":"898","author":"Takens","year":"1981","journal-title":"Lect. Notes Math."},{"key":"ref_31","first-page":"1929","article-title":"Dropout: A simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1275","DOI":"10.1214\/17-BA1082","article-title":"Deep Learning: A Bayesian Perspective","volume":"12","author":"Polson","year":"2017","journal-title":"Bayesian Anal."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"448","DOI":"10.1162\/neco.1992.4.3.448","article-title":"A practical Bayesian framework for backpropagation networks","volume":"4","author":"MacKay","year":"1992","journal-title":"Neural Comput."},{"key":"ref_34","first-page":"85","article-title":"A review of Bayesian variable selection methods: What, how and which","volume":"4","year":"2009","journal-title":"Bayesian Anal."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"881","DOI":"10.1080\/01621459.1993.10476353","article-title":"Variable selection via Gibbs sampling","volume":"88","author":"George","year":"1993","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_36","first-page":"339","article-title":"Approaches for Bayesian variable selection","volume":"7","author":"George","year":"1997","journal-title":"Stat. Sin."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"601","DOI":"10.1198\/016214504000000665","article-title":"Hierarchical Bayesian neural networks: An application to a prostate cancer study","volume":"99","author":"Ghosh","year":"2004","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"681","DOI":"10.1198\/016214508000000337","article-title":"The bayesian lasso","volume":"103","author":"Park","year":"2008","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"465","DOI":"10.1093\/biomet\/asq017","article-title":"The horseshoe estimator for sparse signals","volume":"97","author":"Carvalho","year":"2010","journal-title":"Biometrika"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"431","DOI":"10.1080\/01621459.2016.1260469","article-title":"The spike-and-slab lasso","volume":"113","author":"George","year":"2018","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Belkin, M., and Niyogi, P. (2001, January 3\u20138). Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering. Proceedings of the 14th International Conference on Neural Information Processing Systems: Natural and Synthetic (NIPS\u201901), Vancouver, BC, Canada.","DOI":"10.7551\/mitpress\/1120.003.0080"},{"key":"ref_42","unstructured":"Nair, V., and Hinton, G.E. (2010, January 21\u201324). Rectified linear units improve restricted boltzmann machines. Proceedings of the 27th International Conference on Machine Learning (ICML-10), Haifa, Israel."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1016\/j.acha.2006.04.006","article-title":"Diffusion maps","volume":"21","author":"Coifman","year":"2006","journal-title":"Appl. Comput. Harmon. Anal."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"1087","DOI":"10.1287\/mnsc.22.10.1087","article-title":"Scoring rules for continuous probability distributions","volume":"10","author":"Matheson","year":"1976","journal-title":"Manag. Sci."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1146\/annurev-statistics-062713-085831","article-title":"Probabilistic forecasting","volume":"1","author":"Gneiting","year":"2014","journal-title":"Annu. Rev. Stat. Appl."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1705","DOI":"10.1175\/1520-0469(2003)060<1705:SSFSMR>2.0.CO;2","article-title":"Systematic strategies for stochastic mode reduction in climate","volume":"60","author":"Majda","year":"2003","journal-title":"J. Atmos. Sci."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"4404","DOI":"10.1175\/JCLI3544.1","article-title":"Multilevel regression modeling of nonlinear processes: Derivation and applications to climatic variability","volume":"18","author":"Kravtsov","year":"2005","journal-title":"J. Clim."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"711","DOI":"10.1093\/biomet\/82.4.711","article-title":"Reversible jump Markov chain Monte Carlo computation and Bayesian model determination","volume":"82","author":"Green","year":"1995","journal-title":"Biometrika"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Luko\u0161evi\u010dius, M. (2012). A practical guide to applying echo state networks. Neural Networks: Tricks of the Trade, Springer.","DOI":"10.1007\/978-3-642-35289-8_36"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1175\/1520-0469(1963)020<0130:DNF>2.0.CO;2","article-title":"Deterministic nonperiodic flow","volume":"20","author":"Lorenz","year":"1963","journal-title":"J. Atmos. Sci."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Ma, Q.L., Zheng, Q.L., Peng, H., Zhong, T.W., and Xu, L.Q. (2007, January 19\u201322). Chaotic time series prediction based on evolving recurrent neural networks. Proceedings of the 2007 International Conference on Machine Learning and Cybernetics, Hong Kong, China.","DOI":"10.1109\/ICMLC.2007.4370752"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"116","DOI":"10.1016\/j.neucom.2012.01.014","article-title":"Cooperative coevolution of Elman recurrent neural networks for chaotic time series prediction","volume":"86","author":"Chandra","year":"2012","journal-title":"Neurocomputing"},{"key":"ref_53","unstructured":"Lorenz, E.N. (1995, January 4\u20138). Predictability: A problem partly solved. Proceedings of the Seminar on Predictability, Reading, UK."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1256\/qj.04.03","article-title":"Effects of stochastic parametrizations in the Lorenz\u201996 system","volume":"131","author":"Wilks","year":"2005","journal-title":"Quart. J. R. Meteorol. Soc."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"9804","DOI":"10.1073\/pnas.1512080112","article-title":"Discrete approach to stochastic parametrization and dimension reduction in nonlinear dynamics","volume":"112","author":"Chorin","year":"2015","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"601","DOI":"10.5194\/npg-22-601-2015","article-title":"A framework for variational data assimilation with superparameterization","volume":"22","author":"Grooms","year":"2015","journal-title":"Nonlinear Processes Geophys."},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Hu, S., and Fedorov, A.V. (2017). The extreme El Ni\u00f1o of 2015\u20132016: The role of westerly and easterly wind bursts, and preconditioning by the failed 2014 event. Clim. Dyn., 1\u201319.","DOI":"10.1007\/s00382-017-3531-2"},{"key":"ref_58","first-page":"1363","article-title":"Observing and predicting the 2015-16 El Ni\u00f1o","volume":"98","author":"Takahashi","year":"2017","journal-title":"Bull. Am. Meteorol. Soc."},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"631","DOI":"10.1175\/BAMS-D-11-00111.1","article-title":"Skill of real-time seasonal ENSO model predictions during 2002\u20132011: Is our capability increasing?","volume":"93","author":"Barnston","year":"2012","journal-title":"Bull. Am. Meteorol. Soc."},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1175\/1520-0477(1999)080<0217:PSOSAD>2.0.CO;2","article-title":"Predictive skill of statistical and dynamical climate models in SST forecasts during the 1997\u20131998 El Ni\u00f1o episode and the 1998 La Ni\u00f1a onset","volume":"80","author":"Barnston","year":"1999","journal-title":"Bull. Am. Meteorol. Soc."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"3240","DOI":"10.1175\/JCLI3420.1","article-title":"Did the ECMWF seasonal forecast model outperform statistical ENSO forecast models over the last 15 years?","volume":"18","author":"Balmaseda","year":"2005","journal-title":"J. Clim."},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1175\/1520-0442(1998)011<0029:FEEANN>2.0.CO;2","article-title":"Forecasting ENSO events: A neural network\u2013extended EOF approach","volume":"11","author":"Tangang","year":"1998","journal-title":"J. Clim."},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1002\/env.2266","article-title":"Physically motivated scale interaction parameterization in reduced rank quadratic nonlinear dynamic spatio-temporal models","volume":"25","author":"Gladish","year":"2014","journal-title":"Environmetrics"},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1007\/s11222-005-4786-8","article-title":"Bayesian neural networks for nonlinear time series forecasting","volume":"15","author":"Liang","year":"2005","journal-title":"Stat. Comput."},{"key":"ref_65","doi-asserted-by":"crossref","unstructured":"Sharma, S., and Singh, S. (2016, January 14\u201315). Unemployment rates forecasting using supervised neural networks. Proceedings of the 2016 6th International Conference Cloud System and Big Data Engineering (Confluence), Noida, India.","DOI":"10.1109\/CONFLUENCE.2016.7508042"},{"key":"ref_66","doi-asserted-by":"crossref","first-page":"755","DOI":"10.1016\/j.ijforecast.2005.04.010","article-title":"Linear models, smooth transition autoregressions, and neural networks for forecasting macroeconomic time series: A re-examination","volume":"21","author":"Medeiros","year":"2005","journal-title":"Int. J. Forecast."},{"key":"ref_67","unstructured":"Jones, N.A., and Smith, A.S. (2001). The Two or More Races Population, 2000."},{"key":"ref_68","unstructured":"Welling, M., and Teh, Y.W. (July, January 28). Bayesian learning via stochastic gradient Langevin dynamics. Proceedings of the 28th International Conference on Machine Learning (ICML-11), Bellevue, WA, USA."},{"key":"ref_69","doi-asserted-by":"crossref","first-page":"472","DOI":"10.1080\/01621459.2015.1117471","article-title":"Bayesian spatial change of support for count-valued survey data with application to the american community survey","volume":"111","author":"Bradley","year":"2016","journal-title":"J. Am. Stat. Assoc."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/21\/2\/184\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:32:31Z","timestamp":1760185951000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/21\/2\/184"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,2,15]]},"references-count":69,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2019,2]]}},"alternative-id":["e21020184"],"URL":"https:\/\/doi.org\/10.3390\/e21020184","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,2,15]]}}}