{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,5]],"date-time":"2026-07-05T04:31:50Z","timestamp":1783225910244,"version":"3.54.6"},"reference-count":133,"publisher":"Springer Science and Business Media LLC","issue":"10","license":[{"start":{"date-parts":[[2023,3,15]],"date-time":"2023-03-15T00:00:00Z","timestamp":1678838400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,3,15]],"date-time":"2023-03-15T00:00:00Z","timestamp":1678838400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Artif Intell Rev"],"published-print":{"date-parts":[[2023,10]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The last decade witnessed a growing interest in Bayesian learning. Yet, the technicality of the topic and the multitude of ingredients involved therein, besides the complexity of turning theory into practical implementations, limit the use of the Bayesian learning paradigm, preventing its widespread adoption across different fields and applications. This self-contained survey engages and introduces readers to the principles and algorithms of Bayesian Learning for Neural Networks. It provides an introduction to the topic from an accessible, practical-algorithmic perspective. Upon providing a general introduction to Bayesian Neural Networks, we discuss and present both standard and recent approaches for Bayesian inference, with an emphasis on solutions relying on Variational Inference and the use of Natural gradients. We also discuss the use of manifold optimization as a state-of-the-art approach to Bayesian learning. We examine the characteristic properties of all the discussed methods, and provide pseudo-codes for their implementation, paying attention to practical aspects, such as the computation of the gradients.<\/jats:p>","DOI":"10.1007\/s10462-023-10443-1","type":"journal-article","created":{"date-parts":[[2023,3,15]],"date-time":"2023-03-15T17:03:06Z","timestamp":1678899786000},"page":"11773-11823","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":135,"title":["Bayesian learning for neural networks: an algorithmic survey"],"prefix":"10.1007","volume":"56","author":[{"given":"Martin","family":"Magris","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alexandros","family":"Iosifidis","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,3,15]]},"reference":[{"key":"10443_CR1","volume-title":"Manifolds, tensor analysis, and applications","author":"R Abraham","year":"2012","unstructured":"Abraham R, Marsden JE, Ratiu T (2012) Manifolds, tensor analysis, and applications, vol 75. Springer, New York"},{"key":"10443_CR2","volume-title":"Optimization algorithms on matrix manifolds","author":"PA Absil","year":"2009","unstructured":"Absil PA, Mahony R, Sepulchre R (2009) Optimization algorithms on matrix manifolds. Princeton University Press, Princeton"},{"key":"10443_CR3","unstructured":"Alquier P (2021) User-friendly introduction to PAC-Bayes bounds. Preprint. arXiv:2110.11216"},{"issue":"2","key":"10443_CR4","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1162\/089976698300017746","volume":"10","author":"SI Amari","year":"1998","unstructured":"Amari SI (1998) Natural gradient works efficiently in learning. Neural Comput 10(2):251\u2013276","journal-title":"Neural Comput"},{"key":"10443_CR5","unstructured":"B\u00e9cigneul G, Ganea OE (2018) Riemannian adaptive optimization methods. Preprint. arXiv:1810.00760"},{"issue":"9","key":"10443_CR6","doi-asserted-by":"crossref","first-page":"090502","DOI":"10.1103\/PhysRevLett.114.090502","volume":"114","author":"DW Berry","year":"2015","unstructured":"Berry DW, Childs AM, Cleve R et al (2015) Simulating Hamiltonian dynamics with a truncated Taylor series. Phys Rev Lett 114(9):090502","journal-title":"Phys Rev Lett"},{"issue":"1","key":"10443_CR7","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1214\/aoms\/1177730497","volume":"18","author":"D Blackwell","year":"1947","unstructured":"Blackwell D (1947) Conditional expectation and unbiased sequential estimation. Ann Math Stat 18(1):105\u2013110","journal-title":"Ann Math Stat"},{"issue":"1","key":"10443_CR8","first-page":"17","volume":"1","author":"DM Blei","year":"2007","unstructured":"Blei DM, Lafferty JD (2007) A correlated topic model of science. Ann Appl Stat 1(1):17\u201335","journal-title":"Ann Appl Stat"},{"issue":"518","key":"10443_CR9","doi-asserted-by":"crossref","first-page":"859","DOI":"10.1080\/01621459.2017.1285773","volume":"112","author":"DM Blei","year":"2017","unstructured":"Blei DM, Kucukelbir A, McAuliffe JD (2017) Variational inference: a review for statisticians. J Am Stat Assoc 112(518):859\u2013877","journal-title":"J Am Stat Assoc"},{"key":"10443_CR10","unstructured":"Blundell C, Cornebise J, Kavukcuoglu K et al (2015) Weight uncertainty in neural network. In: International conference on machine learning, 2015, pp 1613\u20131622"},{"issue":"9","key":"10443_CR11","doi-asserted-by":"crossref","first-page":"2217","DOI":"10.1109\/TAC.2013.2254619","volume":"58","author":"S Bonnabel","year":"2013","unstructured":"Bonnabel S (2013) Stochastic gradient descent on Riemannian manifolds. IEEE Trans Autom Control 58(9):2217\u20132229","journal-title":"IEEE Trans Autom Control"},{"key":"10443_CR12","volume-title":"An introduction to differentiable manifolds and Riemannian geometry, Revised","author":"WM Boothby","year":"2003","unstructured":"Boothby WM, Boothby WM (2003) An introduction to differentiable manifolds and Riemannian geometry, Revised, vol 120. Gulf Professional Publishing, Houston"},{"issue":"42","key":"10443_CR13","first-page":"1455","volume":"15","author":"N Boumal","year":"2014","unstructured":"Boumal N, Mishra B, Absil PA et al (2014) Manopt, a MATLAB toolbox for optimization on manifolds. J Mach Learn Res 15(42):1455\u20131459","journal-title":"J Mach Learn Res"},{"issue":"489","key":"10443_CR14","doi-asserted-by":"crossref","first-page":"324","DOI":"10.1198\/jasa.2009.tm08030","volume":"105","author":"M Braun","year":"2010","unstructured":"Braun M, McAuliffe J (2010) Variational inference for large-scale models of discrete choice. J Am Stat Assoc 105(489):324\u2013335","journal-title":"J Am Stat Assoc"},{"key":"10443_CR15","volume-title":"Differentiable manifolds: an introduction","author":"F Brickell","year":"1970","unstructured":"Brickell F, Clark RS (1970) Differentiable manifolds: an introduction. Van Nostrand Reinhold, London"},{"key":"10443_CR16","first-page":"321","volume":"2","author":"DS Broomhead","year":"1988","unstructured":"Broomhead DS, Lowe D (1988) Multivariable functional interpolation and adaptive networks. Complex Syst 2:321\u2013355","journal-title":"Complex Syst"},{"issue":"2","key":"10443_CR17","doi-asserted-by":"crossref","first-page":"476","DOI":"10.1145\/3296957.3173212","volume":"53","author":"R Cai","year":"2018","unstructured":"Cai R, Ren A, Liu N et al (2018) VIBNN: hardware acceleration of Bayesian neural networks. SIGPLAN Not 53(2):476\u2013488","journal-title":"SIGPLAN Not"},{"issue":"2","key":"10443_CR18","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1080\/00031305.1985.10479400","volume":"39","author":"G Casella","year":"1985","unstructured":"Casella G (1985) An introduction to empirical Bayes data analysis. Am Stat 39(2):83\u201387","journal-title":"Am Stat"},{"key":"10443_CR19","volume-title":"Statistical inference","author":"G Casella","year":"2021","unstructured":"Casella G, Berger RL (2021) Statistical inference. Cengage Learning, Boston"},{"issue":"3","key":"10443_CR20","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1080\/00031305.1992.10475878","volume":"46","author":"G Casella","year":"1992","unstructured":"Casella G, George EI (1992) Explaining the Gibbs sampler. Am Stat 46(3):167\u2013174","journal-title":"Am Stat"},{"key":"10443_CR21","first-page":"241","volume-title":"Handbook of statistics","author":"S Chakraborty","year":"2012","unstructured":"Chakraborty S, Ghosh M (2012) Applications of Bayesian neural networks in prostate cancer study. Handbook of statistics, vol 28. Elsevier, Amsterdam, pp 241\u2013262"},{"key":"10443_CR22","unstructured":"Chan A, Alaa A, Qian Z et al (2020) Unlabelled data improves Bayesian uncertainty calibration under covariate shift. In: International conference on machine learning, 2020, pp 1392\u20131402"},{"key":"10443_CR23","doi-asserted-by":"crossref","unstructured":"Cho K, van Merrienboer B, Bahdanau D et al (2014) On the properties of neural machine translation: encoder\u2013decoder approaches. In: Workshop on syntax, semantics and structure in statistical translation, 2014","DOI":"10.3115\/v1\/W14-4012"},{"issue":"7","key":"10443_CR24","doi-asserted-by":"crossref","first-page":"410","DOI":"10.1016\/j.tree.2010.04.001","volume":"25","author":"K Csill\u00e9ry","year":"2010","unstructured":"Csill\u00e9ry K, Blum MG, Gaggiotti OE et al (2010) Approximate Bayesian computation (ABC) in practice. Trends Ecol Evol 25(7):410\u2013418","journal-title":"Trends Ecol Evol"},{"key":"10443_CR25","unstructured":"Damianou A, Lawrence ND (2013) Deep Gaussian processes. In: International conference on artificial intelligence and statistics, 2013, pp 207\u2013215"},{"issue":"2","key":"10443_CR26","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1016\/j.strusafe.2008.06.020","volume":"31","author":"A Der Kiureghian","year":"2009","unstructured":"Der Kiureghian A, Ditlevsen O (2009) Aleatory or epistemic? Does it matter? Struct Saf 31(2):105\u2013112","journal-title":"Struct Saf"},{"key":"10443_CR27","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-41068-1","volume-title":"Machine learning in finance","author":"MF Dixon","year":"2020","unstructured":"Dixon MF, Halperin I, Bilokon P (2020) Machine learning in finance, vol 1170. Springer, Berlin"},{"key":"10443_CR28","volume-title":"Differential geometry of curves and surfaces: revised and updated second edition","author":"MP Do Carmo","year":"2016","unstructured":"Do Carmo MP (2016) Differential geometry of curves and surfaces: revised and updated second edition. Courier Dover Publications, Mineola"},{"key":"10443_CR29","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4757-2201-7","volume-title":"Riemannian geometry","author":"MP Do Carmo","year":"1992","unstructured":"Do Carmo MP, Flaherty Francis J (1992) Riemannian geometry, vol 6. Springer, Berlin"},{"issue":"1","key":"10443_CR30","doi-asserted-by":"crossref","first-page":"350","DOI":"10.1109\/TSG.2020.3019263","volume":"12","author":"Y Du","year":"2020","unstructured":"Du Y, Liu Y, Wang X et al (2020) Predicting weather-related failure risk in distribution systems using Bayesian neural network. IEEE Trans Smart Grid 12(1):350\u2013360","journal-title":"IEEE Trans Smart Grid"},{"issue":"1","key":"10443_CR31","doi-asserted-by":"crossref","first-page":"219","DOI":"10.3758\/s13423-017-1317-5","volume":"25","author":"A Etz","year":"2018","unstructured":"Etz A, Gronau QF, Dablander F et al (2018) How to become a Bayesian in eight easy steps: an annotated reading list. Psychon Bull Rev 25(1):219\u2013234","journal-title":"Psychon Bull Rev"},{"key":"10443_CR32","doi-asserted-by":"crossref","unstructured":"Fichtner A, Gebraad L, Boehm C et al (2020) Auto-tuning Hamiltonian Monte Carlo. In: EGU General Assembly conference abstracts, p 7735","DOI":"10.5194\/egusphere-egu2020-7735"},{"issue":"4","key":"10443_CR33","doi-asserted-by":"crossref","first-page":"933","DOI":"10.1162\/089976600300015655","volume":"12","author":"JFGd Freitas","year":"2000","unstructured":"Freitas JFGd, Niranjan M, Gee AH (2000) Hierarchical Bayesian models for regularization in sequential learning. Neural Comput 12(4):933\u2013953","journal-title":"Neural Comput"},{"key":"10443_CR34","doi-asserted-by":"crossref","DOI":"10.1002\/9781119387596","volume-title":"Markov chains: from theory to implementation and experimentation","author":"PA Gagniuc","year":"2017","unstructured":"Gagniuc PA (2017) Markov chains: from theory to implementation and experimentation. Wiley, Hoboken"},{"key":"10443_CR35","unstructured":"Gal Y, Ghahramani Z (2015) Dropout as a Bayesian approximation: insights and applications. In: Deep learning workshop, international conference on machine learning, 2015, p 2"},{"key":"10443_CR36","unstructured":"Gal Y, Ghahramani Z (2016) Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: International conference on machine learning, 2016, pp 1050\u20131059"},{"key":"10443_CR37","doi-asserted-by":"crossref","DOI":"10.1201\/9781482296426","volume-title":"Markov chain Monte Carlo: stochastic simulation for Bayesian inference","author":"D Gamerman","year":"2006","unstructured":"Gamerman D, Lopes HF (2006) Markov chain Monte Carlo: stochastic simulation for Bayesian inference. CRC Press, Boca Raton"},{"key":"10443_CR38","doi-asserted-by":"crossref","DOI":"10.1201\/9780429258411","volume-title":"Bayesian data analysis","author":"A Gelman","year":"1995","unstructured":"Gelman A, Carlin JB, Stern HS et al (1995) Bayesian data analysis. Chapman and Hall\/CRC, New York"},{"issue":"6","key":"10443_CR39","doi-asserted-by":"crossref","first-page":"721","DOI":"10.1109\/TPAMI.1984.4767596","volume":"6","author":"S Geman","year":"1984","unstructured":"Geman S, Geman D (1984) Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans Pattern Anal Mach Intell 6(6):721\u2013741","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"10443_CR40","unstructured":"Ghahramani Z, Beal M (2000) Propagation algorithms for variational Bayesian learning. In: Advances in neural information processing systems, 2000, vol 13"},{"issue":"1","key":"10443_CR41","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1061\/(ASCE)1090-0241(2005)131:1(84)","volume":"131","author":"AT Goh","year":"2005","unstructured":"Goh AT, Kulhawy FH, Chua C (2005) Bayesian neural network analysis of undrained side resistance of drilled shafts. J Geotech Geoenviron Eng 131(1):84\u201393","journal-title":"J Geotech Geoenviron Eng"},{"key":"10443_CR42","volume-title":"Deep learning","author":"I Goodfellow","year":"2016","unstructured":"Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, Cambridge"},{"issue":"11","key":"10443_CR43","doi-asserted-by":"crossref","first-page":"1529","DOI":"10.3390\/e23111529","volume":"23","author":"B Guedj","year":"2021","unstructured":"Guedj B, Pujol L (2021) Still no free lunches: the price to pay for tighter PAC-Bayes bounds. Entropy 23(11):1529","journal-title":"Entropy"},{"issue":"1","key":"10443_CR44","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1093\/biomet\/57.1.97","volume":"57","author":"WK Hastings","year":"1970","unstructured":"Hastings WK (1970) Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57(1):97\u2013109","journal-title":"Biometrika"},{"key":"10443_CR45","volume-title":"Neural networks: a comprehensive foundation","author":"S Haykin","year":"1998","unstructured":"Haykin S (1998) Neural networks: a comprehensive foundation. Prentice Hall PTR, Hoboken"},{"key":"10443_CR46","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S et al (2015) Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. In: International conference on computer vision, 2015, pp 1026\u20131034","DOI":"10.1109\/ICCV.2015.123"},{"key":"10443_CR47","first-page":"33","volume-title":"A tutorial on learning with Bayesian networks","author":"D Heckerman","year":"2008","unstructured":"Heckerman D (2008) A tutorial on learning with Bayesian networks. Springer, Berlin, pp 33\u201382"},{"issue":"8","key":"10443_CR48","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","volume":"9","author":"S Hochreiter","year":"1997","unstructured":"Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735\u20131780","journal-title":"Neural Comput"},{"issue":"4","key":"10443_CR49","doi-asserted-by":"crossref","first-page":"382","DOI":"10.1214\/ss\/1009212519","volume":"14","author":"JA Hoeting","year":"1999","unstructured":"Hoeting JA, Madigan D, Raftery AE et al (1999) Bayesian model averaging: a tutorial. Stat Sci 14(4):382\u2013417","journal-title":"Stat Sci"},{"key":"10443_CR50","unstructured":"Hoffman MD, Sountsov P (2022) Tuning-free generalized Hamiltonian Monte Carlo. In: International conference on artificial intelligence and statistics, 2022, pp 7799\u20137813"},{"key":"10443_CR51","first-page":"1303","volume":"14","author":"MD Hoffman","year":"2013","unstructured":"Hoffman MD, Blei DM, Wang C et al (2013) Stochastic variational inference. J Mach Learn Res 14:1303\u20131347","journal-title":"J Mach Learn Res"},{"issue":"1","key":"10443_CR52","first-page":"1593","volume":"15","author":"MD Hoffman","year":"2014","unstructured":"Hoffman MD, Gelman A et al (2014) The no-u-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo. J Mach Learn Res 15(1):1593\u20131623","journal-title":"J Mach Learn Res"},{"key":"10443_CR53","unstructured":"Homma T, Atlas LE, Marks RJ (1987) An artificial neural network for spatio-temporal bipolar patters: application to phoneme classification. In: Advances in neural information processing systems, 1987, pp 31\u201340"},{"issue":"2","key":"10443_CR54","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1016\/0370-2693(91)90812-5","volume":"268","author":"AM Horowitz","year":"1991","unstructured":"Horowitz AM (1991) A generalized guided Monte Carlo algorithm. Phys Lett B 268(2):247\u2013252","journal-title":"Phys Lett B"},{"key":"10443_CR55","unstructured":"Hosseini R, Sra S (2015) Matrix manifold optimization for Gaussian mixtures. In: Advances in neural information processing systems, 2015, vol 28"},{"key":"10443_CR56","unstructured":"Hron J, Matthews A, Ghahramani Z (2018) Variational Bayesian dropout: pitfalls and fixes. In: International conference on machine learning, 2018, pp 2019\u20132028"},{"issue":"2","key":"10443_CR57","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1007\/s40305-020-00295-9","volume":"8","author":"J Hu","year":"2020","unstructured":"Hu J, Liu X, Wen ZW et al (2020) A brief introduction to manifold optimization. J Oper Res Soc China 8(2):199\u2013248","journal-title":"J Oper Res Soc China"},{"key":"10443_CR58","unstructured":"Jaakkola TS, Jordan MI (1997) A variational approach to Bayesian logistic regression models and their extensions. In: International workshop on artificial intelligence and statistics, 1997, pp 283\u2013294"},{"key":"10443_CR59","unstructured":"Jakkala K (2021) Deep Gaussian processes: a survey. Preprint. arXiv:2106.12135"},{"key":"10443_CR60","doi-asserted-by":"crossref","first-page":"5427","DOI":"10.1109\/ACCESS.2017.2779181","volume":"6","author":"H Jang","year":"2017","unstructured":"Jang H, Lee J (2017) An empirical study on modeling and prediction of bitcoin prices with Bayesian neural networks based on blockchain information. IEEE Access 6:5427\u20135437","journal-title":"IEEE Access"},{"key":"10443_CR61","first-page":"379","volume":"39","author":"B Jeuris","year":"2012","unstructured":"Jeuris B, Vandebril R, Vandereycken B (2012) A survey and comparison of contemporary algorithms for computing the matrix geometric mean. Electron Trans Numer Anal 39:379\u2013402","journal-title":"Electron Trans Numer Anal"},{"issue":"2","key":"10443_CR62","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1109\/MCI.2022.3155327","volume":"17","author":"LV Jospin","year":"2022","unstructured":"Jospin LV, Laga H, Boussaid F et al (2022) Hands-on Bayesian neural networks\u2014a tutorial for deep learning users. IEEE Comput Intell Mag 17(2):29\u201348","journal-title":"IEEE Comput Intell Mag"},{"key":"10443_CR63","unstructured":"Kasai H, Jawanpuria P, Mishra B (2019) Riemannian adaptive stochastic gradient algorithms on matrix manifolds. In: International conference on machine learning, 2019, pp 3262\u20133271"},{"key":"10443_CR66","doi-asserted-by":"crossref","unstructured":"Khan ME, Nielsen D (2018) Fast yet simple natural-gradient descent for variational inference in complex models. In: International symposium on information theory and its applications, 2018, pp 31\u201335","DOI":"10.23919\/ISITA.2018.8664326"},{"key":"10443_CR64","unstructured":"Khan M, Nielsen D, Tangkaratt V et al (2018a) Fast and scalable Bayesian deep learning by weight-perturbation in ADAM. In: International conference on machine learning, 2018, pp 2611\u20132620"},{"key":"10443_CR65","unstructured":"Khan M, Nielsen D, Tangkaratt V et al (2018b) Fast and scalable Bayesian deep learning by weight-perturbation in ADAM. In: International conference on machine learning, 2018, pp 2611\u20132620"},{"key":"10443_CR67","unstructured":"Kingma DP, Ba J (2014) ADAM: a method for stochastic optimization. arXiv preprint. arXiv:1412.6980"},{"key":"10443_CR68","unstructured":"Knowles D, Minka T (2011) Non-conjugate variational message passing for multinomial and binary regression. In: Advances in neural information processing systems, 2011, vol 24"},{"key":"10443_CR69","volume-title":"Foundations of differential geometry","author":"S Kobayashi","year":"1963","unstructured":"Kobayashi S, Nomizu K (1963) Foundations of differential geometry. Wiley, New York"},{"key":"10443_CR70","unstructured":"Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: Advances in neural information processing systems, 2012, vol 25"},{"issue":"1","key":"10443_CR71","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1214\/aoms\/1177729694","volume":"22","author":"S Kullback","year":"1951","unstructured":"Kullback S, Leibler RA (1951) On information and sufficiency. Ann Math Stat 22(1):79\u201386","journal-title":"Ann Math Stat"},{"issue":"106","key":"10443_CR72","first-page":"816","volume":"142","author":"Y Kwon","year":"2020","unstructured":"Kwon Y, Won JH, Kim BJ et al (2020) Uncertainty quantification using Bayesian neural networks in classification: application to biomedical image segmentation. Comput Stat Data Anal 142(106):816","journal-title":"Comput Stat Data Anal"},{"issue":"3","key":"10443_CR73","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1016\/S0893-6080(00)00098-8","volume":"14","author":"J Lampinen","year":"2001","unstructured":"Lampinen J, Vehtari A (2001) Bayesian approach for neural networks\u2014review and case studies. Neural Netw 14(3):257\u2013274","journal-title":"Neural Netw"},{"key":"10443_CR74","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.patrec.2014.01.008","volume":"42","author":"M L\u00e4ngkvist","year":"2014","unstructured":"L\u00e4ngkvist M, Karlsson L, Loutfi A (2014) A review of unsupervised feature learning and deep learning for time-series modeling. Pattern Recognit Lett 42:11\u201324","journal-title":"Pattern Recognit Lett"},{"key":"10443_CR75","series-title":"Cambridge monographs on applied and computational mathematics","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511614118","volume-title":"Simulating Hamiltonian dynamics","author":"B Leimkuhler","year":"2005","unstructured":"Leimkuhler B, Reich S (2005) Simulating Hamiltonian dynamics. Cambridge monographs on applied and computational mathematics. Cambridge University Press, Cambridge"},{"key":"10443_CR76","doi-asserted-by":"crossref","unstructured":"Lemieux C (2014) Control variates. Statistics Reference Online, Wiley StatsRef, pp 1\u20138","DOI":"10.1002\/9781118445112.stat07947"},{"issue":"1","key":"10443_CR77","doi-asserted-by":"crossref","first-page":"20","DOI":"10.2202\/1544-6115.1038","volume":"3","author":"Y Liang","year":"2004","unstructured":"Liang Y, Kelemen AG (2004) Hierarchical Bayesian neural network for gene expression temporal patterns. Stat Appl Genet Mol Biol 3(1):20","journal-title":"Stat Appl Genet Mol Biol"},{"key":"10443_CR78","unstructured":"Lin W, Schmidt M, Khan ME (2020) Handling the positive-definite constraint in the Bayesian learning rule. In: International conference on machine learning, 2020, pp 6116\u20136126"},{"issue":"1","key":"10443_CR79","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/S0933-3657(03)00033-2","volume":"28","author":"PJ Lisboa","year":"2003","unstructured":"Lisboa PJ, Wong H, Harris P et al (2003) A Bayesian neural network approach for modelling censored data with an application to prognosis after surgery for breast cancer. Artif Intell Med 28(1):1\u201325","journal-title":"Artif Intell Med"},{"key":"10443_CR80","doi-asserted-by":"crossref","DOI":"10.1007\/978-0-387-71265-9","volume-title":"Introduction to applied Bayesian statistics and estimation for social scientists","author":"SM Lynch","year":"2007","unstructured":"Lynch SM (2007) Introduction to applied Bayesian statistics and estimation for social scientists, vol 1. Springer, New York"},{"key":"10443_CR81","doi-asserted-by":"crossref","unstructured":"Lyu Y, Tsang IW (2021) Black-box optimizer with stochastic implicit natural gradient. In: Joint European conference on machine learning and knowledge discovery in databases, 2021. Springer, pp 217\u2013232","DOI":"10.1007\/978-3-030-86523-8_14"},{"issue":"04","key":"10443_CR82","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1142\/S0218213099000294","volume":"8","author":"Q Ma","year":"1999","unstructured":"Ma Q, Wang JT (1999) Biological data mining using Bayesian neural networks: a case study. Int J Artif Intell Tools 8(04):433\u2013451","journal-title":"Int J Artif Intell Tools"},{"key":"10443_CR83","unstructured":"Mackay DJC (1992) Bayesian methods for adaptive models. PhD Thesis, California Institute of Technology"},{"key":"10443_CR84","doi-asserted-by":"crossref","first-page":"469","DOI":"10.1088\/0954-898X_6_3_011","volume":"6","author":"DJC Mackay","year":"1995","unstructured":"Mackay DJC (1995) Probable networks and plausible predictions\u2014a review of practical Bayesian methods for supervised neural networks. Netw Comput Neural Syst 6:469\u2013505","journal-title":"Netw Comput Neural Syst"},{"key":"10443_CR85","doi-asserted-by":"crossref","unstructured":"Magris M, Shabani M, Iosifidis A (2022a) Bayesian bilinear neural network for predicting the mid-price dynamics in limit-order book markets. Preprint. arXiv:2203.03613","DOI":"10.1002\/for.2955"},{"key":"10443_CR86","unstructured":"Magris M, Shabani M, Iosifidis A (2022b) Exact manifold Gaussian variational Bayes. Preprint. arXiv:2210.14598"},{"key":"10443_CR87","unstructured":"Magris M, Shabani M, Iosifidis A (2022c) Quasi black-box variational inference with natural gradients for Bayesian learning. Preprint arXiv:2205.11568"},{"issue":"1","key":"10443_CR88","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1093\/biomet\/71.1.135","volume":"71","author":"KV Mardia","year":"1984","unstructured":"Mardia KV, Marshall RJ (1984) Maximum likelihood estimation of models for residual covariance in spatial regression. Biometrika 71(1):135\u2013146","journal-title":"Biometrika"},{"issue":"6","key":"10443_CR89","doi-asserted-by":"crossref","first-page":"1087","DOI":"10.1063\/1.1699114","volume":"21","author":"N Metropolis","year":"1953","unstructured":"Metropolis N, Rosenbluth AW, Rosenbluth MN et al (1953) Equation of state calculations by fast computing machines. J Chem Phys 21(6):1087\u20131092","journal-title":"J Chem Phys"},{"key":"10443_CR90","doi-asserted-by":"crossref","DOI":"10.1017\/9781139879354","volume-title":"Variational Bayesian learning theory","author":"S Nakajima","year":"2019","unstructured":"Nakajima S, Watanabe K, Sugiyama M (2019) Variational Bayesian learning theory. Cambridge University Press, Cambridge"},{"issue":"11","key":"10443_CR91","first-page":"2","volume":"2","author":"RM Neal","year":"2011","unstructured":"Neal RM et al (2011) MCMC using Hamiltonian dynamics. Handb Markov Chain Monte Carlo 2(11):2","journal-title":"Handb Markov Chain Monte Carlo"},{"key":"10443_CR92","unstructured":"Nielsen F (2020) A note on the natural gradient and its connections with the Riemannian gradient, the mirror descent, and the ordinary gradient"},{"key":"10443_CR93","unstructured":"Nielsen F, Garcia V (2009) Statistical exponential families: a digest with flash cards. Preprint. arXiv:0911.4863"},{"key":"10443_CR94","unstructured":"Oleksiienko I, Iosifidis A (2022) Layer ensembles. arXiv preprint. arXiv:2210.04882"},{"key":"10443_CR95","unstructured":"Oleksiienko I, Tran DT, Iosifidis A (2022) Variational neural networks. arXiv preprint. arXiv:2207.01524"},{"issue":"3","key":"10443_CR96","doi-asserted-by":"crossref","first-page":"786","DOI":"10.1162\/neco.2008.08-07-592","volume":"21","author":"M Opper","year":"2009","unstructured":"Opper M, Archambeau C (2009) The variational Gaussian approximation revisited. Neural Comput 21(3):786\u2013792","journal-title":"Neural Comput"},{"key":"10443_CR97","doi-asserted-by":"crossref","unstructured":"Opper M, Winther O (1999) A Bayesian approach to on-line learning. In: On-line learning in neural networks. Cambridge University Press, Cambridge, pp 363\u2013378","DOI":"10.1017\/CBO9780511569920.017"},{"key":"10443_CR98","unstructured":"Osawa K, Swaroop S, Khan MEE et al (2019) Practical deep learning with Bayesian principles. In: Advances in neural information processing systems, 2019, vol 32"},{"key":"10443_CR99","unstructured":"Osband I, Aslanides J, Cassirer A (2018) Randomized prior functions for deep reinforcement learning. In: Advances in neural information processing systems, 2018, vol 31"},{"key":"10443_CR100","unstructured":"Paisley J, Blei DM, Jordan MI (2012) Variational Bayesian inference with stochastic search. In: International conference on international conference on machine learning, 2012, pp 1363\u20131370"},{"issue":"6","key":"10443_CR101","doi-asserted-by":"crossref","first-page":"774","DOI":"10.1109\/TETCI.2018.2872598","volume":"4","author":"N Passalis","year":"2020","unstructured":"Passalis N, Tefas A, Kanniainen J et al (2020) Temporal bag-of-features learning for predicting mid price movements using high frequency limit order book data. IEEE Trans Emerg Top Comput Intell 4(6):774\u2013785","journal-title":"IEEE Trans Emerg Top Comput Intell"},{"key":"10443_CR102","unstructured":"Ranganath R, Gerrish S, Blei D (2014) Black box variational inference. In: International conference on artificial intelligence and statistics, 2014, pp 814\u2013822"},{"key":"10443_CR103","first-page":"81","volume":"38","author":"CR Rao","year":"1945","unstructured":"Rao CR (1945) Information and the accuracy attainable in the estimation of statistical parameters. Bull Calcutta Math Soc 38:81\u201391","journal-title":"Bull Calcutta Math Soc"},{"key":"10443_CR104","unstructured":"Rezende DJ, Mohamed S, Wierstra D (2014) Stochastic backpropagation and approximate inference in deep generative models. In: International conference on machine learning, 2014, pp 1278\u20131286"},{"issue":"3","key":"10443_CR105","doi-asserted-by":"crossref","first-page":"400","DOI":"10.1214\/aoms\/1177729586","volume":"22","author":"H Robbins","year":"1951","unstructured":"Robbins H, Monro S (1951) A stochastic approximation method. Ann Math Stat 22(3):400\u2013407","journal-title":"Ann Math Stat"},{"key":"10443_CR106","unstructured":"Robert CP, Roberts GO (2021) Rao\u2013Blackwellization in the MCMC era. arXiv preprint. arXiv:2101.01011"},{"key":"10443_CR107","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4757-3071-5","volume-title":"Monte Carlo statistical methods","author":"CP Robert","year":"1999","unstructured":"Robert CP, Casella G, Casella G (1999) Monte Carlo statistical methods, vol 2. Springer, Berlin"},{"key":"10443_CR108","doi-asserted-by":"crossref","first-page":"533","DOI":"10.1038\/323533a0","volume":"323","author":"DE Rumelhart","year":"1986","unstructured":"Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by back-propagating errors. Nature 323:533\u2013536","journal-title":"Nature"},{"issue":"4","key":"10443_CR109","doi-asserted-by":"crossref","first-page":"837","DOI":"10.1214\/13-BA858","volume":"8","author":"T Salimans","year":"2013","unstructured":"Salimans T, Knowles DA (2013) Fixed-form variational posterior approximation through stochastic linear regression. Bayesian Anal 8(4):837\u2013882","journal-title":"Bayesian Anal"},{"issue":"2","key":"10443_CR110","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1080\/14697688.2019.1633014","volume":"20","author":"E Sariev","year":"2020","unstructured":"Sariev E, Germano G (2020) Bayesian regularized artificial neural networks for the estimation of the probability of default. Quant Finance 20(2):311\u2013328","journal-title":"Quant Finance"},{"key":"10443_CR111","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1613\/jair.251","volume":"4","author":"LK Saul","year":"1996","unstructured":"Saul LK, Jaakkola T, Jordan MI (1996) Mean field theory for sigmoid belief networks. J Artif Intell Res 4:61\u201376","journal-title":"J Artif Intell Res"},{"key":"10443_CR112","unstructured":"Serafino L (2013) No free lunch theorem and Bayesian probability theory: two sides of the same coin. Some implications for black-box optimization and metaheuristics. Preprint. arXiv:1311.6041"},{"issue":"9","key":"10443_CR113","doi-asserted-by":"crossref","first-page":"3926","DOI":"10.1109\/TNNLS.2017.2750679","volume":"29","author":"X Shen","year":"2018","unstructured":"Shen X, Tian X, Liu T et al (2018) Continuous dropout. IEEE Trans Neural Netw Learn Syst 29(9):3926\u20133937","journal-title":"IEEE Trans Neural Netw Learn Syst"},{"key":"10443_CR114","unstructured":"Skopek O, Ganea OE, B\u00e9cigneul G (2019) Mixed-curvature variational autoencoders. Preprint. http:\/\/arxiv.org\/abs\/1911.08411arXiv:1911.08411"},{"issue":"1","key":"10443_CR115","doi-asserted-by":"crossref","first-page":"713","DOI":"10.1137\/140978168","volume":"25","author":"S Sra","year":"2015","unstructured":"Sra S, Hosseini R (2015) Conic geometric optimization on the manifold of positive definite matrices. SIAM J Optim 25(1):713\u2013739","journal-title":"SIAM J Optim"},{"issue":"1","key":"10443_CR116","first-page":"1929","volume":"15","author":"N Srivastava","year":"2014","unstructured":"Srivastava N, Hinton G, Krizhevsky A et al (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929\u20131958","journal-title":"J Mach Learn Res"},{"key":"10443_CR117","unstructured":"Tan LS (2021) Analytic natural gradient updates for Cholesky factor in Gaussian variational approximation. Preprint. arXiv:2109.00375"},{"key":"10443_CR118","unstructured":"Tieleman T, Hinton G (2012) Lecture 6.5-RMSProp, COURSERA: neural networks for machine learning. University of Toronto, Technical Report 6"},{"issue":"5","key":"10443_CR119","doi-asserted-by":"crossref","first-page":"1407","DOI":"10.1109\/TNNLS.2018.2869225","volume":"30","author":"DT Tran","year":"2019","unstructured":"Tran DT, Iosifidis A, Kanniainen J et al (2019) Temporal attention-augmented bilinear network for financial time-series data analysis. IEEE Trans Neural Netw Learn Syst 30(5):1407\u20131418","journal-title":"IEEE Trans Neural Netw Learn Syst"},{"issue":"6","key":"10443_CR120","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1007\/s11222-021-10047-1","volume":"31","author":"MN Tran","year":"2021","unstructured":"Tran MN, Nguyen DH, Nguyen D (2021a) Variational Bayes on manifolds. Stat Comput 31(6):71","journal-title":"Stat Comput"},{"key":"10443_CR121","unstructured":"Tran MN, Nguyen TN, Dao VH (2021b) A practical tutorial on variational Bayes. arXiv preprint. arXiv:2103.01327"},{"key":"10443_CR122","doi-asserted-by":"crossref","unstructured":"Trusheim F, Condurache A, Mertins A (2018) Boosting black-box variational inference by incorporating the natural gradient. In: International conference on pattern recognition, 2018, pp 19\u201324","DOI":"10.1109\/ICPR.2018.8546002"},{"key":"10443_CR123","doi-asserted-by":"crossref","unstructured":"Tu LW (2011) Manifolds. In: An introduction to manifolds. Springer, Berlin, pp 47\u201383","DOI":"10.1007\/978-1-4419-7400-6_3"},{"key":"10443_CR124","doi-asserted-by":"crossref","unstructured":"Vehtari A, Lampinen J (1999) Bayesian neural networks for industrial applications. In: IEEE Midnight-Sun workshop on soft computing methods in industrial applications, 1999, pp 63\u201368","DOI":"10.1109\/SMCIA.1999.782709"},{"issue":"3","key":"10443_CR125","doi-asserted-by":"crossref","first-page":"653","DOI":"10.1016\/j.eswa.2005.04.030","volume":"29","author":"S Viaene","year":"2005","unstructured":"Viaene S, Dedene G, Derrig RA (2005) Auto claim fraud detection using Bayesian learning neural networks. Expert Syst Appl 29(3):653\u2013666","journal-title":"Expert Syst Appl"},{"issue":"1\u20132","key":"10443_CR126","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1561\/2200000001","volume":"1","author":"MJ Wainwright","year":"2008","unstructured":"Wainwright MJ, Jordan MI (2008) Graphical models, exponential families, and variational inference. Found Trends Mach Learn 1(1\u20132):1\u2013305","journal-title":"Found Trends Mach Learn"},{"issue":"1","key":"10443_CR127","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s12711-018-0439-1","volume":"50","author":"P Waldmann","year":"2018","unstructured":"Waldmann P (2018) Approximate Bayesian neural networks in genomic prediction. Genet Sel Evol 50(1):1\u20139","journal-title":"Genet Sel Evol"},{"key":"10443_CR128","unstructured":"Wen Y, Tran D, Ba J (2020) BatchEnsemble: an alternative approach to efficient ensemble and lifelong learning. In: International conference on learning representations, 2020"},{"issue":"1","key":"10443_CR129","first-page":"949","volume":"15","author":"D Wierstra","year":"2014","unstructured":"Wierstra D, Schaul T, Glasmachers T et al (2014) Natural evolution strategies. J Mach Learn Res 15(1):949\u2013980","journal-title":"J Mach Learn Res"},{"key":"10443_CR130","unstructured":"Wilson AC, Roelofs R, Stern M et al (2017) The marginal value of adaptive gradient methods in machine learning. In: Advances in neural information processing systems, 2017, vol 30"},{"issue":"7","key":"10443_CR131","doi-asserted-by":"crossref","first-page":"1341","DOI":"10.1162\/neco.1996.8.7.1341","volume":"8","author":"DH Wolpert","year":"1996","unstructured":"Wolpert DH (1996) The lack of a priori distinctions between learning algorithms. Neural Comput 8(7):1341\u20131390","journal-title":"Neural Comput"},{"issue":"3","key":"10443_CR132","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1109\/MCI.2018.2840738","volume":"13","author":"T Young","year":"2018","unstructured":"Young T, Hazarika D, Poria S et al (2018) Recent trends in deep learning based natural language processing. IEEE Comput Intell Mag 13(3):55\u201375","journal-title":"IEEE Comput Intell Mag"},{"issue":"4","key":"10443_CR133","doi-asserted-by":"crossref","first-page":"946","DOI":"10.1080\/10618600.2021.1923516","volume":"30","author":"B Zhou","year":"2021","unstructured":"Zhou B, Gao J, Tran MN et al (2021) Manifold optimization-assisted Gaussian variational approximation. J Comput Graph Stat 30(4):946\u2013957","journal-title":"J Comput Graph Stat"}],"container-title":["Artificial Intelligence Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10462-023-10443-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10462-023-10443-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10462-023-10443-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,16]],"date-time":"2024-10-16T13:41:34Z","timestamp":1729086094000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10462-023-10443-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,15]]},"references-count":133,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2023,10]]}},"alternative-id":["10443"],"URL":"https:\/\/doi.org\/10.1007\/s10462-023-10443-1","relation":{},"ISSN":["0269-2821","1573-7462"],"issn-type":[{"value":"0269-2821","type":"print"},{"value":"1573-7462","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,15]]},"assertion":[{"value":"15 March 2023","order":1,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declaration"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}