{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,27]],"date-time":"2026-06-27T07:13:17Z","timestamp":1782544397690,"version":"3.54.5"},"reference-count":32,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2024,10,23]],"date-time":"2024-10-23T00:00:00Z","timestamp":1729641600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,10,23]],"date-time":"2024-10-23T00:00:00Z","timestamp":1729641600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Comput Stat"],"published-print":{"date-parts":[[2025,6]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>Bayesian neural networks (BNNs) with computationally expensive Hamiltonian Monte Carlo sampling methods are often considered to provide better predictive performance than the maximum a posterior (MAP) solution. Here, as an alternative to sampling all parameters of a BNN (full-random), we experimentally evaluate partially deterministic BNNs that fix some part of the neural network parameters to their MAP solution. In particular, we consider various strategies for fixing half, or all parameters of a layer to the MAP-solution. Over a wide variety of regression and classification tasks, we find that partially deterministic BNNs often significantly improve predictive performance over the MAP-solution, with up to around 24% reduction in negative log-likelihood. Notably, we also find that partially deterministic BNNs that fix half of the parameters in each layer can also reduce under-fitting of full-random BNNs, resulting in up to 7% reduction in negative log-likelihood.<\/jats:p>","DOI":"10.1007\/s00180-024-01561-7","type":"journal-article","created":{"date-parts":[[2024,10,23]],"date-time":"2024-10-23T19:02:23Z","timestamp":1729710143000},"page":"2491-2518","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["On the effectiveness of partially deterministic Bayesian neural networks"],"prefix":"10.1007","volume":"40","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1123-4369","authenticated-orcid":false,"given":"Daniel","family":"Andrade","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Koki","family":"Sato","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,10,23]]},"reference":[{"issue":"2","key":"1561_CR1","first-page":"150","volume":"8","author":"QK Al-Shayea","year":"2011","unstructured":"Al-Shayea QK (2011) Artificial neural networks in medical diagnosis. Int J Comput Sci Issues 8 (2):150\u2013154","journal-title":"Int J Comput Sci Issues"},{"key":"1561_CR2","unstructured":"Chen T, Fox E, Guestrin C (2014) Stochastic gradient hamiltonian monte carlo. In: International conference on machine learning, PMLR, pp 1683\u20131691"},{"key":"1561_CR3","first-page":"20089","volume":"34","author":"E Daxberger","year":"2021","unstructured":"Daxberger E, Kristiadi A, Immer A et\u00a0al. (2021) Laplace redux-effortless Bayesian deep learning. Adv Neural Inf Process Syst 34:20089\u201320103","journal-title":"Adv Neural Inf Process Syst"},{"key":"1561_CR4","unstructured":"Daxberger E, Nalisnick E, Allingham JU, et\u00a0al. (2021b) Bayesian deep learning via subnetwork inference. In: International conference on machine learning, PMLR, pp 2510\u20132521"},{"key":"1561_CR5","first-page":"1","volume":"7","author":"J Dem\u0161ar","year":"2006","unstructured":"Dem\u0161ar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1\u201330","journal-title":"J Mach Learn Res"},{"key":"1561_CR6","unstructured":"Foong AY, Li Y, Hern\u00e1ndez-Lobato JM, et\u00a0al. (2019) \u2019In-between\u2019 uncertainty in Bayesian neural networks. arXiv preprint arXiv:1906.11537"},{"key":"1561_CR7","doi-asserted-by":"publisher","DOI":"10.1201\/b16018","volume-title":"Bayesian data analysis","author":"A Gelman","year":"2013","unstructured":"Gelman A, Stern HS, Carlin JB et\u00a0al. (2013) Bayesian data analysis. Chapman and Hall\/CRC, Boca Raton"},{"issue":"477","key":"1561_CR8","doi-asserted-by":"publisher","first-page":"359","DOI":"10.1198\/016214506000001437","volume":"102","author":"T Gneiting","year":"2007","unstructured":"Gneiting T, Raftery AE (2007) Strictly proper scoring rules, prediction, and estimation. J Am Stat Assoc 102 (477):359\u2013378","journal-title":"J Am Stat Assoc"},{"key":"1561_CR9","unstructured":"Guo C, Pleiss G, Sun Y, et\u00a0al. (2017) On calibration of modern neural networks. In: International conference on machine learning, PMLR, pp 1321\u20131330"},{"key":"1561_CR10","unstructured":"Harrison J, Willes J, Snoek J (2023) Variational Bayesian last layers. In: The twelfth international conference on learning representations"},{"key":"1561_CR11","unstructured":"Izmailov P, Vikram S, Hoffman MD, et\u00a0al. (2021) What are Bayesian neural network posteriors really like? In: International conference on machine learning, PMLR, pp 4629\u20134640"},{"issue":"1","key":"1561_CR12","doi-asserted-by":"publisher","DOI":"10.1016\/j.meaene.2024.100001","volume":"1","author":"B Jin","year":"2024","unstructured":"Jin B, Xu X (2024) Price forecasting through neural networks for crude oil, heating oil, and natural gas. Meas Energy 1 (1):100001","journal-title":"Meas Energy"},{"key":"1561_CR13","unstructured":"Kristiadi A, Hein M, Hennig P (2020) Being bayesian, even just a bit, fixes overconfidence in relu networks. In: International conference on machine learning, PMLR, pp 5436\u20135446"},{"key":"1561_CR14","first-page":"15156","volume":"33","author":"J Lee","year":"2020","unstructured":"Lee J, Schoenholz S, Pennington J et\u00a0al. (2020) Finite versus infinite neural networks: an empirical study. Adv Neural Inf Process Syst 33:15156\u201315172","journal-title":"Adv Neural Inf Process Syst"},{"issue":"523","key":"1561_CR15","doi-asserted-by":"publisher","first-page":"955","DOI":"10.1080\/01621459.2017.1409122","volume":"113","author":"F Liang","year":"2018","unstructured":"Liang F, Li Q, Zhou L (2018) Bayesian neural networks for selection of drug sensitive genes. J Am Stat Assoc 113 (523):955\u2013972","journal-title":"J Am Stat Assoc"},{"key":"1561_CR16","unstructured":"Loshchilov I, Hutter F (2018) Decoupled weight decay regularization. In: International conference on learning representations"},{"key":"1561_CR17","unstructured":"Maddox WJ, Izmailov P, Garipov T, et\u00a0al. (2019) A simple baseline for bayesian uncertainty in deep learning. Adv Neural Inf Process Syst 32"},{"key":"1561_CR18","volume-title":"Probabilistic machine learning: advanced topics","author":"KP Murphy","year":"2023","unstructured":"Murphy KP (2023) Probabilistic machine learning: advanced topics. MIT press, Cambridge"},{"key":"1561_CR19","unstructured":"Ober SW, Rasmussen CE (2019) Benchmarking the neural linear model for regression. In: Second symposium on advances in approximate bayesian inference"},{"issue":"3","key":"1561_CR20","doi-asserted-by":"publisher","first-page":"425","DOI":"10.1214\/21-STS840","volume":"37","author":"T Papamarkou","year":"2022","unstructured":"Papamarkou T, Hinkle J, Young MT et\u00a0al. (2022) Challenges in Markov chain Monte Carlo for Bayesian neural networks. Stat Sci 37 (3):425\u2013442","journal-title":"Stat Sci"},{"key":"1561_CR21","unstructured":"Pitas K, Arbel J (2024) The fine print on tempered posteriors. In: Asian conference on machine learning, PMLR, pp 1087\u20131102"},{"issue":"4","key":"1561_CR22","doi-asserted-by":"publisher","first-page":"1275","DOI":"10.1214\/17-BA1082","volume":"12","author":"NG Polson","year":"2017","unstructured":"Polson NG, Sokolov V (2017) Deep learning: a Bayesian perspective. Bayesian Anal 12 (4):1275\u20131304","journal-title":"Bayesian Anal"},{"key":"1561_CR23","unstructured":"Pourzanjani AA, Jiang RM, Petzold LR (2017) Improving the identifiability of neural networks for bayesian inference. In: NIPS workshop on Bayesian deep learning, p\u00a031"},{"key":"1561_CR24","doi-asserted-by":"publisher","first-page":"1071174","DOI":"10.3389\/fcomp.2023.1071174","volume":"5","author":"S Prabhudesai","year":"2023","unstructured":"Prabhudesai S, Hauth J, Guo D et\u00a0al. (2023) Lowering the computational barrier: partially Bayesian neural networks for transparency in medical imaging AI. Front Comput Sci 5:1071174","journal-title":"Front Comput Sci"},{"key":"1561_CR25","unstructured":"Sharma M, Farquhar S, Nalisnick E, et\u00a0al. (2023) Do Bayesian neural networks need to be fully stochastic? In: International conference on artificial intelligence and statistics, PMLR, pp 7694\u20137722"},{"key":"1561_CR26","unstructured":"Springenberg JT, Klein A, Falkner S, et\u00a0al. (2016) Bayesian optimization with robust Bayesian neural networks. Adv Neural Inf Process Syst 29"},{"issue":"30","key":"1561_CR27","doi-asserted-by":"publisher","first-page":"4605","DOI":"10.1002\/sim.8743","volume":"39","author":"T Sun","year":"2020","unstructured":"Sun T, Wei Y, Chen W et\u00a0al. (2020) Genome-wide association study-based deep learning for survival prediction. Stat Med 39 (30):4605\u20134620","journal-title":"Stat Med"},{"issue":"74","key":"1561_CR28","first-page":"1","volume":"23","author":"BH Tran","year":"2022","unstructured":"Tran BH, Rossi S, Milios D et\u00a0al. (2022) All you need is a good functional prior for Bayesian deep learning. J Mach Learn Res 23 (74):1\u201356","journal-title":"J Mach Learn Res"},{"issue":"2","key":"1561_CR29","doi-asserted-by":"publisher","first-page":"667","DOI":"10.1214\/20-BA1221","volume":"16","author":"A Vehtari","year":"2021","unstructured":"Vehtari A, Gelman A, Simpson D et\u00a0al. (2021) Rank-normalization, folding, and localization: an improved r for assessing convergence of MCMC (with discussion). Bayesian Anal 16 (2):667\u2013718","journal-title":"Bayesian Anal"},{"key":"1561_CR30","unstructured":"Wenzel F, Roth K, Veeling B, et\u00a0al. (2020) How good is the bayes posterior in deep neural networks really? In: International conference on machine learning, PMLR, pp 10248\u201310259"},{"key":"1561_CR31","doi-asserted-by":"publisher","first-page":"196","DOI":"10.1007\/978-1-4612-4380-9_16","volume-title":"Breakthroughs in statistics: methodology and distribution","author":"F Wilcoxon","year":"1992","unstructured":"Wilcoxon F (1992) Individual comparisons by ranking methods. Breakthroughs in statistics: methodology and distribution. Springer, Berlin, pp 196\u2013202"},{"key":"1561_CR32","unstructured":"Zhao Z, Mair S, Sch\u00f6n TB, et\u00a0al. (2024) On feynman-kac training of partial bayesian neural networks. In: International conference on artificial intelligence and statistics, PMLR, pp 3223\u20133231"}],"container-title":["Computational Statistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00180-024-01561-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00180-024-01561-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00180-024-01561-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,24]],"date-time":"2025-05-24T06:16:14Z","timestamp":1748067374000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00180-024-01561-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,23]]},"references-count":32,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2025,6]]}},"alternative-id":["1561"],"URL":"https:\/\/doi.org\/10.1007\/s00180-024-01561-7","relation":{},"ISSN":["0943-4062","1613-9658"],"issn-type":[{"value":"0943-4062","type":"print"},{"value":"1613-9658","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,23]]},"assertion":[{"value":"4 June 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 September 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 October 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}