{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,1]],"date-time":"2026-06-01T21:58:54Z","timestamp":1780351134222,"version":"3.54.1"},"reference-count":50,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2020,9,4]],"date-time":"2020-09-04T00:00:00Z","timestamp":1599177600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.springer.com\/tdm"},{"start":{"date-parts":[[2020,9,4]],"date-time":"2020-09-04T00:00:00Z","timestamp":1599177600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.springer.com\/tdm"}],"funder":[{"DOI":"10.13039\/100000015","name":"U.S. Department of Energy","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000015","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Supercomput"],"published-print":{"date-parts":[[2021,4]]},"DOI":"10.1007\/s11227-020-03401-z","type":"journal-article","created":{"date-parts":[[2020,9,4]],"date-time":"2020-09-04T11:02:56Z","timestamp":1599217376000},"page":"3811-3839","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Bayesian neural networks at scale: a performance analysis and pruning study"],"prefix":"10.1007","volume":"77","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5396-1276","authenticated-orcid":false,"given":"Himanshu","family":"Sharma","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Elise","family":"Jennings","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2020,9,4]]},"reference":[{"key":"3401_CR1","doi-asserted-by":"crossref","unstructured":"Neal RM (1995) Bayesian learning for neural networks. Technical report","DOI":"10.1007\/978-1-4612-0745-0"},{"key":"3401_CR2","unstructured":"Williams C (1996) Computing with infinite networks. In: Advances in neural information processing systems, vol 9. MIT Press, Cambridge, pp 295\u2013301"},{"issue":"3","key":"3401_CR3","doi-asserted-by":"publisher","first-page":"448","DOI":"10.1162\/neco.1992.4.3.448","volume":"4","author":"DJC MacKay","year":"1992","unstructured":"MacKay DJC (1992) A practical Bayesian framework for backpropagation networks. Neural Comput 4(3):448\u2013472","journal-title":"Neural Comput"},{"key":"3401_CR4","doi-asserted-by":"crossref","unstructured":"Hinton G, Van\u00a0Camp D (1993) Keeping neural networks simple by minimizing the description length of the weights. In: Proceedings of the 6th Annual ACM Conference on Computational Learning Theory. Citeseer","DOI":"10.1145\/168304.168306"},{"key":"3401_CR5","first-page":"215","volume":"168","author":"D Barber","year":"1998","unstructured":"Barber D, Bishop CM (1998) Ensemble learning in Bayesian neural networks. NATO ASI Ser Ser F Comput Syst Sci 168:215\u2013237","journal-title":"NATO ASI Ser Ser F Comput Syst Sci"},{"key":"3401_CR6","unstructured":"Graves A (2011) Practical variational inference for neural networks. In: Advances in neural information processing systems. pp 2348\u20132356"},{"issue":"1","key":"3401_CR7","first-page":"1303","volume":"14","author":"MD Hoffman","year":"2013","unstructured":"Hoffman MD, Blei DM, Wang C, Paisley J (2013) Stochastic variational inference. J Mach Learn Res 14(1):1303\u20131347","journal-title":"J Mach Learn Res"},{"key":"3401_CR8","unstructured":"Paisley J, Blei D, Jordan M (2013) Variational Bayesian inference with stochastic search. arXiv preprint arXiv:1206.6430"},{"key":"3401_CR9","unstructured":"Kingma DP, Welling M (2013) Auto-encoding variational Bayes. arXiv preprint: arXiv:1312.6114"},{"key":"3401_CR10","unstructured":"Rezende DJ, Mohamed S, Wierstra D (2014) Stochastic backpropagation and approximate inference in deep generative models"},{"key":"3401_CR11","unstructured":"Titsias M, L\u00e1zaro-Gredilla M (2014) Doubly stochastic variational Bayes for non-conjugate inference. In: International Conference on Machine Learning, pp 1971\u20131979"},{"key":"3401_CR12","unstructured":"Blundell C, Cornebise J, Kavukcuoglu K, Wierstra D (2015) Weight uncertainty in neural networks. arXiv preprint arXiv:1505.05424"},{"key":"3401_CR13","unstructured":"Shridhar K, Laumann F, Liwicki M (2019) A comprehensive guide to Bayesian convolutional neural network with variational inference. arXiv preprint arXiv:1901.02731"},{"key":"3401_CR14","unstructured":"Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov RR (2012) Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580"},{"key":"3401_CR15","unstructured":"Gal Y, Ghahramani Z (2016) Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: International Conference on Machine Learning, pp 1050\u20131059"},{"key":"3401_CR16","unstructured":"Tran D, Dusenberry M, van\u00a0der Wilk M, Hafner D (2019) Bayesian layers: a module for neural network uncertainty. In: Advances in neural information processing systems, pp 14633\u201314645"},{"key":"3401_CR17","unstructured":"Shazeer N, Cheng Y, Parmar N, Tran D, Vaswani A, Koanantakool P, Hawkins P, Lee H, Hong M, Young C et\u00a0al (2018) Mesh-tensorflow: deep learning for supercomputers. In: Advances in neural information processing systems, pp 10414\u201310423"},{"key":"3401_CR18","unstructured":"Wen Y, Vicol P, Ba J, Tran D, Grosse R (2018) Flipout: efficient pseudo-independent weight perturbations on mini-batches. arXiv preprint: arXiv:1803.04386"},{"key":"3401_CR19","unstructured":"Tsyplikhin A (2019) Graphcore delivers 26x performance gains for finance customers"},{"key":"3401_CR20","unstructured":"Nabarro S (2019) Probabilistic-modelling-by-combining-markov-chain-monte-carlo-and-variational-inference-with-ipus. https:\/\/www.graphcore.ai\/posts\/probabilistic-modelling-by-combining-markov-chain-monte-carlo-and-variational-inference-with-ipus"},{"key":"3401_CR21","unstructured":"Cerebras Systems (2019) Cerebras wafer scale engine: an introduction. https:\/\/www.cerebras.net\/wp-content\/uploads\/2019\/08\/Cerebras-Wafer-Scale-Engine-An-Introduction.pdf. Accessed Nov 2019"},{"key":"3401_CR22","doi-asserted-by":"crossref","unstructured":"Baydin AG, Shao L, Bhimji W, Heinrich L, Meadows L, Liu J, Munk A, Naderiparizi S, Gram-Hansen B, Louppe G et\u00a0al (2019) Etalumis: bringing probabilistic programming to scientific simulators at scale. arXiv preprint arXiv:1907.03382","DOI":"10.1145\/3295500.3356180"},{"issue":"1","key":"3401_CR23","doi-asserted-by":"publisher","first-page":"197","DOI":"10.1007\/s11227-017-1994-x","volume":"75","author":"A Viebke","year":"2019","unstructured":"Viebke A, Memeti S, Pllana S, Abraham A (2019) Chaos: a parallelization scheme for training convolutional neural networks on Intel Xeon Phi. J Supercomput 75(1):197\u2013227","journal-title":"J Supercomput"},{"key":"3401_CR24","unstructured":"ALCF (2019\/2020) Xc40 machine overview. Technical report"},{"key":"3401_CR25","unstructured":"Vipin Kumar EK, Ying H, Jing X (2017) Numpy\/Scipy with Intel\u00ae MKL and Intel\u00ae compilers. https:\/\/software.intel.com\/en-us\/articles\/numpyscipy-with-intel-mkl"},{"key":"3401_CR26","unstructured":"Cray-MPICH. The MPICH source wiki"},{"key":"3401_CR27","unstructured":"Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556"},{"key":"3401_CR28","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 770\u2013778","DOI":"10.1109\/CVPR.2016.90"},{"key":"3401_CR29","doi-asserted-by":"crossref","unstructured":"Bowman SR, Vilnis L, Vinyals O, Dai AM, Jozefowicz R, Bengio S (2015) Generating sentences from a continuous space. arXiv preprint arXiv:1511.06349","DOI":"10.18653\/v1\/K16-1002"},{"issue":"5","key":"3401_CR30","first-page":"6","volume":"2","author":"I Higgins","year":"2017","unstructured":"Higgins I, Matthey L, Pal A, Burgess C, Glorot X, Botvinick M, Mohamed S, Lerchner A (2017) beta-vae: learning basic visual concepts with a constrained variational framework. ICLR 2(5):6","journal-title":"ICLR"},{"key":"3401_CR31","unstructured":"Alemi AA, Poole B, Fischer I, Dillon JV, Saurous RA, Murphy K (2017) Fixing a broken ELBO. arXiv preprint arXiv:1711.00464"},{"key":"3401_CR32","unstructured":"Liu X, Gao J, Celikyilmaz A, Carin L et\u00a0al (2019) Cyclical annealing schedule: a simple approach to mitigating kl vanishing. arXiv preprint arXiv:1903.10145"},{"key":"3401_CR33","unstructured":"Ranganath R, Gerrish S, Blei DM (2013) Black box variational inference. arXiv preprint arXiv:1401.0118"},{"key":"3401_CR34","unstructured":"Naesseth CA, Ruiz FJR, Linderman SW, Blei DM (2017) Reparameterization gradients through acceptance\u2013rejection sampling algorithms. In: Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017"},{"key":"3401_CR35","unstructured":"Krizhevsky A et al (2009) Learning multiple layers of features from tiny images. Technical report, Citeseer"},{"key":"3401_CR36","unstructured":"LeCun Y (1998) The MNIST database of handwritten digits. http:\/\/yann.lecun.com\/exdb\/mnist\/. Accessed June 2019"},{"key":"3401_CR37","unstructured":"Loosli G, Canu S, Bottou L (2007) Training invariant support vector machines using selective sampling"},{"key":"3401_CR38","unstructured":"Sergeev A, Del Balso M (2018) Horovod: fast and easy distributed deep learning in TensorFlow. arXiv preprint arXiv:1802.05799"},{"key":"3401_CR39","volume-title":"Variational inference: a review for statisticians","author":"DM Blei","year":"2017","unstructured":"Blei DM, Kucukelbir A, McAuliffe JD (2017) Variational inference: a review for statisticians. University of California, Berkeley"},{"key":"3401_CR40","doi-asserted-by":"crossref","unstructured":"Carpenter B, Gelman A, Hoffman MD, Lee D, Goodrich B, Betancourt M, Brubaker M, Guo J, Li P, Riddell A (2017) Stan: a probabilistic programming language. J Stat Softw 76(1)","DOI":"10.18637\/jss.v076.i01"},{"key":"3401_CR41","unstructured":"Stan\u00a0Development Team et\u00a0al (2017) PyStan: the Python interface to Stan. Version 2.16. 0.0"},{"issue":"1","key":"3401_CR42","first-page":"973","volume":"20","author":"E Bingham","year":"2019","unstructured":"Bingham E, Chen JP, Jankowiak M, Obermeyer F, Pradhan N, Karaletsos T, Singh R, Szerlip P, Horsfall P, Goodman ND (2019) Pyro: deep universal probabilistic programming. J Mach Learn Res 20(1):973\u2013978","journal-title":"J Mach Learn Res"},{"key":"3401_CR43","doi-asserted-by":"crossref","unstructured":"Cusumano-Towner MF, Saad FA, Lew AK, Mansinghka VK (2019) Gen: a general-purpose probabilistic programming system with programmable inference. In: Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2019. ACM, New York, pp 221\u2013236","DOI":"10.1145\/3314221.3314642"},{"key":"3401_CR44","unstructured":"Dillon JV, Langmore I, Tran D, Brevdo E, Vasudevan S, Moore D, Patton B, Alemi A, Hoffman M, Saurous RA (2017) Tensorflow distributions. arXiv preprint arXiv:1711.10604"},{"key":"3401_CR45","unstructured":"Huihuo Z, Elise J. Deep learning with Keras, TensorFlow, PyTorch, and Horovod on Theta. https:\/\/www.alcf.anl.gov\/support-center\/training-assets\/deep-learning-keras-tensorflow-pytorch-and-horovod-theta"},{"key":"3401_CR46","unstructured":"Laanait N, Romero J, Yin J, Young MT, Treichler S, Starchenko V, Borisevich A, Sergeev A, Matheson M (2019) Exascale deep learning for scientific inverse problems. arXiv preprint arXiv:1909.11150"},{"key":"3401_CR47","unstructured":"HPCTW. MPI HPC tool. https:\/\/www.alcf.anl.gov\/files\/Knight_Software_JobSubmission_2.pdf"},{"key":"3401_CR48","unstructured":"LeCun Y, Denker JS, Solla SA (1990) Optimal brain damage. In: Advances in neural information processing systems, pp 598\u2013605"},{"issue":"5","key":"3401_CR49","doi-asserted-by":"publisher","first-page":"848","DOI":"10.1109\/72.317740","volume":"5","author":"CL Giles","year":"1994","unstructured":"Giles CL, Omlin CW (1994) Pruning recurrent neural networks for improved generalization performance. IEEE Trans Neural Netw 5(5):848\u2013851","journal-title":"IEEE Trans Neural Netw"},{"key":"3401_CR50","unstructured":"Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, Ghemawat S, Goodfellow I, Harp A, Irving G, Isard M, Jia Y, Jozefowicz R, Kaiser L, Kudlur M, Levenberg J, Man\u00e9 D, Monga R, Moore S, Murray D, Olah C, Schuster M, Shlens J, Steiner B, Sutskever I, Talwar K, Tucker P, Vanhoucke V, Vasudevan V, Vi\u00e9gas F, Vinyals O, Warden P, Wattenberg M, Wicke M, Yu Y, Zheng X (2015) TensorFlow: large-scale machine learning on heterogeneous systems. Software available from www.tensorflow.org. Accessed June 2019"}],"container-title":["The Journal of Supercomputing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11227-020-03401-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11227-020-03401-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11227-020-03401-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,9,4]],"date-time":"2021-09-04T01:53:36Z","timestamp":1630720416000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11227-020-03401-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,4]]},"references-count":50,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2021,4]]}},"alternative-id":["3401"],"URL":"https:\/\/doi.org\/10.1007\/s11227-020-03401-z","relation":{},"ISSN":["0920-8542","1573-0484"],"issn-type":[{"value":"0920-8542","type":"print"},{"value":"1573-0484","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,9,4]]},"assertion":[{"value":"4 September 2020","order":1,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}