{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,12]],"date-time":"2026-03-12T03:21:11Z","timestamp":1773285671994,"version":"3.50.1"},"reference-count":44,"publisher":"MDPI AG","issue":"21","license":[{"start":{"date-parts":[[2020,10,23]],"date-time":"2020-10-23T00:00:00Z","timestamp":1603411200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004956","name":"Bundesministerium f\u00fcr Verkehr, Innovation und Technologie","doi-asserted-by":"publisher","award":["ASSIC Austrian Smart Systems Integration 521 Research Center"],"award-info":[{"award-number":["ASSIC Austrian Smart Systems Integration 521 Research Center"]}],"id":[{"id":"10.13039\/501100004956","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003413","name":"Bundesministerium f\u00fcr Wissenschaft, Forschung und Wirtschaft","doi-asserted-by":"publisher","award":["ASSIC Austrian Smart Systems Integration 521 Research Center"],"award-info":[{"award-number":["ASSIC Austrian Smart Systems Integration 521 Research Center"]}],"id":[{"id":"10.13039\/501100003413","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Federal Provinces of Carinthia and Styria","award":["ASSIC Austrian Smart Systems Integration 521 Research Center"],"award-info":[{"award-number":["ASSIC Austrian Smart Systems Integration 521 Research Center"]}]},{"DOI":"10.13039\/501100011688","name":"Electronic Components and Systems for European Leadership","doi-asserted-by":"publisher","award":["783163"],"award-info":[{"award-number":["783163"]}],"id":[{"id":"10.13039\/501100011688","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>We present a novel approach for training deep neural networks in a Bayesian way. Compared to other Bayesian deep learning formulations, our approach allows for quantifying the uncertainty in model parameters while only adding very few additional parameters to be optimized. The proposed approach uses variational inference to approximate the intractable a posteriori distribution on basis of a normal prior. By representing the a posteriori uncertainty of the network parameters per network layer and depending on the estimated parameter expectation values, only very few additional parameters need to be optimized compared to a non-Bayesian network. We compare our approach to classical deep learning, Bernoulli dropout and Bayes by Backprop using the MNIST dataset. Compared to classical deep learning, the test error is reduced by 15%. We also show that the uncertainty information obtained can be used to calculate credible intervals for the network prediction and to optimize network architecture for the dataset at hand. To illustrate that our approach also scales to large networks and input vector sizes, we apply it to the GoogLeNet architecture on a custom dataset, achieving an average accuracy of 0.92. Using 95% credible intervals, all but one wrong classification result can be detected.<\/jats:p>","DOI":"10.3390\/s20216011","type":"journal-article","created":{"date-parts":[[2020,10,23]],"date-time":"2020-10-23T08:59:28Z","timestamp":1603443568000},"page":"6011","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Measuring the Uncertainty of Predictions in Deep Neural Networks with Variational Inference"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2465-2527","authenticated-orcid":false,"given":"Jan","family":"Steinbrener","sequence":"first","affiliation":[{"name":"Control of Networked Systems Group, Department of Smart Systems Technologies, Universit\u00e4t Klagenfurt, Universit\u00e4tsstr 65-67, 9020 Klagenfurt, Austria"},{"name":"CTR Carinthian Tech Research AG, Europastr 12, 9524 Villach, Austria"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3602-638X","authenticated-orcid":false,"given":"Konstantin","family":"Posch","sequence":"additional","affiliation":[{"name":"Department of Statistics, Universit\u00e4t Klagenfurt, Universit\u00e4tsstr  65-67, 9020 Klagenfurt, Austria"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9365-4916","authenticated-orcid":false,"given":"J\u00fcrgen","family":"Pilz","sequence":"additional","affiliation":[{"name":"Department of Statistics, Universit\u00e4t Klagenfurt, Universit\u00e4tsstr  65-67, 9020 Klagenfurt, Austria"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,10,23]]},"reference":[{"key":"ref_1","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012). ImageNet Classification with Deep Convolutional Neural Networks, Advances in Neural Information Processing Systems, Curran Associates, Inc."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Bengio, Y., Schwenk, H., Sen\u00e9cal, J.S., Morin, F., and Gauvain, J.L. (2006). Neural Probabilistic Language Models. Innovations in Machine Learning, Springer.","DOI":"10.1007\/3-540-33486-6_6"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"551","DOI":"10.1016\/0893-6080(90)90005-6","article-title":"Universal Approximation of an Unknown Mapping and Its Derivatives Using Multilayer Feedforward Networks","volume":"3","author":"Hornik","year":"1990","journal-title":"Neural Netw."},{"key":"ref_4","unstructured":"Liang, S., and Srikant, R. (2017, January 24\u201326). Why Deep Neural Networks For Function Approximation?. Proceedings of the ICLR, Toulon, France."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2402","DOI":"10.1001\/jama.2016.17216","article-title":"Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs","volume":"316","author":"Gulshan","year":"2016","journal-title":"JAMA J. Am. Med. Assoc."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Greenspan, H., van Ginneken, B., and Summers, R.M. (2016). Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New Technique. IEEE Trans. Med. Imaging, 35.","DOI":"10.1109\/TMI.2016.2553401"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.ress.2017.11.021","article-title":"Remaining useful life estimation in prognostics using deep convolution neural networks","volume":"172","author":"Li","year":"2018","journal-title":"Reliab. Eng. Syst. Saf."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Banerjee, K., Dinh, T.V., and Levkova, L. (2017, January 11\u201314). Velocity estimation from monocular video for automotive applications using convolutional neural networks. Proceedings of the IEEE Intelligent Vehicles Symposium, Los Angeles, CA, USA.","DOI":"10.1109\/IVS.2017.7995747"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1726","DOI":"10.3389\/fpsyg.2017.01726","article-title":"Deep Convolutional Neural Networks Outperform Feature-Based but not Categorical Models in Explaining Object Similarity Judgements","volume":"8","author":"Jozwik","year":"2017","journal-title":"Front. Psychol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1002\/asmb.2209","article-title":"Deep learning for finance: Deep portfolios","volume":"33","author":"Heaton","year":"2017","journal-title":"Appl. Stoch. Model. Bus. Ind."},{"key":"ref_11","unstructured":"Kendall, A., and Gal, Y. (2017). What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?. Advances in Neural Information Processing Systems 30, Curran Associates, Inc."},{"key":"ref_12","unstructured":"Gal, Y., and Ghahramani, Z. (2016, January 19\u201324). Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. Proceedings of the 33rd ICML, New York, NY, USA."},{"key":"ref_13","unstructured":"Rowan, A. (2017). Bayesian Deep Learning with Edward (and a Trick Using Dropout), PyData."},{"key":"ref_14","first-page":"1929","article-title":"Dropout: A Simple Way to Prevent Neural Networks from Overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn."},{"key":"ref_15","unstructured":"Wan, L., Zeiler, M., Zhang, S., LeCun, Y., and Fergus, R. (2013, January 16\u201321). Regularization of Neural Networks Using DropConnect. Proceedings of the 30th ICML, Atlanta, GA, USA."},{"key":"ref_16","unstructured":"Gal, Y., and Ghahramani, Z. (2016, January 2\u20134). Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference. Proceedings of the ICLR, San Juan, PR, USA."},{"key":"ref_17","unstructured":"Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., and Garnett, R. (2015). Variational Dropout and the Local Reparameterization Trick. Advances in Neural Information Processing Systems 28, Curran Associates, Inc."},{"key":"ref_18","unstructured":"Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press."},{"key":"ref_19","unstructured":"Arora, S., Bhaskara, A., Ge, R., and Ma, T. (2014, January 21\u201326). Provable Bounds for Learning Some Deep Representations. Proceedings of the 31st International Conference on Machine Learning, Bejing, China."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1275","DOI":"10.1214\/17-BA1082","article-title":"Deep Learning: A Bayesian Perspective","volume":"12","author":"Polson","year":"2017","journal-title":"Bayesian Anal."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Neal, R.M. (1996). Bayesian Learning for Neural Networks, Springer.","DOI":"10.1007\/978-1-4612-0745-0"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"MacKay, D.J.C. (1996). Bayesian Methods for Backpropagation Networks. Models of Neural Networks III, Springer.","DOI":"10.1007\/978-1-4612-0723-8_6"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1550","DOI":"10.1016\/j.neunet.2006.01.017","article-title":"Invariance priors for Bayesian feed-forward neural networks","volume":"19","author":"Toussaint","year":"2006","journal-title":"Neural Netw."},{"key":"ref_24","unstructured":"Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (2017). Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles. Advances in Neural Information Processing Systems 30, Curran Associates, Inc."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Hinton, G.E., and van Camp, D. (1993, January 26\u201328). Keeping the neural networks simple by minimizing the description length of the weights. Proceedings of the Sixth Annual Conference on Computational Learning Theory, Santa Cruz, CA, USA.","DOI":"10.1145\/168304.168306"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1023\/A:1007665907178","article-title":"An Introduction to Variational Methods for Graphical Models","volume":"37","author":"Jordan","year":"1999","journal-title":"Mach. Learn."},{"key":"ref_27","unstructured":"Blei, D.M., Kucukelbir, A., and McAuliffe, J.D. (2016). Variational Inference: A Review for Statisticians. arXiv."},{"key":"ref_28","unstructured":"Bishop, C.M. (2006). Pattern Recognition and Machine Learning, Springer."},{"key":"ref_29","unstructured":"Gal, Y., and Ghahramani, Z. (2016). Dropout as a Bayesian Approximation: Appendix. arXiv."},{"key":"ref_30","unstructured":"Blundell, C., Cornebise, J., Kavukcuoglu, K., and Wierstra, D. (2015, January 6\u201311). Weight Uncertainty in Neural Networks. Proceedings of the ICML, Lille, France."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-Based Learning Applied to Document Recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proc. IEEE"},{"key":"ref_32","unstructured":"Louizos, C., and Welling, M. (2016). Structured and Efficient Variational Deep Learning with Matrix Gaussian Posteriors. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Amari, S.I. (1985). Differential-Geometrical Methods in Statistics, Springer.","DOI":"10.1007\/978-1-4612-5056-2"},{"key":"ref_34","unstructured":"Hernandez-Lobato, J.M., Li, Y., Rowland, M., Hernandez-Lobato, D., Bui, T., and Turner, R.E. (2015). Black-Box Alpha Divergence Minimization. arXiv."},{"key":"ref_35","unstructured":"Li, Y., and Gal, Y. (2017). Dropout Inference in Bayesian Neural Networks with Alpha-divergences. arXiv."},{"key":"ref_36","unstructured":"Posch, K., and Pilz, J. (2020). Correlated Parameters to Accurately Measure Uncertainty in Deep Neural Networks. IEEE Trans. Neural Netw. Learn. Syst., 1\u201315."},{"key":"ref_37","unstructured":"Kingma, D.P., and Welling, M. (2013). Auto-Encoding Variational Bayes. arXiv."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Hershey, J.R., and Olsen, P.A. (2007, January 15\u201320). Approximating the Kullback Leibler Divergence Between Gaussian Mixture Models. Proceedings of the 2007 IEEE International Conference on Acoustics, Speech, and Signal Processing, Honolulu, HI, USA.","DOI":"10.1109\/ICASSP.2007.366913"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., and Darrell, T. (2014). Caffe: Convolutional Architecture for Fast Feature Embedding. arXiv.","DOI":"10.1145\/2647868.2654889"},{"key":"ref_40","unstructured":"BVLC (2017, May 01). Caffe. Available online: https:\/\/github.com\/BVLC\/caffe."},{"key":"ref_41","unstructured":"Wallach, H., Larochelle, H., Beygelzimer, A., Alch\u00e9-Buc, F., Fox, E., and Garnett, R. (2019). PyTorch: An Imperative Style, High-Performance Deep Learning Library. Advances in Neural Information Processing Systems 32, Curran Associates, Inc."},{"key":"ref_42","unstructured":"LeCun, Y., Cortes, C., and Burges, C.J. (2017, May 01). The MNIST Database of Handwritten Digits, Available online: http:\/\/yann.lecun.com\/exdb\/mnist\/."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_44","first-page":"17","article-title":"Deep Learning of Representations for Unsupervised and Transfer Learning","volume":"27","author":"Bengio","year":"2012","journal-title":"JMLR"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/21\/6011\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:26:38Z","timestamp":1760178398000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/21\/6011"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,23]]},"references-count":44,"journal-issue":{"issue":"21","published-online":{"date-parts":[[2020,11]]}},"alternative-id":["s20216011"],"URL":"https:\/\/doi.org\/10.3390\/s20216011","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,10,23]]}}}