{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,17]],"date-time":"2026-01-17T21:34:48Z","timestamp":1768685688856,"version":"3.49.0"},"reference-count":50,"publisher":"Springer Science and Business Media LLC","issue":"16","license":[{"start":{"date-parts":[[2021,3,1]],"date-time":"2021-03-01T00:00:00Z","timestamp":1614556800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,3,1]],"date-time":"2021-03-01T00:00:00Z","timestamp":1614556800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100012331","name":"Agentschap Innoveren en Ondernemen","doi-asserted-by":"publisher","award":["HBC.2016.0436"],"award-info":[{"award-number":["HBC.2016.0436"]}],"id":[{"id":"10.13039\/100012331","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100012331","name":"Agentschap Innoveren en Ondernemen","doi-asserted-by":"publisher","award":["HBC.2018.2028"],"award-info":[{"award-number":["HBC.2018.2028"]}],"id":[{"id":"10.13039\/100012331","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Neural Comput &amp; Applic"],"published-print":{"date-parts":[[2021,8]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Modern deep learning models achieve state-of-the-art results for many tasks in computer vision, such as image classification and segmentation. However, its adoption into high-risk applications, e.g. automated medical diagnosis systems, happens at a slow pace. One of the main reasons for this is that regular neural networks do not capture uncertainty. To assess uncertainty in classification, several techniques have been proposed casting neural network approaches in a Bayesian setting. Amongst these techniques, Monte Carlo dropout is by far the most popular. This particular technique estimates the moments of the output distribution through sampling with different dropout masks. The output uncertainty of a neural network is then approximated as the sample variance. In this paper, we highlight the limitations of such a variance-based uncertainty metric and propose an novel approach. Our approach is based on the overlap between output distributions of different classes. We show that our technique leads to a better approximation of the inter-class output confusion. We illustrate the advantages of our method using benchmark datasets. In addition, we apply our metric to skin lesion classification\u2014a real-world use case\u2014and show that this yields promising results.\n<\/jats:p>","DOI":"10.1007\/s00521-021-05789-y","type":"journal-article","created":{"date-parts":[[2021,3,1]],"date-time":"2021-03-01T21:02:57Z","timestamp":1614632577000},"page":"10259-10275","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Leveraging the Bhattacharyya coefficient for uncertainty quantification in deep neural networks"],"prefix":"10.1007","volume":"33","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0165-1337","authenticated-orcid":false,"given":"Pieter","family":"Van Molle","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tim","family":"Verbelen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bert","family":"Vankeirsbilck","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jonas","family":"De Vylder","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bart","family":"Diricx","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tom","family":"Kimpe","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pieter","family":"Simoens","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bart","family":"Dhoedt","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,3,1]]},"reference":[{"issue":"2","key":"5789_CR1","doi-asserted-by":"publisher","first-page":"80","DOI":"10.4018\/IJACI.2020040105","volume":"11","author":"DP Acharjya","year":"2020","unstructured":"Acharjya DP et al (2020) Behavioural intention of customers towards smartwatches in an ambient environment using soft computing: an integrated sem-pls and fuzzy rough set approach. Int J Ambient Comput Intell (IJACI) 11(2):80\u2013111","journal-title":"Int J Ambient Comput Intell (IJACI)"},{"key":"5789_CR2","unstructured":"American Cancer Society (2020) Cancer facts & figures 2020"},{"key":"5789_CR3","first-page":"99","volume":"35","author":"A Bhattacharyya","year":"1943","unstructured":"Bhattacharyya A (1943) On a measure of divergence between two statistical populations defined by their probability distributions. Bull Calcutta Math Soc 35:99\u2013109","journal-title":"Bull Calcutta Math Soc"},{"key":"5789_CR4","unstructured":"Blundell C, Cornebise J, Kavukcuoglu K, Wierstra D (2015) Weight uncertainty in neural networks. arXiv preprint arXiv:150505424"},{"key":"5789_CR5","doi-asserted-by":"publisher","first-page":"103735","DOI":"10.1016\/j.compbiomed.2020.103735","volume":"120","author":"A Bria","year":"2020","unstructured":"Bria A, Marrocco C, Tortorella F (2020) Addressing class imbalance in deep learning for small lesion detection on medical images. Comput Biol Med 120:103735","journal-title":"Comput Biol Med"},{"issue":"2","key":"5789_CR6","doi-asserted-by":"publisher","first-page":"474","DOI":"10.1109\/JBHI.2019.2895803","volume":"23","author":"ME Celebi","year":"2019","unstructured":"Celebi ME, Codella N, Halpern A (2019) Dermoscopy image analysis: overview and future directions. IEEE J Biomed Health Inform 23(2):474\u2013478","journal-title":"IEEE J Biomed Health Inform"},{"issue":"2","key":"5789_CR7","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1016\/j.strusafe.2008.06.020","volume":"31","author":"A Der Kiureghian","year":"2009","unstructured":"Der Kiureghian A, Ditlevsen O (2009) Aleatory or epistemic? does it matter? Struct Saf 31(2):105\u2013112","journal-title":"Struct Saf"},{"issue":"7639","key":"5789_CR8","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1038\/nature21056","volume":"542","author":"A Esteva","year":"2017","unstructured":"Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Thrun S (2017) Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639):115\u2013118","journal-title":"Nature"},{"key":"5789_CR9","unstructured":"Gal Y, Ghahramani Z (2016) Bayesian convolutional neural networks with Bernoulli approximate variational inference. arXiv preprint arXiv:150602158"},{"key":"5789_CR10","unstructured":"Gal Y, Ghahramani Z (2016) Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: International conference on machine learning, pp 1050\u20131059"},{"key":"5789_CR11","unstructured":"Gal Y, Islam R, Ghahramani Z (2017) Deep Bayesian active learning with image data. In: Proceedings of the 34th international conference on machine learning, Vol 70, JMLR. org, pp 1183\u20131192"},{"key":"5789_CR12","doi-asserted-by":"publisher","first-page":"106266","DOI":"10.1016\/j.cie.2019.106266","volume":"140","author":"D Gan","year":"2020","unstructured":"Gan D, Shen J, An B, Xu M, Liu N (2020) Integrating tanbn with cost sensitive classification algorithm for imbalanced data in medical diagnosis. Comput Ind Eng 140:106266","journal-title":"Comput Ind Eng"},{"key":"5789_CR13","doi-asserted-by":"crossref","unstructured":"Girshick R (2015) Fast r-cnn. In: Proceedings of the IEEE international conference on computer vision, pp 1440\u20131448","DOI":"10.1109\/ICCV.2015.169"},{"key":"5789_CR14","doi-asserted-by":"crossref","unstructured":"Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 580\u2013587","DOI":"10.1109\/CVPR.2014.81"},{"key":"5789_CR15","unstructured":"Graves A (2011) Practical variational inference for neural networks. In: Advances in neural information processing systems, pp 2348\u20132356"},{"key":"5789_CR16","doi-asserted-by":"crossref","unstructured":"Gupta AK, Nagar DK (2018) Matrix variate distributions. Chapman and Hall\/CRC","DOI":"10.1201\/9780203749289"},{"key":"5789_CR17","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770\u2013778","DOI":"10.1109\/CVPR.2016.90"},{"key":"5789_CR18","unstructured":"Houlsby N, Husz\u00e1r F, Ghahramani Z, Lengyel M (2011) Bayesian active learning for classification and preference learning. arXiv preprint arXiv:11125745"},{"key":"5789_CR19","unstructured":"Jiang H, Kim B, Guan M, Gupta M (2018) To trust or not to trust a classifier. In: Advances in neural information processing systems, pp 5541\u20135552"},{"issue":"2","key":"5789_CR20","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1023\/A:1007665907178","volume":"37","author":"MI Jordan","year":"1999","unstructured":"Jordan MI, Ghahramani Z, Jaakkola TS, Saul LK (1999) An introduction to variational methods for graphical models. Mach Learn 37(2):183\u2013233","journal-title":"Mach Learn"},{"key":"5789_CR21","unstructured":"Kendall A, Badrinarayanan V, Cipolla R (2015) Bayesian segnet: model uncertainty in deep convolutional encoder-decoder architectures for scene understanding. arXiv preprint arXiv:151102680"},{"key":"5789_CR22","doi-asserted-by":"crossref","unstructured":"Kisel\u2019\u00e1k J, Lu Y, \u0160vihra J, Sz\u00e9pe P, Stehl\u00edk M (2020) \u201cspocu\u201d: scaled polynomial constant unit activation function. Neural Comput Appl 1\u201317","DOI":"10.1007\/s00521-020-05412-6"},{"key":"5789_CR23","unstructured":"Krizhevsky A, Hinton G (2009) Learning multiple layers of features from tiny images. Tech. rep, Citeseer"},{"key":"5789_CR24","doi-asserted-by":"publisher","first-page":"106816","DOI":"10.1016\/j.csda.2019.106816","volume":"142","author":"Y Kwon","year":"2020","unstructured":"Kwon Y, Won JH, Kim BJ, Paik MC (2020) Uncertainty quantification using Bayesian neural networks in classification: application to biomedical image segmentation. Comput Stat Data Anal 142:106816","journal-title":"Comput Stat Data Anal"},{"key":"5789_CR25","unstructured":"Lakshminarayanan B, Pritzel A, Blundell C (2017) Simple and scalable predictive uncertainty estimation using deep ensembles. In: Advances in neural information processing systems, pp 6402\u20136413"},{"issue":"11","key":"5789_CR26","doi-asserted-by":"publisher","first-page":"2278","DOI":"10.1109\/5.726791","volume":"86","author":"Y LeCun","year":"1998","unstructured":"LeCun Y, Bottou L, Bengio Y, Haffner P et al (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278\u20132324","journal-title":"Proc IEEE"},{"key":"5789_CR27","unstructured":"Lee K, Lee K, Lee H, Shin J (2018) A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In: Advances in neural information processing systems, pp 7167\u20137177"},{"issue":"1","key":"5789_CR28","doi-asserted-by":"publisher","first-page":"17816","DOI":"10.1038\/s41598-017-17876-z","volume":"7","author":"C Leibig","year":"2017","unstructured":"Leibig C, Allken V, Ayhan MS, Berens P, Wahl S (2017) Leveraging uncertainty information from deep neural networks for disease detection. Sci Rep 7(1):17816","journal-title":"Sci Rep"},{"key":"5789_CR29","doi-asserted-by":"crossref","unstructured":"Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431\u20133440","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"5789_CR30","unstructured":"Louizos C, Welling M (2016) Structured and efficient variational deep learning with matrix gaussian posteriors. In: International conference on machine learning, pp 1708\u20131716"},{"issue":"3","key":"5789_CR31","doi-asserted-by":"publisher","first-page":"448","DOI":"10.1162\/neco.1992.4.3.448","volume":"4","author":"DJ MacKay","year":"1992","unstructured":"MacKay DJ (1992) A practical Bayesian framework for backpropagation networks. Neural Comput 4(3):448\u2013472","journal-title":"Neural Comput"},{"issue":"2","key":"5789_CR32","doi-asserted-by":"publisher","first-page":"270","DOI":"10.1016\/j.jaad.2017.08.016","volume":"78","author":"MA Marchetti","year":"2018","unstructured":"Marchetti MA, Codella NC, Dusza SW, Gutman DA, Helba B, Kalloo A, Mishra N, Carrera C, Celebi ME, DeFazio JL et al (2018) Results of the 2016 international skin imaging collaboration international symposium on biomedical imaging challenge: Comparison of the accuracy of computer algorithms to dermatologists for the diagnosis of melanoma from dermoscopic images. J Am Acad Dermatol 78(2):270\u2013277","journal-title":"J Am Acad Dermatol"},{"issue":"3","key":"5789_CR33","doi-asserted-by":"publisher","first-page":"622","DOI":"10.1016\/j.jaad.2019.07.016","volume":"82","author":"MA Marchetti","year":"2020","unstructured":"Marchetti MA, Liopyris K, Dusza SW, Codella NC, Gutman DA, Helba B, Kalloo A, Halpern AC, Soyer HP, Curiel-Lewandrowski C et al (2020) Computer algorithms show potential for improving dermatologists\u2019 accuracy to diagnose cutaneous melanoma: Results of the international skin imaging collaboration 2017. J Am Acad Dermatol 82(3):622\u2013627","journal-title":"J Am Acad Dermatol"},{"key":"5789_CR34","doi-asserted-by":"publisher","first-page":"101557","DOI":"10.1016\/j.media.2019.101557","volume":"59","author":"T Nair","year":"2020","unstructured":"Nair T, Precup D, Arnold DL, Arbel T (2020) Exploring uncertainty measures in deep networks for multiple sclerosis lesion detection and segmentation. Med Image Anal 59:101557","journal-title":"Med Image Anal"},{"key":"5789_CR35","unstructured":"Neal R (1995) Bayesian learning for neural networks. Toronto, Ontario, Canada: Department of Computer Science, University of Toronto"},{"key":"5789_CR36","unstructured":"Ozdemir O, Woodward B, Berlin AA (2017) Propagating uncertainty in multi-stage bayesian convolutional neural networks with application to pulmonary nodule detection. arXiv preprint arXiv:171200497"},{"key":"5789_CR37","unstructured":"Posch K, Steinbrener J, Pilz J (2019) Variational inference to measure model uncertainty in deep neural networks. arXiv preprint arXiv:190210189"},{"key":"5789_CR38","unstructured":"Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: Towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, pp 91\u201399"},{"key":"5789_CR39","doi-asserted-by":"crossref","unstructured":"Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention, Springer, pp 234\u2013241","DOI":"10.1007\/978-3-319-24574-4_28"},{"issue":"3","key":"5789_CR40","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","volume":"115","author":"O Russakovsky","year":"2015","unstructured":"Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115(3):211\u2013252","journal-title":"Int J Comput Vis"},{"issue":"1","key":"5789_CR41","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1109\/TMI.2019.2919951","volume":"39","author":"P Seebock","year":"2019","unstructured":"Seebock P, Orlando JI, Schlegl T, Waldstein SM, Bogunovic H, Klimscha S, Langs G, Schmidt-Erfurth U (2019) Exploiting epistemic uncertainty of anatomy segmentation for anomaly detection in retinal OCT. IEEE Trans Med Imaging 39(1):87\u201398","journal-title":"IEEE Trans Med Imaging"},{"key":"5789_CR42","unstructured":"Shridhar K, Laumann F, Llopart\u00a0Maurin A, Liwicki M (2018) Bayesian convolutional neural networks. arXiv preprint arXiv:180605978"},{"key":"5789_CR43","unstructured":"Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:14091556"},{"issue":"1","key":"5789_CR44","first-page":"1929","volume":"15","author":"N Srivastava","year":"2014","unstructured":"Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929\u20131958","journal-title":"J Mach Learn Res"},{"key":"5789_CR45","doi-asserted-by":"crossref","unstructured":"Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1\u20139","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"5789_CR46","doi-asserted-by":"publisher","first-page":"429","DOI":"10.1016\/j.ins.2019.11.004","volume":"513","author":"F Thabtah","year":"2020","unstructured":"Thabtah F, Hammoud S, Kamalov F, Gonsalves A (2020) Data imbalance in classification: experimental evaluation. Inf Sci 513:429\u2013441","journal-title":"Inf Sci"},{"key":"5789_CR47","doi-asserted-by":"publisher","first-page":"180161","DOI":"10.1038\/sdata.2018.161","volume":"5","author":"P Tschandl","year":"2018","unstructured":"Tschandl P, Rosendahl C, Kittler H (2018) The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci Data 5:180161","journal-title":"Sci Data"},{"key":"5789_CR48","doi-asserted-by":"publisher","first-page":"34","DOI":"10.1016\/j.neucom.2019.01.103","volume":"338","author":"G Wang","year":"2019","unstructured":"Wang G, Li W, Aertsen M, Deprest J, Ourselin S, Vercauteren T (2019) Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks. Neurocomputing 338:34\u201345","journal-title":"Neurocomputing"},{"key":"5789_CR49","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1016\/j.ins.2020.08.096","volume":"546","author":"Y Xue","year":"2021","unstructured":"Xue Y, Deng Y, Garg H (2021) Uncertain database retrieval with measure-based belief function attribute values under intuitionistic fuzzy set. Inf Sci 546:436\u2013447","journal-title":"Inf Sci"},{"key":"5789_CR50","unstructured":"Yildirim MY, Ozer M, Davulcu H (2019) Leveraging uncertainty in deep learning for selective classification. arXiv preprint arXiv:190509509"}],"container-title":["Neural Computing and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-021-05789-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00521-021-05789-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-021-05789-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,7,8]],"date-time":"2021-07-08T07:29:57Z","timestamp":1625729397000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00521-021-05789-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,1]]},"references-count":50,"journal-issue":{"issue":"16","published-print":{"date-parts":[[2021,8]]}},"alternative-id":["5789"],"URL":"https:\/\/doi.org\/10.1007\/s00521-021-05789-y","relation":{},"ISSN":["0941-0643","1433-3058"],"issn-type":[{"value":"0941-0643","type":"print"},{"value":"1433-3058","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,3,1]]},"assertion":[{"value":"23 June 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 February 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 March 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Compliance with ethical standards"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}