{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,10]],"date-time":"2026-09-10T10:22:03Z","timestamp":1789035723609,"version":"build-2803163510"},"reference-count":29,"publisher":"Wiley","issue":"10","license":[{"start":{"date-parts":[[2023,8,3]],"date-time":"2023-08-03T00:00:00Z","timestamp":1691020800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"funder":[{"DOI":"10.13039\/100012331","name":"Agentschap Innoveren en Ondernemen","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100012331","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Experimental Dermatology"],"published-print":{"date-parts":[[2023,10]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In dermatology, deep learning may be applied for skin lesion classification. However, for a given input image, a neural network only outputs a label, obtained using the class probabilities, which do not model uncertainty. Our group developed a novel method to quantify uncertainty in stochastic neural networks. In this study, we aimed to train such network for skin lesion classification and evaluate its diagnostic performance and uncertainty, and compare the results to the assessments by a group of dermatologists. By passing duplicates of an image through such a stochastic neural network, we obtained distributions per class, rather than a single probability value. We interpreted the overlap between these distributions as the output uncertainty, where a high overlap indicated a high uncertainty, and vice versa. We had 29 dermatologists diagnose a series of skin lesions and rate their confidence. We compared these results to those of the network. The network achieved a sensitivity and specificity of 50% and 88%, comparable to the average dermatologist (respectively 68% and 73%). Higher confidence\/less uncertainty was associated with better diagnostic performance both in the neural network and in dermatologists. We found no correlation between the uncertainty of the neural network and the confidence of dermatologists (<jats:italic>R<\/jats:italic>\u2009=\u2009\u22120.06, <jats:italic>p<\/jats:italic>\u2009=\u20090.77). Dermatologists should not blindly trust the output of a neural network, especially when its uncertainty is high. The addition of an uncertainty score may stimulate the human\u2010computer interaction.<\/jats:p>","DOI":"10.1111\/exd.14892","type":"journal-article","created":{"date-parts":[[2023,8,3]],"date-time":"2023-08-03T13:04:27Z","timestamp":1691067867000},"page":"1744-1751","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Dermatologist versus artificial intelligence confidence in dermoscopy diagnosis: Complementary information that may affect decision\u2010making"],"prefix":"10.1111","volume":"32","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0165-1337","authenticated-orcid":false,"given":"Pieter","family":"Van\u00a0Molle","sequence":"first","affiliation":[{"name":"IDLab, Department of Information Technology Ghent University\u2013IMEC  Ghent Belgium"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9834-0596","authenticated-orcid":false,"given":"Sofie","family":"Mylle","sequence":"additional","affiliation":[{"name":"Department of Dermatology Ghent University Hospital  Ghent Belgium"},{"name":"Cancer Research Institute Ghent (CRIG)  Ghent Belgium"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2731-7262","authenticated-orcid":false,"given":"Tim","family":"Verbelen","sequence":"additional","affiliation":[{"name":"IDLab, Department of Information Technology Ghent University\u2013IMEC  Ghent Belgium"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0763-8114","authenticated-orcid":false,"given":"Cedric","family":"De\u00a0Boom","sequence":"additional","affiliation":[{"name":"IDLab, Department of Information Technology Ghent University\u2013IMEC  Ghent Belgium"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9489-8306","authenticated-orcid":false,"given":"Bert","family":"Vankeirsbilck","sequence":"additional","affiliation":[{"name":"IDLab, Department of Information Technology Ghent University\u2013IMEC  Ghent Belgium"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8771-3614","authenticated-orcid":false,"given":"Evelien","family":"Verhaeghe","sequence":"additional","affiliation":[{"name":"Department of Dermatology Ghent University Hospital  Ghent Belgium"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7271-7479","authenticated-orcid":false,"given":"Bart","family":"Dhoedt","sequence":"additional","affiliation":[{"name":"IDLab, Department of Information Technology Ghent University\u2013IMEC  Ghent Belgium"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6105-6719","authenticated-orcid":false,"given":"Lieve","family":"Brochez","sequence":"additional","affiliation":[{"name":"Department of Dermatology Ghent University Hospital  Ghent Belgium"},{"name":"Cancer Research Institute Ghent (CRIG)  Ghent Belgium"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2023,8,3]]},"reference":[{"key":"e_1_2_11_2_1","unstructured":"European Cancer Information System.Skin melanoma burden in EU\u201027.20211\u20132."},{"key":"e_1_2_11_3_1","doi-asserted-by":"publisher","DOI":"10.5826\/DPC.1003A33"},{"key":"e_1_2_11_4_1","doi-asserted-by":"crossref","unstructured":"SzegedyC LiuW JiaY et al.Going deeper with convolutions. Proceedings of the IEEE conference on computer vision and pattern recognition;2015:1\u20109. doi:10.1109\/CVPR.2015.7298594","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"e_1_2_11_5_1","doi-asserted-by":"crossref","unstructured":"HeK ZhangX RenS SunJ.Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition;2016:770\u2010778. doi:10.1109\/CVPR.2016.90","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_2_11_6_1","unstructured":"TanM LeQV.EfficientNet: rethinking model scaling for convolutional neural networks. International Conference on Machine learning;2019:6105\u20106114."},{"key":"e_1_2_11_7_1","doi-asserted-by":"publisher","DOI":"10.1016\/S2589-7500(19)30123-2"},{"key":"e_1_2_11_8_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature21056"},{"key":"e_1_2_11_9_1","doi-asserted-by":"publisher","DOI":"10.1093\/annonc\/mdy166"},{"key":"e_1_2_11_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.annonc.2019.10.013"},{"key":"e_1_2_11_11_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jaad.2017.08.016"},{"key":"e_1_2_11_12_1","unstructured":"GalY GhahramaniZ.Dropout as a Bayesian approximation: representing model uncertainty in deep learning. International Conference on Machine Learning;2016:1050\u20101059."},{"key":"e_1_2_11_13_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-020-0942-0"},{"key":"e_1_2_11_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00521-021-05789-y"},{"key":"e_1_2_11_15_1","doi-asserted-by":"publisher","DOI":"10.1038\/sdata.2018.161"},{"key":"e_1_2_11_16_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-015-0816-y"},{"key":"e_1_2_11_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01424-7_27"},{"key":"e_1_2_11_18_1","article-title":"Deep learning using rectified linear units (ReLU)","author":"Agarap AF","year":"2018","journal-title":"arXiv"},{"issue":"1","key":"e_1_2_11_19_1","first-page":"1929","article-title":"Dropout: a simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava N","year":"2014","journal-title":"J Mach Learn Res"},{"key":"e_1_2_11_20_1","first-page":"249","article-title":"Understanding the difficulty of training deep feedforward neural networks","volume":"9","author":"Glorot X","year":"2010","journal-title":"J Mach Learn Res"},{"key":"e_1_2_11_21_1","article-title":"Adam: a method for stochastic optimization","author":"Kingma DP","year":"2015","journal-title":"arXiv"},{"key":"e_1_2_11_22_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-02628-8_13"},{"key":"e_1_2_11_23_1","article-title":"Deep inside convolutional networks: visualising image classification models and saliency maps","author":"Simonyan K","year":"2013","journal-title":"arXiv"},{"key":"e_1_2_11_24_1","doi-asserted-by":"crossref","unstructured":"ZhouB KhoslaA LapedrizaA OlivaA TorralbaA.Learning deep features for discriminative localization. Proceedings of the IEEE conference on computer vision and pattern recognition;2016:2921\u20102929. doi:10.1109\/CVPR.2016.319","DOI":"10.1109\/CVPR.2016.319"},{"key":"e_1_2_11_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/JBHI.2018.2839647"},{"key":"e_1_2_11_26_1","article-title":"Skin lesion classification using class activation map","author":"Jia X","year":"2017","journal-title":"arXiv"},{"key":"e_1_2_11_27_1","doi-asserted-by":"publisher","DOI":"10.48550\/arxiv.1711.11279"},{"key":"e_1_2_11_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN48605.2020.9206946"},{"key":"e_1_2_11_29_1","doi-asserted-by":"publisher","DOI":"10.5826\/dpc.0101a04"},{"key":"e_1_2_11_30_1","doi-asserted-by":"publisher","DOI":"10.1016\/S1470-2045(19)30333-X"}],"container-title":["Experimental Dermatology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/exd.14892","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,4,6]],"date-time":"2025-04-06T03:36:42Z","timestamp":1743910602000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/exd.14892"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,3]]},"references-count":29,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2023,10]]}},"alternative-id":["10.1111\/exd.14892"],"URL":"https:\/\/doi.org\/10.1111\/exd.14892","archive":["Portico"],"relation":{},"ISSN":["0906-6705","1600-0625"],"issn-type":[{"value":"0906-6705","type":"print"},{"value":"1600-0625","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,3]]},"assertion":[{"value":"2022-10-21","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-07-13","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-08-03","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}