{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T02:42:25Z","timestamp":1784256145538,"version":"3.55.0"},"reference-count":44,"publisher":"Springer Science and Business Media LLC","issue":"14","license":[{"start":{"date-parts":[[2024,2,28]],"date-time":"2024-02-28T00:00:00Z","timestamp":1709078400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,2,28]],"date-time":"2024-02-28T00:00:00Z","timestamp":1709078400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100004837","name":"Ministerio de Ciencia e Innovaci\u00f3n","doi-asserted-by":"publisher","award":["PID2021-125943OB-I00"],"award-info":[{"award-number":["PID2021-125943OB-I00"]}],"id":[{"id":"10.13039\/501100004837","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004837","name":"Ministerio de Ciencia e Innovaci\u00f3n","doi-asserted-by":"publisher","award":["PID2019-106827GB-I00"],"award-info":[{"award-number":["PID2019-106827GB-I00"]}],"id":[{"id":"10.13039\/501100004837","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004593","name":"Universidad Aut\u00f3noma de Madrid","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100004593","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Neural Comput &amp; Applic"],"published-print":{"date-parts":[[2024,5]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In this paper, we study the post-hoc calibration of modern neural networks, a problem that has drawn a lot of attention in recent years. Despite the plethora of calibration methods proposed, there is no consensus yet on the inherent complexity of the task and, while some authors claim that simple functions solve the problem, others suggest that more expressive models are needed to capture misscalibration. As a first approach, we focus on the task of confidence scaling, specifically on post-hoc methods that generalize Temperature Scaling, which we refer to as the Adaptive Temperature Scaling family. We begin by demonstrating that while complex models like neural networks provide an advantage when there is ample data, they fail in scenarios where it is limited, notably common in fields like medical diagnosis. We then show how under this ideal data conditions the more expressive methods learn a relationship between the entropy of a prediction and its level of overconfidence, and based on this observation, we propose Entropy-based Temperature Scaling, a simple method that scales the confidence of a prediction according to this relationship. Results show that our method obtains state-of-the-art performance and is robust against data scarcity. Moreover, our proposed model enables a deeper understanding of the calibration process by the interpretation of the entropy as a measure of uncertainty in the network outputs.<\/jats:p>","DOI":"10.1007\/s00521-024-09505-4","type":"journal-article","created":{"date-parts":[[2024,2,28]],"date-time":"2024-02-28T20:02:42Z","timestamp":1709150562000},"page":"8073-8095","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":54,"title":["Adaptive temperature scaling for Robust calibration of deep neural networks"],"prefix":"10.1007","volume":"36","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0006-6726","authenticated-orcid":false,"given":"Sergio A.","family":"Balanya","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Juan","family":"Maro\u00f1as","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Ramos","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,2,28]]},"reference":[{"key":"9505_CR1","doi-asserted-by":"publisher","unstructured":"Ding Z, Han X, Niethammer M (2020) Votenet+ : an improved deep learning label fusion method for multi-atlas segmentation. In: Proceedings IEEE international symposium on biomedical imaging pp. 363\u2013367. https:\/\/doi.org\/10.1109\/isbi45749.2020.9098493","DOI":"10.1109\/isbi45749.2020.9098493"},{"issue":"6","key":"9505_CR2","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1109\/MSP.2012.2205597","volume":"29","author":"G Hinton","year":"2012","unstructured":"Hinton G, Deng L, Yu D, Dahl GE, Mohamed A-R, Jaitly N, Senior A, Vanhoucke V, Nguyen P, Sainath TN, Kingsbury B (2012) Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process Mag 29(6):82\u201397. https:\/\/doi.org\/10.1109\/MSP.2012.2205597","journal-title":"IEEE Signal Process Mag"},{"key":"9505_CR3","volume-title":"Pattern classification, 2nd","author":"RO Duda","year":"2001","unstructured":"Duda RO (2001) Pattern classification, 2nd, ed. Wiley, New York","edition":"ed"},{"issue":"379","key":"9505_CR4","doi-asserted-by":"publisher","first-page":"605","DOI":"10.1080\/01621459.1982.10477856","volume":"77","author":"AP Dawid","year":"1982","unstructured":"Dawid AP (1982) The well-calibrated Bayesian. J Am Stat Assoc 77(379):605\u2013610. https:\/\/doi.org\/10.1080\/01621459.1982.10477856","journal-title":"J Am Stat Assoc"},{"issue":"1\u20132","key":"9505_CR5","doi-asserted-by":"publisher","first-page":"12","DOI":"10.2307\/2987588","volume":"32","author":"MH DeGroot","year":"1983","unstructured":"DeGroot MH, Fienberg SE (1983) The comparison and evaluation of forecasters. J R Stat Soc Ser D 32(1\u20132):12\u201322. https:\/\/doi.org\/10.2307\/2987588","journal-title":"J R Stat Soc Ser D"},{"key":"9505_CR6","unstructured":"Guo C, Pleiss G, Sun Y, Weinberger KQ (2017) On calibration of modern neural networks. In: 34th International conference on machine learning ICML 3: 2130\u20132143 arXiv:1706.04599"},{"key":"9505_CR7","unstructured":"Ovadia Y, Fertig E, Ren J, Nado Z, Sculley D, Nowozin S, Dillon JV, Lakshminarayanan B, Snoek J (2019) Can you trust your model\u2019s uncertainty? Evaluating predictive uncertainty under dataset shift. Adv Neural Inform Process Syst 32(NeurIPS) arXiv:1906.02530"},{"key":"9505_CR8","unstructured":"Lakshminarayanan B, Pritzel A, Blundell C (2017) Simple and scalable predictive uncertainty estimation using deep ensembles. In: Proceedings of the 31st international conference on neural information processing systems. NIPS\u201917, pp. 6405\u20136416. Curran Associates Inc., Red Hook, NY, USA"},{"key":"9505_CR9","unstructured":"Shabat TB, Meir R, Azriel D (2022) Empirical bayes approach to truth discovery problems. In: The 38th conference on uncertainty in artificial intelligence"},{"key":"9505_CR10","unstructured":"Kristiadi A, Hein M, Hennig P (2020) Being Bayesian, even just a bit, fixes overconfidence in ReLU networks. In: 37th International conference on machine learning ICML 2020, vol. PartF16814, pp. 5392\u20135402"},{"key":"9505_CR11","doi-asserted-by":"publisher","unstructured":"Singh\u00a0Sambyal A, Krishnan NC, Bathula DR (20222) toward reducing aleatoric uncertainty for medical imaging tasks. In: 2022 IEEE 19th International symposium on biomedical imaging (ISBI), pp. 1\u20134. https:\/\/doi.org\/10.1109\/ISBI52829.2022.9761638","DOI":"10.1109\/ISBI52829.2022.9761638"},{"key":"9505_CR12","doi-asserted-by":"publisher","unstructured":"Patel K, Beluch W, Zhang D, Pfeiffer M, Yang B (2020) On-manifold adversarial data augmentation improves uncertainty calibration. In: Proceedings\u2014International conference on pattern recognition, pp. 8029\u20138036. https:\/\/doi.org\/10.1109\/ICPR48806.2021.9413010","DOI":"10.1109\/ICPR48806.2021.9413010"},{"key":"9505_CR13","unstructured":"Hendrycks D, Mu N, Cubuk ED, Zoph B, Gilmer J, Lakshminarayanan B (2020) AugMix: a simple data processing method to improve robustness and uncertainty. In: Proceedings of the international conference on learning representations (ICLR)"},{"issue":"34","key":"9505_CR14","first-page":"29768","volume":"6","author":"A Karandikar","year":"2021","unstructured":"Karandikar A, Cain N, Tran D, Lakshminarayanan B, Shlens J, Mozer MC, Roelofs B (2021) Soft calibration objectives for neural networks. Adv Neural Inform Process Syst 6(34):29768\u201379","journal-title":"Adv Neural Inform Process Syst"},{"key":"9505_CR15","doi-asserted-by":"crossref","unstructured":"Tomani C, Cremers D, Buettner F (2021) Parameterized temperature scaling for boosting the expressive power in post-hoc uncertainty calibration","DOI":"10.1007\/978-3-031-19778-9_32"},{"key":"9505_CR16","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2106.09613","author":"O Bohdal","year":"2023","unstructured":"Bohdal O, Yang Y, Hospedales T (2023) Meta-calibration: learning of model calibration using differentiable expected calibration error. Trans Mach Learning Res. https:\/\/doi.org\/10.48550\/arXiv.2106.09613","journal-title":"Trans Mach Learning Res"},{"key":"9505_CR17","unstructured":"Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L, Desmaison A, Kopf A, Yang E, DeVito Z, Raison M, Tejani A, Chilamkurthy S, Steiner B, Fang L, Bai J, Chintala S (2019) Pytorch: an imperative style, high-performance deep learning library. In: Proceedings of the 33rd international conference on neural information processing systems. Curran Associates Inc., Red Hook, NY, USA"},{"key":"9505_CR18","unstructured":"Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, Ghemawat S, Goodfellow I, Harp A, Irving G, Isard M, Jia Y, Jozefowicz R, Kaiser L, Kudlur M, Levenberg J, Man\u00e9 D, Monga R, Moore S, Murray D, Olah C, Schuster M, Shlens J, Steiner B, Sutskever I, Talwar K, Tucker P, Vanhoucke V, Vasudevan V, Vi\u00e9gas F, Vinyals O, Warden P, Wattenberg M, Wicke M, Yu Y, Zheng X (2015) TensorFlow: large-scale machine learning on heterogeneous systems. Software available from tensorflow.org. https:\/\/www.tensorflow.org\/"},{"key":"9505_CR19","doi-asserted-by":"publisher","first-page":"194","DOI":"10.1016\/j.neucom.2020.04.103","volume":"407","author":"J Maro\u00f1as","year":"2020","unstructured":"Maro\u00f1as J, Paredes R, Ramos D (2020) Calibration of deep probabilistic models with decoupled Bayesian neural networks. Neurocomputing 407:194\u2013205","journal-title":"Neurocomputing"},{"key":"9505_CR20","unstructured":"Wenger J, Kjellstr\u00f6m H, Triebel R (2020) Non-parametric calibration for classification. In: International conference on artificial intelligence and statistics (AISTATS) pp. 178\u2013190"},{"key":"9505_CR21","first-page":"15682","volume":"34","author":"M Minderer","year":"2021","unstructured":"Minderer M, Djolonga J, Romijnders R, Hubis F, Zhai X, Houlsby N, Tran D, Lucic M (2021) Revisiting the calibration of modern neural networks. Adv Neural Inf Proc Syst 34: 15682\u201315694","journal-title":"Adv Neural Inf Proc Syst"},{"key":"9505_CR22","unstructured":"Zhang J, Kailkhura B, Han T (2020) Mix-n-match: ensemble and compositional methods for uncertainty calibration in deep learning. In: International conference on machine learning (ICML)"},{"key":"9505_CR23","unstructured":"Rahimi A, Shaban A, Cheng CA, Hartley R, Boots B (2020) Intra order-preserving functions for calibration of multi-class neural networks. In: Advances in neural information processing systems, vol. 2020"},{"key":"9505_CR24","doi-asserted-by":"publisher","unstructured":"Sadjadi S, Kheyrkhah T, Tong A, Greenberg C, Reynolds D, Singer E, Mason L, Hernandez-Cordero J (2018) The 2017 nist language recognition evaluation, pp. 82\u201389 (2018). https:\/\/doi.org\/10.21437\/Odyssey.2018-12","DOI":"10.21437\/Odyssey.2018-12"},{"key":"9505_CR25","doi-asserted-by":"publisher","unstructured":"Brummer N, Van\u00a0Leeuwen DA (2006) On calibration of language recognition scores. In: 2006 IEEE Odyssey - the speaker and language recognition workshop pp. 1\u20138. https:\/\/doi.org\/10.1109\/ODYSSEY.2006.248106","DOI":"10.1109\/ODYSSEY.2006.248106"},{"issue":"2","key":"9505_CR26","doi-asserted-by":"publisher","first-page":"401","DOI":"10.1007\/s11517-021-02321-1","volume":"59","author":"TK Yoo","year":"2021","unstructured":"Yoo TK, Choi JY, Kim HK (2021) Feasibility study to improve deep learning in OCT diagnosis of rare retinal diseases with few-shot classification. Med Biol Eng Compu 59(2):401\u2013415. https:\/\/doi.org\/10.1007\/s11517-021-02321-1","journal-title":"Med Biol Eng Compu"},{"key":"9505_CR27","unstructured":"Mitchell TM (1980) The need for biases in learning generalizations"},{"issue":"11","key":"9505_CR28","doi-asserted-by":"publisher","first-page":"2278","DOI":"10.1109\/5.726791","volume":"86","author":"Y Lecun","year":"1998","unstructured":"Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278\u20132324. https:\/\/doi.org\/10.1109\/5.726791","journal-title":"Proc IEEE"},{"key":"9505_CR29","doi-asserted-by":"crossref","unstructured":"Ding Z, Han X, Liu P, Niethammer M (2020) Local Temperature scaling for probability calibration","DOI":"10.1109\/ICCV48922.2021.00681"},{"key":"9505_CR30","unstructured":"Naeini MP, Cooper GF, Hauskrecht M (2015) Obtaining well calibrated probabilities using Bayesian Binning. In: Proceedings of the national conference on artificial intelligence, vol. 4, pp. 2901\u20132907"},{"key":"9505_CR31","volume-title":"Advances in Neural Information Processing Systems","author":"A Kumar","year":"2019","unstructured":"Kumar A, Liang PS, Ma T (2019) Advances in Neural Information Processing Systems. Curran Associates Inc, US. https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2019\/file\/f8c0c968632845cd133308b1a494967f-Paper.pdf"},{"key":"9505_CR32","unstructured":"Vaicenavicius J, Lindsten F, Widmann D, Roll J, Andersson C, Sch\u00f6n TB (2020) Evaluating model calibration in classification. In: AISTATS 2019 - 22nd international conference on artificial intelligence and statistics"},{"key":"9505_CR33","unstructured":"Nixon J, Dusenberry M, Jerfel G, Nguyen T, Liu J, Zhang L, Tran D (2019) Measuring calibration in deep learning. In: Proceedings - 2019 IEEE conference on computer vision and pattern recognition workshops, pp. 38\u201341"},{"issue":"3","key":"9505_CR34","doi-asserted-by":"publisher","first-page":"335","DOI":"10.1016\/S0167-9473(96)00040-0","volume":"23","author":"JS Simonoff","year":"1997","unstructured":"Simonoff JS, Udina F (1997) Measuring the stability of histogram appearance when the anchor position is changed. Comput Stat Data Anal 23(3):335\u2013353. https:\/\/doi.org\/10.1016\/S0167-9473(96)00040-0","journal-title":"Comput Stat Data Anal"},{"issue":"643","key":"9505_CR35","doi-asserted-by":"publisher","first-page":"1512","DOI":"10.1002\/qj.456","volume":"135","author":"J Br\u00f6cker","year":"2009","unstructured":"Br\u00f6cker J (2009) Reliability, sufficiency, and the decomposition of proper scores. Q J R Meteorol Soc 135(643):1512\u20131519","journal-title":"Q J R Meteorol Soc"},{"issue":"2","key":"9505_CR36","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1007\/s40300-014-0039-y","volume":"72","author":"AP Dawid","year":"2014","unstructured":"Dawid AP, Musio M (2014) Theory and applications of proper scoring rules. Metron 72(2):169\u2013183. https:\/\/doi.org\/10.1007\/s40300-014-0039-y","journal-title":"Metron"},{"key":"9505_CR37","unstructured":"Br\u00fcmmer N (2010) Measuring, refining and calibrating speaker and language information extracted from speech. PhD thesis, Stellenbosch University"},{"key":"9505_CR38","doi-asserted-by":"publisher","unstructured":"Pereyra G, Tucker G, Chorowski J, Kaiser \u0141, Hinton G (2019) Regularizing neural networks by penalizing confident output distributions. In: 5th International conference on learning representations, ICLR 2017 - workshop track proceedings. https:\/\/doi.org\/10.48550\/arxiv.1701.06548.https:\/\/arxiv.org\/abs\/1701.06548","DOI":"10.48550\/arxiv.1701.06548."},{"key":"9505_CR39","doi-asserted-by":"publisher","unstructured":"Ji B, Jung H, Yoon J, Kim K, Shin Y (2019) Bin-wise temperature scaling (BTS): improvement in confidence calibration performance through simple scaling techniques. In: Proceedings - 2019 international conference on computer vision workshop, ICCVW 2019, pp. 4190\u20134196. https:\/\/doi.org\/10.1109\/ICCVW.2019.00515","DOI":"10.1109\/ICCVW.2019.00515"},{"key":"9505_CR40","doi-asserted-by":"publisher","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition 2016, 770\u2013778 arXiv:1512.03385. https:\/\/doi.org\/10.1109\/CVPR.2016.90","DOI":"10.1109\/CVPR.2016.90"},{"key":"9505_CR41","unstructured":"Krizhevsky A, Hinton G (2009) Learning multiple layers of features from tiny images. Technical report"},{"key":"9505_CR42","unstructured":"Sutskever I, Martens J, Dahl G, Hinton G (2013) On the importance of initialization and momentum in deep learning. In: Proceedings of the 30th international conference on machine learning, vol. 28, pp. 1139\u20131147. PMLR. https:\/\/proceedings.mlr.press\/v28\/sutskever13.html"},{"key":"9505_CR43","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1038\/s41592-019-0686-2","volume":"17","author":"P Virtanen","year":"2020","unstructured":"...Virtanen P, Gommers R, Oliphant TE, Haberland M, Reddy T, Cournapeau D, Burovski E, Peterson P, Weckesser W, Bright J, van der Walt SJ, Brett M, Wilson J, Millman KJ, Mayorov N, Nelson ARJ, Jones E, Kern R, Larson E, Carey CJ, Polat \u0130, Feng Y, Moore EW, VanderPlas J, Laxalde D, Perktold J, Cimrman R, Henriksen I, Quintero EA, Harris CR, Archibald AM, Ribeiro AH, Pedregosa F, van Mulbregt P (2020) SciPy 1.0 contributors: SciPy 1.0: fundamental algorithms for scientific computing in python. Nat Methods 17:261\u2013272. https:\/\/doi.org\/10.1038\/s41592-019-0686-2","journal-title":"Nat Methods"},{"key":"9505_CR44","doi-asserted-by":"publisher","unstructured":"Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings - 30th IEEE conference on computer vision and pattern recognition, CVPR 2017 2017, 2261\u20132269 arXiv:1608.06993. https:\/\/doi.org\/10.1109\/CVPR.2017.243","DOI":"10.1109\/CVPR.2017.243"}],"container-title":["Neural Computing and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-024-09505-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00521-024-09505-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-024-09505-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,13]],"date-time":"2024-04-13T20:18:09Z","timestamp":1713039489000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00521-024-09505-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,28]]},"references-count":44,"journal-issue":{"issue":"14","published-print":{"date-parts":[[2024,5]]}},"alternative-id":["9505"],"URL":"https:\/\/doi.org\/10.1007\/s00521-024-09505-4","relation":{},"ISSN":["0941-0643","1433-3058"],"issn-type":[{"value":"0941-0643","type":"print"},{"value":"1433-3058","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,28]]},"assertion":[{"value":"14 March 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 January 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 February 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no competing interests to declare that are relevant to the content of this article.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}