{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T16:53:44Z","timestamp":1791910424763,"version":"4.3.4"},"reference-count":45,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2018,3,20]],"date-time":"2018-03-20T00:00:00Z","timestamp":1521504000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Spanish Ministry of Economy and Competitiveness","award":["TEC2015-68172-C2-1-P"],"award-info":[{"award-number":["TEC2015-68172-C2-1-P"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>In this work, we analyze the cross-entropy function, widely used in classifiers both as a performance measure and as an optimization objective. We contextualize cross-entropy in the light of Bayesian decision theory, the formal probabilistic framework for making decisions, and we thoroughly analyze its motivation, meaning and interpretation from an information-theoretical point of view. In this sense, this article presents several contributions: First, we explicitly analyze the contribution to cross-entropy of (i) prior knowledge; and (ii) the value of the features in the form of a likelihood ratio. Second, we introduce a decomposition of cross-entropy into two components: discrimination and calibration. This decomposition enables the measurement of different performance aspects of a classifier in a more precise way; and justifies previously reported strategies to obtain reliable probabilities by means of the calibration of the output of a discriminating classifier. Third, we give different information-theoretical interpretations of cross-entropy, which can be useful in different application scenarios, and which are related to the concept of reference probabilities. Fourth, we present an analysis tool, the Empirical Cross-Entropy (ECE) plot, a compact representation of cross-entropy and its aforementioned decomposition. We show the power of ECE plots, as compared to other classical performance representations, in two diverse experimental examples: a speaker verification system, and a forensic case where some glass findings are present.<\/jats:p>","DOI":"10.3390\/e20030208","type":"journal-article","created":{"date-parts":[[2018,3,20]],"date-time":"2018-03-20T06:57:11Z","timestamp":1521529031000},"page":"208","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":97,"title":["Deconstructing Cross-Entropy for Probabilistic Binary Classifiers"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5998-1489","authenticated-orcid":false,"given":"Daniel","family":"Ramos","sequence":"first","affiliation":[{"name":"AuDIaS-Audio, Data Intelligence and Speech, Escuela Politecnica Superior, Universidad Autonoma de Madrid, Calle Francisco Tomas y Valiente 11, 28049 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Javier","family":"Franco-Pedroso","sequence":"additional","affiliation":[{"name":"AuDIaS-Audio, Data Intelligence and Speech, Escuela Politecnica Superior, Universidad Autonoma de Madrid, Calle Francisco Tomas y Valiente 11, 28049 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5918-8568","authenticated-orcid":false,"given":"Alicia","family":"Lozano-Diez","sequence":"additional","affiliation":[{"name":"AuDIaS-Audio, Data Intelligence and Speech, Escuela Politecnica Superior, Universidad Autonoma de Madrid, Calle Francisco Tomas y Valiente 11, 28049 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0910-2575","authenticated-orcid":false,"given":"Joaquin","family":"Gonzalez-Rodriguez","sequence":"additional","affiliation":[{"name":"AuDIaS-Audio, Data Intelligence and Speech, Escuela Politecnica Superior, Universidad Autonoma de Madrid, Calle Francisco Tomas y Valiente 11, 28049 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2018,3,20]]},"reference":[{"key":"ref_1","unstructured":"Murphy, K.P. (2012). Machine Learning: A Probabilistic Perspective, MIT Press."},{"key":"ref_2","unstructured":"Smola, A.J., Bartlett, P., Sholkopf, B., and Shchuurmans, D. (1999). Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in Large Margin Classifiers, MIT Press. [Book ed.]. Chapter 10."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Zadrozny, B., and Elkan, C. (2002, January 23\u201326). Transforming classifier scores into accurate multiclass probability estimates. In Proceeding of the Eight International Conference on Knowledge Discovery and Data Mining (KDD\u201902), Edmonton, AB, Canada.","DOI":"10.1145\/775107.775151"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Cohen, I., and Goldszmidt, M. (2004). Properties and benefits of calibrated classifiers. Lecture Notes in Computer Science, Springer. Knowledge Discovery in Databases: PKDD 2004.","DOI":"10.1007\/978-3-540-30116-5_14"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Niculescu-Mizil, A., and Caruana, R. (2005, January 7\u201311). Predicting Good Probabilities With Supervised Learning. Proceedings of the 22nd International Conference on Machine Learning, Bonn, Germany.","DOI":"10.1145\/1102351.1102430"},{"key":"ref_6","unstructured":"Guo, C., Pleiss, G., Sun, Y., and Weinberger, K.Q. (2017, January 6\u201311). On Calibration of Modern Neural Networks. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"226","DOI":"10.1109\/34.667881","article-title":"On combining classifiers","volume":"20","author":"Kittler","year":"1998","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_8","unstructured":"Koller, D., and Friedman, N. (2009). Probabilistic Graphical Models: Principles and Techniques\u2014Adaptive Computation and Machine Learning, MIT Press."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"527","DOI":"10.1136\/jamia.2001.0080527","article-title":"Clinical Decision Support Systems for the Practice of Evidence-based Medicine","volume":"8","author":"Sim","year":"2001","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1124","DOI":"10.1126\/science.185.4157.1124","article-title":"Judgment under Uncertainty: Heuristics and Biases","volume":"185","author":"Tversky","year":"1974","journal-title":"Science"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"506","DOI":"10.1037\/0033-295X.98.4.506","article-title":"Probabilistic Mental Models: A Brunswikian Theory of Confidence","volume":"98","author":"Gigerenzer","year":"1991","journal-title":"Psychol. Rev."},{"key":"ref_12","unstructured":"M\u00fcller, C. (2007). An introduction to application-independent evaluation of speaker recognition systems. Speaker Classification, Springer. Lecture Notes in Computer Science\/Artificial Intelligence."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1016\/j.csl.2005.08.001","article-title":"Application Independent Evaluation of Speaker Detection","volume":"20","year":"2006","journal-title":"Comput. Speech Lang."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Tistarelli, M., and Champod, C. (2017). From Bometric Scores to Forensic Likelihood Ratios. Handbook of Biometrics for Forensic Science, Book ed., Springer. Chapter 14.","DOI":"10.1007\/978-3-319-50673-9"},{"key":"ref_15","first-page":"41","article-title":"Reliability of Subjective Probability Forecasts of Precipitation and Temperature","volume":"26","author":"Murphy","year":"1977","journal-title":"J. R. Stat. Soc. Ser. C (Appl. Stat.)"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1016\/j.forsciint.2013.04.014","article-title":"Reliable support: measuring calibration of likelihood ratios","volume":"230","author":"Ramos","year":"2013","journal-title":"Forensic Sci. Int."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.scijus.2011.01.002","article-title":"Expressing evaluative opinions: A position statement","volume":"51","author":"Berger","year":"2011","journal-title":"Sci. Justice"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"12","DOI":"10.2307\/2987588","article-title":"The Comparison and Evaluation of Forecasters","volume":"32","author":"DeGroot","year":"1983","journal-title":"Statistician"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1111\/j.1467-9868.2007.00587.x","article-title":"Probabilistic forecasts, calibration and sharpness","volume":"69","author":"Gneiting","year":"2007","journal-title":"J. R. Stat. Soc. Ser. B"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"605","DOI":"10.1080\/01621459.1982.10477856","article-title":"The well-calibrated Bayesian","volume":"77","author":"Dawid","year":"1982","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"783","DOI":"10.1080\/01621459.1971.10482346","article-title":"The elicitation of personal probabilities and expectations","volume":"66","author":"Savage","year":"1971","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1198\/016214506000001437","article-title":"Strictly Proper Scoring Rules, Prediction and Estimation","volume":"102","author":"Gneiting","year":"2007","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"461","DOI":"10.1162\/neco.1991.3.4.461","article-title":"Neural network classifiers estimate Bayesian a posteriori probabilities","volume":"3","author":"Richard","year":"1991","journal-title":"Neural Comput."},{"key":"ref_24","unstructured":"Willis, S. (2015). ENFSI Guideline for the Formulation of Evaluative Reports in Forensic Science. Monopoly Project MP2010: The Development and Implementation of an ENFSI Standard for Reporting Evaluative Forensic Evidence, European Network of Forensic Science Institutes. Technical Report."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1503","DOI":"10.1111\/1556-4029.12233","article-title":"Information-Theoretical Assessment of the Performance of Likelihood Ratio Models","volume":"58","author":"Ramos","year":"2013","journal-title":"J. Forensic Sci."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1016\/j.specom.2009.08.009","article-title":"An overview of text-independent speaker recognition: From features to supervectors","volume":"52","author":"Kinnunen","year":"2010","journal-title":"Speech Commun."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1175\/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2","article-title":"Verification of forecasts expressed in terms of probability","volume":"78","author":"Brier","year":"1950","journal-title":"Mon. Weather Rev."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A Mathematical Theory of Communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J."},{"key":"ref_29","unstructured":"Cover, T.M., and Thomas, J.A. (2006). Elements of Information Theory, Wiley Interscience. [2nd ed.]."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1007\/s10994-007-5011-0","article-title":"PAV and the ROC convex hull","volume":"68","author":"Fawcett","year":"2007","journal-title":"Mach. Learn."},{"key":"ref_31","unstructured":"Br\u00fcmmer, N. (2010). Measuring, Refining and Calibrating Speaker and Language Information Extracted from Speech. [Ph.D. Thesis, School of Electrical Engineering, University of Stellenbosch]. Available online: http:\/\/sites.google.com\/site\/nikobrummer\/."},{"key":"ref_32","unstructured":"Br\u00fcmmer, N., and du Preez, J. (2018, January 31). The PAV Algorithm Optimizes Binary Proper Scoring Rules. Technical Report, Agnitio, 2009. Available online: https:\/\/sites.google.com\/site\/nikobrummer\/."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"788","DOI":"10.1109\/TASL.2010.2064307","article-title":"Front-End Factor Analysis for Speaker Verification","volume":"19","author":"Dehak","year":"2010","journal-title":"IEEE Trans. Audio Speech Lang. Process."},{"key":"ref_34","unstructured":"Kenny, P. (2010). Bayesian speaker verification with heavy-tailed priors. Odyssey: The Speaker and Language Recognition Workshop, International Speech Communication Association."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"2072","DOI":"10.1109\/TASL.2007.902870","article-title":"Fusion of heterogeneous speaker recognition systems in the STBU submission for the NIST speaker recognition evaluation 2006","volume":"15","author":"Burget","year":"2007","journal-title":"IEEE Trans. Audio Speech Signal Process."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Martin, A., and Greenberg, C. (2010, January 26\u201330). The NIST 2010 speaker recognition evaluation. Proceedings of the Interspeech 2010, Makuhari, Chiba, Japan.","DOI":"10.21437\/Interspeech.2010-722"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Martin, A., and Greenberg, C. (2009, January 6\u201310). NIST 2008 Speaker Recognition Evaluation: Performance Across Telephone and Room Microphone Channels. Proceedings of the Interspeech 2009, Brighton, UK.","DOI":"10.21437\/Interspeech.2009-679"},{"key":"ref_38","first-page":"109","article-title":"Evaluation of trace evidence in the form of multivariate data","volume":"53","author":"Aitken","year":"2004","journal-title":"Appl. Stat."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Franco-Pedroso, J., Ramos, D., and Gonzalez-Rodriguez, J. (2016). Gaussian Mixture Models of Between-Source Variation for Likelihood Ratio Computation from Multivariate Data. PLoS ONE, 11.","DOI":"10.1371\/journal.pone.0149958"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"332","DOI":"10.1037\/lhb0000134","article-title":"Lay understanding of forensic statistics: Evaluation of random match probabilities, likelihood ratios, and verbal equivalents","volume":"39","author":"Thompson","year":"2015","journal-title":"Law Hum. Behav."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"3799","DOI":"10.1016\/j.eswa.2009.11.040","article-title":"A novel measure for evaluating classifiers","volume":"37","author":"Wei","year":"2010","journal-title":"Expert Syst. Appl."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Jurman, G., Riccadonna, S., and Furlanello, C. (2012). A Comparison of MCC and CEN Error Measures in Multi-Class Prediction. PLoS ONE, 7.","DOI":"10.1371\/journal.pone.0041882"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Corzo, R., Hoffman, T., Weis, P., Franco-Pedroso, J., Ramos, D., and Almirall, J. (2018). The Use of LA-ICP-MS Databases to Estimate Likelihood Ratios for the Forensic Analysis of Glass Evidence. Talanta, in press.","DOI":"10.1016\/j.talanta.2018.02.027"},{"key":"ref_44","first-page":"2072","article-title":"Emulating DNA: Rigorous Quantification of Evidential Weight in Transparent and Testable Forensic Speaker Recognition","volume":"15","author":"Rose","year":"2007","journal-title":"IEEE Trans. Audio Speech Lang. Process."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1080\/00450618.2012.733025","article-title":"Tutorial on logistic-regression calibration and fusion: Converting a score to a likelihood ratio","volume":"45","author":"Morrison","year":"2013","journal-title":"Aust. J. Forensic Sci."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/3\/208\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T14:57:44Z","timestamp":1760194664000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/3\/208"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,3,20]]},"references-count":45,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2018,3]]}},"alternative-id":["e20030208"],"URL":"https:\/\/doi.org\/10.3390\/e20030208","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,3,20]]}}}