{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,12]],"date-time":"2026-07-12T02:32:26Z","timestamp":1783823546957,"version":"3.55.0"},"reference-count":25,"publisher":"Springer Science and Business Media LLC","issue":"10","license":[{"start":{"date-parts":[[2022,1,31]],"date-time":"2022-01-31T00:00:00Z","timestamp":1643587200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,1,31]],"date-time":"2022-01-31T00:00:00Z","timestamp":1643587200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100012818","name":"Madrid Autonomous Community","doi-asserted-by":"crossref","award":["IND2018\/TIC-9665"],"award-info":[{"award-number":["IND2018\/TIC-9665"]}],"id":[{"id":"10.13039\/100012818","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100006542","name":"Agencia Antidroga de la Comunidad de Madrid","doi-asserted-by":"crossref","award":["IND2018\/TIC-9665"],"award-info":[{"award-number":["IND2018\/TIC-9665"]}],"id":[{"id":"10.13039\/501100006542","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Spanish Science and Innovation","award":["RTC-2017-6253-1"],"award-info":[{"award-number":["RTC-2017-6253-1"]}]},{"name":"Spanish Science and Innovation","award":["RTI-2018-094269-B-I00"],"award-info":[{"award-number":["RTI-2018-094269-B-I00"]}]},{"name":"Spanish Science and Innovation","award":["RTI-2018-094269-B-I00"],"award-info":[{"award-number":["RTI-2018-094269-B-I00"]}]},{"DOI":"10.13039\/501100007511","name":"Universidad Rey Juan Carlos","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100007511","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Appl Intell"],"published-print":{"date-parts":[[2022,8]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Several performance metrics are currently available to evaluate the performance of <jats:italic>Machine Learning (ML)<\/jats:italic> models in classification problems. <jats:italic>ML<\/jats:italic>\u00a0models are usually assessed using a single measure because it facilitates the comparison between several models. However, there is no silver bullet since each performance metric emphasizes a different aspect of the classification. Thus, the choice depends on the particular requirements and characteristics of the problem. An additional problem arises in multi-class classification problems, since most of the well-known metrics are only directly applicable to binary classification problems. In this paper, we propose the <jats:italic>General Performance Score (GPS)<\/jats:italic>, a methodological approach to build performance metrics for binary and multi-class classification problems. The basic idea behind <jats:italic>GPS<\/jats:italic> is to combine a set of individual metrics, penalising low values in any of them. Thus, users can combine several performance metrics that are relevant in the particular problem based on their preferences obtaining a conservative combination. Different <jats:italic>GPS<\/jats:italic>-based performance metrics are compared with alternatives in classification problems using real and simulated datasets. The metrics built using the proposed method improve the stability and explainability of the usual performance metrics. Finally, the <jats:italic>GPS<\/jats:italic> brings benefits in both new research lines and practical usage, where performance metrics tailored for each particular problem are considered.<\/jats:p>","DOI":"10.1007\/s10489-021-03041-7","type":"journal-article","created":{"date-parts":[[2022,1,31]],"date-time":"2022-01-31T18:02:42Z","timestamp":1643652162000},"page":"12049-12063","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":130,"title":["General Performance Score for classification problems"],"prefix":"10.1007","volume":"52","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5197-2932","authenticated-orcid":false,"given":"Isaac Mart\u00edn","family":"De Diego","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9133-4596","authenticated-orcid":false,"given":"Ana R.","family":"Redondo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4140-1551","authenticated-orcid":false,"given":"Rub\u00e9n R.","family":"Fern\u00e1ndez","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9698-3213","authenticated-orcid":false,"given":"Jorge","family":"Navarro","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1415-1961","authenticated-orcid":false,"given":"Javier M.","family":"Moguerza","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,1,31]]},"reference":[{"key":"3041_CR1","volume-title":"Pattern recognition and machine learning","author":"CM Bishop","year":"2006","unstructured":"Bishop CM (2006) Pattern recognition and machine learning. Springer, New York"},{"key":"3041_CR2","unstructured":"Bland M (2008) Cohen\u2019s kappa. University of York Department of Health Sciences https:\/\/www.usersyorkacuk\/~ mb55\/msc\/clinimet\/week4\/kappash2.pdf. Accessed 13 Feb 2014"},{"issue":"7","key":"3041_CR3","doi-asserted-by":"publisher","first-page":"1145","DOI":"10.1016\/S0031-3203(96)00142-2","volume":"30","author":"AP Bradley","year":"1997","unstructured":"Bradley AP (1997) The use of the area under the roc curve in the evaluation of machine learning algorithms. Pattern Recognition 30(7):1145\u20131159","journal-title":"Pattern Recognition"},{"issue":"1","key":"3041_CR4","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1186\/s12864-019-6413-7","volume":"21","author":"D Chicco","year":"2020","unstructured":"Chicco D, Jurman G (2020) The advantages of the matthews correlation coefficient (mcc) over f1 score and accuracy in binary classification evaluation. BMC Genomics 21(1):6","journal-title":"BMC Genomics"},{"issue":"3","key":"3041_CR5","doi-asserted-by":"publisher","first-page":"247","DOI":"10.1016\/0149-7189(82)90076-3","volume":"5","author":"P Cohen","year":"1982","unstructured":"Cohen P (1982) To be or not to be: Control and balancing of type i and type ii errors. Evaluation and Program Planning 5(3):247\u2013253","journal-title":"Evaluation and Program Planning"},{"key":"3041_CR6","first-page":"1","volume":"7","author":"J Dem\u0161ar","year":"2006","unstructured":"Dem\u0161ar J (2006) Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7:1\u201330","journal-title":"Journal of Machine Learning Research"},{"key":"3041_CR7","unstructured":"Dua D, Graff C (2017) UCI machine learning repository. http:\/\/archive.ics.uci.edu\/ml"},{"issue":"8","key":"3041_CR8","doi-asserted-by":"publisher","first-page":"861","DOI":"10.1016\/j.patrec.2005.10.010","volume":"27","author":"T Fawcett","year":"2006","unstructured":"Fawcett T (2006) An introduction to roc analysis. Pattern Recognition Letters 27(8):861\u2013874","journal-title":"Pattern Recognition Letters"},{"key":"3041_CR9","doi-asserted-by":"crossref","unstructured":"Goodall DW (1967) The distribution of the matching coefficient. Biometrics, 647\u2013656","DOI":"10.2307\/2528419"},{"issue":"5\u20136","key":"3041_CR10","doi-asserted-by":"publisher","first-page":"367","DOI":"10.1016\/j.compbiolchem.2004.09.006","volume":"28","author":"J Gorodkin","year":"2004","unstructured":"Gorodkin J (2004) Comparing two k-category assignments by a k-category correlation coefficient. Computational biology and chemistry 28(5\u20136):367\u2013374","journal-title":"Computational biology and chemistry"},{"key":"3041_CR11","unstructured":"Grandini M, Bagli E, Visani G (2020) Metrics for multi-class classification: an overview. arXiv:200805756"},{"issue":"2\u20133","key":"3041_CR12","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1023\/A:1012801612483","volume":"17","author":"M Halkidi","year":"2001","unstructured":"Halkidi M, Batistakis Y, Vazirgiannis M (2001) On clustering validation techniques. Journal of Intelligent Information Systems 17(2\u20133):107\u2013145","journal-title":"Journal of Intelligent Information Systems"},{"issue":"4","key":"3041_CR13","doi-asserted-by":"publisher","first-page":"932","DOI":"10.1007\/s00330-014-3487-0","volume":"25","author":"S Halligan","year":"2015","unstructured":"Halligan S, Altman DG, Mallett S (2015) Disadvantages of using the area under the receiver operating characteristic curve to assess imaging tests: a discussion and proposal for an alternative approach. European Radiology 25(4):932\u2013939","journal-title":"European Radiology"},{"key":"3041_CR14","doi-asserted-by":"publisher","first-page":"106214","DOI":"10.1016\/j.knosys.2020.106214","volume":"209","author":"C Hu","year":"2020","unstructured":"Hu C, Wang Y, Gu J (2020) Cross-domain intelligent fault classification of bearings based on tensor-aligned invariant subspace learning and two-dimensional convolutional neural networks. Knowledge-Based Systems 209:106214","journal-title":"Knowledge-Based Systems"},{"issue":"4","key":"3041_CR15","doi-asserted-by":"publisher","first-page":"2609","DOI":"10.1007\/s10489-020-02011-9","volume":"51","author":"C Hu","year":"2021","unstructured":"Hu C, He S, Wang Y (2021) A classification method to detect faults in a rotating machinery based on kernelled support tensor machine and multilinear principal component analysis. Applied Intelligence 51(4):2609\u20132621","journal-title":"Applied Intelligence"},{"issue":"2","key":"3041_CR16","doi-asserted-by":"publisher","first-page":"442","DOI":"10.1016\/0005-2795(75)90109-9","volume":"405","author":"BW Matthews","year":"1975","unstructured":"Matthews BW (1975) Comparison of the predicted and observed secondary structure of t4 phage lysozyme. Biochimica et Biophysica Acta (BBA)-Protein Structure 405(2):442\u2013451","journal-title":"Biochimica et Biophysica Acta (BBA)-Protein Structure"},{"issue":"4","key":"3041_CR17","doi-asserted-by":"publisher","first-page":"445","DOI":"10.1214\/aoms\/1177731830","volume":"11","author":"N Norris","year":"1940","unstructured":"Norris N (1940) The standard errors of the geometric and harmonic means and their application to index numbers. The Annals of Mathematical Statistics 11(4):445\u2013448","journal-title":"The Annals of Mathematical Statistics"},{"key":"3041_CR18","unstructured":"Ogbi MSZ (2012) A mathematical property of the harmonic mean. In: The 6th international days of statistics and economics. Prague University of Economics and Business, pp 873\u2013877"},{"key":"3041_CR19","unstructured":"Opitz J, Burst S (2019) Macro f1 and macro f1. arXiv:191103347"},{"key":"3041_CR20","unstructured":"Powers DM (2020) Evaluation: from precision, recall and f-measure to roc, informedness, markedness and correlation. arXiv:201016061"},{"key":"3041_CR21","first-page":"2123","volume":"27","author":"S Puthiya Parambath","year":"2014","unstructured":"Puthiya Parambath S, Usunier N, Grandvalet Y (2014) Optimizing f-measures by cost-sensitive classification. Advances in Neural Information Processing Systems 27:2123\u20132131","journal-title":"Advances in Neural Information Processing Systems"},{"key":"3041_CR22","doi-asserted-by":"crossref","unstructured":"Redondo AR, Navarro J, Fern\u00e1ndez RR, de\u00a0Diego IM, Moguerza JM, Fern\u00e1ndez-Mu\u00f1oz JJ (2020) Unified performance measure for binary classification problems. In: International conference on intelligent data engineering and automated learning. Springer, pp 104\u2013112","DOI":"10.1007\/978-3-030-62365-4_10"},{"key":"3041_CR23","unstructured":"Sasaki Y, Fellow R (2007) The truth of the f-measure, manchester: Mib-school of computer science. University of Manchester p 25"},{"issue":"4","key":"3041_CR24","doi-asserted-by":"publisher","first-page":"427","DOI":"10.1016\/j.ipm.2009.03.002","volume":"45","author":"M Sokolova","year":"2009","unstructured":"Sokolova M, Lapalme G (2009) A systematic analysis of performance measures for classification tasks. Information Processing & Management 45(4):427\u2013437","journal-title":"Information Processing & Management"},{"issue":"1","key":"3041_CR25","first-page":"168","volume":"17","author":"A Tharwat","year":"2020","unstructured":"Tharwat A (2020) Classification assessment methods. New England Journal of Entrepreneurship 17(1):168\u2013192","journal-title":"New England Journal of Entrepreneurship"}],"container-title":["Applied Intelligence"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10489-021-03041-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10489-021-03041-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10489-021-03041-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,6,28]],"date-time":"2022-06-28T05:35:49Z","timestamp":1656394549000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10489-021-03041-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,31]]},"references-count":25,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2022,8]]}},"alternative-id":["3041"],"URL":"https:\/\/doi.org\/10.1007\/s10489-021-03041-7","relation":{},"ISSN":["0924-669X","1573-7497"],"issn-type":[{"value":"0924-669X","type":"print"},{"value":"1573-7497","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1,31]]},"assertion":[{"value":"24 November 2021","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"31 January 2022","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}