{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,3]],"date-time":"2026-04-03T00:08:25Z","timestamp":1775174905994,"version":"3.50.1"},"reference-count":35,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,1,17]],"date-time":"2024-01-17T00:00:00Z","timestamp":1705449600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,1,17]],"date-time":"2024-01-17T00:00:00Z","timestamp":1705449600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"CEF TELECOM-CE","award":["INEA\/CEF\/ICT\/A2020\/2276680"],"award-info":[{"award-number":["INEA\/CEF\/ICT\/A2020\/2276680"]}]},{"DOI":"10.13039\/501100004769","name":"Universit\u00e0 degli Studi di Pavia","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100004769","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Adv Data Anal Classif"],"published-print":{"date-parts":[[2025,3]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>A key point to assess statistical forecasts is the evaluation of their predictive accuracy. Recently, a new measure, called Rank Graduation Accuracy (RGA), based on the concordance between the ranks of the predicted values and the ranks of the actual values of a series of observations to be forecast, was proposed for the assessment of the quality of the predictions. In this paper, we demonstrate that, in a classification perspective, when the response to be predicted is binary, the RGA coincides both with the AUROC and the Wilcoxon-Mann\u2013Whitney statistic, and can be employed to evaluate the accuracy of probability forecasts. When the response to be predicted is real valued, the RGA can still be applied, differently from the AUROC, and similarly to measures such as the RMSE. Differently from the RMSE, the RGA measure evaluates point predictions in terms of their ranks, rather than in terms of their values, improving robustness.<\/jats:p>","DOI":"10.1007\/s11634-023-00574-2","type":"journal-article","created":{"date-parts":[[2024,1,17]],"date-time":"2024-01-17T12:02:16Z","timestamp":1705492936000},"page":"67-93","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":58,"title":["RGA: a unified measure of predictive accuracy"],"prefix":"10.1007","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4198-0127","authenticated-orcid":false,"given":"Paolo","family":"Giudici","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0386-7975","authenticated-orcid":false,"given":"Emanuela","family":"Raffinetti","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,1,17]]},"reference":[{"key":"574_CR1","doi-asserted-by":"publisher","first-page":"1993","DOI":"10.3150\/20-BEJ1298","volume":"27","author":"JR Bracher","year":"2021","unstructured":"Bracher JR, Gneiting T (2021) Scoring interval forecasts: equal-tailed, shortest, and modal interval. Bernoulli 27:1993\u20132010. https:\/\/doi.org\/10.3150\/20-BEJ1298","journal-title":"Bernoulli"},{"key":"574_CR2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pcbi.1008618","volume":"17","author":"JR Bracher","year":"2021","unstructured":"Bracher JR, Ray EL, Gneiting T, Reich NG (2021) Evaluating epidemic forecasts in an interval format. PLOS Comput Biol 17:1\u201315. https:\/\/doi.org\/10.1371\/journal.pcbi.1008618","journal-title":"PLOS Comput Biol"},{"key":"574_CR3","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1175\/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2","volume":"78","author":"GW Brier","year":"1950","unstructured":"Brier GW (1950) Verification of forecasts expressed in terms of probability. Mon Weather Rev 78:1\u20133. https:\/\/doi.org\/10.1175\/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2","journal-title":"Mon Weather Rev"},{"key":"574_CR4","doi-asserted-by":"publisher","first-page":"677","DOI":"10.1007\/s11634-019-00354-x","volume":"14","author":"I Chaabane","year":"2020","unstructured":"Chaabane I, Guermazi R, Hammami M (2020) Enhancing techniques for learning decision trees from imbalanced data. Adv Data Anal Class 14:677\u2013745. https:\/\/doi.org\/10.1007\/s11634-019-00354-x","journal-title":"Adv Data Anal Class"},{"key":"574_CR5","doi-asserted-by":"publisher","first-page":"837","DOI":"10.2307\/2531595","volume":"44","author":"ER DeLong","year":"1988","unstructured":"DeLong ER, DeLong DM, Clarke-Pearson DL (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44:837\u2013845. https:\/\/doi.org\/10.2307\/2531595","journal-title":"Biometrics"},{"key":"574_CR6","doi-asserted-by":"publisher","first-page":"253","DOI":"10.1080\/07350015.1995.10524599","volume":"13","author":"FX Diebold","year":"1995","unstructured":"Diebold FX, Mariano R (1995) Comparing predictive accuracy. J Bus Econ Stat 13:253\u2013263. https:\/\/doi.org\/10.1080\/07350015.1995.10524599","journal-title":"J Bus Econ Stat"},{"key":"574_CR7","doi-asserted-by":"publisher","first-page":"514","DOI":"10.1198\/jcgs.2010.06089","volume":"19","author":"E Dusseldorp","year":"2010","unstructured":"Dusseldorp E, Conversano C, Van Os BJ (2010) Combining an additive and tree-based regression model simultaneously: STIMA. J Comput Graph Stat 19:514\u2013530. https:\/\/doi.org\/10.1198\/jcgs.2010.06089","journal-title":"J Comput Graph Stat"},{"key":"574_CR8","doi-asserted-by":"publisher","first-page":"586","DOI":"10.1214\/aos\/1176345462","volume":"9","author":"E Efron","year":"1981","unstructured":"Efron E, Stein C (1981) The jackknife estimate of variance. Ann Stat 9:586\u2013596. https:\/\/doi.org\/10.1214\/aos\/1176345462","journal-title":"Ann Stat"},{"key":"574_CR9","doi-asserted-by":"publisher","first-page":"248","DOI":"10.1080\/00273171.2014.973099","volume":"50","author":"PA Ferrari","year":"2015","unstructured":"Ferrari PA, Raffinetti E (2015) A different approach to dependence analysis. Multivar Behav Res 50:248\u2013264. https:\/\/doi.org\/10.1080\/00273171.2014.973099","journal-title":"Multivar Behav Res"},{"key":"574_CR10","doi-asserted-by":"publisher","first-page":"823","DOI":"10.1007\/s11634-021-00455-6","volume":"16","author":"S Friedrich","year":"2022","unstructured":"Friedrich S et al (2022) Is there a role for statistics in artificial intelligence? Adv Data Anal Class 16:823\u2013846. https:\/\/doi.org\/10.1007\/s11634-021-00455-6","journal-title":"Adv Data Anal Class"},{"key":"574_CR11","doi-asserted-by":"publisher","first-page":"382","DOI":"10.1080\/00273171.2014.973099","volume":"XV","author":"K Gajowniczek","year":"2014","unstructured":"Gajowniczek K, Ti Zabkowsk, Szupiluk R (2014) Estimating the ROC curve and its significance for classification models\u2019 assessment. Quant Meth Econom XV:382\u2013391. https:\/\/doi.org\/10.1080\/00273171.2014.973099","journal-title":"Quant Meth Econom"},{"key":"574_CR12","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1016\/j.spl.2010.10.005","volume":"81","author":"P Giudici","year":"2011","unstructured":"Giudici P, Raffinetti E (2011) On the Gini measure decomposition. Stat Prob Lett 81:133\u2013139. https:\/\/doi.org\/10.1016\/j.spl.2010.10.005","journal-title":"Stat Prob Lett"},{"key":"574_CR13","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2020.114104","volume":"167","author":"P Giudici","year":"2021","unstructured":"Giudici P, Raffinetti E (2021) Shapley-Lorenz explainable artificial intelligence. Expert Syst Appl 167:114104. https:\/\/doi.org\/10.1016\/j.eswa.2020.114104","journal-title":"Expert Syst Appl"},{"key":"574_CR14","doi-asserted-by":"publisher","first-page":"1318","DOI":"10.1002\/qre.2939","volume":"38","author":"P Giudici","year":"2022","unstructured":"Giudici P, Raffinetti E (2022) Explainable AI methods in cyber risk management. Qual Reliab Eng Int. 38:1318\u20131326. https:\/\/doi.org\/10.1002\/qre.2939","journal-title":"Qual Reliab Eng Int."},{"key":"574_CR15","doi-asserted-by":"publisher","DOI":"10.1016\/j.seps.2023.101560","author":"P Giudici","year":"2023","unstructured":"Giudici P, Gramegna A, Raffinetti E (2023) Machine learning classification model comparison. Socio Econ Plan Sci (Article in Press). https:\/\/doi.org\/10.1016\/j.seps.2023.101560","journal-title":"Socio Econ Plan Sci (Article in Press)"},{"key":"574_CR16","doi-asserted-by":"publisher","first-page":"359","DOI":"10.1198\/016214506000001437","volume":"102","author":"T Gneiting","year":"2007","unstructured":"Gneiting T, Raftery AE (2007) Strictly proper scoring rules, prediction, and estimation. J Am Stat Assoc 102:359\u2013378. https:\/\/doi.org\/10.1198\/016214506000001437","journal-title":"J Am Stat Assoc"},{"key":"574_CR17","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1007\/s11749-008-0114-x","volume":"17","author":"T Gneiting","year":"2008","unstructured":"Gneiting T, Stanberry LI, Grimit EP, Held L, Johnson NA (2008) Assessing probabilistic forecasts of multivariate quantities, with an application to ensemble predictions of surface winds. With Discuss Test 17:211\u2013264. https:\/\/doi.org\/10.1007\/s11749-008-0114-x","journal-title":"With Discuss Test"},{"key":"574_CR18","doi-asserted-by":"publisher","first-page":"746","DOI":"10.1198\/jasa.2011.r10138","volume":"106","author":"T Gneiting","year":"2011","unstructured":"Gneiting T (2011) Making and evaluating point forecasts. J Am Stat Assoc 106:746\u2013762. https:\/\/doi.org\/10.1198\/jasa.2011.r10138","journal-title":"J Am Stat Assoc"},{"key":"574_CR19","doi-asserted-by":"publisher","first-page":"411","DOI":"10.1198\/jbes.2010.08110","volume":"29","author":"T Gneiting","year":"2011","unstructured":"Gneiting T, Ranjan R (2011) Comparing density forecasts using threshold- and quantile-weighted scoring rules. J Bus Econ Stat 29:411\u2013422. https:\/\/doi.org\/10.1198\/jbes.2010.08110","journal-title":"J Bus Econ Stat"},{"key":"574_CR20","volume-title":"Principles of data mining","author":"DJ Hand","year":"2001","unstructured":"Hand DJ, Mannila H, Smyth P (2001) Principles of data mining. MIT Press, Adaptive Computation and Machine Learning Series"},{"key":"574_CR21","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1023\/A:1010920819831","volume":"45","author":"DJ Hand","year":"2011","unstructured":"Hand DJ, Till RJ (2011) A simple generalisation of the area under the ROC curve for multiple class classification problem. Mach Learn 45:171\u2013186. https:\/\/doi.org\/10.1023\/A:1010920819831","journal-title":"Mach Learn"},{"key":"574_CR22","doi-asserted-by":"publisher","first-page":"109","DOI":"10.1007\/s11634-021-00490-3","volume":"17","author":"DJ Hand","year":"2023","unstructured":"Hand DJ, Anagnostopoulos C (2023) Notes on the H-measure of classifier performance. Adv Data Anal Class 17:109\u2013124. https:\/\/doi.org\/10.1007\/s11634-021-00490-3","journal-title":"Adv Data Anal Class"},{"key":"574_CR23","doi-asserted-by":"publisher","first-page":"293","DOI":"10.1214\/aoms\/1177730196","volume":"19","author":"W Hoeffding","year":"1948","unstructured":"Hoeffding W (1948) A class of statistics with asymptotically normal distribution. Ann Math Stat 19:293\u2013325. https:\/\/doi.org\/10.1214\/aoms\/1177730196","journal-title":"Ann Math Stat"},{"key":"574_CR24","doi-asserted-by":"publisher","first-page":"763","DOI":"10.3390\/forecast3040047","volume":"3","author":"T-H Kang","year":"2021","unstructured":"Kang T-H, Sharma A, Marshall L (2021) Assessing goodness of fit for verifying probabilistic forecasts. Forecasting 3:763\u2013773. https:\/\/doi.org\/10.3390\/forecast3040047","journal-title":"Forecasting"},{"key":"574_CR25","doi-asserted-by":"publisher","first-page":"729","DOI":"10.1002\/(sici)1097-0258(19970415)16:7<729::aid-sim491>3.0.co;2-a","volume":"16","author":"WC Lee","year":"1997","unstructured":"Lee WC (1997) Characterising exposure-disease association in human population using the Lorenz curve and the Gini index. Stat Med 16:729\u2013739. https:\/\/doi.org\/10.1002\/(sici)1097-0258(19970415)16:7<729::aid-sim491>3.0.co;2-a","journal-title":"Stat Med"},{"key":"574_CR26","doi-asserted-by":"publisher","first-page":"209","DOI":"10.1080\/15225437.1905.10503443","volume":"9","author":"MO Lorenz","year":"1905","unstructured":"Lorenz MO (1905) Methods of measuring the concentration of wealth. Publ Am Stat Assoc 9:209\u2013219. https:\/\/doi.org\/10.1080\/15225437.1905.10503443","journal-title":"Publ Am Stat Assoc"},{"key":"574_CR27","doi-asserted-by":"crossref","unstructured":"Marshall AW, Olkin I, Arnold BC (2011) Inequalities: theory of majorization and its applications. Springer","DOI":"10.1007\/978-0-387-68276-1"},{"key":"574_CR28","doi-asserted-by":"publisher","first-page":"2145","DOI":"10.1256\/003590002320603584","volume":"128","author":"SJ Mason","year":"2002","unstructured":"Mason SJ, Graham NE (2002) Areas beneath the relative operating characteristics (ROC) and relative operating levels (ROL) curves: Statistical significance and interpretation. Quart J Royal Meteorol Soc 128:2145\u20132166. https:\/\/doi.org\/10.1256\/003590002320603584","journal-title":"Quart J Royal Meteorol Soc"},{"key":"574_CR29","doi-asserted-by":"publisher","first-page":"705","DOI":"10.1016\/j.ijforecast.2021.11.001","volume":"38","author":"F Petropoulos","year":"2022","unstructured":"Petropoulos F, Apiletti D, Assimakopoulo V et al (2022) Forecasting: theory and practice. Int J Forecast 38:705\u2013871. https:\/\/doi.org\/10.1016\/j.ijforecast.2021.11.001","journal-title":"Int J Forecast"},{"key":"574_CR31","doi-asserted-by":"publisher","first-page":"131","DOI":"10.1007\/s11135-023-01613-y","volume":"57","author":"E Raffinetti","year":"2023","unstructured":"Raffinetti E (2023) A rank graduation accuracy measure to mitigate artificial intelligence risks. Qual Quant 57:131\u2013150. https:\/\/doi.org\/10.1007\/s11135-023-01613-y","journal-title":"Qual Quant"},{"key":"574_CR30","doi-asserted-by":"crossref","unstructured":"Raffinetti E, Giudici P (2012) Multivariate Ranks-Based Concordance Indexes. In: Di Ciaccio, A., Coli, M., Ibanez, J.M.A (eds) Advanced Statistical Methods for the Analysis of Large Data-Sets, Series: Studies in Theoretical and Applied Statistics. Springer-Verlag Berlin Heidelberg, pp. 465-473","DOI":"10.1007\/978-3-642-21037-2_42"},{"key":"574_CR32","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1007\/s40300-019-00160-7","volume":"77","author":"E Schechtman","year":"2019","unstructured":"Schechtman E, Schechtman G (2019) The relationship between Gini terminology and the ROC curve. Metron 77:171\u2013178. https:\/\/doi.org\/10.1007\/s40300-019-00160-7","journal-title":"Metron"},{"key":"574_CR33","doi-asserted-by":"publisher","first-page":"683","DOI":"10.1007\/s11634-017-0295-9","volume":"12","author":"JM Vivo","year":"2018","unstructured":"Vivo JM, Franco M, Vicari D (2018) Rethinking an ROC partial area index for evaluating the classification performance at a high specificity range. Adv Data Anal Class 12:683\u2013704. https:\/\/doi.org\/10.1007\/s11634-017-0295-9","journal-title":"Adv Data Anal Class"},{"key":"574_CR34","doi-asserted-by":"publisher","first-page":"785","DOI":"10.1007\/s11634-020-00419-2","volume":"14","author":"S Voj\u00ed\u0159","year":"2020","unstructured":"Voj\u00ed\u0159 S, Kliegr T (2020) Editable machine learning models? A rule-based framework for user studies of explainability. Adv Data Anal Class 14:785\u2013799. https:\/\/doi.org\/10.1007\/s11634-020-00419-2","journal-title":"Adv Data Anal Class"},{"key":"574_CR35","volume-title":"Statistical Methods in the Atmospheric Sciences","author":"DS Wilks","year":"2011","unstructured":"Wilks DS (2011) Statistical Methods in the Atmospheric Sciences. Elsevier Academic Press, Oxford"}],"container-title":["Advances in Data Analysis and Classification"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11634-023-00574-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11634-023-00574-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11634-023-00574-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,30]],"date-time":"2025-03-30T10:01:54Z","timestamp":1743328914000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11634-023-00574-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,17]]},"references-count":35,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,3]]}},"alternative-id":["574"],"URL":"https:\/\/doi.org\/10.1007\/s11634-023-00574-2","relation":{},"ISSN":["1862-5347","1862-5355"],"issn-type":[{"value":"1862-5347","type":"print"},{"value":"1862-5355","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,1,17]]},"assertion":[{"value":"12 October 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 November 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 January 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The Authors declare they have not competing financial interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}