{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,5]],"date-time":"2026-03-05T22:21:07Z","timestamp":1772749267013,"version":"3.50.1"},"reference-count":24,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2008,7,15]],"date-time":"2008-07-15T00:00:00Z","timestamp":1216080000000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/www.springer.com\/tdm"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Mach Learn"],"published-print":{"date-parts":[[2008,9]]},"DOI":"10.1007\/s10994-008-5070-x","type":"journal-article","created":{"date-parts":[[2008,7,14]],"date-time":"2008-07-14T13:17:47Z","timestamp":1216041467000},"page":"247-262","source":"Crossref","is-referenced-by-count":43,"title":["A critical analysis of variants of the AUC"],"prefix":"10.1007","volume":"72","author":[{"given":"Stijn","family":"Vanderlooy","sequence":"first","affiliation":[]},{"given":"Eyke","family":"H\u00fcllermeier","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2008,7,15]]},"reference":[{"key":"5070_CR1","unstructured":"Asuncion, A., & Newman, D. (2007). UCI machine learning repository."},{"issue":"7","key":"5070_CR2","doi-asserted-by":"crossref","first-page":"1145","DOI":"10.1016\/S0031-3203(96)00142-2","volume":"30","author":"A. Bradley","year":"1997","unstructured":"Bradley, A. (1997). The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognition, 30(7), 1145\u20131159.","journal-title":"Pattern Recognition"},{"key":"5070_CR3","unstructured":"Brefeld, U., & Scheffer, T. (2005). AUC maximizing support vector learning. In Ferri, C., Lachiche, N., Macskassy, S., & Rakotomamonjy, A. (Eds.), Proceedings of the 2nd workshop on ROC analysis in machine learning (ROCML 2005). Bonn, Germany, August 11, 2005."},{"key":"5070_CR4","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1007\/978-3-540-74976-9_8","volume-title":"Proceedings of the 11th European conference on principles and practice of knowledge discovery in databases (PKDD 2007)","author":"T. Calders","year":"2007","unstructured":"Calders, T., & Jaroszewicz, S. (2007). Efficient AUC optimization for classification. In J. Kok, J. Koronacki, R. L. de M\u00e1ntaras, S. Matwin, D. Mladenic, & A. Skowron (Eds.), Proceedings of the 11th European conference on principles and practice of knowledge discovery in databases (PKDD 2007) (pp. 42\u201353). Warsaw, Poland, September 17\u201321, 2007. Berlin: Springer."},{"key":"5070_CR5","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1145\/1143844.1143865","volume-title":"Proceedings of the 23rd international conference on machine learning (ICML 2006)","author":"R. Caruana","year":"2006","unstructured":"Caruana, R., & Niculescu-Mizil, A. (2006). An empirical comparison of supervised learning algorithms. In W. Cohen & A. Moore (Eds.), Proceedings of the 23rd international conference on machine learning (ICML 2006) (pp. 161\u2013168). Pittsburgh, PA, USA, June 25\u201329, 2006. New York: Assoc. Comput. Mach."},{"key":"5070_CR6","volume-title":"Advances in neural information processing systems 16 (NIPS 2003)","author":"C. Cortes","year":"2003","unstructured":"Cortes, C., & Mohri, M. (2003). AUC optimization vs. error rate minimization. In S. Thrun, L. Saul, & B. Sch\u00f6lkopf (Eds.), Advances in neural information processing systems 16 (NIPS 2003). Vancouver, BC, Canada, December 8\u201313, 2003. Cambridge: MIT Press."},{"key":"5070_CR7","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1007\/978-3-540-39857-8_13","volume-title":"Proceedings of the 14th European conference on machine learning (ECML 2003)","author":"C. Ferri","year":"2003","unstructured":"Ferri, C., Flach, P., & Hern\u00e1ndez-Orallo, J. (2003). Improving the AUC of probabilistic estimation trees. In N. Lavrac, D. Gamberger, L. Todorovski, & H. Blockeel (Eds.), Proceedings of the 14th European conference on machine learning (ECML 2003) (pp. 121\u2013132). Cavtat-Dubrovnik, Croatia, September 22\u201326, 2003. Berlin: Springer."},{"key":"5070_CR8","unstructured":"Ferri, C., Flach, P., Hern\u00e1ndez-Orallo, J., & Senad, A. (2005). Modifying ROC curves to incorporate predicted probabilities. In C. Ferri, N. Lachiche, S. Macskassy, & A. Rakotomamonjy (Eds.), Proceedings of the 2nd workshop on ROC analysis in machine learning (ROCML 2005). Bonn, Germany, August 11, 2005."},{"key":"5070_CR9","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1023\/A:1009778005914","volume":"1","author":"J. Friedman","year":"1997","unstructured":"Friedman, J. (1997). On bias, variance, 0\/1-loss, and the curse-of-dimensionality. Data Mining and Knowledge Discovery, 1, 55\u201377.","journal-title":"Data Mining and Knowledge Discovery"},{"key":"5070_CR10","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1023\/A:1010920819831","volume":"45","author":"D. Hand","year":"2001","unstructured":"Hand, D., & Till, R. (2001). A simple generalisation of the area under the ROC curve for multiple class classification problems. Machine Learning, 45, 171\u2013186.","journal-title":"Machine Learning"},{"issue":"1","key":"5070_CR11","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1148\/radiology.143.1.7063747","volume":"143","author":"J. Hanley","year":"1982","unstructured":"Hanley, J., & McNeil, B. (1982). The meaning and use of the area under a receiver operator characteristic ROC curve. Radiology, 143(1), 29\u201336.","journal-title":"Radiology"},{"key":"5070_CR12","volume-title":"Proceedings of the 21st international conference on machine learning (ICML 2004)","author":"A. Herschtal","year":"2004","unstructured":"Herschtal, A., & Raskutti, B. (2004). Optimising area under the ROC curve using gradient descent. In C. Brodley (Ed.), Proceedings of the 21st international conference on machine learning (ICML 2004). Banff, Alberta, Canada, July 4\u20138, 2004. New York: Assoc. Comput. Mach."},{"key":"5070_CR13","first-page":"519","volume-title":"Proceedings of the 18th international joint conference on artificial intelligence (IJCAI 2003)","author":"C. Ling","year":"2003","unstructured":"Ling, C., Huang, J., & Zhang, H. (2003). AUC: a statistically consistent and more discriminating measure than accuracy. In G. Gottlob & T. Walsh (Eds.), Proceedings of the 18th international joint conference on artificial intelligence (IJCAI 2003) (pp. 519\u2013526). Acapulco, Mexico, August 9\u201315, 2003. Menlo Park: AAAI Press."},{"issue":"1","key":"5070_CR14","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1214\/aoms\/1177730491","volume":"18","author":"H. Mann","year":"1947","unstructured":"Mann, H., & Whitney, D. (1947). On a test whether one of two random variables is stochastically larger than the other. The Annals of Mathematical Statistics, 18(1), 50\u201360.","journal-title":"The Annals of Mathematical Statistics"},{"issue":"3","key":"5070_CR15","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1023\/A:1024099825458","volume":"52","author":"F. Provost","year":"2003","unstructured":"Provost, F., & Domingos, P. (2003). Tree-induction fir probability based ranking. Machine Learning, 52(3), 199\u2013215.","journal-title":"Machine Learning"},{"issue":"3","key":"5070_CR16","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1023\/A:1007601015854","volume":"42","author":"F. Provost","year":"2001","unstructured":"Provost, F., & Fawcett, T. (2001). Robust classification for imprecise environments. Machine Learning, 42(3), 203\u2013231.","journal-title":"Machine Learning"},{"key":"5070_CR17","first-page":"43","volume-title":"Proceedings of the 15th international conference on machine learning (ICML 1998)","author":"F. Provost","year":"1998","unstructured":"Provost, F., Fawcett, T., & Kohavi, R. (1998). The case against accuracy estimation for comparing induction algorithms. In J. Shavlik (Ed.), Proceedings of the 15th international conference on machine learning (ICML 1998) (pp. 43\u201348). Madison, WI, USA, July 24\u201327, 1998. San Mateo: Morgan Kaufmann."},{"key":"5070_CR18","unstructured":"Rakotomamonjy, A. (2004). Optimizing area under ROC curve with SVMs. In J. Hern\u00e1ndez-Orallo, C. Ferri, N. Lachiche, & P. Flach (Eds.), Proceedings of the 1st workshop on ROC analysis and artificial intelligence (ROCAI 2004) (pp. 71\u201380). Valencia, Spain, August 22, 2004."},{"key":"5070_CR19","doi-asserted-by":"crossref","first-page":"347","DOI":"10.1007\/978-3-540-74958-5_33","volume-title":"Proceedings of the 18th European conference on machine learning (ECML 2007)","author":"H. Steck","year":"2007","unstructured":"Steck, H. (2007). Hinge rank loss and the area under the ROC curve. In J. Kok, J. Koronacki, R. L. de M\u00e1ntaras, S. Matwin, D. Mladenic, & A. Skowron (Eds.), Proceedings of the 18th European conference on machine learning (ECML 2007) (pp. 347\u2013358). Warsaw, Poland, September 17\u201321, 2007. Berlin: Springer."},{"key":"5070_CR20","first-page":"224","volume-title":"Proceedings of the 17th Belgium-Netherlands conference on artificial intelligence (BNAIC 2005)","author":"D. Tax","year":"2005","unstructured":"Tax, D., & Veenman, C. (2005). Tuning the hyperparameter of an AUC-optimized classifier. In K. Verbeeck, K. Tuyls, A. Nowe, B. Manderick, & B. Kuijpers (Eds.), Proceedings of the 17th Belgium-Netherlands conference on artificial intelligence (BNAIC 2005) (pp. 224\u2013231). Brussels, Belgium, October 17\u201318, 2005. Brussels: Royal Flemish Academy of Belgium for Science and Arts."},{"key":"5070_CR21","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1109\/ICPR.2006.766","volume-title":"Proceedings of the 18th international conference on pattern recognition (ICPR 2006)","author":"D. Tax","year":"2006","unstructured":"Tax, D., Duin, R., & Arzhaeva, Y. (2006). Linear model combining by optimizing the area under the ROC curve. In Y. Tang, P. Wang, G. Lorette, D. Yeung, & H. Yan (Eds.), Proceedings of the 18th international conference on pattern recognition (ICPR 2006) (pp. 119\u2013122). Hong Kong, China, August 20\u201324, 2006. Los Alamitos: IEEE Comput. Soc."},{"key":"5070_CR22","volume-title":"Data mining: practical machine learning tools and techniques","author":"I. Witten","year":"2005","unstructured":"Witten, I., & Frank, E. (2005). Data mining: practical machine learning tools and techniques (2nd ed.). San\u00a0Mateo: Morgan\u00a0Kaufmann.","edition":"2"},{"key":"5070_CR23","doi-asserted-by":"crossref","first-page":"478","DOI":"10.1007\/978-3-540-74958-5_44","volume-title":"Proceedings of the 18th European conference on machine learning (ECML 2007)","author":"S. Wu","year":"2007","unstructured":"Wu, S., Flach, P., & Ferri, C. (2007). An improved model selection heuristic for AUC. In J. Kok, J. Koronacki, R. L. de M\u00e1ntaras, S. Matwin, D. Mladenic, & A. Skowron (Eds.), Proceedings of the 18th European conference on machine learning (ECML 2007) (pp. 478\u2013489). Warsaw, Poland, September 17\u201321, 2007. Berlin: Springer."},{"key":"5070_CR24","first-page":"848","volume-title":"Proceedings of the 20th international conference on machine learning (ICML 2003)","author":"L. Yan","year":"2003","unstructured":"Yan, L., Dodier, R., Mozer, M., & Wolniewicz, R. (2003). Optimizing classifier performance via an approximation to the Wilcoxon-Mann-Whitney statistic. In T. Fawcett & N. Mishra (Eds.), Proceedings of the 20th international conference on machine learning (ICML 2003) (pp. 848\u2013855). Washington, DC, USA, August 21\u201324, 2003. Menlo Park: AAAI Press."}],"container-title":["Machine Learning"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-008-5070-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s10994-008-5070-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-008-5070-x","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,9,13]],"date-time":"2021-09-13T03:37:29Z","timestamp":1631504249000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s10994-008-5070-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2008,7,15]]},"references-count":24,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2008,9]]}},"alternative-id":["5070"],"URL":"https:\/\/doi.org\/10.1007\/s10994-008-5070-x","relation":{},"ISSN":["0885-6125","1573-0565"],"issn-type":[{"value":"0885-6125","type":"print"},{"value":"1573-0565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2008,7,15]]}}}