{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:36:02Z","timestamp":1760146562734,"version":"build-2065373602"},"reference-count":30,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2024,11,17]],"date-time":"2024-11-17T00:00:00Z","timestamp":1731801600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Scientific and Technological Research Council of Turkey (TUBITAK)"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Performance metrics are measures of success or performance that can be used to evaluate how well a model makes accurate predictions or classifications. However, there is no single measure since each performance metric emphasizes a different classification aspect. Model selection procedures based on information criteria offer a quantitative measure that balances model complexity with goodness of fit, providing a better alternative to classical approaches. In this paper, we introduce and develop a novel Information Complexity\u2013Receiver Operating Characteristic, abbreviated as ICOMP-ROC, criterion approach to fit and study the performance of ROC curve models. We construct and derive the Universal ROC (UROC) for a combination of sixteen Bi-distributional ROC models to choose the best Bi-distributional ROC by minimizing the ICOMP-ROC criterion. We conduct large-scale Monte Carlo simulations using the sixteen Bi-distributional ROC models with the Normal\u2013Normal and Weibull\u2013Gamma pairs as the pseudo-true ROC models. We report the frequency of hits of the ICOMP-ROC criterion, showing its remarkable recovery rate. In addition to Bi-distributional fitting, we consider a high-dimensional real Magnetic Resonance Imaging (MRI) of the Brain dataset and Wisconsin Breast Cancer (WBC) dataset to study the performance of the common performance metrics and the ICOMP-ROC criterion using several machine learning (ML) classification algorithms. We use the genetic algorithm (GA) to reduce the dimensions of these two datasets to choose the best subset of the features to study and compare the performance of the newly proposed ICOMP-ROC criterion along with the traditional performance metrics. The choice of a suitable metric is not just contingent upon the ML model used, but it also depends upon the complexity and high dimensionality of the input datasets, since the traditional performance metrics give different results and have inherent limitations. Our numerical results show the consistency and reliability of the ICOMP-ROC criterion over the traditional performance metrics as a clever model selection criterion to choose the best fitting Bi-distributional ROC model and the best classification algorithm among the ones considered. This shows the utility and the versatility of our newly proposed approach in ROC curve modeling that integrates and robustifies currently used procedures.<\/jats:p>","DOI":"10.3390\/e26110988","type":"journal-article","created":{"date-parts":[[2024,11,20]],"date-time":"2024-11-20T09:14:38Z","timestamp":1732094078000},"page":"988","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["A Novel Information Complexity Approach to Score Receiver Operating Characteristic (ROC) Curve Modeling"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8469-9247","authenticated-orcid":false,"given":"Aylin","family":"Gocoglu","sequence":"first","affiliation":[{"name":"Department of Statistics, The Graduate School of Natural and Applied Sciences, Dokuz Eylul University, Izmir 35390, Turkey"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5394-4721","authenticated-orcid":false,"given":"Neslihan","family":"Demirel","sequence":"additional","affiliation":[{"name":"Department of Statistics, Faculty of Science, Dokuz Eylul University, Izmir 35390, Turkey"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8238-7469","authenticated-orcid":false,"given":"Hamparsum","family":"Bozdogan","sequence":"additional","affiliation":[{"name":"Department of Business Analytics and Statistics, University of Tennessee, Knoxville, TN 37996, USA"}]}],"member":"1968","published-online":{"date-parts":[[2024,11,17]]},"reference":[{"key":"ref_1","unstructured":"Egan, J.P. (1975). Signal detection theory and roc analysis. Second International Symposium on Information Theory, Academic Press."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"264","DOI":"10.1016\/j.jocn.2008.04.010","article-title":"Direct comparison of thallium-201 and technetium-99m MIBI SPECT of a glioma by Receiver Operating Characteristic analysis","volume":"16","author":"Shibata","year":"2009","journal-title":"J. Clin. Neurosci."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1126","DOI":"10.1016\/j.patcog.2008.08.028","article-title":"Microcalcification classification assisted by content-based image retrieval for breast cancer diagnosis","volume":"42","author":"Wei","year":"2009","journal-title":"Pattern Recognit."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2143","DOI":"10.1002\/(SICI)1097-0258(19971015)16:19<2143::AID-SIM655>3.0.CO;2-3","article-title":"Smooth non-parametric Receiver Operating Characteristic (ROC) curves for continuous diagnostic tests","volume":"16","author":"Zou","year":"1997","journal-title":"Stat. Med."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"3540","DOI":"10.1016\/j.jspi.2010.05.026","article-title":"Compare diagnostic tests using transformation-invariant smoothed roc curves","volume":"140","author":"Tang","year":"2010","journal-title":"J. Stat. Plan. Inference"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"309","DOI":"10.6000\/1927-5129.2012.08.02.09","article-title":"The Bi-gamma ROC curve in a straightforward manner","volume":"8","author":"Hussain","year":"2012","journal-title":"J. Basic Appl. Sci."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"52","DOI":"10.9756\/BIJDM.1362","article-title":"Estimation of area under the roc curve using exponential and weibull distributions","volume":"2","author":"Vardhan","year":"2012","journal-title":"Bonfring Int. J. Data Min."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2769","DOI":"10.1007\/s10994-021-06114-3","article-title":"Receiver Operating Characteristic (ROC) movies, Universal ROC (UROC) curves, and coefficient of predictive ability (CPA)","volume":"111","author":"Gneiting","year":"2022","journal-title":"Mach. Learn."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2147","DOI":"10.1007\/s10994-021-06115-2","article-title":"Receiver Operating Characteristic (ROC) curves: Equivalences, beta model, and minimum distance estimation","volume":"111","author":"Gneiting","year":"2022","journal-title":"Mach. Learn."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Pendrill, L.R., Melin, J., Stavelin, A., and Nordin, G. (2023). Modernising Receiver Operating Characteristic (ROC) Curves. Algorithms, 16.","DOI":"10.3390\/a16050253"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Reshan, M.S.A., Amin, S., Zeb, M.A., Sulaiman, A., Alshahrani, H., Azar, A.T., and Shaikh, A. (2023). Enhancing breast cancer detection and classification using advanced multi-model features and ensemble machine learning techniques. Life, 13.","DOI":"10.3390\/life13102093"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Han, H. (2022). The utility of Receiver Operating Characteristic curve in educational assessment: Performance prediction. Mathematics, 10.","DOI":"10.3390\/math10091493"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Hichri, A., Hajji, M., Mansouri, M., Abodayeh, K., Bouzrara, K., Nounou, H., and Nounou, M. (2022). Genetic-algorithm-based neural network for fault detection and diagnosis: Application to grid-connected photovoltaic systems. Sustainability, 14.","DOI":"10.3390\/su141710518"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Ibrahim, A., Anayi, F., Packianather, M., and Alomari, O.A. (2022). New hybrid invasive weed optimization and machine learning approach for fault detection. Energies, 15.","DOI":"10.3390\/en15041488"},{"key":"ref_15","unstructured":"Petrov, B.N., and Csaki, F. (1973). Information theory and an extension of maximum likelihood principle. Second International Symposium on Information Theory, Akademiai Kiado."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"345","DOI":"10.1007\/BF02294361","article-title":"Model selection and Akaike\u2019s Information Criterion (AIC): The general theory and its analytical extensions","volume":"52","author":"Bozdogan","year":"1987","journal-title":"Psychometrika"},{"key":"ref_17","unstructured":"van Endem, M.H. (1971). An analysis of complexity. Mathematical Centre Tracts, Mathematisch Centrum."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1006\/jmps.1999.1277","article-title":"Akaike\u2019s information criterion and recent developments in Information Complexity","volume":"44","author":"Bozdogan","year":"2000","journal-title":"J. Math. Psychol."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Sun, Y., and Bozdogan, H. (2020). Segmentation of high dimensional time-series data using mixture of sparse principal component regression model with Information Complexity. Entropy, 10.","DOI":"10.3390\/e22101170"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1035","DOI":"10.1090\/S0025-5718-00-01174-1","article-title":"Computation of Gauss-Kronrod quadrature rules","volume":"69","author":"Calvetti","year":"2000","journal-title":"Math. Comput."},{"key":"ref_21","unstructured":"Chakrabarty, N. (2024, June 01). Brain MRI Images for Brain Tumor Detection Dataset. Available online: https:\/\/www.kaggle.com\/navoneel\/brain-mri-images-for-brain-tumor-detection."},{"key":"ref_22","unstructured":"(2024, October 10). Wisconsin Breast Cancer Dataset. Available online: https:\/\/www.kaggle.com\/datasets\/uciml\/breast-cancer-wisconsin-data."},{"key":"ref_23","unstructured":"Holland, J. (1975). An introductory analysis with applications to biology, control, and artificial intelligence. Adaptation in Natural and Artificial Systems, The University of Michigan. [1st ed.]."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"James, G., Witten, D., Hastie, T., Tibshirani, R., and Taylor, J. (2023). Statistical learning. An Introduction to Statistical Learning: With Applications in Python, Springer International Publishing.","DOI":"10.1007\/978-3-031-38747-0"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1148\/radiology.143.1.7063747","article-title":"The meaning and use of the area under a Receiver Operating Characteristic (ROC) curve","volume":"143","author":"Hanley","year":"1982","journal-title":"Radiology"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"932","DOI":"10.1007\/s00330-014-3487-0","article-title":"Disadvantages of using the area under the Receiver Operating Characteristic curve to assess imaging tests: A discussion and proposal for an alternative approach","volume":"25","author":"Halligan","year":"2015","journal-title":"Eur. Radiol."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1023\/A:1010920819831","article-title":"A simple generalisation of the area under the ROC curve for multiple class classification problems","volume":"45","author":"Hand","year":"2021","journal-title":"Mach. Learn."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1023\/A:1024099825458","article-title":"Tree induction for probability-based ranking","volume":"52","author":"Provost","year":"2003","journal-title":"Mach. Learn."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Vose, M.D. (1999). The Simple Genetic Algorithm: Foundations and Theory, The MIT Press.","DOI":"10.7551\/mitpress\/6229.001.0001"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1016\/j.patcog.2019.02.023","article-title":"The impact of class imbalance in classification performance metrics based on the binary confusion matrix","volume":"91","author":"Luque","year":"2019","journal-title":"Pattern Recognit."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/11\/988\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:33:58Z","timestamp":1760114038000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/11\/988"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,17]]},"references-count":30,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2024,11]]}},"alternative-id":["e26110988"],"URL":"https:\/\/doi.org\/10.3390\/e26110988","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2024,11,17]]}}}