{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,24]],"date-time":"2026-08-24T21:24:08Z","timestamp":1787606648090,"version":"build-2736575974"},"reference-count":19,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR AI"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Fairness evaluation is essential for trustworthy clinical risk prediction. However, existing fairness-oriented discrimination metrics either ignore cross-group comparisons or rely on exhaustive pairwise evaluations, making them difficult to interpret and impractical for model selection.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study aimed to develop and evaluate novel fairness-oriented discrimination metrics for clinical risk prediction that address limitations of within-group and pairwise cross-group approaches.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n We examined theoretical properties of existing U-statistic\u2013based metrics, including concordance index (CI) and area under the receiver operating characteristic curve (AUC), when applied to subgroups. We highlighted the distinction between within-group discrimination (ranking within a subgroup) and group-level discrimination (ranking relative to the broader population). Building on this framework, we proposed group-level extensions of the CI and AUC that summarize subgroup-specific performance in a single interpretable measure. We then applied these metrics to the PREVENT (Predicting Risk of Cardiovascular Disease Events) equation, a recently developed model for atherosclerotic cardiovascular disease.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n The traditional subgroup-specific CI and AUC captured within-group but not group-level discrimination, obscuring inequities in clinical decision-making. Existing cross-group approaches (eg, the xCI and xAUC metrics) addressed this limitation but became computationally and interpretively burdensome with multiple subgroups due to pairwise comparisons. Our proposed metrics provided a streamlined alternative, yielding 1 summary statistic per subgroup while retaining sensitivity to cross-group ranking disparities. Applied to PREVENT, these metrics revealed differences in subgroup performance not apparent from within-group evaluations.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n By distinguishing between within-group and group-level discrimination, our framework clarifies a common source of misinterpretation in fairness evaluation. The proposed group-level extensions of the CI and AUC provide practical, interpretable tools for evaluating fairness in clinical prediction models, enabling more transparent and equitable risk assessment.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/85822","type":"journal-article","created":{"date-parts":[[2026,8,24]],"date-time":"2026-08-24T21:01:58Z","timestamp":1787605318000},"page":"e85822-e85822","source":"Crossref","is-referenced-by-count":0,"title":["A Simplified Metric to Streamline Between-Group Fairness Assessment for Predictive Models: Algorithm Development and Evaluation Study"],"prefix":"10.2196","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-1175-6881","authenticated-orcid":false,"given":"Haoyuan","family":"Wang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7056-9559","authenticated-orcid":false,"given":"Chuan","family":"Hong","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1968-2641","authenticated-orcid":false,"given":"Michael","family":"Pencina","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4112-9639","authenticated-orcid":false,"given":"Matthew","family":"Engelhard","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,8,24]]},"reference":[{"issue":"13","key":"R1","doi-asserted-by":"publisher","first-page":"2109","DOI":"10.1002\/sim.1802","article-title":"Overall C as a measure of discrimination in survival analysis: model specific population value and confidence interval estimation","volume":"23","author":"Pencina","journal-title":"Stat Med"},{"issue":"1","key":"R2","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1148\/radiology.143.1.7063747","article-title":"The meaning and use of the area under a receiver operating characteristic (ROC) curve","volume":"143","author":"Hanley","journal-title":"Radiology"},{"key":"R3","doi-asserted-by":"crossref","unstructured":"Korolyuk VS Borovskich YV . Theory of U-Statistics. Springer; 2013. [doi: 10.1007\/978-94-017-3515-5]","DOI":"10.1007\/978-94-017-3515-5"},{"key":"R4","doi-asserted-by":"crossref","unstructured":"Pessach D Shmueli E . Algorithmic fairness. In: Rokach L Maimon O Shmueli E , editors. Machine Learning for Data Science Handbook. Springer; 2023:867-886. [doi: 10.1007\/978-3-031-24628-9_37]","DOI":"10.1007\/978-3-031-24628-9_37"},{"key":"R5","unstructured":"Barocas S Hardt M Narayanan A . Fairness and Machine Learning: Limitations and Opportunities. MIT Press; 2023. ISBN: 9780262376525"},{"key":"R6","unstructured":"Hardt M Price E Srebro N . Equality of opportunity in supervised learning. In: NIPS\u201916: Proceedings of the 30th International Conference on Neural Information Processing Systems. Curran Associates; 2016. ISBN: 9781510838819"},{"key":"R7","unstructured":"Jiang Z Han X Fan C Yang F Mostafavi A Hu X . Generalized demographic parity for group fairness. Presented at: Tenth International Conference on Learning Representations ICLR 2022; Apr 25-29, 2022. URL: https:\/\/openreview.net\/pdf?id=YigKlMJwjye [Accessed 24-07-2026]"},{"key":"R8","unstructured":"Pleiss G Raghavan M Wu F Kleinberg J Weinberger KQ . On fairness and calibration. In: von Luxburg U Guyon I Bengio S Wallach H Fergus R , editors. NIPS\u201917: Proceedings of the 31st International Conference on Neural Information Processing Systems. Curran Associates; 2017. [doi: 10.5555\/3295222.3295319]"},{"key":"R9","unstructured":"Kleinberg J Mullainathan S Raghavan M . Inherent trade-offs in the fair determination of risk scores. arXiv. Preprint posted online on Sep 19, 2016. [doi: 10.48550\/arXiv.1609.05807]"},{"issue":"10","key":"R10","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2020.23242","article-title":"Performance of the pooled cohort equations to estimate atherosclerotic cardiovascular disease risk by body mass index","volume":"3","author":"Khera","journal-title":"JAMA Netw Open"},{"issue":"6","key":"R11","doi-asserted-by":"publisher","first-page":"633","DOI":"10.1001\/jamacardio.2021.0122","article-title":"Use of machine learning models to predict death after acute myocardial infarction","volume":"6","author":"Khera","journal-title":"JAMA Cardiol"},{"issue":"1","key":"R12","doi-asserted-by":"publisher","DOI":"10.1007\/s40520-025-03063-y","article-title":"Heterogeneity in mortality risk prediction: a study of vulnerable adults in the Canadian Longitudinal Study on Aging","volume":"37","author":"Ndiaye","journal-title":"Aging Clin Exp Res"},{"key":"R13","unstructured":"Kallus N Zhou A . The fairness of risk scores beyond classification: bipartite ranking and the xAUC metric. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems. Curran Associates; 2019. [doi: 10.5555\/3454287.3454596]"},{"key":"R14","doi-asserted-by":"publisher","DOI":"10.1016\/j.artmed.2025.103130","article-title":"Exploring trade-offs in equitable stroke risk prediction with parity-constrained and race-free models","volume":"164","author":"Engelhard","journal-title":"Artif Intell Med"},{"issue":"6","key":"R15","doi-asserted-by":"publisher","first-page":"430","DOI":"10.1161\/CIRCULATIONAHA.123.067626","article-title":"Development and validation of the American Heart Association\u2019s PREVENT equations","volume":"149","author":"Khan","journal-title":"Circulation"},{"issue":"4","key":"R16","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2026.6838","article-title":"Performance of PREVENT cardiovascular risk in electronic health record-based clinical practice","volume":"9","author":"Hong","journal-title":"JAMA Netw Open"},{"issue":"3","key":"R17","doi-asserted-by":"crossref","first-page":"837","DOI":"10.2307\/2531595","volume":"44","author":"DeLong","journal-title":"Biometrics"},{"key":"R18","unstructured":"engelhard-lab\/gCI. GitHub. URL: https:\/\/github.com\/engelhard-lab\/gci [Accessed 18-08-2026]"},{"key":"R19","unstructured":"Truveta. URL: https:\/\/www.truveta.com\/ [Accessed 18-08-2026]"}],"container-title":["JMIR AI"],"original-title":[],"language":"en","deposited":{"date-parts":[[2026,8,24]],"date-time":"2026-08-24T21:01:59Z","timestamp":1787605319000},"score":1,"resource":{"primary":{"URL":"https:\/\/ai.jmir.org\/2026\/1\/e85822"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,24]]},"references-count":19,"URL":"https:\/\/doi.org\/10.2196\/85822","relation":{},"ISSN":["2817-1705"],"issn-type":[{"value":"2817-1705","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,24]]},"article-number":"v5i13e85822"}}