{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,23]],"date-time":"2026-04-23T18:01:25Z","timestamp":1776967285520,"version":"3.51.4"},"reference-count":56,"publisher":"Public Library of Science (PLoS)","issue":"4","license":[{"start":{"date-parts":[[2026,4,23]],"date-time":"2026-04-23T00:00:00Z","timestamp":1776902400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS One"],"abstract":"<jats:sec id=\"sec001\">\n                    <jats:title>Introduction<\/jats:title>\n                    <jats:p>Risk adjustment is critical in observational epidemiology to control for confounding of the exposure-outcome relationship. Accurate prediction of outcomes, such as mortality, can improve risk adjustment. In the present study, we compared logistic regression with a range of tree-based ensemble methods to predict 1-year mortality in the general population of Ontario, Canada.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec id=\"sec002\">\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>Ontario adults (age 18 years and older) who were alive as of January 1, 2022 were included. Using a window of up to 3 years, various measures of health and healthcare utilization were captured from administrative databases. To predict 1-year mortality, we applied logistic regression, random forests, extremely randomized trees, adaptive boosting, gradient boosting, extreme gradient boosting, Newton boosting, and CatBoost. All models also included age and sex. Performance was evaluated using the area under the ROC curve (AUROC), the area under the precision-recall curve (PR-AUC), the Brier score, and a quantile-based version of the Integrated Calibration Index (ICI), reported in the 30% test set. Feature importance was assessed using CatBoost\u2019s internal model structure, supplemented with permutation feature importance, explainable boosted machines, and marginal effects.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec id=\"sec003\">\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>A total of 12,080,801 Ontarians were included and 121,951 (1.0%) died within 1 year. Logistic regression showed excellent discrimination (AUROC 0.926; PR-AUC 0.256) and acceptable calibration (ICI 0.0022). The best model was CatBoost, which had the best discrimination (AUROC 0.933, PR-AUC 0.280) and calibration (ICI 0.0003). In sensitivity analyses of the CatBoost model, including more detailed definitions of cancer (to include its subtype) and chronic kidney disease (defined using serum creatinine instead of diagnostic codes) produced modest improvements in PR-AUC (0.290), along with substantially improved calibration amongst the highest-risk (70\u2013100%) individuals. The most influential model-building feature was age. Residence in long-term care and receipt of palliative care was associated with the largest marginal effects.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec id=\"sec004\">\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>The machine learning model CatBoost yielded the most accurate predictive model for 1-year mortality using individual comorbidities and additional measures of healthcare utilization for the general population. These findings demonstrate that machine learning methods can enhance risk adjustment efforts in observational studies, leading to more accurate confounder control and better support for health policy and epidemiologic research.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1371\/journal.pone.0347302","type":"journal-article","created":{"date-parts":[[2026,4,23]],"date-time":"2026-04-23T17:25:26Z","timestamp":1776965126000},"page":"e0347302","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":0,"title":["Using tree-based ensemble methods to produce a population-based mortality risk score in Ontario, Canada"],"prefix":"10.1371","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9650-9105","authenticated-orcid":true,"given":"Steven","family":"Habbous","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Peter C.","family":"Austin","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shabnam","family":"Balamchi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Davood","family":"Astaraky","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Roozbeh","family":"Yousefi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Munaza","family":"Chaudhry","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Erik","family":"Hellsten","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"340","published-online":{"date-parts":[[2026,4,23]]},"reference":[{"issue":"15","key":"pone.0347302.ref001","doi-asserted-by":"crossref","first-page":"1553","DOI":"10.1001\/jama.2011.451","article-title":"A predictive model for progression of chronic kidney disease to kidney failure","volume":"305","author":"N Tangri","year":"2011","journal-title":"JAMA"},{"issue":"10","key":"pone.0347302.ref002","doi-asserted-by":"crossref","first-page":"910","DOI":"10.1056\/NEJMoa1214726","article-title":"Probability of cancer in pulmonary nodules detected on first screening CT","volume":"369","author":"A McWilliams","year":"2013","journal-title":"N Engl J Med"},{"issue":"6","key":"pone.0347302.ref003","doi-asserted-by":"crossref","first-page":"1338","DOI":"10.1097\/ALN.0b013e3181fc6e0a","article-title":"Prediction of postoperative pulmonary complications in a population-based surgical cohort","volume":"113","author":"J Canet","year":"2010","journal-title":"Anesthesiology"},{"issue":"6","key":"pone.0347302.ref004","doi-asserted-by":"crossref","first-page":"743","DOI":"10.1161\/CIRCULATIONAHA.107.699579","article-title":"General cardiovascular risk profile for use in primary care: the Framingham Heart Study","volume":"117","author":"RB D\u2019Agostino","year":"2008","journal-title":"Circulation"},{"issue":"1","key":"pone.0347302.ref005","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s12553-023-00806-7","article-title":"The underuse of AI in the health sector: opportunity costs, success stories, risks and recommendations","volume":"14","author":"U Pagallo","year":"2024","journal-title":"Health Technol (Berl)"},{"key":"pone.0347302.ref006","doi-asserted-by":"crossref","first-page":"4942","DOI":"10.57187\/4942","article-title":"A qualitative assessment of the use of artificial intelligence in public sector health organisations in Ontario, Canada","volume":"156","author":"S Habbous","year":"2026","journal-title":"Swiss Med Wkly"},{"issue":"1","key":"pone.0347302.ref007","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1093\/cid\/cix731","article-title":"Machine learning for healthcare: on the verge of a major shift in healthcare epidemiology","volume":"66","author":"J Wiens","year":"2018","journal-title":"Clin Infect Dis"},{"issue":"5","key":"pone.0347302.ref008","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1016\/0021-9681(87)90171-8","article-title":"A new method of classifying prognostic comorbidity in longitudinal studies: development and validation","volume":"40","author":"ME Charlson","year":"1987","journal-title":"J Chronic Dis"},{"issue":"4","key":"pone.0347302.ref009","first-page":"188","article-title":"Charlson comorbidity index: ICD-9 update and ICD-10 translation","volume":"12","author":"WP Glasheen","year":"2019","journal-title":"Am Health Drug Benefits"},{"issue":"4","key":"pone.0347302.ref010","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1016\/j.jclinepi.2014.12.002","article-title":"Look back for the Charlson Index did not improve risk adjustment of cancer surgical outcomes","volume":"68","author":"TA Dobbins","year":"2015","journal-title":"J Clin Epidemiol"},{"key":"pone.0347302.ref011","volume-title":"Ensemble methods for machine learning","author":"G. Kunapuli","year":"2023"},{"issue":"1","key":"pone.0347302.ref012","doi-asserted-by":"crossref","DOI":"10.1186\/s40537-020-00369-8","article-title":"CatBoost for big data: an interdisciplinary review","volume":"7","author":"JT Hancock","year":"2020","journal-title":"J Big Data"},{"issue":"23","key":"pone.0347302.ref013","article-title":"Using tree-based machine learning for health studies: literature review and case series","volume":"19","author":"L Hu","year":"2022","journal-title":"Int J Environ Res Public Health"},{"key":"pone.0347302.ref014","first-page":"1","volume-title":"Grokking machine learning","author":"LG. Serrano","year":"2021"},{"issue":"3","key":"pone.0347302.ref015","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pone.0118432","article-title":"The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets","volume":"10","author":"T Saito","year":"2015","journal-title":"PLoS One"},{"issue":"6","key":"pone.0347302.ref016","doi-asserted-by":"crossref","first-page":"100994","DOI":"10.1016\/j.patter.2024.100994","article-title":"The receiver operating characteristic curve accurately assesses imbalanced datasets","volume":"5","author":"E Richardson","year":"2024","journal-title":"Patterns"},{"issue":"8","key":"pone.0347302.ref017","doi-asserted-by":"crossref","first-page":"938","DOI":"10.1016\/j.jclinepi.2009.11.009","article-title":"Use of Brier score to assess binary predictions","volume":"63","author":"K Rufibach","year":"2010","journal-title":"J Clin Epidemiol"},{"issue":"1","key":"pone.0347302.ref018","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1186\/s12916-019-1466-7","article-title":"Calibration: the Achilles heel of predictive analytics","volume":"17","author":"Topic Group \u2018Evaluating diagnostic tests and prediction models\u2019 of the STRATOS initiative","year":"2019","journal-title":"BMC Med"},{"issue":"3","key":"pone.0347302.ref019","doi-asserted-by":"crossref","first-page":"517","DOI":"10.1002\/sim.5941","article-title":"Graphical assessment of internal and external calibration of logistic regression models by using loess smoothers","volume":"33","author":"PC Austin","year":"2014","journal-title":"Stat Med"},{"issue":"21","key":"pone.0347302.ref020","doi-asserted-by":"crossref","first-page":"4051","DOI":"10.1002\/sim.8281","article-title":"The Integrated Calibration Index (ICI) and related metrics for quantifying the calibration of logistic regression models","volume":"38","author":"PC Austin","year":"2019","journal-title":"Stat Med"},{"key":"pone.0347302.ref021","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1016\/j.jclinepi.2019.09.016","article-title":"Validation of clinical prediction models: what does the \u201ccalibration slope\u201d really measure?","volume":"118","author":"RJ Stevens","year":"2020","journal-title":"J Clin Epidemiol"},{"issue":"1","key":"pone.0347302.ref022","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1097\/EDE.0b013e3181c30fb2","article-title":"Assessing the performance of prediction models: a framework for traditional and novel measures","volume":"21","author":"EW Steyerberg","year":"2010","journal-title":"Epidemiology"},{"key":"pone.0347302.ref023","doi-asserted-by":"crossref","first-page":"562","DOI":"10.1093\/biomet\/45.3-4.562","article-title":"Two further applications of a model for binary regression","volume":"45","author":"DR Cox","year":"1958","journal-title":"Biometrika"},{"issue":"10","key":"pone.0347302.ref024","doi-asserted-by":"crossref","first-page":"751","DOI":"10.7326\/0003-4819-149-10-200811180-00009","article-title":"Assessing the value of risk predictions by using risk stratification tables","volume":"149","author":"H Janes","year":"2008","journal-title":"Ann Intern Med"},{"issue":"5","key":"pone.0347302.ref025","doi-asserted-by":"crossref","first-page":"952","DOI":"10.1093\/jalm\/jfad047","article-title":"The CKD-EPI 2021 equation and other creatinine-based race-independent eGFR equations in chronic kidney disease diagnosis and staging","volume":"8","author":"S Lu","year":"2023","journal-title":"J Appl Lab Med"},{"key":"pone.0347302.ref026","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1186\/1471-2369-14-81","article-title":"Detecting chronic kidney disease in population-based administrative databases using an algorithm of hospital encounter and physician claim codes","volume":"14","author":"JL Fleet","year":"2013","journal-title":"BMC Nephrol"},{"key":"pone.0347302.ref027","doi-asserted-by":"crossref","first-page":"278","DOI":"10.1002\/ansa.202200018","article-title":"Cross-validated permutation feature importance considering correlation between features","volume":"3","author":"H Kaneko","year":"2022","journal-title":"Anal Sci Adv"},{"issue":"2","key":"pone.0347302.ref028","doi-asserted-by":"crossref","first-page":"308","DOI":"10.1177\/1536867X1201200209","article-title":"Using the margins command to estimate and interpret adjusted predictions and marginal effects","volume":"12","author":"R Williams","year":"2012","journal-title":"Stata J"},{"issue":"1","key":"pone.0347302.ref029","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1186\/s40635-024-00639-2","article-title":"Explainable Boosting Machine approach identifies risk factors for acute renal failure","volume":"12","author":"A K\u00f6rner","year":"2024","journal-title":"Intensive Care Med Exp"},{"issue":"3","key":"pone.0347302.ref030","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1002\/sim.3782","article-title":"Improving propensity score weighting using machine learning","volume":"29","author":"BK Lee","year":"2010","journal-title":"Stat Med"},{"issue":"1","key":"pone.0347302.ref031","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1097\/00005650-199801000-00004","article-title":"Comorbidity measures for use with administrative data","volume":"36","author":"A Elixhauser","year":"1998","journal-title":"Med Care"},{"issue":"6","key":"pone.0347302.ref032","doi-asserted-by":"crossref","first-page":"626","DOI":"10.1097\/MLR.0b013e31819432e5","article-title":"A modification of the Elixhauser comorbidity measures into a point system for hospital death using administrative data","volume":"47","author":"C van Walraven","year":"2009","journal-title":"Med Care"},{"issue":"11","key":"pone.0347302.ref033","doi-asserted-by":"crossref","first-page":"1130","DOI":"10.1097\/01.mlr.0000182534.19832.83","article-title":"Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data","volume":"43","author":"H Quan","year":"2005","journal-title":"Med Care"},{"issue":"10","key":"pone.0347302.ref034","doi-asserted-by":"crossref","first-page":"1423","DOI":"10.7326\/M21-4204","article-title":"Development and validation of the summary elixhauser comorbidity score for use with ICD-10-CM-coded data among older adults","volume":"175","author":"HB Mehta","year":"2022","journal-title":"Ann Intern Med"},{"issue":"10","key":"pone.0347302.ref035","doi-asserted-by":"crossref","first-page":"932","DOI":"10.1097\/MLR.0b013e318215d5e2","article-title":"Using the Johns Hopkins Aggregated Diagnosis Groups (ADGs) to predict mortality in a general adult population cohort in Ontario, Canada","volume":"49","author":"PC Austin","year":"2011","journal-title":"Med Care"},{"issue":"10","key":"pone.0347302.ref036","doi-asserted-by":"crossref","first-page":"940","DOI":"10.1097\/MLR.0b013e318229360e","article-title":"The mortality risk score and the ADG score: two points-based scoring systems for the Johns Hopkins aggregated diagnosis groups to predict mortality in a general adult population cohort in Ontario, Canada","volume":"49","author":"PC Austin","year":"2011","journal-title":"Med Care"},{"key":"pone.0347302.ref037","volume-title":"CIHI\u2019s Population Grouping Methodology 1.4 \u2014 Overview and Outputs, 2023","author":"Canadian Institute for Health Information","year":"2023"},{"issue":"10","key":"pone.0347302.ref038","doi-asserted-by":"crossref","first-page":"3853","DOI":"10.21037\/tcr-22-1626","article-title":"Utilization of model-agnostic explainable artificial intelligence frameworks in oncology: a narrative review","volume":"11","author":"C Ladbury","year":"2022","journal-title":"Transl Cancer Res"},{"key":"pone.0347302.ref039","article-title":"Glass box and black box machine learning approaches to exploit compositional descriptors of molecules in drug discovery and aid the medicinal chemist","author":"B Robson","year":"2024","journal-title":"ChemMedChem"},{"issue":"12","key":"pone.0347302.ref040","doi-asserted-by":"crossref","first-page":"971","DOI":"10.1136\/bmjqs-2018-009285","article-title":"mHOMR: a feasibility study of an automated system for identifying inpatients having an elevated risk of 1-year mortality","volume":"28","author":"P Wegier","year":"2019","journal-title":"BMJ Qual Saf"},{"key":"pone.0347302.ref041","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1007\/978-1-4614-6849-3","volume-title":"Applied predictive modeling","author":"M Kuhn","year":"2013"},{"key":"pone.0347302.ref042","doi-asserted-by":"crossref","first-page":"102256","DOI":"10.1016\/j.artmed.2022.102256","article-title":"A novel lifelong machine learning-based method to eliminate calibration drift in clinical prediction models","volume":"125","author":"S Chi","year":"2022","journal-title":"Artif Intell Med"},{"key":"pone.0347302.ref043","doi-asserted-by":"crossref","first-page":"103611","DOI":"10.1016\/j.jbi.2020.103611","article-title":"Detection of calibration drift in clinical prediction models to inform model updating","volume":"112","author":"SE Davis","year":"2020","journal-title":"J Biomed Inform"},{"issue":"4","key":"pone.0347302.ref044","article-title":"Myocardial infarction signs and symptoms: females vs. males","volume":"15","author":"KJ Schulte","year":"2023","journal-title":"Cureus"},{"issue":"3","key":"pone.0347302.ref045","first-page":"177","article-title":"Race and gender disparities are evident in COPD underdiagnoses across all severities of measured airflow obstruction","volume":"5","author":"AJ Mamary","year":"2018","journal-title":"Chronic Obstr Pulm Dis"},{"issue":"1","key":"pone.0347302.ref046","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1038\/s41746-023-00858-z","article-title":"Bias in AI-based models for medical applications: challenges and mitigation strategies","volume":"6","author":"M Mittermaier","year":"2023","journal-title":"NPJ Digit Med"},{"issue":"6","key":"pone.0347302.ref047","doi-asserted-by":"crossref","first-page":"719","DOI":"10.1038\/s41551-023-01056-8","article-title":"Algorithm fairness in artificial intelligence for medicine and healthcare","volume":"7","author":"RJ Chen","year":"2023","journal-title":"Nat Biomed Eng"},{"issue":"6","key":"pone.0347302.ref048","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pdig.0000278","article-title":"Bias in artificial intelligence algorithms and recommendations for mitigation","volume":"2","author":"LH Nazer","year":"2023","journal-title":"PLOS Digit Health"},{"issue":"3","key":"pone.0347302.ref049","doi-asserted-by":"crossref","DOI":"10.1136\/bmjopen-2021-054058","article-title":"Validation and adaptation of the Charlson Comorbidity Index using administrative data from the Colombian health system: retrospective cohort study","volume":"12","author":"H Oliveros","year":"2022","journal-title":"BMJ Open"},{"issue":"6","key":"pone.0347302.ref050","doi-asserted-by":"crossref","first-page":"676","DOI":"10.1093\/aje\/kwq433","article-title":"Updating and validating the Charlson comorbidity index and score for risk adjustment in hospital discharge abstracts using data from 6 countries","volume":"173","author":"H Quan","year":"2011","journal-title":"Am J Epidemiol"},{"issue":"9","key":"pone.0347302.ref051","doi-asserted-by":"crossref","first-page":"940","DOI":"10.1016\/j.jclinepi.2005.12.013","article-title":"Length of comorbidity lookback period affected regression model performance of administrative health data","volume":"59","author":"DB Preen","year":"2006","journal-title":"J Clin Epidemiol"},{"issue":"1","key":"pone.0347302.ref052","doi-asserted-by":"crossref","DOI":"10.1186\/s12913-018-3148-0","article-title":"Identifying diabetes cases from administrative data: a population-based validation study","volume":"18","author":"LL Lipscombe","year":"2018","journal-title":"BMC Health Serv Res"},{"issue":"3","key":"pone.0347302.ref053","doi-asserted-by":"crossref","first-page":"160","DOI":"10.24095\/hpcdp.33.3.06","article-title":"Identifying cases of congestive heart failure from administrative data: a validation study using primary care patient records","volume":"33","author":"SE Schultz","year":"2013","journal-title":"Chronic Dis Inj Can"},{"issue":"7","key":"pone.0347302.ref054","article-title":"Validation of physician billing and hospitalization data to identify patients with ischemic heart disease using data from the Electronic Medical Record Administrative data Linked Database (EMRALD)","volume":"26","author":"K Tu","year":"2010","journal-title":"Can J Cardiol"},{"issue":"6","key":"pone.0347302.ref055","doi-asserted-by":"crossref","first-page":"765","DOI":"10.1017\/cjn.2016.278","article-title":"Accuracy of administrative data for the coding of acute stroke and TIAs","volume":"43","author":"R Hall","year":"2016","journal-title":"Can J Neurol Sci"},{"issue":"10","key":"pone.0347302.ref056","doi-asserted-by":"crossref","first-page":"1733","DOI":"10.1007\/s00464-006-9136-1","article-title":"Accuracy of administrative health data for the diagnosis of upper gastrointestinal diseases","volume":"21","author":"SR Lopushinsky","year":"2007","journal-title":"Surg Endosc"}],"container-title":["PLOS One"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0347302","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,23]],"date-time":"2026-04-23T17:25:37Z","timestamp":1776965137000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0347302"}},"subtitle":[],"editor":[{"given":"Suyan","family":"Tian","sequence":"first","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2026,4,23]]},"references-count":56,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2026,4,23]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0347302","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,23]]}}}