{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T12:20:16Z","timestamp":1791894016810,"version":"4.3.4"},"reference-count":51,"publisher":"Oxford University Press (OUP)","issue":"4","license":[{"start":{"date-parts":[[2022,2,25]],"date-time":"2022-02-25T00:00:00Z","timestamp":1645747200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,11,29]]},"abstract":"<jats:title>Summary<\/jats:title>\n                  <jats:p>Variable importance measures are the main tools used to analyse the black-box mechanisms of random forests. Although the mean decrease accuracy is widely accepted as the most efficient variable importance measure for random forests, little is known about its statistical properties. In fact, the definition of mean decrease accuracy varies across the main random forest software. In this article, our objective is to rigorously analyse the behaviour of the main mean decrease accuracy implementations. Consequently, we mathematically formalize the various implemented mean decrease accuracy algorithms, and then establish their limits when the sample size increases. This asymptotic analysis reveals that these mean decrease accuracy versions differ as importance measures, since they converge towards different quantities. More importantly, we break down these limits into three components: the first two terms are related to Sobol indices, which are well-defined measures of a covariate contribution to the response variance, widely used in the sensitivity analysis field, as opposed to the third term, whose value increases with dependence within covariates. Thus, we theoretically demonstrate that the mean decrease accuracy does not target the right quantity to detect influential covariates in a dependent setting, a fact that has already been noticed experimentally. To address this issue, we define a new importance measure for random forests, the Sobol-mean decrease accuracy, which fixes the flaws of the original mean decrease accuracy, and consistently estimates the accuracy decrease of the forest retrained without a given covariate, but with an efficient computational cost. The Sobol-mean decrease accuracy empirically outperforms its competitors on both simulated and real data for variable selection.<\/jats:p>","DOI":"10.1093\/biomet\/asac017","type":"journal-article","created":{"date-parts":[[2022,2,24]],"date-time":"2022-02-24T15:13:41Z","timestamp":1645715621000},"page":"881-900","source":"Crossref","is-referenced-by-count":83,"title":["Mean decrease accuracy for random forests: inconsistency, and a practical solution via the Sobol-MDA"],"prefix":"10.1093","volume":"109","author":[{"given":"Cl\u00e9ment","family":"B\u00e9nard","sequence":"first","affiliation":[{"name":"Digital Sciences & Technologies Safran Tech, , 78114 Magny-Les-Hameaux, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"S\u00e9bastien","family":"Da Veiga","sequence":"additional","affiliation":[{"name":"Digital Sciences & Technologies Safran Tech, , 78114 Magny-Les-Hameaux, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Erwan","family":"Scornet","sequence":"additional","affiliation":[{"name":"Institut Polytechnique de Paris Center for Applied Mathematics, UMR7641, \u00c9cole polytechnique, , 91120 Palaiseau, France"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2022,2,25]]},"reference":[{"key":"2022112917595172400_B1","doi-asserted-by":"crossref","first-page":"103502","DOI":"10.1016\/j.artint.2021.103502","article-title":"Explaining individual predictions when features are dependent: more accurate approximations to Shapley values","volume":"298","author":"Aas,","year":"2021","journal-title":"Artif. Intel."},{"key":"2022112917595172400_B2","first-page":"107","article-title":"Random forests for global sensitivity analysis: a selective review","volume":"206","author":"Antoniadis,","year":"2020","journal-title":"Reliab. Eng. Syst. Safety"},{"key":"2022112917595172400_B3","doi-asserted-by":"crossref","first-page":"2249","DOI":"10.1016\/j.csda.2007.08.015","article-title":"Empirical characterization of random forest variable importance measures","volume":"52","author":"Archer,","year":"2008","journal-title":"Comp. Statist. Data Anal."},{"key":"2022112917595172400_B4","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1016\/j.chemolab.2010.12.004","article-title":"Empirical comparison of tree ensemble variable importance measures","volume":"105","author":"Auret,","year":"2011","journal-title":"Chemom. Intell. Lab. Syst."},{"key":"2022112917595172400_B5","doi-asserted-by":"crossref","first-page":"493","DOI":"10.1002\/widm.1072","article-title":"Overview of random forest methodology and practical guidance with emphasis on computational biology and bioinformatics","volume":"2","author":"Boulesteix,","year":"2012","journal-title":"Data Mining Know. Disc."},{"key":"2022112917595172400_B6","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman,","year":"2001","journal-title":"Mach. Learn."},{"key":"2022112917595172400_B7","volume-title":"Classification and Regression Trees","author":"Breiman,","year":"1984"},{"key":"2022112917595172400_B8","doi-asserted-by":"crossref","first-page":"551","DOI":"10.1111\/rssb.12265","article-title":"Panning for gold: \u2018Model-X\u2019 knockoffs for high-dimensional controlled variable selection","volume":"80","author":"Cand\u00e8s,","year":"2018","journal-title":"J. R. Statist. Soc. B"},{"key":"2022112917595172400_B9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/1471-2105-7-3","article-title":"Gene selection and classification of microarray data using random forest","volume":"7","author":"D\u00edaz-Uriarte,","year":"2006","journal-title":"BMC Bioinformatics"},{"key":"2022112917595172400_B10","article-title":"UCI machine learning repository","author":"Dua,","year":"2017"},{"key":"2022112917595172400_B11","doi-asserted-by":"crossref","first-page":"2225","DOI":"10.1016\/j.patrec.2010.03.014","article-title":"Variable selection using random forests","volume":"31","author":"Genuer,","year":"2010","journal-title":"Pat. Recog. Lett."},{"key":"2022112917595172400_B12","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-12385-1","volume-title":"Handbook of Uncertainty Quantification","author":"Ghanem,","year":"2017"},{"key":"2022112917595172400_B13","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/j.csda.2015.04.002","article-title":"Grouped variable importance with random forests and application to multiple functional data analysis","volume":"90","author":"Gregorutti,","year":"2015","journal-title":"Comp. Statist. Data Anal."},{"key":"2022112917595172400_B14","doi-asserted-by":"crossref","first-page":"659","DOI":"10.1007\/s11222-016-9646-1","article-title":"Correlation and variable importance in random forests","volume":"27","author":"Gregorutti,","year":"2017","journal-title":"Statist. Comp."},{"key":"2022112917595172400_B15","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1023\/A:1012487302797","article-title":"Gene selection for cancer classification using support vector machines","volume":"46","author":"Guyon,","year":"2002","journal-title":"Mach. Learn."},{"key":"2022112917595172400_B16","volume-title":"A Distribution-Free Theory of Nonparametric Regression","author":"Gy\u00f6rfi,","year":"2006"},{"key":"2022112917595172400_B17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s11222-021-10057-z","article-title":"Unrestricted permutation forces extrapolation: variable importance requires at least one more model, or there is no free variable importance","volume":"31","author":"Hooker,","year":"2021","journal-title":"Statist. Comp."},{"key":"2022112917595172400_B18","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1007\/978-1-4899-7547-8_5","article-title":"A review on global sensitivity analysis methods","volume-title":"Uncertainty Management in Simulation-Optimization of Complex Systems","author":"Iooss,","year":"2015"},{"key":"2022112917595172400_B19","doi-asserted-by":"crossref","first-page":"493","DOI":"10.1615\/Int.J.UncertaintyQuantification.2019028372","article-title":"Shapley effects for sensitivity analysis with correlated inputs: comparisons with Sobol\u2019indices, numerical estimation and applications","volume":"9","author":"Iooss,","year":"2019","journal-title":"Int. J. Uncertain. Quantif."},{"key":"2022112917595172400_B20","doi-asserted-by":"crossref","first-page":"519","DOI":"10.1214\/07-EJS039","article-title":"Variable importance in binary regression trees and forests","volume":"1","author":"Ishwaran,","year":"2007","journal-title":"Electron. J. Statist."},{"key":"2022112917595172400_B21","article-title":"Fast unified random forests for survival, regression, and classification (RF-SRC)","author":"Ishwaran,","year":"2020"},{"key":"2022112917595172400_B22","doi-asserted-by":"crossref","first-page":"841","DOI":"10.1214\/08-AOAS169","article-title":"Random survival forests","volume":"2","author":"Ishwaran,","year":"2008","journal-title":"Ann. Appl. Statist."},{"key":"2022112917595172400_B23","doi-asserted-by":"crossref","first-page":"937","DOI":"10.1016\/j.cpc.2011.12.020","article-title":"Estimation of global sensitivity indices for models with dependent variables","volume":"183","author":"Kucherenko,","year":"2012","journal-title":"Comp. Phys. Commun."},{"key":"2022112917595172400_B24","article-title":"A debiased MDI feature importance measure for random forests","volume-title":"Advances in Neural Information Processing Systems","author":"Li,","year":"2019"},{"key":"2022112917595172400_B25","first-page":"18","article-title":"Classification and regression by randomforest","volume":"2","author":"Liaw,","year":"2002","journal-title":"R News"},{"key":"2022112917595172400_B26","doi-asserted-by":"crossref","first-page":"1413","DOI":"10.1080\/03610926.2020.1764042","article-title":"Unbiased variable importance for random forests","volume":"51","author":"Loecher,","year":"2022","journal-title":"Commun. Statist. A"},{"key":"2022112917595172400_B27","article-title":"Consistent individualized feature attribution for tree ensembles","author":"Lundberg,","year":"2019"},{"key":"2022112917595172400_B28","first-page":"4768","article-title":"A unified approach to interpreting model predictions","volume-title":"Advances in Neural Information Processing Systems","author":"Lundberg,","year":"2017"},{"key":"2022112917595172400_B29","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1016\/j.envsoft.2015.07.010","article-title":"Non-parametric methods for global sensitivity analysis of model output with dependent inputs","volume":"72","author":"Mara,","year":"2015","journal-title":"Envir. Mod. Software"},{"key":"2022112917595172400_B30","first-page":"983","article-title":"Quantile regression forests","volume":"7","author":"Meinshausen,","year":"2006","journal-title":"J. Mach. Learn. Res."},{"key":"2022112917595172400_B31","first-page":"841","article-title":"Quantifying uncertainty in random forests via confidence intervals and hypothesis tests","volume":"17","author":"Mentch,","year":"2016","journal-title":"J. Mach. Learn. Res."},{"key":"2022112917595172400_B32","article-title":"Getting better from worse: augmented bagging and a cautionary tale of variable importance","author":"Mentch,","year":"2020"},{"key":"2022112917595172400_B33","doi-asserted-by":"crossref","first-page":"1884","DOI":"10.1093\/bioinformatics\/btp331","article-title":"Predictor correlation impacts machine learning algorithms: implications for genomic studies","volume":"25","author":"Nicodemus,","year":"2009","journal-title":"Bioinformatics"},{"key":"2022112917595172400_B34","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1137\/130936233","article-title":"Sobol\u2019indices and Shapley value","volume":"2","author":"Owen,","year":"2014","journal-title":"SIAM-ASA J. Uncertainty Quantification"},{"key":"2022112917595172400_B35","first-page":"2825","article-title":"Scikit-learn: machine learning in python","volume":"12","author":"Pedregosa,","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"2022112917595172400_B36","doi-asserted-by":"crossref","first-page":"232","DOI":"10.1214\/21-EJS1958","article-title":"Rates of convergence for random forests via generalized U-statistics","volume":"16","author":"Peng,","year":"2022","journal-title":"Electron. J. Statist."},{"key":"2022112917595172400_B37","volume-title":"R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing.","year":"2022"},{"key":"2022112917595172400_B38","doi-asserted-by":"crossref","first-page":"280","DOI":"10.1016\/S0010-4655(02)00280-1","article-title":"Making best use of model evaluations to compute sensitivity indices","volume":"145","author":"Saltelli,","year":"2002","journal-title":"Comp. Phys. Commun."},{"key":"2022112917595172400_B39","article-title":"Trees, forests, and impurity-based variable importance","author":"Scornet,","year":"2021"},{"key":"2022112917595172400_B40","doi-asserted-by":"crossref","first-page":"1716","DOI":"10.1214\/15-AOS1321","article-title":"Consistency of random forests","volume":"43","author":"Scornet,","year":"2015","journal-title":"Ann. Statist."},{"key":"2022112917595172400_B41","first-page":"307","article-title":"A value for n-person games","volume-title":"Contributions to the Theory of Games","author":"Shapley,","year":"1953"},{"key":"2022112917595172400_B42","first-page":"407","article-title":"Sensitivity estimates for nonlinear mathematical models","volume":"1","author":"Sobol,","year":"1993","journal-title":"Math. Mod. Comp. Exp."},{"key":"2022112917595172400_B43","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1186\/1471-2105-9-307","article-title":"Conditional variable importance for random forests","volume":"9","author":"Strobl,","year":"2008","journal-title":"BMC Bioinformatics"},{"key":"2022112917595172400_B44","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1186\/1471-2105-8-25","article-title":"Bias in random forest variable importance measures: illustrations, sources and a solution","volume":"8","author":"Strobl,","year":"2007","journal-title":"BMC Bioinformatics"},{"key":"2022112917595172400_B45","article-title":"Danger: high power!\u2013exploring the statistical properties of a test for random forest variable importance","volume-title":"Proc. 18th Int. Conf. Comp Statist.","author":"Strobl,","year":"2008"},{"key":"2022112917595172400_B46","doi-asserted-by":"crossref","first-page":"1986","DOI":"10.1093\/bioinformatics\/btr300","article-title":"Classification with correlated features: unreliability of feature ranking and solutions","volume":"27","author":"Tolo\u015fi,","year":"2011","journal-title":"Bioinformatics"},{"key":"2022112917595172400_B47","doi-asserted-by":"crossref","first-page":"1228","DOI":"10.1080\/01621459.2017.1319839","article-title":"Estimation and inference of heterogeneous treatment effects using random forests","volume":"113","author":"Wager,","year":"2018","journal-title":"J. Am. Statist. Assoc."},{"key":"2022112917595172400_B48","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.2021.2003200","article-title":"A general framework for inference on algorithm-agnostic variable importance","author":"Williamson,","year":"2022","journal-title":"J. Am. Statist. Assoc."},{"key":"2022112917595172400_B49","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v077.i01","article-title":"ranger: a fast implementation of random forests for high dimensional data in C++ and R","volume":"77","author":"Wright,","year":"2017","journal-title":"J. Statist. Software"},{"key":"2022112917595172400_B50","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3429445","article-title":"Unbiased measurement of feature importance in tree-based methods","volume":"15","author":"Zhou,","year":"2021","journal-title":"ACM Trans. Know. Disc. Data"},{"key":"2022112917595172400_B51","doi-asserted-by":"crossref","first-page":"1770","DOI":"10.1080\/01621459.2015.1036994","article-title":"Reinforcement learning trees","volume":"110","author":"Zhu,","year":"2015","journal-title":"J. Am. Statist. Assoc."}],"container-title":["Biometrika"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/biomet\/advance-article-pdf\/doi\/10.1093\/biomet\/asac017\/43557265\/asac017.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/biomet\/article-pdf\/109\/4\/881\/47415734\/asac017.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/biomet\/article-pdf\/109\/4\/881\/47415734\/asac017.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,11,29]],"date-time":"2022-11-29T13:02:33Z","timestamp":1669726953000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/biomet\/article\/109\/4\/881\/6536961"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,2,25]]},"references-count":51,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2022,2,25]]},"published-print":{"date-parts":[[2022,11,29]]}},"URL":"https:\/\/doi.org\/10.1093\/biomet\/asac017","relation":{},"ISSN":["0006-3444","1464-3510"],"issn-type":[{"value":"0006-3444","type":"print"},{"value":"1464-3510","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022,12,1]]},"published":{"date-parts":[[2022,2,25]]}}}