{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,26]],"date-time":"2026-04-26T20:53:50Z","timestamp":1777236830583,"version":"3.51.4"},"reference-count":16,"publisher":"Oxford University Press (OUP)","issue":"10","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2010,5,15]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Motivation: In life sciences, interpretability of machine learning models is as important as their prediction accuracy. Linear models are probably the most frequently used methods for assessing feature relevance, despite their relative inflexibility. However, in the past years effective estimators of feature relevance have been derived for highly complex or non-parametric models such as support vector machines and RandomForest (RF) models. Recently, it has been observed that RF models are biased in such a way that categorical variables with a large number of categories are preferred.<\/jats:p>\n               <jats:p>Results: In this work, we introduce a heuristic for normalizing feature importance measures that can correct the feature importance bias. The method is based on repeated permutations of the outcome vector for estimating the distribution of measured importance for each variable in a non-informative setting. The P-value of the observed importance provides a corrected measure of feature importance. We apply our method to simulated data and demonstrate that (i) non-informative predictors do not receive significant P-values, (ii) informative variables can successfully be recovered among non-informative variables and (iii) P-values computed with permutation importance (PIMP) are very helpful for deciding the significance of variables, and therefore improve model interpretability. Furthermore, PIMP was used to correct RF-based importance measures for two real-world case studies. We propose an improved RF model that uses the significant variables with respect to the PIMP measure and show that its prediction accuracy is superior to that of other existing models.<\/jats:p>\n               <jats:p>Availability: R code for the method presented in this article is available at http:\/\/www.mpi-inf.mpg.de\/\u223caltmann\/download\/PIMP.R<\/jats:p>\n               <jats:p>Contact: \u00a0altmann@mpi-inf.mpg.de, laura.tolosi@mpi-inf.mpg.de<\/jats:p>\n               <jats:p>Supplementary information: \u00a0Supplementary data are available at Bioinformatics online.<\/jats:p>","DOI":"10.1093\/bioinformatics\/btq134","type":"journal-article","created":{"date-parts":[[2010,4,13]],"date-time":"2010-04-13T02:32:32Z","timestamp":1271125952000},"page":"1340-1347","source":"Crossref","is-referenced-by-count":2386,"title":["Permutation importance: a corrected feature importance measure"],"prefix":"10.1093","volume":"26","author":[{"given":"Andr\u00e9","family":"Altmann","sequence":"first","affiliation":[{"name":"Department of Computational Biology and Applied Algorithmics, Max Planck Institute for Informatics, Saarbr\u00fccken, Germany"}]},{"given":"Laura","family":"Tolo\u015fi","sequence":"additional","affiliation":[{"name":"Department of Computational Biology and Applied Algorithmics, Max Planck Institute for Informatics, Saarbr\u00fccken, Germany"}]},{"given":"Oliver","family":"Sander","sequence":"additional","affiliation":[{"name":"Department of Computational Biology and Applied Algorithmics, Max Planck Institute for Informatics, Saarbr\u00fccken, Germany"}]},{"given":"Thomas","family":"Lengauer","sequence":"additional","affiliation":[{"name":"Department of Computational Biology and Applied Algorithmics, Max Planck Institute for Informatics, Saarbr\u00fccken, Germany"}]}],"member":"286","published-online":{"date-parts":[[2010,4,12]]},"reference":[{"key":"2023012507511769800_B1","doi-asserted-by":"crossref","first-page":"965","DOI":"10.1016\/j.sigpro.2004.11.020","article-title":"Criteria based on mutual information minimization for blind source separation in post nonlinear mixtures","volume":"85","author":"Achard","year":"2005","journal-title":"Signal Processing"},{"key":"2023012507511769800_B2","doi-asserted-by":"crossref","first-page":"537","DOI":"10.1109\/72.298224","article-title":"Using mutual information for selecting features in supervised neural net learning","volume":"5","author":"Battiti","year":"1994","journal-title":"IEEE Trans. Neural Netw."},{"key":"2023012507511769800_B3","doi-asserted-by":"crossref","first-page":"901","DOI":"10.2307\/1914138","article-title":"Decomposable income inequality measures","volume":"47","author":"Bourguignon","year":"1979","journal-title":"Econometrica"},{"key":"2023012507511769800_B4","volume-title":"Classification and regression trees.","author":"Breiman","year":"1984"},{"key":"2023012507511769800_B5","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random Forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn."},{"key":"2023012507511769800_B6","doi-asserted-by":"crossref","first-page":"132","DOI":"10.1186\/1471-2105-5-132","article-title":"Simple statistical models predict C-to-U edited sites in plant mitochondrial RNA","volume":"5","author":"Cummings","year":"2004","journal-title":"BMC Bioinformatics"},{"key":"2023012507511769800_B7","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1186\/1471-2105-7-3","article-title":"Gene selection and classification of microarray data using random forest","volume":"7","author":"Di\u00e1z-Uriarte","year":"2006","journal-title":"BMC Bioinformatics"},{"key":"2023012507511769800_B8","first-page":"239","article-title":"The permutations test for feature selection by mutual information","volume-title":"ESANN 2006, European Symposium on Artificial Neural Networks","author":"Fran\u00e7ois","year":"2006"},{"key":"2023012507511769800_B9","first-page":"1157","article-title":"An introduction to variable and feature selection","volume":"3","author":"Guyon","year":"2003","journal-title":"J. Mach. Learn. Res."},{"key":"2023012507511769800_B10","doi-asserted-by":"crossref","DOI":"10.1007\/978-0-387-21606-5","volume-title":"The Elements of Statistical Learning.","author":"Hastie","year":"2001"},{"key":"2023012507511769800_B11","doi-asserted-by":"crossref","first-page":"651","DOI":"10.1198\/106186006X133933","article-title":"Unbiased recursive partitioning: a conditional inference framework","volume":"15","author":"Hothorn","year":"2006","journal-title":"J. Comput. Graph. Stat."},{"key":"2023012507511769800_B12","doi-asserted-by":"crossref","first-page":"1407","DOI":"10.1038\/nbt1371","article-title":"Bioinformatics prediction of HIV coreceptor usage","volume":"25","author":"Lengauer","year":"2007","journal-title":"Nat. Biotechnol."},{"key":"2023012507511769800_B13","doi-asserted-by":"crossref","first-page":"451","DOI":"10.2307\/1885088","article-title":"The distribution of income by factor components","volume":"95","author":"Pyatt","year":"1980","journal-title":"Q. J. Econ."},{"key":"2023012507511769800_B14","doi-asserted-by":"crossref","first-page":"i6","DOI":"10.1093\/bioinformatics\/btn170","article-title":"POIMs: positional oligomer importance matrices \u2013 understanding support vector machine-based signal detectors","volume":"24","author":"Sonnenburg","year":"2008","journal-title":"Bioinformatics"},{"key":"2023012507511769800_B15","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1186\/1471-2105-8-25","article-title":"Bias in random forest variable importance measures: Illustrations, sources and a solution","volume":"8","author":"Strobl","year":"2007","journal-title":"BMC Bioinformatics"},{"issue":"Suppl. 3","key":"2023012507511769800_B16","first-page":"A100","article-title":"Improved genotypic prediction of HIV-1 coreceptor usage by incorporating V2 loop sequence variation","volume":"13","author":"Thielen","year":"2008","journal-title":"Antivir. Ther."}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/26\/10\/1340\/48851160\/bioinformatics_26_10_1340.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/26\/10\/1340\/48851160\/bioinformatics_26_10_1340.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,25]],"date-time":"2023-01-25T07:51:33Z","timestamp":1674633093000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/26\/10\/1340\/193348"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2010,4,12]]},"references-count":16,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2010,5,15]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btq134","relation":{},"ISSN":["1367-4811","1367-4803"],"issn-type":[{"value":"1367-4811","type":"electronic"},{"value":"1367-4803","type":"print"}],"subject":[],"published-other":{"date-parts":[[2010,5,15]]},"published":{"date-parts":[[2010,4,12]]}}}