{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,4]],"date-time":"2026-11-04T23:54:45Z","timestamp":1793836485521,"version":"4.6.1"},"reference-count":37,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,9,19]],"date-time":"2023-09-19T00:00:00Z","timestamp":1695081600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,9,19]],"date-time":"2023-09-19T00:00:00Z","timestamp":1695081600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"German federal state of Hesse"},{"name":"German federal state of Hesse"},{"DOI":"10.13039\/501100002347","name":"Bundesministerium f\u00fcr Bildung und Forschung","doi-asserted-by":"crossref","award":["13FH019KX1"],"award-info":[{"award-number":["13FH019KX1"]}],"id":[{"id":"10.13039\/501100002347","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100002347","name":"Bundesministerium f\u00fcr Bildung und Forschung","doi-asserted-by":"crossref","award":["13FH019KX1"],"award-info":[{"award-number":["13FH019KX1"]}],"id":[{"id":"10.13039\/501100002347","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100018453","name":"Hochschule Darmstadt University of Applied Sciences","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100018453","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Res Methodol"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    Random Forests are a powerful and frequently applied Machine Learning tool. The permutation variable importance (VIMP) has been proposed to improve the explainability of such a pure prediction model. It describes the expected increase in prediction error after randomly permuting a variable and disturbing its association with the outcome. However, VIMPs measure a variable\u2019s marginal influence only, that can make its interpretation difficult or even misleading. In the present work we address the general need for improving the explainability of prediction models by exploring VIMPs in the presence of correlated variables. In particular, we propose to use a variable\u2019s residual information for investigating if its permutation importance partially or totally originates from correlated predictors. Hypotheses tests are derived by a resampling algorithm that can further support results by providing test decisions and\n                    <jats:italic>p<\/jats:italic>\n                    -values. In simulation studies we show that the proposed test controls type I error rates. When applying the methods to a Random Forest analysis of post-transplant survival after kidney transplantation, the importance of kidney donor quality for predicting post-transplant survival is shown to be high. However, the transplant allocation policy introduces correlations with other well-known predictors, which raises the concern that the importance of kidney donor quality may simply originate from these predictors. By using the proposed method, this concern is addressed and it is demonstrated that kidney donor quality plays an important role in post-transplant survival, regardless of correlations with other predictors.\n                  <\/jats:p>","DOI":"10.1186\/s12874-023-02023-2","type":"journal-article","created":{"date-parts":[[2023,9,19]],"date-time":"2023-09-19T04:03:50Z","timestamp":1695096230000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":23,"title":["Exploring the variable importance in random forests under correlations: a general concept applied to donor organ quality in post-transplant survival"],"prefix":"10.1186","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7136-298X","authenticated-orcid":false,"given":"Christoph","family":"Wies","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7385-1151","authenticated-orcid":false,"given":"Robert","family":"Miltenberger","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8489-1589","authenticated-orcid":false,"given":"Gunter","family":"Grieser","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2397-5340","authenticated-orcid":false,"given":"Antje","family":"Jahn-Eimermacher","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,9,19]]},"reference":[{"key":"2023_CR1","doi-asserted-by":"crossref","unstructured":"Hart A, et al. OPTN\/SRTR 2016 Annual Data Report: Kidney. Am J Transplant. 2018;Suppl 1(Suppl 1):18\u2013113.","DOI":"10.1111\/ajt.14557"},{"key":"2023_CR2","doi-asserted-by":"publisher","first-page":"231","DOI":"10.1097\/TP.0b013e3181ac620b","volume":"88","author":"P Rao","year":"2009","unstructured":"Rao P, et al. A comprehensive risk quantification score for deceased donor kidneys: the kidney donor risk index. Transplantation. 2009;88:231\u20136.","journal-title":"Transplantation."},{"issue":"8","key":"2023_CR3","doi-asserted-by":"publisher","first-page":"1842","DOI":"10.1681\/ASN.2013070784","volume":"25","author":"A Israni","year":"2014","unstructured":"Israni A, et al. New national allocation policy for deceased donor kidneys in the United States and possible effect on patient outcomes. J Am Soc Nephrol. 2014;25(8):1842\u20138.","journal-title":"J Am Soc Nephrol."},{"issue":"4","key":"2023_CR4","doi-asserted-by":"publisher","first-page":"399","DOI":"10.1097\/MOT.0000000000000774","volume":"25","author":"D Guijo-Rubio","year":"2020","unstructured":"Guijo-Rubio D, Guti\u00e9rrez P, Herv\u00e1s-Mart\u00ednez C. Machine learning methods in organ transplantation. Curr Opin Organ Transplant. 2020;25(4):399\u2013405.","journal-title":"Curr Opin Organ Transplant."},{"issue":"4","key":"2023_CR5","doi-asserted-by":"publisher","first-page":"393","DOI":"10.1097\/MOT.0000000000000775","volume":"25","author":"J Brice\u00f1o","year":"2020","unstructured":"Brice\u00f1o J. Artificial intelligence and organ transplantation: challenges and expectations. Curr Opin Organ Transplant. 2020;25(4):393\u20138.","journal-title":"Curr Opin Organ Transplant."},{"key":"2023_CR6","doi-asserted-by":"crossref","unstructured":"Ravindhran B, et al. Machine learning models in predicting graft survival in kidney transplantation: meta-analysis. BJS Open. 2023;7(2):zrad011.","DOI":"10.1093\/bjsopen\/zrad011"},{"issue":"11","key":"2023_CR7","doi-asserted-by":"publisher","first-page":"1472","DOI":"10.1111\/tri.13695","volume":"33","author":"S Bae","year":"2020","unstructured":"Bae S, Massie AB, Caffo BS, Jackson KR, Segev DL. Machine learning to predict transplant outcomes: helpful or hype? A national cohort study. Transpl Int. 2020;33(11):1472\u201380.","journal-title":"Transpl Int."},{"issue":"5","key":"2023_CR8","doi-asserted-by":"publisher","first-page":"936","DOI":"10.1016\/j.kint.2022.12.011","volume":"103","author":"A Truchot","year":"2023","unstructured":"Truchot A, et al. Machine learning does not outperform traditional statistical modelling for kidney allograft failure prediction. Kidney Int. 2023;103(5):936\u201348.","journal-title":"Kidney Int."},{"key":"2023_CR9","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","volume":"45","author":"L Breiman","year":"2001","unstructured":"Breiman L. Random Forests. Mach Learn. 2001;45:5\u201332.","journal-title":"Mach Learn."},{"issue":"3","key":"2023_CR10","doi-asserted-by":"publisher","first-page":"841","DOI":"10.1214\/08-AOAS169","volume":"2","author":"H Ishwaran","year":"2008","unstructured":"Ishwaran H, Kogalur UB, Blackstone EH, Lauer MS. Random survival forests. Ann Appl Stat. 2008;2(3):841\u201360.","journal-title":"Ann Appl Stat."},{"key":"2023_CR11","doi-asserted-by":"publisher","first-page":"4793","DOI":"10.1109\/TNNLS.2020.3027314","volume":"32","author":"E Tjoa","year":"2020","unstructured":"Tjoa E, Guan C. A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI. IEEE Trans Neural Netw Learn Syst. 2020;32:4793\u2013813.","journal-title":"IEEE Trans Neural Netw Learn Syst."},{"key":"2023_CR12","doi-asserted-by":"publisher","first-page":"519","DOI":"10.1214\/07-EJS039","volume":"1","author":"H Ishwaran","year":"2007","unstructured":"Ishwaran H. Variable importance in binary regression trees and forests. Electron J Stat. 2007;1:519\u201337.","journal-title":"Electron J Stat."},{"key":"2023_CR13","first-page":"431","volume":"26","author":"G Louppe","year":"2013","unstructured":"Louppe G, Wehenkel L, Sutera A, Geurts P. Understanding variable importances in forests of randomized trees. Adv Neural Inf Process Syst. 2013;26:431\u20139.","journal-title":"Adv Neural Inf Process Syst."},{"issue":"1","key":"2023_CR14","doi-asserted-by":"publisher","first-page":"230","DOI":"10.1186\/s12859-017-1650-8","volume":"18","author":"I Epifanio","year":"2017","unstructured":"Epifanio I. Intervention in prediction measure: a new approach to assessing variable importance for random forests. BMC Bioinformatics. 2017;18(1):230.","journal-title":"BMC Bioinformatics."},{"issue":"530","key":"2023_CR15","doi-asserted-by":"publisher","first-page":"636","DOI":"10.1080\/01621459.2020.1762613","volume":"115","author":"B Efron","year":"2020","unstructured":"Efron B. Prediction, Estimation, and Attribution. J Am Stat Assoc. 2020;115(530):636\u201355.","journal-title":"J Am Stat Assoc."},{"key":"2023_CR16","unstructured":"Paluszynska A, Biecek P, Jiang Y. randomForestExplainer: explaining and visualizing Random Forests in terms of variable importance. R package version 0.10.1. 2020. https:\/\/CRAN.R-project.org\/package=randomForestExplainer."},{"issue":"3","key":"2023_CR17","doi-asserted-by":"publisher","first-page":"659","DOI":"10.1007\/s11222-016-9646-1","volume":"27","author":"B Gregorutti","year":"2017","unstructured":"Gregorutti B, Michel B, Saint-Pierre P. Correlation and variable importance in random forests. Stat Comput. 2017;27(3):659\u201378.","journal-title":"Stat Comput."},{"issue":"1","key":"2023_CR18","doi-asserted-by":"publisher","first-page":"307","DOI":"10.1186\/s12859-020-03622-2","volume":"21","author":"D Debeer","year":"2020","unstructured":"Debeer D, Strobl C. Conditional permutation importance revisited. BMC Bioinformatics. 2020;21(1):307.","journal-title":"BMC Bioinformatics."},{"issue":"8","key":"2023_CR19","doi-asserted-by":"publisher","first-page":"2107","DOI":"10.1007\/s10994-021-06030-6","volume":"110","author":"D Watson","year":"2021","unstructured":"Watson D, Wright M. Testing conditional independence in supervised learning algorithms. Mach Learn. 2021;110(8):2107\u201329.","journal-title":"Mach Learn."},{"issue":"3","key":"2023_CR20","doi-asserted-by":"publisher","first-page":"551","DOI":"10.1111\/rssb.12265","volume":"80","author":"E Cand\u00e8s","year":"2018","unstructured":"Cand\u00e8s E, Fan Y, Janson L, Lv J. Panning for gold: \u2018model-X\u2019 knockoffs for high dimensional controlled variable selection. J R Stat Soc Ser B (Stat Methodol). 2018;80(3):551\u201377.","journal-title":"J R Stat Soc Ser B (Stat Methodol)."},{"issue":"1","key":"2023_CR21","doi-asserted-by":"publisher","first-page":"307","DOI":"10.1186\/1471-2105-9-307","volume":"9","author":"C Strobl","year":"2008","unstructured":"Strobl C, Boulesteix A, Kneib T, Augustin T, Zeileis A. Conditional variable importance for random forests. BMC Bioinformatics. 2008;9(1):307.","journal-title":"BMC Bioinformatics."},{"issue":"1","key":"2023_CR22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18637\/jss.v077.i01","volume":"77","author":"M Wright","year":"2017","unstructured":"Wright M, Ziegler A. ranger: a fast implementation of Random Forests for high dimensional data in C++ and R. J Stat Softw. 2017;77(1):1\u201317.","journal-title":"J Stat Softw."},{"issue":"8","key":"2023_CR23","doi-asserted-by":"publisher","first-page":"e1910312","DOI":"10.1001\/jamanetworkopen.2019.10312","volume":"2","author":"S Husain","year":"2019","unstructured":"Husain S, et al. Association between declined offers of deceased donor kidney allograft and outcomes in kidney transplant candidates. JAMA Netw Open. 2019;2(8):e1910312.","journal-title":"JAMA Netw Open."},{"key":"2023_CR24","unstructured":"Organ Procurement and Transplantation Network: Data Request. https:\/\/optn.transplant.hrsa.gov\/data\/request-data\/. Accessed 1 Jan 2023."},{"issue":"3","key":"2023_CR25","doi-asserted-by":"publisher","first-page":"355","DOI":"10.1093\/biostatistics\/kxj011","volume":"7","author":"T Hothorn","year":"2006","unstructured":"Hothorn T, B\u00fchlmann P, Dudoit S, Molinaro A, Van Der Laan MJ. Survival ensembles. Biostatistics. 2006;7(3):355\u201373.","journal-title":"Biostatistics"},{"key":"2023_CR26","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-84858-7","volume-title":"The elements of statistical learning data (2nd)","author":"T Hastie","year":"2009","unstructured":"Hastie T, Tibshirani R, Friedman J. The elements of statistical learning data (2nd). US: Springer; 2009."},{"issue":"4","key":"2023_CR27","doi-asserted-by":"publisher","first-page":"558","DOI":"10.1002\/sim.7803","volume":"38","author":"H Ishwaran","year":"2019","unstructured":"Ishwaran H, Lu M. Standard errors and confidence intervals for variable importance in random forest regression, classification, and survival. Stat Med. 2019;38(4):558\u201382.","journal-title":"Stat Med."},{"key":"2023_CR28","doi-asserted-by":"publisher","first-page":"68","DOI":"10.1093\/ndt\/gfw405","volume":"23","author":"R Kabore","year":"2017","unstructured":"Kabore R, Haller MC, Harambat J, Heinze G, Leffondre K. Risk prediction models for graft failure in kidney transplantation: a systematic review. Nephrol Dial Transplant. 2017;23:68\u201376.","journal-title":"Nephrol Dial Transplant."},{"key":"2023_CR29","doi-asserted-by":"publisher","first-page":"331","DOI":"10.1186\/s12874-022-01823-2","volume":"22","author":"M Gholamzadeh","year":"2022","unstructured":"Gholamzadeh M, Abtahi H, Safdari R. Machine learning-based techniques to improve lung transplantation outcomes and complications: a systematic review. BMC Med Res Methodol. 2022;22:331.","journal-title":"BMC Med Res Methodol."},{"key":"2023_CR30","doi-asserted-by":"publisher","first-page":"89","DOI":"10.1038\/s41746-022-00637-2","volume":"5","author":"N Gotlieb","year":"2022","unstructured":"Gotlieb N, et al. The promise of machine learning applications in solid organ transplantation. NPJ Digit Med. 2022;5:89.","journal-title":"NPJ Digit Med."},{"key":"2023_CR31","doi-asserted-by":"publisher","first-page":"10014","DOI":"10.1038\/s41598-019-46202-y","volume":"9","author":"M Ellies-Oury","year":"2019","unstructured":"Ellies-Oury M, et al. Statistical model choice including variable selection based on variable importance: A relevant way for biomarkers selection to predict meat tenderness. Sci Rep. 2019;9:10014.","journal-title":"Sci Rep."},{"issue":"2","key":"2023_CR32","doi-asserted-by":"publisher","first-page":"492","DOI":"10.1093\/bib\/bbx124","volume":"20","author":"F Degenhardt","year":"2019","unstructured":"Degenhardt F, Seifert S, Szymczak S. Evaluation of variable selection methods for random forests and omics data sets. Brief Bioinforma. 2019;20(2):492\u2013503.","journal-title":"Brief Bioinforma."},{"issue":"489","key":"2023_CR33","doi-asserted-by":"publisher","first-page":"205","DOI":"10.1198\/jasa.2009.tm08622","volume":"105","author":"H Ishwaran","year":"2010","unstructured":"Ishwaran H, Kogalur UB, Gorodeski EZ, Minn AJ, Lauer MS. High-dimensional variable selection for survival data. J Am Stat Assoc. 2010;105(489):205\u201317.","journal-title":"J Am Stat Assoc."},{"key":"2023_CR34","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1016\/j.eswa.2019.05.028","volume":"134","author":"JL Speiser","year":"2019","unstructured":"Speiser JL, Miller ME, Tooze J, Ip E. A comparison of random forest variable selection methods for classification prediction modeling. Expert Syst Appl. 2019;134:93\u2013101.","journal-title":"Expert Syst Appl."},{"issue":"1","key":"2023_CR35","first-page":"1","volume":"23","author":"A Bommert","year":"2021","unstructured":"Bommert A, Welchowski T, Schmid M, Rahnenf\u00fchrer J. Benchmark of filter methods for feature selection in high-dimensional gene expression survival data. Brief Bioinforma. 2021;23(1):1\u201313.","journal-title":"Brief Bioinforma."},{"key":"2023_CR36","doi-asserted-by":"publisher","first-page":"107689","DOI":"10.1016\/j.csda.2022.107689","volume":"181","author":"A Hapfelmeier","year":"2023","unstructured":"Hapfelmeier A, Hornung R, Haller B. Efficient permutation testing of variable importance measures by the example of random forests. Comput Stat Data Anal. 2023;181:107689.","journal-title":"Comput Stat Data Anal."},{"issue":"2","key":"2023_CR37","doi-asserted-by":"publisher","first-page":"425","DOI":"10.1111\/ajt.14978","volume":"19","author":"S Bae","year":"2019","unstructured":"Bae S, et al. Who can tolerate a marginal kidney? Predicting survival after deceased donor kidney transplant by donor-recipient combination. Am J Transplant. 2019;19(2):425\u201333.","journal-title":"Am J Transplant."}],"container-title":["BMC Medical Research Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12874-023-02023-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12874-023-02023-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12874-023-02023-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,17]],"date-time":"2023-11-17T13:07:20Z","timestamp":1700226440000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedresmethodol.biomedcentral.com\/articles\/10.1186\/s12874-023-02023-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,19]]},"references-count":37,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["2023"],"URL":"https:\/\/doi.org\/10.1186\/s12874-023-02023-2","relation":{},"ISSN":["1471-2288"],"issn-type":[{"value":"1471-2288","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,19]]},"assertion":[{"value":"1 March 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 August 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 September 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The data analysis presented in this manuscript uses retrospective and anonymized data provided by UNOS, where informed consent has been given by each subject and\/or their legal guardian(s). Therefore, for this research the definition of human subjects research does not apply and all methods were carried out in accordance with relevant guidelines and regulations. Ethical approval was waived by the Ethics Committee of the State Medical Association of Hesse.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"209"}}