{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,11]],"date-time":"2026-07-11T02:23:49Z","timestamp":1783736629843,"version":"3.55.0"},"reference-count":43,"publisher":"Springer Science and Business Media LLC","issue":"1","content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Bioinformatics"],"published-print":{"date-parts":[[2009,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Background<\/jats:title><jats:p>Regularized regression methods such as principal component or partial least squares regression perform well in learning tasks on high dimensional spectral data, but cannot explicitly eliminate irrelevant features. The random forest classifier with its associated Gini feature importance, on the other hand, allows for an explicit feature elimination, but may not be optimally adapted to spectral data due to the topology of its constituent classification trees which are based on orthogonal splits in feature space.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>We propose to combine the best of both approaches, and evaluated the joint use of a feature selection based on a recursive feature elimination using the Gini importance of random forests' together with regularized classification methods on spectral data sets from medical diagnostics, chemotaxonomy, biomedical analytics, food science, and synthetically modified spectral data. Here, a feature selection using the Gini feature importance with a regularized classification by discriminant partial least squares regression performed as well as or better than a filtering according to different univariate statistical tests, or using regression coefficients in a backward feature elimination. It outperformed the direct application of the random forest classifier, or the direct application of the regularized classifiers on the full set of features.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p>The Gini importance of the random forest provided superior means for measuring feature relevance on spectral data, but \u2013 on an optimal subset of features \u2013 the regularized classifiers might be preferable over the random forest classifier, in spite of their limitation to model linear dependencies only. A feature selection based on Gini importance, however, may precede a regularized linear classification to identify this optimal subset of features, and to earn a double benefit of both dimensionality reduction and the elimination of noise from the classification task.<\/jats:p><\/jats:sec>","DOI":"10.1186\/1471-2105-10-213","type":"journal-article","created":{"date-parts":[[2009,7,10]],"date-time":"2009-07-10T18:50:42Z","timestamp":1247251842000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1134,"title":["A comparison of random forest and its Gini importance with standard chemometric methods for the feature selection and classification of spectral data"],"prefix":"10.1186","volume":"10","author":[{"given":"Bjoern H","family":"Menze","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"B Michael","family":"Kelm","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ralf","family":"Masuch","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Uwe","family":"Himmelreich","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peter","family":"Bachert","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wolfgang","family":"Petrich","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fred A","family":"Hamprecht","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2009,7,10]]},"reference":[{"key":"2943_CR1","doi-asserted-by":"publisher","first-page":"1157","DOI":"10.1162\/153244303322753616","volume":"3","author":"I Guyon","year":"2003","unstructured":"Guyon I, Elisseeff A: An introduction to variable and feature selection. J Mach Learn Res 2003, 3: 1157\u201382. 10.1162\/153244303322753616","journal-title":"J Mach Learn Res"},{"key":"2943_CR2","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1111\/j.2517-6161.1990.tb01786.x","volume":"52","author":"M Stone","year":"1990","unstructured":"Stone M, J R, Brooks Continuum regression: Cross-validated sequentially constructed prediction embracing ordinary least squares, partial least squares and principal components regression. J Roy Stat Soc B (Meth) 1990, 52: 237\u2013269.","journal-title":"J Roy Stat Soc B (Meth)"},{"key":"2943_CR3","doi-asserted-by":"publisher","first-page":"109","DOI":"10.1080\/00401706.1993.10485033","volume":"35","author":"IE Frank","year":"1993","unstructured":"Frank IE, Friedman JH: A statistical view of some Chemometrics regression tools. Technometrics 1993, 35: 109\u2013135. 10.2307\/1269656","journal-title":"Technometrics"},{"key":"2943_CR4","doi-asserted-by":"publisher","first-page":"106","DOI":"10.1186\/1471-2105-9-106","volume":"9","author":"M Bylesj\u00f6","year":"2008","unstructured":"Bylesj\u00f6 M, Rantalainen M, Nicholson JK, Holmes E, Trygg J: K-OPLS package: Kernel-based orthogonal projections to latent structures for prediction and interpretation in feature space. BMC Bioinformatics 2008, 9: 106. 10.1186\/1471-2105-9-106","journal-title":"BMC Bioinformatics"},{"key":"2943_CR5","doi-asserted-by":"publisher","first-page":"117","DOI":"10.1255\/jnirs.271","volume":"117","author":"F Westad","year":"2000","unstructured":"Westad F, Martens H: Variable selection in near infrared spectroscopy based on significance testing in partial least squares regression. J Near Infrared Spectrosc 2000, 117: 117\u2013124. 10.1255\/jnirs.271","journal-title":"J Near Infrared Spectrosc"},{"key":"2943_CR6","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1002\/cem.915","volume":"19","author":"B Nadler","year":"2005","unstructured":"Nadler B, Coifman RR: The prediction error in CLS and PLS: the importance of feature selection prior to multivariate calibration. J Chemometrics 2005, 19: 107\u2013118. 10.1002\/cem.915","journal-title":"J Chemometrics"},{"key":"2943_CR7","doi-asserted-by":"publisher","first-page":"515","DOI":"10.2307\/2986329","volume":"42","author":"MC Denham","year":"1993","unstructured":"Denham MC, Brown PJ: Calibration with many variables. Appl Stat 1993, 42: 515\u2013528. 10.2307\/2986329","journal-title":"Appl Stat"},{"key":"2943_CR8","doi-asserted-by":"publisher","first-page":"339","DOI":"10.1002\/cem.730","volume":"16","author":"K Baumann","year":"2002","unstructured":"Baumann K, von Korff M, Albert H: Asystematic evaluation of the benefits and hazards of variable selection in latent variable regression. Part I. Search Algorithm, theory and simulations. J Chemometrics 2002, 16: 339\u2013350. 10.1002\/cem.730","journal-title":"J Chemometrics"},{"key":"2943_CR9","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1186\/gb-2000-1-2-research0003","volume":"1","author":"T Hastie","year":"2000","unstructured":"Hastie T, Tibshirani R, Eisen MB, Alizadeh A, Levy R, Staudt L, Chan WC, Botstein D, Brown P: 'Gene shaving' as a method for identifying distinct sets of genes with similar expression patterns. Genome Biol 2000, 1: 3. 10.1186\/gb-2000-1-2-research0003","journal-title":"Genome Biol"},{"key":"2943_CR10","doi-asserted-by":"publisher","first-page":"S8","DOI":"10.1186\/1471-2105-9-S6-S8","volume":"9","author":"XQ Zeng","year":"2008","unstructured":"Zeng XQ, Li GZ, Yang JY, Yang MQ, Wu GF: Dimension reduction with redundant gene elimination for tumor classification. BMC Bioinformatics 2008, 9: S8. 10.1186\/1471-2105-9-S6-S8","journal-title":"BMC Bioinformatics"},{"key":"2943_CR11","doi-asserted-by":"publisher","first-page":"559","DOI":"10.1002\/cem.651","volume":"15","author":"R Leardi","year":"2001","unstructured":"Leardi R: Genetic algorithms in chemometrics and chemistry: a review. J Chemometrics 2001, 15: 559\u2013569. 10.1002\/cem.651","journal-title":"J Chemometrics"},{"key":"2943_CR12","doi-asserted-by":"publisher","first-page":"3851","DOI":"10.1021\/ac960321m","volume":"68","author":"V Centner","year":"1996","unstructured":"Centner V, Massart DL, de Noord OE, de Jong S, Vandeginste BM, Sterna C: Elimination of uninformative variables for multivariate calibration. Anal Chem 1996, 68: 3851\u201358. 10.1021\/ac960321m","journal-title":"Anal Chem"},{"key":"2943_CR13","doi-asserted-by":"publisher","first-page":"165","DOI":"10.1002\/(SICI)1099-128X(199903\/04)13:2<165::AID-CEM535>3.0.CO;2-Y","volume":"13","author":"M Forina","year":"1999","unstructured":"Forina M, Casolino C, Millan CP: Iterative predictor weighting (IPW) PLS: a technique for the elimination of useless predictors in regression problems. J Chemometrics 1999, 13: 165\u201384. Publisher Full Text10.1002\/(SICI)1099-128X(199903\/04)13:2<165::AID-CEM535>3.0.CO;2-Y","journal-title":"J Chemometrics"},{"key":"2943_CR14","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","volume":"45","author":"L Breiman","year":"2001","unstructured":"Breiman L: Random forests. J Mach Learn 2001, 45: 5\u201332. 10.1023\/A:1010933404324","journal-title":"J Mach Learn"},{"key":"2943_CR15","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/1471-2105-5-81","volume":"5","author":"H Jiang","year":"2004","unstructured":"Jiang H, Deng Y, Chen H-S, Tao L, Sha Q, Chen J, Tsai C-J, Zhang S: Joint analysis of two microarray gene-expression data sets to select lung adenocarcinoma marker genes. BMC Bioinformatics 2004, 5: 1\u201312. 10.1186\/1471-2105-5-81","journal-title":"BMC Bioinformatics"},{"key":"2943_CR16","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/1471-2105-7-3","volume":"7","author":"R Diaz-Uriarte","year":"2006","unstructured":"Diaz-Uriarte R, Alvarez de Andres S: Gene selection and classification of microarray data using random forest. BMC Bioinformatics 2006, 7: 1\u201325. 10.1186\/1471-2105-7-3","journal-title":"BMC Bioinformatics"},{"key":"2943_CR17","doi-asserted-by":"publisher","first-page":"952","DOI":"10.1021\/ci050049u","volume":"45","author":"S Li","year":"2005","unstructured":"Li S, Fedorowicz A, Singh H, Soderholm SC: Application of the random forest method in studies of local lymph node assay based skin sensitization data. J Chem Inf Comp Sci 2005, 45: 952\u201364.","journal-title":"J Chem Inf Comp Sci"},{"key":"2943_CR18","doi-asserted-by":"publisher","first-page":"313","DOI":"10.1093\/bioinformatics\/bti494","volume":"21","author":"P Geurts","year":"2005","unstructured":"Geurts P, Fillet M, de Seny D, Meuwis M-A, Malaise M, Merville M-P, Wehenkel L: Proteomic mass spectra classification using decision tree based ensemble methods. Bioinformatics 2005, 21: 313\u2013845. 10.1093\/bioinformatics\/bti494","journal-title":"Bioinformatics"},{"key":"2943_CR19","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1198\/016214502753479248","volume":"97","author":"S Dudoit","year":"2002","unstructured":"Dudoit S, Fridlyand J, Speed TP: Comparison of discrimination methods for the classification of tumors using gene expression data. J Am Stat Assoc 2002, 97: 77\u201388. 10.1198\/016214502753479248","journal-title":"J Am Stat Assoc"},{"key":"2943_CR20","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1186\/1471-2105-9-319","volume":"9","author":"A Statnikov","year":"2008","unstructured":"Statnikov A, Wang L, Aliferis CF: A comprehensive comparison of random forests and support vector machines for microarray-based cancer classification. BMC Bioinformatics 2008, 9: 319. 10.1186\/1471-2105-9-319","journal-title":"BMC Bioinformatics"},{"key":"2943_CR21","doi-asserted-by":"publisher","first-page":"1231","DOI":"10.1109\/TBME.2007.890733","volume":"54","author":"KQ Shen","year":"2007","unstructured":"Shen KQ, Ong CJ, Li XP, Zheng H, Wilder-Smith EPV: A Feature Selection Method for Multi-Level Mental Fatigue EEG Classification. IEEE Trans Biomed Engin 2007, 54: 1231\u20131237. 10.1109\/TBME.2007.890733","journal-title":"IEEE Trans Biomed Engin"},{"key":"2943_CR22","doi-asserted-by":"publisher","first-page":"801","DOI":"10.1007\/s00216-006-1070-5","volume":"387","author":"BH Menze","year":"2007","unstructured":"Menze BH, Petrich W, Hamprecht FA: Multivariate feature selection and hierarchical classification for infrared spectroscopy: serum-based detection of bovine spongiform encephalopathy. Anal Bioanal Chem 2007, 387: 801\u20131807. 10.1007\/s00216-006-1070-5","journal-title":"Anal Bioanal Chem"},{"key":"2943_CR23","doi-asserted-by":"publisher","first-page":"83","DOI":"10.1016\/j.chemolab.2006.01.007","volume":"83","author":"P Granitto","year":"2006","unstructured":"Granitto P, Furlanello C, Biasioli F, Gasperi F: Recursive Feature Elimination with Random Forest for PTR-MS analysis of agroindustrial products. Chem Intell Lab Sys 2006, 83: 83\u201390. 10.1016\/j.chemolab.2006.01.007","journal-title":"Chem Intell Lab Sys"},{"key":"2943_CR24","doi-asserted-by":"publisher","first-page":"1947","DOI":"10.1021\/ci034160g","volume":"43","author":"V Svetnik","year":"2003","unstructured":"Svetnik V, Liaw A, Tong C, Culberson JC, Sheridan RP, Feuston BP: Random Forest: A Classification and Regression Tool for Compound Classification and QSAR Modeling. J Chem Inf Comp Sci 2003, 43: 1947\u201358.","journal-title":"J Chem Inf Comp Sci"},{"key":"2943_CR25","doi-asserted-by":"publisher","first-page":"578","DOI":"10.1198\/016214505000001230","volume":"101","author":"Y Lin","year":"2006","unstructured":"Lin Y, Jeon Y: Random Forests and adaptive nearest neighbor. J Am Stat Assoc 2006, 101: 578\u2013590. 10.1198\/016214505000001230","journal-title":"J Am Stat Assoc"},{"key":"2943_CR26","first-page":"2015","volume":"9","author":"G Biau","year":"2008","unstructured":"Biau G, Devroye L, Lugosi G: Consistency of Random Forests and Other Averaging Classifiers. J Mach Learn Res 2008, 9: 2015\u20132033.","journal-title":"J Mach Learn Res"},{"key":"2943_CR27","volume-title":"Technical Report 670","author":"L Breiman","year":"2004","unstructured":"Breiman L: Consistency for a simple model of random forests. In Technical Report 670. Technical report, Department of Statistics, University of California, Berkeley, USA; 2004."},{"key":"2943_CR28","doi-asserted-by":"publisher","first-page":"307","DOI":"10.1186\/1471-2105-9-307","volume":"9","author":"C Strobl","year":"2008","unstructured":"Strobl C, Boulesteix A-L, Kneib T, Augustin T, Zeileis A: Conditional variable importance for random forests. BMC Bioinformatics 2008, 9: 307. 10.1186\/1471-2105-9-307","journal-title":"BMC Bioinformatics"},{"issue":"Suppl 1","key":"2943_CR29","doi-asserted-by":"publisher","first-page":"S65","DOI":"10.1186\/1471-2105-10-S1-S65","volume":"10","author":"R Jiang","year":"2009","unstructured":"Jiang R, Tang W, Wu X, Fu W: A random forest approach to the detection of epistatic interactions in case-control studies. BMC Bioinformatics 2009, 10(Suppl 1):S65.","journal-title":"BMC Bioinformatics"},{"key":"2943_CR30","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1186\/1471-2105-8-25","volume":"8","author":"C Strobl","year":"2007","unstructured":"Strobl C, Boulesteix AL, Zeileis A, Hothorn T: Bias in Random Forest Variable Importance Measures: Illustrations, Sources and a Solution. BMC Bioinformatics 2007, 8: 25. 10.1186\/1471-2105-8-25","journal-title":"BMC Bioinformatics"},{"key":"2943_CR31","doi-asserted-by":"publisher","first-page":"611","DOI":"10.1198\/106186008X344522","volume":"17","author":"M Sandri","year":"2008","unstructured":"Sandri M, Zuccoletto P: A bias correction algorithm for the Gini variable importance measure in classification trees. J Comp Graph Stat 2008, 17: 611\u2013628. 10.1198\/106186008X344522","journal-title":"J Comp Graph Stat"},{"key":"2943_CR32","doi-asserted-by":"publisher","first-page":"2249","DOI":"10.1016\/j.csda.2007.08.015","volume":"52","author":"KJ Archer","year":"2008","unstructured":"Archer KJ, Kimes RV: Empirical characterization of random forest variable importance measures. Comp Stat Data Anal 2008, 52: 2249\u20132260. 10.1016\/j.csda.2007.08.015","journal-title":"Comp Stat Data Anal"},{"key":"2943_CR33","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1016\/S0169-7439(01)00158-7","volume":"58","author":"J-P Gauchi","year":"2001","unstructured":"Gauchi J-P, Chagnon P: Comparison of selection methods of explanatory variables in PLS regression with application to manufacturing process data. Chem Intell Lab Sys 2001, 58: 171\u201393. 10.1016\/S0169-7439(01)00158-7","journal-title":"Chem Intell Lab Sys"},{"key":"2943_CR34","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1613\/jair.63","volume":"2","author":"SK Murthy","year":"1994","unstructured":"Murthy SK, Kasif S, Salzberg S: A System for Induction of Oblique Decision Trees. J Artif Intell Res 1994, 2: 1\u201332.","journal-title":"J Artif Intell Res"},{"key":"2943_CR35","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1111\/1467-9469.00134","volume":"26","author":"A Bjorkstrom","year":"1999","unstructured":"Bjorkstrom A: A generalized view on continuum regression. Scand J Stat 1999, 26: 17\u201330. 10.1111\/1467-9469.00134","journal-title":"Scand J Stat"},{"key":"2943_CR36","doi-asserted-by":"publisher","first-page":"897","DOI":"10.1039\/b408950m","volume":"129","author":"TC Martin","year":"2004","unstructured":"Martin TC, Moecks J, Belooussov A, Cawthraw S, Dolenko B, Eiden M, Von Frese J, Kohler W, Schmitt J, Somorjai RL, Udelhoven T, Verzakov S, Petrich W: Classification of signatures of bovine spongiform encephalopathy in serum using infrared spectroscopy. Analyst 2004, 129: 897\u2013901. 10.1039\/b408950m","journal-title":"Analyst"},{"key":"2943_CR37","doi-asserted-by":"publisher","first-page":"599","DOI":"10.1002\/nbm.1041","volume":"19","author":"BH Menze","year":"2006","unstructured":"Menze BH, Lichy MP, Bachert P, Kelm BM, Schlemmer H-P, Hamprecht FA: Optimal classification of long echo time in vivo magnetic resonance spectra in the detection of recurrent brain tumors. NMR in Biomedicine 2006, 19: 599\u201360. 10.1002\/nbm.1041","journal-title":"NMR in Biomedicine"},{"key":"2943_CR38","first-page":"31","volume-title":"Proceedings of BVM","author":"BH Menze","year":"2006","unstructured":"Menze BH, Kelm BM, Heck D, Lichy MP, Hamprecht FA: Machine based rejection of low-quality spectra and estimation of brain tumor probabilities from magnetic resonance spectroscopic images. In Proceedings of BVM. Edited by: Handels H, Ehrhardt J, Horsch A, Meinzer H-P, Tolxdorff T. Springer, New York; 2006:31\u201335."},{"key":"2943_CR39","doi-asserted-by":"publisher","first-page":"4566","DOI":"10.1128\/AEM.69.8.4566-4574.2003","volume":"69","author":"U Himmelreich","year":"2003","unstructured":"Himmelreich U, Somorjai RL, Dolenko B, Lee OC, Daniel HM, Murray R, Mountford CE, Sorrell TC: Rapid identification of candida species by using nuclear magnetic resonance spectroscopy and a statistical classification strategy. Appl Environm Microbiol 2003, 69: 4566\u201374. 10.1128\/AEM.69.8.4566-4574.2003","journal-title":"Appl Environm Microbiol"},{"key":"2943_CR40","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18637\/jss.v018.i02","volume":"18","author":"B-H Mevik","year":"2007","unstructured":"Mevik B-H, Wehrens R: The pls Package: Principal Component and Partial Least Squares Regression in R. J Stat Software 2007, 18: 1\u201324.","journal-title":"J Stat Software"},{"key":"2943_CR41","first-page":"18","volume":"2","author":"A Liaw","year":"2002","unstructured":"Liaw A, Wiener M: Classification and Regression by randomForest. R News 2002, 2: 18\u201322.","journal-title":"R News"},{"key":"2943_CR42","doi-asserted-by":"publisher","first-page":"575","DOI":"10.1198\/106186005X59630","volume":"14","author":"T Hothorn","year":"2005","unstructured":"Hothorn T, Leisch F, Zeileis A, Hornik K: The design and analysis of benchmark experiments. J Comp Graph Stat 2005, 14: 575\u2013699.","journal-title":"J Comp Graph Stat"},{"key":"2943_CR43","first-page":"1","volume":"7","author":"J Demsar","year":"2006","unstructured":"Demsar J: Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 2006, 7: 1\u201330.","journal-title":"J Mach Learn Res"}],"container-title":["BMC Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/1471-2105-10-213.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,2,10]],"date-time":"2025-02-10T22:14:02Z","timestamp":1739225642000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/1471-2105-10-213"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2009,7,10]]},"references-count":43,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2009,12]]}},"alternative-id":["2943"],"URL":"https:\/\/doi.org\/10.1186\/1471-2105-10-213","relation":{},"ISSN":["1471-2105"],"issn-type":[{"value":"1471-2105","type":"electronic"}],"subject":[],"published":{"date-parts":[[2009,7,10]]},"assertion":[{"value":"23 February 2009","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 July 2009","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 July 2009","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"213"}}