{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,11]],"date-time":"2026-04-11T20:26:31Z","timestamp":1775939191819,"version":"3.50.1"},"reference-count":47,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2022,11,3]],"date-time":"2022-11-03T00:00:00Z","timestamp":1667433600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,11,3]],"date-time":"2022-11-03T00:00:00Z","timestamp":1667433600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Ministry For Innovation and Technology Hungary","award":["\u00daNKP-20-3-II-CORVINUS-68"],"award-info":[{"award-number":["\u00daNKP-20-3-II-CORVINUS-68"]}]},{"DOI":"10.13039\/100018253","name":"Corvinus University of Budapest","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100018253","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Comput Stat"],"published-print":{"date-parts":[[2024,4]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In this paper, the properties of 10 different feature selection algorithms for generalized additive models (GAMs) are compared on one simulated and two real-world datasets under concurvity. Concurvity can be interpreted as a redundancy in the feature set of a GAM. Like multicollinearity in linear models, concurvity causes unstable parameter estimates in GAMs and makes the marginal effect of features harder interpret. Feature selection algorithms for GAMs can be separated into four clusters: stepwise, boosting, regularization and concurvity controlled methods. Our numerical results show that algorithms with no constraints on concurvity tend to select a large feature set, without significant improvements in predictive performance compared to a more parsimonious feature set. A large feature set is accompanied by harmful concurvity in the proposed models. To tackle the concurvity phenomenon, recent feature selection algorithms such as the mRMR and the HSIC-Lasso incorporated some constraints on concurvity in their objective function. However, these algorithms interpret concurvity as pairwise non-linear relationship between features, so they do not account for the case when a feature can be accurately estimated as a multivariate function of several other features. This is confirmed by our numerical results. Our own solution to the problem, a hybrid genetic\u2013harmony search algorithm (HA) introduces constrains on multivariate concurvity directly. Due to this constraint, the HA proposes a small and not redundant feature set with predictive performance similar to that of models with far more features.<\/jats:p>","DOI":"10.1007\/s00180-022-01292-7","type":"journal-article","created":{"date-parts":[[2022,11,3]],"date-time":"2022-11-03T15:04:36Z","timestamp":1667487876000},"page":"461-493","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":20,"title":["Feature selection algorithms in generalized additive models under concurvity"],"prefix":"10.1007","volume":"39","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9032-402X","authenticated-orcid":false,"given":"L\u00e1szl\u00f3","family":"Kov\u00e1cs","sequence":"first","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,11,3]]},"reference":[{"issue":"4","key":"1292_CR1","doi-asserted-by":"publisher","first-page":"281","DOI":"10.1038\/nmeth.3812","volume":"13","author":"N Altman","year":"2016","unstructured":"Altman N, Krzywinski M (2016) Analyzing outliers: Influential or nuisance? Nat Methods 13(4):281\u2013283","journal-title":"Nat Methods"},{"issue":"1","key":"1292_CR2","first-page":"85","volume":"74","author":"S Amodio","year":"2014","unstructured":"Amodio S, Aria M, D\u2019Ambrosio A (2014) On concurvity in nonlinear and nonparametric regression models. Statistica 74(1):85\u201398","journal-title":"Statistica"},{"issue":"8","key":"1292_CR3","doi-asserted-by":"publisher","first-page":"2404","DOI":"10.1016\/j.csda.2012.01.026","volume":"56","author":"NH Augustin","year":"2012","unstructured":"Augustin NH, Sauleau EA, Wood SN (2012) On quantile quantile plots for generalized linear models. Comput Stat Data Anal 56(8):2404\u20132409. https:\/\/doi.org\/10.1016\/j.csda.2012.01.026","journal-title":"Comput Stat Data Anal"},{"issue":"1","key":"1292_CR4","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1016\/j.csda.2008.05.032","volume":"53","author":"C Belitz","year":"2008","unstructured":"Belitz C, Lang S (2008) Simultaneous selection of variables and smoothing parameters in structured additive regression models. Comput Stat Data Anal 53(1):61\u201381. https:\/\/doi.org\/10.1016\/j.csda.2008.05.032","journal-title":"Comput Stat Data Anal"},{"issue":"1","key":"1292_CR5","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1007\/s11222-007-9040-0","volume":"18","author":"H Binder","year":"2008","unstructured":"Binder H, Tutz G (2008) A comparison of methods for the fitting of generalized additive models. Stat Comput 18(1):87\u201399. https:\/\/doi.org\/10.1007\/s11222-007-9040-0","journal-title":"Stat Comput"},{"issue":"1","key":"1292_CR6","doi-asserted-by":"publisher","first-page":"232","DOI":"10.1214\/10-AOAS388","volume":"5","author":"P Breheny","year":"2011","unstructured":"Breheny P, Huang J (2011) Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection. Ann Appl Stat 5(1):232\u2013253","journal-title":"Ann Appl Stat"},{"issue":"3","key":"1292_CR7","doi-asserted-by":"publisher","first-page":"237","DOI":"10.1177\/1471082X1001100304","volume":"11","author":"E Cantoni","year":"2011","unstructured":"Cantoni E, Flemming JM, Ronchetti E (2011) Variable selection in additive models by non-negative garrote. Stat Model 11(3):237\u2013252. https:\/\/doi.org\/10.1177\/1471082X1001100304","journal-title":"Stat Model"},{"issue":"1\u20132","key":"1292_CR8","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1016\/j.chemolab.2004.12.011","volume":"78","author":"IG Chong","year":"2005","unstructured":"Chong IG, Jun CH (2005) Performance of some variable selection methods when multicollinearity is present. Chemom Intell Lab Syst 78(1\u20132):103\u2013112. https:\/\/doi.org\/10.1016\/j.chemolab.2004.12.011","journal-title":"Chemom Intell Lab Syst"},{"issue":"14","key":"1292_CR9","doi-asserted-by":"publisher","first-page":"i427","DOI":"10.1093\/bioinformatics\/btz333","volume":"35","author":"H Climente-Gonz\u00e1lez","year":"2019","unstructured":"Climente-Gonz\u00e1lez H, Azencott CA, Kaski S, Yamada M (2019) Block HSIC Lasso: model-free biomarker detection for ultra-high dimensional data. Bioinformatics 35(14):i427\u2013i435. https:\/\/doi.org\/10.1093\/bioinformatics\/btz333","journal-title":"Bioinformatics"},{"issue":"18","key":"1292_CR10","doi-asserted-by":"publisher","first-page":"2365","DOI":"10.1093\/bioinformatics\/btt383","volume":"29","author":"N De Jay","year":"2013","unstructured":"De Jay N, Papillon-Cavanagh S, Olsen C, El-Hachem N, Bontempi G, Haibe-Kains B (2013) mRMRe: an R package for parallelized mRMR ensemble feature selection. Bioinformatics 29(18):2365\u20132368. https:\/\/doi.org\/10.1093\/bioinformatics\/btt383","journal-title":"Bioinformatics"},{"issue":"1","key":"1292_CR11","doi-asserted-by":"publisher","first-page":"68","DOI":"10.1145\/3359786","volume":"63","author":"M Du","year":"2019","unstructured":"Du M, Liu N, Hu X (2019) Techniques for interpretable machine learning. Commun ACM 63(1):68\u201377. https:\/\/doi.org\/10.1145\/3359786","journal-title":"Commun ACM"},{"key":"1292_CR12","unstructured":"Efroymson MA (1960) Multiple regression analysis. In: Ralston A, Wilf HS (eds) Mathematical methods for digital computers. John Wiley, New York, pp 191\u2013203"},{"key":"1292_CR13","doi-asserted-by":"crossref","unstructured":"Gretton A, Bousquet O, Smola A, Sch\u00f6lkopf B (2005) Measuring statistical dependence with Hilbert-Schmidt norms. In: International conference on algorithmic learning theory. Springer, Berlin, pp 63\u201377","DOI":"10.1007\/11564089_7"},{"issue":"3","key":"1292_CR14","doi-asserted-by":"publisher","first-page":"531","DOI":"10.1198\/jcgs.2010.07139","volume":"19","author":"H Gu","year":"2010","unstructured":"Gu H, Kenney T, Zhu M (2010) Partial generalized additive models: an information-theoretic approach for dealing with concurvity and selecting variables. J Comput Graph Stat 19(3):531\u2013551. https:\/\/doi.org\/10.1198\/jcgs.2010.07139","journal-title":"J Comput Graph Stat"},{"key":"1292_CR15","unstructured":"Hall MA (1999) Correlation-based feature selection for machine learning. Dissertation, University of Waikato."},{"issue":"1","key":"1292_CR16","doi-asserted-by":"publisher","first-page":"100","DOI":"10.2307\/2346830","volume":"28","author":"JA Hartigan","year":"1979","unstructured":"Hartigan JA, Wong MA (1979) Algorithm AS 136: A k-means clustering algorithm. J R Stat Soc Ser C Appl Stat 28(1):100\u2013108. https:\/\/doi.org\/10.2307\/2346830","journal-title":"J R Stat Soc Ser C Appl Stat"},{"key":"1292_CR17","volume-title":"Generalized additive models","author":"TJ Hastie","year":"1990","unstructured":"Hastie TJ, Tibshirani RJ (1990) Generalized additive models. Chapman and Hall, London"},{"key":"1292_CR18","unstructured":"Hastie TJ (2018) gam: generalized additive models. R package version 1.16. https:\/\/CRAN.R-project.org\/package=gam"},{"key":"1292_CR19","doi-asserted-by":"crossref","unstructured":"Huo X, Ni X (2007) When do stepwise algorithms meet subset selection criteria?. Ann Stat. pp 870\u2013887. https:\/\/www.jstor.org\/stable\/25463581","DOI":"10.1214\/009053606000001334"},{"key":"1292_CR20","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4614-7138-7","volume-title":"An introduction to statistical learning: with applications in R","author":"G James","year":"2013","unstructured":"James G, Witten D, Hastie TJ, Tibshirani R (2013) An introduction to statistical learning: with applications in R. Springer, New York"},{"key":"1292_CR21","first-page":"595","volume":"20","author":"J Jia","year":"2010","unstructured":"Jia J, Yu B (2010) On model selection consistency of the elastic net. Stat Sin 20:595\u2013611","journal-title":"Stat Sin"},{"key":"1292_CR22","unstructured":"Kuhn M, Wing J, Weston S, Williams A, Keefer C, Engelhardt A, Cooper T, Mayer Z, Kenkel B, the R Core Team, Benesty M, Lescarbeau R, Ziem A, Scrucca L, Tang Y, Candan C, Tyler H (2019) caret: Classification and Regression Training. R package version 6.0\u201384. https:\/\/CRAN.R-project.org\/package=caret"},{"issue":"1","key":"1292_CR23","doi-asserted-by":"publisher","first-page":"e02567","DOI":"10.1002\/ecs2.2567","volume":"10","author":"J Lai","year":"2019","unstructured":"Lai J, Lortie CJ, Muenchen RA, Yang J, Ma K (2019) Evaluating the popularity of R in ecology. Ecosphere 10(1):e02567. https:\/\/doi.org\/10.1002\/ecs2.2567","journal-title":"Ecosphere"},{"issue":"1","key":"1292_CR24","first-page":"2","volume":"11","author":"B L\u00e1ng","year":"2017","unstructured":"L\u00e1ng B, Kov\u00e1cs L, Moh\u00e1csi L (2017) Linear regression model selection using a hybrid genetic \u2013 Improved harmony search parallelized algorithm. SEFBIS J 11(1):2\u20139","journal-title":"SEFBIS J"},{"issue":"5","key":"1292_CR25","doi-asserted-by":"publisher","first-page":"2272","DOI":"10.1214\/009053606000000722","volume":"34","author":"Y Lin","year":"2006","unstructured":"Lin Y, Zhang HH (2006) Component selection and smoothing in multivariate nonparametric regression. Ann Stat 34(5):2272\u20132297. https:\/\/doi.org\/10.1214\/009053606000000722","journal-title":"Ann Stat"},{"issue":"3a","key":"1292_CR26","doi-asserted-by":"publisher","first-page":"158","DOI":"10.1080\/00031305.1982.10482818","volume":"36","author":"ER Mansfield","year":"1982","unstructured":"Mansfield ER, Helms BP (1982) Detecting multicollinearity. Am Stat 36(3a):158\u2013160","journal-title":"Am Stat"},{"issue":"7","key":"1292_CR27","doi-asserted-by":"publisher","first-page":"2372","DOI":"10.1016\/j.csda.2011.02.004","volume":"55","author":"G Marra","year":"2011","unstructured":"Marra G, Wood SN (2011) Practical variable selection for generalized additive models. Comput Stat Data Anal 55(7):2372\u20132387. https:\/\/doi.org\/10.1016\/j.csda.2011.02.004","journal-title":"Comput Stat Data Anal"},{"key":"1292_CR28","first-page":"105","volume-title":"Frontiers in econometrics","author":"D McFadden","year":"1974","unstructured":"McFadden D (1974) Conditional logit analysis of qualitative choice behaviour. In: Zarembka P (ed) Frontiers in econometrics. Academic Press, New York, pp 105\u2013142"},{"key":"1292_CR29","volume-title":"Interpretable machine learning","author":"C Molnar","year":"2020","unstructured":"Molnar C (2020) Interpretable machine learning. Leanpub, Victoria"},{"issue":"1","key":"1292_CR30","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12874-019-0666-3","volume":"19","author":"A Perperoglou","year":"2019","unstructured":"Perperoglou A, Sauerbrei W, Abrahamowicz M, Schmid M (2019) A review of spline function procedures in R. BMC Med Res Methodol 19(1):1\u201316. https:\/\/doi.org\/10.1186\/s12874-019-0666-3","journal-title":"BMC Med Res Methodol"},{"issue":"1","key":"1292_CR31","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1097\/00001648-200301000-00009","volume":"14","author":"TO Ramsay","year":"2003","unstructured":"Ramsay TO, Burnett RT, Krewski D (2003) The effect of concurvity in generalized additive models linking mortality to ambient particulate matter. Epidemiology 14(1):18\u201323","journal-title":"Epidemiology"},{"key":"1292_CR32","doi-asserted-by":"publisher","first-page":"197","DOI":"10.1007\/BF00116037","volume":"5","author":"RE Schapire","year":"1990","unstructured":"Schapire RE (1990) The strength of weak learnability. Mach Learn 5:197\u2013227","journal-title":"Mach Learn"},{"issue":"2","key":"1292_CR33","doi-asserted-by":"publisher","first-page":"298","DOI":"10.1016\/j.csda.2008.09.009","volume":"53","author":"M Schmid","year":"2008","unstructured":"Schmid M, Hothorn T (2008) Boosting additive models using component-wise P-splines. Comput Stat Data Anal 53(2):298\u2013311. https:\/\/doi.org\/10.1016\/j.csda.2008.09.009","journal-title":"Comput Stat Data Anal"},{"key":"1292_CR34","unstructured":"Signoretto M, Pelckmans K, Suykens JA (2008) Functional ANOVA Models: Convex-concave approach and concurvity analysis (No. 08\u2013203). Internal Report."},{"key":"1292_CR35","unstructured":"Therneau T, Atkinson B (2018) rpart: recursive partitioning and regression trees. R package version 4.1\u201313. https:\/\/CRAN.R-project.org\/package=rpart"},{"issue":"1","key":"1292_CR36","doi-asserted-by":"publisher","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","volume":"58","author":"R Tibshirani","year":"1996","unstructured":"Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Stat Soc: Ser B (methodol) 58(1):267\u2013288. https:\/\/doi.org\/10.1111\/j.2517-6161.1996.tb02080.x","journal-title":"J R Stat Soc: Ser B (methodol)"},{"issue":"4","key":"1292_CR37","doi-asserted-by":"publisher","first-page":"961","DOI":"10.1111\/j.1541-0420.2006.00578.x","volume":"62","author":"G Tutz","year":"2006","unstructured":"Tutz G, Binder H (2006) Generalized additive modeling with implicit variable selection by likelihood-based boosting. Biometrics 62(4):961\u2013971. https:\/\/doi.org\/10.1111\/j.1541-0420.2006.00578.x","journal-title":"Biometrics"},{"key":"1292_CR38","unstructured":"Weston S (2019a) foreach: provides foreach looping construct. R package version 1.4.7. https:\/\/CRAN.R-project.org\/package=foreach"},{"key":"1292_CR39","unstructured":"Weston S (2019b) doParallel: Foreach Parallel Adaptor for the 'parallel' Package. R package version 1.0.15. https:\/\/CRAN.R-project.org\/package=doParallel"},{"issue":"1","key":"1292_CR41","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1111\/j.1467-9868.2010.00749.x","volume":"73","author":"SN Wood","year":"2011","unstructured":"Wood SN (2011) Fast stable restricted maximum likelihood and marginal likelihood estimation of semiparametric generalized linear models. J R Stat Soc Ser B Stat Methodol 73(1):3\u201336. https:\/\/doi.org\/10.1111\/j.1467-9868.2010.00749.x","journal-title":"J R Stat Soc Ser B Stat Methodol"},{"key":"1292_CR42","doi-asserted-by":"publisher","DOI":"10.1201\/9781315370279","volume-title":"Generalized additive models: an introduction with R","author":"SN Wood","year":"2017","unstructured":"Wood SN (2017) Generalized additive models: an introduction with R, 2nd edn. Chapman and Hall\/CRC, London","edition":"2"},{"key":"1292_CR43","volume-title":"Introductory econometrics: a modern approach","author":"JM Wooldridge","year":"2016","unstructured":"Wooldridge JM (2016) Introductory econometrics: a modern approach. Nelson Education, Toronto"},{"issue":"05","key":"1292_CR44","doi-asserted-by":"publisher","first-page":"115","DOI":"10.4236\/iim.2018.105010","volume":"10","author":"S Yang","year":"2018","unstructured":"Yang S, Zhang H (2018) Comparison of several data mining methods in credit card default prediction. Intell Inf Manag 10(05):115\u2013122. https:\/\/doi.org\/10.4236\/iim.2018.105010","journal-title":"Intell Inf Manag"},{"issue":"12","key":"1292_CR45","doi-asserted-by":"publisher","first-page":"1797","DOI":"10.1016\/S0008-8846(98)00165-3","volume":"28","author":"IC Yeh","year":"1998","unstructured":"Yeh IC (1998) Modeling of strength of high-performance concrete using artificial neural networks. Cem Concr Res 28(12):1797\u20131808. https:\/\/doi.org\/10.1016\/S0008-8846(98)00165-3","journal-title":"Cem Concr Res"},{"issue":"2","key":"1292_CR46","doi-asserted-by":"publisher","first-page":"2473","DOI":"10.1016\/j.eswa.2007.12.020","volume":"36","author":"IC Yeh","year":"2009","unstructured":"Yeh IC, Lien CH (2009) The comparisons of data mining techniques for the predictive accuracy of probability of default of credit card clients. Expert Syst Appl 36(2):2473\u20132480. https:\/\/doi.org\/10.1016\/j.eswa.2007.12.020","journal-title":"Expert Syst Appl"},{"key":"1292_CR47","unstructured":"Zhang HH, Lin CY (2013) cosso: fit regularized nonparametric regression models using COSSO penalty. R package version 2.1\u20131. https:\/\/CRAN.R-project.org\/package=cosso"},{"key":"1292_CR48","first-page":"2541","volume":"7","author":"P Zhao","year":"2006","unstructured":"Zhao P, Yu B (2006) On model selection consistency of Lasso. J Mach Learn Res 7:2541\u20132563","journal-title":"J Mach Learn Res"}],"container-title":["Computational Statistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00180-022-01292-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00180-022-01292-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00180-022-01292-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,14]],"date-time":"2024-03-14T09:07:10Z","timestamp":1710407230000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00180-022-01292-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,3]]},"references-count":47,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,4]]}},"alternative-id":["1292"],"URL":"https:\/\/doi.org\/10.1007\/s00180-022-01292-7","relation":{},"ISSN":["0943-4062","1613-9658"],"issn-type":[{"value":"0943-4062","type":"print"},{"value":"1613-9658","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,3]]},"assertion":[{"value":"22 October 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 October 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 November 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"We declare that there are no financial interests or personal relationships that could have influenced this work.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}