{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,14]],"date-time":"2025-10-14T00:37:18Z","timestamp":1760402238264,"version":"build-2065373602"},"reference-count":37,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2020,1,28]],"date-time":"2020-01-28T00:00:00Z","timestamp":1580169600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Polish National Science Center","award":["2015\/17\/B\/ST6\/01878"],"award-info":[{"award-number":["2015\/17\/B\/ST6\/01878"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>We consider selection of random predictors for a high-dimensional regression problem with a binary response for a general loss function. An important special case is when the binary model is semi-parametric and the response function is misspecified under a parametric model fit. When the true response coincides with a postulated parametric response for a certain value of parameter, we obtain a common framework for parametric inference. Both cases of correct specification and misspecification are covered in this contribution. Variable selection for such a scenario aims at recovering the support of the minimizer of the associated risk with large probability. We propose a two-step selection Screening-Selection (SS) procedure which consists of screening and ordering predictors by Lasso method and then selecting the subset of predictors which minimizes the Generalized Information Criterion for the corresponding nested family of models. We prove consistency of the proposed selection method under conditions that allow for a much larger number of predictors than the number of observations. For the semi-parametric case when distribution of random predictors satisfies linear regressions condition, the true and the estimated parameters are collinear and their common support can be consistently identified. This partly explains robustness of selection procedures to the response function misspecification.<\/jats:p>","DOI":"10.3390\/e22020153","type":"journal-article","created":{"date-parts":[[2020,1,28]],"date-time":"2020-01-28T09:37:09Z","timestamp":1580204229000},"page":"153","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Selection Consistency of Lasso-Based Procedures for Misspecified High-Dimensional Binary Model and Random Regressors"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1453-5589","authenticated-orcid":false,"given":"Mariusz","family":"Kubkowski","sequence":"first","affiliation":[{"name":"Institute of Computer Science, Polish Academy of Sciences, Jana Kazimierza 5, 01-248 Warsaw, Poland"},{"name":"Faculty of Mathematics and Information Science, Warsaw University of Technology, Koszykowa 75, 00-662 Warsaw, Poland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2621-2303","authenticated-orcid":false,"given":"Jan","family":"Mielniczuk","sequence":"additional","affiliation":[{"name":"Institute of Computer Science, Polish Academy of Sciences, Jana Kazimierza 5, 01-248 Warsaw, Poland"},{"name":"Faculty of Mathematics and Information Science, Warsaw University of Technology, Koszykowa 75, 00-662 Warsaw, Poland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,1,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Cover, T., and Thomas, J. (2006). Elements of Information Theory, Wiley.","DOI":"10.1002\/047174882X"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"B\u00fchlmann, P., and van de Geer, S. (2011). Statistics for High-dimensional Data, Springer.","DOI":"10.1007\/978-3-642-20192-9"},{"key":"ref_3","unstructured":"van de Geer, S. (2009). Estimation and Testing Under Sparsity, Springer. Lecture Notes in Mathematics."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Hastie, T., Tibshirani, R., and Wainwright, M. (2015). Statistical Learning with Sparsity, Springer.","DOI":"10.1201\/b18401"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1009","DOI":"10.1214\/aos\/1176347254","article-title":"Regression analysis under link violation","volume":"17","author":"Li","year":"1989","journal-title":"Ann. Stat."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1023","DOI":"10.1080\/02331888.2017.1290096","article-title":"Active set of predictors for misspecified logistic regression","volume":"51","author":"Kubkowski","year":"2017","journal-title":"Statistics"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"152","DOI":"10.1016\/j.laa.2017.09.013","article-title":"Projections of a general binary model on logistic regrssion","volume":"536","author":"Kubkowski","year":"2018","journal-title":"Linear Algebra Appl."},{"key":"ref_8","unstructured":"Kubkowski, M. (2019). Misspecification of Binary Regression Model: Properties and Inferential Procedures. [Ph.D. Thesis, Warsaw University of Technology]."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"717","DOI":"10.1093\/biomet\/ass027","article-title":"On the robustness of the adaptive lasso to model misspecification","volume":"99","author":"Lu","year":"2012","journal-title":"Biometrika"},{"key":"ref_10","unstructured":"Brillinger, D. (1982). A Generalized linear model with \u2018gaussian\u2019 regressor variables. A Festschfrift for Erich Lehmann, Wadsworth International Group."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"225","DOI":"10.2307\/1912257","article-title":"Suffcient conditions for the consistency of maximum likelihood estimation despite misspecifcation of distribution in multinomial discrete choice models","volume":"51","author":"Ruud","year":"1983","journal-title":"Econometrica"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"547","DOI":"10.1080\/10618600.2016.1256816","article-title":"Semismooth Newton coordinate descent algorithm for elastic-net penalized Huber loss regression and quantile regression","volume":"26","author":"Yi","year":"2017","journal-title":"J. Comput. Graph. Stat."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1","DOI":"10.2307\/1912526","article-title":"Maximum likelihood estimation of misspecified models","volume":"50","author":"White","year":"1982","journal-title":"Econometrica"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"307","DOI":"10.2307\/1912557","article-title":"Likelihood ratio testts for model selection and not-nested hypotheses","volume":"57","author":"Vuong","year":"1989","journal-title":"Econometrica"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1705","DOI":"10.1214\/08-AOS620","article-title":"Simultaneous analysis of Lasso and Dantzig selector","volume":"37","author":"Bickel","year":"2009","journal-title":"Ann. Stat."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"538","DOI":"10.1214\/12-STS400","article-title":"A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers","volume":"27","author":"Negahban","year":"2012","journal-title":"Stat. Sci."},{"key":"ref_17","first-page":"961","article-title":"Combined \u21131 and greedy \u21130 penalized least squares for linear model selection","volume":"16","author":"Pokarowski","year":"2015","journal-title":"J. Mach. Learn. Res."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"867","DOI":"10.1214\/aos\/1176349155","article-title":"On almost Linearity of Low Dimensional Projections from High Dimensional Data","volume":"21","author":"Hall","year":"1993","journal-title":"Ann. Stat."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"759","DOI":"10.1093\/biomet\/asn034","article-title":"Extended bayesian information criterion for model selection with large model spaces","volume":"95","author":"Chen","year":"2008","journal-title":"Biometrika"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"555","DOI":"10.5705\/ss.2010.216","article-title":"Extended BIC for small-n-large-p sparse GLM","volume":"22","author":"Chen","year":"2012","journal-title":"Stat. Sin."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1007\/978-3-319-13881-7_13","article-title":"Selection consistency of Generalized Information Criterion for sparse logistic model","volume":"Volume 122","author":"Steland","year":"2015","journal-title":"Stochastic Models, Statistics and Their Applications"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Vershynin, R. (2012). Introduction to the non-asymptotic analysis of random matrices. Compressed Sensing, Theory and Applications, Cambridge University Press.","DOI":"10.1017\/CBO9780511794308.006"},{"key":"ref_23","unstructured":"Fan, J., Xue, L., and Zou, H. (2020, January 25). Supplement to \u201cStrong Oracle Optimality of Folded Concave Penalized Estimation\u201d. Available online: NIHMS649192-supplement-suppl.pdf."},{"key":"ref_24","first-page":"819","article-title":"Strong Oracle Optimality of folded concave penalized estimation","volume":"43","author":"Fan","year":"2014","journal-title":"Ann. Stat."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"384","DOI":"10.1214\/09-EJS521","article-title":"Self-concordant analysis for logistic regression","volume":"4","author":"Bach","year":"2010","journal-title":"Electron. J. Stat."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1007\/BF02506337","article-title":"Statistical predictor identification","volume":"22","author":"Akaike","year":"1970","journal-title":"Ann. Inst. Stat. Math."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"461","DOI":"10.1214\/aos\/1176344136","article-title":"Estimating the Dimension of a Model","volume":"6","author":"Schwarz","year":"1978","journal-title":"Ann. Stat."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"2467","DOI":"10.1214\/15-AOS1413","article-title":"Consistent model selection criteria for quadratically supported risks","volume":"44","author":"Kim","year":"2016","journal-title":"Ann. Stat."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"van der Vaart, A.W., and Wellner, J.A. (1996). Weak Convergence and Empirical Processes with Applications to Statistics, Springer.","DOI":"10.1007\/978-1-4757-2545-2"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Ledoux, M., and Talagrand, M. (1991). Probability in Banach Spaces: Isoperimetry and Processes, Springe.","DOI":"10.1007\/978-3-642-20212-4"},{"key":"ref_31","first-page":"1603","article-title":"Adaptive Lasso for sparse high-dimensional regression models","volume":"18","author":"Huang","year":"2008","journal-title":"Stat. Sin."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1456","DOI":"10.1214\/13-EJS815","article-title":"The lasso problem and uniqueness","volume":"7","author":"Tibshirani","year":"2013","journal-title":"Electron. J. Stat."},{"key":"ref_33","unstructured":"Zhou, S. (2010). Thresholded Lasso for high dimensional variable selection and statistical estimation. arXiv."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v033.i01","article-title":"Regularization paths for generalized linear models via coordinate descent","volume":"33","author":"Friedman","year":"2010","journal-title":"J. Stat. Softw."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"531","DOI":"10.1111\/rssb.12001","article-title":"Tuning parameter selection in high dimensional penalized likelihood","volume":"75","author":"Fan","year":"2013","journal-title":"J. R. Stat. Soc. Ser. B"},{"key":"ref_36","first-page":"941","article-title":"Boosting as a regularized path to a maximum margin classifier","volume":"5","author":"Rosset","year":"2004","journal-title":"J. Mach. Learn. Res."},{"key":"ref_37","unstructured":"Devroye, L., and Lugosi, G. (2012). Combinatorial Methods in Density Estimation, Springer Science & Business Media."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/2\/153\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T13:30:33Z","timestamp":1760362233000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/2\/153"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,1,28]]},"references-count":37,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2020,2]]}},"alternative-id":["e22020153"],"URL":"https:\/\/doi.org\/10.3390\/e22020153","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2020,1,28]]}}}