{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,16]],"date-time":"2026-10-16T16:53:00Z","timestamp":1792169580862,"version":"4.4.0"},"reference-count":42,"publisher":"Institute of Mathematical Statistics","issue":"4","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Ann. Statist."],"published-print":{"date-parts":[[2011,8,1]]},"abstract":"<jats:p>We consider the problem of estimating a sparse linear regression vector \u03b2* under a Gaussian noise model, for the purpose of both prediction and model selection. We assume that prior knowledge is available on the sparsity pattern, namely the set of variables is partitioned into prescribed groups, only few of which are relevant in the estimation process. This group sparsity assumption suggests us to consider the Group Lasso method as a means to estimate \u03b2*. We establish oracle inequalities for the prediction and \u21132 estimation errors of this estimator. These bounds hold under a restricted eigenvalue condition on the design matrix. Under a stronger condition, we derive bounds for the estimation error for mixed (2, p)-norms with 1 \u2264 p \u2264 \u221e. When p=\u221e, this result implies that a thresholded version of the Group Lasso estimator selects the sparsity pattern of \u03b2* with high probability. Next, we prove that the rate of convergence of our upper bounds is optimal in a minimax sense, up to a logarithmic factor, for all estimators over a class of group sparse vectors. Furthermore, we establish lower bounds for the prediction and \u21132 estimation errors of the usual Lasso estimator. Using this result, we demonstrate that the Group Lasso can achieve an improvement in the prediction and estimation errors as compared to the Lasso. An important application of our results is provided by the problem of estimating multiple regression equations simultaneously or multi-task learning. In this case, we obtain refinements of the results in [In Proc. of the 22nd Annual Conference on Learning Theory (COLT) (2009)], which allow us to establish a quantitative advantage of the Group Lasso over the usual Lasso in the multi-task setting. Finally, within the same setting, we show how our results can be extended to more general noise distributions, of which we only require the fourth moment to be finite. To obtain this extension, we establish a new maximal moment inequality, which may be of independent interest.<\/jats:p>","DOI":"10.1214\/11-aos896","type":"journal-article","created":{"date-parts":[[2011,10,25]],"date-time":"2011-10-25T23:10:55Z","timestamp":1319584255000},"page":"2164-2204","source":"Crossref","is-referenced-by-count":217,"title":["Oracle inequalities and optimal inference under group sparsity"],"prefix":"10.1214","volume":"39","author":[{"given":"Karim","family":"Lounici","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Massimiliano","family":"Pontil","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sara","family":"van de Geer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alexandre B.","family":"Tsybakov","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"108","reference":[{"key":"4","doi-asserted-by":"publisher","unstructured":"[4] Bickel, P. J., Ritov, Y. and Tsybakov, A. B. (2009). Simultaneous analysis of Lasso and Dantzig selector. <i>Ann. Statist.<\/i> <b>37<\/b> 1705\u20131732.","DOI":"10.1214\/08-AOS620"},{"key":"9","doi-asserted-by":"publisher","unstructured":"[9] Cavalier, L., Golubev, G. K., Picard, D. and Tsybakov, A. B. (2002). Oracle inequalities for inverse problems. <i>Ann. Statist.<\/i> <b>30<\/b> 843\u2013874.","DOI":"10.1214\/aos\/1028674843"},{"key":"2","doi-asserted-by":"crossref","unstructured":"[2] Argyriou, A., Evgeniou, T. and Pontil, M. (2008). Convex multi-task feature learning. <i>Machine Learning<\/i> <b>73<\/b> 243\u2013272.","DOI":"10.1007\/s10994-007-5040-8"},{"key":"3","unstructured":"[3] Bach, F. R. (2008). Consistency of the group lasso and multiple kernel learning. <i>J. Mach. Learn. Res.<\/i> <b>9<\/b> 1179\u20131225."},{"key":"5","doi-asserted-by":"crossref","unstructured":"[5] Borwein, J. M. and Lewis, A. S. (2006). <i>Convex Analysis and Nonlinear Optimization: Theory and Examples<\/i>, 2nd ed. <i>CMS Books in Mathematics\/Ouvrages de Math\u00e9matiques de la SMC<\/i> <b>3<\/b>. Springer, New York.","DOI":"10.1007\/978-0-387-31256-9"},{"key":"6","doi-asserted-by":"publisher","unstructured":"[6] Bunea, F., Tsybakov, A. and Wegkamp, M. (2007). Sparsity oracle inequalities for the Lasso. <i>Electron. J. Stat.<\/i> <b>1<\/b> 169\u2013194 (electronic).","DOI":"10.1214\/07-EJS008"},{"key":"7","doi-asserted-by":"publisher","unstructured":"[7] Bunea, F., Tsybakov, A. B. and Wegkamp, M. H. (2007). Aggregation for Gaussian regression. <i>Ann. Statist.<\/i> <b>35<\/b> 1674\u20131697.","DOI":"10.1214\/009053606000001587"},{"key":"8","doi-asserted-by":"publisher","unstructured":"[8] Candes, E. and Tao, T. (2007). The Dantzig selector: Statistical estimation when <i>p<\/i> is much larger than <i>n. Ann. Statist.<\/i> <b>35<\/b> 2313\u20132351.","DOI":"10.1214\/009053606000001523"},{"key":"10","doi-asserted-by":"publisher","unstructured":"[10] Chesneau, C. and Hebiri, M. (2008). Some theoretical results on the grouped variables Lasso. <i>Math. Methods Statist.<\/i> <b>17<\/b> 317\u2013326.","DOI":"10.3103\/S1066530708040030"},{"key":"11","doi-asserted-by":"crossref","unstructured":"[11] Diggle, P. J., Heagerty, P. J., Liang, K.-Y. and Zeger, S. L. (2002). <i>Analysis of Longitudinal Data<\/i>, 2nd ed. <i>Oxford Statistical Science Series<\/i> <b>25<\/b>. Oxford Univ. Press, Oxford.","DOI":"10.1093\/oso\/9780198524847.001.0001"},{"key":"12","doi-asserted-by":"publisher","unstructured":"[12] Donoho, D. L., Elad, M. and Temlyakov, V. N. (2006). Stable recovery of sparse overcomplete representations in the presence of noise. <i>IEEE Trans. Inform. Theory<\/i> <b>52<\/b> 6\u201318.","DOI":"10.1109\/TIT.2005.860430"},{"key":"13","doi-asserted-by":"publisher","unstructured":"[13] D\u00fcmbgen, L., van de Geer, S. A., Veraar, M. C. and Wellner, J. A. (2010). Nemirovski\u2019s inequalities revisited. <i>Amer. Math. Monthly<\/i> <b>117<\/b> 138\u2013160.","DOI":"10.4169\/000298910X476059"},{"key":"14","doi-asserted-by":"crossref","unstructured":"[14] Evgeniou, T., Pontil, M. and Toubia, O. (2007). A convex optimization approach to modeling consumer heterogeneity in conjoint estimation. <i>Marketing Science<\/i> <b>26<\/b> 805\u2013818.","DOI":"10.1287\/mksc.1070.0291"},{"key":"15","unstructured":"[15] Hsiao, C. (2003). <i>Analysis of Panel Data<\/i>, 2nd ed. <i>Econometric Society Monographs<\/i> <b>34<\/b>. Cambridge Univ. Press, Cambridge."},{"key":"16","doi-asserted-by":"publisher","unstructured":"[16] Huang, J., Horowitz, J. L. and Wei, F. (2010). Variable selection in nonparametric additive models. <i>Ann. Statist.<\/i> <b>38<\/b> 2282\u20132313.","DOI":"10.1214\/09-AOS781"},{"key":"17","doi-asserted-by":"publisher","unstructured":"[17] Huang, J. and Zhang, T. (2010). The benefit of group sparsity. <i>Ann. Statist.<\/i> <b>38<\/b> 1978\u20132004.","DOI":"10.1214\/09-AOS778"},{"key":"18","doi-asserted-by":"crossref","unstructured":"[18] Koltchinskii, V. (2011). <i>Oracle Inequalities in Empirical Risk Minimization and Sparse Recovery Problems. Lecture Notes in Math.<\/i> <b>2033<\/b>. Springer, Berlin.","DOI":"10.1007\/978-3-642-22147-7"},{"key":"20","doi-asserted-by":"crossref","unstructured":"[20] Lenk, P. J., DeSarbo, W. S., Green, P. E. and Young, M. R. (1996). Hierarchical Bayes conjoint analysis: Recovery of partworth heterogeneity from reduced experimental designs. <i>Marketing Science<\/i> <b>15<\/b> 173\u2013191.","DOI":"10.1287\/mksc.15.2.173"},{"key":"21","doi-asserted-by":"publisher","unstructured":"[21] Lounici, K. (2008). Sup-norm convergence rate and sign concentration property of Lasso and Dantzig estimators. <i>Electron. J. Stat.<\/i> <b>2<\/b> 90\u2013102.","DOI":"10.1214\/08-EJS177"},{"key":"23","unstructured":"[23] Maurer, A. (2006). Bounds for linear multi-task learning. <i>J. Mach. Learn. Res.<\/i> <b>7<\/b> 117\u2013139."},{"key":"24","doi-asserted-by":"publisher","unstructured":"[24] Meier, L., van de Geer, S. and B\u00fchlmann, P. (2008). The group Lasso for logistic regression. <i>J. R. Stat. Soc. Ser. B Stat. Methodol.<\/i> <b>70<\/b> 53\u201371.","DOI":"10.1111\/j.1467-9868.2007.00627.x"},{"key":"25","doi-asserted-by":"publisher","unstructured":"[25] Meier, L., van de Geer, S. and B\u00fchlmann, P. (2009). High-dimensional additive modeling. <i>Ann. Statist.<\/i> <b>37<\/b> 3779\u20133821.","DOI":"10.1214\/09-AOS692"},{"key":"26","doi-asserted-by":"publisher","unstructured":"[26] Nardi, Y. and Rinaldo, A. (2008). On the asymptotic properties of the group lasso estimator for linear models. <i>Electron. J. Stat.<\/i> <b>2<\/b> 605\u2013633.","DOI":"10.1214\/08-EJS200"},{"key":"27","unstructured":"[27] Nemirovski, A. (2000). Topics in non-parametric statistics. In <i>Lectures on Probability Theory and Statistics (Saint-Flour, 1998). Lecture Notes in Math.<\/i> <b>1738<\/b> 85\u2013277. Springer, Berlin."},{"key":"28","doi-asserted-by":"publisher","unstructured":"[28] Obozinski, G., Wainwright, M. J. and Jordan, M. I. (2011). Support union recovery in high-dimensional multivariate regression. <i>Ann. Statist.<\/i> <b>39<\/b> 1\u201347.","DOI":"10.1214\/09-AOS776"},{"key":"29","doi-asserted-by":"crossref","unstructured":"[29] Petrov, V. V. (1995). <i>Limit Theorems of Probability Theory: Sequences of Independent Random Variables. Oxford Studies in Probability<\/i> <b>4<\/b>. The Clarendon Press, New York.","DOI":"10.1093\/oso\/9780198534990.001.0001"},{"key":"30","doi-asserted-by":"publisher","unstructured":"[30] Raskutti, G., Wainwright, M. J. and Yu, B. (2009). Minimax rates of estimation for high-dimensional linear regression over <i>\u2113<\/i><sub><i>q<\/i><\/sub>-balls. Available at <a href=\"arXiv:0910.2042\">arXiv:0910.2042<\/a>.","DOI":"10.1109\/TIT.2009.2016018"},{"key":"32","doi-asserted-by":"publisher","unstructured":"[32] Rigollet, P. and Tsybakov, A. (2010). Exponential Screening and optimal rates of sparse estimation. Available at <a href=\"arXiv:1003.2654\">arXiv:1003.2654<\/a>.","DOI":"10.1214\/10-AOS854"},{"key":"33","doi-asserted-by":"publisher","unstructured":"[33] Rio, E. (2009). Moment inequalities for sums of dependent random variables under projective conditions. <i>J. Theoret. Probab.<\/i> <b>22<\/b> 146\u2013163.","DOI":"10.1007\/s10959-008-0155-9"},{"key":"34","unstructured":"[34] Srivastava, V. K. and Giles, D. E. A. (1987). <i>Seemingly Unrelated Regression Equations Models: Estimation and Inference. Statistics: Textbooks and Monographs<\/i> <b>80<\/b>. Dekker, New York."},{"key":"36","doi-asserted-by":"publisher","unstructured":"[36] van de Geer, S. A. (2008). High-dimensional generalized linear models and the lasso. <i>Ann. Statist.<\/i> <b>36<\/b> 614\u2013645.","DOI":"10.1214\/009053607000000929"},{"key":"39","doi-asserted-by":"publisher","unstructured":"[39] Yuan, M. and Lin, Y. (2006). Model selection and estimation in regression with grouped variables. <i>J. R. Stat. Soc. Ser. B Stat. Methodol.<\/i> <b>68<\/b> 49\u201367.","DOI":"10.1111\/j.1467-9868.2005.00532.x"},{"key":"40","doi-asserted-by":"publisher","unstructured":"[40] Zellner, A. (1962). An efficient method of estimating seemingly unrelated regressions and tests for aggregation bias. <i>J. Amer. Statist. Assoc.<\/i> <b>57<\/b> 348\u2013368.","DOI":"10.2307\/2281644"},{"key":"41","doi-asserted-by":"publisher","unstructured":"[41] Zhang, C.-H. and Huang, J. (2008). The sparsity and bias of the LASSO selection in high-dimensional linear regression. <i>Ann. Statist.<\/i> <b>36<\/b> 1567\u20131594.","DOI":"10.1214\/07-AOS520"},{"key":"42","doi-asserted-by":"publisher","unstructured":"[42] Zhao, P., Rocha, G. and Yu, B. (2009). The composite absolute penalties family for grouped and hierarchical variable selection. <i>Ann. Statist.<\/i> <b>37<\/b> 3468\u20133497.","DOI":"10.1214\/07-AOS584"},{"key":"1","unstructured":"[1] Aaker, D. A., Day, G. S. and Kumar, V. (1995). <i>Marketing Research<\/i>. Wiley."},{"key":"19","unstructured":"[19] Koltchinskii, V. and Yuan, M. (2008). Sparse recovery in large ensembles of kernel machines. In 21<i>st Annual Conference on Learning Theory\u2014COLT<\/i> 2008, <i>Helsinki<\/i>, <i>Finland<\/i>, <i>July<\/i> 9-12, 2008 (R. A. Servedio and T. Zhang, eds.) 229\u2013238. Omnipress."},{"key":"22","unstructured":"[22] Lounici, K., Pontil, M., B Tsybakov, A. and van de Geer, S. A. (2009). Taking advantage of sparsity in multi-task learning. In <i>Proc. of the<\/i> 22<i>nd Annual Conference on Learning Theory<\/i> (<i>COLT<\/i> 2009) 73\u201382. Omnipress."},{"key":"31","doi-asserted-by":"publisher","unstructured":"[31] Ravikumar, P., Liu, H., Lafferty, J. and Wasserman, L. (2008). Spam: Sparse additive models. In <i>Advances in Neural Information Processing Systems (NIPS)<\/i> (J. C. Platt, D. Koller, Y. Singer and S. Roweis, eds.) <b>22<\/b> 1201\u20131208. MIT Press, Cambridge, MA.","DOI":"10.1111\/j.1467-9868.2009.00718.x"},{"key":"35","doi-asserted-by":"crossref","unstructured":"[35] Tsybakov, A. B. (2009). <i>Introduction to Nonparametric Estimation<\/i>. Springer, New York. Revised and extended from the 2004 French original, Translated by Vladimir Zaiats.","DOI":"10.1007\/b13794"},{"key":"37","doi-asserted-by":"crossref","unstructured":"[37] van der Vaart, A. W. and Wellner, J. A. (1996). <i>Weak Convergence and Empirical Processes<\/i>. Springer, New York.","DOI":"10.1007\/978-1-4757-2545-2"},{"key":"38","unstructured":"[38] Wooldridge, J. M. (2002). <i>Econometric Analysis of Cross Section and Panel Data<\/i>. MIT Press, Cambridge, MA."}],"container-title":["The Annals of Statistics"],"original-title":[],"link":[{"URL":"https:\/\/projecteuclid.org\/download\/pdfview_1\/euclid.aos\/1319595462","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T21:01:46Z","timestamp":1791579706000},"score":1,"resource":{"primary":{"URL":"https:\/\/projecteuclid.org\/journals\/annals-of-statistics\/volume-39\/issue-4\/Oracle-inequalities-and-optimal-inference-under-group-sparsity\/10.1214\/11-AOS896.full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2011,8,1]]},"references-count":42,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2011,8,1]]}},"URL":"https:\/\/doi.org\/10.1214\/11-aos896","relation":{},"ISSN":["0090-5364","2168-8966"],"issn-type":[{"value":"0090-5364","type":"print"},{"value":"2168-8966","type":"electronic"}],"subject":[],"published":{"date-parts":[[2011,8,1]]}}}