{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T17:53:13Z","timestamp":1785520393297,"version":"3.56.0"},"reference-count":55,"publisher":"Oxford University Press (OUP)","issue":"2","license":[{"start":{"date-parts":[[2024,9,18]],"date-time":"2024-09-18T00:00:00Z","timestamp":1726617600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"DOI":"10.13039\/100000001","name":"NSF","doi-asserted-by":"publisher","award":["2DMS-1945428"],"award-info":[{"award-number":["2DMS-1945428"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"NSF","doi-asserted-by":"publisher","award":["MTH230020"],"award-info":[{"award-number":["MTH230020"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,4,11]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Generalized cross-validation (GCV) is a widely used method for estimating the squared out-of-sample prediction risk that employs scalar degrees of freedom adjustment (in a multiplicative sense) to the squared training error. In this paper, we examine the consistency of GCV for estimating the prediction risk of arbitrary ensembles of penalized least-squares estimators. We show that GCV is inconsistent for any finite ensemble of size greater than one. Towards repairing this shortcoming, we identify a correction that involves an additional scalar correction (in an additive sense) based on degrees of freedom adjusted training errors from each ensemble component. The proposed estimator (termed CGCV) maintains the computational advantages of GCV and requires neither sample splitting, model refitting, or out-of-bag risk estimation. The estimator stems from a finer inspection of the ensemble risk decomposition and two intermediate risk estimators for the components in this decomposition. We provide a non-asymptotic analysis of the CGCV and the two intermediate risk estimators for ensembles of convex penalized estimators under Gaussian features and a linear response model. Furthermore, in the special case of ridge regression, we extend the analysis to general feature and response distributions using random matrix theory, which establishes model-free uniform consistency of CGCV.<\/jats:p>","DOI":"10.1093\/jrsssb\/qkae092","type":"journal-article","created":{"date-parts":[[2024,9,18]],"date-time":"2024-09-18T08:06:16Z","timestamp":1726646776000},"page":"289-318","source":"Crossref","is-referenced-by-count":3,"title":["Corrected generalized cross-validation for finite ensembles of penalized estimators"],"prefix":"10.1093","volume":"87","author":[{"given":"Pierre C","family":"Bellec","sequence":"first","affiliation":[{"name":"Department of Statistics, Rutgers University , New Brunswick, NJ 08854 ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9683-4146","authenticated-orcid":false,"given":"Jin-Hong","family":"Du","sequence":"additional","affiliation":[{"name":"Department of Statistics and Data Science, Carnegie Mellon University , Pittsburgh, PA 15213 ,","place":["USA"]},{"name":"Machine Learning Department, Carnegie Mellon University , Pittsburgh, PA 15213 ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1152-3264","authenticated-orcid":false,"given":"Takuya","family":"Koriyama","sequence":"additional","affiliation":[{"name":"Booth School of Business, The University of Chicago , Chicago, IL 60637 ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pratik","family":"Patil","sequence":"additional","affiliation":[{"name":"Department of Statistics, University of California , Berkeley, CA 94720 ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kai","family":"Tan","sequence":"additional","affiliation":[{"name":"Department of Statistics, Rutgers University , New Brunswick, NJ 08854 ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2024,9,18]]},"reference":[{"key":"2025091919460057700_qkae092-B1","author":"Adlam","year":"2020"},{"key":"2025091919460057700_qkae092-B2","author":"Adlam","year":"2020"},{"key":"2025091919460057700_qkae092-B3","author":"Auddy","year":"2024"},{"key":"2025091919460057700_qkae092-B4","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4419-0661-8","volume-title":"Spectral analysis of large dimensional random matrices","author":"Bai","year":"2010","edition":"2nd ed."},{"key":"2025091919460057700_qkae092-B5","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1017\/S0962492921000027","article-title":"Deep learning: A statistical viewpoint","volume":"30","author":"Bartlett","year":"2021","journal-title":"Acta Numerica"},{"key":"2025091919460057700_qkae092-B6","article-title":"Estimating LASSO risk and noise level","volume":"26","author":"Bayati","year":"2013","journal-title":"Advances in neural information processing systems"},{"issue":"4","key":"2025091919460057700_qkae092-B7","doi-asserted-by":"publisher","first-page":"1997","DOI":"10.1109\/TIT.2011.2174612","article-title":"The lasso risk for Gaussian matrices","volume":"58","author":"Bayati","year":"2011","journal-title":"IEEE Transactions on Information Theory"},{"issue":"4","key":"2025091919460057700_qkae092-B8","doi-asserted-by":"publisher","first-page":"2782","DOI":"10.1093\/imaiai\/iaad031","article-title":"Out-of-sample error estimation for m-estimators with convex penalty","volume":"12","author":"Bellec","year":"2023","journal-title":"Information and Inference: A Journal of the IMA"},{"key":"2025091919460057700_qkae092-B9","author":"Bellec","year":"2022"},{"issue":"4","key":"2025091919460057700_qkae092-B10","doi-asserted-by":"publisher","first-page":"1864","DOI":"10.1214\/20-AOS2005","article-title":"Second-order stein: Sure for sure and other applications in high-dimensional inference","volume":"49","author":"Bellec","year":"2021","journal-title":"The Annals of Statistics"},{"issue":"2","key":"2025091919460057700_qkae092-B11","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1007\/BF00058655","article-title":"Bagging predictors","volume":"24","year":"1996","journal-title":"Machine Learning"},{"issue":"1","key":"2025091919460057700_qkae092-B12","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Machine Learning"},{"issue":"4","key":"2025091919460057700_qkae092-B13","doi-asserted-by":"publisher","first-page":"927","DOI":"10.1214\/aos\/1031689014","article-title":"Analyzing bagging","volume":"30","author":"B\u00fchlmann","year":"2002","journal-title":"The Annals of Statistics"},{"issue":"5","key":"2025091919460057700_qkae092-B14","doi-asserted-by":"publisher","first-page":"2194","DOI":"10.1214\/23-AOS2327","article-title":"The lasso with general Gaussian designs with applications to hypothesis testing","volume":"51","author":"Celentano","year":"2023","journal-title":"The Annals of Statistics"},{"issue":"4","key":"2025091919460057700_qkae092-B15","doi-asserted-by":"publisher","first-page":"377","DOI":"10.1007\/BF01404567","article-title":"Estimating the correct degree of smoothing by the method of generalized cross-validation","volume":"31","author":"Craven","year":"1979","journal-title":"Numerische Mathematik"},{"key":"2025091919460057700_qkae092-B16","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1023\/A:1007607513941","article-title":"An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting and randomization","volume":"32","author":"Dietterich","year":"1998","journal-title":"Machine Learning"},{"key":"2025091919460057700_qkae092-B17","author":"Dietterich","year":"2000"},{"issue":"1","key":"2025091919460057700_qkae092-B18","doi-asserted-by":"publisher","first-page":"247","DOI":"10.1214\/17-AOS1549","article-title":"High-dimensional asymptotics of prediction: Ridge regression and classification","volume":"46","author":"Dobriban","year":"2018","journal-title":"The Annals of Statistics"},{"key":"2025091919460057700_qkae092-B19","doi-asserted-by":"publisher","first-page":"809","DOI":"10.5705\/ss.2011.281","article-title":"The degrees of freedom of the lasso for general design matrix","author":"Dossal","year":"2013","journal-title":"Statistica Sinica"},{"key":"2025091919460057700_qkae092-B20","author":"Du","year":"2023"},{"issue":"3","key":"2025091919460057700_qkae092-B21","doi-asserted-by":"publisher","first-page":"1061","DOI":"10.1080\/10618600.2023.2288194","article-title":"Extrapolated cross-validation for randomized ensembles","volume":"33","author":"Du","year":"2024","journal-title":"Journal of Computational and Graphical Statistics"},{"issue":"1","key":"2025091919460057700_qkae092-B22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1214\/08-AOS648","article-title":"The spectrum of kernel random matrices","volume":"38","author":"El Karoui","year":"2010","journal-title":"The Annals of Statistics"},{"issue":"1","key":"2025091919460057700_qkae092-B23","first-page":"170","article-title":"Can we trust the bootstrap in high-dimensions? The case of linear models","volume":"19","author":"El Karoui","year":"2018","journal-title":"The Journal of Machine Learning Research"},{"issue":"3","key":"2025091919460057700_qkae092-B24","doi-asserted-by":"publisher","first-page":"669","DOI":"10.1016\/j.jspi.2006.06.002","article-title":"On bagging and nonlinear estimation","volume":"137","author":"Friedman","year":"2007","journal-title":"Journal of Statistical Planning and Inference"},{"issue":"2","key":"2025091919460057700_qkae092-B25","doi-asserted-by":"publisher","first-page":"215","DOI":"10.1080\/00401706.1979.10489751","article-title":"Generalized cross-validation as a method for choosing a good ridge parameter","volume":"21","author":"Golub","year":"1979","journal-title":"Technometrics"},{"issue":"2","key":"2025091919460057700_qkae092-B26","doi-asserted-by":"publisher","first-page":"949","DOI":"10.1214\/21-AOS2133","article-title":"Surprises in high-dimensional ridgeless least squares interpolation","volume":"50","author":"Hastie","year":"2022","journal-title":"The Annals of Statistics"},{"key":"2025091919460057700_qkae092-B27","doi-asserted-by":"crossref","DOI":"10.1007\/978-0-387-84858-7","volume-title":"The elements of statistical learning: Data mining, inference, and prediction","author":"Hastie","year":"2009","edition":"2nd ed."},{"issue":"3","key":"2025091919460057700_qkae092-B28","doi-asserted-by":"publisher","first-page":"661","DOI":"10.3150\/08-BEJ127","article-title":"Evaluation and selection of models for out-of-sample prediction when the sample size is small relative to the complexity of the data-generating process","volume":"14","author":"Leeb","year":"2008","journal-title":"Bernoulli"},{"key":"2025091919460057700_qkae092-B29","author":"LeJeune","year":"2020"},{"issue":"4","key":"2025091919460057700_qkae092-B30","doi-asserted-by":"publisher","first-page":"1352","DOI":"10.1214\/aos\/1176349742","article-title":"From Stein\u2019s unbiased risk estimates to the method of generalized cross validation","volume":"13","author":"Li","year":"1985","journal-title":"The Annals of Statistics"},{"issue":"3","key":"2025091919460057700_qkae092-B31","doi-asserted-by":"publisher","first-page":"1101","DOI":"10.1214\/aos\/1176350052","article-title":"Asymptotic optimality of C\u2113 and generalized cross-validation in ridge regression with application to spline smoothing","volume":"14","author":"Li","year":"1986","journal-title":"The Annals of Statistics"},{"issue":"3","key":"2025091919460057700_qkae092-B32","doi-asserted-by":"publisher","first-page":"958","DOI":"10.1214\/aos\/1176350486","article-title":"Asymptotic optimality for Cp, C\u2113, cross-validation and generalized cross-validation: Discrete index set","volume":"15","author":"Li","year":"1987","journal-title":"The Annals of Statistics"},{"issue":"2","key":"2025091919460057700_qkae092-B33","doi-asserted-by":"publisher","first-page":"1088","DOI":"10.1214\/18-AOS1707","article-title":"Estimating the algorithmic variance of randomized ensembles via the bootstrap","volume":"47","author":"Lopes","year":"2019","journal-title":"The Annals of Statistics"},{"issue":"4","key":"2025091919460057700_qkae092-B34","doi-asserted-by":"publisher","first-page":"921","DOI":"10.1137\/20M1343300","article-title":"Measuring the algorithmic convergence of randomized ensembles: The regression setting","volume":"2","author":"Lopes","year":"2020","journal-title":"SIAM Journal on Mathematics of Data Science"},{"key":"2025091919460057700_qkae092-B35","author":"Loureiro","year":"2022"},{"issue":"4","key":"2025091919460057700_qkae092-B36","doi-asserted-by":"publisher","first-page":"2313","DOI":"10.1214\/20-AOS2038","article-title":"The distribution of the lasso: Uniform control over sparse balls and adaptive parameter tuning","volume":"49","author":"Miolane","year":"2021","journal-title":"The Annals of Statistics"},{"key":"2025091919460057700_qkae092-B37","author":"M\u00fccke","year":"2022"},{"key":"2025091919460057700_qkae092-B38","author":"Patil","year":"2023"},{"issue":"319","key":"2025091919460057700_qkae092-B39","first-page":"1","article-title":"Bagging in overparameterized learning: Risk characterization and risk monotonization","volume":"24","author":"Patil","year":"2023","journal-title":"Journal of Machine Learning Research"},{"key":"2025091919460057700_qkae092-B40","author":"Patil","year":"2022"},{"key":"2025091919460057700_qkae092-B41","author":"Patil","year":"2021"},{"issue":"4","key":"2025091919460057700_qkae092-B42","doi-asserted-by":"publisher","first-page":"965","DOI":"10.1111\/rssb.12374","article-title":"A scalable estimate of the out-of-sample prediction error via approximate leave-one-out cross-validation","volume":"82","author":"Rad","year":"2020","journal-title":"Journal of the Royal Statistical Society: Series B (Statistical Methodology)"},{"issue":"6","key":"2025091919460057700_qkae092-B43","doi-asserted-by":"publisher","first-page":"1135","DOI":"10.1214\/aos\/1176345632","article-title":"Estimation of the mean of a multivariate normal distribution","volume":"9","author":"Stein","year":"1981","journal-title":"The Annals of Statistics"},{"issue":"3","key":"2025091919460057700_qkae092-B44","doi-asserted-by":"publisher","first-page":"1695","DOI":"10.3150\/23-BEJ1644","article-title":"Noise covariance estimation in multi-task high-dimensional linear models","volume":"30","author":"Tan","year":"2024","journal-title":"Bernoulli"},{"issue":"8","key":"2025091919460057700_qkae092-B45","doi-asserted-by":"publisher","first-page":"5592","DOI":"10.1109\/TIT.2018.2840720","article-title":"Precise error analysis of regularized M-estimators in high dimensions","volume":"64","author":"Thrampoulidis","year":"2018","journal-title":"IEEE Transactions on Information Theory"},{"issue":"2","key":"2025091919460057700_qkae092-B46","doi-asserted-by":"publisher","first-page":"1198","DOI":"10.1214\/12-AOS1003","article-title":"Degrees of freedom in lasso problems","volume":"40","author":"Tibshirani","year":"2012","journal-title":"The Annals of Statistics"},{"issue":"4","key":"2025091919460057700_qkae092-B47","doi-asserted-by":"publisher","first-page":"791","DOI":"10.1007\/s10463-016-0563-z","article-title":"The degrees of freedom of partly smooth regularizers","volume":"69","author":"Vaiter","year":"2017","journal-title":"Annals of the Institute of Statistical Mathematics"},{"issue":"5","key":"2025091919460057700_qkae092-B48","doi-asserted-by":"publisher","first-page":"2791","DOI":"10.1214\/19-AOS1906","article-title":"Which bridge estimator is the best for variable selection?","volume":"48","author":"Wang","year":"2020","journal-title":"The Annals of Statistics"},{"key":"2025091919460057700_qkae092-B49","author":"Wang","year":"2018"},{"key":"2025091919460057700_qkae092-B50","volume-title":"All of nonparametric statistics","author":"Wasserman","year":"2006"},{"issue":"6","key":"2025091919460057700_qkae092-B51","doi-asserted-by":"publisher","first-page":"378","DOI":"10.1080\/00029890.1983.11971235","article-title":"Do symmetric problems have symmetric solutions?","volume":"90","author":"Waterhouse","year":"1983","journal-title":"The American Mathematical Monthly"},{"key":"2025091919460057700_qkae092-B52","author":"Wei","year":"2022"},{"issue":"2","key":"2025091919460057700_qkae092-B53","doi-asserted-by":"publisher","first-page":"241","DOI":"10.1016\/S0893-6080(05)80023-1","article-title":"Stacked generalization","volume":"5","author":"Wolpert","year":"1992","journal-title":"Neural Networks"},{"issue":"9","key":"2025091919460057700_qkae092-B54","doi-asserted-by":"publisher","first-page":"5997","DOI":"10.1109\/TIT.2021.3095375","article-title":"Consistent risk estimation in moderately high-dimensional linear regression","volume":"67","author":"Xu","year":"2021","journal-title":"IEEE Transactions on Information Theory"},{"issue":"5","key":"2025091919460057700_qkae092-B55","doi-asserted-by":"publisher","first-page":"2173","DOI":"10.1214\/009053607000000127","article-title":"On the degrees of freedom of the lasso","volume":"35","author":"Zou","year":"2007","journal-title":"The Annals of Statistics"}],"container-title":["Journal of the Royal Statistical Society Series B: Statistical Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/87\/2\/289\/59180613\/qkae092.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/87\/2\/289\/59180613\/qkae092.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,19]],"date-time":"2025-09-19T23:46:09Z","timestamp":1758325569000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jrsssb\/article\/87\/2\/289\/7760017"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,18]]},"references-count":55,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2024,9,18]]},"published-print":{"date-parts":[[2025,4,11]]}},"URL":"https:\/\/doi.org\/10.1093\/jrsssb\/qkae092","relation":{},"ISSN":["1369-7412","1467-9868"],"issn-type":[{"value":"1369-7412","type":"print"},{"value":"1467-9868","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,4]]},"published":{"date-parts":[[2024,9,18]]}}}