{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T04:16:20Z","timestamp":1790741780114,"version":"4.1.0"},"reference-count":45,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2019,5,24]],"date-time":"2019-05-24T00:00:00Z","timestamp":1558656000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/501100008982","name":"National Science Foundation","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100008982","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2019,9,1]]},"abstract":"<jats:title>Summary<\/jats:title><jats:p>The lasso has been studied extensively as a tool for estimating the coefficient vector in the high-dimensional linear model; however, considerably less is known about estimating the error variance in this context. In this paper, we propose the natural lasso estimator for the error variance, which maximizes a penalized likelihood objective. A key aspect of the natural lasso is that the likelihood is expressed in terms of the natural parameterization of the multi-parameter exponential family of a Gaussian with unknown mean and variance. The result is a remarkably simple estimator of the error variance with provably good performance in terms of mean squared error. These theoretical results do not require placing any assumptions on the design matrix or the true regression coefficients. We also propose a companion estimator, called the organic lasso, which theoretically does not require tuning of the regularization parameter. Both estimators do well empirically compared to pre-existing methods, especially in settings where successful recovery of the true support of the coefficient vector is hard. Finally, we show that existing methods can do well under fewer assumptions than previously known, thus providing a fuller story about the problem of estimating the error variance in high-dimensional linear models.<\/jats:p>","DOI":"10.1093\/biomet\/asz017","type":"journal-article","created":{"date-parts":[[2019,3,8]],"date-time":"2019-03-08T12:35:27Z","timestamp":1552048527000},"page":"533-546","source":"Crossref","is-referenced-by-count":19,"title":["Estimating the error variance in a high-dimensional linear model"],"prefix":"10.1093","volume":"106","author":[{"given":"Guo","family":"Yu","sequence":"first","affiliation":[{"name":"Department of Statistics, University of Washington, Box 354322, Seattle, Washington 98105, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jacob","family":"Bien","sequence":"additional","affiliation":[{"name":"Data Sciences and Operations, Marshall School of Business, University of Southern California, 3670 Trousdale Pkwy, Los Angeles, California 90089, U.S.A. jbien@usc.edu"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2019,5,24]]},"reference":[{"key":"2019091012113408600_B1","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1007\/s11749-010-0198-y","article-title":"Comments on: $\\ell_1$-penalization for mixture regression models","volume":"19","author":"Antoniadis,","year":"2010","journal-title":"Test"},{"key":"2019091012113408600_B2","first-page":"485","article-title":"Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data","volume":"9","author":"Banerjee,","year":"2008","journal-title":"J. Mach. Learn. Res."},{"key":"2019091012113408600_B3","first-page":"944","article-title":"Estimating lasso risk and noise level","volume-title":"Advances in Neural Information Processing Systems 26 (NIPS\u201913)","author":"Bayati,","year":"2013"},{"key":"2019091012113408600_B4","doi-asserted-by":"crossref","first-page":"791","DOI":"10.1093\/biomet\/asr043","article-title":"Square-root lasso: Pivotal recovery of sparse signals via conic programming","volume":"98","author":"Belloni,","year":"2011","journal-title":"Biometrika"},{"key":"2019091012113408600_B5","article-title":"The simulator: an engine to streamline simulations","author":"Bien,","year":"2016"},{"key":"2019091012113408600_B6","doi-asserted-by":"crossref","first-page":"807","DOI":"10.1093\/biomet\/asr054","article-title":"Sparse estimation of a covariance matrix","volume":"98","author":"Bien,","year":"2011","journal-title":"Biometrika"},{"key":"2019091012113408600_B7","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511804441","volume-title":"Convex Optimization","author":"Boyd,","year":"2004"},{"key":"2019091012113408600_B8","doi-asserted-by":"crossref","first-page":"1212","DOI":"10.3150\/12-BEJSP11","article-title":"Statistical significance in high-dimensional linear models","volume":"19","author":"B\u00fchlmann,","year":"2013","journal-title":"Bernoulli"},{"key":"2019091012113408600_B9","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-642-20192-9","volume-title":"Statistics for High-Dimensional Data: Methods, Theory and Applications","author":"B\u00fchlmann,","year":"2011"},{"key":"2019091012113408600_B10","doi-asserted-by":"crossref","first-page":"4220","DOI":"10.1214\/17-EJS1317","article-title":"Within group variable selection through the exclusive lasso","volume":"11","author":"Campbell,","year":"2017","journal-title":"Electron. J. Statist."},{"key":"2019091012113408600_B11","article-title":"Prediction error of cross-validated lasso","author":"Chatterjee,","year":"2016"},{"key":"2019091012113408600_B12","first-page":"1259","article-title":"Fused sparsity and robust estimation for linear models with unknown variance","volume-title":"Advances in Neural Information Processing Systems 25 (NIPS\u201912)","author":"Dalalyan,","year":"2012"},{"key":"2019091012113408600_B13","doi-asserted-by":"crossref","first-page":"552","DOI":"10.3150\/15-BEJ756","article-title":"On the prediction performance of the lasso","volume":"23","author":"Dalalyan,","year":"2017","journal-title":"Bernoulli"},{"key":"2019091012113408600_B14","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1093\/biomet\/ast065","article-title":"Variance estimation in high-dimensional linear models","volume":"101","author":"Dicker,","year":"2014","journal-title":"Biometrika"},{"key":"2019091012113408600_B15","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1111\/j.1467-9868.2011.01005.x","article-title":"Variance estimation using refitted cross-validation in ultrahigh dimensional regression","volume":"74","author":"Fan,","year":"2012","journal-title":"J. R. Statist. Soc. B"},{"key":"2019091012113408600_B16","doi-asserted-by":"crossref","first-page":"302","DOI":"10.1214\/07-AOAS131","article-title":"Pathwise coordinate optimization","volume":"1","author":"Friedman,","year":"2007","journal-title":"Ann. Appl. Statist."},{"key":"2019091012113408600_B17","doi-asserted-by":"crossref","first-page":"432","DOI":"10.1093\/biostatistics\/kxm045","article-title":"Sparse inverse covariance estimation with the graphical lasso","volume":"9","author":"Friedman,","year":"2008","journal-title":"Biostatistics"},{"key":"2019091012113408600_B18","article-title":"Prediction and estimation consistency of sparse multi-class penalized optimal scoring","author":"Gaynanova,","year":"2018"},{"key":"2019091012113408600_B19","doi-asserted-by":"crossref","DOI":"10.1201\/b18401","volume-title":"Statistical Learning with Sparsity: The Lasso and Generalizations","author":"Hastie,","year":"2015"},{"key":"2019091012113408600_B20","doi-asserted-by":"crossref","first-page":"1846","DOI":"10.1109\/TIT.2012.2227680","article-title":"How correlations influence lasso prediction","volume":"59","author":"Hebiri,","year":"2013","journal-title":"IEEE Trans. Info. Theory"},{"key":"2019091012113408600_B21","first-page":"2869","article-title":"Confidence intervals and hypothesis testing for high-dimensional regression","volume":"15","author":"Javanmard,","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"2019091012113408600_B22","article-title":"Oracle inequalities for high-dimensional prediction","author":"Lederer,","year":"2018"},{"key":"2019091012113408600_B23","doi-asserted-by":"crossref","first-page":"907","DOI":"10.1214\/15-AOS1371","article-title":"Exact post-selection inference, with application to the lasso","volume":"44","author":"Lee,","year":"2016","journal-title":"Ann. Statist."},{"key":"2019091012113408600_B24","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1214\/13-AOS1175","article-title":"A significance test for the lasso","volume":"42","author":"Lockhart,","year":"2014","journal-title":"Ann. Statist."},{"key":"2019091012113408600_B25","first-page":"477","article-title":"Exploiting covariate similarity in sparse regression via the pairwise elastic net","volume":"9","author":"Lorbert,","year":"2010","journal-title":"Proc. Mach. Learn. Res"},{"key":"2019091012113408600_B26","doi-asserted-by":"crossref","first-page":"158","DOI":"10.1214\/16-AOS1448","article-title":"A general theory of hypothesis tests and confidence regions for sparse high dimensional models","volume":"45","author":"Ning,","year":"2017","journal-title":"Ann. Statist."},{"key":"2019091012113408600_B27","volume-title":"R: A Language and Environment for Statistical Computing","year":"2019"},{"key":"2019091012113408600_B28","first-page":"35","article-title":"A study of error variance estimation in lasso regression","volume":"26","author":"Reid,","year":"2016","journal-title":"Statist. Sinica"},{"key":"2019091012113408600_B29","doi-asserted-by":"crossref","first-page":"731","DOI":"10.1214\/10-AOS854","article-title":"Exponential screening and optimal rates of sparse estimation","volume":"39","author":"Rigollet,","year":"2011","journal-title":"Ann. Statist."},{"key":"2019091012113408600_B30","doi-asserted-by":"crossref","volume-title":"Convex Analysis","author":"Rockafellar,","DOI":"10.1515\/9781400873173"},{"key":"2019091012113408600_B31","doi-asserted-by":"crossref","first-page":"209","DOI":"10.1007\/s11749-010-0197-z","article-title":"$\\ell_1$-penalization for mixture regression models (with discussion)","volume":"19","author":"St\u00e4dler,","year":"2010","journal-title":"Test"},{"key":"2019091012113408600_B32","doi-asserted-by":"crossref","first-page":"270","DOI":"10.1007\/s11749-010-0201-7","article-title":"Comments on: $\\ell_1$-penalization for mixture regression models","volume":"19","author":"Sun,","year":"2010","journal-title":"Test"},{"key":"2019091012113408600_B33","doi-asserted-by":"crossref","first-page":"879","DOI":"10.1093\/biomet\/ass043","article-title":"Scaled sparse linear regression","volume":"99","author":"Sun,","year":"2012","journal-title":"Biometrika"},{"key":"2019091012113408600_B34","first-page":"3385","article-title":"Sparse matrix inversion with scaled lasso","volume":"14","author":"Sun,","year":"2013","journal-title":"J. Mach. Learn. Res."},{"key":"2019091012113408600_B35","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1002\/cjs.11313","article-title":"Post-selection inference for $\\ell_1$-penalized likelihood models","volume":"46","author":"Taylor,","year":"2017","journal-title":"Can. J. Statist."},{"key":"2019091012113408600_B36","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","article-title":"Regression shrinkage and selection via the lasso","volume":"58","author":"Tibshirani,","year":"1996","journal-title":"J. R. Statist. Soc. B"},{"key":"2019091012113408600_B37","doi-asserted-by":"crossref","first-page":"1255","DOI":"10.1214\/17-AOS1584","article-title":"Uniform asymptotic inference and the bootstrap after model selection","volume":"46","author":"Tibshirani,","year":"2018","journal-title":"Ann. Statist."},{"key":"2019091012113408600_B38","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1080\/01621459.2015.1108848","article-title":"Exact post-selection inference for sequential regression procedures","volume":"111","author":"Tibshirani,","year":"2016","journal-title":"J. Am. Statist. Assoc."},{"key":"2019091012113408600_B39","doi-asserted-by":"crossref","first-page":"475","DOI":"10.1023\/A:1017501703105","article-title":"Convergence of a block coordinate descent method for nondifferentiable minimization","volume":"109","author":"Tseng,","year":"2001","journal-title":"J. Optimiz. Theory Appl."},{"key":"2019091012113408600_B40","doi-asserted-by":"crossref","first-page":"1166","DOI":"10.1214\/14-AOS1221","article-title":"On asymptotically optimal confidence regions and tests for high-dimensional models","volume":"42","author":"van de Geer,","year":"2014","journal-title":"Ann. Statist."},{"key":"2019091012113408600_B41","doi-asserted-by":"crossref","first-page":"1360","DOI":"10.1214\/09-EJS506","article-title":"On the conditions used to prove oracle results for the lasso","volume":"3","author":"van de Geer,","year":"2009","journal-title":"Electron. J. Statist."},{"key":"2019091012113408600_B42","author":"Yu,","year":"2017","journal-title":"natural: Estimating the Error Variance in a High-Dimensional Linear Model"},{"key":"2019091012113408600_B43","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1093\/biomet\/asm018","article-title":"Model selection and estimation in the Gaussian graphical model","volume":"94","author":"Yuan,","year":"2007","journal-title":"Biometrika"},{"key":"2019091012113408600_B44","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1111\/rssb.12026","article-title":"Confidence intervals for low dimensional parameters in high dimensional linear models","volume":"76","author":"Zhang,","year":"2014","journal-title":"J. R. Statist. Soc. B"},{"key":"2019091012113408600_B45","first-page":"988","article-title":"Exclusive lasso for multi-task feature selection","volume":"9","author":"Zhou,","year":"2010","journal-title":"Proc. Mach. Learn. Res"}],"container-title":["Biometrika"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/biomet\/advance-article-pdf\/doi\/10.1093\/biomet\/asz017\/28701987\/asz017.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/biomet\/article-pdf\/106\/3\/533\/29133541\/asz017.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,15]],"date-time":"2024-07-15T15:46:46Z","timestamp":1721058406000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/biomet\/article\/106\/3\/533\/5498375"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,5,24]]},"references-count":45,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2019,5,24]]},"published-print":{"date-parts":[[2019,9,1]]}},"URL":"https:\/\/doi.org\/10.1093\/biomet\/asz017","relation":{},"ISSN":["0006-3444","1464-3510"],"issn-type":[{"value":"0006-3444","type":"print"},{"value":"1464-3510","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2019,9]]},"published":{"date-parts":[[2019,5,24]]}}}