{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,30]],"date-time":"2026-08-30T01:57:29Z","timestamp":1788055049079,"version":"build-2784847793"},"reference-count":47,"publisher":"Oxford University Press (OUP)","issue":"2","license":[{"start":{"date-parts":[[2023,2,24]],"date-time":"2023-02-24T00:00:00Z","timestamp":1677196800000},"content-version":"vor","delay-in-days":1120,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2020,4,1]]},"abstract":"<jats:title>Summary<\/jats:title><jats:p>We consider the problem of estimating a high dimensional p \u00d7 p covariance matrix \u03a3, given n observations of confounded data with covariance \u03a3+\u0393\u0393T, where \u0393 is an unknown p \u00d7 q matrix of latent factor loadings. We propose a simple and scalable estimator based on the projection onto the right singular vectors of the observed data matrix, which we call right singular vector projection (RSVP). Our theoretical analysis of this method reveals that, in contrast with approaches based on the removal of principal components, RSVP can cope well with settings where the smallest eigenvalue of \u0393T\u0393 is relatively close to the largest eigenvalue of \u03a3, as well as when the eigenvalues of \u0393T\u0393 are diverging fast. RSVP does not require knowledge or estimation of the number of latent factors q, but it recovers \u03a3 only up to an unknown positive scale factor. We argue that this suffices in many applications, e.g. if an estimate of the correlation matrix is desired. We also show that, by using subsampling, we can further improve the performance of the method. We demonstrate the favourable performance of RSVP through simulation experiments and an analysis of gene expression data sets collated by the GTEX consortium.<\/jats:p>","DOI":"10.1111\/rssb.12359","type":"journal-article","created":{"date-parts":[[2020,1,31]],"date-time":"2020-01-31T15:53:52Z","timestamp":1580486032000},"page":"361-389","source":"Crossref","is-referenced-by-count":13,"title":["Right Singular Vector Projection Graphs: Fast High Dimensional Covariance Matrix Estimation under Latent Confounding"],"prefix":"10.1093","volume":"82","author":[{"given":"Rajen D.","family":"Shah","sequence":"first","affiliation":[{"name":"University of Cambridge \u00a0 UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Benjamin","family":"Frot","sequence":"additional","affiliation":[{"name":"Eidgen\u00f6ssiche Technische Hochschule Z\u00fcrich \u00a0 Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gian-Andrea","family":"Thanei","sequence":"additional","affiliation":[{"name":"Eidgen\u00f6ssiche Technische Hochschule Z\u00fcrich \u00a0 Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nicolai","family":"Meinshausen","sequence":"additional","affiliation":[{"name":"Eidgen\u00f6ssiche Technische Hochschule Z\u00fcrich \u00a0 Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2020,1,31]]},"reference":[{"key":"2023022408363499700_","first-page":"204","article-title":"Local genetic effects on gene expression across 44 human tissues","volume":"550","author":"Aguet","year":"2016","journal-title":"Nature"},{"key":"2023022408363499700_","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1038\/75556","article-title":"Gene ontology: tool for the unification of biology","volume":"25","author":"Ashburner","year":"2000","journal-title":"Nat. Genet."},{"key":"2023022408363499700_","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1111\/1468-0262.00273","article-title":"Determining the number of factors in approximate factor models","volume":"70","author":"Bai","year":"2002","journal-title":"Econometrica"},{"key":"2023022408363499700_","volume-title":"Consistent estimation of high-dimensional factor models when the factor number is over-estimated","author":"Barigozzi","year":"2018"},{"key":"2023022408363499700_","doi-asserted-by":"crossref","first-page":"791","DOI":"10.1093\/biomet\/asr043","article-title":"Square-root lasso: pivotal recovery of sparse signals via conic programming","volume":"98","author":"Belloni","year":"2011","journal-title":"Biometrika"},{"key":"2023022408363499700_","doi-asserted-by":"crossref","first-page":"2577","DOI":"10.1214\/08-AOS600","article-title":"Covariance regularization by thresholding","volume":"36","author":"Bickel","year":"2008","journal-title":"Ann. Statist."},{"key":"2023022408363499700_","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1007\/BF00058655","article-title":"Bagging predictors","volume":"24","author":"Breiman","year":"1996","journal-title":"Mach. Learn."},{"key":"2023022408363499700_","doi-asserted-by":"crossref","first-page":"594","DOI":"10.1198\/jasa.2011.tm10155","article-title":"A constrained \u21131 minimization approach to sparse precision matrix estimation","volume":"106","author":"Cai","year":"2011","journal-title":"J. Am. Statist. Ass."},{"key":"2023022408363499700_","first-page":"1","article-title":"Estimating structured high-dimensional covariance and precision matrices: optimal rates and adaptive estimation","volume":"10","author":"Cai","year":"2016","journal-title":"Electron. J. Statist."},{"key":"2023022408363499700_","doi-asserted-by":"crossref","DOI":"10.1145\/1970392.1970395","article-title":"Robust principal component analysis?","volume":"58","author":"Cand\u00e8s","year":"2011","journal-title":"J. Ass. Comput. Mach."},{"key":"2023022408363499700_","volume-title":"Spectral deconfounding and perturbed sparse linear models","author":"\u0106evid","year":"2018"},{"key":"2023022408363499700_","first-page":"1935","article-title":"Latent variable graphical model selection via convex optimization","volume":"40","author":"Chandrasekaran","year":"2012","journal-title":"Ann. Statist."},{"key":"2023022408363499700_","doi-asserted-by":"crossref","first-page":"572","DOI":"10.1137\/090761793","article-title":"Rank-sparsity incoherence for matrix decomposition","volume":"21","author":"Chandrasekaran","year":"2011","journal-title":"SIAM J. Optimzn"},{"key":"2023022408363499700_","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1214\/16-AOS1434","article-title":"A lava attack on the recovery of sums of dense and sparse signals","volume":"45","author":"Chernozhukov","year":"2017","journal-title":"Ann. Statist."},{"key":"2023022408363499700_","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1137\/0707001","article-title":"The rotation of eigenvectors by a perturbation: iii","volume":"7","author":"Davis","year":"1970","journal-title":"SIAM J. Numer. Anal."},{"key":"2023022408363499700_","doi-asserted-by":"crossref","first-page":"1742","DOI":"10.1214\/17-AOS1601","article-title":"Optimal shrinkage of eigenvalues in the spiked covariance model","volume":"46","author":"Donoho","year":"2018","journal-title":"Ann. Statist."},{"key":"2023022408363499700_","doi-asserted-by":"crossref","first-page":"603","DOI":"10.1111\/rssb.12016","article-title":"Large covariance estimation by thresholding principal orthogonal complements (with discussion)","volume":"75","author":"Fan","year":"2013","journal-title":"J. R. Statist. Soc."},{"key":"2023022408363499700_","doi-asserted-by":"crossref","first-page":"1383","DOI":"10.1214\/17-AOS1588","article-title":"Large covariance estimation through elliptical factor models","volume":"46","author":"Fan","year":"2018","journal-title":"Ann. Statist."},{"key":"2023022408363499700_","doi-asserted-by":"crossref","first-page":"432","DOI":"10.1093\/biostatistics\/kxm045","article-title":"Sparse inverse covariance estimation with the graphical lasso","volume":"9","author":"Friedman","year":"2008","journal-title":"Biostatistics"},{"key":"2023022408363499700_","volume-title":"glasso: graphical lasso: estimation of Gaussian graphical models","author":"Friedman","year":"2018"},{"key":"2023022408363499700_","doi-asserted-by":"crossref","first-page":"723","DOI":"10.1080\/01621459.2018.1434531","article-title":"Graphical model selection for Gaussian conditional random fields in the presence of latent variables","volume":"114","author":"Frot","year":"2019","journal-title":"J. Am. Statist. Ass."},{"key":"2023022408363499700_","doi-asserted-by":"crossref","first-page":"459","DOI":"10.1111\/rssb.12315","article-title":"Robust causal structure learning with some hidden variables","volume":"81","author":"Frot","year":"2019","journal-title":"J. R. Statist. Soc."},{"key":"2023022408363499700_","volume-title":"Removing unwanted variation from high dimensional data with negative controls","author":"Gagnon-Bartsch","year":"2013"},{"key":"2023022408363499700_","doi-asserted-by":"crossref","first-page":"2693","DOI":"10.3150\/17-BEJ941","article-title":"Max-linear models on directed acyclic graphs","volume":"24","author":"Gissibl","year":"2018","journal-title":"Bernoulli"},{"key":"2023022408363499700_","doi-asserted-by":"crossref","first-page":"1","DOI":"10.2307\/1906935","article-title":"The probability approach in econometrics","volume":"12","author":"Haavelmo","year":"1944","journal-title":"Econometrica"},{"key":"2023022408363499700_","doi-asserted-by":"crossref","first-page":"603","DOI":"10.1198\/016214506000001275","article-title":"Determining the number of factors in the general dynamic factor model","volume":"102","author":"Hallin","year":"2007","journal-title":"J. Am. Statist. Ass."},{"key":"2023022408363499700_","first-page":"3365","article-title":"PC algorithm for nonparanormal graphical models","volume":"14","author":"Harris","year":"2013","journal-title":"J. Mach. Learn. Res."},{"key":"2023022408363499700_","doi-asserted-by":"crossref","first-page":"371","DOI":"10.1146\/annurev-statistics-031017-100630","article-title":"Causal structure learning","volume":"5","author":"Heinze-Deml","year":"2018","journal-title":"A. Rev. Statist. Appl."},{"key":"2023022408363499700_","doi-asserted-by":"crossref","first-page":"1150","DOI":"10.1214\/15-EJS1029","article-title":"Preconditioning the lasso for sign consistency","volume":"9","author":"Jia","year":"2015","journal-title":"Electron. J. Statist."},{"key":"2023022408363499700_","first-page":"613","article-title":"Estimating high-dimensional directed acyclic graphs with the PC-algorithm","volume":"8","author":"Kalisch","year":"2007","journal-title":"J. Mach. Learn. Res."},{"key":"2023022408363499700_","volume-title":"Uniform Hanson-Wright type concentration inequalities for unbounded entries via the entropy method","author":"Klochkov","year":"2018"},{"key":"2023022408363499700_","doi-asserted-by":"crossref","DOI":"10.1093\/oso\/9780198522195.001.0001","volume-title":"Graphical Models","author":"Lauritzen","year":"1996"},{"key":"2023022408363499700_","doi-asserted-by":"crossref","first-page":"365","DOI":"10.1016\/S0047-259X(03)00096-4","article-title":"A well-conditioned estimator for large-dimensional covariance matrices","volume":"88","author":"Ledoit","year":"2004","journal-title":"J. Multiv. Anal."},{"key":"2023022408363499700_","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pgen.0030161","article-title":"Capturing heterogeneity in gene expression studies by surrogate variable analysis","volume":"3","author":"Leek","year":"2007","journal-title":"PLOS Genet."},{"key":"2023022408363499700_","first-page":"411","volume-title":"Uncertainty in Artificial Intelligence","author":"Meek","year":"1995"},{"key":"2023022408363499700_","doi-asserted-by":"crossref","first-page":"1436","DOI":"10.1214\/009053606000000281","article-title":"High dimensional graphs and variable selection with the Lasso","volume":"34","author":"Meinshausen","year":"2006","journal-title":"Ann. Statist."},{"key":"2023022408363499700_","doi-asserted-by":"crossref","first-page":"439","DOI":"10.1007\/978-0-387-77439-8_15","volume-title":"Handbook of Portfolio Construction","author":"Menchero","year":"2010"},{"key":"2023022408363499700_","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511803161","volume-title":"Causality","author":"Pearl","year":"2009"},{"key":"2023022408363499700_","doi-asserted-by":"crossref","first-page":"991","DOI":"10.1214\/14-AOS1286","article-title":"Asymptotic normality and optimalities in estimation of large Gaussian graphical models","volume":"43","author":"Ren","year":"2015","journal-title":"Ann. Statist."},{"key":"2023022408363499700_","doi-asserted-by":"crossref","first-page":"1393","DOI":"10.1016\/0270-0255(86)90088-6","article-title":"A new approach to causal inference in mortality studies with a sustained exposure period: application to control of the healthy worker survivor effect","volume":"7","author":"Robins","year":"1986","journal-title":"Math. Modllng"},{"key":"2023022408363499700_","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1002\/sta4.86","article-title":"Preconditioning for classical relationships: a note relating ridge regression and OLS p-values to preconditioned sparse penalized regression","volume":"4","author":"Rohe","year":"2015","journal-title":"Stat"},{"key":"2023022408363499700_","volume-title":"Causation, Prediction, and Search","author":"Spirtes","year":"2000"},{"key":"2023022408363499700_","doi-asserted-by":"crossref","first-page":"500","DOI":"10.1038\/nprot.2011.457","article-title":"Using probabilistic estimation of expression residuals (PEER) to obtain increased power and interpretability of gene expression analyses","volume":"7","author":"Stegle","year":"2012","journal-title":"Nat. Protcls"},{"key":"2023022408363499700_","first-page":"1343","article-title":"The xyz algorithm for fast interaction search in high-dimensional data","volume":"19","author":"Thanei","year":"2018","journal-title":"J. Mach. Learn. Res."},{"key":"2023022408363499700_","doi-asserted-by":"crossref","first-page":"589","DOI":"10.1111\/rssb.12127","article-title":"High dimensional ordinary least squares projection for screening variables","volume":"78","author":"Wang","year":"2015","journal-title":"J. R. Statist. Soc."},{"key":"2023022408363499700_","first-page":"2261","article-title":"High dimensional inverse covariance matrix estimation via linear programming","volume":"11","author":"Yuan","year":"2010","journal-title":"J. Mach. Learn. Res."},{"key":"2023022408363499700_","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1093\/biomet\/asm018","article-title":"Model selection and estimation in the Gaussian graphical model","volume":"94","author":"Yuan","year":"2007","journal-title":"Biometrika"}],"container-title":["Journal of the Royal Statistical Society Series B: Statistical Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1111%2Frssb.12359","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/82\/2\/361\/49321412\/jrsssb_82_2_361.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/82\/2\/361\/49321412\/jrsssb_82_2_361.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,30]],"date-time":"2024-07-30T22:17:10Z","timestamp":1722377830000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jrsssb\/article\/82\/2\/361\/7056088"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,1,31]]},"references-count":47,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2020,1,31]]},"published-print":{"date-parts":[[2020,4,1]]}},"URL":"https:\/\/doi.org\/10.1111\/rssb.12359","relation":{},"ISSN":["1369-7412","1467-9868"],"issn-type":[{"value":"1369-7412","type":"print"},{"value":"1467-9868","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2020,4]]},"published":{"date-parts":[[2020,1,31]]}}}