{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T11:17:13Z","timestamp":1782386233029,"version":"3.54.5"},"reference-count":45,"publisher":"Oxford University Press (OUP)","issue":"4","license":[{"start":{"date-parts":[[2016,12,28]],"date-time":"2016-12-28T00:00:00Z","timestamp":1482883200000},"content-version":"vor","delay-in-days":799,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2015,2,15]]},"abstract":"<jats:p>Motivation: There are a number of well-established methods such as principal component analysis (PCA) for automatically capturing systematic variation due to latent variables in large-scale genomic data. PCA and related methods may directly provide a quantitative characterization of a complex biological variable that is otherwise difficult to precisely define or model. An unsolved problem in this context is how to systematically identify the genomic variables that are drivers of systematic variation captured by PCA. Principal components (PCs) (and other estimates of systematic variation) are directly constructed from the genomic variables themselves, making measures of statistical significance artificially inflated when using conventional methods due to over-fitting.<\/jats:p><jats:p>Results: We introduce a new approach called the jackstraw that allows one to accurately identify genomic variables that are statistically significantly associated with any subset or linear combination of PCs. The proposed method can greatly simplify complex significance testing problems encountered in genomics and can be used to identify the genomic variables significantly associated with latent variables. Using simulation, we demonstrate that our method attains accurate measures of statistical significance over a range of relevant scenarios. We consider yeast cell-cycle gene expression data, and show that the proposed method can be used to straightforwardly identify genes that are cell-cycle regulated with an accurate measure of statistical significance. We also analyze gene expression data from post-trauma patients, allowing the gene expression data to provide a molecularly driven phenotype. Using our method, we find a greater enrichment for inflammatory-related gene sets compared to the original analysis that uses a clinically defined, although likely imprecise, phenotype. The proposed method provides a useful bridge between large-scale quantifications of systematic variation and gene-level significance analyses.<\/jats:p><jats:p>Availability and implementation: An R software package, called jackstraw, is available in CRAN.<\/jats:p><jats:p>Contact: jstorey@princeton.edu<\/jats:p>","DOI":"10.1093\/bioinformatics\/btu674","type":"journal-article","created":{"date-parts":[[2014,10,22]],"date-time":"2014-10-22T05:07:31Z","timestamp":1413954451000},"page":"545-554","source":"Crossref","is-referenced-by-count":214,"title":["Statistical significance of variables driving systematic variation in high-dimensional data"],"prefix":"10.1093","volume":"31","author":[{"given":"Neo Christopher","family":"Chung","sequence":"first","affiliation":[{"name":"1 \u00a01Lewis-Sigler Institute for Integrative Genomics and 2Department of Molecular Biology, Princeton University, Princeton, NJ 08544, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"John D.","family":"Storey","sequence":"additional","affiliation":[{"name":"1 \u00a01Lewis-Sigler Institute for Integrative Genomics and 2Department of Molecular Biology, Princeton University, Princeton, NJ 08544, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2014,10,21]]},"reference":[{"key":"2023020108564478600_btu674-B1","doi-asserted-by":"crossref","first-page":"503","DOI":"10.1038\/35000501","article-title":"Distinct types of diffuse large b-cell lymphoma identified by gene expression profiling","volume":"403","author":"Alizadeh","year":"2000","journal-title":"Nature"},{"key":"2023020108564478600_btu674-B2","doi-asserted-by":"crossref","first-page":"10101","DOI":"10.1073\/pnas.97.18.10101","article-title":"Singular value decomposition for genome-wide expression data processing and modeling","volume":"97","author":"Alter","year":"2000","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"2023020108564478600_btu674-B3","doi-asserted-by":"crossref","first-page":"122","DOI":"10.1214\/aoms\/1177704248","article-title":"Asymptotic theory for principal component analysis","volume":"34","author":"Anderson","year":"1963","journal-title":"Ann. Math. Stat."},{"key":"2023020108564478600_btu674-B4","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1207\/s15327906mbr2704_2","article-title":"Remarks on parallel analysis","volume":"27","author":"Buja","year":"1992","journal-title":"Multivar. Behav. Res."},{"key":"2023020108564478600_btu674-B5","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1016\/S1097-2765(00)80114-8","article-title":"A genome-wide transcriptional analysis of the mitotic cell cycle","volume":"2","author":"Cho","year":"1998","journal-title":"Mol. Cell"},{"key":"2023020108564478600_btu674-B6","doi-asserted-by":"crossref","first-page":"4801","DOI":"10.1073\/pnas.0409768102","article-title":"Tompkins, and inflammation host response injury. Application of genome-wide expression analysis to human health and disease","volume":"102","author":"Cobb","year":"2005","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"2023020108564478600_btu674-B7","doi-asserted-by":"crossref","first-page":"457","DOI":"10.1038\/ng1296-457","article-title":"Use of a cDNA microarray to analyse gene expression patterns in human cancer","volume":"14","author":"DeRisi","year":"1996","journal-title":"Nat. Genet."},{"key":"2023020108564478600_btu674-B8","doi-asserted-by":"crossref","first-page":"e1001093","DOI":"10.1371\/journal.pmed.1001093","article-title":"Dissecting inflammatory complications in critically injured patients by within-patient gene expression changes: A longitudinal clinical genomics study","volume":"8","author":"Desai","year":"2011","journal-title":"PLoS Med."},{"key":"2023020108564478600_btu674-B9","first-page":"225","article-title":"K-means clustering via principal component analysis","volume-title":"Proceedings of the 21st International Conference on Machine learning","author":"Ding","year":"2004"},{"key":"2023020108564478600_btu674-B10","doi-asserted-by":"crossref","first-page":"126","DOI":"10.1214\/aos\/1176344552","article-title":"Bootstrap methods: Another look at the jackknife","volume":"7","author":"Efron","year":"1979","journal-title":"Ann. Stat."},{"key":"2023020108564478600_btu674-B11","doi-asserted-by":"crossref","first-page":"e1001117","DOI":"10.1371\/journal.pgen.1001117","article-title":"Analysis of population structure: a unifying framework and novel methods based on sparse factor analysis","volume":"6","author":"Engelhardt","year":"2010","journal-title":"PLoS Genet."},{"key":"2023020108564478600_btu674-B12","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1214\/aoms\/1177732180","article-title":"On the sampling theory of roots of determinantal equations","volume":"10","author":"Girshick","year":"1939","journal-title":"Ann. Math. Stat."},{"key":"2023020108564478600_btu674-B13","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1111\/j.1541-0420.2012.01808.x","article-title":"Corrected confidence bands for functional data using principal components","volume":"69","author":"Goldsmith","year":"2013","journal-title":"Biometrics"},{"key":"2023020108564478600_btu674-B14","volume-title":"The Elements of Statistical Learning: Data Mining, Inference, and Prediction","author":"Hastie","year":"2011","edition":"2nd edn"},{"key":"2023020108564478600_btu674-B15","doi-asserted-by":"crossref","first-page":"8409","DOI":"10.1073\/pnas.150242097","article-title":"Fundamental patterns underlying gene expression profiles: simplicity from complexity","volume":"97","author":"Holter","year":"2000","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"2023020108564478600_btu674-B16","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1214\/aos\/1009210544","article-title":"On the distribution of the largest eigenvalue in principal components analysis","volume":"29","author":"Johnstone","year":"2001","journal-title":"Ann. Stat."},{"key":"2023020108564478600_btu674-B17","volume-title":"Principal Component Analysis","author":"Jolliffe","year":"2002","edition":"2nd edn"},{"key":"2023020108564478600_btu674-B18","first-page":"531","article-title":"A modified principal component technique based on the lasso","volume":"12","author":"Jolliffe","year":"2003","journal-title":"JCGS"},{"key":"2023020108564478600_btu674-B19","doi-asserted-by":"crossref","first-page":"344","DOI":"10.1111\/j.1541-0420.2010.01455.x","article-title":"Asymptotic conditional singular value decomposition for high-dimensional genomic data","volume":"67","author":"Leek","year":"2010","journal-title":"Biometrics"},{"key":"2023020108564478600_btu674-B20","doi-asserted-by":"crossref","first-page":"e161","DOI":"10.1371\/journal.pgen.0030161","article-title":"Capturing heterogeneity in gene expression studies by surrogate variable analysis","volume":"3","author":"Leek","year":"2007","journal-title":"PLoS Genet."},{"key":"2023020108564478600_btu674-B21","doi-asserted-by":"crossref","first-page":"18718","DOI":"10.1073\/pnas.0808709105","article-title":"A general framework for multiple testing dependence","volume":"105","author":"Leek","year":"2008","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"2023020108564478600_btu674-B22","doi-asserted-by":"crossref","first-page":"Article 28","DOI":"10.2202\/1544-6115.1673","article-title":"The joint null criterion for multiple hypothesis tests","volume":"10","author":"Leek","year":"2011","journal-title":"Stat. Appl. Genet. Mol. Biol."},{"key":"2023020108564478600_btu674-B23","volume-title":"Testing Statistical Hypotheses","author":"Lehmann","year":"1997","edition":"2nd edn"},{"key":"2023020108564478600_btu674-B24","doi-asserted-by":"crossref","first-page":"440","DOI":"10.1007\/s11336-011-9216-6","article-title":"Statistical significance of the contribution of variables to the PCA solution: an alternative permutation strategy","volume":"76","author":"Linting","year":"2011","journal-title":"Psychometrika"},{"key":"2023020108564478600_btu674-B25","doi-asserted-by":"crossref","first-page":"e1035","DOI":"10.1371\/journal.pone.0001035","article-title":"Assembly of inflammation-related genes for pathway-focused genetic analysis","volume":"2","author":"Loza","year":"2007","journal-title":"PLoS One"},{"key":"2023020108564478600_btu674-B26","doi-asserted-by":"crossref","first-page":"1638","DOI":"10.1097\/00003246-199510000-00007","article-title":"Multiple organ dysfunction score\u2014a reliable descriptor of a complex clinical outcome","volume":"23","author":"Marshall","year":"1995","journal-title":"Crit. Care Med."},{"key":"2023020108564478600_btu674-B27","first-page":"903","article-title":"Link analysis, eigenvectors and stability","volume":"2","author":"Ng","year":"2001","journal-title":"Int. Jt. Conf. Artif. Intell."},{"key":"2023020108564478600_btu674-B28","doi-asserted-by":"crossref","first-page":"2347","DOI":"10.1890\/00-0634","article-title":"Giving meaningful interpretation to ordination axes: assessing loading significance in principal component analysis","volume":"84","author":"Peres-Neto","year":"2003","journal-title":"Ecology"},{"key":"2023020108564478600_btu674-B29","doi-asserted-by":"crossref","first-page":"2266","DOI":"10.1101\/gad.1450606","article-title":"The forkhead transcription factor hcm1 regulates chromosome segregation genes and fills the s-phase gap in the transcriptional circuitry of the cell cycle","volume":"20","author":"Pramila","year":"2006","journal-title":"Genes Dev."},{"key":"2023020108564478600_btu674-B30","doi-asserted-by":"crossref","first-page":"904","DOI":"10.1038\/ng1847","article-title":"Principal components analysis corrects for stratification in genome-wide association studies","volume":"38","author":"Price","year":"2006","journal-title":"Nat. Genet."},{"key":"2023020108564478600_btu674-B31","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1111\/j.2517-6161.1949.tb00023.x","article-title":"Approximate tests of correlation in time series","volume":"11","author":"Quenouille","year":"1949","journal-title":"J. R. Stat. Soc. Ser. B"},{"key":"2023020108564478600_btu674-B32","doi-asserted-by":"crossref","DOI":"10.1007\/b98888","volume-title":"Functional Data Analysis","author":"Ramsay","year":"2005"},{"key":"2023020108564478600_btu674-B33","first-page":"452","article-title":"Principal components analysis to summarize microarray experiments: application to sporulation time series","volume":"5","author":"Raychaudhuri","year":"2000","journal-title":"Pac. Symp. Biocomput."},{"key":"2023020108564478600_btu674-B34","doi-asserted-by":"crossref","first-page":"16892","DOI":"10.1073\/pnas.0706022104","article-title":"High-resolution timing of cell cycle-regulated gene expression","volume":"104","author":"Rowicka","year":"2007","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"2023020108564478600_btu674-B35","doi-asserted-by":"crossref","first-page":"3273","DOI":"10.1091\/mbc.9.12.3273","article-title":"Comprehensive identification of cell cycle-regulated genes of the yeast Saccharomyces cerevisiae by microarray hybridization","volume":"9","author":"Spellman","year":"1998","journal-title":"Mol. Biol. Cell"},{"key":"2023020108564478600_btu674-B36","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1348\/000711006X109636","article-title":"Estimating confidence intervals for principal component loadings: a comparison between the bootstrap and asymptotic results","volume":"60","author":"Timmerman","year":"2010","journal-title":"Br. J. Math. Stat. Psychol."},{"key":"2023020108564478600_btu674-B37","doi-asserted-by":"crossref","first-page":"727","DOI":"10.1007\/BF02099545","article-title":"On orthogonal and symplectic matrix ensembles","volume":"177","author":"Tracy","year":"1996","journal-title":"Commun. Math. Phys."},{"key":"2023020108564478600_btu674-B38","doi-asserted-by":"crossref","first-page":"1152","DOI":"10.1126\/science.1120499","article-title":"Logic of the yeast metabolic cycle: temporal compartmentalization of cellular processes","volume":"310","author":"Tu","year":"2005","journal-title":"Science"},{"key":"2023020108564478600_btu674-B39","first-page":"614","article-title":"Bias and confidence in not quite large samples","volume":"29","author":"Tukey","year":"1958","journal-title":"Ann. Math. Stat."},{"key":"2023020108564478600_btu674-B40","doi-asserted-by":"crossref","first-page":"515","DOI":"10.1093\/biostatistics\/kxp008","article-title":"A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis","volume":"10","author":"Witten","year":"2009","journal-title":"Biostatistics"},{"key":"2023020108564478600_btu674-B41","doi-asserted-by":"crossref","first-page":"522","DOI":"10.1186\/1471-2105-9-522","article-title":"Systematic identification of yeast cell cycle transcription factors using multiple data sources","volume":"9","author":"Wu","year":"2008","journal-title":"BMC Bioinf."},{"key":"2023020108564478600_btu674-B42","doi-asserted-by":"crossref","first-page":"577","DOI":"10.1198\/016214504000001745","article-title":"Functional data analysis for sparse longitudinal data","volume":"100","author":"Yao","year":"2005","journal-title":"J. Am. Stat. Assoc."},{"key":"2023020108564478600_btu674-B43","article-title":"Spectral relaxation for k-means clustering","volume-title":"Neural Information Processing Systems vol.14 (NIPS 2001)","author":"Zha","year":"2001"},{"key":"2023020108564478600_btu674-B44","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1002\/gepi.210","article-title":"Association mapping, using a mixture model for complex traits","volume":"23","author":"Zhu","year":"2002","journal-title":"Genet. Epidemiol."},{"key":"2023020108564478600_btu674-B45","first-page":"262","article-title":"Sparse principal component analysis","volume":"15","author":"Zou","year":"2006","journal-title":"JCGS"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/31\/4\/545\/49011114\/bioinformatics_31_4_545.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/31\/4\/545\/49011114\/bioinformatics_31_4_545.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,4]],"date-time":"2024-06-04T07:09:27Z","timestamp":1717484967000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/31\/4\/545\/2748186"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,10,21]]},"references-count":45,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2015,2,15]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btu674","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published":{"date-parts":[[2014,10,21]]}}}