{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,12]],"date-time":"2026-04-12T19:30:35Z","timestamp":1776022235830,"version":"3.50.1"},"reference-count":19,"publisher":"Springer Science and Business Media LLC","issue":"1","content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Bioinformatics"],"published-print":{"date-parts":[[2007,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:sec>\n            <jats:title>Background<\/jats:title>\n            <jats:p>Multiple data-analytic methods have been proposed for evaluating gene-expression levels in specific biological pathways, assessing differential expression associated with a binary phenotype. Following Goeman and B\u00fchlmann's recent review, we compared statistical performance of three methods, namely Global Test, ANCOVA Global Test, and SAM-GS, that test \"self-contained null hypotheses\" Via. subject sampling. The three methods were compared based on a simulation experiment and analyses of three real-world microarray datasets.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Results<\/jats:title>\n            <jats:p>In the simulation experiment, we found that the use of the asymptotic distribution in the two Global Tests leads to a statistical test with an incorrect size. Specifically, p-values calculated by the scaled <jats:italic>\u03c7<\/jats:italic>\n              <jats:sup>2<\/jats:sup> distribution of Global Test and the asymptotic distribution of ANCOVA Global Test are too liberal, while the asymptotic distribution with a quadratic form of the Global Test results in p-values that are too conservative. The two Global Tests with permutation-based inference, however, gave a correct size. While the three methods showed similar power using permutation inference after a proper standardization of gene expression data, SAM-GS showed slightly higher power than the Global Tests. In the analysis of a real-world microarray dataset, the two Global Tests gave markedly different results, compared to SAM-GS, in identifying pathways whose gene expressions are associated with <jats:italic>p53<\/jats:italic> mutation in cancer cell lines. A proper standardization of gene expression variances is necessary for the two Global Tests in order to produce biologically sensible results. After the standardization, the three methods gave very similar biologically-sensible results, with slightly higher statistical significance given by SAM-GS. The three methods gave similar patterns of results in the analysis of the other two microarray datasets.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Conclusion<\/jats:title>\n            <jats:p>An appropriate standardization makes the performance of all three methods similar, given the use of permutation-based inference. SAM-GS tends to have slightly higher power in the lower <jats:italic>\u03b1<\/jats:italic>-level region (i.e. gene sets that are of the greatest interest). Global Test and ANCOVA Global Test have the important advantage of being able to analyze continuous and survival phenotypes and to adjust for covariates. A free Microsoft Excel Add-In to perform SAM-GS is available from <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" xlink:href=\"http:\/\/www.ualberta.ca\/~yyasui\/homepage.html\" ext-link-type=\"uri\">http:\/\/www.ualberta.ca\/~yyasui\/homepage.html<\/jats:ext-link>.<\/jats:p>\n          <\/jats:sec>","DOI":"10.1186\/1471-2105-8-431","type":"journal-article","created":{"date-parts":[[2007,11,7]],"date-time":"2007-11-07T19:29:18Z","timestamp":1194463758000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":80,"title":["Comparative evaluation of gene-set analysis methods"],"prefix":"10.1186","volume":"8","author":[{"given":"Qi","family":"Liu","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Irina","family":"Dinu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Adeniyi J","family":"Adewale","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"John D","family":"Potter","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yutaka","family":"Yasui","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2007,11,7]]},"reference":[{"key":"1803_CR1","doi-asserted-by":"publisher","first-page":"5116","DOI":"10.1073\/pnas.091062498","volume":"98","author":"VG Tusher","year":"2001","unstructured":"Tusher VG, Tibshirani R, Chu G: Significance analysis of microarrays applied to the ionizing radiation response. Proc Natl Acad Sci USA 2001, 98: 5116\u20135121. 10.1073\/pnas.091062498","journal-title":"Proc Natl Acad Sci USA"},{"key":"1803_CR2","doi-asserted-by":"publisher","first-page":"267","DOI":"10.1038\/ng1180","volume":"34","author":"VK Mootha","year":"2003","unstructured":"Mootha VK, Lindgren CM, Eriksson KF, Subramanian A, Sihag S, Lehar J, Puigserver P, Carlsson E, Ridderstrale M, Laurila E, Houstis N, Daly MJ, Patterson N, Mesirov JP, Golub TR, Tamayo P, Spiegelman B, Lander ES, Hirschhorn JN, Altshuler D, Groop LC: PGC-1alpha-responsive genes involved in oxidative phosphorylation are coordinately downregulated in human diabetes. Nat Genet 2003, 34: 267\u2013273. 10.1038\/ng1180","journal-title":"Nat Genet"},{"key":"1803_CR3","doi-asserted-by":"publisher","first-page":"15545","DOI":"10.1073\/pnas.0506580102","volume":"102","author":"A Subramanian","year":"2005","unstructured":"Subramanian A, Tamayo P, Mootha VK, Mukherjee S, Ebert BL, Gillette MA, Paulovich A, Pomeroy SL, Golub TR, Lander ES, Mesirov JP: Gene set enrichment analysis: a knowledge-based approach for interpreting genome-wide expression profiles. Proc Natl Acad Sci USA 2005, 102: 15545\u201315550. 10.1073\/pnas.0506580102","journal-title":"Proc Natl Acad Sci USA"},{"key":"1803_CR4","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1093\/bioinformatics\/btg382","volume":"20","author":"JJ Goeman","year":"2004","unstructured":"Goeman JJ, van de Geer SA, de Kort F, van Houwelingen HC: A global test for groups of genes: testing association with a clinical outcome. Bioinformatics 2004, 20: 93\u201399. 10.1093\/bioinformatics\/btg382","journal-title":"Bioinformatics"},{"key":"1803_CR5","doi-asserted-by":"publisher","first-page":"1950","DOI":"10.1093\/bioinformatics\/bti267","volume":"21","author":"JJ Goeman","year":"2005","unstructured":"Goeman JJ, Oosting J, Cleton-Jansen AM, Anninga JK, van Houwelingen HC: Testing association of a pathway with survival using gene expression data. Bioinformatics 2005, 21: 1950\u20131957. 10.1093\/bioinformatics\/bti267","journal-title":"Bioinformatics"},{"key":"1803_CR6","doi-asserted-by":"crossref","first-page":"449","DOI":"10.1055\/s-0038-1633982","volume":"44","author":"U Mansmann","year":"2005","unstructured":"Mansmann U, Meister R: Testing differential gene expression in functional groups. Goeman's global test versus an ANCOVA approach. Methods Inf Med 2005, 44: 449\u2013453.","journal-title":"Methods Inf Med"},{"key":"1803_CR7","doi-asserted-by":"publisher","first-page":"242","DOI":"10.1186\/1471-2105-8-242","volume":"8","author":"I Dinu","year":"2007","unstructured":"Dinu I, Potter JD, Mueller T, Liu Q, Adewale AJ, Jhangri GS, Einecke G, Famulski KS, Halloran P, Yasui Y: Improving GSEA for Analysis of Biologic Pathways for Differential Gene Expression across a Binary Phenotype. BMC Bioinformatics 2007, 8: 242. 10.1186\/1471-2105-8-242","journal-title":"BMC Bioinformatics"},{"key":"1803_CR8","doi-asserted-by":"publisher","first-page":"980","DOI":"10.1093\/bioinformatics\/btm051","volume":"23","author":"JJ Goeman","year":"2007","unstructured":"Goeman JJ, B\u00fchlmann P: Analyzing gene expression data in terms of gene sets: methodological issues. Bioinformatics 2007, 23: 980\u2013987. 10.1093\/bioinformatics\/btm051","journal-title":"Bioinformatics"},{"key":"1803_CR9","unstructured":"Gene Set Enrichment Analysis[http:\/\/www.broad.mit.edu\/gsea]"},{"issue":"Suppl 1","key":"1803_CR10","doi-asserted-by":"publisher","first-page":"S96","DOI":"10.1093\/bioinformatics\/18.suppl_1.S96","volume":"18","author":"W Huber","year":"2002","unstructured":"Huber W, von Heydebreck A, Sultmann H, Poustka A, Vingron M: Variance stabilization applied to microarray data calibration and to the quantification of differential expression. Bioinformatics 2002, 18(Suppl 1):S96\u2013104.","journal-title":"Bioinformatics"},{"key":"1803_CR11","doi-asserted-by":"crossref","first-page":"Article3","DOI":"10.2202\/1544-6115.1008","volume":"2","author":"W Huber","year":"2003","unstructured":"Huber W, von Heydebreck A, Sueltmann H, Poustka A, Vingron M: Parameter estimation for the calibration and variance stabilization of microarray data. Stat Appl Genet Mol Biol 2003, 2: Article3.","journal-title":"Stat Appl Genet Mol Biol"},{"key":"1803_CR12","doi-asserted-by":"publisher","first-page":"477","DOI":"10.1111\/j.1467-9868.2006.00551.x","volume":"68","author":"JJ Goeman","year":"2006","unstructured":"Goeman JJ, Van de Geer SA, van Houwelingen HC: Testing against a high dimensional alternative. J R Statist Soc B 2006, 68: 477\u2013493. 10.1111\/j.1467-9868.2006.00551.x","journal-title":"J R Statist Soc B"},{"key":"1803_CR13","doi-asserted-by":"publisher","first-page":"479","DOI":"10.1111\/1467-9868.00346","volume":"64","author":"JD Storey","year":"2002","unstructured":"Storey JD: A direct approach to false discovery rates. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 2002, 64: 479\u2013498. 10.1111\/1467-9868.00346","journal-title":"Journal of the Royal Statistical Society: Series B (Statistical Methodology)"},{"key":"1803_CR14","doi-asserted-by":"publisher","first-page":"13544","DOI":"10.1073\/pnas.0506577102","volume":"102","author":"L Tian","year":"2005","unstructured":"Tian L, Greenberg SA, Kong SW, Altschuler J, Kohane IS, Park PJ: Discovering statistically significant pathways in expression profiling studies. Proc Natl Acad Sci USA 2005, 102: 13544\u201313549. 10.1073\/pnas.0506577102","journal-title":"Proc Natl Acad Sci USA"},{"key":"1803_CR15","doi-asserted-by":"publisher","first-page":"225","DOI":"10.1186\/1471-2105-6-225","volume":"6","author":"J Tomfohr","year":"2005","unstructured":"Tomfohr J, Lu J, Kepler TB: Pathway level analysis of gene expression using singular value decomposition. BMC Bioinformatics 2005, 6: 225. 10.1186\/1471-2105-6-225","journal-title":"BMC Bioinformatics"},{"key":"1803_CR16","doi-asserted-by":"publisher","first-page":"1943","DOI":"10.1093\/bioinformatics\/bti260","volume":"21","author":"WT Barry","year":"2005","unstructured":"Barry WT, Nobel AB, Wright FA: Significance analysis of functional categories in gene expression studies: a structured permutation approach. Bioinformatics 2005, 21: 1943\u20131949. 10.1093\/bioinformatics\/bti260","journal-title":"Bioinformatics"},{"key":"1803_CR17","doi-asserted-by":"publisher","first-page":"98","DOI":"10.1016\/S0888-7543(02)00021-6","volume":"81","author":"S Draghici","year":"2003","unstructured":"Draghici S, Khatri P, Martins RP, Ostermeier GC, Krawetz SA: Global functional profiling of gene expression. Genomics 2003, 81: 98\u2013104. 10.1016\/S0888-7543(02)00021-6","journal-title":"Genomics"},{"key":"1803_CR18","doi-asserted-by":"publisher","first-page":"600","DOI":"10.2307\/2532948","volume":"51","author":"S le Cessie","year":"1995","unstructured":"le Cessie S, van Houwelingen HC: Testing the fit of a regression model via score tests in random effects models. Biometrics 1995, 51: 600\u2013614. 10.2307\/2532948","journal-title":"Biometrics"},{"key":"1803_CR19","doi-asserted-by":"publisher","first-page":"1292","DOI":"10.2307\/2533260","volume":"51","author":"JJ Houwing-Duistermaat","year":"1995","unstructured":"Houwing-Duistermaat JJ, Derkx BH, Rosendaal FR, van Houwelingen HC: Testing familial aggregation. Biometrics 1995, 51: 1292\u20131301. 10.2307\/2533260","journal-title":"Biometrics"}],"container-title":["BMC Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/1471-2105-8-431.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,9,1]],"date-time":"2021-09-01T10:15:25Z","timestamp":1630491325000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/1471-2105-8-431"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2007,11,7]]},"references-count":19,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2007,12]]}},"alternative-id":["1803"],"URL":"https:\/\/doi.org\/10.1186\/1471-2105-8-431","relation":{},"ISSN":["1471-2105"],"issn-type":[{"value":"1471-2105","type":"electronic"}],"subject":[],"published":{"date-parts":[[2007,11,7]]},"assertion":[{"value":"5 September 2007","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 November 2007","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 November 2007","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"431"}}