{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,19]],"date-time":"2025-09-19T11:09:04Z","timestamp":1758280144853,"version":"3.37.3"},"reference-count":27,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2016,12,9]],"date-time":"2016-12-09T00:00:00Z","timestamp":1481241600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2016,12,9]],"date-time":"2016-12-09T00:00:00Z","timestamp":1481241600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Bioinformatics"],"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Background<\/jats:title>\n                <jats:p>Competitive gene set analysis is a standard exploratory tool for gene expression data. Permutation-based competitive gene set analysis methods are preferable to parametric ones because the latter make strong statistical assumptions which are not always met. For permutation-based methods, we permute samples, as opposed to genes, as doing so preserves the inter-gene correlation structure. Unfortunately, up until now, sample permutation-based methods have required a minimum of six replicates per sample group.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>We propose a new permutation-based competitive gene set analysis method for multi-group gene expression data with as few as three replicates per group. The method is based on advanced sample permutation technique that utilizes all groups within a data set for pairwise comparisons. We present a comprehensive evaluation of different permutation techniques, using multiple data sets and contrast the performance of our method, mGSZm, with other state of the art methods. We show that mGSZm is robust, and that, despite only using less than six replicates, we are able to consistently identify a high proportion of the top ranked gene sets from the analysis of a substantially larger data set. Further, we highlight other methods where performance is highly variable and appears dependent on the underlying data set being analyzed.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusions<\/jats:title>\n                <jats:p>Our results demonstrate that robust gene set analysis of multi-group gene expression data is permissible with as few as three replicates. In doing so, we have extended the applicability of such approaches to resource constrained experiments where additional data generation is prohibitively difficult or expensive. An R package implementing the proposed method and supplementary materials are available from the website <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"http:\/\/ekhidna.biocenter.helsinki.fi\/downloads\/pashupati\/mGSZm.html\">http:\/\/ekhidna.biocenter.helsinki.fi\/downloads\/pashupati\/mGSZm.html<\/jats:ext-link>.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1186\/s12859-016-1403-0","type":"journal-article","created":{"date-parts":[[2016,12,9]],"date-time":"2016-12-09T12:41:14Z","timestamp":1481287274000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Robust multi-group gene set analysis with few replicates"],"prefix":"10.1186","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5177-3431","authenticated-orcid":false,"given":"Pashupati P.","family":"Mishra","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alan","family":"Medlar","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Liisa","family":"Holm","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Petri","family":"T\u00f6r\u00f6nen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2016,12,9]]},"reference":[{"key":"1403_CR1","doi-asserted-by":"publisher","first-page":"600","DOI":"10.1038\/ng.3557","volume":"48","author":"J Kim","year":"2016","unstructured":"Kim J, Mouw KW, Polak P, Braunstein LZ, Kamburov A, Tiao G, Kwiatkowski DJ, Rosenberg JE, Van Allen EM, D D\u2019Andrea A, et al.Somatic ercc2 mutations are associated with a distinct genomic signature in urothelial tumors. Nature genetics. 2016; 48:600\u2013606.","journal-title":"Nature genetics"},{"issue":"15","key":"1403_CR2","doi-asserted-by":"publisher","first-page":"1899","DOI":"10.1038\/onc.2014.136","volume":"34","author":"Q Miow","year":"2015","unstructured":"Miow Q, Tan T, Ye J, Lau J, Yokomizo T, Thiery J, Mori S. Epithelial\u2013mesenchymal status renders differential responses to cisplatin in ovarian cancer. Oncogene. 2015; 34(15):1899\u20131907.","journal-title":"Oncogene"},{"key":"1403_CR3","doi-asserted-by":"publisher","first-page":"134","DOI":"10.1038\/srep00134","volume":"1","author":"RH Houtkooper","year":"2011","unstructured":"Houtkooper RH, Argmann C, Houten SM, Cant\u00f3 C, Jeninga EH, Andreux PA, Thomas C, Doenlen R, Schoonjans K, Auwerx J. The metabolic footprint of aging in mice. Scientific reports. 2011; 1:134.","journal-title":"Scientific reports"},{"key":"1403_CR4","doi-asserted-by":"publisher","first-page":"451","DOI":"10.1038\/nrm.2016.25","volume":"17","author":"CH Johnson","year":"2016","unstructured":"Johnson CH, Ivanisevic J, Siuzdak G. Metabolomics: beyond biomarkers and towards mechanisms. Nat Rev Mol Cell Biol. 2016; 17:451\u2013459.","journal-title":"Nat Rev Mol Cell Biol"},{"key":"1403_CR5","doi-asserted-by":"crossref","unstructured":"Perry JRB, McCarthy MI, Hattersley AT, Zeggini E, Wellcome Trust Case Control Consortium, Weedon MN, Frayling TM. Interrogating Type 2 Diabetes Genome-Wide Association Data Using a Biological Pathway-Based Approach. Diabetes. 2009; 58(6):1463\u20131467. doi:http:\/\/dx.doi.org\/10.2337\/db08-1378.","DOI":"10.2337\/db08-1378"},{"key":"1403_CR6","doi-asserted-by":"crossref","unstructured":"Elbers CC, van Eijk KR, Franke L, Mulder F, van der Schouw YT, Wijmenga C, Onland-Moret NC. Using genome-wide pathway analysis to unravel the etiology of complex diseases. Genet Epidemiol. 2009; 33(5):419\u201331. doi:http:\/\/dx.doi.org\/10.1002\/gepi.20395.","DOI":"10.1002\/gepi.20395"},{"issue":"1","key":"1403_CR7","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1038\/75556","volume":"25","author":"M Ashburner","year":"2000","unstructured":"Ashburner M, Ball CA, Blake JA, Botstein D, Butler H, Cherry JM, Davis AP, Dolinski K, Dwight SS, Eppig JT, et al.Gene ontology: tool for the unification of biology. Nat Genet. 2000; 25(1):25\u20139.","journal-title":"Nat Genet"},{"issue":"1","key":"1403_CR8","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1093\/nar\/28.1.27","volume":"28","author":"M Kanehisa","year":"2000","unstructured":"Kanehisa M, Goto S. Kegg: kyoto encyclopedia of genes and genomes. Nucleic Acids Res. 2000; 28(1):27\u201330.","journal-title":"Nucleic Acids Res"},{"issue":"8","key":"1403_CR9","doi-asserted-by":"publisher","first-page":"980","DOI":"10.1093\/bioinformatics\/btm051","volume":"23","author":"JJ Goeman","year":"2007","unstructured":"Goeman JJ, B\u00fchlmann P. Analyzing gene expression data in terms of gene sets: methodological issues. Bioinformatics. 2007; 23(8):980\u20137.","journal-title":"Bioinformatics"},{"issue":"1","key":"1403_CR10","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1214\/07-AOAS101","volume":"1","author":"B Efron","year":"2006","unstructured":"Efron B, Tibshirani R. On testing the significance of sets of genes. Ann Appl Stat. 2006; 1(1):107\u201329.","journal-title":"Ann Appl Stat"},{"key":"1403_CR11","doi-asserted-by":"publisher","first-page":"504","DOI":"10.1093\/bib\/bbt002","volume":"15","author":"H Maciejewski","year":"2013","unstructured":"Maciejewski H. Gene set analysis methods: statistical models and methodological differences. Briefings in bioinformatics. 2013; 15:504\u2013518.","journal-title":"Briefings in bioinformatics"},{"issue":"43","key":"1403_CR12","doi-asserted-by":"publisher","first-page":"15545","DOI":"10.1073\/pnas.0506580102","volume":"102","author":"A Subramanian","year":"2005","unstructured":"Subramanian A, Tamayo P, Mootha VK, Mukherjee S, Ebert BL, Gillette MA, Paulovich A, Pomeroy SL, Golub TR, Lander ES, et al.Gene set enrichment analysis: A knowledge-based approach for interpreting genome-wide expression profiles. Proc Natl Acad Sci U S A. 2005; 102(43):15545\u201315550.","journal-title":"Proc Natl Acad Sci U S A"},{"issue":"17","key":"1403_CR13","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1093\/nar\/gks461","volume":"40","author":"D Wu","year":"2012","unstructured":"Wu D, Smyth GK. Camera: a competitive gene set test accounting for inter-gene correlation. Nucleic Acids Res. 2012; 40(17):133.","journal-title":"Nucleic Acids Res"},{"issue":"1","key":"1403_CR14","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1214\/07-AOAS104","volume":"1","author":"MA Newton","year":"2007","unstructured":"Newton MA, Quintana FA, Boon JAD, Sengupta S, Ahlquist P. Random-set methods identify distinct aspects of the enrichment signal in gene-set analysis. Ann Appl Stat. 2007; 1(1):85\u2013106.","journal-title":"Ann Appl Stat"},{"key":"1403_CR15","doi-asserted-by":"publisher","first-page":"144","DOI":"10.1186\/1471-2105-6-144","volume":"6","author":"SY Kim","year":"2005","unstructured":"Kim SY, Volsky DJ. Page: parametric analysis of gene set enrichment. BMC Bioinforma. 2005; 6:144.","journal-title":"BMC Bioinforma"},{"issue":"19","key":"1403_CR16","doi-asserted-by":"publisher","first-page":"2747","DOI":"10.1093\/bioinformatics\/btu374","volume":"30","author":"P Mishra","year":"2014","unstructured":"Mishra P, T\u00f6r\u00f6nen P, Leino Y, Holm L. Gene set analysis: limitations in popular existing methods and proposed improvements. Bioinformatics. 2014; 30(19):2747\u2013756.","journal-title":"Bioinformatics"},{"issue":"1","key":"1403_CR17","doi-asserted-by":"publisher","first-page":"307","DOI":"10.1186\/1471-2105-10-307","volume":"10","author":"P T\u00f6r\u00f6nen","year":"2009","unstructured":"T\u00f6r\u00f6nen P, Ojala PJ, Marttinen P, Holm L. Robust extraction of functional signals from gene set analysis using a generalized threshold free scoring function. BMC Bioinforma. 2009; 10(1):307.","journal-title":"BMC Bioinforma"},{"issue":"10","key":"1403_CR18","doi-asserted-by":"publisher","first-page":"1544","DOI":"10.1093\/bioinformatics\/btu851","volume":"31","author":"P Koskinen","year":"2015","unstructured":"Koskinen P, T\u00f6r\u00f6nen P, Nokso-Koivisto J, Holm L. Pannzer: high-throughput functional annotation of uncharacterized proteins in an error-prone environment. Bioinformatics. 2015; 31(10):1544\u20131552.","journal-title":"Bioinformatics"},{"issue":"1","key":"1403_CR19","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/1756-0381-5-18","volume":"5","author":"H Wirth","year":"2012","unstructured":"Wirth H, von Bergen M, Binder H. Mining som expression portraits: Feature selection and integrating concepts of molecular function. BioData Min. 2012; 5(1):1.","journal-title":"BioData Min"},{"key":"1403_CR20","volume-title":"Low-Oxygen Stress in Plants","author":"OB Blokhina","year":"2014","unstructured":"Blokhina OB, T\u00f6r\u00f6nen P, Fagerstedt KV. Oxidative stress components explored in anoxic and hypoxic global gene expression data. In: Low-Oxygen Stress in Plants. Vienna: Springer: 2014. p. 19\u201339."},{"issue":"1","key":"1403_CR21","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1186\/1471-2105-10-161","volume":"10","author":"W Luo","year":"2009","unstructured":"Luo W, Friedman MS, Shedden K, Hankenson KD, Woolf PJ. Gage: generally applicable gene set enrichment for pathway analysis. BMC Bioinforma. 2009; 10(1):161.","journal-title":"BMC Bioinforma"},{"key":"1403_CR22","doi-asserted-by":"publisher","first-page":"e170","DOI":"10.1093\/nar\/gkt660","volume":"41","author":"G Yaari","year":"2013","unstructured":"Yaari G, Bolen CR, Thakar J, Kleinstein SH. Quantitative set analysis for gene expression: a method to quantify gene set differential expression including gene-gene correlations. Nucleic acids research. 2013; 41:e170.","journal-title":"Nucleic acids research"},{"issue":"1","key":"1403_CR23","doi-asserted-by":"crossref","first-page":"1","DOI":"10.2202\/1544-6115.1027","volume":"3","author":"GK Smyth","year":"2004","unstructured":"Smyth GK. Linear models and empirical bayes methods for assessing differential expression in microarray experiments. Stat Appl Genet Mol Biol. 2004; 3(1):1\u201325.","journal-title":"Stat Appl Genet Mol Biol"},{"key":"1403_CR24","doi-asserted-by":"publisher","first-page":"e47","DOI":"10.1093\/nar\/gkv007","volume":"43","author":"ME Ritchie","year":"2015","unstructured":"Ritchie ME, Phipson B, Wu D, Hu Y, Law CW, Shi W, Smyth GK. limma powers differential expression analyses for rna-sequencing and microarray studies. Nucleic acids research. 2015; 43:e47.","journal-title":"Nucleic acids research"},{"issue":"5","key":"1403_CR25","doi-asserted-by":"publisher","first-page":"731","DOI":"10.1182\/blood-2009-12-260760","volume":"116","author":"IJ Majewski","year":"2010","unstructured":"Majewski IJ, Ritchie ME, Phipson B, Corbin J, Pakusch M, Ebert A, Busslinger M, Koseki H, Hu Y, Smyth GK, et al. Opposing roles of polycomb repressive complexes in hematopoietic stem and progenitor cells. Blood. 2010; 116(5):731\u20139.","journal-title":"Blood"},{"issue":"5","key":"1403_CR26","doi-asserted-by":"publisher","first-page":"64483","DOI":"10.1371\/journal.pone.0064483","volume":"8","author":"Y Song","year":"2013","unstructured":"Song Y, Ahn J, Suh Y, Davis ME, Lee K. Identification of novel tissue-specific genes by analysis of microarray databases: a human and mouse model. PloS one. 2013; 8(5):64483.","journal-title":"PloS one"},{"key":"1403_CR27","doi-asserted-by":"publisher","first-page":"194","DOI":"10.1016\/j.immuni.2015.12.006","volume":"44","author":"J Godec","year":"2016","unstructured":"Godec J, Tan Y, Liberzon A, Tamayo P, Bhattacharya S, Butte AJ, Mesirov JP, Haining WN. Compendium of immune signatures identifies conserved and species-specific biology in response to inflammation. Immunity. 2016; 44:194\u2013206.","journal-title":"Immunity"}],"container-title":["BMC Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-016-1403-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12859-016-1403-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-016-1403-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,2,1]],"date-time":"2024-02-01T18:19:04Z","timestamp":1706811544000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/s12859-016-1403-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,12,9]]},"references-count":27,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2016,12]]}},"alternative-id":["1403"],"URL":"https:\/\/doi.org\/10.1186\/s12859-016-1403-0","relation":{},"ISSN":["1471-2105"],"issn-type":[{"type":"electronic","value":"1471-2105"}],"subject":[],"published":{"date-parts":[[2016,12,9]]},"assertion":[{"value":"1 July 2016","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 December 2016","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 December 2016","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"526"}}