{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T21:42:59Z","timestamp":1774474979390,"version":"3.50.1"},"reference-count":24,"publisher":"Oxford University Press (OUP)","issue":"12","license":[{"start":{"date-parts":[[2016,10,2]],"date-time":"2016-10-02T00:00:00Z","timestamp":1475366400000},"content-version":"vor","delay-in-days":2685,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/2.0\/uk\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2009,6,15]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Motivation: Permutation tests have become a standard tool to assess the statistical significance of an event under investigation. The statistical significance, as expressed in a P-value, is calculated as the fraction of permutation values that are at least as extreme as the original statistic, which was derived from non-permuted data. This empirical method directly couples both the minimal obtainable P-value and the resolution of the P-value to the number of permutations. Thereby, it imposes upon itself the need for a very large number of permutations when small P-values are to be accurately estimated. This is computationally expensive and often infeasible.<\/jats:p>\n               <jats:p>Results: A method of computing P-values based on tail approximation is presented. The tail of the distribution of permutation values is approximated by a generalized Pareto distribution. A good fit and thus accurate P-value estimates can be obtained with a drastically reduced number of permutations when compared with the standard empirical way of computing P-values.<\/jats:p>\n               <jats:p>Availability: The Matlab code can be obtained from the corresponding author on request.<\/jats:p>\n               <jats:p>Contact: \u00a0tknijnenburg@systemsbiology.org<\/jats:p>\n               <jats:p>Supplementary information: \u00a0Supplementary data are available at Bioinformatics online.<\/jats:p>","DOI":"10.1093\/bioinformatics\/btp211","type":"journal-article","created":{"date-parts":[[2009,5,28]],"date-time":"2009-05-28T15:48:54Z","timestamp":1243525734000},"page":"i161-i168","source":"Crossref","is-referenced-by-count":192,"title":["Fewer permutations, more accurate <i>P<\/i>-values"],"prefix":"10.1093","volume":"25","author":[{"given":"Theo A.","family":"Knijnenburg","sequence":"first","affiliation":[{"name":"1 Institute for Systems Biology, Seattle, WA, USA, 2Bioinformatics and Statistics, The Netherlands Cancer Institute, Amsterdam and 3Information and Communication Theory Group, Delft University of Technology, Delft, The Netherlands"}]},{"given":"Lodewyk F. A.","family":"Wessels","sequence":"additional","affiliation":[{"name":"1 Institute for Systems Biology, Seattle, WA, USA, 2Bioinformatics and Statistics, The Netherlands Cancer Institute, Amsterdam and 3Information and Communication Theory Group, Delft University of Technology, Delft, The Netherlands"}]},{"given":"Marcel J. T.","family":"Reinders","sequence":"additional","affiliation":[{"name":"1 Institute for Systems Biology, Seattle, WA, USA, 2Bioinformatics and Statistics, The Netherlands Cancer Institute, Amsterdam and 3Information and Communication Theory Group, Delft University of Technology, Delft, The Netherlands"}]},{"given":"Ilya","family":"Shmulevich","sequence":"additional","affiliation":[{"name":"1 Institute for Systems Biology, Seattle, WA, USA, 2Bioinformatics and Statistics, The Netherlands Cancer Institute, Amsterdam and 3Information and Communication Theory Group, Delft University of Technology, Delft, The Netherlands"}]}],"member":"286","published-online":{"date-parts":[[2009,5,27]]},"reference":[{"key":"2023013112002575900_B1","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1038\/75556","article-title":"Gene ontology: tool for the unification of biology. The Gene Ontology Consortium","volume":"25","author":"Ashburner","year":"2000","journal-title":"Nat. Genet."},{"key":"2023013112002575900_B2","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1186\/1471-2105-5-34","article-title":"Iterative Group Analysis (iga): a simple tool to enhance sensitivity and facilitate interpretation of microarray experiments","volume":"5","author":"Breitling","year":"2004","journal-title":"BMC Bioinformatics"},{"key":"2023013112002575900_B3","doi-asserted-by":"crossref","first-page":"478","DOI":"10.1198\/00401700152672573","article-title":"Goodness-of-fit tests for the generalized pareto distribution","volume":"43","author":"Choulakian","year":"2001","journal-title":"Technometrics"},{"key":"2023013112002575900_B4","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1093\/genetics\/142.1.285","article-title":"Permutation tests for multiple loci affecting a quantitative character","volume":"142","author":"Doerge","year":"1996","journal-title":"Genetics"},{"key":"2023013112002575900_B5","article-title":"Randomization Tests","author":"Edgington","year":"1980","journal-title":"Marcel Dekker, Inc."},{"key":"2023013112002575900_B6","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1214\/07-AOAS101","article-title":"On testing the significance of sets of genes","volume":"1","author":"Efron","year":"2007","journal-title":"Ann. Appl. Stat."},{"key":"2023013112002575900_B7","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1080\/00401706.1993.10485040","article-title":"Computing maximum likelihood estimates for the generalized pareto distribution","volume":"35","author":"Grimshaw","year":"1993","journal-title":"Technometrics"},{"key":"2023013112002575900_B8","doi-asserted-by":"crossref","DOI":"10.7312\/gumb92958","volume-title":"Statistics of extremes.","author":"Gumbel","year":"1958"},{"key":"2023013112002575900_B9","doi-asserted-by":"crossref","first-page":"339","DOI":"10.1080\/00401706.1987.10488243","article-title":"Parameter and quantile estimation for the generalized pareto distribution","volume":"29","author":"Hosking","year":"1987","journal-title":"Technometrics"},{"key":"2023013112002575900_B10","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1093\/nar\/28.1.27","article-title":"Kegg: kyoto encyclopedia of genes and genomes","volume":"28","author":"Kanehisa","year":"2000","journal-title":"Nucleic Acids Res."},{"key":"2023013112002575900_B11","doi-asserted-by":"crossref","first-page":"290","DOI":"10.1186\/1471-2105-8-290","article-title":"Computation of significance scores of unweighted gene set enrichment analyses","volume":"8","author":"Keller","year":"2007","journal-title":"BMC Bioinformatics"},{"key":"2023013112002575900_B12","doi-asserted-by":"crossref","DOI":"10.1186\/1471-2164-10-53","article-title":"Combinatorial effects of environmental parameters on transcriptional regulation in Saccharomyces cerevisiae: A quantitative analysis of a compendium of chemostat-based transcriptome data","volume":"10","author":"Knijnenburg","year":"2009","journal-title":"BMC Genomics"},{"key":"2023013112002575900_B13","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1093\/nar\/25.1.28","article-title":"Mips: a database for protein sequences, homology data and yeast genome information","volume":"25","author":"Mewes","year":"1997","journal-title":"Nucleic Acids Res"},{"key":"2023013112002575900_B14","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1038\/ng1180","article-title":"Pgc-1alpha-responsive genes involved in oxidative phosphorylation are coordinately downregulated in human diabetes","volume":"34","author":"Mootha","year":"2003","journal-title":"Nat. Genet."},{"key":"2023013112002575900_B15","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1111\/j.0006-341X.2000.00052.x","article-title":"Accounting for variability in the use of permutation testing to detect quantitative trait loci","volume":"56","author":"Nettleton","year":"2000","journal-title":"Biometrics"},{"key":"2023013112002575900_B16","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1089\/cmb.2008.0137","article-title":"Exact calculation of distributions on integers, with application to sequence alignment","volume":"16","author":"Newberg","year":"2009","journal-title":"J. Comput. Biol."},{"key":"2023013112002575900_B17","first-page":"119","article-title":"Statistical inference using extreme order statistics","volume":"3","author":"Pickands","year":"1975","journal-title":"Ann. Stat."},{"key":"2023013112002575900_B18","doi-asserted-by":"crossref","first-page":"6211","DOI":"10.1007\/978-94-017-3069-3_48","article-title":"Threshold methods for sample extremes","volume-title":"Statistical Extremes and Application.","author":"Smith","year":"1984"},{"key":"2023013112002575900_B19","doi-asserted-by":"crossref","first-page":"1","DOI":"10.2202\/1544-6115.1027","article-title":"Linear models and empirical bayes methods for assessing differential expression in microarray experiments","volume":"3","author":"Smyth","year":"2004","journal-title":"Stat. Appl. Genet Mol. Biol."},{"key":"2023013112002575900_B20","doi-asserted-by":"crossref","first-page":"15545","DOI":"10.1073\/pnas.0506580102","article-title":"Gene set enrichment analysis: a knowledge-based approach for interpreting genome-wide expression profiles","volume":"102","author":"Subramanian","year":"2005","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"2023013112002575900_B21","doi-asserted-by":"crossref","first-page":"5116","DOI":"10.1073\/pnas.091062498","article-title":"Significance analysis of microarrays applied to the ionizing radiation response","volume":"98","author":"Tusher","year":"2001","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"2023013112002575900_B22","doi-asserted-by":"crossref","first-page":"1999","DOI":"10.1056\/NEJMoa021967","article-title":"A gene-expression signature as a predictor of survival in breast cancer","volume":"347","author":"van de","year":"2002","journal-title":"N. Engl. J. Med."},{"key":"2023013112002575900_B23","doi-asserted-by":"crossref","first-page":"e1000158","DOI":"10.1371\/journal.pcbi.1000158","article-title":"Modeling chip sequencing in silico with applications","volume":"4","author":"Zhang","year":"2008","journal-title":"PLoS Comput. Biol."},{"key":"2023013112002575900_B24","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1159\/000022901","article-title":"Model-free analysis and permutation tests for allelic associations","volume":"50","author":"Zhao","year":"2000","journal-title":"Hum. Hered"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/25\/12\/i161\/48990923\/bioinformatics_25_12_i161.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/25\/12\/i161\/48990923\/bioinformatics_25_12_i161.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,31]],"date-time":"2023-01-31T21:07:36Z","timestamp":1675199256000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/25\/12\/i161\/189924"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2009,5,27]]},"references-count":24,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2009,6,15]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btp211","relation":{},"ISSN":["1367-4811","1367-4803"],"issn-type":[{"value":"1367-4811","type":"electronic"},{"value":"1367-4803","type":"print"}],"subject":[],"published-other":{"date-parts":[[2009,6,15]]},"published":{"date-parts":[[2009,5,27]]}}}