{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,11]],"date-time":"2026-04-11T18:34:23Z","timestamp":1775932463829,"version":"3.50.1"},"reference-count":35,"publisher":"Oxford University Press (OUP)","issue":"14","license":[{"start":{"date-parts":[[2018,12,6]],"date-time":"2018-12-06T00:00:00Z","timestamp":1544054400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"name":"Augusta University Medical College of Georgia"},{"DOI":"10.13039\/501100004912","name":"Sichuan University","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100004912","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012226","name":"Fundamental Research Funds for the Central Universities of China","doi-asserted-by":"publisher","award":["20822041B4009"],"award-info":[{"award-number":["20822041B4009"]}],"id":[{"id":"10.13039\/501100012226","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012226","name":"Fundamental Research Funds for the Central Universities of China","doi-asserted-by":"publisher","award":["20822041A4202"],"award-info":[{"award-number":["20822041A4202"]}],"id":[{"id":"10.13039\/501100012226","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["J1310022"],"award-info":[{"award-number":["J1310022"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"NIH","doi-asserted-by":"publisher","award":["5P30CA118100"],"award-info":[{"award-number":["5P30CA118100"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"NIH","doi-asserted-by":"publisher","award":["P30CA046592"],"award-info":[{"award-number":["P30CA046592"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2019,7,15]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Motivation<\/jats:title><jats:p>Small P-values are often required to be accurately estimated in large-scale genomic studies for the adjustment of multiple hypothesis tests and the ranking of genomic features based on their statistical significance. For those complicated test statistics whose cumulative distribution functions are analytically intractable, existing methods usually do not work well with small P-values due to lack of accuracy or computational restrictions. We propose a general approach for accurately and efficiently estimating small P-values for a broad range of complicated test statistics based on the principle of the cross-entropy method and Markov chain Monte Carlo sampling techniques.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>We evaluate the performance of the proposed algorithm through simulations and demonstrate its application to three real-world examples in genomic studies. The results show that our approach can accurately evaluate small to extremely small P-values (e.g. 10-6 to 10-100). The proposed algorithm is helpful for the improvement of some existing test procedures and the development of new test procedures in genomic studies.<\/jats:p><\/jats:sec><jats:sec><jats:title>Availability and implementation<\/jats:title><jats:p>R programs for implementing the algorithm and reproducing the results are available at: https:\/\/github.com\/shilab2017\/MCMC-CE-codes.<\/jats:p><\/jats:sec><jats:sec><jats:title>Supplementary information<\/jats:title><jats:p>Supplementary data are available at Bioinformatics online.<\/jats:p><\/jats:sec>","DOI":"10.1093\/bioinformatics\/bty1005","type":"journal-article","created":{"date-parts":[[2018,12,5]],"date-time":"2018-12-05T20:13:02Z","timestamp":1544040782000},"page":"2441-2448","source":"Crossref","is-referenced-by-count":8,"title":["Accurate and efficient estimation of small<i>P<\/i>-values with the cross-entropy method: applications in genomic data analysis"],"prefix":"10.1093","volume":"35","author":[{"given":"Yang","family":"Shi","sequence":"first","affiliation":[{"name":"Division of Biostatistics and Data Science, Department of Population Health Sciences, Medical College of Georgia, Augusta University, Augusta, Georgia, USA"},{"name":"Department of Epidemiology and Biostatistics, West China School of Public Health, Sichuan University, Chengdu, Sichuan, China"},{"name":"Biostatistics Shared Resource, University of New Mexico Comprehensive Cancer Center and Department of Internal Medicine, University of New Mexico, Albuquerque, New Mexico, USA"},{"name":"Department of Biostatistics, University of Michigan, Ann Arbor, Michigan, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mengqiao","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Epidemiology and Biostatistics, West China School of Public Health, Sichuan University, Chengdu, Sichuan, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Weiping","family":"Shi","sequence":"additional","affiliation":[{"name":"College of Mathematics, Jilin University, Changchun, Jilin, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ji-Hyun","family":"Lee","sequence":"additional","affiliation":[{"name":"Division of Quantitative Sciences, University of Florida Health Cancer Center and Department of Biostatistics, University of Florida, Gainesville, Florida, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Huining","family":"Kang","sequence":"additional","affiliation":[{"name":"Biostatistics Shared Resource, University of New Mexico Comprehensive Cancer Center and Department of Internal Medicine, University of New Mexico, Albuquerque, New Mexico, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hui","family":"Jiang","sequence":"additional","affiliation":[{"name":"Department of Biostatistics, University of Michigan, Ann Arbor, Michigan, USA"},{"name":"Center for Computational Medicine and Bioinformatics"},{"name":"University of Michigan Rogel Cancer Center, University of Michigan, Ann Arbor, Michigan, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2018,12,6]]},"reference":[{"key":"2023062712302788300_bty1005-B1","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1038\/75556","article-title":"Gene Ontology: tool for the unification of biology","volume":"25","author":"Ashburner","year":"2000","journal-title":"Nat. Genet"},{"key":"2023062712302788300_bty1005-B2","doi-asserted-by":"crossref","first-page":"2446","DOI":"10.1016\/j.csda.2008.11.028","article-title":"How accurate are the extremely small P-values used in genomic research: an evaluation of numerical libraries","volume":"53","author":"Bangalore","year":"2009","journal-title":"Comput. Stat. Data Anal"},{"key":"2023062712302788300_bty1005-B3","doi-asserted-by":"crossref","first-page":"505202.","DOI":"10.1088\/1751-8113\/46\/50\/505202","article-title":"On the efficient calculation of a linear combination of chi-square random variables with an application in counting string vacua","volume":"46","author":"Bausch","year":"2013","journal-title":"J. Phys. A Math. Theor"},{"key":"2023062712302788300_bty1005-B4","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1111\/j.2517-6161.1995.tb02031.x","article-title":"Controlling the false discovery rate: a practical and powerful approach to multiple testing","author":"Benjamini","year":"1995","journal-title":"J. R. Stat. Soc. Ser. B (Methodological)"},{"key":"2023062712302788300_bty1005-B5","doi-asserted-by":"crossref","first-page":"228.","DOI":"10.1186\/1471-2105-12-228","article-title":"Proportion statistics to detect differentially expressed genes: a comparison with log-ratio statistics","volume":"12","author":"Bergemann","year":"2011","journal-title":"BMC Bioinformatics"},{"key":"2023062712302788300_bty1005-B6","doi-asserted-by":"crossref","first-page":"176","DOI":"10.1158\/2159-8290.CD-15-0859","article-title":"Recurrent fusions in MYB and MYBL1 define a common, transcription factor-driven oncogenic pathway in salivary gland adenoid cystic carcinoma","volume":"6","author":"Brayer","year":"2016","journal-title":"Cancer Disc"},{"key":"2023062712302788300_bty1005-B151","first-page":"161","article-title":"A Family of MCMC Methods on Implicitly Defined Manifolds","author":"Brubaker","year":"2012","journal-title":"Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics"},{"key":"2023062712302788300_bty1005-B7","doi-asserted-by":"crossref","first-page":"661","DOI":"10.1038\/nature05911","article-title":"Genome-wide association study of 14,\u2009000 cases of seven common diseases and 3,000 shared controls","volume":"447","author":"Burton","year":"2007","journal-title":"Nature"},{"key":"2023062712302788300_bty1005-B8","doi-asserted-by":"crossref","first-page":"1031","DOI":"10.1007\/s11222-011-9275-7","article-title":"Improved cross-entropy method for estimation","volume":"22","author":"Chan","year":"2012","journal-title":"Stat. Comput"},{"key":"2023062712302788300_bty1005-B9","doi-asserted-by":"crossref","first-page":"251","DOI":"10.2307\/1390645","article-title":"Performance of the Gibbs, hit-and-run, and Metropolis samplers","volume":"2","author":"Chen","year":"1993","journal-title":"J. Comput. Graph. Stat"},{"key":"2023062712302788300_bty1005-B10","doi-asserted-by":"crossref","first-page":"1207","DOI":"10.1093\/bioinformatics\/18.9.1207","article-title":"Ratio statistics of gene expression levels and applications to microarray data analysis","volume":"18","author":"Chen","year":"2002","journal-title":"Bioinformatics"},{"key":"2023062712302788300_bty1005-B11","first-page":"323","article-title":"Algorithm AS 155: the distribution of a linear combination of \u03c7 2 random variables","volume":"29","author":"Davies","year":"1980","journal-title":"J. R. Stat. Soc. Ser. C (Appl. Stat.)"},{"key":"2023062712302788300_bty1005-B12","doi-asserted-by":"crossref","first-page":"858","DOI":"10.1016\/j.csda.2009.11.025","article-title":"Computing the distribution of quadratic forms: further comparisons between the Liu\u2013Tang\u2013Zhang approximation and exact methods","volume":"54","author":"Duchesne","year":"2010","journal-title":"Comput. Stat. Data Anal"},{"key":"2023062712302788300_bty1005-B13","first-page":"332","article-title":"Algorithm AS 204: the distribution of a positive linear combination of \u03c7 2 random variables","volume":"33","author":"Farebrother","year":"1984","journal-title":"J. R. Stat. Soc. Ser. C (Appl. Stat.)"},{"key":"2023062712302788300_bty1005-B14","doi-asserted-by":"crossref","first-page":"D331","DOI":"10.1093\/nar\/gkw1108","article-title":"Expansion of the Gene Ontology knowledgebase and resources","volume":"45","year":"2017","journal-title":"Nucleic Acids Res"},{"key":"2023062712302788300_bty1005-B15","first-page":"571","author":"Geweke","year":"1991"},{"key":"2023062712302788300_bty1005-B16","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1093\/bioinformatics\/btg382","article-title":"A global test for groups of genes: testing association with a clinical outcome","volume":"20","author":"Goeman","year":"2004","journal-title":"Bioinformatics"},{"key":"2023062712302788300_bty1005-B17","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1093\/biomet\/asr016","article-title":"Testing against a high-dimensional alternative in the generalized linear model: asymptotic type I error control","volume":"98","author":"Goeman","year":"2011","journal-title":"Biometrika"},{"key":"2023062712302788300_bty1005-B18","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1093\/biomet\/48.3-4.419","article-title":"Computing the distribution of quadratic forms in normal variables","volume":"48","author":"Imhof","year":"1961","journal-title":"Biometrika"},{"key":"2023062712302788300_bty1005-B19","first-page":"1757","author":"Kotecha","year":"1999"},{"key":"2023062712302788300_bty1005-B20","doi-asserted-by":"crossref","DOI":"10.1002\/9781118014967","volume-title":"Handbook of Monte Carlo Methods","author":"Kroese","year":"2011"},{"key":"2023062712302788300_bty1005-B21","first-page":"357","article-title":"Markov Chain Monte Carlo from Lagrangian Dynamics","volume":"24","author":"Lan","year":"2015","journal-title":"J. Comput. Graph. Stat. Jt. Publ. Am. Stat. Assoc. Inst. Math. Stat. Interface Found. N. Am"},{"key":"2023062712302788300_bty1005-B22","doi-asserted-by":"crossref","first-page":"781","DOI":"10.1093\/bioinformatics\/bti053","article-title":"An efficient Monte Carlo approach to assessing statistical significance in genomic studies","volume":"21","author":"Lin","year":"2005","journal-title":"Bioinformatics"},{"key":"2023062712302788300_bty1005-B23","doi-asserted-by":"crossref","first-page":"1079","DOI":"10.1111\/j.1541-0420.2007.00799.x","article-title":"Semiparametric Regression of multidimensional genetic pathway data: least \u2013 Squares Kernel Machines and Linear Mixed Models","volume":"63","author":"Liu","year":"2007","journal-title":"Biometrics"},{"key":"2023062712302788300_bty1005-B24","doi-asserted-by":"crossref","first-page":"853","DOI":"10.1016\/j.csda.2008.11.025","article-title":"A new chi-square approximation to the distribution of non-negative definite quadratic forms in non-central normal variables","volume":"53","author":"Liu","year":"2009","journal-title":"Comput. Stat. Data Anal"},{"key":"2023062712302788300_bty1005-B25","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1089\/106652701300099074","article-title":"On differential variability of expression ratios: improving statistical inference about gene expression changes from microarray data","volume":"8","author":"Newton","year":"2001","journal-title":"J. Comput. Biol"},{"key":"2023062712302788300_bty1005-B26","article-title":"Auxiliary-variable exact Hamiltonian Monte Carlo samplers for binary distributions","author":"Pakman","year":"2013","journal-title":"Advances in Neural Information Processing Systems 26 (NIPS 2013). Neural Information Processing Systems Foundation, Inc"},{"key":"2023062712302788300_bty1005-B27","doi-asserted-by":"crossref","first-page":"518","DOI":"10.1080\/10618600.2013.788448","article-title":"Exact hamiltonian monte carlo for truncated multivariate gaussians","volume":"23","author":"Pakman","year":"2014","journal-title":"J. Comput. Graph. Stat"},{"key":"2023062712302788300_bty1005-B28","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1093\/bioinformatics\/btp616","article-title":"edgeR: a Bioconductor package for differential expression analysis of digital gene expression data","volume":"26","author":"Robinson","year":"2010","journal-title":"Bioinformatics"},{"key":"2023062712302788300_bty1005-B29","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4757-4321-0","volume-title":"The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation and Machine Learning","author":"Rubinstein","year":"2004"},{"key":"2023062712302788300_bty1005-B30","doi-asserted-by":"crossref","first-page":"196","DOI":"10.1111\/biom.12731","article-title":"Fast approximation of small p-values in permutation tests by partitioning the permutations","volume":"74","author":"Segal","year":"2017","journal-title":"Biometrics"},{"key":"2023062712302788300_bty1005-B31","author":"Shi","year":"2016"},{"key":"2023062712302788300_bty1005-B32","doi-asserted-by":"crossref","first-page":"5116","DOI":"10.1073\/pnas.091062498","article-title":"Significance analysis of microarrays applied to the ionizing radiation response","volume":"98","author":"Tusher","year":"2001","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"2023062712302788300_bty1005-B33","doi-asserted-by":"crossref","first-page":"879","DOI":"10.1038\/ng1840","article-title":"Genome-wide genetic association of complex traits in heterogeneous stock mice","volume":"38","author":"Valdar","year":"2006","journal-title":"Nat. Genet"},{"key":"2023062712302788300_bty1005-B34","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/j.ajhg.2011.05.029","article-title":"Rare-variant association testing for sequencing data with the sequence kernel association test","volume":"89","author":"Wu","year":"2011","journal-title":"Am. J. Hum. Genet"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/35\/14\/2441\/50720469\/bioinformatics_35_14_2441.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/35\/14\/2441\/50720469\/bioinformatics_35_14_2441.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,13]],"date-time":"2024-07-13T01:49:21Z","timestamp":1720835361000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/35\/14\/2441\/5232222"}},"subtitle":[],"editor":[{"given":"Bonnie","family":"Berger","sequence":"additional","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2018,12,6]]},"references-count":35,"journal-issue":{"issue":"14","published-print":{"date-parts":[[2019,7,15]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/bty1005","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2019,7]]},"published":{"date-parts":[[2018,12,6]]}}}