{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,26]],"date-time":"2026-02-26T20:34:37Z","timestamp":1772138077622,"version":"3.50.1"},"reference-count":69,"publisher":"Oxford University Press (OUP)","issue":"11","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2012,6,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Motivation: Several statistical tests are available to detect the enrichment of differential expression in gene sets. Such tests were originally proposed for analyzing gene sets associated with biological processes. The objective evaluation of tests on real measurements has not been possible as it is difficult to decide a priori, which processes will be affected in given experiments.<\/jats:p>\n                  <jats:p>Results: We present a first large study to rigorously assess and compare the performance of gene set enrichment tests on real expression measurements. Gene sets are defined based on the targets of given regulators such as transcription factors (TFs) and microRNAs (miRNAs). In contrast to processes, TFs and miRNAs are amenable to direct perturbations, e.g. regulator over-expression or deletion. We assess the ability of 14 different statistical tests to predict the perturbations from expression measurements in Escherichia coli, Saccharomyces cerevisiae and human. We also analyze how performance depends on the quality and comprehensiveness of the regulator targets via a permutation approach. We find that ANOVA and Wilcoxons test consistently perform better than for instance Kolmogorov\u2013Smirnov and hypergeometric tests. For scenarios where the optimal test is not known, we suggest to combine all evaluated tests into an unweighted consensus, which also performs well in our assessment. Our results provide a guide for the selection of existing tests as well as a basis for the development and assessment of novel tests.<\/jats:p>\n                  <jats:p>Contact: \u00a0robert.kueffner@bio.ifi.lmu.de<\/jats:p>\n                  <jats:p>Supplementary information: \u00a0Supplementary data are available at Bioinformatics online.<\/jats:p>","DOI":"10.1093\/bioinformatics\/bts164","type":"journal-article","created":{"date-parts":[[2012,4,7]],"date-time":"2012-04-07T23:53:17Z","timestamp":1333842797000},"page":"1480-1486","source":"Crossref","is-referenced-by-count":25,"title":["Rigorous assessment of gene set enrichment tests"],"prefix":"10.1093","volume":"28","author":[{"given":"Haroon","family":"Naeem","sequence":"first","affiliation":[{"name":"Department of Informatics, Ludwig-Maximilians Universit\u00e4t, Amalienstr. 17, 80333 M\u00fcnchen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ralf","family":"Zimmer","sequence":"additional","affiliation":[{"name":"Department of Informatics, Ludwig-Maximilians Universit\u00e4t, Amalienstr. 17, 80333 M\u00fcnchen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pegah","family":"Tavakkolkhah","sequence":"additional","affiliation":[{"name":"Department of Informatics, Ludwig-Maximilians Universit\u00e4t, Amalienstr. 17, 80333 M\u00fcnchen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Robert","family":"K\u00fcffner","sequence":"additional","affiliation":[{"name":"Department of Informatics, Ludwig-Maximilians Universit\u00e4t, Amalienstr. 17, 80333 M\u00fcnchen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2012,4,5]]},"reference":[{"key":"2023012512310690100_B1","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1186\/1471-2105-10-47","article-title":"A general modular framework for gene set enrichment analysis","volume":"10","author":"Ackermann","year":"2009","journal-title":"BMC Bioinformatics"},{"key":"2023012512310690100_B2","doi-asserted-by":"crossref","first-page":"578","DOI":"10.1093\/bioinformatics\/btg455","article-title":"FatiGO: a web tool for finding significant associations of Gene Ontology terms with groups of genes","volume":"20","author":"Al-Shahrour","year":"2004","journal-title":"Bioinformatics"},{"key":"2023012512310690100_B3","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1038\/nature07242","article-title":"The impact of microRNAs on protein output","volume":"455","author":"Baek","year":"2006","journal-title":"Nature"},{"key":"2023012512310690100_B4","doi-asserted-by":"crossref","first-page":"1943","DOI":"10.1093\/bioinformatics\/bti260","article-title":"Significance analysis of functional categories in gene expression studies: a structured permutation approach","volume":"21","author":"Barry","year":"2005","journal-title":"Bioinformatics"},{"key":"2023012512310690100_B5","doi-asserted-by":"crossref","first-page":"286","DOI":"10.1214\/07-AOAS146","article-title":"A statistical framework for testing functional categories in microarray data","volume":"2","author":"Barry","year":"2008","journal-title":"Ann. Appl. Stat."},{"key":"2023012512310690100_B6","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1016\/j.cell.2009.01.002","article-title":"MicroRNAs: target recognition and regulatory functions","volume":"136","author":"Bartel","year":"2009","journal-title":"Cell"},{"key":"2023012512310690100_B7","doi-asserted-by":"crossref","first-page":"3523","DOI":"10.1093\/nar\/gkq045","article-title":"GOing Bayesian: model-based gene set analysis of genome-scale data","volume":"38","author":"Bauer","year":"2010","journal-title":"Nucleic Acids Res."},{"key":"2023012512310690100_B8","doi-asserted-by":"crossref","first-page":"e3112","DOI":"10.1371\/journal.pone.0003112","article-title":"Inferring condition-specific modulation of transcription factor activity in yeast through regulon-based analysis of genomewide expression","volume":"3","author":"Boorsma","year":"2008","journal-title":"PLoS One"},{"key":"2023012512310690100_B9","article-title":"Memoire sur les elections au scrutin","volume-title":"Histoire de l'Academie des Sciences","author":"Borda","year":"1781"},{"key":"2023012512310690100_B10","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1038\/nrg1990","article-title":"The evolution of gene regulation by transcription factors and microRNAs","volume":"8","author":"Chen","year":"2007","journal-title":"Nat. Rev. Genet."},{"key":"2023012512310690100_B11","doi-asserted-by":"crossref","first-page":"R90","DOI":"10.1186\/gb-2009-10-9-r90","article-title":"mRNA expression profiles show differential regulatory effects of microRNAs between estrogen receptor-positive and estrogen receptor-negative breast cancer","volume":"10","author":"Cheng","year":"2009","journal-title":"Genome Biol."},{"key":"2023012512310690100_B12","doi-asserted-by":"crossref","first-page":"12045","DOI":"10.1073\/pnas.0605140103","article-title":"Identifying transcription factor functions and targets by phenotypic activation","volume":"103","author":"Chua","year":"2006","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"2023012512310690100_B13","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4899-4541-9","volume-title":"An Introduction to the Bootstrap.","author":"Efron","year":"1993"},{"key":"2023012512310690100_B14","first-page":"107","article-title":"On testing the significance of sets of genes","volume":"1","author":"Efron","year":"2006","journal-title":"Ann. Appl. Stat."},{"key":"2023012512310690100_B15","doi-asserted-by":"crossref","first-page":"e120","DOI":"10.1093\/nar\/gkq149","article-title":"Transcription factor regulation can be accurately predicted from the presence of target gene signatures in microarray gene expression data","volume":"38","author":"Essaghir","year":"2010","journal-title":"Nucleic Acids Res."},{"key":"2023012512310690100_B16","doi-asserted-by":"crossref","first-page":"D866","DOI":"10.1093\/nar\/gkm815","article-title":"Many microbe microarrays database: uniformly normalized Affymetrix compendia with structured experimental metadata","volume":"36","author":"Faith","year":"2008","journal-title":"Nucleic Acids Res."},{"key":"2023012512310690100_B17","doi-asserted-by":"crossref","first-page":"1817","DOI":"10.1126\/science.1121158","article-title":"The widespread impact of mammalian microRNAs on mRNA repression and evolution","volume":"10","author":"Farh","year":"2005","journal-title":"Science"},{"key":"2023012512310690100_B18","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1101\/gr.082701.108","article-title":"Most mammalian mRNAs are conserved targets of microRNAs","volume":"19","author":"Friedman","year":"2009","journal-title":"Genome Res."},{"key":"2023012512310690100_B19","doi-asserted-by":"crossref","first-page":"D98","DOI":"10.1093\/nar\/gkq1110","article-title":"RegulonDB version 7.0: transcriptional regulation of Escherichia coli K-12 integrated within genetic sensory response units (Gensor Units)","volume":"39","author":"Gama-Castro","year":"2011","journal-title":"Nucleic Acids Res."},{"key":"2023012512310690100_B20","doi-asserted-by":"crossref","first-page":"574","DOI":"10.1186\/1471-2164-11-574","article-title":"Heading down the wrong pathway: on the influence of correlation within gene sets","volume":"11","author":"Gatti","year":"2010","journal-title":"BMC Genomics"},{"key":"2023012512310690100_B21","doi-asserted-by":"crossref","first-page":"10105","DOI":"10.1158\/0008-5472.CAN-08-1846","article-title":"Coordinated regulation of cell cycle transcripts by p53-Inducible microRNAs, miR-192 and miR-215","volume":"68","author":"Georges","year":"2008","journal-title":"Cancer Res."},{"key":"2023012512310690100_B22","doi-asserted-by":"crossref","first-page":"980","DOI":"10.1093\/bioinformatics\/btm051","article-title":"Analyzing gene expression data in terms of gene sets: methodological issues","volume":"23","author":"Goeman","year":"2007","journal-title":"Bioinformatics"},{"key":"2023012512310690100_B23","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1016\/j.molcel.2007.06.017","article-title":"MicroRNA targeting specificity in mammals: determinants beyond seed pairing","volume":"27","author":"Grimson","year":"2007","journal-title":"Mol. Cell"},{"key":"2023012512310690100_B24","doi-asserted-by":"crossref","first-page":"D258","DOI":"10.1093\/nar\/gkh036","article-title":"The Gene Ontology (GO) database and informatics resource","volume":"32","author":"Harris","year":"2004","journal-title":"Nucleic Acids Res."},{"key":"2023012512310690100_B25","doi-asserted-by":"crossref","first-page":"1130","DOI":"10.1038\/nature05939","article-title":"A microRNA component of the p53 tumour suppressor network","volume":"447","author":"He","year":"2007","journal-title":"Nature"},{"key":"2023012512310690100_B26","doi-asserted-by":"crossref","first-page":"2423","DOI":"10.1101\/gr.1330003","article-title":"Reconciling gene expression data with known genome-scale regulatory network structures","volume":"13","author":"Herrg\u00e5rd","year":"2003","journal-title":"Genome Res."},{"key":"2023012512310690100_B27","doi-asserted-by":"crossref","first-page":"1785","DOI":"10.1126\/science.1151651","article-title":"Gene regulation by transcription factors and microRNAs","volume":"319","author":"Hobert","year":"2008","journal-title":"Science"},{"key":"2023012512310690100_B28","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1016\/j.ygeno.2009.12.006","article-title":"An efficient algorithm to identify coordinately activated transcription factors","volume":"95","author":"Hu","year":"2010","journal-title":"Genomics"},{"key":"2023012512310690100_B29","doi-asserted-by":"crossref","first-page":"683","DOI":"10.1038\/ng2012","article-title":"Genetic reconstruction of a functional transcriptional regulatory network","volume":"39","author":"Hu","year":"2007","journal-title":"Nat. Genet."},{"key":"2023012512310690100_B30","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1093\/nar\/gkn923","article-title":"Bioinformatics enrichment tools: paths toward the comprehensive functional analysis of large gene lists","volume":"37","author":"Huang da","year":"2009","journal-title":"Nucleic Acids Res."},{"key":"2023012512310690100_B31","doi-asserted-by":"crossref","first-page":"D98","DOI":"10.1093\/nar\/gkn714","article-title":"miR2Disease: a manually curated database for microRNA deregulation in human disease","volume":"37","author":"Jiang","year":"2009","journal-title":"Nucleic Acids Res."},{"key":"2023012512310690100_B32","doi-asserted-by":"crossref","first-page":"1278","DOI":"10.1038\/ng2135","article-title":"The role of site accessibility in microRNA target recognition","volume":"39","author":"Kertesz","year":"2007","journal-title":"Nat. Genet."},{"key":"2023012512310690100_B33","doi-asserted-by":"crossref","first-page":"3587","DOI":"10.1093\/bioinformatics\/bti565","article-title":"Ontological analysis of gene expression data: current tools, limitations, and open problems","volume":"21","author":"Khatri","year":"2005","journal-title":"Bioinformatics"},{"key":"2023012512310690100_B34","doi-asserted-by":"crossref","first-page":"495","DOI":"10.1038\/ng1536","article-title":"Combinatorial microRNA target predictions","volume":"37","author":"Krek","year":"2005","journal-title":"Nat. Genet."},{"key":"2023012512310690100_B35","volume-title":"Nonparametric Statistical Methods Based on Ranks.","author":"Lehmann","year":"1975"},{"key":"2023012512310690100_B36","doi-asserted-by":"crossref","first-page":"R93","DOI":"10.1186\/gb-2006-7-10-r93","article-title":"Pathway and gene-set activation measurement from mRNA expression data: the tissue distribution of human pathways","volume":"7","author":"Levine","year":"2006","journal-title":"Genome Biol."},{"issue":"Suppl. 11","key":"2023012512310690100_B37","doi-asserted-by":"crossref","first-page":"S5","DOI":"10.1186\/1471-2105-11-S11-S5","article-title":"TF-centered downstream gene set enrichment analysis: inference of causal regulators by integrating TF-DNA interactions and protein post-translational modifications information","volume":"11","author":"Liu","year":"2010","journal-title":"BMC Bioinformatics"},{"key":"2023012512310690100_B38","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1186\/1471-2105-7-113","article-title":"An improved map of conserved regulatory sites for Saccharomyces cerevisiae","volume":"7","author":"MacIsaac","year":"2006","journal-title":"BMC Bioinformatics"},{"key":"2023012512310690100_B39","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1214\/aoms\/1177730491","article-title":"On a test of whether one of two random variables is stochastically larger than the other","volume":"18","author":"Mann","year":"1947","journal-title":"Ann. Math. Stat."},{"key":"2023012512310690100_B40","doi-asserted-by":"crossref","first-page":"R101","DOI":"10.1186\/gb-2004-5-12-r101","article-title":"GOToolBox: functional analysis of gene datasets based on Gene Ontology","volume":"5","author":"Martin","year":"2004","journal-title":"Genome Biol."},{"key":"2023012512310690100_B41","doi-asserted-by":"crossref","first-page":"435","DOI":"10.1002\/bies.200800212","article-title":"The interplay between transcription factors and microRNAs in genome-scale regulatory networks","volume":"31","author":"Martinez","year":"2009","journal-title":"Bioessays"},{"key":"2023012512310690100_B42","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1186\/1752-0509-3-49","article-title":"Comparative analysis of module-based versus direct methods for reverse-engineering transcriptional regulatory networks","volume":"3","author":"Michoel","year":"2009","journal-title":"BMC Syst. Biol."},{"key":"2023012512310690100_B43","doi-asserted-by":"crossref","DOI":"10.1201\/b15236","volume-title":"Beyond ANOVA: Basics of Applied Statistics.","author":"Miller","year":"1997"},{"key":"2023012512310690100_B44","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1038\/ng1180","article-title":"PGC-1 alpha-responsive genes involved in oxidative phosphorylation are coordinately downregulated in human diabetes","volume":"34","author":"Mootha","year":"2003","journal-title":"Nat. Genet."},{"key":"2023012512310690100_B45","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/1471-2105-11-135","article-title":"miRSel: automated extraction of associations between microRNAs and genes from the biomedical literature","volume":"11","author":"Naeem","year":"2010","journal-title":"BMC Bioinformatics"},{"key":"2023012512310690100_B46","doi-asserted-by":"crossref","first-page":"e22519","DOI":"10.1371\/journal.pone.0022519","article-title":"MIRTFnet: analysis of miRNA regulated transcription factors","volume":"6","author":"Naeem","year":"2011","journal-title":"PLoS One"},{"key":"2023012512310690100_B47","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1093\/bib\/bbn001","article-title":"Gene-set approach for expression pattern analysis","volume":"9","author":"Nam","year":"2008","journal-title":"Brief Bioinform."},{"key":"2023012512310690100_B48","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1016\/j.ygeno.2010.10.003","article-title":"A comprehensive assessment of methods for de-novo reverse-engineering of genome-scale regulatory networks","volume":"97","author":"Narendra","year":"2011","journal-title":"Genomics"},{"key":"2023012512310690100_B49","doi-asserted-by":"crossref","first-page":"265","DOI":"10.2307\/2986126","article-title":"Algorithm AS 288: exact Smirnov two-sample tests for arbitrary distributions","volume":"43","author":"Nikiforov","year":"1994","journal-title":"Applied Statistics"},{"key":"2023012512310690100_B50","doi-asserted-by":"crossref","first-page":"e16250","DOI":"10.1371\/journal.pone.0016250","article-title":"MicroRNAs differentially expressed in postnatal aortic development downregulate elastin via 3' UTR and coding-sequence binding sites","volume":"6","author":"Ott","year":"2011","journal-title":"PLoS One"},{"key":"2023012512310690100_B51","doi-asserted-by":"crossref","first-page":"D155","DOI":"10.1093\/nar\/gkn809","article-title":"The database of experimentally supported targets: a functional update of TarBase","volume":"37","author":"Papadopoulos","year":"2009","journal-title":"Nucleic Acids Res."},{"key":"2023012512310690100_B52","doi-asserted-by":"crossref","first-page":"1213","DOI":"10.1023\/B:NERE.0000023608.29741.45","article-title":"Using the gene ontology for microarray data mining: a comparison of methods and application to age effects in human prefrontal cortex","volume":"29","author":"Pavlidis","year":"2004","journal-title":"Neurochem. Res."},{"key":"2023012512310690100_B53","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1186\/1471-2105-6-162","article-title":"Theme discovery from gene lists for identification and viewing of multiple functional groups","volume":"6","author":"Pehkonen","year":"2005","journal-title":"BMC Bioinformatics"},{"key":"2023012512310690100_B54","doi-asserted-by":"crossref","first-page":"e9202","DOI":"10.1371\/journal.pone.0009202","article-title":"Towards a rigorous assessment of systems biology models: the DREAM3 challenges","volume":"5","author":"Prill","year":"2010","journal-title":"PLoS One"},{"key":"2023012512310690100_B55","doi-asserted-by":"crossref","first-page":"397","DOI":"10.1038\/nmeth0609-397","article-title":"Predicting microRNA targets and functions: traps for the unwary","volume":"6","author":"Ritchie","year":"2009","journal-title":"Nat. Methods"},{"key":"2023012512310690100_B56","doi-asserted-by":"crossref","first-page":"401","DOI":"10.1093\/bioinformatics\/btl633","article-title":"Enrichment or depletion of a GO category within a class of genes: which test?","volume":"23","author":"Rivals","year":"2007","journal-title":"Bioinformatics"},{"key":"2023012512310690100_B57","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1038\/nature07228","article-title":"Widespread changes in protein synthesis induced by microRNAs","volume":"455","author":"Selbach","year":"2008","journal-title":"Nature"},{"key":"2023012512310690100_B58","volume-title":"Nonparametric Statistics for the Behavioral Sciences.","author":"Siegel","year":"1956"},{"key":"2023012512310690100_B59","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1093\/bioinformatics\/bti1120","article-title":"Identifying active transcription factors and kinases from expression data using pathway queries","volume":"21","author":"Sohler","year":"2005","journal-title":"Bioinformatics"},{"key":"2023012512310690100_B60","doi-asserted-by":"crossref","first-page":"2746","DOI":"10.1073\/pnas.0511045103","article-title":"Cell-type-specific signatures of microRNAs on target mRNA expression","volume":"103","author":"Sood","year":"2006","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"2023012512310690100_B61","first-page":"113","volume-title":"Theory and Problems of Probability and Statistics.","author":"Spiegel","year":"1992"},{"key":"2023012512310690100_B62","doi-asserted-by":"crossref","first-page":"15545","DOI":"10.1073\/pnas.0506580102","article-title":"Gene set enrichment analysis: a knowledge-based approach for interpreting genome-wide expression profiles","volume":"102","author":"Subramanian","year":"2005","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"2023012512310690100_B63","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1186\/1471-2105-10-307","article-title":"Robust extraction of functional signals from gene set analysis using a generalized threshold free scoring function","volume":"10","author":"T\u00f6r\u00f6nen","year":"2009","journal-title":"BMC Bioinformatics"},{"key":"2023012512310690100_B64","doi-asserted-by":"crossref","first-page":"5969","DOI":"10.1093\/nar\/gkp638","article-title":"Combinatorial network of primary and secondary microRNA-driven regulatory mechanisms","volume":"37","author":"Tu","year":"2009","journal-title":"Nucleic Acids Res."},{"key":"2023012512310690100_B65","doi-asserted-by":"crossref","first-page":"D105","DOI":"10.1093\/nar\/gkn851","article-title":"miRecords: an integrated resource for microRNA-target interactions","volume":"37","author":"Xiao","year":"2009","journal-title":"Nucleic Acids Res."},{"key":"2023012512310690100_B66","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1186\/1471-2105-7-30","article-title":"Wholepathwayscope: a comprehensive pathway-based analysis tool for high-throughput data","volume":"7","author":"Yi","year":"2006","journal-title":"BMC Bioinformatics"},{"key":"2023012512310690100_B67","doi-asserted-by":"crossref","first-page":"R28","DOI":"10.1186\/gb-2003-4-4-r28","article-title":"GoMiner: a resource for biological interpretation of genomic and proteomic data","volume":"4","author":"Zeeberg","year":"2003","journal-title":"Genome Biol."},{"key":"2023012512310690100_B68","first-page":"407","article-title":"Analysis of gene expression data with pathway scores","volume":"8","author":"Zien","year":"2000","journal-title":"Proc. Int. Conf. Intell. Syst. Mol. Biol."},{"key":"2023012512310690100_B69","doi-asserted-by":"crossref","first-page":"561","DOI":"10.1093\/clinchem\/39.4.561","article-title":"Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine","volume":"39","author":"Zweig","year":"1993","journal-title":"Clin. Chem."}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/28\/11\/1480\/48866976\/bioinformatics_28_11_1480.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/28\/11\/1480\/48866976\/bioinformatics_28_11_1480.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,25]],"date-time":"2023-01-25T10:59:40Z","timestamp":1674644380000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/28\/11\/1480\/266349"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2012,4,5]]},"references-count":69,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2012,6,1]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/bts164","relation":{"has-review":[{"id-type":"doi","id":"10.3410\/f.717950212.793455463","asserted-by":"object"}]},"ISSN":["1367-4811","1367-4803"],"issn-type":[{"value":"1367-4811","type":"electronic"},{"value":"1367-4803","type":"print"}],"subject":[],"published-other":{"date-parts":[[2012,6,1]]},"published":{"date-parts":[[2012,4,5]]}}}