{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,20]],"date-time":"2026-04-20T10:26:43Z","timestamp":1776680803369,"version":"3.51.2"},"reference-count":44,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2018,9,18]],"date-time":"2018-09-18T00:00:00Z","timestamp":1537228800000},"content-version":"vor","delay-in-days":1,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000054","name":"National Cancer Institute","doi-asserted-by":"publisher","award":["R01CA158472"],"award-info":[{"award-number":["R01CA158472"]}],"id":[{"id":"10.13039\/100000054","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000054","name":"National Cancer Institute","doi-asserted-by":"publisher","award":["NIH\/NCI R01 CA200987"],"award-info":[{"award-number":["NIH\/NCI R01 CA200987"]}],"id":[{"id":"10.13039\/100000054","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000054","name":"National Cancer Institute","doi-asserted-by":"publisher","award":["NIH\/NCI U24 CA210954"],"award-info":[{"award-number":["NIH\/NCI U24 CA210954"]}],"id":[{"id":"10.13039\/100000054","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2019,11,27]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Epigenome-wide association studies (EWASs) have become increasingly popular for studying DNA methylation (DNAm) variations in complex diseases. The Illumina methylation arrays provide an economical, high-throughput and comprehensive platform for measuring methylation status in EWASs. A number of software tools have been developed for identifying disease-associated differentially methylated regions (DMRs) in the epigenome. However, in practice, we found these tools typically had multiple parameter settings that needed to be specified and the performance of the software tools under different parameters was often unclear. To help users better understand and choose optimal parameter settings when using DNAm analysis tools, we conducted a comprehensive evaluation of 4 popular DMR analysis tools under 60 different parameter settings. In addition to evaluating power, precision, area under precision-recall curve, Matthews correlation coefficient, F1 score and type I error rate, we also compared several additional characteristics of the analysis results, including the size of the DMRs, overlap between the methods and execution time. The results showed that none of the software tools performed best under their default parameter settings, and power varied widely when parameters were changed. Overall, the precision of these software tools were good. In contrast, all methods lacked power when effect size was consistent but small. Across all simulation scenarios, comb-p consistently had the best sensitivity as well as good control of false-positive rate.<\/jats:p>","DOI":"10.1093\/bib\/bby085","type":"journal-article","created":{"date-parts":[[2018,8,24]],"date-time":"2018-08-24T19:12:21Z","timestamp":1535137941000},"page":"2224-2235","source":"Crossref","is-referenced-by-count":99,"title":["An evaluation of supervised methods for identifying differentially methylated regions in Illumina methylation arrays"],"prefix":"10.1093","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4107-6784","authenticated-orcid":false,"given":"Saurav","family":"Mallik","sequence":"first","affiliation":[{"name":"Division of Biostatistics, Department of Public Health Sciences, University of Miami, Miller School of Medicine, Miami, FL, USA"},{"name":"Joint First Authors"}]},{"given":"Gabriel J","family":"Odom","sequence":"first","affiliation":[{"name":"Division of Biostatistics, Department of Public Health Sciences, University of Miami, Miller School of Medicine, Miami, FL, USA"},{"name":"Joint First Authors"}]},{"given":"Zhen","family":"Gao","sequence":"first","affiliation":[{"name":"Sylvester Comprehensive Cancer Center, University of Miami Miller School of Medicine, Miami, FL, USA"}]},{"given":"Lissette","family":"Gomez","sequence":"first","affiliation":[{"name":"Dr. John T. Macdonald Foundation, Department of Human Genetics, and John P. Hussman Institute for Human Genomics, University of Miami, Miami, FL, USA"}]},{"given":"Xi","family":"Chen","sequence":"first","affiliation":[{"name":"Division of Biostatistics, Department of Public Health Sciences, University of Miami, Miller School of Medicine, Miami, FL, USA"},{"name":"Sylvester Comprehensive Cancer Center, University of Miami Miller School of Medicine, Miami, FL, USA"}]},{"given":"Lily","family":"Wang","sequence":"first","affiliation":[{"name":"Division of Biostatistics, Department of Public Health Sciences, University of Miami, Miller School of Medicine, Miami, FL, USA"},{"name":"Sylvester Comprehensive Cancer Center, University of Miami Miller School of Medicine, Miami, FL, USA"},{"name":"Dr. John T. Macdonald Foundation, Department of Human Genetics, and John P. Hussman Institute for Human Genomics, University of Miami, Miami, FL, USA"}]}],"member":"286","published-online":{"date-parts":[[2018,9,17]]},"reference":[{"key":"2020060321500282500_ref1","doi-asserted-by":"crossref","first-page":"1057","DOI":"10.1038\/nbt.1685","article-title":"Epigenetic modifications and human disease","volume":"28","author":"Portela","year":"2010","journal-title":"Nat Biotechnol"},{"key":"2020060321500282500_ref2","doi-asserted-by":"crossref","first-page":"288","DOI":"10.1016\/j.ygeno.2011.07.007","article-title":"High density DNA methylation array with single CpG site reslution","volume":"98","author":"Bibikova","year":"2011","journal-title":"Genomics"},{"key":"2020060321500282500_ref3","doi-asserted-by":"crossref","first-page":"178","DOI":"10.1038\/ng.298","article-title":"The human colon cancer methylome shows similar hypo- and hypermethylation at conserved tissue-specific CpG island shores","volume":"41","author":"Irizarry","year":"2009","journal-title":"Nat Genet"},{"key":"2020060321500282500_ref4","doi-asserted-by":"crossref","first-page":"389","DOI":"10.2217\/epi.15.114","article-title":"Validation of a DNA methylation microarray for 850,000 CpG sites of the human genome enriched in enhancer sequences","volume":"8","author":"Moran","year":"2016","journal-title":"Epigenomics"},{"key":"2020060321500282500_ref5","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1038\/nature11247","article-title":"An integrated encyclopedia of DNA elements in the human genome","volume":"489","author":"ENCODE Project Consortium","year":"2012","journal-title":"Nature"},{"key":"2020060321500282500_ref6","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1111\/joim.12231","article-title":"Epigenetics, chromatin and genome organization: recent advances from the ENCODE project","volume":"276","author":"Siggens","year":"2014","journal-title":"J Intern Med"},{"key":"2020060321500282500_ref7","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1186\/s13059-014-0560-6","article-title":"Gateways to the FANTOM5 promoter level mammalian expression atlas","volume":"16","author":"Lizio","year":"2015","journal-title":"Genome Biol"},{"key":"2020060321500282500_ref8","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nrg3000","article-title":"Epigenome-wide association studies for common human diseases","volume":"12","author":"Rakyan","year":"2011","journal-title":"Nat Rev Genet"},{"key":"2020060321500282500_ref9","doi-asserted-by":"crossref","first-page":"1156","DOI":"10.1038\/nn.3786","article-title":"Alzheimer's disease: early alterations in brain DNA methylation at ANK1, BIN1, RHBDF2 and other loci","volume":"17","author":"De Jager","year":"2014","journal-title":"Nat Neurosci"},{"key":"2020060321500282500_ref10","first-page":"310","article-title":"Epigenetics and epigenetic alterations in pancreatic cancer","volume":"2","author":"Omura","year":"2009","journal-title":"Int J Clin Exp Pathol"},{"key":"2020060321500282500_ref11","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1016\/j.gene.2017.12.033","article-title":"The role of DNA methylation in coronary artery disease","volume":"646","author":"Duan","year":"2018","journal-title":"Gene"},{"key":"2020060321500282500_ref12","doi-asserted-by":"crossref","first-page":"686","DOI":"10.1038\/nrgastro.2011.173","article-title":"Epigenetics and colorectal cancer","volume":"8","author":"Lao","year":"2011","journal-title":"Nat Rev Gastroenterol Hepatol"},{"key":"2020060321500282500_ref13","doi-asserted-by":"crossref","first-page":"862","DOI":"10.1038\/mp.2013.114","article-title":"Common DNA methylation alterations in multiple brain regions in autism","volume":"19","author":"Ladd-Acosta","year":"2014","journal-title":"Mol Psychiatry"},{"key":"2020060321500282500_ref14","doi-asserted-by":"crossref","first-page":"142","DOI":"10.1038\/nbt.2487","article-title":"Epigenome-wide association data implicate DNA methylation as an intermediary of genetic risk in rheumatoid arthritis","volume":"31","author":"Liu","year":"2013","journal-title":"Nat Biotechnol"},{"key":"2020060321500282500_ref15","doi-asserted-by":"crossref","first-page":"200","DOI":"10.1093\/ije\/dyr238","article-title":"Bump hunting to identify differentially methylated regions in epigenetic epidemiology studies","volume":"41","author":"Jaffe","year":"2012","journal-title":"Int J Epidemiol"},{"key":"2020060321500282500_ref16","doi-asserted-by":"crossref","first-page":"2986","DOI":"10.1093\/bioinformatics\/bts545","article-title":"Comb-p: software for combining, analyzing, grouping and correcting spatially correlated P-values","volume":"28","author":"Pedersen","year":"2012","journal-title":"Bioinformatics"},{"key":"2020060321500282500_ref17","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1186\/1756-8935-8-6","article-title":"De novo identification of differentially methylated regions in the human genome","volume":"8","author":"Peters","year":"2015","journal-title":"Epigenetics Chromatin"},{"key":"2020060321500282500_ref18","doi-asserted-by":"crossref","first-page":"729","DOI":"10.1093\/bioinformatics\/bts013","article-title":"IMA: an R package for high-throughput analysis of Illumina's 450K Infinium methylation data","volume":"28","author":"Wang","year":"2012","journal-title":"Bioinformatics"},{"key":"2020060321500282500_ref19","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1016\/j.ymeth.2014.10.036","article-title":"Probe Lasso: a novel method to rope in differentially methylated regions with 450K DNA methylation data","volume":"72","author":"Butcher","year":"2015","journal-title":"Methods"},{"key":"2020060321500282500_ref20","doi-asserted-by":"crossref","first-page":"e0156895","DOI":"10.1371\/journal.pone.0156895","article-title":"A comparative study of five association tests based on CpG set for epigenome-wide association studies","volume":"11","author":"Zhang","year":"2016","journal-title":"PLoS One"},{"key":"2020060321500282500_ref21","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1186\/s12859-015-0641-x","article-title":"An evaluation of statistical methods for DNA methylation microarray data analysis","volume":"16","author":"Li","year":"2015","journal-title":"BMC Bioinformatics"},{"key":"2020060321500282500_ref22","doi-asserted-by":"crossref","first-page":"2884","DOI":"10.1093\/bioinformatics\/btt498","article-title":"A-clustering: a novel method for the detection of co-regulated methylation regions, and regions associated with exposure","volume":"29","author":"Sofer","year":"2013","journal-title":"Bioinformatics"},{"key":"2020060321500282500_ref23","doi-asserted-by":"crossref","first-page":"587","DOI":"10.1186\/1471-2105-11-587","article-title":"Comparison of Beta-value and M-value methods for quantifying methylation levels by microarray analysis","volume":"11","author":"Du","year":"2010","journal-title":"BMC Bioinformatics"},{"key":"2020060321500282500_ref24","doi-asserted-by":"crossref","first-page":"110","DOI":"10.2307\/3002019","article-title":"An approximate distribution of estimates of variance components","volume":"2","author":"Satterthwaite","year":"1946","journal-title":"Biometrics"},{"key":"2020060321500282500_ref25","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1111\/j.2517-6161.1995.tb02031.x","article-title":"Controlling the false discovery rate: a practical and powerful approach to multiple testing","volume":"57","author":"Benjamini","year":"1995","journal-title":"Journal of the Royal Statistical Society, Series B"},{"key":"2020060321500282500_ref26","volume-title":"The American Soldier: Adjustment During Army Life.","author":"Riley","year":"1949"},{"key":"2020060321500282500_ref27","doi-asserted-by":"crossref","first-page":"29","DOI":"10.2202\/1544-6115.1434","article-title":"Generalizing moving averages for tiling arrays using combined p-value statistic","volume":"9","author":"Kechris","year":"2010","journal-title":"Stat Appl Genet Mol Biol"},{"key":"2020060321500282500_ref28","doi-asserted-by":"crossref","first-page":"1113","DOI":"10.1038\/ng.2764","article-title":"The Cancer Genome Atlas Pan-Cancer analysis project","volume":"45","author":"Cancer Genome Atlas Research Network","year":"2013","journal-title":"Nat Genet"},{"key":"2020060321500282500_ref29","doi-asserted-by":"crossref","first-page":"711","DOI":"10.1016\/j.ccell.2016.04.001","article-title":"Comprehensive characterization of molecular differences in cancer between male and female patients","volume":"29","author":"Yuan","year":"2016","journal-title":"Cancer Cell"},{"key":"2020060321500282500_ref30","doi-asserted-by":"crossref","first-page":"585","DOI":"10.1038\/nrg3405","article-title":"From promises to practical strategies in epigenetic epidemiology","volume":"14","author":"Mill","year":"2013","journal-title":"Nat Rev Genet"},{"key":"2020060321500282500_ref31","doi-asserted-by":"crossref","first-page":"e1002629","DOI":"10.1371\/journal.pgen.1002629","article-title":"Epigenome-wide scans identify differentially methylated regions for age and age-related phenotypes in a healthy ageing population","volume":"8","author":"Bell","year":"2012","journal-title":"PLoS Genet"},{"key":"2020060321500282500_ref32","doi-asserted-by":"crossref","first-page":"530","DOI":"10.1080\/07853890600989211","article-title":"Complex disease, gender and epigenetics","volume":"38","author":"Kaminsky","year":"2006","journal-title":"Ann Med"},{"key":"2020060321500282500_ref33","doi-asserted-by":"crossref","first-page":"1425","DOI":"10.1289\/ehp.1205412","article-title":"450K epigenome-wide scan identifies differential DNA methylation in newborns related to maternal smoking during pregnancy","volume":"120","author":"Joubert","year":"2012","journal-title":"Environ Health Perspect"},{"key":"2020060321500282500_ref34","doi-asserted-by":"crossref","first-page":"R31","DOI":"10.1186\/gb-2014-15-2-r31","article-title":"Accounting for cellular heterogeneity is critical in epigenome-wide association studies","volume":"15","author":"Jaffe","year":"2014","journal-title":"Genome Biol"},{"key":"2020060321500282500_ref35","doi-asserted-by":"crossref","first-page":"R43","DOI":"10.1186\/gb-2012-13-6-r43","article-title":"Functional annotation of the human brain methylome identifies tissue-specific epigenetic variation across brain and blood","volume":"13","author":"Davies","year":"2012","journal-title":"Genome Biol"},{"key":"2020060321500282500_ref36","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1186\/s13148-017-0433-4","article-title":"Exposure to NO2, CO, and PM2.5 is linked to regional DNA methylation differences in asthma","volume":"10","author":"Prunicki","year":"2018","journal-title":"Clin Epigenetics"},{"key":"2020060321500282500_ref37","doi-asserted-by":"crossref","DOI":"10.2202\/1544-6115.1484","article-title":"A unified mixed effects model for gene set analysis of time course microarray experiments","volume":"8","author":"Wang","year":"2009","journal-title":"Stat Appl Genet Mol Biol"},{"key":"2020060321500282500_ref38","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1198\/016214505000000628","article-title":"Prediction by supervised principal components","volume":"101","author":"Bair","year":"2006","journal-title":"J Am Stat Assoc"},{"key":"2020060321500282500_ref39","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.ygeno.2011.04.006","article-title":"Gene set analysis of genome-wide association studies: methodological issues and perspectives","volume":"98","author":"Wang","year":"2011","journal-title":"Genomics"},{"key":"2020060321500282500_ref40","doi-asserted-by":"crossref","first-page":"686","DOI":"10.1093\/bioinformatics\/btq728","article-title":"An efficient hierarchical generalized linear mixed model for pathway analysis of genome-wide association studies","volume":"27","author":"Wang","year":"2011","journal-title":"Bioinformatics"},{"key":"2020060321500282500_ref41","doi-asserted-by":"crossref","first-page":"843","DOI":"10.1038\/nrg2884","article-title":"Analysing biological pathways in genome-wide association studies","volume":"11","author":"Wang","year":"2010","journal-title":"Nat Rev Genet"},{"key":"2020060321500282500_ref42","doi-asserted-by":"crossref","first-page":"667","DOI":"10.1016\/j.ajhg.2013.03.022","article-title":"Sherlock: detecting gene-disease associations by matching patterns of expression QTL and GWAS","volume":"92","author":"He","year":"2013","journal-title":"Am J Hum Genet"},{"key":"2020060321500282500_ref43","doi-asserted-by":"crossref","first-page":"1091","DOI":"10.1038\/ng.3367","article-title":"A gene-based association method for mapping traits using reference transcriptome data","volume":"47","author":"Gamazon","year":"2015","journal-title":"Nat Genet"},{"key":"2020060321500282500_ref44","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1038\/ng.3506","volume":"48","author":"Gusev","year":"2016","journal-title":"Nat Genet"}],"container-title":["Briefings in Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/bib\/article-pdf\/20\/6\/2224\/33324841\/bby085.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/bib\/article-pdf\/20\/6\/2224\/33324841\/bby085.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,9]],"date-time":"2024-07-09T16:01:34Z","timestamp":1720540894000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bib\/article\/20\/6\/2224\/5096828"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,9,17]]},"references-count":44,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2018,9,17]]},"published-print":{"date-parts":[[2019,11,27]]}},"URL":"https:\/\/doi.org\/10.1093\/bib\/bby085","relation":{},"ISSN":["1467-5463","1477-4054"],"issn-type":[{"value":"1467-5463","type":"print"},{"value":"1477-4054","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2019,11]]},"published":{"date-parts":[[2018,9,17]]}}}