{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,9]],"date-time":"2026-01-09T12:34:13Z","timestamp":1767962053124,"version":"3.49.0"},"reference-count":50,"publisher":"Oxford University Press (OUP)","issue":"18","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2008,9,15]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Motivation: Differentially expressed gene (DEG) lists detected from different microarray studies for a same disease are often highly inconsistent. Even in technical replicate tests using identical samples, DEG detection still shows very low reproducibility. It is often believed that current small microarray studies will largely introduce false discoveries.<\/jats:p><jats:p>Results: Based on a statistical model, we show that even in technical replicate tests using identical samples, it is highly likely that the selected DEG lists will be very inconsistent in the presence of small measurement variations. Therefore, the apparently low reproducibility of DEG detection from current technical replicate tests does not indicate low quality of microarray technology. We also demonstrate that heterogeneous biological variations existing in real cancer data will further reduce the overall reproducibility of DEG detection. Nevertheless, in small subsamples from both simulated and real data, the actual false discovery rate (FDR) for each DEG list tends to be low, suggesting that each separately determined list may comprise mostly true DEGs. Rather than simply counting the overlaps of the discovery lists from different studies for a complex disease, novel metrics are needed for evaluating the reproducibility of discoveries characterized with correlated molecular changes.<\/jats:p><jats:p>Contact: \u00a0guoz@ems.hrbmu.edu.cn; lixia@ems.hrbmu.edu.cn<\/jats:p><jats:p>Supplementaty information: \u00a0Supplementary data are available at Bioinformatics online.<\/jats:p>","DOI":"10.1093\/bioinformatics\/btn365","type":"journal-article","created":{"date-parts":[[2008,7,17]],"date-time":"2008-07-17T00:34:11Z","timestamp":1216254851000},"page":"2057-2063","source":"Crossref","is-referenced-by-count":104,"title":["Apparently low reproducibility of true differential expression discoveries in microarray studies"],"prefix":"10.1093","volume":"24","author":[{"given":"Min","family":"Zhang","sequence":"first","affiliation":[{"name":"1 School of Bioinformatics Science and Technology, Harbin Medical University, Harbin 150086 and 2Bioinformatics Centre and School of Life Science, University of Electronic Science and Technology of China, Chengdu 610054, China"}]},{"given":"Chen","family":"Yao","sequence":"additional","affiliation":[{"name":"1 School of Bioinformatics Science and Technology, Harbin Medical University, Harbin 150086 and 2Bioinformatics Centre and School of Life Science, University of Electronic Science and Technology of China, Chengdu 610054, China"}]},{"given":"Zheng","family":"Guo","sequence":"additional","affiliation":[{"name":"1 School of Bioinformatics Science and Technology, Harbin Medical University, Harbin 150086 and 2Bioinformatics Centre and School of Life Science, University of Electronic Science and Technology of China, Chengdu 610054, China"},{"name":"1 School of Bioinformatics Science and Technology, Harbin Medical University, Harbin 150086 and 2Bioinformatics Centre and School of Life Science, University of Electronic Science and Technology of China, Chengdu 610054, China"}]},{"given":"Jinfeng","family":"Zou","sequence":"additional","affiliation":[{"name":"1 School of Bioinformatics Science and Technology, Harbin Medical University, Harbin 150086 and 2Bioinformatics Centre and School of Life Science, University of Electronic Science and Technology of China, Chengdu 610054, China"}]},{"given":"Lin","family":"Zhang","sequence":"additional","affiliation":[{"name":"1 School of Bioinformatics Science and Technology, Harbin Medical University, Harbin 150086 and 2Bioinformatics Centre and School of Life Science, University of Electronic Science and Technology of China, Chengdu 610054, China"}]},{"given":"Hui","family":"Xiao","sequence":"additional","affiliation":[{"name":"1 School of Bioinformatics Science and Technology, Harbin Medical University, Harbin 150086 and 2Bioinformatics Centre and School of Life Science, University of Electronic Science and Technology of China, Chengdu 610054, China"}]},{"given":"Dong","family":"Wang","sequence":"additional","affiliation":[{"name":"1 School of Bioinformatics Science and Technology, Harbin Medical University, Harbin 150086 and 2Bioinformatics Centre and School of Life Science, University of Electronic Science and Technology of China, Chengdu 610054, China"}]},{"given":"Da","family":"Yang","sequence":"additional","affiliation":[{"name":"1 School of Bioinformatics Science and Technology, Harbin Medical University, Harbin 150086 and 2Bioinformatics Centre and School of Life Science, University of Electronic Science and Technology of China, Chengdu 610054, China"}]},{"given":"Xue","family":"Gong","sequence":"additional","affiliation":[{"name":"1 School of Bioinformatics Science and Technology, Harbin Medical University, Harbin 150086 and 2Bioinformatics Centre and School of Life Science, University of Electronic Science and Technology of China, Chengdu 610054, China"}]},{"given":"Jing","family":"Zhu","sequence":"additional","affiliation":[{"name":"1 School of Bioinformatics Science and Technology, Harbin Medical University, Harbin 150086 and 2Bioinformatics Centre and School of Life Science, University of Electronic Science and Technology of China, Chengdu 610054, China"}]},{"given":"Yanhui","family":"Li","sequence":"additional","affiliation":[{"name":"1 School of Bioinformatics Science and Technology, Harbin Medical University, Harbin 150086 and 2Bioinformatics Centre and School of Life Science, University of Electronic Science and Technology of China, Chengdu 610054, China"}]},{"given":"Xia","family":"Li","sequence":"additional","affiliation":[{"name":"1 School of Bioinformatics Science and Technology, Harbin Medical University, Harbin 150086 and 2Bioinformatics Centre and School of Life Science, University of Electronic Science and Technology of China, Chengdu 610054, China"}]}],"member":"286","published-online":{"date-parts":[[2008,7,16]]},"reference":[{"key":"2023020211101141100_B1","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1111\/j.2517-6161.1995.tb02031.x","article-title":"Controlling the false discovery rate: a practical and powerful approach to multiple testing","volume":"57","author":"Benjamini","year":"1995","journal-title":"J. R. Stat. Soc. B Met."},{"key":"2023020211101141100_B2","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1007\/s11306-006-0037-z","article-title":"Statistical strategies for avoiding false discoveries in metabolomics and related experiments","volume":"2","author":"Broadhurst","year":"2006","journal-title":"Metabolomics"},{"key":"2023020211101141100_B3","doi-asserted-by":"crossref","first-page":"412","DOI":"10.1186\/1471-2105-8-412","article-title":"Reproducibility of microarray data: a further analysis of microarray quality control (MAQC) data","volume":"8","author":"Chen","year":"2007","journal-title":"BMC Bioinformatics"},{"key":"2023020211101141100_B4","doi-asserted-by":"crossref","first-page":"1929","DOI":"10.1091\/mbc.02-02-0023","article-title":"Gene expression patterns in human liver cancers","volume":"13","author":"Chen","year":"2002","journal-title":"Mol. Biol. Cell"},{"key":"2023020211101141100_B5","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1093\/nar\/gkg014","article-title":"SOURCE: a unified genomic resource of functional annotations, ontologies, and gene expression data","volume":"31","author":"Diehn","year":"2003","journal-title":"Nucleic Acids Res."},{"key":"2023020211101141100_B6","doi-asserted-by":"crossref","first-page":"254","DOI":"10.1016\/S1016-8478(23)17418-8","article-title":"Normalization of microarray data: single-labeled and dual-labeled arrays","volume":"22","author":"Do","year":"2006","journal-title":"Mol. Cells"},{"key":"2023020211101141100_B7","doi-asserted-by":"crossref","first-page":"565","DOI":"10.1158\/1078-0432.565.11.2","article-title":"Interlaboratory comparability study of cancer gene expression analysis using oligonucleotide microarrays","volume":"11","author":"Dobbin","year":"2005","journal-title":"Clin. Cancer Res."},{"key":"2023020211101141100_B8","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1093\/bioinformatics\/bth469","article-title":"Outcome signature genes in breast cancer: is there a unique set?","volume":"21","author":"Ein-Dor","year":"2005","journal-title":"Bioinformatics"},{"key":"2023020211101141100_B9","doi-asserted-by":"crossref","first-page":"5923","DOI":"10.1073\/pnas.0601231103","article-title":"Thousands of samples are needed to generate a robust gene list for predicting outcome in cancer","volume":"103","author":"Ein-Dor","year":"2006","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"2023020211101141100_B10","doi-asserted-by":"crossref","first-page":"362","DOI":"10.1038\/nrd1746","article-title":"An array of problems","volume":"4","author":"Frantz","year":"2005","journal-title":"Nat. Rev. Drug Discov."},{"key":"2023020211101141100_B11","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1093\/bioinformatics\/btg405","article-title":"affy\u2013analysis of Affymetrix GeneChip data at the probe level","volume":"20","author":"Gautier","year":"2004","journal-title":"Bioinformatics"},{"key":"2023020211101141100_B12","doi-asserted-by":"crossref","first-page":"1162","DOI":"10.1038\/nbt1238","article-title":"Rat toxicogenomic study reveals analytical consistency across microarray platforms","volume":"24","author":"Guo","year":"2006","journal-title":"Nat. Biotechnol."},{"key":"2023020211101141100_B13","doi-asserted-by":"crossref","first-page":"2121","DOI":"10.1093\/bioinformatics\/btm294","article-title":"Edge-based scoring and searching method for identifying condition-responsive protein-protein interaction sub-network","volume":"23","author":"Guo","year":"2007","journal-title":"Bioinformatics"},{"key":"2023020211101141100_B14","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1186\/1471-2105-6-58","article-title":"Towards precise classification of cancers based on robust gene functional expression profiles","volume":"6","author":"Guo","year":"2005","journal-title":"BMC Bioinformatics"},{"key":"2023020211101141100_B15","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1038\/nbt0108-69","article-title":"Protein-protein interaction networks and biology-what's the connection?","volume":"26","author":"Hakes","year":"2008","journal-title":"Nat. Biotechnol."},{"key":"2023020211101141100_B16","doi-asserted-by":"crossref","first-page":"R70","DOI":"10.1186\/gb-2003-4-10-r70","article-title":"Identifying biological themes within lists of genes with EASE","volume":"4","author":"Hosack","year":"2003","journal-title":"Genome Biol."},{"key":"2023020211101141100_B17","doi-asserted-by":"crossref","first-page":"345","DOI":"10.1038\/nmeth756","article-title":"Multiple-laboratory comparison of microarray platforms","volume":"2","author":"Irizarry","year":"2005","journal-title":"Nat. Methods"},{"key":"2023020211101141100_B18","doi-asserted-by":"crossref","DOI":"10.2202\/1544-6115.1189","article-title":"A new type of stochastic dependence revealed in gene expression data","volume":"5","author":"Klebanov","year":"2006","journal-title":"Stat. Appl. Genet. Mol. Biol."},{"key":"2023020211101141100_B19","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1038\/nbt0107-25","article-title":"Statistical methods and microarray data","volume":"25","author":"Klebanov","year":"2007","journal-title":"Nat. Biotechnol."},{"key":"2023020211101141100_B20","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1186\/1745-6150-2-9","article-title":"How high is the level of technical noise in microarray data?","volume":"2","author":"Klebanov","year":"2007","journal-title":"Biol. Direct"},{"key":"2023020211101141100_B21","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1038\/4427","article-title":"Array of hope","volume":"21","author":"Lander","year":"1999","journal-title":"Nat. Genet."},{"key":"2023020211101141100_B22","doi-asserted-by":"crossref","first-page":"811","DOI":"10.1073\/pnas.0304146101","article-title":"Gene expression profiling identifies clinically relevant subtypes of prostate cancer","volume":"101","author":"Lapointe","year":"2004","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"2023020211101141100_B23","doi-asserted-by":"crossref","first-page":"2685","DOI":"10.1093\/nar\/gkh563","article-title":"Gene mining: a novel and powerful ensemble decision approach to hunting for disease genes using microarray expression profiling","volume":"32","author":"Li","year":"2004","journal-title":"Nucleic Acids Res."},{"key":"2023020211101141100_B24","doi-asserted-by":"crossref","first-page":"630","DOI":"10.1126\/science.306.5696.630","article-title":"Getting the noise out of gene arrays","volume":"306","author":"Marshall","year":"2004","journal-title":"Science"},{"key":"2023020211101141100_B25","doi-asserted-by":"crossref","first-page":"615","DOI":"10.1038\/nbt965","article-title":"Microarray reality checks in the context of a complex disease","volume":"22","author":"Miklos","year":"2004","journal-title":"Nat. Biotechnol."},{"key":"2023020211101141100_B26","doi-asserted-by":"crossref","first-page":"1620","DOI":"10.1093\/bioinformatics\/btg227","article-title":"The effect of replication on gene expression microarray experiments","volume":"19","author":"Pavlidis","year":"2003","journal-title":"Bioinformatics"},{"key":"2023020211101141100_B27","doi-asserted-by":"crossref","first-page":"3017","DOI":"10.1093\/bioinformatics\/bti448","article-title":"False discovery rate, sensitivity and sample size for microarray studies","volume":"21","author":"Pawitan","year":"2005","journal-title":"Bioinformatics"},{"key":"2023020211101141100_B28","doi-asserted-by":"crossref","first-page":"3865","DOI":"10.1093\/bioinformatics\/bti626","article-title":"Bias in the estimation of false discovery rate in microarray studies","volume":"21","author":"Pawitan","year":"2005","journal-title":"Bioinformatics"},{"key":"2023020211101141100_B29","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1186\/1471-2105-8-28","article-title":"Detecting differential expression in microarray data: comparison of optimal procedures","volume":"8","author":"Perelman","year":"2007","journal-title":"BMC Bioinformatics"},{"key":"2023020211101141100_B30","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1186\/1471-2105-7-50","article-title":"Assessing stability of gene selection in microarray data analysis","volume":"7","author":"Qiu","year":"2006","journal-title":"BMC Bioinformatics"},{"issue":"Suppl","key":"2023020211101141100_B31","doi-asserted-by":"crossref","first-page":"496","DOI":"10.1038\/ng1032","article-title":"Microarray data normalization and transformation","volume":"32","author":"Quackenbush","year":"2002","journal-title":"Nat. Genet."},{"key":"2023020211101141100_B32","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1038\/nrc1322","article-title":"Rules of evidence for cancer molecular-marker discovery and validation","volume":"4","author":"Ransohoff","year":"2004","journal-title":"Nat. Rev. Cancer"},{"key":"2023020211101141100_B33","doi-asserted-by":"crossref","first-page":"142","DOI":"10.1038\/nrc1550","article-title":"Bias as a threat to the validity of cancer molecular-marker research","volume":"5","author":"Ransohoff","year":"2005","journal-title":"Nat. Rev. Cancer"},{"key":"2023020211101141100_B34","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1093\/jnci\/dji054","article-title":"Lessons from controversy: ovarian cancer screening and serum proteomics","volume":"97","author":"Ransohoff","year":"2005","journal-title":"J. Natl Cancer Inst."},{"key":"2023020211101141100_B35","doi-asserted-by":"crossref","first-page":"166","DOI":"10.1593\/neo.07112","article-title":"Oncomine 3.0: genes, pathways, and networks in a collection of 18,000 cancer gene expression profiles","volume":"9","author":"Rhodes","year":"2007","journal-title":"Neoplasia"},{"issue":"Suppl. 2","key":"2023020211101141100_B36","doi-asserted-by":"crossref","first-page":"S12","DOI":"10.1186\/1471-2105-6-S2-S12","article-title":"Cross-platform comparability of microarray technology: intra-platform consistency and appropriate data analysis procedures are essential","volume":"6","author":"Shi","year":"2005","journal-title":"BMC Bioinformatics"},{"key":"2023020211101141100_B37","doi-asserted-by":"crossref","first-page":"1151","DOI":"10.1038\/nbt1239","article-title":"The MicroArray Quality Control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements","volume":"24","author":"Shi","year":"2006","journal-title":"Nat. Biotechnol."},{"key":"2023020211101141100_B38","doi-asserted-by":"crossref","first-page":"15545","DOI":"10.1073\/pnas.0506580102","article-title":"Gene set enrichment analysis: a knowledge-based approach for interpreting genome-wide expression profiles","volume":"102","author":"Subramanian","year":"2005","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"2023020211101141100_B39","doi-asserted-by":"crossref","first-page":"5676","DOI":"10.1093\/nar\/gkg763","article-title":"Evaluation of gene expression measurements from commercial microarray platforms","volume":"31","author":"Tan","year":"2003","journal-title":"Nucleic Acids Res."},{"key":"2023020211101141100_B40","doi-asserted-by":"crossref","first-page":"1132","DOI":"10.1038\/nbt1237","article-title":"Evaluation of external RNA controls for the assessment of microarray performance","volume":"24","author":"Tong","year":"2006","journal-title":"Nat. Biotechnol."},{"key":"2023020211101141100_B41","doi-asserted-by":"crossref","first-page":"520","DOI":"10.1093\/bioinformatics\/17.6.520","article-title":"Missing value estimation methods for DNA microarrays","volume":"17","author":"Troyanskaya","year":"2001","journal-title":"Bioinformatics"},{"key":"2023020211101141100_B42","doi-asserted-by":"crossref","first-page":"5116","DOI":"10.1073\/pnas.091062498","article-title":"Significance analysis of microarrays applied to the ionizing radiation response","volume":"98","author":"Tusher","year":"2001","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"2023020211101141100_B43","doi-asserted-by":"crossref","first-page":"4280","DOI":"10.1093\/bioinformatics\/bti685","article-title":"A note on using permutation-based false discovery rate estimates to compare different analysis methods for microarray data","volume":"21","author":"Xie","year":"2005","journal-title":"Bioinformatics"},{"key":"2023020211101141100_B44","doi-asserted-by":"crossref","first-page":"25","DOI":"10.2119\/2005-00036.Xu","article-title":"Peeling off the hidden genetic heterogeneities of cancers based on disease-relevant functional modules","volume":"12","author":"Xu","year":"2006","journal-title":"Mol. Med."},{"key":"2023020211101141100_B45","doi-asserted-by":"crossref","first-page":"1284","DOI":"10.1093\/bioinformatics\/btg155","article-title":"A comparison of parametric versus permutation methods with applications to general and temporal microarray gene expression data","volume":"19","author":"Xu","year":"2003","journal-title":"Bioinformatics"},{"key":"2023020211101141100_B46","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1093\/bioinformatics\/btm558","article-title":"Gaining confidence in biological interpretation of the microarray data: the functional consistence of the significant GO categories","volume":"24","author":"Yang","year":"2008","journal-title":"Bioinformatics"},{"key":"2023020211101141100_B47","doi-asserted-by":"crossref","first-page":"e15","DOI":"10.1093\/nar\/30.4.e15","article-title":"Normalization for cDNA microarray data: a robust composite method addressing single and multiple slide systematic variation","volume":"30","author":"Yang","year":"2002","journal-title":"Nucleic Acids Res."},{"key":"2023020211101141100_B48","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1016\/S1535-6108(02)00032-6","article-title":"Classification, subtype discovery, and prediction of outcome in pediatric acute lymphoblastic leukemia by gene expression profiling","volume":"1","author":"Yeoh","year":"2002","journal-title":"Cancer Cell"},{"key":"2023020211101141100_B49","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1186\/1471-2105-8-230","article-title":"A comprehensive evaluation of SAM, the SAM R-package and a simple modification to improve its performance","volume":"8","author":"Zhang","year":"2007","journal-title":"BMC Bioinformatics"},{"key":"2023020211101141100_B50","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1186\/1471-2164-8-30","article-title":"GO-2D: identifying 2-dimensional cellular-localized functional modules in Gene Ontology","volume":"8","author":"Zhu","year":"2007","journal-title":"BMC Genomics"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/24\/18\/2057\/49050204\/bioinformatics_24_18_2057.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/24\/18\/2057\/49050204\/bioinformatics_24_18_2057.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,31]],"date-time":"2025-01-31T01:38:58Z","timestamp":1738287538000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/24\/18\/2057\/191118"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2008,7,16]]},"references-count":50,"journal-issue":{"issue":"18","published-print":{"date-parts":[[2008,9,15]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btn365","relation":{},"ISSN":["1367-4811","1367-4803"],"issn-type":[{"value":"1367-4811","type":"electronic"},{"value":"1367-4803","type":"print"}],"subject":[],"published-other":{"date-parts":[[2008,9,15]]},"published":{"date-parts":[[2008,7,16]]}}}