{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T17:54:42Z","timestamp":1775066082040,"version":"3.50.1"},"reference-count":43,"publisher":"Oxford University Press (OUP)","issue":"5","license":[{"start":{"date-parts":[[2022,7,25]],"date-time":"2022-07-25T00:00:00Z","timestamp":1658707200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["CA121974"],"award-info":[{"award-number":["CA121974"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["CA196530"],"award-info":[{"award-number":["CA196530"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,9,20]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In biomedical research, the replicability of findings across studies is highly desired. In this study, we focus on cancer omics data, for which the examination of replicability has been mostly focused on important omics variables identified in different studies. In published literature, although there have been extensive attention and ad hoc discussions, there is insufficient quantitative research looking into replicability measures and their properties. The goal of this study is to fill this important knowledge gap. In particular, we consider three sensible replicability measures, for which we examine distributional properties and develop a way of making inference. Applying them to three The Cancer Genome Atlas (TCGA) datasets reveals in general low replicability and significant across-data variations. To further comprehend such findings, we resort to simulation, which confirms the validity of the findings with the TCGA data and further informs the dependence of replicability on signal level (or equivalently sample size). Overall, this study can advance our understanding of replicability for cancer omics and other studies that have identification as a key goal.<\/jats:p>","DOI":"10.1093\/bib\/bbac304","type":"journal-article","created":{"date-parts":[[2022,7,25]],"date-time":"2022-07-25T11:19:02Z","timestamp":1658747942000},"source":"Crossref","is-referenced-by-count":7,"title":["Replicability in cancer omics data analysis: measures and empirical explorations"],"prefix":"10.1093","volume":"23","author":[{"given":"Jiping","family":"Wang","sequence":"first","affiliation":[{"name":"Department of Biostatistics, Yale School of Public Health , New Haven, CT, USA"}]},{"given":"Hongmin","family":"Liang","sequence":"additional","affiliation":[{"name":"Department of Statistics, School of Economics, Xiamen University , Xiamen, Fujian, China"}]},{"given":"Qingzhao","family":"Zhang","sequence":"additional","affiliation":[{"name":"Department of Statistics, School of Economics, Xiamen University , Xiamen, Fujian, China"},{"name":"The Wang Yanan Institute for Studies in Economics, Xiamen University , Xiamen, Fujian, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9001-4999","authenticated-orcid":false,"given":"Shuangge","family":"Ma","sequence":"additional","affiliation":[{"name":"Department of Biostatistics, Yale School of Public Health , New Haven, CT, USA"}]}],"member":"286","published-online":{"date-parts":[[2022,7,25]]},"reference":[{"key":"2022092013221839500_ref1","volume-title":"National Academies of Sciences, Engineering, and Medicine. Reproducibility and replicability in science","year":"2019"},{"key":"2022092013221839500_ref2","article-title":"Estimating the reproducibility of psychological science","volume":"349","author":"Collaboration","year":"2015","journal-title":"Science"},{"key":"2022092013221839500_ref3","doi-asserted-by":"crossref","DOI":"10.1126\/scitranslmed.aaf5027","article-title":"What does research reproducibility mean?","volume":"8","author":"Goodman","year":"2016","journal-title":"Sci Transl Med"},{"key":"2022092013221839500_ref4","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pmed.0020124","article-title":"Why most published research findings are false","volume":"2","author":"Ioannidis","year":"2005","journal-title":"PLoS Med"},{"key":"2022092013221839500_ref5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/bcr2124","article-title":"Meta-analysis of gene expression profiles in breast cancer: toward a unified understanding of breast cancer subtyping and prognosis signatures","volume":"10","author":"Wirapati","year":"2008","journal-title":"Breast Cancer Res"},{"key":"2022092013221839500_ref6","doi-asserted-by":"crossref","first-page":"1065","DOI":"10.1093\/bioinformatics\/btv734","article-title":"Positive and negative forms of replicability in gene network analysis","volume":"32","author":"Verleyen","year":"2016","journal-title":"Bioinformatics"},{"key":"2022092013221839500_ref7","doi-asserted-by":"crossref","first-page":"1125","DOI":"10.1080\/01621459.2019.1671197","article-title":"Modeling between-study heterogeneity for improved replicability in gene signature selection and clinical prediction","volume":"115","author":"Rashid","year":"2020","journal-title":"J Am Stat Assoc"},{"key":"2022092013221839500_ref8","doi-asserted-by":"crossref","first-page":"464","DOI":"10.1093\/jnci\/djq025","article-title":"Gene expression\u2013based prognostic signatures in lung cancer: ready for clinical use?","volume":"102","author":"Subramanian","year":"2010","journal-title":"J Natl Cancer Inst"},{"key":"2022092013221839500_ref9","doi-asserted-by":"crossref","first-page":"1748","DOI":"10.1016\/j.jtho.2020.07.005","article-title":"Multi-institutional prospective validation of prognostic mRNA signatures in early stage squamous lung cancer (alliance)","volume":"15","author":"Bueno","year":"2020","journal-title":"J Thorac Oncol"},{"key":"2022092013221839500_ref10","doi-asserted-by":"crossref","DOI":"10.1093\/jnci\/dju049","article-title":"Comparative meta-analysis of prognostic gene signatures for late-stage ovarian cancer","volume":"106","author":"Waldron","year":"2014","journal-title":"J Natl Cancer Inst"},{"key":"2022092013221839500_ref11","doi-asserted-by":"crossref","first-page":"306","DOI":"10.1038\/ng749","article-title":"Replication validity of genetic association studies","volume":"29","author":"Ioannidis","year":"2001","journal-title":"Nat Genet"},{"key":"2022092013221839500_ref12","doi-asserted-by":"crossref","first-page":"545","DOI":"10.1038\/nrc2173","article-title":"Taking gene-expression profiling to the clinic: when will molecular signatures become relevant to patient care?","volume":"7","author":"Sotiriou","year":"2007","journal-title":"Nat Rev Cancer"},{"key":"2022092013221839500_ref13","doi-asserted-by":"crossref","first-page":"1715","DOI":"10.1093\/jnci\/djm216","article-title":"Challenges in projecting clustering results across gene expression\u2013profiling datasets","volume":"99","author":"Lusa","year":"2007","journal-title":"J Natl Cancer Inst"},{"key":"2022092013221839500_ref14","doi-asserted-by":"crossref","DOI":"10.1093\/jnci\/dju357","article-title":"Absolute assignment of breast cancer intrinsic molecular subtype","volume":"107","author":"Paquet","year":"2015","journal-title":"J Natl Cancer Inst"},{"key":"2022092013221839500_ref15","doi-asserted-by":"crossref","first-page":"5923","DOI":"10.1073\/pnas.0601231103","article-title":"Thousands of samples are needed to generate a robust gene list for predicting outcome in cancer","volume":"103","author":"Ein-Dor","year":"2006","journal-title":"Proc Natl Acad Sci U S A"},{"key":"2022092013221839500_ref16","doi-asserted-by":"crossref","first-page":"671","DOI":"10.1093\/bib\/bbt044","article-title":"Similarity of markers identified from cancer gene expression studies: observations from GEO","volume":"15","author":"Shi","year":"2014","journal-title":"Brief Bioinform"},{"key":"2022092013221839500_ref17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/1471-2164-10-535","article-title":"Identification of genes associated with multiple cancers via integrative analysis","volume":"10","author":"Ma","year":"2009","journal-title":"BMC Genomics"},{"key":"2022092013221839500_ref18","doi-asserted-by":"crossref","first-page":"488","DOI":"10.1016\/S0140-6736(05)17866-0","article-title":"Prediction of cancer outcome with microarrays: a multiple random validation strategy","volume":"365","author":"Michiels","year":"2005","journal-title":"Lancet"},{"key":"2022092013221839500_ref19","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pcbi.1002240","article-title":"Most random gene expression signatures are significantly associated with breast cancer outcome","volume":"7","author":"Venet","year":"2011","journal-title":"PLoS Comput Biol"},{"key":"2022092013221839500_ref20","doi-asserted-by":"crossref","first-page":"735","DOI":"10.1093\/bib\/bbu049","article-title":"Measures for the degree of overlap of gene signatures and applications to TCGA","volume":"16","author":"Shi","year":"2015","journal-title":"Brief Bioinform"},{"key":"2022092013221839500_ref21","doi-asserted-by":"crossref","first-page":"2971","DOI":"10.1093\/bioinformatics\/btu434","article-title":"repfdr: a tool for replicability analysis for genome-wide association studies","volume":"30","author":"Heller","year":"2014","journal-title":"Bioinformatics"},{"key":"2022092013221839500_ref22","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pcbi.1005700","article-title":"Extracting replicable associations across multiple studies: empirical Bayes algorithms for controlling the false discovery rate","volume":"13","author":"Amar","year":"2017","journal-title":"PLoS Comput Biol"},{"key":"2022092013221839500_ref23","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41467-021-21226-z","article-title":"Model-based assessment of replicability for genome-wide association meta-analysis","volume":"12","author":"McGuire","year":"2021","journal-title":"Nat Commun"},{"key":"2022092013221839500_ref24","first-page":"A68","article-title":"The Cancer Genome Atlas (TCGA): an immeasurable source of knowledge","volume":"19","author":"Tomczak","year":"2015","journal-title":"Contemp Oncol (Pozn)"},{"key":"2022092013221839500_ref25","doi-asserted-by":"crossref","first-page":"1050","DOI":"10.1038\/s41416-020-0742-9","article-title":"Long non-coding RNA dysregulation is a frequent event in non-small cell lung carcinoma pathogenesis","volume":"122","author":"Acha-Sagredo","year":"2020","journal-title":"Br J Cancer"},{"key":"2022092013221839500_ref26","doi-asserted-by":"crossref","first-page":"744","DOI":"10.3390\/cancers11060744","article-title":"False discovery rate control in cancer biomarker selection using knockoffs","volume":"11","author":"Shen","year":"2019","journal-title":"Cancer"},{"key":"2022092013221839500_ref27","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s12885-021-08021-1","article-title":"Reference-free transcriptome signatures for prostate cancer prognosis","volume":"21","author":"Nguyen","year":"2021","journal-title":"BMC Cancer"},{"key":"2022092013221839500_ref28","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1177\/0962280209105024","article-title":"Survival analysis with high-dimensional covariates","volume":"19","author":"Witten","year":"2010","journal-title":"Stat Methods Med Res"},{"key":"2022092013221839500_ref29","doi-asserted-by":"crossref","first-page":"392","DOI":"10.1093\/bib\/bbn027","article-title":"Penalized feature selection and classification in bioinformatics","volume":"9","author":"Ma","year":"2008","journal-title":"Brief Bioinform"},{"key":"2022092013221839500_ref30","doi-asserted-by":"crossref","DOI":"10.1002\/0470041102","volume-title":"Cancer Diagnostics with DNA Microarrays","author":"Knudsen","year":"2006"},{"key":"2022092013221839500_ref31","first-page":"33","article-title":"Package \u2018survival\u2019","volume":"128","author":"Therneau","year":"2015","journal-title":"R Top Doc"},{"key":"2022092013221839500_ref32","first-page":"3","article-title":"Teoria statistica delle classi e calcolo delle probabilita","volume":"8","author":"Bonferroni","year":"1936","journal-title":"Pubblicazioni del R Istituto Superiore di Scienze Economiche e Commericiali di Firenze"},{"key":"2022092013221839500_ref33","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1111\/j.2517-6161.1995.tb02031.x","article-title":"Controlling the false discovery rate: a practical and powerful approach to multiple testing","volume":"57","author":"Benjamini","year":"1995","journal-title":"J R Stat Soc B Methodol"},{"key":"2022092013221839500_ref34","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1111\/j.1467-9868.2005.00503.x","article-title":"Regularization and variable selection via the elastic net","volume":"67","author":"Zou","year":"2005","journal-title":"J R Stat Soc Series B Stat Methodology"},{"key":"2022092013221839500_ref35","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v033.i01","article-title":"Regularization paths for generalized linear models via coordinate descent","volume":"33","author":"Friedman","year":"2010","journal-title":"J Stat Softw"},{"key":"2022092013221839500_ref36","doi-asserted-by":"crossref","first-page":"3641","DOI":"10.1093\/hmg\/ddy271","article-title":"Meta-analysis of genome-wide association studies for height and body mass index in\u223c 700000 individuals of European ancestry","volume":"27","author":"Yengo","year":"2018","journal-title":"Hum Mol Genet"},{"key":"2022092013221839500_ref37","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41467-018-04951-w","article-title":"Genome-wide association analyses identify 143 risk variants and putative regulatory mechanisms for type 2 diabetes","volume":"9","author":"Xue","year":"2018","journal-title":"Nat Commun"},{"key":"2022092013221839500_ref38","doi-asserted-by":"crossref","first-page":"3055","DOI":"10.1093\/bioinformatics\/bty1054","article-title":"DIABLO: an integrative approach for identifying key molecular drivers from multi-omics assays","volume":"35","author":"Singh","year":"2019","journal-title":"Bioinformatics"},{"key":"2022092013221839500_ref39","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/1752-0509-8-S2-I1","article-title":"Data integration in the era of omics: current and future challenges","volume":"8","author":"Gomez-Cabrero","year":"2014","journal-title":"BMC Syst Biol"},{"key":"2022092013221839500_ref40","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1038\/nrg1318","article-title":"The Bayesian revolution in genetics","volume":"5","author":"Beaumont","year":"2004","journal-title":"Nat Rev Genet"},{"key":"2022092013221839500_ref41","doi-asserted-by":"crossref","DOI":"10.1214\/11-AOAS463","article-title":"Incorporating biological information into linear models: a Bayesian approach to the selection of pathways and genes","volume":"5","author":"Stingo","year":"2011","journal-title":"Ann Appl Stat"},{"key":"2022092013221839500_ref42","doi-asserted-by":"crossref","first-page":"176","DOI":"10.1002\/gepi.21956","article-title":"Incorporating functional genomic information in genetic association studies using an empirical Bayes approach","volume":"40","author":"Spencer","year":"2016","journal-title":"Genet Epidemiol"},{"key":"2022092013221839500_ref43","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1006\/ceps.2000.1040","article-title":"Measures of effect size for comparative studies: applications, interpretations, and limitations","volume":"25","author":"Olejnik","year":"2000","journal-title":"Contemp Educ Psychol"}],"container-title":["Briefings in Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/23\/5\/bbac304\/45937226\/bbac304.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/23\/5\/bbac304\/45937226\/bbac304.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,29]],"date-time":"2024-09-29T20:39:14Z","timestamp":1727642354000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bib\/article\/doi\/10.1093\/bib\/bbac304\/6649493"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,25]]},"references-count":43,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2022,9,20]]}},"URL":"https:\/\/doi.org\/10.1093\/bib\/bbac304","relation":{},"ISSN":["1467-5463","1477-4054"],"issn-type":[{"value":"1467-5463","type":"print"},{"value":"1477-4054","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022,9]]},"published":{"date-parts":[[2022,7,25]]},"article-number":"bbac304"}}