{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,11]],"date-time":"2026-08-11T21:06:40Z","timestamp":1786482400617,"version":"build-2736575974"},"reference-count":40,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2016,11,14]],"date-time":"2016-11-14T00:00:00Z","timestamp":1479081600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2016,11,14]],"date-time":"2016-11-14T00:00:00Z","timestamp":1479081600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100008762","name":"Genome Canada","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100008762","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000233","name":"Genome British Columbia","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100000233","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Genome Quebec"},{"DOI":"10.13039\/501100000024","name":"Canadian Institutes of Health Research","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100000024","id-type":"DOI","asserted-by":"publisher"}]},{"name":"PROOF Centre"},{"name":"St. Pauls' Hospital Foundation"},{"name":"Providence Healthcare"},{"name":"COPD Clinical Research Network"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Bioinformatics"],"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Background<\/jats:title>\n                <jats:p>Gene network inference (GNI) algorithms can be used to identify sets of coordinately expressed genes, termed network modules from whole transcriptome gene expression data. The identification of such modules has become a popular approach to systems biology, with important applications in translational research. Although diverse computational and statistical approaches have been devised to identify such modules, their performance behavior is still not fully understood, particularly in complex human tissues. Given human heterogeneity, one important question is how the outputs of these computational methods are sensitive to the input sample set, or stability. A related question is how this sensitivity depends on the size of the sample set. We describe here the SABRE (<jats:bold>S<\/jats:bold>imilarity <jats:bold>A<\/jats:bold>cross <jats:bold>B<\/jats:bold>ootstrap <jats:bold>RE<\/jats:bold>-sampling) procedure for assessing the stability of gene network modules using a re-sampling strategy, introduce a novel criterion for identifying stable modules, and demonstrate the utility of this approach in a clinically-relevant cohort, using two different gene network module discovery algorithms.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>The stability of modules increased as sample size increased and stable modules were more likely to be replicated in larger sets of samples. Random modules derived from permutated gene expression data were consistently unstable, as assessed by SABRE, and provide a useful baseline value for our proposed stability criterion. Gene module sets identified by different algorithms varied with respect to their stability, as assessed by SABRE. Finally, stable modules were more readily annotated in various curated gene set databases.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusions<\/jats:title>\n                <jats:p>The SABRE procedure and proposed stability criterion may provide guidance when designing systems biology studies in complex human disease and tissues.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1186\/s12859-016-1319-8","type":"journal-article","created":{"date-parts":[[2016,11,14]],"date-time":"2016-11-14T13:27:30Z","timestamp":1479130050000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["SABRE: a method for assessing the stability of gene modules in complex tissues and subject populations"],"prefix":"10.1186","volume":"17","author":[{"given":"Casey P.","family":"Shannon","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Virginia","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mandeep","family":"Takhar","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zsuzsanna","family":"Hollander","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Robert","family":"Balshaw","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bruce M.","family":"McManus","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Scott J.","family":"Tebbutt","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Don D.","family":"Sin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Raymond T.","family":"Ng","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2016,11,14]]},"reference":[{"key":"1319_CR1","doi-asserted-by":"publisher","first-page":"e59","DOI":"10.1371\/journal.pcbi.0020059","volume":"2","author":"H Kurata","year":"2006","unstructured":"Kurata H, El-Samad H, Iwasaki R, Ohtake H, Doyle JC, Grigorova I, et al. Module-based analysis of robustness tradeoffs in the heat shock response system. PLoS Comput Biol. 2006;2:e59.","journal-title":"PLoS Comput Biol"},{"key":"1319_CR2","doi-asserted-by":"publisher","first-page":"e145","DOI":"10.1371\/journal.pcbi.0020145","volume":"2","author":"K Xia","year":"2006","unstructured":"Xia K, Xue H, Dong D, Zhu S, Wang J, Zhang Q, et al. Identification of the proliferation\/differentiation switch in the cellular network of multicellular organisms. PLoS Comput Biol. 2006;2:e145.","journal-title":"PLoS Comput Biol"},{"key":"1319_CR3","doi-asserted-by":"publisher","first-page":"e130","DOI":"10.1371\/journal.pgen.0020130","volume":"2","author":"A Ghazalpour","year":"2006","unstructured":"Ghazalpour A, Doss S, Zhang B, Wang S, Plaisier C, Castellanos R, et al. Integrating genetic and network analysis to characterize genes related to mouse weight. PLoS Genet. 2006;2:e130.","journal-title":"PLoS Genet"},{"key":"1319_CR4","doi-asserted-by":"publisher","first-page":"e1000521","DOI":"10.1371\/journal.pcbi.1000521","volume":"5","author":"J Wang","year":"2009","unstructured":"Wang J, Zhang S, Wang Y, Chen L, Zhang X-S. Disease-aging network reveals significant roles of aging genes in connecting genetic diseases. PLoS Comput Biol. 2009;5:e1000521.","journal-title":"PLoS Comput Biol"},{"key":"1319_CR5","doi-asserted-by":"publisher","first-page":"e1000479","DOI":"10.1371\/journal.pgen.1000479","volume":"5","author":"EA Stone","year":"2009","unstructured":"Stone EA, Ayroles JF. Modulated modularity clustering as an exploratory tool for functional genomic inference. PLoS Genet. 2009;5:e1000479.","journal-title":"PLoS Genet"},{"key":"1319_CR6","doi-asserted-by":"publisher","first-page":"e1000642","DOI":"10.1371\/journal.pgen.1000642","volume":"5","author":"CL Plaisier","year":"2009","unstructured":"Plaisier CL, Horvath S, Huertas-Vazquez A, Cruz-Bautista I, Herrera MF, Tusie-Luna T, et al. A systems genetics approach implicates USF1, FADS3, and other causal candidate genes for familial combined hyperlipidemia. PLoS Genet. 2009;5:e1000642.","journal-title":"PLoS Genet"},{"key":"1319_CR7","doi-asserted-by":"publisher","first-page":"166","DOI":"10.1038\/ng1165","volume":"34","author":"E Segal","year":"2003","unstructured":"Segal E, Shapira M, Regev A, Pe\u2019er D, Botstein D, Koller D, et al. Module networks: identifying regulatory modules and their condition-specific regulators from gene expression data. Nat Genet. 2003;34:166\u201376.","journal-title":"Nat Genet"},{"key":"1319_CR8","doi-asserted-by":"publisher","first-page":"799","DOI":"10.1126\/science.1094068","volume":"303","author":"N Friedman","year":"2004","unstructured":"Friedman N. Inferring cellular networks using probabilistic graphical models. Science. 2004;303:799\u2013805.","journal-title":"Science"},{"key":"1319_CR9","doi-asserted-by":"publisher","first-page":"382","DOI":"10.1038\/ng1532","volume":"37","author":"K Basso","year":"2005","unstructured":"Basso K, Margolin AA, Stolovitzky G, Klein U, Dalla-Favera R, Califano A. Reverse engineering of regulatory networks in human B cells. Nat Genet. 2005;37:382\u201390.","journal-title":"Nat Genet"},{"key":"1319_CR10","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1038\/ng.2007.70","volume":"40","author":"I Lee","year":"2008","unstructured":"Lee I, Lehner B, Crombie C, Wong W, Fraser AG, Marcotte EM. A single gene network accurately predicts phenotypic effects of gene perturbation in Caenorhabditis elegans. Nat Genet. 2008;40:181\u20138.","journal-title":"Nat Genet"},{"key":"1319_CR11","doi-asserted-by":"crossref","first-page":"Article17","DOI":"10.2202\/1544-6115.1128","volume":"4","author":"B Zhang","year":"2005","unstructured":"Zhang B, Horvath S. A general framework for weighted gene co-expression network analysis. Stat Appl Genet Mol Biol. 2005;4:Article17.","journal-title":"Stat Appl Genet Mol Biol"},{"key":"1319_CR12","doi-asserted-by":"publisher","first-page":"e12776","DOI":"10.1371\/journal.pone.0012776","volume":"5","author":"VA Huynh-Thu","year":"2010","unstructured":"Huynh-Thu VA, Irrthum A, Wehenkel L, Geurts P. Inferring regulatory networks from expression data using tree-based methods. PLoS ONE. 2010;5:e12776.","journal-title":"PLoS ONE"},{"key":"1319_CR13","doi-asserted-by":"publisher","first-page":"S17","DOI":"10.1186\/1471-2105-12-S13-S17","volume":"12","author":"JC Rajapakse","year":"2011","unstructured":"Rajapakse JC, Mundra PA. Stability of building gene regulatory networks with sparse autoregressive models. BMC Bioinformatics. 2011;12:S17.","journal-title":"BMC Bioinformatics"},{"key":"1319_CR14","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1186\/1752-0509-6-145","volume":"6","author":"A-C Haury","year":"2012","unstructured":"Haury A-C, Mordelet F, Vera-Licona P, Vert J-P. TIGRESS: trustful inference of gene REgulation using stability selection. BMC Syst Biol. 2012;6:145.","journal-title":"BMC Syst Biol"},{"key":"1319_CR15","doi-asserted-by":"publisher","first-page":"1376","DOI":"10.1093\/bioinformatics\/bts143","volume":"28","author":"R Kuffner","year":"2012","unstructured":"Kuffner R, Petri T, Tavakkolkhah P, Windhager L, Zimmer R. Inferring gene regulatory networks by ANOVA. Bioinformatics. 2012;28:1376\u201382.","journal-title":"Bioinformatics"},{"key":"1319_CR16","doi-asserted-by":"publisher","first-page":"e92709","DOI":"10.1371\/journal.pone.0092709","volume":"9","author":"J Ruyssinck","year":"2014","unstructured":"Ruyssinck J, Huynh-Thu VA, Geurts P, Dhaene T, Demeester P, Saeys Y. NIMEFI: gene regulatory network inference using multiple ensemble feature importance algorithms. PLoS ONE. 2014;9:e92709.","journal-title":"PLoS ONE"},{"key":"1319_CR17","doi-asserted-by":"publisher","first-page":"95","DOI":"10.1186\/1471-2105-11-95","volume":"11","author":"R Gill","year":"2010","unstructured":"Gill R, Datta S, Datta S. A statistical framework for differential network analysis from microarray data. BMC Bioinformatics. 2010;11:95.","journal-title":"BMC Bioinformatics"},{"key":"1319_CR18","doi-asserted-by":"publisher","first-page":"561","DOI":"10.1093\/bioinformatics\/btm640","volume":"24","author":"V Pihur","year":"2008","unstructured":"Pihur V, Datta S, Datta S. Reconstruction of genetic association networks from microarray data: a partial least squares approach. Bioinformatics. 2008;24:561\u20138.","journal-title":"Bioinformatics"},{"key":"1319_CR19","doi-asserted-by":"crossref","unstructured":"de Jong S, Boks MPM, Fuller TF, Strengman E, Janson E, de Kovel CGF, et al. A Gene Co-Expression Network in Whole Blood of Schizophrenia Patients Is Independent of Antipsychotic-Use and Enriched for Brain-Expressed Genes. Mazza M, editor. PLoS ONE. 2012;7:e39498.","DOI":"10.1371\/journal.pone.0039498"},{"key":"1319_CR20","doi-asserted-by":"publisher","first-page":"636","DOI":"10.1186\/1471-2164-13-636","volume":"13","author":"KR Van Eijk","year":"2012","unstructured":"Van Eijk KR, de Jong S, Boks MP, Langeveld T, Colas F, Veldink JH, et al. Genetic analysis of DNA methylation and gene expression levels in whole blood of healthy human subjects. BMC Genomics. 2012;13:636.","journal-title":"BMC Genomics"},{"key":"1319_CR21","doi-asserted-by":"publisher","first-page":"e1001057","DOI":"10.1371\/journal.pcbi.1001057","volume":"7","author":"P Langfelder","year":"2011","unstructured":"Langfelder P, Luo R, Oldham MC, Horvath S. Is my network module preserved and reproducible? PLoS Comput Biol. 2011;7:e1001057.","journal-title":"PLoS Comput Biol"},{"key":"1319_CR22","doi-asserted-by":"crossref","unstructured":"Brock G, Pihur V, Datta S, Datta S, others. clValid, an R package for cluster validation. J. Stat. Softw. Brock Al March 2008 [Internet]. 2011 [cited 2016 Sep 8]; Available from: http:\/\/cran.us.r-project.org\/web\/packages\/clValid\/vignettes\/clValid.pdf","DOI":"10.18637\/jss.v025.i04"},{"key":"1319_CR23","doi-asserted-by":"publisher","first-page":"459","DOI":"10.1093\/bioinformatics\/btg025","volume":"19","author":"S Datta","year":"2003","unstructured":"Datta S, Datta S. Comparisons and validation of statistical clustering techniques for microarray gene expression data. Bioinformatics. 2003;19:459\u201366.","journal-title":"Bioinformatics"},{"key":"1319_CR24","doi-asserted-by":"publisher","first-page":"150","DOI":"10.1016\/j.immuni.2008.05.012","volume":"29","author":"D Chaussabel","year":"2008","unstructured":"Chaussabel D, Quinn C, Shen J, Patel P, Glaser C, Baldwin N, et al. A modular analysis framework for blood genomics studies: application to systemic lupus erythematosus. Immunity. 2008;29:150\u201364.","journal-title":"Immunity"},{"key":"1319_CR25","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1186\/1741-7007-8-84","volume":"8","author":"D Chaussabel","year":"2010","unstructured":"Chaussabel D, Pascual V, Banchereau J. Assessing the human immune system through blood transcriptomics. BMC Biol. 2010;8:84.","journal-title":"BMC Biol"},{"key":"1319_CR26","doi-asserted-by":"publisher","first-page":"195","DOI":"10.1038\/ni.2789","volume":"15","author":"S Li","year":"2013","unstructured":"Li S, Rouphael N, Duraisingham S, Romero-Steiner S, Presnell S, Davis C, et al. Molecular signatures of antibody responses derived from a systems biology study of five human vaccines. Nat Immunol. 2013;15:195\u2013204.","journal-title":"Nat Immunol"},{"key":"1319_CR27","doi-asserted-by":"publisher","first-page":"765","DOI":"10.1016\/S0140-6736(07)61380-4","volume":"370","author":"DM Mannino","year":"2007","unstructured":"Mannino DM, Buist AS. Global burden of COPD: risk factors, prevalence, and future trends. Lancet. 2007;370:765\u201373.","journal-title":"Lancet"},{"key":"1319_CR28","doi-asserted-by":"publisher","first-page":"786","DOI":"10.1016\/S0140-6736(07)61382-8","volume":"370","author":"JA Wedzicha","year":"2007","unstructured":"Wedzicha JA, Seemungal TA. COPD exacerbations: defining their cause and prevention. Lancet. 2007;370:786\u201396.","journal-title":"Lancet"},{"key":"1319_CR29","doi-asserted-by":"publisher","first-page":"869","DOI":"10.1183\/09031936.00111707","volume":"31","author":"J Vestbo","year":"2008","unstructured":"Vestbo J, Anderson W, Coxson HO, Crim C, Dawber F, Edwards L, et al. Evaluation of COPD longitudinally to identify predictive surrogate end-points (eclipse). Eur Respir J. 2008;31:869\u201373.","journal-title":"Eur Respir J"},{"key":"1319_CR30","doi-asserted-by":"publisher","first-page":"185","DOI":"10.1093\/bioinformatics\/19.2.185","volume":"19","author":"BM Bolstad","year":"2003","unstructured":"Bolstad BM, Irizarry R, \u00c5strand M, Speed TP. A comparison of normalization methods for high density oligonucleotide array data based on variance and bias. Bioinformatics. 2003;19:185\u201393.","journal-title":"Bioinformatics"},{"key":"1319_CR31","doi-asserted-by":"publisher","first-page":"943","DOI":"10.1093\/bioinformatics\/btl033","volume":"22","author":"S Hochreiter","year":"2006","unstructured":"Hochreiter S, Clevert D-A, Obermayer K. A new summarization method for affymetrix probe level data. Bioinformatics. 2006;22:943\u20139.","journal-title":"Bioinformatics"},{"key":"1319_CR32","doi-asserted-by":"publisher","first-page":"559","DOI":"10.1186\/1471-2105-9-559","volume":"9","author":"P Langfelder","year":"2008","unstructured":"Langfelder P, Horvath S. WGCNA: an R package for weighted correlation network analysis. BMC Bioinformatics. 2008;9:559.","journal-title":"BMC Bioinformatics"},{"key":"1319_CR33","doi-asserted-by":"publisher","first-page":"1169","DOI":"10.1038\/nmeth.2728","volume":"10","author":"JI Fuxman Bass","year":"2013","unstructured":"Fuxman Bass JI, Diallo A, Nelson J, Soto JM, Myers CL, Walhout AJM. Using networks to measure similarity between genes: association index selection. Nat Methods. 2013;10:1169\u201376.","journal-title":"Nat Methods"},{"key":"1319_CR34","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1214\/aos\/1176344552","volume":"7","author":"B Efron","year":"1979","unstructured":"Efron B. Bootstrap methods: another look at the Jackknife. Ann Stat. 1979;7:1\u201326.","journal-title":"Ann Stat"},{"key":"1319_CR35","first-page":"1","volume":"1695","author":"G Csardi","year":"2006","unstructured":"Csardi G, Nepusz T. The igraph software package for complex network research. Int J Complex Syst. 2006;1695:1\u20139.","journal-title":"Int J Complex Syst"},{"key":"1319_CR36","doi-asserted-by":"publisher","first-page":"15545","DOI":"10.1073\/pnas.0506580102","volume":"102","author":"A Subramanian","year":"2005","unstructured":"Subramanian A, Tamayo P, Mootha VK, Mukherjee S, Ebert BL, Gillette MA, et al. Gene set enrichment analysis: a knowledge-based approach for interpreting genome-wide expression profiles. Proc Natl Acad Sci U S A. 2005;102:15545\u201350.","journal-title":"Proc Natl Acad Sci U S A"},{"key":"1319_CR37","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1093\/nar\/gkn923","volume":"37","author":"DW Huang","year":"2009","unstructured":"Huang DW, Sherman BT, Lempicki RA. Bioinformatics enrichment tools: paths toward the comprehensive functional analysis of large gene lists. Nucleic Acids Res. 2009;37:1\u201313.","journal-title":"Nucleic Acids Res"},{"key":"1319_CR38","doi-asserted-by":"publisher","first-page":"16569","DOI":"10.1073\/pnas.0507655102","volume":"102","author":"JE Hirsch","year":"2005","unstructured":"Hirsch JE. An index to quantify an individual\u2019s scientific research output. Proc Natl Acad Sci U S A. 2005;102:16569\u201372.","journal-title":"Proc Natl Acad Sci U S A"},{"key":"1319_CR39","first-page":"67","volume":"4","author":"JD Campbell","year":"2012","unstructured":"Campbell JD, McDonough JE, Zeskind JE, Hackett TL, Pechkovsky DV, Brandsma C-A, et al. A gene expression signature of emphysema-related lung destruction and its reversal by the tripeptide GHK. Genome Med. 2012;4:67.","journal-title":"Genome Med"},{"key":"1319_CR40","doi-asserted-by":"publisher","first-page":"531","DOI":"10.1165\/rcmb.2012-0492TR","volume":"48","author":"K Hoenderdos","year":"2013","unstructured":"Hoenderdos K, Condliffe A. The neutrophil in chronic obstructive pulmonary disease. Too little, too late or too much, too soon? Am J Respir Cell Mol Biol. 2013;48:531\u20139.","journal-title":"Am J Respir Cell Mol Biol"}],"container-title":["BMC Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-016-1319-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12859-016-1319-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-016-1319-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,2,1]],"date-time":"2024-02-01T18:04:59Z","timestamp":1706810699000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/s12859-016-1319-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,11,14]]},"references-count":40,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2016,12]]}},"alternative-id":["1319"],"URL":"https:\/\/doi.org\/10.1186\/s12859-016-1319-8","relation":{},"ISSN":["1471-2105"],"issn-type":[{"value":"1471-2105","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,11,14]]},"assertion":[{"value":"25 June 2016","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 November 2016","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 November 2016","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"460"}}