{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,17]],"date-time":"2025-11-17T17:16:38Z","timestamp":1763399798311,"version":"3.37.3"},"reference-count":23,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2018,8,29]],"date-time":"2018-08-29T00:00:00Z","timestamp":1535500800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/501100007750","name":"National Cheng Kung University","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100007750","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004663","name":"Ministry of Science and Technology in Taiwan","doi-asserted-by":"crossref","award":["105-2218-E-006-016-MY2"],"award-info":[{"award-number":["105-2218-E-006-016-MY2"]}],"id":[{"id":"10.13039\/501100004663","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2019,3,15]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Motivation<\/jats:title><jats:p>Inference of gene regulatory networks (GRNs) from perturbation data can give detailed mechanistic insights of a biological system. Many inference methods exist, but the resulting GRN is generally sensitive to the choice of method-specific parameters. Even though the inferred GRN is optimal given the parameters, many links may be wrong or missing if the data is not informative. To make GRN inference reliable, a method is needed to estimate the support of each predicted link as the method parameters are varied.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>To achieve this we have developed a method called nested bootstrapping, which applies a bootstrapping protocol to GRN inference, and by repeated bootstrap runs assesses the stability of the estimated support values. To translate bootstrap support values to false discovery rates we run the same pipeline with shuffled data as input. This provides a general method to control the false discovery rate of GRN inference that can be applied to any setting of inference parameters, noise level, or data properties. We evaluated nested bootstrapping on a simulated dataset spanning a range of such properties, using the LASSO, Least Squares, RNI, GENIE3 and CLR inference methods. An improved inference accuracy was observed in almost all situations. Nested bootstrapping was incorporated into the GeneSPIDER package, which was also used for generating the simulated networks and data, as well as running and analyzing the inferences.<\/jats:p><\/jats:sec><jats:sec><jats:title>Availability and implementation<\/jats:title><jats:p>https:\/\/bitbucket.org\/sonnhammergrni\/genespider\/src\/NB\/%2B Methods\/NestBoot.m<\/jats:p><\/jats:sec>","DOI":"10.1093\/bioinformatics\/bty764","type":"journal-article","created":{"date-parts":[[2018,8,29]],"date-time":"2018-08-29T19:37:49Z","timestamp":1535571469000},"page":"1026-1032","source":"Crossref","is-referenced-by-count":13,"title":["A generalized framework for controlling FDR in gene regulatory network inference"],"prefix":"10.1093","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8326-6178","authenticated-orcid":false,"given":"Daniel","family":"Morgan","sequence":"first","affiliation":[{"name":"Department of Biochemistry and Biophysics, Stockholm Bioinformatics Center, Science for Life Laboratory, Stockholm University, Stockholm, Sweden"}]},{"given":"Andreas","family":"Tj\u00e4rnberg","sequence":"additional","affiliation":[{"name":"Department of Physics, Chemistry and Biology\/Bioinformatics, Link\u00f6ping University, Link\u00f6ping, Sweden"}]},{"given":"Torbj\u00f6rn E M","family":"Nordling","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, National Cheng Kung University, Tainan, Taiwan"}]},{"given":"Erik L L","family":"Sonnhammer","sequence":"additional","affiliation":[{"name":"Department of Biochemistry and Biophysics, Stockholm Bioinformatics Center, Science for Life Laboratory, Stockholm University, Stockholm, Sweden"}]}],"member":"286","published-online":{"date-parts":[[2018,8,29]]},"reference":[{"key":"2023013107262175100_bty764-B1","doi-asserted-by":"crossref","first-page":"839","DOI":"10.15252\/msb.20156236","article-title":"An experimentally supported model of the bacillus subtilis global transcriptional regulatory network","volume":"11","author":"Arrieta-Ortiz","year":"2015","journal-title":"Mol. Syst. Biol"},{"key":"2023013107262175100_bty764-B2","doi-asserted-by":"crossref","DOI":"10.1201\/9780429246593","volume-title":"An Introduction to the Bootstrap","author":"Efron","year":"1994"},{"key":"2023013107262175100_bty764-B3","doi-asserted-by":"crossref","first-page":"e8.","DOI":"10.1371\/journal.pbio.0050008","article-title":"Large-scale mapping and validation of escherichia coli transcriptional regulation from a compendium of expression profiles","volume":"5","author":"Faith","year":"2007","journal-title":"PLoS Biol"},{"key":"2023013107262175100_bty764-B4","article-title":"glmnet: lasso and elastic-net regularized generalized linear models","volume":"1","author":"Friedman","year":"2009","journal-title":"R Package Version"},{"key":"2023013107262175100_bty764-B5","doi-asserted-by":"crossref","first-page":"102","DOI":"10.1126\/science.1081900","article-title":"Inferring genetic networks and identifying compound mode of action via expression profiling","volume":"301","author":"Gardner","year":"2003","journal-title":"Science"},{"key":"2023013107262175100_bty764-B6","doi-asserted-by":"crossref","first-page":"e13397.","DOI":"10.1371\/journal.pone.0013397","article-title":"Dream4: combining genetic and dynamic information to identify biological networks and dynamical models","volume":"5","author":"Greenfield","year":"2010","journal-title":"PLoS One"},{"key":"2023013107262175100_bty764-B7","doi-asserted-by":"crossref","first-page":"145.","DOI":"10.1186\/1752-0509-6-145","article-title":"Tigress: trustful inference of gene regulation using stability selection","volume":"6","author":"Haury","year":"2012","journal-title":"BMC Syst. Biol"},{"key":"2023013107262175100_bty764-B8","doi-asserted-by":"crossref","first-page":"435","DOI":"10.1093\/biomet\/76.3.435","article-title":"Importance sampling and the nested bootstrap","volume":"76","author":"Hinkley","year":"1989","journal-title":"Biometrika"},{"key":"2023013107262175100_bty764-B9","doi-asserted-by":"crossref","first-page":"e12776","DOI":"10.1371\/journal.pone.0012776","article-title":"Inferring regulatory networks from expression data using tree-based methods","volume":"5","author":"Huynh-Thu","year":"2010","journal-title":"PLoS One"},{"key":"2023013107262175100_bty764-B10","doi-asserted-by":"crossref","first-page":"305","DOI":"10.1142\/9789814447331_0029","volume-title":"Biocomputing 2000","author":"Ideker","year":"1999"},{"key":"2023013107262175100_bty764-B11","doi-asserted-by":"crossref","first-page":"391.","DOI":"10.1214\/12-AOAS589","article-title":"Bootstrap inference for network construction with an application to a breast cancer microarray study","volume":"7","author":"Li","year":"2013","journal-title":"Ann. Appl. Stat"},{"key":"2023013107262175100_bty764-B12","doi-asserted-by":"crossref","first-page":"1145","DOI":"10.1073\/pnas.0812551106","article-title":"A network biology approach to aging in yeast","volume":"106","author":"Lorenz","year":"2009","journal-title":"Proc. Natl. Acad. Sci. USA"},{"year":"2017","key":"2023013107262175100_bty764-B13"},{"key":"2023013107262175100_bty764-B14","doi-asserted-by":"crossref","first-page":"442","DOI":"10.1016\/0005-2795(75)90109-9","article-title":"Comparison of the predicted and observed secondary structure of t4 phage lysozyme","volume":"405","author":"Matthews","year":"1975","journal-title":"Biochim. Biophys. Acta (BBA)-Protein Struct"},{"key":"2023013107262175100_bty764-B15","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1111\/j.1467-9868.2010.00740.x","article-title":"Stability selection","volume":"72","author":"Meinshausen","year":"2010","journal-title":"J. R. Stat. Soc. Ser. B (Stat. Methodol.)"},{"year":"2013","author":"Nordling","key":"2023013107262175100_bty764-B16"},{"key":"2023013107262175100_bty764-B17","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1016\/j.tig.2006.11.003","article-title":"Perturbations to uncover gene networks","volume":"23","author":"Tegn\u00e9r","year":"2007","journal-title":"Trends Genet"},{"key":"2023013107262175100_bty764-B18","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","article-title":"Regression shrinkage and selection via the lasso","volume":"58","author":"Tibshirani","year":"1996","journal-title":"J. R. Stat. Soc. Ser. B (Methodological"},{"key":"2023013107262175100_bty764-B19","doi-asserted-by":"crossref","first-page":"398","DOI":"10.1089\/cmb.2012.0268","article-title":"Optimal sparsity criteria for network inference","volume":"20","author":"Tj\u00e4rnberg","year":"2013","journal-title":"J. Comput. Biol. J. Comput. Mol. Cell Biol"},{"key":"2023013107262175100_bty764-B20","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1039\/C4MB00419A","article-title":"Avoiding pitfalls in L1-regularised inference of gene networks","volume":"11","author":"Tj\u00e4rnberg","year":"2015","journal-title":"Mol. Biosyst"},{"key":"2023013107262175100_bty764-B21","doi-asserted-by":"crossref","first-page":"1304","DOI":"10.1039\/C7MB00058H","article-title":"Genespider\u2013gene regulatory network inference benchmarking with controlled network and data properties","volume":"13","author":"Tj\u00e4rnberg","year":"2017","journal-title":"Mol. BioSyst"},{"key":"2023013107262175100_bty764-B22","doi-asserted-by":"crossref","first-page":"468.","DOI":"10.1214\/10-AOAS377","article-title":"Random lasso","volume":"5","author":"Wang","year":"2011","journal-title":"Ann. Appl. Stat"},{"key":"2023013107262175100_bty764-B23","first-page":"2541","article-title":"On model selection consistency of lasso","volume":"7","author":"Zhao","year":"2006","journal-title":"J. Mach. Learn. Res"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/35\/6\/1026\/48967463\/bioinformatics_35_6_1026.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/35\/6\/1026\/48967463\/bioinformatics_35_6_1026.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,9]],"date-time":"2024-07-09T18:48:38Z","timestamp":1720550918000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/35\/6\/1026\/5086392"}},"subtitle":[],"editor":[{"given":"Bonnie","family":"Berger","sequence":"additional","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2018,8,29]]},"references-count":23,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2019,3,15]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/bty764","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"type":"print","value":"1367-4803"},{"type":"electronic","value":"1367-4811"}],"subject":[],"published-other":{"date-parts":[[2019,3,15]]},"published":{"date-parts":[[2018,8,29]]}}}