{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,7]],"date-time":"2026-08-07T19:08:38Z","timestamp":1786129718833,"version":"3.56.0"},"reference-count":41,"publisher":"Oxford University Press (OUP)","issue":"16","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2011,8,15]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Motivation: Over the last decade, numerous methods have been developed for inference of regulatory networks from gene expression data. However, accurate and systematic evaluation of these methods is hampered by the difficulty of constructing adequate benchmarks and the lack of tools for a differentiated analysis of network predictions on such benchmarks.<\/jats:p>\n                  <jats:p>Results: Here, we describe a novel and comprehensive method for in silico benchmark generation and performance profiling of network inference methods available to the community as an open-source software called GeneNetWeaver (GNW). In addition to the generation of detailed dynamical models of gene regulatory networks to be used as benchmarks, GNW provides a network motif analysis that reveals systematic prediction errors, thereby indicating potential ways of improving inference methods. The accuracy of network inference methods is evaluated using standard metrics such as precision-recall and receiver operating characteristic curves. We show how GNW can be used to assess the performance and identify the strengths and weaknesses of six inference methods. Furthermore, we used GNW to provide the international Dialogue for Reverse Engineering Assessments and Methods (DREAM) competition with three network inference challenges (DREAM3, DREAM4 and DREAM5).<\/jats:p>\n                  <jats:p>Availability: GNW is available at http:\/\/gnw.sourceforge.net along with its Java source code, user manual and supporting data.<\/jats:p>\n                  <jats:p>Supplementary information: \u00a0Supplementary data are available at Bioinformatics online.<\/jats:p>\n                  <jats:p>Contact: \u00a0dario.floreano@epfl.ch<\/jats:p>","DOI":"10.1093\/bioinformatics\/btr373","type":"journal-article","created":{"date-parts":[[2011,6,22]],"date-time":"2011-06-22T20:58:29Z","timestamp":1308776309000},"page":"2263-2270","source":"Crossref","is-referenced-by-count":500,"title":["GeneNetWeaver:\n                    <i>in silico<\/i>\n                    benchmark generation and performance profiling of network inference methods"],"prefix":"10.1093","volume":"27","author":[{"given":"Thomas","family":"Schaffter","sequence":"first","affiliation":[{"name":"1 Laboratory of Intelligent Systems, Ecole Polytechnique F\u00e9d\u00e9rale de Lausanne (EPFL), Lausanne, Switzerland, 2MIT Computer Science and Artificial Intelligence Laboratory, Cambridge and 3Broad Institute of MIT and Harvard, Cambridge, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Marbach","sequence":"additional","affiliation":[{"name":"1 Laboratory of Intelligent Systems, Ecole Polytechnique F\u00e9d\u00e9rale de Lausanne (EPFL), Lausanne, Switzerland, 2MIT Computer Science and Artificial Intelligence Laboratory, Cambridge and 3Broad Institute of MIT and Harvard, Cambridge, MA, USA"},{"name":"1 Laboratory of Intelligent Systems, Ecole Polytechnique F\u00e9d\u00e9rale de Lausanne (EPFL), Lausanne, Switzerland, 2MIT Computer Science and Artificial Intelligence Laboratory, Cambridge and 3Broad Institute of MIT and Harvard, Cambridge, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dario","family":"Floreano","sequence":"additional","affiliation":[{"name":"1 Laboratory of Intelligent Systems, Ecole Polytechnique F\u00e9d\u00e9rale de Lausanne (EPFL), Lausanne, Switzerland, 2MIT Computer Science and Artificial Intelligence Laboratory, Cambridge and 3Broad Institute of MIT and Harvard, Cambridge, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2011,6,22]]},"reference":[{"key":"2023012511513191100_B1","doi-asserted-by":"crossref","first-page":"1129","DOI":"10.1073\/pnas.79.4.1129","article-title":"Quantitative model for gene regulation by lambda phage repressor","volume":"79","author":"Ackers","year":"1982","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"2023012511513191100_B2","doi-asserted-by":"crossref","first-page":"2937","DOI":"10.1093\/bioinformatics\/btp511","article-title":"Learning gene regulatory networks from gene expression measurements using non-parametric molecular kinetics","volume":"25","author":"\u00c4ij\u00f6","year":"2009","journal-title":"Bioinformatics"},{"key":"2023012511513191100_B3","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1038\/msb4100120","article-title":"How to infer gene networks from expression profiles","volume":"3","author":"Bansal","year":"2007","journal-title":"Molecular Syst. Biol."},{"key":"2023012511513191100_B4","doi-asserted-by":"crossref","first-page":"590","DOI":"10.1038\/35014651","article-title":"Engineering stability in gene networks by autoregulation","volume":"405","author":"Becskei","year":"2000","journal-title":"Nature"},{"key":"2023012511513191100_B5","doi-asserted-by":"crossref","first-page":"13004","DOI":"10.1073\/pnas.0605420103","article-title":"Quantification of protein half-lives in the budding yeast proteome","volume":"103","author":"Belle","year":"2006","journal-title":"Proc. Natl Acad. Sci."},{"key":"2023012511513191100_B6","doi-asserted-by":"crossref","first-page":"R36","DOI":"10.1186\/gb-2006-7-5-r36","article-title":"The Inferelator: an algorithm for learning parsimonious regulatory networks from systems-biology data sets de novo","volume":"7","author":"Bonneau","year":"2006","journal-title":"Genome Biol."},{"key":"2023012511513191100_B7","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1016\/j.cell.2009.03.032","article-title":"Systems biology strikes gold","volume":"137","author":"Camacho","year":"2009","journal-title":"Cell"},{"key":"2023012511513191100_B8","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1016\/j.cell.2009.01.055","article-title":"A yeast synthetic network for in vivo assessment of reverse-engineering and modeling approaches","volume":"137","author":"Cantone","year":"2009","journal-title":"Cell"},{"key":"2023012511513191100_B9","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1145\/1143844.1143874","article-title":"The relationship between precision-recall and roc curves","volume-title":"Proceedings of the 23rd International Conference on Machine Learning.","author":"Davis","year":"2006"},{"key":"2023012511513191100_B10","doi-asserted-by":"crossref","first-page":"467","DOI":"10.1126\/science.270.5235.467","article-title":"Quantitative monitoring of gene expression patterns with a complementary DNA microarray","volume":"270","author":"Davis","year":"1995","journal-title":"Science"},{"key":"2023012511513191100_B11","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1111\/j.1749-6632.2008.03756.x","article-title":"A gene network simulator to assess reverse engineering algorithms","volume":"1158","author":"Di Camillo","year":"2009","journal-title":"Ann. N Y Acad. Sci."},{"key":"2023012511513191100_B12","doi-asserted-by":"crossref","first-page":"e8","DOI":"10.1371\/journal.pbio.0050008","article-title":"Large-scale mapping and validation of Escherichia coli transcriptional regulation from a compendium of expression profiles","volume":"5","author":"Faith","year":"2007","journal-title":"PLoS Biol."},{"issue":"Suppl. 1","key":"2023012511513191100_B13","doi-asserted-by":"crossref","first-page":"D98","DOI":"10.1093\/nar\/gkq1110","article-title":"RegulonDB version 7.0: transcriptional regulation of Escherichia coli K-12 integrated within genetic sensory response units (Gensor Units)","volume":"39","author":"Gama-Castro","year":"2011","journal-title":"Nucleic Acids Res."},{"key":"2023012511513191100_B14","doi-asserted-by":"crossref","first-page":"520","DOI":"10.1038\/35014708","article-title":"Neutralizing noise in gene networks","volume":"405","author":"Gardner","year":"2000","journal-title":"Nature"},{"key":"2023012511513191100_B15","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1063\/1.481811","article-title":"The chemical Langevin equation","volume":"113","author":"Gillespie","year":"2000","journal-title":"J. Chem. Phys."},{"key":"2023012511513191100_B16","doi-asserted-by":"crossref","first-page":"1205","DOI":"10.1093\/bioinformatics\/btp115","article-title":"GeNGe: systematic generation of gene regulatory networks","volume":"25","author":"Hache","year":"2009","journal-title":"Bioinformatics"},{"key":"2023012511513191100_B17","doi-asserted-by":"crossref","first-page":"801","DOI":"10.1093\/bioinformatics\/btp068","article-title":"Benchmarking regulatory network reconstruction with GRENDEL","volume":"25","author":"Haynes","year":"2009","journal-title":"Bioinformatics"},{"key":"2023012511513191100_B18","doi-asserted-by":"crossref","first-page":"e12776","DOI":"10.1371\/journal.pone.0012776","article-title":"Inferring Regulatory Networks from Expression Data Using Tree-Based Methods","volume":"5","author":"Huynh-Thu","year":"2010","journal-title":"PLoS One"},{"key":"2023012511513191100_B19","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1093\/bib\/4.3.228","article-title":"Inferring gene networks from time series microarray data using dynamic Bayesian networks","volume":"4","author":"Kim","year":"2003","journal-title":"Brief. Bioinformatics"},{"key":"2023012511513191100_B20","doi-asserted-by":"crossref","first-page":"2160","DOI":"10.1093\/bioinformatics\/btq342","article-title":"TRANSWESD: inferring cellular networks with transitive reduction","volume":"26","author":"Klamt","year":"2010","journal-title":"Bioinformatics"},{"key":"2023012511513191100_B21","doi-asserted-by":"crossref","first-page":"1773","DOI":"10.1101\/gr.1226004","article-title":"A benchmark for methods in reverse engineering and model discrimination: problem formulation and solutions","volume":"14","author":"Kremling","year":"2004","journal-title":"Genome Res."},{"key":"2023012511513191100_B22","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1016\/j.ygeno.2009.08.009","article-title":"ReTRN: A retriever of real transcriptional regulatory network and expression data for evaluating structure learning algorithm","volume":"94","author":"Li","year":"2009","journal-title":"Genomics"},{"key":"2023012511513191100_B23","doi-asserted-by":"crossref","first-page":"1675","DOI":"10.1038\/nbt1296-1675","article-title":"Expression monitoring by hybridization to high-density oligonucleotide arrays","volume":"14","author":"Lockhart","year":"1996","journal-title":"Nat. Biotechnol."},{"key":"2023012511513191100_B24","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1089\/cmb.2008.09TT","article-title":"Generating realistic in silico gene networks for performance assessment of reverse engineering methods","volume":"16","author":"Marbach","year":"2009","journal-title":"J. Comput. Biol."},{"key":"2023012511513191100_B25","doi-asserted-by":"crossref","first-page":"6286","DOI":"10.1073\/pnas.0913357107","article-title":"Revealing strengths and weaknesses of methods for gene network inference","volume":"107","author":"Marbach","year":"2010","journal-title":"Proc. Natl Acad. Sci. USA"},{"issue":"Suppl. 1","key":"2023012511513191100_B26","doi-asserted-by":"crossref","first-page":"S7","DOI":"10.1186\/1471-2105-7-S1-S7","article-title":"ARACNE: an algorithm for the reconstruction of gene regulatory networks in a mammalian cellular context","volume":"7","author":"Margolin","year":"2006","journal-title":"BMC Bioinformatics"},{"issue":"Suppl. 2","key":"2023012511513191100_B27","doi-asserted-by":"crossref","first-page":"ii122","DOI":"10.1093\/bioinformatics\/btg1069","article-title":"Artificial gene networks for objective comparison of analysis algorithms","volume":"19","author":"Mendes","year":"2003","journal-title":"Bioinformatics"},{"key":"2023012511513191100_B28","doi-asserted-by":"crossref","first-page":"e14147","DOI":"10.1371\/journal.pone.0014147","article-title":"Gene regulatory networks from multifactorial perturbations using graphical lasso: application to the dream4 challenge","volume":"5","author":"Men\u00e9ndez","year":"2010","journal-title":"PloS One"},{"key":"2023012511513191100_B29","doi-asserted-by":"crossref","first-page":"621","DOI":"10.1038\/nmeth.1226","article-title":"Mapping and quantifying mammalian transcriptomes by RNA-Seq","volume":"5","author":"Mortazavi","year":"2008","journal-title":"Nat. Methods"},{"key":"2023012511513191100_B30","doi-asserted-by":"crossref","first-page":"218","DOI":"10.1371\/journal.pone.0012912","article-title":"From knockouts to networks: establishing direct cause-effect relationships through graph analysis","volume":"5","author":"Pinna","year":"2010","journal-title":"PloS One"},{"key":"2023012511513191100_B31","doi-asserted-by":"crossref","first-page":"e9202","DOI":"10.1371\/journal.pone.0009202","article-title":"Towards a rigorous assessment of systems biology models: the DREAM3 challenges","volume":"5","author":"Prill","year":"2010","journal-title":"PloS One"},{"key":"2023012511513191100_B32","doi-asserted-by":"crossref","first-page":"1551","DOI":"10.1126\/science.1073374","article-title":"Hierarchical organization of modularity in metabolic networks","volume":"297","author":"Ravasz","year":"2002","journal-title":"Science"},{"key":"2023012511513191100_B33","doi-asserted-by":"crossref","first-page":"765","DOI":"10.1093\/bioinformatics\/bti064","article-title":"Reconstructing biological networks using conditional correlation analysis","volume":"21","author":"Rice","year":"2005","journal-title":"Bioinformatics"},{"key":"2023012511513191100_B34","doi-asserted-by":"crossref","first-page":"1318","DOI":"10.1093\/bioinformatics\/btn126","article-title":"A system for generating transcription regulatory networks with combinatorial control of transcription","volume":"24","author":"Roy","year":"2008","journal-title":"Bioinformatics"},{"key":"2023012511513191100_B35","article-title":"Numerical integration of SDEs: a short tutorial","volume-title":"Technical Report LIS-REPORT-2010-001","author":"Schaffter","year":"2010"},{"key":"2023012511513191100_B36","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1038\/ng881","article-title":"Network motifs in the transcriptional regulation network of Escherichia coli","volume":"31","author":"Shen-Orr","year":"2002","journal-title":"Nat. Genet."},{"key":"2023012511513191100_B37","doi-asserted-by":"crossref","first-page":"1402","DOI":"10.1073\/pnas.0406555102","article-title":"Statistical analysis of MPSS measurements: application to the study of LPS-activated macrophage gene expression","volume":"102","author":"Stolovitzky","year":"2005","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"2023012511513191100_B38","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1186\/1471-2105-7-43","article-title":"SynTReN: a generator of synthetic gene expression data for design and analysis of structure learning algorithms","volume":"7","author":"Van den Bulcke","year":"2006","journal-title":"BMC Bioinformatics"},{"key":"2023012511513191100_B39","doi-asserted-by":"crossref","first-page":"S19","DOI":"10.1186\/1471-2105-11-S6-S19","article-title":"Time lagged information theoretic approaches to the reverse engineering of gene regulatory networks","volume":"11","author":"Vijender","year":"2010","journal-title":"BMC Bioinformatics"},{"key":"2023012511513191100_B40","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pone.0008121","article-title":"Improved reconstruction of in silico gene regulatory networks by integrating knockout and perturbation data","volume":"5","author":"Yip","year":"2010","journal-title":"PloS One"},{"key":"2023012511513191100_B41","doi-asserted-by":"crossref","first-page":"3594","DOI":"10.1093\/bioinformatics\/bth448","article-title":"Advances to Bayesian network inference for generating causal networks from observational biological data","volume":"20","author":"Yu","year":"2004","journal-title":"Bioinformatics"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/27\/16\/2263\/48863257\/bioinformatics_27_16_2263.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/27\/16\/2263\/48863257\/bioinformatics_27_16_2263.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,7]],"date-time":"2024-04-07T18:20:33Z","timestamp":1712514033000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/27\/16\/2263\/254752"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2011,6,22]]},"references-count":41,"journal-issue":{"issue":"16","published-print":{"date-parts":[[2011,8,15]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btr373","relation":{"has-review":[{"id-type":"doi","id":"10.3410\/f.12002958.13156058","asserted-by":"object"}]},"ISSN":["1367-4811","1367-4803"],"issn-type":[{"value":"1367-4811","type":"electronic"},{"value":"1367-4803","type":"print"}],"subject":[],"published-other":{"date-parts":[[2011,8,15]]},"published":{"date-parts":[[2011,6,22]]}}}