{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,5]],"date-time":"2026-03-05T11:02:58Z","timestamp":1772708578021,"version":"3.50.1"},"reference-count":35,"publisher":"Oxford University Press (OUP)","issue":"9","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2006,5,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Motivation: In recent years, there have been various efforts to overcome the limitations of standard clustering approaches for the analysis of gene expression data by grouping genes and samples simultaneously. The underlying concept, which is often referred to as biclustering, allows to identify sets of genes sharing compatible expression patterns across subsets of samples, and its usefulness has been demonstrated for different organisms and datasets. Several biclustering methods have been proposed in the literature; however, it is not clear how the different techniques compare with each other with respect to the biological relevance of the clusters as well as with other characteristics such as robustness and sensitivity to noise. Accordingly, no guidelines concerning the choice of the biclustering method are currently available.<\/jats:p>\n                  <jats:p>Results: First, this paper provides a methodology for comparing and validating biclustering methods that includes a simple binary reference model. Although this model captures the essential features of most biclustering approaches, it is still simple enough to exactly determine all optimal groupings; to this end, we propose a fast divide-and-conquer algorithm (Bimax). Second, we evaluate the performance of five salient biclustering algorithms together with the reference model and a hierarchical clustering method on various synthetic and real datasets for Saccharomyces cerevisiae and Arabidopsis thaliana. The comparison reveals that (1) biclustering in general has advantages over a conventional hierarchical clustering approach, (2) there are considerable performance differences between the tested methods and (3) already the simple reference model delivers relevant patterns within all considered settings.<\/jats:p>\n                  <jats:p>Availability: The datasets used, the outcomes of the biclustering algorithms and the Bimax implementation for the reference model are available at<\/jats:p>\n                  <jats:p>Contact: \u00a0bleuler@tik.ee.ethz.ch<\/jats:p>\n                  <jats:p>Supplementary information: \u00a0Supplementary data are available at<\/jats:p>","DOI":"10.1093\/bioinformatics\/btl060","type":"journal-article","created":{"date-parts":[[2006,2,24]],"date-time":"2006-02-24T20:14:23Z","timestamp":1140812063000},"page":"1122-1129","source":"Crossref","is-referenced-by-count":684,"title":["A systematic comparison and evaluation of biclustering methods for gene expression data"],"prefix":"10.1093","volume":"22","author":[{"given":"Amela","family":"Preli\u0107","sequence":"first","affiliation":[{"name":"Computer Engineering and Networks Laboratory 1 \u00a0 1 \u00a0 \u00a0 ETH Zurich, 8092 Zurich, Switzerland"}]},{"given":"Stefan","family":"Bleuler","sequence":"additional","affiliation":[{"name":"Computer Engineering and Networks Laboratory 1 \u00a0 1 \u00a0 \u00a0 ETH Zurich, 8092 Zurich, Switzerland"}]},{"given":"Philip","family":"Zimmermann","sequence":"additional","affiliation":[{"name":"Institute for Plant Sciences and Functional Genomics Center Zurich 2 \u00a0 2 \u00a0 \u00a0 ETH Zurich, 8092 Zurich, Switzerland"}]},{"given":"Anja","family":"Wille","sequence":"additional","affiliation":[{"name":"Colab 3 \u00a0 3 \u00a0 \u00a0 ETH Zurich, 8092 Zurich, Switzerland"},{"name":"Seminar for Statistics 4 \u00a0 4 \u00a0 \u00a0 ETH Zurich, 8092 Zurich, Switzerland"}]},{"given":"Peter","family":"B\u00fchlmann","sequence":"additional","affiliation":[{"name":"Seminar for Statistics 4 \u00a0 4 \u00a0 \u00a0 ETH Zurich, 8092 Zurich, Switzerland"}]},{"given":"Wilhelm","family":"Gruissem","sequence":"additional","affiliation":[{"name":"Institute for Plant Sciences and Functional Genomics Center Zurich 2 \u00a0 2 \u00a0 \u00a0 ETH Zurich, 8092 Zurich, Switzerland"}]},{"given":"Lars","family":"Hennig","sequence":"additional","affiliation":[{"name":"Institute for Plant Sciences and Functional Genomics Center Zurich 2 \u00a0 2 \u00a0 \u00a0 ETH Zurich, 8092 Zurich, Switzerland"}]},{"given":"Lothar","family":"Thiele","sequence":"additional","affiliation":[{"name":"Computer Engineering and Networks Laboratory 1 \u00a0 1 \u00a0 \u00a0 ETH Zurich, 8092 Zurich, Switzerland"}]},{"given":"Eckart","family":"Zitzler","sequence":"additional","affiliation":[{"name":"Computer Engineering and Networks Laboratory 1 \u00a0 1 \u00a0 \u00a0 ETH Zurich, 8092 Zurich, Switzerland"}]}],"member":"286","published-online":{"date-parts":[[2006,2,24]]},"reference":[{"key":"2023012409134188600_b1","article-title":"Consensus algorithms for the generation of all maximal bicliques","volume-title":"Technical Report TF-DIMACS-2002-52","author":"Alexe","year":"2002"},{"key":"2023012409134188600_b2","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1093\/bioinformatics\/18.2.319","article-title":"A cluster validity framework for genome expression data","volume":"18","author":"Azuaje","year":"2002","journal-title":"Bioinformatics"},{"key":"2023012409134188600_b3","first-page":"49","article-title":"Discovering local structure in gene expression data: the order-preserving sub-matrix problem","author":"Ben-Dor","year":"2002"},{"key":"2023012409134188600_b4","doi-asserted-by":"crossref","first-page":"031902","DOI":"10.1103\/PhysRevE.67.031902","article-title":"Iterative signature algorithm for the analysis of large-scale gene expression data","volume":"67","author":"Bergmann","year":"2003","journal-title":"Phys. Rev. E Stat. Nonlin. Soft Matter Phys."},{"key":"2023012409134188600_b5","doi-asserted-by":"crossref","first-page":"2502","DOI":"10.1093\/bioinformatics\/btg363","article-title":"Characterizing gene sets with FuncAssociate","volume":"19","author":"Berriz","year":"2003","journal-title":"BioInformatics"},{"key":"2023012409134188600_b6","first-page":"93","article-title":"Biclustering of expression data","volume-title":"Proc. Int. Conf. Intell. Syst. Mol. Biol.","author":"Cheng","year":"2000"},{"key":"2023012409134188600_b7","doi-asserted-by":"crossref","first-page":"459","DOI":"10.1093\/bioinformatics\/btg025","article-title":"Comparisons and validation of statistical clustering techniques for microarray gene expression data","volume":"19","author":"Datta","year":"2003","journal-title":"Bioinformatics"},{"key":"2023012409134188600_b8","first-page":"25","article-title":"Gene Ontology: tool for the unification of biology. The Gene Ontology Consortium","volume-title":"Nat. Genet.","author":"Gene Ontology Consortium","year":"2000"},{"key":"2023012409134188600_b9","doi-asserted-by":"crossref","first-page":"12079","DOI":"10.1073\/pnas.210134797","article-title":"Coupled two-way clustering analysis of gene microarray data","volume":"97","author":"Getz","year":"2000","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"2023012409134188600_b10","doi-asserted-by":"crossref","first-page":"1079","DOI":"10.1093\/bioinformatics\/btf876","article-title":"Coupled two-way clustering analysis of breast cancer and colon cancer gene expression data","volume":"19","author":"Getz","year":"2003","journal-title":"Bioinformatics"},{"key":"2023012409134188600_b11","doi-asserted-by":"crossref","first-page":"689","DOI":"10.1093\/bioinformatics\/btg469","article-title":"A statistical framework for combining and interpreting proteomic datasets","volume":"20","author":"Gilchrist","year":"2004","journal-title":"Bioinformatics"},{"key":"2023012409134188600_b12","doi-asserted-by":"crossref","first-page":"4241","DOI":"10.1091\/mbc.11.12.4241","article-title":"Genomic expression programs in the response of yeast cells to environmental changes","volume":"11","author":"Gasch","year":"2000","journal-title":"Mol. Biol. Cell"},{"key":"2023012409134188600_b13","doi-asserted-by":"crossref","first-page":"2381","DOI":"10.1093\/bioinformatics\/btg330","article-title":"Scoring clustering solutions by their biological relevance","volume":"19","author":"Gat-Viks","year":"2003","journal-title":"Bioinformatics"},{"key":"2023012409134188600_b14","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1023\/A:1012801612483","article-title":"On clustering validation techniques","volume":"17","author":"Halkidi","year":"2001","journal-title":"J. Intell. Inform. Syst."},{"key":"2023012409134188600_b15","doi-asserted-by":"crossref","first-page":"3201","DOI":"10.1093\/bioinformatics\/bti517","article-title":"Computational cluster validation in post-genomic data analysis","volume":"21","author":"Handl","year":"2005","journal-title":"Bioinformatics"},{"key":"2023012409134188600_b16","doi-asserted-by":"crossref","first-page":"100","DOI":"10.2307\/2346830","article-title":"A k-means clustering algorithm","volume":"28","author":"Hartigan","year":"1979","journal-title":"Appl. Stat."},{"key":"2023012409134188600_b17","doi-asserted-by":"crossref","first-page":"S233","DOI":"10.1093\/bioinformatics\/18.suppl_1.S233","article-title":"Discovering regulatory and signaling circuits in molecular interaction networks","volume":"18","author":"Ideker","year":"2002","journal-title":"Bioinformatics"},{"key":"2023012409134188600_b18","doi-asserted-by":"crossref","first-page":"370","DOI":"10.1038\/ng941","article-title":"Revealing modular organization in the yeast transcriptional network","volume":"31","author":"Ihmels","year":"2002","journal-title":"Nat. Genet."},{"key":"2023012409134188600_b19","doi-asserted-by":"crossref","first-page":"1993","DOI":"10.1093\/bioinformatics\/bth166","article-title":"Defining transcription modules using large-scale gene expression data","volume":"20","author":"Ihmels","year":"2004","journal-title":"Bioinformatics"},{"key":"2023012409134188600_b20","doi-asserted-by":"crossref","first-page":"8961","DOI":"10.1073\/pnas.161273698","article-title":"Bootstrapping cluster analysis: assessing the reliability of conclusions from microarray experiments","volume":"98","author":"Kerr","year":"2001","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"2023012409134188600_b21","doi-asserted-by":"crossref","first-page":"703","DOI":"10.1101\/gr.648603","article-title":"Spectral biclustering of microarray cancer data: co-clustering genes and conditions","volume":"13","author":"Kluger","year":"2003","journal-title":"Genome Res."},{"key":"2023012409134188600_b22","first-page":"187","article-title":"OP-clusters: clustering by tendency in high dimensional space","author":"Liu","year":"2003"},{"key":"2023012409134188600_b23","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1109\/TCBB.2004.2","article-title":"Biclustering algorithms for biological data analysis: a survey","volume":"1","author":"Madeira","year":"2004","journal-title":"IEEE\/ACM Trans. Comput. Biol. Bioinform."},{"key":"2023012409134188600_b24","first-page":"77","article-title":"Extracting conserved gene expression motifs from gene expression data","volume":"8","author":"Murali","year":"2003","journal-title":"Pac. Symp. Biocomput."},{"key":"2023012409134188600_b25","doi-asserted-by":"crossref","first-page":"1787","DOI":"10.1093\/bioinformatics\/btg232","article-title":"CLICK and EXPANDER: a system for clustering and visualizing gene expression data","volume":"14","author":"Sharan","year":"2003","journal-title":"Bioinformatics"},{"key":"2023012409134188600_b26","first-page":"1409","article-title":"A statistical method for evaluating systematic relationships","volume":"38","author":"Sokal","year":"1958","journal-title":"Univ. Kansas Sci. Bull."},{"key":"2023012409134188600_b27","doi-asserted-by":"crossref","first-page":"D449","DOI":"10.1093\/nar\/gkh086","article-title":"The Database of Interacting Proteins: 2004 update","volume":"32","author":"Salwinski","year":"2004","journal-title":"Nucleic Acids Res."},{"key":"2023012409134188600_b28","doi-asserted-by":"crossref","first-page":"S136","DOI":"10.1093\/bioinformatics\/18.suppl_1.S136","article-title":"Discovering statistically significant biclusters in gene expression data","volume":"18","author":"Tanay","year":"2002","journal-title":"Bioinformatics"},{"key":"2023012409134188600_b29","doi-asserted-by":"crossref","first-page":"2981","DOI":"10.1073\/pnas.0308661100","article-title":"Revealing modularity and organization in the yeast molecular network by integrated analysis of highly heterogeneous genomewide data","volume":"101","author":"Tanay","year":"2004","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"2023012409134188600_b30","volume-title":"Resampling-Based Multiple Testing","author":"Westfall","year":"1993"},{"key":"2023012409134188600_b31","doi-asserted-by":"crossref","first-page":"R92","DOI":"10.1186\/gb-2004-5-11-r92","article-title":"Sparse graphical Gaussian modeling of the isoprenoid gene network in Arabidopsis thaliana","volume":"5","author":"Wille","year":"2004","journal-title":"Genome Biol."},{"key":"2023012409134188600_b32","first-page":"517","article-title":"Delta-clusters: capturing subspace correlation in a large data set","author":"Yang","year":"2002"},{"key":"2023012409134188600_b33","first-page":"321","article-title":"Enhanced biclustering on expression data","author":"Yang","year":"2003"},{"key":"2023012409134188600_b34","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1093\/bioinformatics\/17.4.309","article-title":"Validating clustering for gene expression data","volume":"17","author":"Yeung","year":"2001","journal-title":"Bioinformatics"},{"key":"2023012409134188600_b35","first-page":"407","article-title":"Analysis of gene expression data with pathway scores","volume":"8","author":"Zien","year":"2000","journal-title":"Proc. Int. Conf. Intell. Syst. Mol. Biol."}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/22\/9\/1122\/48839984\/bioinformatics_22_9_1122.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/22\/9\/1122\/48839984\/bioinformatics_22_9_1122.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,24]],"date-time":"2023-01-24T04:52:16Z","timestamp":1674535936000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/22\/9\/1122\/200492"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2006,2,24]]},"references-count":35,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2006,5,1]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btl060","relation":{"has-review":[{"id-type":"doi","id":"10.3410\/f.1087930.540878","asserted-by":"object"}]},"ISSN":["1367-4811","1367-4803"],"issn-type":[{"value":"1367-4811","type":"electronic"},{"value":"1367-4803","type":"print"}],"subject":[],"published-other":{"date-parts":[[2006,5,1]]},"published":{"date-parts":[[2006,2,24]]}}}