{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T14:54:56Z","timestamp":1775660096270,"version":"3.50.1"},"reference-count":41,"publisher":"Oxford University Press (OUP)","issue":"13","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2015,7,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Motivation: Comparing time courses of gene expression with time courses of phenotypic data may provide new insights in cellular mechanisms. In this study, we compared the performance of five pattern recognition methods with respect to their ability to relate genes and phenotypic data: one classical method (k-means) and four methods especially developed for time series [Short Time-series Expression Miner (STEM), Linear Mixed Model mixtures, Dynamic Time Warping for -Omics and linear modeling with R\/Bioconductor limma package]. The methods were evaluated using data available from toxicological studies that had the aim to relate gene expression with phenotypic endpoints (i.e. to develop biomarkers for adverse outcomes). Additionally, technical aspects (influence of noise, number of time points and number of replicates) were evaluated on simulated data.<\/jats:p>\n               <jats:p>Results: None of the methods outperforms the others in terms of biology. Linear modeling with limma is mostly influenced by noise. STEM is mostly influenced by the number of biological replicates in the dataset, whereas k-means and linear modeling with limma are mostly influenced by the number of time points. In most cases, the results of the methods complement each other. We therefore provide recommendations to integrate the five methods.<\/jats:p>\n               <jats:p>Availability: The Matlab code for the simulations performed in this research is available in the Supplementary Data (Word file). The microarray data analysed in this paper are available at ArrayExpress (E-TOXM-22 and E-TOXM-23) and Gene Expression Omnibus (GSE39291). The phenotypic data are available in the Supplementary Data (Excel file). Links to the pattern recognition tools compared in this paper are provided in the main text.<\/jats:p>\n               <jats:p>Contact: \u00a0d.hendrickx@maastrichtuniversity.nl<\/jats:p>\n               <jats:p>Supplementary information: \u00a0Supplementary data are available at Bioinformatics online.<\/jats:p>","DOI":"10.1093\/bioinformatics\/btv108","type":"journal-article","created":{"date-parts":[[2015,2,21]],"date-time":"2015-02-21T15:10:09Z","timestamp":1424531409000},"page":"2115-2122","source":"Crossref","is-referenced-by-count":8,"title":["Pattern recognition methods to relate time profiles of gene expression with phenotypic data: a comparative study"],"prefix":"10.1093","volume":"31","author":[{"given":"Diana M.","family":"Hendrickx","sequence":"first","affiliation":[{"name":"Department of Toxicogenomics, School of Oncology and Developmental Biology (GROW), Maastricht University, 6200 MD Maastricht, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Danyel G. J.","family":"Jennen","sequence":"additional","affiliation":[{"name":"Department of Toxicogenomics, School of Oncology and Developmental Biology (GROW), Maastricht University, 6200 MD Maastricht, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jacob J.","family":"Bried\u00e9","sequence":"additional","affiliation":[{"name":"Department of Toxicogenomics, School of Oncology and Developmental Biology (GROW), Maastricht University, 6200 MD Maastricht, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rachel","family":"Cavill","sequence":"additional","affiliation":[{"name":"Department of Toxicogenomics, School of Oncology and Developmental Biology (GROW), Maastricht University, 6200 MD Maastricht, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Theo M.","family":"de Kok","sequence":"additional","affiliation":[{"name":"Department of Toxicogenomics, School of Oncology and Developmental Biology (GROW), Maastricht University, 6200 MD Maastricht, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jos C. S.","family":"Kleinjans","sequence":"additional","affiliation":[{"name":"Department of Toxicogenomics, School of Oncology and Developmental Biology (GROW), Maastricht University, 6200 MD Maastricht, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2015,2,19]]},"reference":[{"key":"2023020202145252000_btv108-B1","doi-asserted-by":"crossref","first-page":"1226","DOI":"10.1021\/pr900723c","article-title":"In\u00a0vitro neurotoxicity of PBDE-99: immediate and concentration-dependent effects on protein expression in cerebral cortex cells","volume":"9","author":"Alm","year":"2009","journal-title":"J. Proteome Res."},{"key":"2023020202145252000_btv108-B2","doi-asserted-by":"crossref","first-page":"2493","DOI":"10.1093\/bioinformatics\/bth283","article-title":"Analyzing time series gene expression data","volume":"20","author":"Bar-Joseph","year":"2004","journal-title":"Bioinformatics"},{"key":"2023020202145252000_btv108-B3","doi-asserted-by":"crossref","first-page":"552","DOI":"10.1038\/nrg3244","article-title":"Studying and modelling dynamic biological processes using time-series gene expression data","volume":"13","author":"Bar-Joseph","year":"2012","journal-title":"Nat. Rev. Genet."},{"key":"2023020202145252000_btv108-B4","first-page":"552","article-title":"Benzo[a]pyrene-dependent activation of transcription factors NF-kappaB and AP-1 related to tumor promotion in hepatoma cell cultures","volume":"72","author":"Bolotina","year":"2007","journal-title":"Biochemistry"},{"key":"2023020202145252000_btv108-B5","first-page":"451","article-title":"Cancer risk assessment, indicators, and guidelines for polycyclic aromatic hydrocarbons in the ambient air","volume":"110","author":"Bostrom","year":"2002","journal-title":"Environ. Health Perspect."},{"key":"2023020202145252000_btv108-B6","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1093\/toxsci\/kfp309","article-title":"Global gene expression analysis reveals differences in cellular responses to hydroxyl- and superoxide anion radical-induced oxidative stress in caco-2 cells","volume":"114","author":"Briede","year":"2010","journal-title":"Toxicol. Sci."},{"key":"2023020202145252000_btv108-B7","doi-asserted-by":"crossref","first-page":"e71823","DOI":"10.1371\/journal.pone.0071823","article-title":"DTW4Omics: comparing patterns in biological time series","volume":"8","author":"Cavill","year":"2013","journal-title":"PLoS one"},{"key":"2023020202145252000_btv108-B8","first-page":"4566","article-title":"Mixture of linear mixed models. Application to repeated data clustering","author":"Celeux","year":"2002","journal-title":"Inria Research Report"},{"key":"2023020202145252000_btv108-B9","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1191\/1471082X05st096oa","article-title":"Mixture of linear mixed models for clustering gene expression profiles from repeated microarray experiments","volume":"5","author":"Celeux","year":"2005","journal-title":"Stat. Model."},{"key":"2023020202145252000_btv108-B10","doi-asserted-by":"crossref","first-page":"D1104","DOI":"10.1093\/nar\/gks994","article-title":"The comparative toxicogenomics database: update 2013","volume":"41","author":"Davis","year":"2013","journal-title":"Nucleic Acids Res."},{"key":"2023020202145252000_btv108-B11","doi-asserted-by":"crossref","first-page":"633","DOI":"10.1093\/bib\/bbt020","article-title":"Pattern recognition in bioinformatics","volume":"14","author":"de Ridder","year":"2013","journal-title":"Brief Bioinform."},{"key":"2023020202145252000_btv108-B12","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1016\/j.tox.2013.02.001","article-title":"Time series analysis of oxidative stress response patterns in HepG2: a toxicogenomics approach","volume":"306","author":"Deferme","year":"2013","journal-title":"Toxicology"},{"key":"2023020202145252000_btv108-B13","doi-asserted-by":"crossref","first-page":"e82340","DOI":"10.1371\/journal.pone.0082340","article-title":"Analysis of time-resolved gene expression measurements across individuals","volume":"8","author":"Elo","year":"2013","journal-title":"PLoS one"},{"key":"2023020202145252000_btv108-B14","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1186\/1471-2105-7-191","article-title":"STEM: a tool for the analysis of short time series gene expression data","volume":"7","author":"Ernst","year":"2006","journal-title":"BMC bioinformatics"},{"key":"2023020202145252000_btv108-B15","doi-asserted-by":"crossref","first-page":"i159","DOI":"10.1093\/bioinformatics\/bti1022","article-title":"Clustering short time series gene expression data","volume":"21","author":"Ernst","year":"2005","journal-title":"Bioinformatics"},{"key":"2023020202145252000_btv108-B16","doi-asserted-by":"crossref","first-page":"35","DOI":"10.2217\/14622416.9.1.35","article-title":"Pathway analysis tools and toxicogenomics reference databases for risk assessment","volume":"9","author":"Ganter","year":"2008","journal-title":"Pharmacogenomics"},{"key":"2023020202145252000_btv108-B17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v031.i07","article-title":"Computing and visualizing dynamic time warping alignments in R: the dtw package","volume":"31","author":"Giorgino","year":"2009","journal-title":"J. Stat. Softw."},{"key":"2023020202145252000_btv108-B18","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v028.i04","article-title":"FlexMix version 2: finite mixtures with concomitant variables and varying and constant parameters","volume":"28","author":"Gr\u00fcn","year":"2008","journal-title":"J. Stat. Softw."},{"key":"2023020202145252000_btv108-B19","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1016\/S0092-8674(00)81683-9","article-title":"The hallmarks of cancer","volume":"100","author":"Hanahan","year":"2000","journal-title":"Cell"},{"key":"2023020202145252000_btv108-B20","doi-asserted-by":"crossref","first-page":"646","DOI":"10.1016\/j.cell.2011.02.013","article-title":"Hallmarks of cancer: the next generation","volume":"144","author":"Hanahan","year":"2011","journal-title":"Cell"},{"key":"2023020202145252000_btv108-B21","doi-asserted-by":"crossref","first-page":"480","DOI":"10.1289\/ehp.1205657","article-title":"Performance in omics analyses of blood samples in long-term storage: opportunities for the exploitation of existing biobanks in environmental health research","volume":"121","author":"Hebels","year":"2013","journal-title":"Environ. Health Perspect"},{"key":"2023020202145252000_btv108-B22","doi-asserted-by":"crossref","first-page":"D793","DOI":"10.1093\/nar\/gks1055","article-title":"The ConsensusPathDB interaction database: 2013 update","volume":"41","author":"Kamburov","year":"2013","journal-title":"Nucleic Acids Res."},{"key":"2023020202145252000_btv108-B23","doi-asserted-by":"crossref","first-page":"31","DOI":"10.2307\/280052","article-title":"Heuristic approaches to spatial analysis in archaeology","volume":"47","author":"Kintigh","year":"1982","journal-title":"Am. Antiq."},{"key":"2023020202145252000_btv108-B24","doi-asserted-by":"crossref","first-page":"559","DOI":"10.1186\/1471-2105-9-559","article-title":"WGCNA: an R package for weighted correlation network analysis","volume":"9","author":"Langfelder","year":"2008","journal-title":"BMC Bioinformatics"},{"key":"2023020202145252000_btv108-B25","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v011.i08","article-title":"FlexMix: a general framework for finite mixture models and latent class regression in R","volume":"11","author":"Leisch","year":"2004","journal-title":"J. Stat. Softw."},{"key":"2023020202145252000_btv108-B26","doi-asserted-by":"crossref","first-page":"1857","DOI":"10.1016\/j.patcog.2005.01.025","article-title":"Clustering of time series data\u2014a survey","volume":"38","author":"Liao","year":"2005","journal-title":"Pattern Recogn."},{"key":"2023020202145252000_btv108-B27","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1002\/jcp.10119","article-title":"Cellular response to oxidative stress: signaling for suicide and survival","volume":"192","author":"Martindale","year":"2002","journal-title":"J. Cell Physiol."},{"key":"2023020202145252000_btv108-B28","doi-asserted-by":"crossref","first-page":"ii122","DOI":"10.1093\/bioinformatics\/btg1069","article-title":"Artificial gene networks for objective comparison of analysis algorithms","volume":"19","author":"Mendes","year":"2003","journal-title":"Bioinformatics"},{"key":"2023020202145252000_btv108-B29","first-page":"849","article-title":"On spectral clustering: analysis and an algorithm","volume-title":"Proceedings of Advances in Neural Information Processing Systems","author":"Ng","year":"2001"},{"key":"2023020202145252000_btv108-B30","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1186\/1471-2105-7-349","article-title":"Simulation of microarray data with realistic characteristics","volume":"7","author":"Nykter","year":"2006","journal-title":"BMC Bioinformatics"},{"key":"2023020202145252000_btv108-B31","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1093\/toxsci\/kfl030","article-title":"Phenotypic anchoring of acetominophen-induced oxidative stress with gene expression profiles in rat liver","volume":"93","author":"Powell","year":"2006","journal-title":"Toxicol. Sci."},{"key":"2023020202145252000_btv108-B32","doi-asserted-by":"crossref","DOI":"10.1093\/nar\/gkv007","article-title":"Limma powers differential expression analysis for RNA-sequencing and microarray studies","author":"Ritchie","year":"2015","journal-title":"Nucleic Acids Res."},{"key":"2023020202145252000_btv108-B33","doi-asserted-by":"crossref","first-page":"370","DOI":"10.1093\/bioinformatics\/btp686","article-title":"Mixtures of regression models for time course gene expression data: evaluation of initialization and random effects","volume":"26","author":"Scharl","year":"2010","journal-title":"Bioinformatics"},{"key":"2023020202145252000_btv108-B34","doi-asserted-by":"crossref","first-page":"i119","DOI":"10.1093\/bioinformatics\/btp206","article-title":"Clustered alignments of gene-expression time series data","volume":"25","author":"Smith","year":"2009","journal-title":"Bioinformatics"},{"key":"2023020202145252000_btv108-B35","doi-asserted-by":"crossref","first-page":"R94","DOI":"10.1186\/gb-2004-5-11-r94","article-title":"Consensus clustering and functional interpretation of gene-expression data","volume":"5","author":"Swift","year":"2004","journal-title":"Genome Biol."},{"key":"2023020202145252000_btv108-B36","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1093\/bioinformatics\/bti737","article-title":"Integrating time-course microarray gene expression profiles with cytotoxicity for identification of biomarkers in primary rat hepatocytes exposed to cadmium","volume":"22","author":"Tan","year":"2006","journal-title":"Bioinformatics"},{"key":"2023020202145252000_btv108-B37","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1177\/1533210110392953","article-title":"Emerging issues in vitamin K research","volume":"16","author":"Truong","year":"2011","journal-title":"J. Evid. Based Complementary Altern. Med."},{"key":"2023020202145252000_btv108-B38","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1093\/toxsci\/kfq214","article-title":"Time series analysis of benzo[A]pyrene-induced transcriptome changes suggests that a network of transcription factors regulates the effects on functional gene sets","volume":"117","author":"van Delft","year":"2010","journal-title":"Toxicol. Sci."},{"key":"2023020202145252000_btv108-B39","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1093\/bioinformatics\/bts668","article-title":"Efficient statistical significance approximation for local similarity analysis of high-throughput time series data","volume":"29","author":"Xia","year":"2013","journal-title":"Bioinformatics"},{"key":"2023020202145252000_btv108-B40","first-page":"1","article-title":"Kernlab-an S4 package for kernel methods in R","volume":"11","author":"Zeileis","year":"2004","journal-title":"J. Stat. Softw."},{"key":"2023020202145252000_btv108-B41","first-page":"1","article-title":"A general framework for weighted gene co-expression network analysis","volume":"4","author":"Zhang","year":"2005","journal-title":"Stat. Appl. Genet. Mol."}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/31\/13\/2115\/49035030\/bioinformatics_31_13_2115.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/31\/13\/2115\/49035030\/bioinformatics_31_13_2115.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,2,2]],"date-time":"2023-02-02T03:41:43Z","timestamp":1675309303000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/31\/13\/2115\/196010"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,2,19]]},"references-count":41,"journal-issue":{"issue":"13","published-print":{"date-parts":[[2015,7,1]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btv108","relation":{},"ISSN":["1367-4811","1367-4803"],"issn-type":[{"value":"1367-4811","type":"electronic"},{"value":"1367-4803","type":"print"}],"subject":[],"published-other":{"date-parts":[[2015,7,1]]},"published":{"date-parts":[[2015,2,19]]}}}