{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,8]],"date-time":"2025-10-08T15:23:25Z","timestamp":1759937005270},"reference-count":34,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2016,10,2]],"date-time":"2016-10-02T00:00:00Z","timestamp":1475366400000},"content-version":"vor","delay-in-days":2804,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/2.0\/uk\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2009,3,15]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Motivation: In recent years, the biological literature has seen a significant increase of reported methods for identifying both structure and parameters of ordinary differential equations (ODEs) from time series data. A natural way to evaluate the performance of such methods is to try them on a sufficient number of realistic test cases. However, weak practices in specifying identification problems and lack of commonly accepted benchmark problems makes it difficult to evaluate and compare different methods.<\/jats:p>\n               <jats:p>Results: To enable better evaluation and comparisons between different methods, we propose how to specify identification problems as optimization problems with a model space of allowed reactions (e.g. reaction kinetics like Michaelis\u2013Menten or S-systems), ranges for the parameters, time series data and an error function. We also define a file format for such problems.<\/jats:p>\n               <jats:p>We then present a collection of more than 40 benchmark problems for ODE model identification of cellular systems. The collection includes realistic problems of different levels of difficulty w.r.t. size and quality of data. We consider both problems with simulated data from known systems, and problems with real data. Finally, we present results based on our identification algorithm for all benchmark problems. In comparison with publications on which we have based some of the benchmark problems, our approach allows all problems to be solved without the use of supercomputing.<\/jats:p>\n               <jats:p>Availability: The benchmark problems are available at www.odeidentification.org<\/jats:p>\n               <jats:p>Contact: \u00a0peterg@chalmers.se<\/jats:p>\n               <jats:p>Supplementary information: \u00a0Supplementary data are available at Bioinformatics online.<\/jats:p>","DOI":"10.1093\/bioinformatics\/btp050","type":"journal-article","created":{"date-parts":[[2009,1,29]],"date-time":"2009-01-29T01:31:13Z","timestamp":1233192673000},"page":"780-786","source":"Crossref","is-referenced-by-count":29,"title":["Benchmarks for identification of ordinary differential equations from time series data"],"prefix":"10.1093","volume":"25","author":[{"given":"Peter","family":"Gennemark","sequence":"first","affiliation":[{"name":"1 Department of Mathematical Sciences, University of G\u00f6teborg, SE-412 96 G\u00f6teborg, 2Department of Mathematics, Uppsala University, SE-751 06 Uppsala and 3Department of Computer Science and Engineering, Chalmers University of Technology, SE-412 96 G\u00f6teborg, Sweden"},{"name":"1 Department of Mathematical Sciences, University of G\u00f6teborg, SE-412 96 G\u00f6teborg, 2Department of Mathematics, Uppsala University, SE-751 06 Uppsala and 3Department of Computer Science and Engineering, Chalmers University of Technology, SE-412 96 G\u00f6teborg, Sweden"}]},{"given":"Dag","family":"Wedelin","sequence":"additional","affiliation":[{"name":"1 Department of Mathematical Sciences, University of G\u00f6teborg, SE-412 96 G\u00f6teborg, 2Department of Mathematics, Uppsala University, SE-751 06 Uppsala and 3Department of Computer Science and Engineering, Chalmers University of Technology, SE-412 96 G\u00f6teborg, Sweden"}]}],"member":"286","published-online":{"date-parts":[[2009,1,28]]},"reference":[{"key":"2023051209121295900_B1","doi-asserted-by":"crossref","first-page":"970","DOI":"10.1021\/j100003a020","article-title":"Statistical construction of chemical reaction mechanisms from measured time-series","volume":"99","author":"Arkin","year":"1995","journal-title":"J. Phys. Chem."},{"key":"2023051209121295900_B2","doi-asserted-by":"crossref","first-page":"D562","DOI":"10.1093\/nar\/gki022","article-title":"NCBI GEO: mining millions of expression profiles\u2013database and tools","volume":"33","author":"Barrett","year":"2005","journal-title":"Nucleic Acids Res."},{"key":"2023051209121295900_B3","doi-asserted-by":"crossref","first-page":"9943","DOI":"10.1073\/pnas.0609476104","article-title":"Automated reverse engineering of nonlinear dynamical systems","volume":"104","author":"Bongard","year":"2007","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"2023051209121295900_B4","doi-asserted-by":"crossref","first-page":"1631","DOI":"10.1093\/bioinformatics\/btl122","article-title":"Identification of biochemical networks by S-tree based genetic programming","volume":"22","author":"Cho","year":"2006","journal-title":"Bioinformatics"},{"key":"2023051209121295900_B5","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1186\/1742-4682-3-25","article-title":"Parameter estimation in biochemical systems models with alternating regression","volume":"3","author":"Chou","year":"2006","journal-title":"Theor. Biol. Med. Model"},{"key":"2023051209121295900_B6","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1016\/j.pbiomolbio.2004.04.002","article-title":"Mathematical and computational techniques to deduce complex biochemical reaction mechanisms","volume":"86","author":"Crampin","year":"2004","journal-title":"Prog. Biophy. Mol. Biol."},{"key":"2023051209121295900_B7","doi-asserted-by":"crossref","first-page":"503","DOI":"10.1142\/S0219720006001886","article-title":"Inference of scale-free networks from gene expression time series","volume":"4","author":"Daisuke","year":"2006","journal-title":"J. Bioinform. Comput. Biol."},{"key":"2023051209121295900_B8","article-title":"Reconstructing Gene Networks from Large Scale Gene Expression Data","volume-title":"PhD Thesis.","author":"D'haeseleer","year":"2000"},{"key":"2023051209121295900_B9","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1049\/iet-syb:20050098","article-title":"Efficient algorithms for ordinary differential equation model identification of biological systems","volume":"1","author":"Gennemark","year":"2007","journal-title":"IET Syst. Biol."},{"key":"2023051209121295900_B10","doi-asserted-by":"crossref","first-page":"480","DOI":"10.1093\/bioinformatics\/btl522","article-title":"Parameter estimation using Simulated Annealing for S-system models of biochemical networks","volume":"23","author":"Gonzalez","year":"2007","journal-title":"Bioinformatics"},{"key":"2023051209121295900_B11","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1006\/jmbi.1996.0011","article-title":"Rules for coupled expression of regulator and effector genes in inducible circuits","volume":"255","author":"Hlavacek","year":"1996","journal-title":"J. Mol. Biol."},{"key":"2023051209121295900_B12","doi-asserted-by":"crossref","first-page":"3459","DOI":"10.1529\/biophysj.106.093344","article-title":"Numerical Matrices Method for nonlinear system identification and description of dynamics of biochemical reaction networks","volume":"92","author":"Karnaukhov","year":"2007","journal-title":"Biophys. J."},{"key":"2023051209121295900_B13","doi-asserted-by":"crossref","first-page":"643","DOI":"10.1093\/bioinformatics\/btg027","article-title":"Dynamic modeling of genetic networks using genetic algorithm and S-system","volume":"19","author":"Kikuchi","year":"2003","journal-title":"Bioinformatics"},{"key":"2023051209121295900_B14","doi-asserted-by":"crossref","first-page":"1154","DOI":"10.1093\/bioinformatics\/bti071","article-title":"Inference of S-system models of genetic networks using a cooperative coevolutionary algorithm","volume":"21","author":"Kimura","year":"2005","journal-title":"Bioinformatics"},{"key":"2023051209121295900_B15","first-page":"26","article-title":"CadC-mediated activation of the cadBA promoter in Escherichia coli","volume":"10","author":"Kuper","year":"2005","journal-title":"J. Mol. Microbiol. Biotechnol."},{"key":"2023051209121295900_B16","doi-asserted-by":"crossref","first-page":"174","DOI":"10.1049\/iet-syb:20060064","article-title":"S-system parameter estimation for noisy metabolic profiles using newton-flow analysis","volume":"1","author":"Kutalik","year":"2007","journal-title":"IET Syst. Biol."},{"key":"2023051209121295900_B17","doi-asserted-by":"crossref","first-page":"1085","DOI":"10.1093\/bioinformatics\/btn075","article-title":"Inference of biochemical network models in S-system using multiobjective optimization approach","volume":"24","author":"Liu","year":"2008","journal-title":"Bioinformatics"},{"key":"2023051209121295900_B18","first-page":"446","article-title":"Development of a system for the inference of large scale genetic networks","author":"Maki","year":"2001","journal-title":"Pac. Symp. Biocomput."},{"key":"2023051209121295900_B19","doi-asserted-by":"crossref","first-page":"665","DOI":"10.1142\/S0219720006002259","article-title":"An automated procedure for the extraction of metabolic network information from time series data","volume":"4","author":"Marino","year":"2006","journal-title":"J. Bioinform. Comput. Biol."},{"key":"2023051209121295900_B20","doi-asserted-by":"crossref","first-page":"021912","DOI":"10.1103\/PhysRevE.73.021912","article-title":"Hybrid grammar-based approach to nonlinear dynamical system identification from biological time series","volume":"73","author":"McKinney","year":"2006","journal-title":"Phys. Rev. E. Stat. Nonlin. Soft Matter Phys."},{"issue":"Suppl. 2","key":"2023051209121295900_B21","doi-asserted-by":"crossref","first-page":"ii122","DOI":"10.1093\/bioinformatics\/btg1069","article-title":"Artificial gene networks for objective comparison of analysis algorithms","volume":"19","author":"Mendes","year":"2003","journal-title":"Bioinformatics"},{"key":"2023051209121295900_B22","doi-asserted-by":"crossref","first-page":"2467","DOI":"10.1101\/gr.1262503","article-title":"Parameter estimation in biochemical pathways: a comparison of global optimization methods","volume":"13","author":"Moles","year":"2003","journal-title":"Genome Res."},{"key":"2023051209121295900_B23","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1038\/msb.2008.53","article-title":"Models from experiments: combinatorial drug perturbations of cancer cells","volume":"4","author":"Nelander","year":"2008","journal-title":"Mol. Syst. Biol."},{"key":"2023051209121295900_B24","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1086\/382510","article-title":"Adverse events after smallpox immunizations are associated with alterations in systemic cytokine levels","volume":"189","author":"Rock","year":"2004","journal-title":"J. Infect Dis."},{"key":"2023051209121295900_B25","doi-asserted-by":"crossref","first-page":"10555","DOI":"10.1073\/pnas.152046799","article-title":"Assigning numbers to the arrows: parameterizing a gene regulation network by using accurate expression kinetics","volume":"99","author":"Ronen","year":"2002","journal-title":"Proc. Natl Acad. Sci. USA."},{"key":"2023051209121295900_B26","volume-title":"Biochemical systems analysis: a study of function and design in molecular biology.","author":"Savageau","year":"1976"},{"key":"2023051209121295900_B27","article-title":"Numerical data fitting in dynamical systems: a practical introduction with applications and software","volume-title":"Applied Optimization.","author":"Schittkowski","year":"2002"},{"key":"2023051209121295900_B28","doi-asserted-by":"crossref","first-page":"1180","DOI":"10.1093\/bioinformatics\/bti099","article-title":"Evolutionary optimization with data collocation for reverse engineering of biological networks","volume":"21","author":"Tsai","year":"2005","journal-title":"Bioinformatics"},{"key":"2023051209121295900_B29","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1007\/s11155-006-9009-2","article-title":"Parameter reconstruction for biochemical networks using interval analysis","volume":"12","author":"Tucker","year":"2006","journal-title":"Reliable Computing"},{"key":"2023051209121295900_B30","doi-asserted-by":"crossref","first-page":"607","DOI":"10.1016\/j.mbs.2006.11.009","article-title":"Estimating parameters for generalized mass action models using constraint propagation","volume":"208","author":"Tucker","year":"2007","journal-title":"Math. Bioscience"},{"key":"2023051209121295900_B31","first-page":"176","article-title":"Parameter estimation","volume-title":"Computational Analysis of Biochemical Systems. A Practical Guide for Biochemists and Molecular Biologists.","author":"Voit","year":"2000"},{"key":"2023051209121295900_B32","doi-asserted-by":"crossref","first-page":"1670","DOI":"10.1093\/bioinformatics\/bth140","article-title":"Decoupling dynamical systems for pathway identification from metabolic profiles","volume":"20","author":"Voit","year":"2004","journal-title":"Bioinformatics"},{"key":"2023051209121295900_B33","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1016\/S0303-2647(99)00090-8","article-title":"Coarse-grained reverse engineering of genetic regulatory networks","volume":"55","author":"Wahde","year":"2000","journal-title":"Biosystems"},{"key":"2023051209121295900_B34","doi-asserted-by":"crossref","first-page":"2876","DOI":"10.1021\/ie000544+","article-title":"Hybrid differential evolution for problems of kinetic parameter estimation and dynamic optimization of an ethanol fermentation process","volume":"40","author":"Wang","year":"2001","journal-title":"Ind. Eng. Chem. Res."}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/25\/6\/780\/50286272\/bioinformatics_25_6_780.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/25\/6\/780\/50286272\/bioinformatics_25_6_780.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,12]],"date-time":"2023-05-12T09:12:52Z","timestamp":1683882772000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/25\/6\/780\/251452"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2009,1,28]]},"references-count":34,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2009,3,15]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btp050","relation":{},"ISSN":["1367-4811","1367-4803"],"issn-type":[{"value":"1367-4811","type":"electronic"},{"value":"1367-4803","type":"print"}],"subject":[],"published-other":{"date-parts":[[2009,3,15]]},"published":{"date-parts":[[2009,1,28]]}}}