{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,2]],"date-time":"2026-07-02T20:26:22Z","timestamp":1783023982790,"version":"3.54.6"},"reference-count":118,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2021,10,7]],"date-time":"2021-10-07T00:00:00Z","timestamp":1633564800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"name":"European Union\u2019s Horizon 2020 Research and Innovation Programme","award":["686282"],"award-info":[{"award-number":["686282"]}]},{"name":"Ram\u00f3n y Cajal Fellowship","award":["RYC-2019-027537-I"],"award-info":[{"award-number":["RYC-2019-027537-I"]}]},{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["390873048"],"award-info":[{"award-number":["390873048"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["390685813"],"award-info":[{"award-number":["390685813"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]},{"name":"German Federal Ministry of Economic Affairs and Energy","award":["16KN074236"],"award-info":[{"award-number":["16KN074236"]}]},{"DOI":"10.13039\/501100004837","name":"Ministerio de Ciencia e Innovaci\u00f3n","doi-asserted-by":"publisher","award":["PID2020-117271RB-C22"],"award-info":[{"award-number":["PID2020-117271RB-C22"]}],"id":[{"id":"10.13039\/501100004837","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,1,17]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Ordinary differential equation models are nowadays widely used for the mechanistic description of biological processes and their temporal evolution. These models typically have many unknown and nonmeasurable parameters, which have to be determined by fitting the model to experimental data. In order to perform this task, known as parameter estimation or model calibration, the modeller faces challenges such as poor parameter identifiability, lack of sufficiently informative experimental data and the existence of local minima in the objective function landscape. These issues tend to worsen with larger model sizes, increasing the computational complexity and the number of unknown parameters. An incorrectly calibrated model is problematic because it may result in inaccurate predictions and misleading conclusions. For nonexpert users, there are a large number of potential pitfalls. Here, we provide a protocol that guides the user through all the steps involved in the calibration of dynamic models. We illustrate the methodology with two models and provide all the code required to reproduce the results and perform the same analysis on new models. Our protocol provides practitioners and researchers in biological modelling with a one-stop guide that is at the same time compact and sufficiently comprehensive to cover all aspects of the problem.<\/jats:p>","DOI":"10.1093\/bib\/bbab387","type":"journal-article","created":{"date-parts":[[2021,9,8]],"date-time":"2021-09-08T11:20:51Z","timestamp":1631100051000},"source":"Crossref","is-referenced-by-count":137,"title":["A protocol for dynamic model calibration"],"prefix":"10.1093","volume":"23","author":[{"given":"Alejandro F","family":"Villaverde","sequence":"first","affiliation":[{"name":"Universidade de Vigo, Department of Systems Engineering & Control, Vigo 36310, Galicia, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dilan","family":"Pathirana","sequence":"additional","affiliation":[{"name":"Faculty of Mathematics and Natural Sciences, University of Bonn, Bonn 53115, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fabian","family":"Fr\u00f6hlich","sequence":"additional","affiliation":[{"name":"Institute of Computational Biology, Helmholtz Zentrum M\u00fcnchen, Neuherberg 85764, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jan","family":"Hasenauer","sequence":"additional","affiliation":[{"name":"Center for Mathematics, Technische Universit\u00e4t M\u00fcnchen, Garching 85748, Germany"},{"name":"Harvard Medical School, Cambridge, MA 02115, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Julio R","family":"Banga","sequence":"additional","affiliation":[{"name":"Bioprocess Engineering Group, IIM-CSIC, Vigo 36208, Galicia, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2021,10,9]]},"reference":[{"issue":"9","key":"2022011921120650700_ref1","doi-asserted-by":"crossref","first-page":"1001","DOI":"10.1038\/nbt1330","article-title":"Ensemble modeling for analysis of cell signaling dynamics","volume":"25","author":"Kuepfer","year":"2007","journal-title":"Nat Biotechnol"},{"key":"2022011921120650700_ref2","first-page":"63","volume-title":"Pacific Symposium on Biocomputing. Pacific Symposium on Biocomputing","author":"Sachs","year":"2009"},{"issue":"6","key":"2022011921120650700_ref3","doi-asserted-by":"crossref","first-page":"567","DOI":"10.1016\/j.cels.2018.10.013","article-title":"Efficient parameter estimation enables the prediction of drug response using a mechanistic pan-cancer pathway model","volume":"7","author":"Fr\u00f6hlich","year":"2018","journal-title":"Cell Syst"},{"key":"2022011921120650700_ref4","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pcbi.1005379","article-title":"Data-driven reverse engineering of signaling pathways using ensembles of dynamic models","volume":"13","author":"Henriques","year":"2017","journal-title":"PLoS Comput Biol"},{"issue":"4","key":"2022011921120650700_ref5","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1016\/j.coche.2013.08.004","article-title":"Modeling metabolic systems: the need for dynamics","volume":"2","author":"Song","year":"2013","journal-title":"Curr Opin Chem Eng"},{"key":"2022011921120650700_ref6","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1016\/j.ymben.2014.03.007","article-title":"Kinetic models in industrial biotechnology\u2013improving cell factory performance","volume":"24","author":"Almquist","year":"2014","journal-title":"Metab Eng"},{"key":"2022011921120650700_ref7","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.jbiotec.2016.01.005","article-title":"Metabolic engineering with multi-objective optimization of kinetic models","volume":"222","author":"Villaverde","year":"2016","journal-title":"J Biotechnol"},{"issue":"2","key":"2022011921120650700_ref8","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1021\/acssynbio.7b00188","article-title":"Perfect adaptation and optimal equilibrium productivity in a simple microbial biofuel metabolic pathway using dynamic integral control","volume":"7","author":"Briat","year":"2018","journal-title":"ACS Synth Biol"},{"issue":"4","key":"2022011921120650700_ref9","doi-asserted-by":"crossref","first-page":"410","DOI":"10.1039\/C7ME00032D","article-title":"Computational design of biological circuits: putting parts into context","volume":"2","author":"Karamasioti","year":"2017","journal-title":"Mol Syst Design Eng"},{"issue":"3","key":"2022011921120650700_ref10","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1109\/MCS.2018.2810459","article-title":"Control theory for synthetic biology: recent advances in system characterization, control design, and controller implementation for synthetic biology","volume":"38","author":"Hsiao","year":"2018","journal-title":"IEEE Control Syst"},{"issue":"2","key":"2022011921120650700_ref11","doi-asserted-by":"crossref","first-page":"807","DOI":"10.1109\/TCNS.2018.2790039","article-title":"Design constraints for biological systems that achieve adaptation and disturbance rejection","volume":"5","author":"Steel","year":"2018","journal-title":"IEEE Trans Control Netw Syst"},{"issue":"4","key":"2022011921120650700_ref12","doi-asserted-by":"crossref","first-page":"508","DOI":"10.1016\/j.cels.2018.03.013","article-title":"Computational re-design of synthetic genetic oscillators for independent amplitude and frequency modulation","volume":"6","author":"Tomazou","year":"2018","journal-title":"Cell Syst"},{"issue":"1","key":"2022011921120650700_ref13","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1093\/nar\/28.1.27","article-title":"Kegg: kyoto encyclopedia of genes and genomes","volume":"28","author":"Kanehisa","year":"2000","journal-title":"Nucleic Acids Res"},{"key":"2022011921120650700_ref14","article-title":"The string database in 2017: quality-controlled protein\u2013protein association networks, made broadly accessible","volume":"45","author":"Szklarczyk","year":"2016","journal-title":"Nucleic Acids Res"},{"issue":"D1","key":"2022011921120650700_ref15","doi-asserted-by":"crossref","first-page":"D649","DOI":"10.1093\/nar\/gkx1132","article-title":"The reactome pathway knowledgebase","volume":"46","author":"Fabregat","year":"2017","journal-title":"Nucleic Acids Res"},{"issue":"13","key":"2022011921120650700_ref16","doi-asserted-by":"crossref","first-page":"2143","DOI":"10.1093\/bioinformatics\/bth200","article-title":"Web-based kinetic modelling using JWS online","volume":"20","author":"Olivier","year":"2004","journal-title":"Bioinformatics"},{"issue":"database issue","key":"2022011921120650700_ref17","doi-asserted-by":"crossref","first-page":"D689","DOI":"10.1093\/nar\/gkj092","article-title":"BioModels database: a free, centralized database of curated, published, quantitative kinetic models of biochemical and cellular systems","volume":"34","author":"Le Nov\u00e8re","year":"2006","journal-title":"Nucleic Acids Res"},{"key":"2022011921120650700_ref18","article-title":"Brenda in 2015: exciting developments in its 25th year of existence","volume":"43","author":"Chang","year":"2014","journal-title":"Nucleic Acids Res"},{"issue":"D1","key":"2022011921120650700_ref19","doi-asserted-by":"crossref","first-page":"D790","DOI":"10.1093\/nar\/gkr1046","article-title":"Sabio-rk database for biochemical reaction kinetics","volume":"40","author":"Wittig","year":"2012","journal-title":"Nucleic Acids Res"},{"issue":"4","key":"2022011921120650700_ref20","doi-asserted-by":"crossref","first-page":"364","DOI":"10.1093\/bib\/bbl040","article-title":"Dynamic modelling and analysis of biochemical networks: mechanism-based models and model-based experiments","volume":"7","author":"Riel","year":"2006","journal-title":"Brief Bioinform"},{"key":"2022011921120650700_ref21","doi-asserted-by":"crossref","first-page":"524","DOI":"10.1093\/bioinformatics\/btg015","article-title":"The systems biology markup language (SBML): a medium for representation and exchange of biochemical network models","volume":"19","author":"Hucka","year":"2003","journal-title":"Bioinformatics"},{"issue":"5","key":"2022011921120650700_ref22","doi-asserted-by":"crossref","first-page":"830","DOI":"10.1093\/bioinformatics\/bty736","article-title":"Benchmarking optimization methods for parameter estimation in large kinetic models","volume":"35","author":"Villaverde","year":"2018","journal-title":"Bioinformatics"},{"issue":"17","key":"2022011921120650700_ref23","doi-asserted-by":"crossref","first-page":"3073","DOI":"10.1093\/bioinformatics\/btz020","article-title":"Benchmark problems for dynamic modeling of intracellular processes","volume":"35","author":"Hass","year":"2019","journal-title":"Bioinformatics"},{"issue":"11","key":"2022011921120650700_ref24","doi-asserted-by":"crossref","first-page":"813","DOI":"10.1038\/nrm2030","article-title":"Linking data to models: data regression","volume":"7","author":"Jaqaman","year":"2006","journal-title":"Nat Rev Mol Cell Bio"},{"issue":"4","key":"2022011921120650700_ref25","doi-asserted-by":"crossref","first-page":"886","DOI":"10.1111\/j.1742-4658.2008.06844.x","article-title":"Systems biology: parameter estimation for biochemical models","volume":"276","author":"Ashyraliyev","year":"2009","journal-title":"FEBS J"},{"key":"2022011921120650700_ref26","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1007\/978-1-61779-833-7_2","volume-title":"Computational Modeling of Signaling Networks, Volume 880 of Methods in Molecular Biology","author":"Geier","year":"2012"},{"issue":"9","key":"2022011921120650700_ref27","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pone.0074335","article-title":"Lessons learned from quantitative dynamical modeling in systems biology","volume":"8","author":"Raue","year":"2013","journal-title":"PLoS One"},{"issue":"5","key":"2022011921120650700_ref28","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1002\/psp4.12071","article-title":"A six-stage workflow for robust application of systems pharmacology","volume":"5","author":"Kapil","year":"2016","journal-title":"CPT Pharmacometrics Syst Pharmacol"},{"issue":"91","key":"2022011921120650700_ref29","doi-asserted-by":"crossref","first-page":"20130505","DOI":"10.1098\/rsif.2013.0505","article-title":"Reverse engineering and identification in systems biology: strategies, perspectives and challenges","volume":"11","author":"Villaverde","year":"2014","journal-title":"J R Soc Interface"},{"issue":"6","key":"2022011921120650700_ref30","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1109\/MCS.2019.2938121","article-title":"Nonlinear system identification: a user-oriented road map","volume":"39","author":"Schoukens","year":"2019","journal-title":"IEEE Control Syst Mag"},{"key":"2022011921120650700_ref31","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1186\/1752-0509-4-11","article-title":"An iterative identification procedure for dynamic modeling of biochemical networks","volume":"4","author":"Balsa-Canto","year":"2010","journal-title":"BMC Syst Biol"},{"key":"2022011921120650700_ref32","first-page":"317","volume-title":"ODE-Based Modeling of Complex Regulatory Circuits","author":"Seaton","year":"2017"},{"key":"2022011921120650700_ref33","article-title":"Modeling of biocatalytic reactions: a workflow for model calibration, selection and validation using Bayesian statistics","volume":"66","author":"Eisenkolb","year":"2019","journal-title":"AIChE Jl"},{"issue":"4","key":"2022011921120650700_ref34","doi-asserted-by":"crossref","first-page":"575","DOI":"10.2166\/hydro.2011.041","article-title":"A practical protocol for calibration of nutrient removal wastewater treatment models","volume":"13","author":"Mannina","year":"2011","journal-title":"J Hydroinf"},{"key":"2022011921120650700_ref35","doi-asserted-by":"crossref","first-page":"8493","DOI":"10.1038\/srep08493","article-title":"A novel protocol for model calibration in biological wastewater treatment","volume":"5","author":"Zhu","year":"2015","journal-title":"Sci Rep"},{"issue":"3","key":"2022011921120650700_ref36","first-page":"436","article-title":"Toward predictive food process models: a protocol for parameter estimation","volume":"58","author":"Vilas","year":"2018","journal-title":"Crit Rev Food Sci Nutr"},{"key":"2022011921120650700_ref37","volume-title":"Proceedings of the 58th IEEE Conference on Decision and Control","author":"Tuza","year":"2019"},{"key":"2022011921120650700_ref38","doi-asserted-by":"crossref","DOI":"10.1002\/wsbm.1482","article-title":"Calibration of ionic and cellular cardiac electrophysiology models","volume":"12","author":"Whittaker","year":"2020","journal-title":"Wiley Interdiscip Rev Syst Biol Med"},{"key":"2022011921120650700_ref39","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1007\/978-1-4939-9102-0_16","volume-title":"Modeling Biomolecular Site Dynamics","author":"Steiert","year":"2019"},{"issue":"21","key":"2022011921120650700_ref40","doi-asserted-by":"crossref","first-page":"3558","DOI":"10.1093\/bioinformatics\/btv405","article-title":"Data2dynamics: a modeling environment tailored to parameter estimation in dynamical systems","volume":"31","author":"Raue","year":"2015","journal-title":"Bioinformatics"},{"issue":"21","key":"2022011921120650700_ref41","doi-asserted-by":"crossref","first-page":"5993","DOI":"10.1093\/jxb\/erw356","article-title":"Enzymatic study on atccd4 and atccd7 and their potential to form acyclic regulatory metabolites","volume":"67","author":"Bruno","year":"2016","journal-title":"J Exp Bot"},{"issue":"132","key":"2022011921120650700_ref42","doi-asserted-by":"crossref","first-page":"ra56","DOI":"10.1126\/scisignal.2000810","article-title":"Decoupling of receptor and downstream signals in the Akt pathway by its low-pass filter characteristics","volume":"3","author":"Fujita","year":"2010","journal-title":"Sci Signal"},{"issue":"1","key":"2022011921120650700_ref43","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pcbi.1008646","article-title":"Petab-interoperable specification of parameter estimation problems in systems biology","volume":"17","author":"Schmiester","year":"2021","journal-title":"PLoS Comput Biol"},{"issue":"156","key":"2022011921120650700_ref44","doi-asserted-by":"crossref","first-page":"20190043","DOI":"10.1098\/rsif.2019.0043","article-title":"Full observability and estimation of unknown inputs, states and parameters of nonlinear biological models","volume":"16","author":"Villaverde","year":"2019","journal-title":"J R Soc Interface"},{"issue":"1","key":"2022011921120650700_ref45","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pcbi.1005331","article-title":"Scalable parameter estimation for genome-scale biochemical reaction networks","volume":"13","author":"Fr\u00f6hlich","year":"2017","journal-title":"PLoS Comput Biol"},{"issue":"4","key":"2022011921120650700_ref46","doi-asserted-by":"crossref","first-page":"705","DOI":"10.1093\/bioinformatics\/btx676","article-title":"Pesto: parameter estimation toolbox","volume":"34","author":"Stapor","year":"2017","journal-title":"Bioinformatics"},{"key":"2022011921120650700_ref47","doi-asserted-by":"crossref","DOI":"10.1101\/2021.05.20.445065","article-title":"Fides: Reliable trust-region optimization for parameter estimation of ordinary differential equation models","author":"Froehlich","year":"2021"},{"key":"2022011921120650700_ref48","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1038\/s41592-019-0686-2","article-title":"SciPy 1.0: fundamental algorithms for scientific computing in Python","volume":"17","author":"Virtanen","year":"2020","journal-title":"Nat Methods"},{"issue":"1","key":"2022011921120650700_ref49","first-page":"3","article-title":"On identifiability of nonlinear ode models and applications in viral dynamics","volume":"53","author":"Miao","year":"2011","journal-title":"SIAM Rev Soc Ind Appl Math"},{"issue":"11","key":"2022011921120650700_ref50","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pone.0027755","article-title":"Structural identifiability of systems biology models: a critical comparison of methods","volume":"6","author":"Chis","year":"2011","journal-title":"PLoS One"},{"key":"2022011921120650700_ref51","doi-asserted-by":"crossref","DOI":"10.1155\/2019\/8497093","article-title":"Observability and structural identifiability of nonlinear biological systems","volume":"2019","author":"Villaverde","year":"2019","journal-title":"Complexity"},{"issue":"5","key":"2022011921120650700_ref52","doi-asserted-by":"crossref","first-page":"735","DOI":"10.1006\/jsco.2002.0532","article-title":"A probabilistic algorithm to test local algebraic observability in polynomial time","volume":"33","author":"Sedoglavic","year":"2002","journal-title":"J Symbolic Comput"},{"key":"2022011921120650700_ref53","first-page":"941","volume-title":"16th IFAC Symposium on System Identification","author":"Karlsson","year":"2012"},{"issue":"5","key":"2022011921120650700_ref54","doi-asserted-by":"crossref","first-page":"607","DOI":"10.1109\/TAC.2005.847062","article-title":"Model structure simplification of nonlinear systems via immersion","volume":"50","author":"Ohtsuka","year":"2005","journal-title":"IEEE Trans Automatic Control"},{"issue":"3","key":"2022011921120650700_ref55","doi-asserted-by":"crossref","first-page":"574","DOI":"10.1002\/stc.1690","article-title":"On the observability and identifiability of nonlinear structural and mechanical systems","volume":"22","author":"Chatzis","year":"2015","journal-title":"Struct Control Health Monit"},{"issue":"8","key":"2022011921120650700_ref56","doi-asserted-by":"crossref","first-page":"1421","DOI":"10.1093\/bioinformatics\/btx735","article-title":"Genssi 2.0: multi-experiment structural identifiability analysis of sbml models","volume":"34","author":"Ligon","year":"2017","journal-title":"Bioinformatics"},{"issue":"16","key":"2022011921120650700_ref57","doi-asserted-by":"crossref","first-page":"2873","DOI":"10.1093\/bioinformatics\/bty1069","article-title":"Sian: software for structural identifiability analysis of ode models","volume":"35","author":"Hong","year":"2019","journal-title":"Bioinformatics"},{"issue":"10","key":"2022011921120650700_ref58","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pone.0110261","article-title":"On finding and using identifiable parameter combinations in nonlinear dynamic systems biology models and combos: a novel web implementation","volume":"9","author":"Meshkat","year":"2014","journal-title":"PLoS One"},{"key":"2022011921120650700_ref59","first-page":"329","volume-title":"International Conference on Computational Methods in Systems Biology","author":"Saccomani","year":"2019"},{"key":"2022011921120650700_ref60","doi-asserted-by":"crossref","first-page":"118","DOI":"10.1016\/j.automatica.2015.05.004","article-title":"A fast algorithm to assess local structural identifiability","volume":"58","author":"Stigter","year":"2015","journal-title":"Automatica"},{"key":"2022011921120650700_ref61","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1016\/j.automatica.2017.01.029","article-title":"Identifiability of affine linear parameter-varying models","volume":"80","author":"Alkhoury","year":"2017","journal-title":"Automatica"},{"issue":"11","key":"2022011921120650700_ref62","doi-asserted-by":"crossref","first-page":"2884","DOI":"10.1016\/j.automatica.2008.03.019","article-title":"Identifiability of discrete-time nonlinear systems: the local state isomorphism approach","volume":"44","author":"Anstett","year":"2008","journal-title":"Automatica"},{"key":"2022011921120650700_ref63","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1016\/j.automatica.2016.01.054","article-title":"Further results on identifiability of discrete-time nonlinear systems","volume":"68","author":"N\u00f5mm","year":"2016","journal-title":"Automatica"},{"issue":"173","key":"2022011921120650700_ref64","doi-asserted-by":"crossref","first-page":"20200652","DOI":"10.1098\/rsif.2020.0652","article-title":"Identifiability analysis for stochastic differential equation models in systems biology","volume":"17","author":"Browning","year":"2020","journal-title":"J R Soc Interface"},{"key":"2022011921120650700_ref65","article-title":"Structural identifiability analysis of pdes: a case study in continuous age-structured epidemic models","author":"Renardy","year":"2021"},{"key":"2022011921120650700_ref66","volume-title":"Identification of Parametric Models from Experimental Data","author":"Walter","year":"1997"},{"key":"2022011921120650700_ref67","volume-title":"Dynamic Systems Biology Modeling and Simulation","author":"DiStefano","year":"2015"},{"issue":"13","key":"2022011921120650700_ref68","doi-asserted-by":"crossref","first-page":"i494","DOI":"10.1093\/bioinformatics\/bty229","article-title":"Bayesian parameter estimation for biochemical reaction networks using region-based adaptive parallel tempering","volume":"34","author":"Ballnus","year":"2018","journal-title":"Bioinformatics"},{"key":"2022011921120650700_ref69","article-title":"Repairing dynamic models: a method to obtain identifiable and observable reparameterizations with mechanistic insights","author":"Massonis","year":"2020"},{"issue":"1","key":"2022011921120650700_ref70","doi-asserted-by":"crossref","DOI":"10.1103\/PhysRevE.92.012920","article-title":"Higher-order lie symmetries in identifiability and predictability analysis of dynamic models","volume":"92","author":"Merkt","year":"2015","journal-title":"Phy Rev E"},{"issue":"19","key":"2022011921120650700_ref71","doi-asserted-by":"crossref","first-page":"2612","DOI":"10.1093\/bioinformatics\/btm382","article-title":"Data-based identifiability analysis of non-linear dynamical models","volume":"23","author":"Hengl","year":"2007","journal-title":"Bioinformatics"},{"key":"2022011921120650700_ref72","doi-asserted-by":"crossref","first-page":"104","DOI":"10.1007\/978-3-662-54345-0_27","volume-title":"Bildverarbeitung f\u00fcr die Medizin 2017","author":"Maier","year":"2017"},{"issue":"1","key":"2022011921120650700_ref73","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41467-018-06439-z","article-title":"Using both qualitative and quantitative data in parameter identification for systems biology models","volume":"9","author":"Mitra","year":"2018","journal-title":"Nat Commun"},{"issue":"10","key":"2022011921120650700_ref74","doi-asserted-by":"crossref","first-page":"3177","DOI":"10.1093\/bioinformatics\/btaa084","article-title":"Bayesian inference using qualitative observations of underlying continuous variables","volume":"36","author":"Mitra","year":"2020","journal-title":"Bioinformatics"},{"issue":"2","key":"2022011921120650700_ref75","doi-asserted-by":"crossref","first-page":"603","DOI":"10.1007\/s00285-020-01522-w","article-title":"Parameterization of mechanistic models from qualitative data using an efficient optimal scaling approach","volume":"81","author":"Schmiester","year":"2020","journal-title":"J Math Biol"},{"key":"2022011921120650700_ref76","doi-asserted-by":"crossref","DOI":"10.1093\/bioinformatics\/btab512","article-title":"Efficient gradient-based parameter estimation for dynamic models using qualitative data","author":"Schmiester","year":"2021","journal-title":"Bioinformatics"},{"key":"2022011921120650700_ref77","first-page":"49","article-title":"Sur les probl\u00e8mes aux d\u00e9riv\u00e9es partielles et leur signification physique","volume":"13","author":"Hadamard","year":"1902","journal-title":"Princeton Univ Bull"},{"key":"2022011921120650700_ref78","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1016\/j.compchemeng.2015.03.002","article-title":"Nonlinear ill-posed problem analysis in model-based parameter estimation and experimental design","volume":"77","author":"Lopez","year":"2015","journal-title":"Comput Chem Eng"},{"issue":"15","key":"2022011921120650700_ref79","doi-asserted-by":"crossref","first-page":"2321","DOI":"10.1093\/bioinformatics\/btw131","article-title":"Analysis of CFSE time-series data using division-, age-and label-structured population models","volume":"32","author":"Hross","year":"2016","journal-title":"Bioinformatics"},{"issue":"26","key":"2022011921120650700_ref80","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1016\/j.ifacol.2016.12.104","article-title":"New concepts for evaluating the performance of computational methods","volume":"49","author":"Kreutz","year":"2016","journal-title":"IFAC-Papers OnLine"},{"issue":"24","key":"2022011921120650700_ref81","doi-asserted-by":"crossref","first-page":"4266","DOI":"10.1093\/bioinformatics\/bty514","article-title":"Hierarchical optimization for the efficient parametrization of ode models","volume":"34","author":"Loos","year":"2018","journal-title":"Bioinformatics"},{"issue":"2","key":"2022011921120650700_ref82","doi-asserted-by":"crossref","first-page":"594","DOI":"10.1093\/bioinformatics\/btz581","article-title":"Efficient parameterization of large-scale dynamic models based on relative measurements","volume":"36","author":"Schmiester","year":"2020","journal-title":"Bioinformatics"},{"issue":"1","key":"2022011921120650700_ref83","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1186\/s12859-016-1452-4","article-title":"Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy","volume":"18","author":"Penas","year":"2017","journal-title":"BMC Bioinformatics"},{"issue":"8","key":"2022011921120650700_ref84","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pone.0183250","article-title":"Assessing the accuracy of predictive models for numerical data: Not r nor r2, why not? then what?","volume":"12","author":"Li","year":"2017","journal-title":"PLoS One"},{"key":"2022011921120650700_ref85","first-page":"54","article-title":"Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy","volume":"1","author":"Efron","year":"1986","journal-title":"Stat Sci"},{"issue":"3","key":"2022011921120650700_ref86","doi-asserted-by":"crossref","first-page":"657","DOI":"10.1016\/j.automatica.2014.01.001","article-title":"Kernel methods in system identification, machine learning and function estimation: a survey","volume":"50","author":"Pillonetto","year":"2014","journal-title":"Automatica"},{"key":"2022011921120650700_ref87","volume-title":"Mathematical Methods of Statistics (PMS-9)","author":"Cram\u00e9r","year":"2016"},{"key":"2022011921120650700_ref88","article-title":"On structural and practical identifiability","volume":"25","author":"Wieland","year":"2021","journal-title":"Curr Opin Syst Biol"},{"key":"2022011921120650700_ref89","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1042\/bse0450195","article-title":"Parameter estimation and optimal experimental design","volume":"45","author":"Banga","year":"2008","journal-title":"Essays Biochem"},{"key":"2022011921120650700_ref90","doi-asserted-by":"crossref","first-page":"447","DOI":"10.1016\/j.ymben.2006.04.003","article-title":"Exploiting the bootstrap method for quantifying parameter confidence intervals in dynamical systems","volume":"8","author":"Joshi","year":"2006","journal-title":"Metab Eng"},{"key":"2022011921120650700_ref91","first-page":"61","volume-title":"International Conference on Computational Methods in Systems Biology","author":"Fr\u00f6hlich","year":"2014"},{"key":"2022011921120650700_ref92","first-page":"614","article-title":"Bias and confidence in not-quite large samples","volume":"29","author":"Tukey","year":"1958","journal-title":"Ann Math Statist"},{"key":"2022011921120650700_ref93","doi-asserted-by":"crossref","first-page":"586","DOI":"10.1214\/aos\/1176345462","article-title":"The Jackknife estimate of variance","volume":"9","author":"Efron","year":"1981","journal-title":"Ann Stat"},{"issue":"31","key":"2022011921120650700_ref94","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1098\/rsif.2008.0172","article-title":"Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems","volume":"6","author":"Toni","year":"2009","journal-title":"J R Soc Interface"},{"issue":"2","key":"2022011921120650700_ref95","doi-asserted-by":"crossref","first-page":"439","DOI":"10.1038\/nprot.2014.025","article-title":"A framework for parameter estimation and model selection from experimental data in systems biology using approximate Bayesian computation","volume":"9","author":"Liepe","year":"2014","journal-title":"Nat Protoc"},{"issue":"2","key":"2022011921120650700_ref96","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1016\/j.mbs.2013.04.002","article-title":"High-dimensional Bayesian parameter estimation: case study for a model of jak2\/stat5 signaling","volume":"246","author":"Hug","year":"2013","journal-title":"Math Biosci"},{"issue":"8","key":"2022011921120650700_ref97","doi-asserted-by":"crossref","first-page":"1130","DOI":"10.1093\/bioinformatics\/bts088","article-title":"An integrated strategy for prediction uncertainty analysis","volume":"28","author":"Vanlier","year":"2012","journal-title":"Bioinformatics"},{"issue":"15","key":"2022011921120650700_ref98","doi-asserted-by":"crossref","first-page":"1923","DOI":"10.1093\/bioinformatics\/btp358","article-title":"Structural and practical identifiability analysis of partially observed dynamical models by exploiting the profile likelihood","volume":"25","author":"Raue","year":"2009","journal-title":"Bioinformatics"},{"issue":"4","key":"2022011921120650700_ref99","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1049\/iet-syb:20070069","article-title":"Computational procedures for optimal experimental design in biological systems","volume":"2","author":"Balsa-Canto","year":"2008","journal-title":"IET Syst Biol"},{"issue":"7","key":"2022011921120650700_ref100","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pone.0040052","article-title":"Experimental design for parameter estimation of gene regulatory networks","volume":"7","author":"Steiert","year":"2012","journal-title":"PLoS One"},{"key":"2022011921120650700_ref101","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/978-3-642-30367-8","volume-title":"Model Based Parameter Estimation","author":"Bock","year":"2013"},{"issue":"19","key":"2022011921120650700_ref102","doi-asserted-by":"crossref","first-page":"4846","DOI":"10.1016\/j.ces.2007.11.034","article-title":"Model-based design of experiments for parameter precision: state of the art","volume":"63","author":"Franceschini","year":"2008","journal-title":"Chem Eng Sci"},{"issue":"2","key":"2022011921120650700_ref103","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1016\/j.automatica.2007.05.016","article-title":"Optimal experimental design and some related control problems","volume":"44","author":"Pronzato","year":"2008","journal-title":"Automatica"},{"issue":"11","key":"2022011921120650700_ref104","doi-asserted-by":"crossref","first-page":"2564","DOI":"10.1111\/febs.12276","article-title":"Profile likelihood in systems biology","volume":"280","author":"Kreutz","year":"2013","journal-title":"FEBS J"},{"issue":"4","key":"2022011921120650700_ref105","doi-asserted-by":"crossref","first-page":"20130008","DOI":"10.1098\/rsfs.2013.0008","article-title":"Convergence in parameters and predictions using computational experimental design","volume":"3","author":"Hagen","year":"2013","journal-title":"Interface Focus"},{"issue":"4\u20135","key":"2022011921120650700_ref106","doi-asserted-by":"crossref","first-page":"335","DOI":"10.3166\/ejc.11.335-352","article-title":"Identification for control: from the early achievements to the revival of experiment design","volume":"11","author":"Gevers","year":"2005","journal-title":"Eur J Control"},{"issue":"3","key":"2022011921120650700_ref107","doi-asserted-by":"crossref","first-page":"190","DOI":"10.1049\/iet-syb:20060065","article-title":"Optimal experimental design in an epidermal growth factor receptor signalling and down-regulation model","volume":"1","author":"Casey","year":"2007","journal-title":"IET Syst Biol"},{"key":"2022011921120650700_ref108","doi-asserted-by":"crossref","DOI":"10.1021\/acs.iecr.9b04089","article-title":"Closed-loop model-based design of experiments for kinetic model discrimination and parameter estimation: benzoic acid esterification on a heterogeneous catalyst","volume":"58","author":"Waldron","year":"2019","journal-title":"Ind Eng Chem Res"},{"key":"2022011921120650700_ref109","article-title":"A comparison of methods for quantifying prediction uncertainty in systems biology","volume":"52","author":"Villaverde","year":"2019","journal-title":"IFAC-Papers OnLine"},{"issue":"3","key":"2022011921120650700_ref110","doi-asserted-by":"crossref","first-page":"1244","DOI":"10.1021\/acs.iecr.8b03047","article-title":"Sequential model-based a-optimal design of experiments when the fisher information matrix is noninvertible","volume":"58","author":"Shahmohammadi","year":"2019","journal-title":"Ind Eng Chem Res"},{"issue":"1","key":"2022011921120650700_ref111","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1186\/1752-0509-6-120","article-title":"Likelihood based observability analysis and confidence intervals for predictions of dynamic models","volume":"6","author":"Kreutz","year":"2012","journal-title":"BMC Syst Biol"},{"issue":"8","key":"2022011921120650700_ref112","doi-asserted-by":"crossref","first-page":"1204","DOI":"10.1093\/bioinformatics\/btv743","article-title":"Fast integration-based prediction bands for ordinary differential equation models","volume":"32","author":"Hass","year":"2015","journal-title":"Bioinformatics"},{"issue":"3","key":"2022011921120650700_ref113","doi-asserted-by":"crossref","first-page":"184","DOI":"10.1088\/1478-3967\/1\/3\/006","article-title":"The statistical mechanics of complex signaling networks: nerve growth factor signaling","volume":"1","author":"Brown","year":"2004","journal-title":"Phys Biol"},{"issue":"1","key":"2022011921120650700_ref114","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1016\/j.cmpb.2015.02.001","article-title":"A consensus approach for estimating the predictive accuracy of dynamic models in biology","volume":"119","author":"Villaverde","year":"2015","journal-title":"Comput Methods Programs Biomed"},{"issue":"3","key":"2022011921120650700_ref115","doi-asserted-by":"crossref","first-page":"345","DOI":"10.1007\/BF02294361","article-title":"Model selection and Akaike\u2019s information criterion (AIC): the general theory and its analytical extensions","volume":"52","author":"Bozdogan","year":"1987","journal-title":"Psychometrika"},{"issue":"6","key":"2022011921120650700_ref116","doi-asserted-by":"crossref","first-page":"833","DOI":"10.1093\/bioinformatics\/btm607","article-title":"Bayesian ranking of biochemical system models","volume":"24","author":"Vyshemirsky","year":"2008","journal-title":"Bioinformatics"},{"key":"2022011921120650700_ref117","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","article-title":"Regression shrinkage and selection via the LASSO","volume":"58","author":"Tibshirani","year":"1996","journal-title":"J R Stat Soc B Methodol"},{"issue":"17","key":"2022011921120650700_ref118","doi-asserted-by":"crossref","first-page":"i718","DOI":"10.1093\/bioinformatics\/btw461","article-title":"L 1 regularization facilitates detection of cell type-specific parameters in dynamical systems","volume":"32","author":"Steiert","year":"2016","journal-title":"Bioinformatics"}],"container-title":["Briefings in Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/23\/1\/bbab387\/42230172\/bbab387.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/23\/1\/bbab387\/42230172\/bbab387.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,8]],"date-time":"2024-09-08T01:04:27Z","timestamp":1725757467000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bib\/article\/doi\/10.1093\/bib\/bbab387\/6383562"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,9]]},"references-count":118,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,1,17]]}},"URL":"https:\/\/doi.org\/10.1093\/bib\/bbab387","relation":{},"ISSN":["1467-5463","1477-4054"],"issn-type":[{"value":"1467-5463","type":"print"},{"value":"1477-4054","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022,1]]},"published":{"date-parts":[[2021,10,9]]},"article-number":"bbab387"}}