{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,5]],"date-time":"2026-08-05T15:55:50Z","timestamp":1785945350424,"version":"3.56.0"},"reference-count":58,"publisher":"Oxford University Press (OUP)","issue":"2","license":[{"start":{"date-parts":[[2023,1,31]],"date-time":"2023-01-31T00:00:00Z","timestamp":1675123200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Conseller\u00eda de Cultura, Educaci\u00f3n e Ordenaci\u00f3n Universitaria","award":["ED431F 2021\/003"],"award-info":[{"award-number":["ED431F 2021\/003"]}]},{"name":"Conseller\u00eda de Cultura, Educaci\u00f3n e Ordenaci\u00f3n Universitaria","award":["MCIN\/AEI\/10.13039\/501100011033"],"award-info":[{"award-number":["MCIN\/AEI\/10.13039\/501100011033"]}]},{"name":"Conseller\u00eda de Cultura, Educaci\u00f3n e Ordenaci\u00f3n Universitaria","award":["PID2020-113992RA-I00"],"award-info":[{"award-number":["PID2020-113992RA-I00"]}]},{"name":"PREDYCTBIO","award":["MCIN\/AEI\/10.13039\/501100011033"],"award-info":[{"award-number":["MCIN\/AEI\/10.13039\/501100011033"]}]},{"name":"PREDYCTBIO","award":["RYC-2019-027537-I"],"award-info":[{"award-number":["RYC-2019-027537-I"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,2,3]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Motivation<\/jats:title>\n                  <jats:p>The theoretical possibility of determining the state and parameters of a dynamic model by measuring its outputs is given by its structural identifiability and its observability. These properties should be analysed before attempting to calibrate a model, but their a priori analysis can be challenging, requiring symbolic calculations that often have a high computational cost. In recent years, a number of software tools have been developed for this task, mostly in the systems biology community. These tools have vastly different features and capabilities, and a critical assessment of their performance is still lacking.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>Here, we present a comprehensive study of the computational resources available for analysing structural identifiability. We consider 13 software tools developed in 7 programming languages and evaluate their performance using a set of 25 case studies created from 21 models. Our results reveal their strengths and weaknesses, provide guidelines for choosing the most appropriate tool for a given problem and highlight opportunities for future developments.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Availability and implementation<\/jats:title>\n                  <jats:p>https:\/\/github.com\/Xabo-RB\/Benchmarking_files.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btad065","type":"journal-article","created":{"date-parts":[[2023,1,30]],"date-time":"2023-01-30T21:20:05Z","timestamp":1675113605000},"source":"Crossref","is-referenced-by-count":60,"title":["Benchmarking tools for a priori identifiability analysis"],"prefix":"10.1093","volume":"39","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2390-4462","authenticated-orcid":false,"given":"Xabier","family":"Rey Barreiro","sequence":"first","affiliation":[{"name":"Department of Systems and Control Engineering, Universidade de Vigo , 36310 Vigo, Galicia, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7401-7380","authenticated-orcid":false,"given":"Alejandro F","family":"Villaverde","sequence":"additional","affiliation":[{"name":"Department of Systems and Control Engineering, Universidade de Vigo , 36310 Vigo, Galicia, Spain"},{"name":"CITMAga , 15782 Santiago de Compostela, Galicia, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2023,1,31]]},"reference":[{"key":"2023021015010538100_btad065-B1","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1016\/j.mbs.2012.04.005","article-title":"Minimal output sets for identifiability","volume":"239","author":"Anguelova","year":"2012","journal-title":"Math. Biosci"},{"key":"2023021015010538100_btad065-B2","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1016\/j.arcontrol.2020.10.006","article-title":"A priori identifiability: an overview on definitions and approaches","volume":"50","author":"Anstett-Collin","year":"2020","journal-title":"Annu. Rev. Control"},{"key":"2023021015010538100_btad065-B3","doi-asserted-by":"crossref","first-page":"516","DOI":"10.1038\/msb.2011.50","article-title":"Division of labor by dual feedback regulators controls JAK2\/STAT5 signaling over broad ligand range","volume":"7","author":"Bachmann","year":"2011","journal-title":"Mol. Syst. Biol"},{"key":"2023021015010538100_btad065-B4","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/1752-0509-4-11","article-title":"An iterative identification procedure for dynamic modeling of biochemical networks","volume":"4","author":"Balsa-Canto","year":"2010","journal-title":"BMC Syst. Biol"},{"key":"2023021015010538100_btad065-B5","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1016\/j.cmpb.2007.07.002","article-title":"DAISY: a new software tool to test global identifiability of biological and physiological systems","volume":"88","author":"Bellu","year":"2007","journal-title":"Comput. Methods Programs Biomed"},{"key":"2023021015010538100_btad065-B6","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511565847","volume-title":"Gr\u00f6bner Bases and Applications","author":"Buchberger","year":"1998"},{"key":"2023021015010538100_btad065-B7","doi-asserted-by":"crossref","first-page":"2610","DOI":"10.1093\/bioinformatics\/btr431","article-title":"GenSSI: a software toolbox for structural identifiability analysis of biological models","volume":"27","author":"Chi\u015f","year":"2011","journal-title":"Bioinformatics"},{"key":"2023021015010538100_btad065-B8","doi-asserted-by":"crossref","first-page":"e27755","DOI":"10.1371\/journal.pone.0027755","article-title":"Structural identifiability of systems biology models: a critical comparison of methods","volume":"6","author":"Chis","year":"2011","journal-title":"PLoS One"},{"key":"2023021015010538100_btad065-B9","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1038\/newbio239251a0","article-title":"Application of a new model of species competition to drosophila","volume":"239","author":"Coleman","year":"1972","journal-title":"Nat. New Biol"},{"key":"2023021015010538100_btad065-B10","doi-asserted-by":"crossref","first-page":"507","DOI":"10.1016\/j.bpj.2017.11.3787","article-title":"Dynamics of posttranslational modification systems: recent progress and future directions","volume":"114","author":"Conradi","year":"2018","journal-title":"Biophys. J"},{"key":"2023021015010538100_btad065-B11","doi-asserted-by":"crossref","first-page":"btac748","DOI":"10.1093\/bioinformatics\/btac748","article-title":"STRIKE-GOLDD 4.0: user-friendly, efficient analysis of structural identifiability and observability","volume":"39","author":"D\u00edaz","year":"2023","journal-title":"Bioinformatics"},{"key":"2023021015010538100_btad065-B12","volume-title":"Dynamic Systems Biology Modeling and Simulation","author":"Distefano","year":"2015"},{"key":"2023021015010538100_btad065-B13","author":"Dong","year":"2022"},{"key":"2023021015010538100_btad065-B14","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1016\/j.jtbi.2017.07.018","article-title":"A confidence building exercise in data and identifiability: modeling cancer chemotherapy as a case study","volume":"431","author":"Eisenberg","year":"2017","journal-title":"J. Theor. Biol"},{"key":"2023021015010538100_btad065-B15","doi-asserted-by":"crossref","first-page":"728","DOI":"10.1109\/TAC.1977.1101601","article-title":"Nonlinear controllability and observability","volume":"22","author":"Hermann","year":"1977","journal-title":"IEEE Trans. Automat. Contr"},{"key":"2023021015010538100_btad065-B16","doi-asserted-by":"crossref","first-page":"2873","DOI":"10.1093\/bioinformatics\/bty1069","article-title":"SIAN: software for structural identifiability analysis of ode models","volume":"35","author":"Hong","year":"2019","journal-title":"Bioinformatics"},{"key":"2023021015010538100_btad065-B17","doi-asserted-by":"crossref","first-page":"1831","DOI":"10.1002\/cpa.21921","article-title":"Global identifiability of differential models","volume":"73","author":"Hong","year":"2020","journal-title":"Comm. Pure Appl. Math"},{"key":"2023021015010538100_btad065-B18","first-page":"254","author":"Ilmer","year":"2021"},{"key":"2023021015010538100_btad065-B19","doi-asserted-by":"crossref","first-page":"590","DOI":"10.3389\/fphys.2016.00590","article-title":"Parameter identifiability of fundamental pharmacodynamic models","volume":"7","author":"Janz\u00e9n","year":"2016","journal-title":"Front. Physiol"},{"key":"2023021015010538100_btad065-B20","doi-asserted-by":"crossref","first-page":"941","DOI":"10.3182\/20120711-3-BE-2027.00381","article-title":"An efficient method for structural identifiability analysis of large dynamic systems","volume":"45","author":"Karlsson","year":"2012","journal-title":"IFAC Proc. Vol"},{"key":"2023021015010538100_btad065-B21","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1016\/j.jtbi.2017.01.032","article-title":"Model distinguishability and inference robustness in mechanisms of cholera transmission and loss of immunity","volume":"420","author":"Lee","year":"2017","journal-title":"J. Theor. Biol"},{"key":"2023021015010538100_btad065-B22","doi-asserted-by":"crossref","first-page":"1421","DOI":"10.1093\/bioinformatics\/btx735","article-title":"GenSSI 2.0: multi-experiment structural identifiability analysis of SBML models","volume":"34","author":"Ligon","year":"2018","journal-title":"Bioinformatics"},{"key":"2023021015010538100_btad065-B23","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1016\/j.jtbi.2004.01.001","article-title":"Mathematical model of NF-\u03baB regulatory module","volume":"228","author":"Lipniacki","year":"2004","journal-title":"J. Theor. Biol"},{"key":"2023021015010538100_btad065-B24","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1016\/0005-1098(94)90029-9","article-title":"On global identifiability for arbitrary model parametrizations","volume":"30","author":"Ljung","year":"1994","journal-title":"Automatica"},{"key":"2023021015010538100_btad065-B25","doi-asserted-by":"crossref","first-page":"383","DOI":"10.1016\/j.jtbi.2004.11.038","article-title":"Modelling genetic networks with noisy and varied experimental data: the circadian clock in Arabidopsis thaliana","volume":"234","author":"Locke","year":"2005","journal-title":"J. Theor. Biol"},{"key":"2023021015010538100_btad065-B26","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41467-017-01498-0","article-title":"Balancing a genetic toggle switch by real-time feedback control and periodic forcing","volume":"8","author":"Lugagne","year":"2017","journal-title":"Nat. Commun"},{"key":"2023021015010538100_btad065-B27","doi-asserted-by":"crossref","first-page":"378","DOI":"10.1016\/j.ymssp.2019.05.010","article-title":"Observability of nonlinear systems with unmeasured inputs","volume":"130","author":"Maes","year":"2019","journal-title":"Mech. Syst. Signal Process"},{"key":"2023021015010538100_btad065-B28","doi-asserted-by":"crossref","DOI":"10.1002\/rnc.5887","article-title":"AutoRepar: a method to obtain identifiable and observable reparameterizations of dynamic models with mechanistic insights","author":"Massonis","year":"2021","journal-title":"Int. J. Robust Nonlinear Control"},{"key":"2023021015010538100_btad065-B29","doi-asserted-by":"crossref","first-page":"012920","DOI":"10.1103\/PhysRevE.92.012920","article-title":"Higher-order lie symmetries in identifiability and predictability analysis of dynamic models","volume":"92","author":"Merkt","year":"2015","journal-title":"Phys. Rev. E"},{"key":"2023021015010538100_btad065-B30","doi-asserted-by":"crossref","first-page":"e110261","DOI":"10.1371\/journal.pone.0110261","article-title":"On finding and using identifiable parameter combinations in nonlinear dynamic systems biology models and COMBOS: a novel web implementation","volume":"9","author":"Meshkat","year":"2014","journal-title":"PLoS One"},{"key":"2023021015010538100_btad065-B31","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1137\/090757009","article-title":"On identifiability of nonlinear ODE models and applications in viral dynamics","volume":"53","author":"Miao","year":"2011","journal-title":"SIAM Rev"},{"key":"2023021015010538100_btad065-B32","doi-asserted-by":"crossref","first-page":"731","DOI":"10.3168\/jds.2007-0398","article-title":"Kinetics of ruminal lipolysis of triacylglycerol and biohydrogenation of long-chain fatty acids: new insights from old data","volume":"91","author":"Moate","year":"2008","journal-title":"J. Dairy Sci"},{"key":"2023021015010538100_btad065-B33","doi-asserted-by":"crossref","first-page":"701","DOI":"10.1017\/S1751731117002774","article-title":"To be or not to be an identifiable model. Is this a relevant question in animal science modelling?","volume":"12","author":"Mu\u00f1oz-Tamayo","year":"2018","journal-title":"Animal"},{"key":"2023021015010538100_btad065-B34","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/srep12569","article-title":"DYVIPAC: an integrated analysis and visualisation framework to probe multi-dimensional biological networks","volume":"5","author":"Nguyen","year":"2015","journal-title":"Sci. Rep"},{"key":"2023021015010538100_btad065-B35","doi-asserted-by":"crossref","first-page":"105030","DOI":"10.1016\/j.sysconle.2021.105030","article-title":"Computing all identifiable functions of parameters for ode models","volume":"157","author":"Ovchinnikov","year":"2021","journal-title":"Syst. Control Lett"},{"key":"2023021015010538100_btad065-B36","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1137\/S0036144598335107","article-title":"Mathematical analysis of HIV-1 dynamics in vivo","volume":"41","author":"Perelson","year":"1999","journal-title":"SIAM Rev"},{"key":"2023021015010538100_btad065-B37","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1016\/0025-5564(78)90063-9","article-title":"System identifiability based on the power series expansion of the solution","volume":"41","author":"Pohjanpalo","year":"1978","journal-title":"Math. Biosci"},{"key":"2023021015010538100_btad065-B38","doi-asserted-by":"crossref","first-page":"693","DOI":"10.1158\/0008-5472.CAN-10-2987","article-title":"Dynamic mathematical modeling of il13-induced signaling in hodgkin and primary mediastinal b-cell lymphoma allows prediction of therapeutic targets","volume":"71","author":"Raia","year":"2011","journal-title":"Cancer Res"},{"key":"2023021015010538100_btad065-B39","doi-asserted-by":"crossref","first-page":"1923","DOI":"10.1093\/bioinformatics\/btp358","article-title":"Structural and practical identifiability analysis of partially observed dynamical models by exploiting the profile likelihood","volume":"25","author":"Raue","year":"2009","journal-title":"Bioinformatics"},{"key":"2023021015010538100_btad065-B40","doi-asserted-by":"crossref","first-page":"1440","DOI":"10.1093\/bioinformatics\/btu006","article-title":"Comparison of approaches for parameter identifiability analysis of biological systems","volume":"30","author":"Raue","year":"2014","journal-title":"Bioinformatics"},{"key":"2023021015010538100_btad065-B41","first-page":"430","volume-title":"XLIII Jornadas de Autom\u00e1tica: libro de Actas","author":"Rey Rostro","year":"2022"},{"key":"2023021015010538100_btad065-B42","doi-asserted-by":"crossref","DOI":"10.1090\/coll\/033","volume-title":"Differential Algebra","author":"Ritt","year":"1950"},{"key":"2023021015010538100_btad065-B43","first-page":"3108","author":"Saccomani","year":"2001"},{"key":"2023021015010538100_btad065-B44","doi-asserted-by":"crossref","first-page":"619","DOI":"10.1016\/S0005-1098(02)00302-3","article-title":"Parameter identifiability of nonlinear systems: the role of initial conditions","volume":"39","author":"Saccomani","year":"2003","journal-title":"Automatica"},{"key":"2023021015010538100_btad065-B45","doi-asserted-by":"crossref","first-page":"735","DOI":"10.1006\/jsco.2002.0532","article-title":"A probabilistic algorithm to test local algebraic observability in polynomial time","volume":"33","author":"Sedoglavic","year":"2002","journal-title":"J. Symb. Comput"},{"key":"2023021015010538100_btad065-B46","doi-asserted-by":"crossref","first-page":"108345","DOI":"10.1016\/j.ymssp.2021.108345","article-title":"An efficient algorithm to test the observability of rational nonlinear systems with unmeasured inputs","volume":"165","author":"Shi","year":"2022","journal-title":"Mech. Syst. Signal Process"},{"key":"2023021015010538100_btad065-B47","doi-asserted-by":"crossref","first-page":"138","DOI":"10.1016\/j.ifacol.2021.08.348","article-title":"Computing measures of identifiability, observability, and controllability for a dynamic system model with the StrucID app","volume":"54","author":"Stigter","year":"2021","journal-title":"IFAC-PapersOnLine"},{"key":"2023021015010538100_btad065-B48","doi-asserted-by":"crossref","first-page":"118","DOI":"10.1016\/j.automatica.2015.05.004","article-title":"A fast algorithm to assess local structural identifiability","volume":"58","author":"Stigter","year":"2015","journal-title":"Automatica"},{"key":"2023021015010538100_btad065-B49","first-page":"3290","article-title":"Effect of dose, molecular size, affinity, and protein binding on tumor uptake of antibody or ligand: a biomathematical model","volume":"49","author":"Thomas","year":"1989","journal-title":"Cancer Res"},{"key":"2023021015010538100_btad065-B50","doi-asserted-by":"crossref","first-page":"605","DOI":"10.1006\/jtbi.2000.2150","article-title":"A model of \u03b2-cell mass, insulin, and glucose kinetics: pathways to diabetes","volume":"206","author":"Topp","year":"2000","journal-title":"J. Theor. Biol"},{"key":"2023021015010538100_btad065-B51","first-page":"517","article-title":"Identifiability and estimation of pharmacokinetic parameters for the ligands of the macrophage mannose receptor","volume":"15","author":"Verdiere","year":"2005","journal-title":"Int. J. Appl. Math. Com.-Pol"},{"key":"2023021015010538100_btad065-B52","doi-asserted-by":"crossref","first-page":"e1005153","DOI":"10.1371\/journal.pcbi.1005153","article-title":"Structural identifiability of dynamic systems biology models","volume":"12","author":"Villaverde","year":"2016","journal-title":"PLoS Comput. Biol"},{"key":"2023021015010538100_btad065-B53","doi-asserted-by":"crossref","first-page":"20190043","DOI":"10.1098\/rsif.2019.0043","article-title":"Full observability and estimation of unknown inputs, states and parameters of nonlinear biological models","volume":"16","author":"Villaverde","year":"2019","journal-title":"J. R Soc. Interface"},{"key":"2023021015010538100_btad065-B54","doi-asserted-by":"crossref","first-page":"bbab387","DOI":"10.1093\/bib\/bbab387","article-title":"A protocol for dynamic model calibration","volume":"23","author":"Villaverde","year":"2022","journal-title":"Brief. Bioinform"},{"key":"2023021015010538100_btad065-B55","doi-asserted-by":"crossref","first-page":"472","DOI":"10.1016\/0378-4754(82)90645-0","article-title":"Global approaches to identifiability testing for linear and nonlinear state space models","volume":"24","author":"Walter","year":"1982","journal-title":"Math. Comput. Simul"},{"key":"2023021015010538100_btad065-B56","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1016\/S0025-5564(01)00066-9","article-title":"Modeling epidemics caused by respiratory syncytial virus (RSV)","volume":"172","author":"Weber","year":"2001","journal-title":"Math. Biosci"},{"key":"2023021015010538100_btad065-B57","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1016\/j.coisb.2021.03.005","article-title":"On structural and practical identifiability","volume":"25","author":"Wieland","year":"2021","journal-title":"Curr. Opin. Syst. Biol"},{"key":"2023021015010538100_btad065-B58","doi-asserted-by":"crossref","first-page":"14464","DOI":"10.1073\/pnas.96.25.14464","article-title":"Specific therapy regimes could lead to long-term immunological control of HIV","volume":"96","author":"Wodarz","year":"1999","journal-title":"Proc. Natl. Acad. Sci. USA"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btad065\/49002669\/btad065.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/39\/2\/btad065\/49156528\/btad065.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/39\/2\/btad065\/49156528\/btad065.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,2,10]],"date-time":"2023-02-10T15:44:46Z","timestamp":1676043886000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/doi\/10.1093\/bioinformatics\/btad065\/7017524"}},"subtitle":[],"editor":[{"given":"Jonathan","family":"Wren","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2023,1,31]]},"references-count":58,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2023,2,3]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btad065","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,2,1]]},"published":{"date-parts":[[2023,1,31]]},"article-number":"btad065"}}