{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,11]],"date-time":"2026-04-11T00:54:07Z","timestamp":1775868847940,"version":"3.50.1"},"reference-count":40,"publisher":"Oxford University Press (OUP)","issue":"22-23","license":[{"start":{"date-parts":[[2020,12,1]],"date-time":"2020-12-01T00:00:00Z","timestamp":1606780800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"name":"Model Driven Computation of Treatments for Infertility Related Endocrinological Diseases","award":["600773"],"award-info":[{"award-number":["600773"]}]},{"name":"INdAM ``GNCS Project"},{"name":"Sapienza University 2018 project","award":["RG11816436BD4F21"],"award-info":[{"award-number":["RG11816436BD4F21"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,4,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Motivation<\/jats:title><jats:p>Model-based approaches to safety and efficacy assessment of pharmacological drugs, treatment strategies or medical devices (In Silico Clinical Trial, ISCT) aim to decrease time and cost for the needed experimentations, reduce animal and human testing, and enable precision medicine. Unfortunately, in presence of non-identifiable models (e.g. reaction networks), parameter estimation is not enough to generate complete populations of Virtual Patients (VPs), i.e. populations guaranteed to show the entire spectrum of model behaviours (phenotypes), thus ensuring representativeness of the trial.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>We present methods and software based on global search driven by statistical model checking that, starting from a (non-identifiable) quantitative model of the human physiology (plus drugs PK\/PD) and suitable biological and medical knowledge elicited from experts, compute a population of VPs whose behaviours are representative of the whole spectrum of phenotypes entailed by the model (completeness) and pairwise distinguishable according to user-provided criteria. This enables full granularity control on the size of the population to employ in an ISCT, guaranteeing representativeness while avoiding over-representation of behaviours. We proved the effectiveness of our algorithm on a non-identifiable ODE-based model of the female Hypothalamic-Pituitary-Gonadal axis, by generating a population of 4\u00a0830\u00a0264 VPs stratified into 7 levels (at different granularity of behaviours), and assessed its representativeness against 86 retrospective health records from Pfizer, Hannover Medical School and University Hospital of Lausanne. The datasets are respectively covered by our VPs within Average Normalized Mean Absolute Error of 15%, 20% and 35% (90% of the latter dataset is covered within 20% error).<\/jats:p><jats:p>Availability and implementation. Our open-source software is available at https:\/\/bitbucket.org\/mclab\/vipgenerator<\/jats:p><\/jats:sec><jats:sec><jats:title>Supplementary information<\/jats:title><jats:p>Supplementary data are available at Bioinformatics online.<\/jats:p><\/jats:sec>","DOI":"10.1093\/bioinformatics\/btaa1026","type":"journal-article","created":{"date-parts":[[2020,11,30]],"date-time":"2020-11-30T12:11:16Z","timestamp":1606738276000},"page":"5465-5472","source":"Crossref","is-referenced-by-count":40,"title":["Complete populations of virtual patients for<i>in silico<\/i>clinical trials"],"prefix":"10.1093","volume":"36","author":[{"given":"S","family":"Sinisi","sequence":"first","affiliation":[{"name":"Computer Science Department, Sapienza University of Rome , Rome, Italy"}]},{"given":"V","family":"Alimguzhin","sequence":"additional","affiliation":[{"name":"Computer Science Department, Sapienza University of Rome , Rome, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3355-2170","authenticated-orcid":false,"given":"T","family":"Mancini","sequence":"additional","affiliation":[{"name":"Computer Science Department, Sapienza University of Rome , Rome, Italy"}]},{"given":"E","family":"Tronci","sequence":"additional","affiliation":[{"name":"Computer Science Department, Sapienza University of Rome , Rome, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4027-6151","authenticated-orcid":false,"given":"B","family":"Leeners","sequence":"additional","affiliation":[{"name":"Department of Reproductive Endocrinology, University Hospital Zurich , Zurich, Switzerland"}]}],"member":"286","published-online":{"date-parts":[[2020,12,16]]},"reference":[{"key":"2023062707202085400_btaa1026-B1","doi-asserted-by":"crossref","first-page":"140","DOI":"10.1002\/psp4.12063","article-title":"Efficient generation and selection of virtual populations in quantitative systems pharmacology models","volume":"5","author":"Allen","year":"2016","journal-title":"CPT Pharm. Syst. Pharmacol"},{"key":"2023062707202085400_btaa1026-B2","year":"2016"},{"key":"2023062707202085400_btaa1026-B3","doi-asserted-by":"crossref","first-page":"208","DOI":"10.1093\/molehr\/gat078","article-title":"Species-specific differences in follicular antral sizes result from diffusion-based limitations on the thickness of the granulosa cell layer","volume":"20","author":"B\u00e4chler","year":"2014","journal-title":"Mol. Hum. Reprod"},{"key":"2023062707202085400_btaa1026-B4","doi-asserted-by":"crossref","first-page":"788","DOI":"10.1002\/psp4.12359","article-title":"A quantitative systems pharmacology kidney model of diabetes associated renal hyperfiltration and the effects of SGLT inhibitors","volume":"7","author":"Balazki","year":"2018","journal-title":"CPT Pharm. Syst. Pharmacol"},{"key":"2023062707202085400_btaa1026-B5","doi-asserted-by":"crossref","first-page":"e1004591","DOI":"10.1371\/journal.pcbi.1004591","article-title":"Computational modeling, formal analysis, and tools for systems biology","volume":"12","author":"Bartocci","year":"2016","journal-title":"PLoS Comput. Biol"},{"key":"2023062707202085400_btaa1026-B6","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1007\/s10928-017-9567-4","article-title":"Boolean network modeling in systems pharmacology","volume":"45","author":"Bloomingdale","year":"2018","journal-title":"J. Pharm. Pharmacodyn"},{"key":"2023062707202085400_btaa1026-B8","doi-asserted-by":"crossref","first-page":"593","DOI":"10.1111\/j.1525-1594.2009.00755.x","article-title":"A mathematical model to evaluate control strategies for mechanical circulatory support","volume":"33","author":"Cox","year":"2009","journal-title":"Artif. Organs"},{"key":"2023062707202085400_btaa1026-B9","year":"2019"},{"key":"2023062707202085400_btaa1026-B10","author":"Eykholt","year":"2018"},{"key":"2023062707202085400_btaa1026-B11","doi-asserted-by":"crossref","first-page":"D649","DOI":"10.1093\/nar\/gkx1132","article-title":"The Reactome pathway knowledgebase","volume":"46","author":"Fabregat","year":"2018","journal-title":"Nucleic Acids Res"},{"key":"2023062707202085400_btaa1026-B12","year":"2018"},{"key":"2023062707202085400_btaa1026-B13","volume-title":"TACAS 2005), LNCS vol. 3440","author":"Grosu","year":"2005"},{"key":"2023062707202085400_btaa1026-B14","doi-asserted-by":"crossref","first-page":"12","DOI":"10.3389\/fphys.2011.00012","article-title":"Hummod: a modeling environment for the simulation of integrative human physiology","volume":"2","author":"Hester","year":"2011","journal-title":"Front. Physiol"},{"key":"2023062707202085400_btaa1026-B15","doi-asserted-by":"crossref","first-page":"524","DOI":"10.1093\/bioinformatics\/btg015","article-title":"The Systems Biology Markup language (SBML): a medium for representation and exchange of biochemical network models","volume":"19","author":"Hucka","year":"2003","journal-title":"Bioinformatics"},{"key":"2023062707202085400_btaa1026-B16","doi-asserted-by":"crossref","first-page":"1040","DOI":"10.1093\/bioinformatics\/btw747","article-title":"Advanced Boolean modeling of biological networks applied to systems pharmacology","volume":"33","author":"Irurzun-Arana","year":"2017","journal-title":"Bioinformatics"},{"key":"2023062707202085400_btaa1026-B17","author":"Jenn","year":"2020"},{"key":"2023062707202085400_btaa1026-B18","doi-asserted-by":"crossref","first-page":"D353","DOI":"10.1093\/nar\/gkw1092","article-title":"KEGG: new perspectives on genomes, pathways, diseases and drugs","volume":"45","author":"Kanehisa","year":"2017","journal-title":"Nucleic Acids Res"},{"key":"2023062707202085400_btaa1026-B19","doi-asserted-by":"crossref","first-page":"198","DOI":"10.1038\/s41467-017-00268-2","article-title":"Unraveling a tumor type-specific regulatory core underlying e2f1-mediated epithelial-mesenchymal transition to predict receptor protein signatures","volume":"8","author":"Khan","year":"2017","journal-title":"Nat. Commun"},{"key":"2023062707202085400_btaa1026-B20","first-page":"44","article-title":"In silico preclinical trials: a proof of concept in closed-loop control of type 1 diabetes","volume":"3","author":"Kovatchev","year":"2009","journal-title":"JDST"},{"key":"2023062707202085400_btaa1026-B21","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1038\/nrg3885","article-title":"Quantitative and logic modelling of molecular and gene networks","volume":"16","author":"Le Nov\u00e8re","year":"2015","journal-title":"Nat. Rev. Genet"},{"key":"2023062707202085400_btaa1026-B22","doi-asserted-by":"crossref","first-page":"878","DOI":"10.1002\/psp4.12473","article-title":"Open systems pharmacology community\u2014an open access, open source, open science approach to modeling and simulation in pharmaceutical sciences","volume":"8","author":"Lippert","year":"2019","journal-title":"CPT Pharm. Syst. Pharmacol"},{"key":"2023062707202085400_btaa1026-B23","doi-asserted-by":"crossref","first-page":"2165","DOI":"10.1093\/bioinformatics\/btz860","article-title":"SBML2Modelica: integrating biochemical models within open-standard simulation ecosystems","volume":"36","author":"Maggioli","year":"2020","journal-title":"Bioinformatics"},{"key":"2023062707202085400_btaa1026-B24","volume-title":"CAV 2013), LNCS vol. 8044","author":"Mancini","year":"2013"},{"key":"2023062707202085400_btaa1026-B25","volume-title":"IWBBIO 2015), LNCS vol. 9044","author":"Mancini","year":"2015"},{"key":"2023062707202085400_btaa1026-B26","first-page":"101","article-title":"SyLVaaS: system level formal verification as a service","volume":"149","author":"Mancini","year":"2016","journal-title":"Fund. Inf"},{"key":"2023062707202085400_btaa1026-B2514191","article-title":"Computing personalised treatments through in silico clinical trials. A case study on downregulation in assisted reproduction","author":"Mancini","year":"2018","journal-title":"In: Int. Works. RCRA 2018. CEUR-WS\u00a0vol.\u00a02271."},{"key":"2023062707202085400_btaa1026-B27","doi-asserted-by":"crossref","first-page":"1699","DOI":"10.1093\/bib\/bby043","article-title":"In silico clinical trials: concepts and early adoptions","volume":"20","author":"Pappalardo","year":"2019","journal-title":"Brief. Bioinf"},{"key":"2023062707202085400_btaa1026-B28","doi-asserted-by":"crossref","first-page":"e1006538","DOI":"10.1371\/journal.pcbi.1006538","article-title":"Computational discovery of dynamic cell line specific boolean networks from multiplex time-course data","volume":"14","author":"Razzaq","year":"2018","journal-title":"PLoS Comput. Biol"},{"key":"2023062707202085400_btaa1026-B29","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/j.pbiomolbio.2018.06.002","article-title":"Improving the generation and selection of virtual populations in quantitative systems pharmacology models","volume":"139","author":"Rieger","year":"2018","journal-title":"Progr. Biophys. Mol. Biol"},{"key":"2023062707202085400_btaa1026-B30","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1016\/j.jtbi.2012.11.020","article-title":"A mathematical model of the human menstrual cycle for the administration of GnRH analogues","volume":"321","author":"R\u00f6blitz","year":"2013","journal-title":"J. Theor. Biol"},{"key":"2023062707202085400_btaa1026-B31","doi-asserted-by":"crossref","first-page":"1717","DOI":"10.1111\/j.2042-7158.2010.01154.x","article-title":"Molecular docking and QSAR studies of aromatase inhibitor androstenedione derivatives","volume":"62","author":"Roy","year":"2010","journal-title":"JPP"},{"key":"2023062707202085400_btaa1026-B32","doi-asserted-by":"crossref","first-page":"594","DOI":"10.1093\/bioinformatics\/btz581","article-title":"Efficient parameterization of large-scale dynamic models based on relative measurements","volume":"36","author":"Schmiester","year":"2019","journal-title":"Bioinformatics"},{"key":"2023062707202085400_btaa1026-B59984157","doi-asserted-by":"crossref","DOI":"10.3233\/FI-2020-1943","article-title":"Optimal personalised treatment computation through in silico clinical trials on patient digital twins","volume":"174","author":"Sinisi","year":"2020","journal-title":"Fundam. Inform"},{"key":"2023062707202085400_btaa1026-B33","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4612-0577-7","volume-title":"Mathematical Control Theory: Deterministic Finite Dimensional Systems","author":"Sontag","year":"1998","edition":"2nd edn"},{"key":"2023062707202085400_btaa1026-B34","doi-asserted-by":"crossref","first-page":"3228","DOI":"10.1007\/s11095-015-1699-x","article-title":"Generating virtual patients by multivariate and discrete re-sampling techniques","volume":"32","author":"Teutonico","year":"2015","journal-title":"Pharm. Res"},{"key":"2023062707202085400_btaa1026-B35","volume-title":"FMCAD 2014)","author":"Tronci","year":"2014"},{"key":"2023062707202085400_btaa1026-B36","volume-title":"Advanced Digital Signal Processing and Noise Reduction","author":"Vaseghi","year":"2009"},{"key":"2023062707202085400_btaa1026-B37","doi-asserted-by":"crossref","first-page":"141","DOI":"10.3389\/fbioe.2020.00141","article-title":"Conducting a virtual clinical trial in HER2-negative breast cancer using a quantitative systems pharmacology model with an epigenetic modulator and immune checkpoint inhibitors","volume":"8","author":"Wang","year":"2020","journal-title":"Front. Bioeng. Biotechnol"},{"key":"2023062707202085400_btaa1026-B38","doi-asserted-by":"crossref","first-page":"055001","DOI":"10.1088\/1478-3975\/9\/5\/055001","article-title":"Boolean modeling in systems biology: an overview of methodology and applications","volume":"9","author":"Wang","year":"2012","journal-title":"Phys. Biol"},{"key":"2023062707202085400_btaa1026-B39","doi-asserted-by":"crossref","first-page":"e60593","DOI":"10.1371\/journal.pone.0060593","article-title":"An efficient algorithm for computing attractors of synchronous and asynchronous Boolean networks","volume":"8","author":"Zheng","year":"2013","journal-title":"PLoS One"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btaa1026\/35383724\/btaa1026.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/36\/22-23\/5465\/50715663\/btaa1026.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/36\/22-23\/5465\/50715663\/btaa1026.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,14]],"date-time":"2023-10-14T07:10:20Z","timestamp":1697267420000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/36\/22-23\/5465\/6039108"}},"subtitle":[],"editor":[{"given":"Jinbo","family":"Xu","sequence":"additional","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2020,12,1]]},"references-count":40,"journal-issue":{"issue":"22-23","published-print":{"date-parts":[[2021,4,1]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btaa1026","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2020,12,1]]},"published":{"date-parts":[[2020,12,1]]}}}