{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,23]],"date-time":"2025-10-23T16:57:20Z","timestamp":1761238640937,"version":"3.41.2"},"reference-count":58,"publisher":"Oxford University Press (OUP)","issue":"2","license":[{"start":{"date-parts":[[2019,8,2]],"date-time":"2019-08-02T00:00:00Z","timestamp":1564704000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["1DP2MH103909-01","5U01HG009088-02","U54GM088558-09","5R37AI051164-12","1R01AI112339-01","U54GM088558-06"],"award-info":[{"award-number":["1DP2MH103909-01","5U01HG009088-02","U54GM088558-09","5R37AI051164-12","1R01AI112339-01","U54GM088558-06"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001711","name":"Swiss National Science Foundation","doi-asserted-by":"publisher","award":["CR12I1_156229","105218_163196"],"award-info":[{"award-number":["CR12I1_156229","105218_163196"]}],"id":[{"id":"10.13039\/501100001711","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2020,4,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Network models are applied across many domains where data can be represented as a network. Two prominent paradigms for modelling networks are statistical models (probabilistic models for the observed network) and mechanistic models (models for network growth and\/or evolution). Mechanistic models are better suited for incorporating domain knowledge, to study effects of interventions (such as changes to specific mechanisms) and to forward simulate, but they typically have intractable likelihoods. As such, and in a stark contrast to statistical models, there is a relative dearth of research on model selection for such models despite the otherwise large body of extant work. In this article, we propose a simulator-based procedure for mechanistic network model selection that borrows aspects from Approximate Bayesian Computation along with a means to quantify the uncertainty in the selected model. To select the most suitable network model, we consider and assess the performance of several learning algorithms, most notably the so-called Super Learner, which makes our framework less sensitive to the choice of a particular learning algorithm. Our approach takes advantage of the ease to forward simulate from mechanistic network models to circumvent their intractable likelihoods. The overall process is flexible and widely applicable. Our simulation results demonstrate the approach\u2019s ability to accurately discriminate between competing mechanistic models. Finally, we showcase our approach with a protein\u2013protein interaction network model from the literature for yeast (Saccharomyces cerevisiae).<\/jats:p>","DOI":"10.1093\/comnet\/cnz024","type":"journal-article","created":{"date-parts":[[2019,6,25]],"date-time":"2019-06-25T18:31:25Z","timestamp":1561487485000},"source":"Crossref","is-referenced-by-count":10,"title":["Flexible model selection for mechanistic network models"],"prefix":"10.1093","volume":"8","author":[{"given":"Sixing","family":"Chen","sequence":"first","affiliation":[{"name":"Department of Biostatistics, T.H. Chan School of Public Health, Harvard University 655 Huntington Avenue, Building 2, 4th Floor, Boston, MA 02115, USA"}]},{"given":"Antonietta","family":"Mira","sequence":"additional","affiliation":[{"name":"Data Science Lab, Institute of Computational Science, Universit\u00e0 della Svizzera italiana Via Buffi 6, 6900 Lugano, Switzerland and Dipartimento di Scienza e Alta Tecnologia, Universit\u00e0 degli Studi dell\u2019Insubria Via Valleggio, 11 - 22100 Como, Italy"}]},{"given":"Jukka-Pekka","family":"Onnela","sequence":"additional","affiliation":[{"name":"Department of Biostatistics, T.H. Chan School of Public Health, Harvard University 655 Huntington Avenue, Building 2, 4th Floor, Boston, MA 02115, USA"}]}],"member":"286","published-online":{"date-parts":[[2019,8,2]]},"reference":[{"key":"2020072910570128800_B1","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511815478","volume-title":"Social Network Analysis: Methods and Applications","author":"Wasserman,","year":"1994"},{"volume-title":"Evolution and Structure of the Internet: A Statistical Physics Approach","year":"2007","author":"Pastor-Satorras,","key":"2020072910570128800_B2"},{"key":"2020072910570128800_B3","doi-asserted-by":"crossref","DOI":"10.1093\/acprof:oso\/9780199206650.001.0001","volume-title":"Networks: An Introduction","author":"Newman,","year":"2010"},{"key":"2020072910570128800_B4","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511894701","volume-title":"Exponential Random Graph Models for Social Networks: Theory, Methods, and Applications","author":"Lusher,","year":"2012"},{"volume-title":"Introduction to Biological Networks","year":"2013","author":"Raval,","key":"2020072910570128800_B5"},{"key":"2020072910570128800_B6","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1016\/j.socnet.2006.08.002","article-title":"An introduction to exponential random graph ($p^*$) models for social networks","volume":"29","author":"Robins,","year":"2007","journal-title":"Soc. Netw."},{"key":"2020072910570128800_B7","doi-asserted-by":"crossref","first-page":"1090","DOI":"10.1198\/016214502388618906","article-title":"Latent space approaches to social network analysis","volume":"97","author":"Hoff,","year":"2002","journal-title":"J. Am. Stat. Assoc."},{"key":"2020072910570128800_B8","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1017\/nws.2014.2","article-title":"Sampling networks from their posterior predictive distribution","volume":"2","author":"Goyal,","year":"2014","journal-title":"Netw. Sci."},{"key":"2020072910570128800_B9","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1126\/science.286.5439.509","article-title":"Emergence of scaling in random networks","volume":"286","author":"Barab\u00e1si,","year":"1999","journal-title":"Science"},{"key":"2020072910570128800_B10","doi-asserted-by":"crossref","first-page":"440","DOI":"10.1038\/30918","article-title":"Collective dynamics of \u2018small-world\u2019 networks","volume":"393","author":"Watts,","year":"1998","journal-title":"Nature"},{"key":"2020072910570128800_B11","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1142\/S021952590200047X","article-title":"A model of large-scale proteome evolution","volume":"5","author":"Sol\u00e9,","year":"2002","journal-title":"Adv. Complex Syst."},{"key":"2020072910570128800_B12","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1159\/000067642","article-title":"Modeling of protein interaction networks","volume":"1","author":"V\u00e1zquez,","year":"2003","journal-title":"Complexus"},{"key":"2020072910570128800_B13","doi-asserted-by":"crossref","first-page":"036123","DOI":"10.1103\/PhysRevE.65.036123","article-title":"Highly clustered scale-free networks","volume":"65","author":"Klemm,","year":"2002","journal-title":"Phys. Rev. E"},{"key":"2020072910570128800_B14","doi-asserted-by":"crossref","first-page":"228701","DOI":"10.1103\/PhysRevLett.99.228701","article-title":"Emergence of communities in weighted networks","volume":"99","author":"Kumpula,","year":"2007","journal-title":"Phys. Rev. Lett."},{"key":"2020072910570128800_B15","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1145\/1592665.1592675","article-title":"On the evolution of user interaction in facebook","volume-title":"Proceedings of the 2nd ACM Workshop on Online Social Networks","author":"Viswanath,","year":"2009"},{"key":"2020072910570128800_B16","doi-asserted-by":"crossref","first-page":"7332","DOI":"10.1073\/pnas.0610245104","article-title":"Structure and tie strengths in mobile communication networks","volume":"104","author":"Onnela,","year":"2007","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"2020072910570128800_B17","doi-asserted-by":"crossref","first-page":"164","DOI":"10.1140\/epjb\/e2015-60106-6","article-title":"From seconds to months: an overview of multi-scale dynamics of mobile telephone calls","volume":"88","author":"Saram\u00e4ki,","year":"2015","journal-title":"Eur. Phys. J. B"},{"journal-title":"Mechanistic and probabilistic network models (in progress)","author":"Goyal,","key":"2020072910570128800_B18"},{"key":"2020072910570128800_B19","doi-asserted-by":"crossref","first-page":"508","DOI":"10.1214\/12-AOS1044","article-title":"Consistency under sampling of exponential random graph models","volume":"41","author":"Shalizi,","year":"2013","journal-title":"Ann. Stat."},{"key":"2020072910570128800_B20","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1177\/1740774514523351","article-title":"Sample size considerations in the design of cluster randomized trials of combination HIV prevention","volume":"11","author":"Wang,","year":"2014","journal-title":"Clin. Trials"},{"key":"2020072910570128800_B21","doi-asserted-by":"crossref","first-page":"3192","DOI":"10.1073\/pnas.0409515102","article-title":"Inferring network mechanisms: the Drosophila melanogaster protein interaction network","volume":"102","author":"Middendorf,","year":"2005","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"2020072910570128800_B22","doi-asserted-by":"crossref","first-page":"10576","DOI":"10.1073\/pnas.0807882106","article-title":"Model criticism based on likelihood-free inference, with an application to protein network evolution","volume":"106","author":"Ratmann,","year":"2009","journal-title":"Proc. Nat. Acad. Sci., USA"},{"key":"2020072910570128800_B23","doi-asserted-by":"crossref","first-page":"2653","DOI":"10.1098\/rsif.2012.0220","article-title":"Graph spectral analysis of protein interaction network evolution","volume":"9","author":"Thorne,","year":"2012","journal-title":"J. R. Soc. Interface"},{"journal-title":"Statistical inference and model selection for mechanistic network models (in progress)","author":"Onnela,","key":"2020072910570128800_B24"},{"key":"2020072910570128800_B25","doi-asserted-by":"crossref","first-page":"1167","DOI":"10.1007\/s11222-011-9288-2","article-title":"Approximate Bayesian computational methods","volume":"22","author":"Marin,","year":"2012","journal-title":"Stat. Comput."},{"key":"2020072910570128800_B26","doi-asserted-by":"crossref","first-page":"e1002803","DOI":"10.1371\/journal.pcbi.1002803","article-title":"Approximate Bayesian computation","volume":"9","author":"Sunn\u00e5ker,","year":"2013","journal-title":"PLoS Comput. Biol."},{"key":"2020072910570128800_B27","first-page":"e66","article-title":"Fundamentals and recent developments in approximate Bayesian computation","volume":"66","author":"Lintusaari,","year":"2017","journal-title":"Syst. Biol."},{"key":"2020072910570128800_B28","doi-asserted-by":"crossref","DOI":"10.1201\/9781315117195","volume-title":"Handbook of Approximate Bayesian Computation","author":"Sisson,","year":"2018"},{"key":"2020072910570128800_B29","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1146\/annurev-ecolsys-102209-144621","article-title":"Approximate Bayesian computation in evolution and ecology","volume":"41","author":"Beaumont,","year":"2010","journal-title":"Annu. Rev. Ecol. Evol. Syst."},{"key":"2020072910570128800_B30","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1214\/09-BA412","article-title":"ABC likelihood-free methods for model choice in Gibbs random fields","volume":"4","author":"Grelaud,","year":"2009","journal-title":"Bayesian Anal."},{"key":"2020072910570128800_B31","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1098\/rsif.2008.0172","article-title":"Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems","volume":"6","author":"Toni,","year":"2009","journal-title":"J. R. Soc. Interface"},{"key":"2020072910570128800_B32","doi-asserted-by":"crossref","first-page":"1760","DOI":"10.1073\/pnas.0607208104","article-title":"Sequential monte carlo without likelihoods","volume":"104","author":"Sisson,","year":"2007","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"2020072910570128800_B33","doi-asserted-by":"crossref","first-page":"198","DOI":"10.1111\/biom.12249","article-title":"Model choice problems using approximate Bayesian computation with applications to pathogen transmission data sets","volume":"71","author":"Lee,","year":"2015","journal-title":"Biometrics"},{"key":"2020072910570128800_B34","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1111\/j.1467-9868.2011.01010.x","article-title":"Constructing summary statistics for approximate Bayesian computation: semi-automatic approximate Bayesian computation","volume":"74","author":"Fearnhead,","year":"2012","journal-title":"J. R. Stat. Soc. Ser. B (Stat. Methodol.)"},{"key":"2020072910570128800_B35","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1007\/s11222-014-9514-9","article-title":"Adaptive ABC model choice and geometric summary statistics for hidden Gibbs random fields","volume":"25","author":"Stoehr,","year":"2015","journal-title":"Stat. Comput."},{"key":"2020072910570128800_B36","doi-asserted-by":"crossref","first-page":"859","DOI":"10.1093\/bioinformatics\/btv684","article-title":"Reliable ABC model choice via random forests","volume":"32","author":"Pudlo,","year":"2015","journal-title":"Bioinformatics"},{"key":"2020072910570128800_B37","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1214\/15-EJS988","article-title":"The rate of convergence for approximate Bayesian computation","volume":"9","author":"Barber,","year":"2015","journal-title":"Electron. J. Stat."},{"key":"2020072910570128800_B38","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1111\/j.1467-9876.2010.00747.x","article-title":"Approximate Bayesian computation using indirect inference","volume":"60","author":"Drovandi,","year":"2011","journal-title":"J. R. Stat. Soc. Ser. C (Appl. Stat.)"},{"key":"2020072910570128800_B39","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1515\/sagmb-2013-0012","article-title":"Semi-automatic selection of summary statistics for ABC model choice","volume":"13","author":"Prangle,","year":"2014","journal-title":"Stat. Appl. Genet. Mol. Biol."},{"key":"2020072910570128800_B40","doi-asserted-by":"crossref","first-page":"15112","DOI":"10.1073\/pnas.1102900108","article-title":"Lack of confidence in approximate Bayesian computation model choice","volume":"108","author":"Robert,","year":"2011","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"2020072910570128800_B41","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1007\/978-1-4419-9782-1_3","article-title":"Super Learning","volume-title":"Targeted Learning","author":"Polley,","year":"2011"},{"key":"2020072910570128800_B42","article-title":"Super Learner","volume":"6","author":"Van der,","year":"2007","journal-title":"Stat. Appl. Genet. Mol. Biol."},{"key":"2020072910570128800_B43","article-title":"Unified cross-validation methodology for selection among estimators and a general cross-validated adaptive epsilon-net estimator: finite sample oracle inequalities and examples","volume-title":"Technical Report","author":"Van Der Laan,","year":"2003"},{"key":"2020072910570128800_B44","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1016\/j.stamet.2005.02.003","article-title":"Asymptotics of cross-validated risk estimation in estimator selection and performance assessment","volume":"2","author":"Dudoit,","year":"2005","journal-title":"Stat. Methodol."},{"key":"2020072910570128800_B45","doi-asserted-by":"crossref","first-page":"1145","DOI":"10.1016\/S0031-3203(96)00142-2","article-title":"The use of the area under the ROC curve in the evaluation of machine learning algorithms","volume":"30","author":"Bradley,","year":"1997","journal-title":"Pattern Recognit."},{"key":"2020072910570128800_B46","first-page":"519","article-title":"AUC: a statistically consistent and more discriminating measure than accuracy","volume":"3","author":"Ling,","year":"2003","journal-title":"IJCAI"},{"key":"2020072910570128800_B47","doi-asserted-by":"crossref","first-page":"290","DOI":"10.5486\/PMD.1959.6.3-4.12","article-title":"On random graphs I","volume":"6","author":"Erd\u00f6s,","year":"1959","journal-title":"Publ. Math. Debrecen"},{"key":"2020072910570128800_B48","doi-asserted-by":"crossref","first-page":"1360","DOI":"10.1086\/225469","article-title":"The strength of weak ties","volume":"78","author":"Granovetter,","year":"1973","journal-title":"Am. J. Sociol."},{"key":"2020072910570128800_B49","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1093\/aje\/kwu253","article-title":"Improving propensity score estimators\u2019 robustness to model misspecification using super learner","volume":"181","author":"Pirracchio,","year":"2014","journal-title":"Am. J. Epidemiol."},{"key":"2020072910570128800_B50","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1016\/S2213-2600(14)70239-5","article-title":"Mortality prediction in intensive care units with the Super ICU Learner Algorithm (SICULA): a population-based study","volume":"3","author":"Pirracchio,","year":"2015","journal-title":"Lancet Respiratory Med."},{"key":"2020072910570128800_B51","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1097\/QAI.0000000000000548","article-title":"Super learner analysis of electronic adherence data improves viral prediction and may provide strategies for selective HIV RNA monitoring","volume":"69","author":"Petersen,","year":"2015","journal-title":"J. Acquir. Immune Defic. Syndr. (1999)"},{"key":"2020072910570128800_B52","doi-asserted-by":"crossref","first-page":"D449","DOI":"10.1093\/nar\/gkh086","article-title":"The database of interacting proteins: 2004 update","volume":"32","author":"Salwinski,","year":"2004","journal-title":"Nucleic Acids Res."},{"key":"2020072910570128800_B53","doi-asserted-by":"crossref","first-page":"e118","DOI":"10.1371\/journal.pcbi.0030118","article-title":"Not all scale-free networks are born equal: the role of the seed graph in PPI network evolution","volume":"3","author":"Hormozdiari,","year":"2007","journal-title":"PLoS Comput. Biol."},{"key":"2020072910570128800_B54","doi-asserted-by":"crossref","first-page":"i142","DOI":"10.1093\/bioinformatics\/btr201","article-title":"Generative probabilistic models for protein\u2014protein interaction networks\u2013the biclique perspective","volume":"27","author":"Schweiger,","year":"2011","journal-title":"Bioinformatics"},{"key":"2020072910570128800_B55","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1016\/S0022-5193(03)00028-6","article-title":"Evolving protein interaction networks through gene duplication","volume":"222","author":"Pastor-Satorras,","year":"2003","journal-title":"J. Theoretical Biol."},{"key":"2020072910570128800_B56","doi-asserted-by":"crossref","first-page":"2443","DOI":"10.1093\/nar\/gkg340","article-title":"Topological structure analysis of the protein\u2013protein interaction network in budding yeast","volume":"31","author":"Bu,","year":"2003","journal-title":"Nucleic Acids Res."},{"key":"2020072910570128800_B57","doi-asserted-by":"crossref","first-page":"989","DOI":"10.1093\/bioinformatics\/btl020","article-title":"Discovering motif pairs at interaction sites from protein sequences on a proteome-wide scale","volume":"22","author":"Li,","year":"2006","journal-title":"Bioinformatics"},{"key":"2020072910570128800_B58","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman,","year":"2001","journal-title":"Mach. learn."}],"container-title":["Journal of Complex Networks"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/comnet\/article-pdf\/8\/2\/cnz024\/33543529\/cnz024.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/comnet\/article-pdf\/8\/2\/cnz024\/33543529\/cnz024.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,17]],"date-time":"2023-09-17T19:14:32Z","timestamp":1694978072000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/comnet\/article\/doi\/10.1093\/comnet\/cnz024\/5543002"}},"subtitle":[],"editor":[{"given":"Matjaz","family":"Perc","sequence":"additional","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2019,8,2]]},"references-count":58,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2020,4,1]]}},"URL":"https:\/\/doi.org\/10.1093\/comnet\/cnz024","relation":{},"ISSN":["2051-1329"],"issn-type":[{"type":"electronic","value":"2051-1329"}],"subject":[],"published-other":{"date-parts":[[2020,4]]},"published":{"date-parts":[[2019,8,2]]},"article-number":"cnz024"}}