{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T23:22:41Z","timestamp":1785540161052,"version":"3.56.0"},"update-to":[{"DOI":"10.1371\/journal.pcbi.1009355","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2021,9,29]],"date-time":"2021-09-29T00:00:00Z","timestamp":1632873600000}}],"reference-count":41,"publisher":"Public Library of Science (PLoS)","issue":"9","license":[{"start":{"date-parts":[[2021,9,17]],"date-time":"2021-09-17T00:00:00Z","timestamp":1631836800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100007601","name":"Horizon 2020","doi-asserted-by":"publisher","award":["800925"],"award-info":[{"award-number":["800925"]}],"id":[{"id":"10.13039\/501100007601","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100007601","name":"Horizon 2020","doi-asserted-by":"publisher","award":["800925"],"award-info":[{"award-number":["800925"]}],"id":[{"id":"10.13039\/501100007601","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100007601","name":"Horizon 2020","doi-asserted-by":"publisher","award":["800925"],"award-info":[{"award-number":["800925"]}],"id":[{"id":"10.13039\/501100007601","id-type":"DOI","asserted-by":"publisher"}]},{"name":"NWO","award":["016.Veni.178.023"],"award-info":[{"award-number":["016.Veni.178.023"]}]},{"DOI":"10.13039\/501100001826","name":"ZonMw","doi-asserted-by":"publisher","award":["10430022010001"],"award-info":[{"award-number":["10430022010001"]}],"id":[{"id":"10.13039\/501100001826","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001826","name":"ZonMw","doi-asserted-by":"publisher","award":["10430022010001"],"award-info":[{"award-number":["10430022010001"]}],"id":[{"id":"10.13039\/501100001826","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001826","name":"ZonMw","doi-asserted-by":"publisher","award":["10430022010001"],"award-info":[{"award-number":["10430022010001"]}],"id":[{"id":"10.13039\/501100001826","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>Many countries are currently dealing with the COVID-19 epidemic and are searching for an exit strategy such that life in society can return to normal. To support this search, computational models are used to predict the spread of the virus and to assess the efficacy of policy measures before actual implementation. The model output has to be interpreted carefully though, as computational models are subject to uncertainties. These can stem from, e.g., limited knowledge about input parameters values or from the intrinsic stochastic nature of some computational models. They lead to uncertainties in the model predictions, raising the question what distribution of values the model produces for key indicators of the severity of the epidemic. Here we show how to tackle this question using techniques for uncertainty quantification and sensitivity analysis. We assess the uncertainties and sensitivities of four exit strategies implemented in an agent-based transmission model with geographical stratification. The exit strategies are termed Flattening the Curve, Contact Tracing, Intermittent Lockdown and Phased Opening. We consider two key indicators of the ability of exit strategies to avoid catastrophic health care overload: the maximum number of prevalent cases in intensive care (IC), and the total number of IC patient-days in excess of IC bed capacity. Our results show that uncertainties not directly related to the exit strategies are secondary, although they should still be considered in comprehensive analysis intended to inform policy makers. The sensitivity analysis discloses the crucial role of the intervention uptake by the population and of the capability to trace infected individuals. Finally, we explore the existence of a safe operating space. For Intermittent Lockdown we find only a small region in the model parameter space where the key indicators of the model stay within safe bounds, whereas this region is larger for the other exit strategies.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1009355","type":"journal-article","created":{"date-parts":[[2021,9,17]],"date-time":"2021-09-17T13:36:14Z","timestamp":1631885774000},"page":"e1009355","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":18,"title":["Uncertainty quantification and sensitivity analysis of COVID-19 exit strategies in an individual-based transmission model"],"prefix":"10.1371","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1988-3555","authenticated-orcid":true,"given":"Federica","family":"Gugole","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4425-2264","authenticated-orcid":true,"given":"Luc E.","family":"Coffeng","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wouter","family":"Edeling","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9483-1988","authenticated-orcid":true,"given":"Benjamin","family":"Sanderse","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1830-5668","authenticated-orcid":true,"given":"Sake J.","family":"de Vlas","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0542-6337","authenticated-orcid":true,"given":"Daan","family":"Crommelin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2021,9,17]]},"reference":[{"issue":"6","key":"pcbi.1009355.ref001","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1371\/journal.ppat.1004206","article-title":"The contribution of social behaviour to the transmission of influenza A in a human population","volume":"10","author":"AJ Kucharski","year":"2014","journal-title":"PLOS Pathog"},{"key":"pcbi.1009355.ref002","unstructured":"Ferguson NM, Laydon D, Nedjati-Gilani G, Imai N, Ainslie K, Baguelin M, et al. Report 9\u2014Impact of non-pharmaceutical interventions (NPIs) to reduce COVID-19 mortality and healthcare demand; 2020. Available from: https:\/\/www.imperial.ac.uk\/mrc-global-infectious-disease-analysis\/covid-19\/report-9-impact-of-npis-on-covid-19\/."},{"key":"pcbi.1009355.ref003","article-title":"Covasim: an agent-based model of COVID-19 dynamics and interventions","author":"CC Kerr","year":"2021","journal-title":"medRxiv"},{"key":"pcbi.1009355.ref004","unstructured":"Modelling the spread of the novel coronavirus;. https:\/\/www.rivm.nl\/en\/novel-coronavirus-covid-19\/modelling."},{"issue":"4445","key":"pcbi.1009355.ref005","article-title":"Achieving herd immunity against COVID-19 at the country level by the exit strategy of a phased lift of control","volume":"11","author":"S De Vlas","year":"2021","journal-title":"Sci Rep"},{"key":"pcbi.1009355.ref006","unstructured":"Coffeng LE. virsim; 2020. https:\/\/gitlab.com\/luccoffeng\/virsim\/-\/tree\/v1.0.5."},{"key":"pcbi.1009355.ref007","doi-asserted-by":"crossref","unstructured":"Xiu D. Numerical Methods for Stochastic Computations: A Spectral Method Approach. Princeton University Press; 2010. Available from: http:\/\/www.jstor.org\/stable\/j.ctv7h0skv.","DOI":"10.1515\/9781400835348"},{"key":"pcbi.1009355.ref008","doi-asserted-by":"crossref","DOI":"10.1137\/1.9781611973228","volume-title":"Uncertainty Quantification\u2014Theory, Implementation and Applications","author":"RC Smith","year":"2013"},{"key":"pcbi.1009355.ref009","unstructured":"Dabladar F, Coffeng LE. Interactive Exploration of COVID-19 Exit Strategies; 2020. Available from: https:\/\/scienceversuscorona.shinyapps.io\/covid-exit\/."},{"issue":"5","key":"pcbi.1009355.ref010","doi-asserted-by":"crossref","first-page":"553","DOI":"10.1016\/S1473-3099(20)30144-4","article-title":"Early dynamics of transmission and control of COVID-19: a mathematical modelling study","volume":"20","author":"AJ Kucharski","year":"2020","journal-title":"Lancet Infect Dis"},{"issue":"19","key":"pcbi.1009355.ref011","doi-asserted-by":"crossref","first-page":"1915","DOI":"10.1001\/jama.2020.6130","article-title":"Association of public health interventions with the epidemiology of the COVID-19 outbreak in Wuhan, China","volume":"323","author":"A Pan","year":"2020","journal-title":"JAMA"},{"key":"pcbi.1009355.ref012","doi-asserted-by":"crossref","DOI":"10.1016\/S2468-2667(20)30073-6","article-title":"The effect of control strategies to reduce social mixing on outcomes of the COVID-19 epidemic in Wuhan, China: a modelling study","volume":"5","author":"K Prem","year":"2020","journal-title":"Lancet Public Health"},{"key":"pcbi.1009355.ref013","doi-asserted-by":"crossref","DOI":"10.1016\/S2468-2667(20)30090-6","article-title":"Impact assessment of non-pharmaceutical interventions against coronavirus disease 2019 and influenza in Hong Kong: an observational study","volume":"5","author":"BJ Cowling","year":"2020","journal-title":"Lancet Public Health"},{"key":"pcbi.1009355.ref014","doi-asserted-by":"crossref","first-page":"107815","DOI":"10.1016\/j.ress.2021.107815","article-title":"Global sensitivity analysis for stochastic simulators based on generalized lambda surrogate models","volume":"214","author":"X Zhu","year":"2021","journal-title":"Reliab Eng Syst Safe"},{"key":"pcbi.1009355.ref015","volume-title":"Climate Change 2013: The Physical Science Basis. Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change","author":"G Flato","year":"2013"},{"key":"pcbi.1009355.ref016","first-page":"407","article-title":"Sensitivity estimates for nonlinear mathematical models","volume":"1","author":"IM Sobol\u2019","year":"1993","journal-title":"Math Comput Model"},{"key":"pcbi.1009355.ref017","doi-asserted-by":"crossref","DOI":"10.2514\/6.2009-976","volume-title":"Comparison of Non-Intrusive Polynomial Chaos and Stochastic Collocation Methods for Uncertainty Quantification","author":"M Eldred","year":"2009"},{"key":"pcbi.1009355.ref018","doi-asserted-by":"crossref","unstructured":"Zhu X, Sudret B. Emulation of stochastic simulators using generalized lambda models. arXiv. 2021;(2007.00996).","DOI":"10.1137\/20M1337302"},{"key":"pcbi.1009355.ref019","doi-asserted-by":"crossref","first-page":"280","DOI":"10.1016\/S0010-4655(02)00280-1","article-title":"Making best use of model evaluations to compute sensitivity indices","volume":"145","author":"A Saltelli","year":"2002","journal-title":"Comput Phys Commun"},{"key":"pcbi.1009355.ref020","doi-asserted-by":"crossref","DOI":"10.5334\/jors.303","article-title":"EasyVVUQ: A Library for Verification, Validation and Uncertainty Quantification in High Performance Computing","volume":"8","author":"RA Richardson","year":"2020","journal-title":"J Open Res Softw"},{"issue":"8","key":"pcbi.1009355.ref021","doi-asserted-by":"crossref","first-page":"1900246","DOI":"10.1002\/adts.201900246","article-title":"Building Confidence in Simulation: Applications of EasyVVUQ","volume":"3","author":"DW Wright","year":"2020","journal-title":"Adv Theory Simul"},{"key":"pcbi.1009355.ref022","unstructured":"VECMA. EasyVVUQ; 2020. https:\/\/github.com\/UCL-CCS\/EasyVVUQ\/tree\/v0.8."},{"issue":"Supplement C","key":"pcbi.1009355.ref023","doi-asserted-by":"crossref","first-page":"375","DOI":"10.1016\/j.cpc.2016.05.020","article-title":"FabSim: Facilitating computational research through automation on large-scale and distributed e-infrastructures","volume":"207","author":"D Groen","year":"2016","journal-title":"Comput Phys Commun"},{"key":"pcbi.1009355.ref024","unstructured":"Groen DJ. FabSim3; 2020. https:\/\/github.com\/djgroen\/FabSim3\/tree\/v2.6."},{"key":"pcbi.1009355.ref025","unstructured":"VECMA. QCG-PilotJob; 2020. https:\/\/github.com\/vecma-project\/QCG-PilotJob\/tree\/v0.10.0."},{"key":"pcbi.1009355.ref026","doi-asserted-by":"crossref","first-page":"479","DOI":"10.1007\/978-3-030-22747-0_36","volume-title":"Computational Science\u2014ICCS 2019","author":"D Groen","year":"2019"},{"key":"pcbi.1009355.ref027","doi-asserted-by":"crossref","unstructured":"Groen D, Arabnejad H, Jancauskas V, Edeling WN, Jansson F, Richardson RA, et al. VECMAtk: A Scalable Verification, Validation and Uncertainty Quantification Toolkit for Scientific Simulations. arXiv. 2020.","DOI":"10.1098\/rsta.2020.0221"},{"key":"pcbi.1009355.ref028","unstructured":"Gugole F. UQ_covid19; 2020. https:\/\/github.com\/FGugole\/UQ_covid19."},{"issue":"3","key":"pcbi.1009355.ref029","doi-asserted-by":"crossref","first-page":"642","DOI":"10.1214\/aoms\/1177728174","article-title":"Asymptotic minimax character of the sample distribution function and of the classical multinomial estimator","volume":"27","author":"A Dvoretzky","year":"1956","journal-title":"Ann Math Stat"},{"issue":"3","key":"pcbi.1009355.ref030","doi-asserted-by":"crossref","first-page":"1269","DOI":"10.1214\/aop\/1176990746","article-title":"The tight constant in the Dvoretzky-Kiefer-Wolfowitz inequality","volume":"18","author":"P Massart","year":"1990","journal-title":"Ann Probab"},{"key":"pcbi.1009355.ref031","doi-asserted-by":"crossref","first-page":"166","DOI":"10.1016\/j.envsoft.2015.03.020","article-title":"An introduction to sensitivity assessment of simulation models","volume":"69","author":"J Norton","year":"2015","journal-title":"Environ Model Softw"},{"key":"pcbi.1009355.ref032","doi-asserted-by":"crossref","first-page":"104588","DOI":"10.1016\/j.envsoft.2019.104588","article-title":"Certain trends in uncertainty and sensitivity analysis: An overview of software tools and techniques","volume":"124","author":"D Douglas-Smith","year":"2020","journal-title":"Environ Model Softw"},{"key":"pcbi.1009355.ref033","doi-asserted-by":"crossref","first-page":"937","DOI":"10.1016\/j.cpc.2011.12.020","article-title":"Estimation of global sensitivity indices for models with dependent variables","volume":"183","author":"S Kucherenko","year":"2012","journal-title":"Comput Phys Commun"},{"issue":"86","key":"pcbi.1009355.ref034","doi-asserted-by":"crossref","first-page":"20121018","DOI":"10.1098\/rsif.2012.1018","article-title":"Sensitivity analysis of infectious disease models: methods, advances and their application","volume":"10","author":"J Wu","year":"2013","journal-title":"J R Soc Interface"},{"key":"pcbi.1009355.ref035","doi-asserted-by":"crossref","first-page":"104011","DOI":"10.1016\/j.compbiomed.2020.104011","article-title":"Uncertainty quantification in epidemiological models for the COVID-19 pandemic","volume":"125","author":"L Taghizadeh","year":"2020","journal-title":"Comput Biol Med"},{"issue":"4","key":"pcbi.1009355.ref036","first-page":"382","article-title":"Bayesian Model Averaging: A Tutorial","volume":"14","author":"JA Hoeting","year":"1999","journal-title":"Stat Sci"},{"key":"pcbi.1009355.ref037","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1016\/j.jcp.2014.06.052","article-title":"Predictive RANS simulations via Bayesian Model-Scenario Averaging","volume":"275","author":"WN Edeling","year":"2014","journal-title":"J Comput Phys"},{"key":"pcbi.1009355.ref038","doi-asserted-by":"crossref","DOI":"10.1016\/S2468-2667(20)30133-X","article-title":"Effects of non-pharmaceutical interventions on COVID-19 cases, deaths, and demand for hospital services in the UK: a modelling study","volume":"5","author":"N Davies","year":"2020","journal-title":"Lancet Public Health"},{"key":"pcbi.1009355.ref039","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1016\/j.epidem.2013.11.002","article-title":"Probabilistic uncertainty analysis of epidemiological modeling to guide public health intervention policy","volume":"6","author":"JA Gilbert","year":"2014","journal-title":"Epidemics"},{"issue":"128","key":"pcbi.1009355.ref040","article-title":"The impact of uncertainty on predictions of the CovidSim epidemiological code","volume":"1","author":"W Edeling","year":"2021","journal-title":"Nat Comput Sci"},{"key":"pcbi.1009355.ref041","doi-asserted-by":"crossref","DOI":"10.1016\/S2214-109X(20)30074-7","article-title":"Feasibility of controlling COVID-19 outbreaks by isolation of cases and contacts","volume":"8","author":"J Hellewell","year":"2020","journal-title":"Lancet Glob Health"}],"updated-by":[{"DOI":"10.1371\/journal.pcbi.1009355","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2021,9,29]],"date-time":"2021-09-29T00:00:00Z","timestamp":1632873600000}}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1009355","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,8]],"date-time":"2023-11-08T18:59:56Z","timestamp":1699469996000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1009355"}},"subtitle":[],"editor":[{"given":"Don","family":"Klinkenberg","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2021,9,17]]},"references-count":41,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2021,9,17]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1009355","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2021.03.24.21254218","asserted-by":"object"}]},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,17]]}}}