{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T16:47:30Z","timestamp":1791478050794,"version":"4.3.4"},"reference-count":38,"publisher":"Oxford University Press (OUP)","issue":"7","license":[{"start":{"date-parts":[[2025,4,28]],"date-time":"2025-04-28T00:00:00Z","timestamp":1745798400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"name":"INSERM and the Investissements d\u2019Avenir program, Vaccine Research Institute (VRI), managed by the ANR","award":["ANR-10-LABX-77-01"],"award-info":[{"award-number":["ANR-10-LABX-77-01"]}]},{"name":"French Agency for Research on AIDS and Emerging Infectious Diseases","award":["ANRS0151"],"award-info":[{"award-number":["ANRS0151"]}]},{"name":"Digital Public Health Graduate Program","award":["17-EURE-0019"],"award-info":[{"award-number":["17-EURE-0019"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,7,8]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Numerous studies assessing the effectiveness of nonpharmaceutical interventions (NPIs) against COVID-19 have produced conflicting results, partly due to methodological differences. This study aims to clarify these discrepancies by comparing 2 frequently used approaches in terms of parameter bias and CI coverage of NPI effectiveness parameters. We compared 2-step approaches, where NPI effects are regressed on by-products of the first analysis, such as the effective reproduction number ${\\mathcal{R}}_t$, with more integrated models that jointly estimate NPI effects and transmission rates in a single-step approach. We simulated datasets with mechanistic and agent-based models and analyzed them with both mechanistic models and a 2-step regression procedure. In the latter, ${\\mathcal{R}}_t$ was estimated first and then used as the outcome in a linear regression with NPI variables as predictors. Mechanistic models consistently outperformed 2-step regressions, exhibiting minimal bias (0%-5%) and accurate CI coverage. Conversely, the 2-step regression showed bias up to 25%, with significantly lower-than-nominal CI coverage, reflecting challenges in uncertainty propagation. We identified additional challenges in the 2-step regression method, such high depletion of susceptibles and time lags in observational data. Our findings suggest caution when using 2-step regression methods for estimating NPI effectiveness.<\/jats:p>","DOI":"10.1093\/aje\/kwaf035","type":"journal-article","created":{"date-parts":[[2025,4,24]],"date-time":"2025-04-24T06:46:03Z","timestamp":1745477163000},"page":"1888-1896","source":"Crossref","is-referenced-by-count":3,"title":["Comparative evaluation of methodologies for estimating the effectiveness of nonpharmaceutical interventions in the context of COVID-19: a simulation study"],"prefix":"10.1093","volume":"195","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1895-1248","authenticated-orcid":false,"given":"Iris","family":"Ganser","sequence":"first","affiliation":[{"name":"BPH Research Center, SISTM Team, Univ. Bordeaux, Inserm, UMR 1219 , Bordeaux ,","place":["France"]},{"name":"McGill Health Informatics, School of Population and Global Health, McGill University , Montreal, QC ,","place":["Canada"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5143-6256","authenticated-orcid":false,"given":"Juliette","family":"Paireau","sequence":"additional","affiliation":[{"name":"Mathematical Modelling of Infectious Diseases Unit, Institut Pasteur , Paris ,","place":["France"]},{"name":"Infectious Diseases Department, Sant\u00e9 publique France , Saint Maurice ,","place":["France"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1817-5047","authenticated-orcid":false,"given":"David L","family":"Buckeridge","sequence":"additional","affiliation":[{"name":"McGill Health Informatics, School of Population and Global Health, McGill University , Montreal, QC ,","place":["Canada"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9186-4549","authenticated-orcid":false,"given":"Simon","family":"Cauchemez","sequence":"additional","affiliation":[{"name":"Mathematical Modelling of Infectious Diseases Unit, Institut Pasteur , Paris ,","place":["France"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5235-3962","authenticated-orcid":false,"given":"Rodolphe","family":"Thi\u00e9baut","sequence":"additional","affiliation":[{"name":"BPH Research Center, SISTM Team, Univ. Bordeaux, Inserm, UMR 1219 , Bordeaux ,","place":["France"]},{"name":"Inria, Inria Bordeaux - Sud-Ouest , Talence ,","place":["France"]},{"name":"Vaccine Research Institute , Creteil ,","place":["France"]},{"name":"Medical Information Department, Bordeaux University Hospital , Bordeaux ,","place":["France"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9809-7848","authenticated-orcid":false,"given":"M\u00e9lanie","family":"Prague","sequence":"additional","affiliation":[{"name":"BPH Research Center, SISTM Team, Univ. Bordeaux, Inserm, UMR 1219 , Bordeaux ,","place":["France"]},{"name":"Inria, Inria Bordeaux - Sud-Ouest , Talence ,","place":["France"]},{"name":"Vaccine Research Institute , Creteil ,","place":["France"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2025,4,28]]},"reference":[{"issue":"11","key":"2026070807570820900_ref1","doi-asserted-by":"publisher","first-page":"e0260371","DOI":"10.1371\/journal.pone.0260371","article-title":"Effectiveness of non-pharmaceutical public health interventions against COVID-19: a systematic review and meta-analysis","volume":"16","author":"Iezadi","year":"2021","journal-title":"PloS One."},{"issue":"3","key":"2026070807570820900_ref2","doi-asserted-by":"publisher","first-page":"281","DOI":"10.1016\/j.jinf.2021.06.018","article-title":"Systematic review of empirical studies comparing the effectiveness of non-pharmaceutical interventions against COVID-19","volume":"83","author":"Mendez-Brito","year":"2021","journal-title":"J Infect."},{"key":"2026070807570820900_ref3","doi-asserted-by":"publisher","DOI":"10.1136\/bmj-2021-068302","article-title":"Effectiveness of public health measures in reducing the incidence of covid-19, SARS-CoV-2 transmission, and covid-19 mortality: systematic review and meta-analysis","volume":"375","author":"Talic","year":"2021","journal-title":"BMJ."},{"issue":"7","key":"2026070807570820900_ref4","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0236619","article-title":"Enacting national social distancing policies corresponds with dramatic reduction in COVID19 infection rates","volume":"15","author":"McGrail","year":"2020","journal-title":"PloS One."},{"issue":"10","key":"2026070807570820900_ref5","doi-asserted-by":"publisher","first-page":"1601","DOI":"10.1093\/infdis\/jiaa491","article-title":"Coronavirus disease 2019 epidemic doubling time in the United States before and during stay-at-home restrictions","volume":"222","author":"Lurie","year":"2020","journal-title":"J Infect Dis."},{"issue":"6531","key":"2026070807570820900_ref6","doi-asserted-by":"publisher","DOI":"10.1126\/science.abd9338","article-title":"Inferring the effectiveness of government interventions against COVID-19","volume":"371","author":"Brauner","year":"2021","journal-title":"Science."},{"issue":"7820","key":"2026070807570820900_ref7","doi-asserted-by":"publisher","first-page":"257","DOI":"10.1038\/s41586-020-2405-7","article-title":"Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe","volume":"584","author":"Flaxman","year":"2020","journal-title":"Nature."},{"key":"2026070807570820900_ref8","doi-asserted-by":"publisher","DOI":"10.1016\/j.epidem.2024.100744","article-title":"Estimating the population effectiveness of interventions against COVID-19 in France: a modelling study","volume":"46","author":"Ganser","year":"2024","journal-title":"Epidemics."},{"issue":"6500","key":"2026070807570820900_ref9","doi-asserted-by":"publisher","first-page":"208","DOI":"10.1126\/science.abc3517","article-title":"Estimating the burden of SARS-CoV-2 in France","volume":"369","author":"Salje","year":"2020","journal-title":"Science."},{"key":"2026070807570820900_ref10","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1016\/j.puhe.2020.04.016","article-title":"Examining the effect of social distancing on the compound growth rate of COVID-19 at the county level (United States) using statistical analyses and a random forest machine learning model","volume":"185","author":"Cobb","year":"2020","journal-title":"Public Health."},{"issue":"9","key":"2026070807570820900_ref11","doi-asserted-by":"publisher","DOI":"10.1016\/j.patter.2020.100145","article-title":"A machine learning-aided global diagnostic and comparative tool to assess effect of quarantine control in COVID-19 spread","volume":"1","author":"Dandekar","year":"2020","journal-title":"Patterns."},{"issue":"1","key":"2026070807570820900_ref12","doi-asserted-by":"publisher","first-page":"3106","DOI":"10.1038\/s41467-022-30897-1","article-title":"Untangling the changing impact of non-pharmaceutical interventions and vaccination on European COVID-19 trajectories","volume":"13","author":"Ge","year":"2022","journal-title":"Nat Commun."},{"issue":"1","key":"2026070807570820900_ref13","doi-asserted-by":"publisher","first-page":"965","DOI":"10.1186\/s12889-021-10950-2","article-title":"Differential impact of non-pharmaceutical public health interventions on COVID-19 epidemics in the United States","volume":"21","author":"Liu","year":"2021","journal-title":"BMC Public Health."},{"issue":"8","key":"2026070807570820900_ref14","doi-asserted-by":"publisher","first-page":"1504","DOI":"10.1093\/aje\/kwaa293","article-title":"Estimating the effect of social distancing interventions on COVID-19 in the United States","volume":"190","author":"Olney","year":"2021","journal-title":"Am J Epidemiol."},{"issue":"10","key":"2026070807570820900_ref15","doi-asserted-by":"publisher","first-page":"1003","DOI":"10.1007\/s10654-022-00908-y","article-title":"The methodologies to assess the effectiveness of non-pharmaceutical interventions during COVID-19: a systematic review","volume":"37","author":"Banholzer","year":"2022","journal-title":"Eur J Epidemiol."},{"issue":"1","key":"2026070807570820900_ref16","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1186\/s12916-020-01872-8","article-title":"The impact of non-pharmaceutical interventions on SARS-CoV-2 transmission across 130 countries and territories","volume":"19","author":"Liu","year":"2021","journal-title":"BMC Med."},{"issue":"12","key":"2026070807570820900_ref17","doi-asserted-by":"publisher","first-page":"1303","DOI":"10.1038\/s41562-020-01009-0","article-title":"Ranking the effectiveness of worldwide COVID-19 government interventions","volume":"4","author":"Haug","year":"2020","journal-title":"Nat Hum Behav."},{"issue":"2","key":"2026070807570820900_ref18","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1016\/S1473-3099(20)30785-4","article-title":"The temporal association of introducing and lifting nonpharmaceutical interventions with the time-varying reproduction number (R) of SARS-CoV-2: a modelling study across 131 countries","volume":"21","author":"Li","year":"2021","journal-title":"Lancet Infect Dis."},{"key":"2026070807570820900_ref19","doi-asserted-by":"publisher","first-page":"190","DOI":"10.1186\/s12879-023-08106-1","article-title":"Impact of non-pharmaceutical interventions, weather, vaccination, and variants on COVID-19 transmission across departments in France","volume":"23","author":"Paireau","year":"2023","journal-title":"BMC Infect Dis."},{"issue":"9","key":"2026070807570820900_ref20","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pone.0274590","article-title":"The impact of multi-level interventions on the second-wave SARS-CoV-2 transmission in China","volume":"17","author":"He","year":"2022","journal-title":"PloS One."},{"issue":"1","key":"2026070807570820900_ref21","doi-asserted-by":"publisher","first-page":"294","DOI":"10.1111\/biom.12564","article-title":"Dynamic models for estimating the effect of HAART on CD4 in observational studies: application to the Aquitaine cohort and the Swiss HIV cohort study","volume":"73","author":"Prague","year":"2017","journal-title":"Biometrics."},{"issue":"1","key":"2026070807570820900_ref22","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1515\/ijb-2022-0087","article-title":"Using a population-based Kalman estimator to model the COVID-19 epidemic in France: estimating associations between disease transmission and non-pharmaceutical interventions","volume":"20","author":"Collin","year":"2023","journal-title":"Int J Biostat."},{"issue":"1609","key":"2026070807570820900_ref23","doi-asserted-by":"publisher","first-page":"599","DOI":"10.1098\/rspb.2006.3754","article-title":"How generation intervals shape the relationship between growth rates and reproductive numbers","volume":"274","author":"Wallinga","year":"2007","journal-title":"Proc Biol Sci."},{"key":"2026070807570820900_ref24","author":"Lixoft","year":"2021"},{"issue":"7","key":"2026070807570820900_ref25","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1009149","article-title":"Covasim: an agent-based model of COVID-19 dynamics and interventions","volume":"17","author":"Kerr","year":"2021","journal-title":"PLoS Comput Biol."},{"issue":"4","key":"2026070807570820900_ref26","doi-asserted-by":"publisher","first-page":"1020","DOI":"10.1016\/j.csda.2004.07.002","article-title":"Maximum likelihood estimation in nonlinear mixed effects models","volume":"49","author":"Kuhn","year":"2005","journal-title":"Comput Statist Data Anal."},{"issue":"1","key":"2026070807570820900_ref27","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1007\/s10928-010-9175-z","article-title":"The use of the SAEM algorithm in MONOLIX software for estimation of population pharmacokinetic-pharmacodynamicviral dynamics parameters of maraviroc in asymptomatic HIV subjects","volume":"38","author":"Chan","year":"2011","journal-title":"J Pharmacokinet Pharmacodyn."},{"key":"2026070807570820900_ref28","doi-asserted-by":"publisher","DOI":"10.1201\/b17203","volume-title":"Mixed Effects Models for the Population Approach:Models, Tasks, Methods and Tools","author":"Lavielle","year":"2014"},{"key":"2026070807570820900_ref29","volume-title":"R: A Language and Environment for Statistical Computing","author":"Team RC","year":"2023"},{"key":"2026070807570820900_ref30","author":"Cori","year":"2021"},{"issue":"9","key":"2026070807570820900_ref31","doi-asserted-by":"publisher","first-page":"1505","DOI":"10.1093\/aje\/kwt133","article-title":"A new framework and software to estimate time-varying reproduction numbers during epidemics","volume":"178","author":"Cori","year":"2013","journal-title":"Am J Epidemiol."},{"issue":"6","key":"2026070807570820900_ref32","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pdig.0000052","article-title":"Real-time estimation of the epidemic reproduction number: scoping review of the applications and challenges","volume":"1","author":"Nash","year":"2022","journal-title":"PLoS Digit Health."},{"issue":"12","key":"2026070807570820900_ref33","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1008409","article-title":"Practical considerations for measuring the effective reproductive number, Rt","volume":"16","author":"Gostic","year":"2020","journal-title":"PLoS Comput Biol."},{"issue":"1","key":"2026070807570820900_ref34","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18637\/jss.v067.i01","article-title":"Fitting linear mixed-effects models using lme4","volume":"67","author":"Bates","year":"2015","journal-title":"J Stat Softw."},{"issue":"4","key":"2026070807570820900_ref35","doi-asserted-by":"publisher","first-page":"281","DOI":"10.1098\/rsif.2005.0042","article-title":"Perspectives on the basic reproductive ratio","volume":"2","author":"Heffernan","year":"2005","journal-title":"J R Soc Interface."},{"key":"2026070807570820900_ref36","first-page":"112","article-title":"Estimating the time-varying reproduction number of SARSCoV- 2 using national and subnational case counts","volume-title":"Wellcome Open Res","author":"Abbott","year":"2020"},{"issue":"1","key":"2026070807570820900_ref37","doi-asserted-by":"publisher","first-page":"310","DOI":"10.1186\/s12859-023-05428-4","article-title":"estimateR: an R package to estimate and monitor the effective reproductive number","volume":"24","author":"Scire","year":"2023","journal-title":"BMC Bioinformatics."},{"issue":"6","key":"2026070807570820900_ref38","doi-asserted-by":"publisher","first-page":"e408","DOI":"10.1016\/S2468-2667(21)00064-5","article-title":"Monitoring the proportion of the population infected by SARS-CoV-2 using age-stratified hospitalisation and serological data: a modelling study","volume":"6","author":"Hoz","year":"2021","journal-title":"Lancet Public Health."}],"container-title":["American Journal of Epidemiology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/aje\/advance-article-pdf\/doi\/10.1093\/aje\/kwaf035\/63030574\/kwaf035.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/aje\/article-pdf\/195\/7\/1888\/63030574\/kwaf035.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/aje\/article-pdf\/195\/7\/1888\/63030574\/kwaf035.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,8]],"date-time":"2026-07-08T11:58:37Z","timestamp":1783511917000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/aje\/article\/195\/7\/1888\/8122376"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,28]]},"references-count":38,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2025,4,28]]},"published-print":{"date-parts":[[2026,7,8]]}},"URL":"https:\/\/doi.org\/10.1093\/aje\/kwaf035","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2024.10.14.24314896","asserted-by":"object"}]},"ISSN":["0002-9262","1476-6256"],"issn-type":[{"value":"0002-9262","type":"print"},{"value":"1476-6256","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,7]]},"published":{"date-parts":[[2025,4,28]]}}}