{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T16:33:16Z","timestamp":1791477196318,"version":"4.3.4"},"reference-count":34,"publisher":"Oxford University Press (OUP)","issue":"5","license":[{"start":{"date-parts":[[2023,4,6]],"date-time":"2023-04-06T00:00:00Z","timestamp":1680739200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"DOI":"10.13039\/100006545","name":"National Institute on Minority Health and Health Disparities","doi-asserted-by":"publisher","award":["1K01MD014163-01A1"],"award-info":[{"award-number":["1K01MD014163-01A1"]}],"id":[{"id":"10.13039\/100006545","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,10,5]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>As the interest in and use of quasi-experimental methods to evaluate impacts of health policies have dramatically increased in the epidemiological literature, we set out this study to (i) systematically compare several quasi-experimental methods that use data before and after an intervention and contrast their performance within a simulation framework while providing a brief overview of the methods; and (ii) discuss challenges that could arise from using these methods as well as directions for future research in the context of epidemiological applications.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We considered single-group designs [pre-post and interrupted time series (ITS)] and multiple-group designs [controlled interrupted time series\/difference-in-differences, synthetic control methods (SCMs): traditional SCMs and generalized SCMs]. We assessed performance based on bias and root mean squared error.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>We identified settings in which each method failed to provide unbiased estimates. We found that, among the methods investigated, when data for multiple time points and for multiple control groups are available (multiple-group designs), data-adaptive methods such as the generalized SCM were generally less biased than other methods evaluated in our study. In addition, when all of the included units have been exposed to treatment (single-group designs) and data for a sufficiently long pre-intervention period are available, then the ITS performs very well, provided the underlying model is correctly specified.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>When using a quasi-experimental method using data before and after an intervention, epidemiologists should strive to use, whenever feasible, data-adaptive methods that nest alternative identifying assumptions including relaxing the parallel trend assumption (e.g. generalized SCMs).<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/ije\/dyad032","type":"journal-article","created":{"date-parts":[[2023,3,7]],"date-time":"2023-03-07T18:53:45Z","timestamp":1678215225000},"page":"1522-1533","source":"Crossref","is-referenced-by-count":52,"title":["A comparison of quasi-experimental methods with data before and after an intervention: an introduction for epidemiologists and a simulation study"],"prefix":"10.1093","volume":"52","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5932-6169","authenticated-orcid":false,"given":"Roch A","family":"Nianogo","sequence":"first","affiliation":[{"name":"Department of Epidemiology, Fielding School of Public Health, University of California, Los Angeles (UCLA) , Los Angeles, CA, USA"},{"name":"California Center for Population Research, UCLA , Los Angeles, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tarik","family":"Benmarhnia","sequence":"additional","affiliation":[{"name":"Scripps Institution of Oceanography, UC San Diego , La Jolla, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stephen","family":"O\u2019Neill","sequence":"additional","affiliation":[{"name":"Department of Health Services Research and Policy, London School of Hygiene and Tropical Medicine , London, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2023,4,6]]},"reference":[{"key":"2023100820065305700_dyad032-B1","doi-asserted-by":"crossref","first-page":"351","DOI":"10.1146\/annurev-publhealth-031816-044208","article-title":"Evaluating the health impact of large-scale public policy changes: classical and novel approaches","volume":"38","author":"Basu","year":"2017","journal-title":"Annu Rev Public Health"},{"key":"2023100820065305700_dyad032-B2","doi-asserted-by":"crossref","first-page":"688","DOI":"10.1037\/h0037350","article-title":"Estimating causal effects of treatments in randomized and nonrandomized studies","volume":"66","author":"Rubin","year":"1974","journal-title":"J Educ Psychol"},{"key":"2023100820065305700_dyad032-B3","doi-asserted-by":"crossref","first-page":"945","DOI":"10.1080\/01621459.1986.10478354","article-title":"Statistics and causal inference","volume":"81","author":"Holland","year":"1986","journal-title":"J Am Stat Assoc"},{"key":"2023100820065305700_dyad032-B4","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1016\/j.socscimed.2017.12.005","article-title":"Understanding and misunderstanding randomized controlled trials","volume":"210","author":"Deaton","year":"2018","journal-title":"Soc Sci Med"},{"key":"2023100820065305700_dyad032-B5","doi-asserted-by":"crossref","first-page":"2720","DOI":"10.1001\/jama.291.22.2720","article-title":"Participation in cancer clinical trials: race-, sex-, and age-based disparities","volume":"291","author":"Murthy","year":"2004","journal-title":"JAMA"},{"key":"2023100820065305700_dyad032-B6","first-page":"348","article-title":"Interrupted time series regression for the evaluation of public health interventions: a tutorial","volume":"46","author":"Bernal","year":"2017","journal-title":"Int J Epidemiol"},{"key":"2023100820065305700_dyad032-B7","doi-asserted-by":"crossref","first-page":"950","DOI":"10.1016\/j.jclinepi.2014.12.018","article-title":"Interrupted time series analysis in drug utilization research is increasing: systematic review and recommendations. In","volume":"68","author":"Jandoc","year":"2015","journal-title":"J Clin Epidemiol"},{"key":"2023100820065305700_dyad032-B8","doi-asserted-by":"crossref","first-page":"480","DOI":"10.1177\/1536867X1501500208","article-title":"Conducting interrupted time-series analysis for single- and multiple-group comparisons","volume":"15","author":"Linden","year":"2015","journal-title":"Stata J"},{"key":"2023100820065305700_dyad032-B9","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1007\/s40471-020-00245-2","article-title":"Difference-in-difference in the time of cholera: a gentle introduction for epidemiologists","volume":"7","author":"Caniglia","year":"2020","journal-title":"Curr Epidemiol Rep"},{"key":"2023100820065305700_dyad032-B10","doi-asserted-by":"crossref","first-page":"673","DOI":"10.1136\/jech-2017-210106","article-title":"Synthetic control methodology as a tool for evaluating population-level health interventions","volume":"0","author":"Bouttell","year":"2018","journal-title":"J Epidemiol Commun Health"},{"key":"2023100820065305700_dyad032-B11","doi-asserted-by":"crossref","first-page":"503","DOI":"10.1097\/EDE.0000000000000837","article-title":"A new tool for case studies in epidemiology: the synthetic control method","volume":"29","author":"Rehkopf","year":"2018","journal-title":"Epidemiology"},{"key":"2023100820065305700_dyad032-B12","doi-asserted-by":"crossref","first-page":"2700","DOI":"10.1093\/aje\/kwab211","article-title":"Synthetic control methods for the evaluation of single-unit interventions in epidemiology: a tutorial","author":"Bonander","year":"2021","journal-title":"Am J Epidemiol"},{"key":"2023100820065305700_dyad032-B13","doi-asserted-by":"crossref","first-page":"2010","DOI":"10.1093\/ije\/dyaa152","article-title":"Can synthetic controls improve causal inference in interrupted time series evaluations of public health interventions?","volume":"49","author":"Degli Esposti","year":"2021","journal-title":"Int J Epidemiol"},{"key":"2023100820065305700_dyad032-B14","doi-asserted-by":"publisher","first-page":"328","DOI":"10.1111\/1475-6773.13274","article-title":"A comparison of methods for health policy evaluation with controlled pre-post designs","volume":"55","author":"O\u2019Neill","year":"2020","journal-title":"Health Serv Res"},{"key":"2023100820065305700_dyad032-B15","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s10742-016-0146-8","article-title":"Estimating causal effects: considering three alternatives to difference-in-differences estimation","volume":"16","author":"O'Neill","year":"2016","journal-title":"Health Serv Outcomes Res Methodol"},{"key":"2023100820065305700_dyad032-B16","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1146\/annurev-publhealth-040617-014128","article-title":"Selecting and improving quasi-experimental designs in effectiveness and implementation research","volume":"39","author":"Handley","year":"2018","journal-title":"Annu Rev Public Health"},{"key":"2023100820065305700_dyad032-B17","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1146\/annurev-publhealth-031816-044327","article-title":"Natural experiments: an overview of methods, approaches, and contributions to public health intervention research","volume":"38","author":"Craig","year":"2017","journal-title":"Annu Rev Public Health"},{"key":"2023100820065305700_dyad032-B18","doi-asserted-by":"crossref","first-page":"493","DOI":"10.1198\/jasa.2009.ap08746","article-title":"Synthetic control methods for comparative case studies: estimating the effect of California\u2019s tobacco control program","volume":"105","author":"Abadie","year":"2010","journal-title":"J Am Stat Assoc"},{"key":"2023100820065305700_dyad032-B19","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1017\/pan.2016.2","article-title":"Generalized synthetic control method: causal inference with interactive fixed effects models","volume":"25","author":"Xu","year":"2017","journal-title":"Polit Anal"},{"key":"2023100820065305700_dyad032-B20","first-page":"305","article-title":"Pretest with caution: Event-study estimates after testing for parallel trends","volume-title":"AER: Insights","author":"Roth","year":"2022"},{"key":"2023100820065305700_dyad032-B21","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1093\/epirev\/mxab005","article-title":"What to do when everything happens at once: analytic approaches to estimate the health effects of co-occurring social policies","volume":"43","author":"Matthay","year":"2021","journal-title":"Epidemiol Rev"},{"key":"2023100820065305700_dyad032-B22","author":"Roth"},{"key":"2023100820065305700_dyad032-B23","first-page":"200","article-title":"Difference-in-differences with multiple time periods","volume":"225","author":"Callaway","year":"2020","journal-title":"J Econ"},{"key":"2023100820065305700_dyad032-B24","author":"R Core Team","year":"2022"},{"key":"2023100820065305700_dyad032-B25","doi-asserted-by":"publisher","first-page":"1197","DOI":"10.3982\/qe1596","article-title":"Synthetic controls with imperfect pretreatment fit","volume":"12","author":"Ferman","year":"2021","journal-title":"Quant Econ"},{"key":"2023100820065305700_dyad032-B26","doi-asserted-by":"publisher","first-page":"788","DOI":"10.1097\/EDE.0000000000001539","article-title":"Using the generalized synthetic control method to estimate the impact of extreme weather events on population health","volume":"33","author":"Sheridan","year":"2022","journal-title":"Epidemiology"},{"key":"2023100820065305700_dyad032-B27","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1214\/09-STS313T4","article-title":"Matching methods for causal inference","volume":"25","author":"Stuart","year":"2010","journal-title":"Stat Sci"},{"key":"2023100820065305700_dyad032-B28","doi-asserted-by":"crossref","first-page":"166","DOI":"10.1007\/s10742-014-0123-z","article-title":"Using propensity scores in difference-in-differences models to estimate the effects of a policy change","volume":"14","author":"Stuart","year":"2014","journal-title":"Health Serv Outcomes Res Methodol"},{"key":"2023100820065305700_dyad032-B29","doi-asserted-by":"crossref","first-page":"1789","DOI":"10.1080\/01621459.2021.1929245","article-title":"The augmented synthetic control method","volume":"116","author":"Ben-Michael","year":"2021","journal-title":"J Am Stat Assoc"},{"key":"2023100820065305700_dyad032-B30","doi-asserted-by":"crossref","first-page":"1716","DOI":"10.1080\/01621459.2021.1891924","article-title":"Matrix completion methods for causal panel data models","volume":"116","author":"Athey","year":"2021","journal-title":"J Am Stat Assoc"},{"key":"2023100820065305700_dyad032-B31","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1017\/pan.2021.22","article-title":"A Bayesian alternative to synthetic control for comparative case studies","volume":"30","author":"Pang","year":"2022","journal-title":"Polit Anal"},{"key":"2023100820065305700_dyad032-B32","doi-asserted-by":"crossref","first-page":"254","DOI":"10.1016\/j.jeconom.2021.03.014","article-title":"Difference-in-differences with variation in treatment timing","volume":"225","author":"Goodman-Bacon","year":"2021","journal-title":"J Econ"},{"key":"2023100820065305700_dyad032-B33","author":"Borusyak"},{"key":"2023100820065305700_dyad032-B34","doi-asserted-by":"publisher","first-page":"2964","DOI":"10.1257\/aer.20181169","article-title":"Two-way fixed effects estimators with heterogeneous treatment effects","volume":"110","author":"de Chaisemartin","year":"2020","journal-title":"Am Econ Rev"}],"container-title":["International Journal of Epidemiology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/ije\/article-pdf\/52\/5\/1522\/51950800\/dyad032.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/ije\/article-pdf\/52\/5\/1522\/51950800\/dyad032.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,9]],"date-time":"2023-10-09T04:27:07Z","timestamp":1696825627000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/ije\/article\/52\/5\/1522\/7110226"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,6]]},"references-count":34,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2023,4,6]]},"published-print":{"date-parts":[[2023,10,5]]}},"URL":"https:\/\/doi.org\/10.1093\/ije\/dyad032","relation":{},"ISSN":["0300-5771","1464-3685"],"issn-type":[{"value":"0300-5771","type":"print"},{"value":"1464-3685","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,10,1]]},"published":{"date-parts":[[2023,4,6]]}}}