{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,25]],"date-time":"2026-09-25T01:30:39Z","timestamp":1790299839533,"version":"4.1.0"},"reference-count":49,"publisher":"Oxford University Press (OUP)","issue":"5","license":[{"start":{"date-parts":[[2018,8,13]],"date-time":"2018-08-13T00:00:00Z","timestamp":1534118400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2018,11,1]]},"abstract":"<jats:title>Summary<\/jats:title>\n                  <jats:p>Although attractive from a theoretical perspective, finely stratified experiments such as paired designs suffer from certain analytical limitations that are not present in block-randomized experiments with multiple treated and control individuals in each block. In short, when using a weighted difference in means to estimate the sample average treatment effect, the traditional variance estimator in a paired experiment is conservative unless the pairwise average treatment effects are constant across pairs; however, in more coarsely stratified experiments, the corresponding variance estimator is unbiased if treatment effects are constant within blocks, even if they vary across blocks. Using insights from classical least squares theory, we present an improved variance estimator that is appropriate in finely stratified experiments. The variance estimator remains conservative in expectation but is asymptotically no more conservative than the classical estimator and can be considerably less conservative. The magnitude of the improvement depends on the extent to which effect heterogeneity can be explained by observed covariates. Aided by this estimator, a new test for the null hypothesis of a constant treatment effect is proposed. These findings extend to some, but not all, superpopulation models, depending on whether the covariates are viewed as fixed across samples.<\/jats:p>","DOI":"10.1111\/rssb.12290","type":"journal-article","created":{"date-parts":[[2018,8,13]],"date-time":"2018-08-13T23:17:44Z","timestamp":1534202264000},"page":"1035-1056","source":"Crossref","is-referenced-by-count":30,"title":["On Mitigating the Analytical Limitations of Finely Stratified Experiments"],"prefix":"10.1093","volume":"80","author":[{"given":"Colin B.","family":"Fogarty","sequence":"first","affiliation":[{"name":"Massachusetts Institute of Technology , Cambridge , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2018,8,13]]},"reference":[{"key":"2023022002394474700_","volume-title":"Sampling-based vs. design-based uncertainty in regression analysis","author":"Abadie","year":"2017"},{"key":"2023022002394474700_","first-page":"175","article-title":"Estimation of the conditional variance in paired experiments","volume":"91","author":"Abadie","year":"2008","journal-title":"Ann. Econ. Statist."},{"key":"2023022002394474700_","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1515\/jci-2012-0009","article-title":"A class of unbiased estimators of the average treatment effect in randomized experiments","volume":"1","author":"Aronow","year":"2013","journal-title":"J. Causl Inf."},{"key":"2023022002394474700_","volume-title":"Reading with Television: an Evaluation of The Electric Company; a Report to the Children\u2019s Television Workshop","author":"Ball","year":"1973"},{"key":"2023022002394474700_","doi-asserted-by":"crossref","first-page":"1012","DOI":"10.1080\/01621459.1994.10476836","article-title":"P values maximized over a confidence set for the nuisance parameter","volume":"89","author":"Berger","year":"1994","journal-title":"J. Am. Statist. Ass."},{"key":"2023022002394474700_","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1177\/0193841X13513025","article-title":"Covariance adjustments for the analysis of randomized field experiments","volume":"37","author":"Berk","year":"2013","journal-title":"Evaln Rev."},{"key":"2023022002394474700_","doi-asserted-by":"crossref","first-page":"7383","DOI":"10.1073\/pnas.1510506113","article-title":"Lasso adjustments of treatment effect estimates in randomized experiments","volume":"113","author":"Bloniarz","year":"2016","journal-title":"Proc. Natn. Acad. Sci. USA"},{"key":"2023022002394474700_","doi-asserted-by":"crossref","first-page":"822","DOI":"10.1080\/01621459.2015.1058264","article-title":"Nonparametric variance estimation under fine stratification: an alternative to collapsed strata","volume":"111","author":"Breidt","year":"2016","journal-title":"J. Am. Statist. Ass."},{"key":"2023022002394474700_","volume-title":"Models as approximations, part i: a conspiracy of nonlinearity and random regressors in linear regression","author":"Buja","year":"2016"},{"key":"2023022002394474700_","volume-title":"Experimental Designs","author":"Cochran","year":"1957"},{"key":"2023022002394474700_","volume-title":"Planning of Experiments","author":"Cox","year":"1958"},{"key":"2023022002394474700_","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1214\/07-AOAS113","article-title":"Applied statistics: a review","volume":"1","author":"Cox","year":"2007","journal-title":"Ann. Appl. Statist."},{"key":"2023022002394474700_","first-page":"331","article-title":"A paradox from randomization-based causal inference","volume":"32","author":"Ding","year":"2017","journal-title":"Statist. Sci."},{"key":"2023022002394474700_","doi-asserted-by":"crossref","first-page":"655","DOI":"10.1111\/rssb.12124","article-title":"Randomization inference for treatment effect variation","volume":"78","author":"Ding","year":"2016","journal-title":"J. R. Statist. Soc."},{"key":"2023022002394474700_","volume-title":"The Design of Experiments","author":"Fisher","year":"1935"},{"key":"2023022002394474700_","doi-asserted-by":"crossref","DOI":"10.1093\/biomet\/asy034","article-title":"Regression-assisted inference for the average treatment effect in paired experiments.","volume-title":"Biometrika","author":"Fogarty","year":"2018"},{"key":"2023022002394474700_","doi-asserted-by":"crossref","first-page":"180","DOI":"10.1016\/j.aam.2006.12.003","article-title":"On regression adjustments to experimental data","volume":"40","author":"Freedman","year":"2008","journal-title":"Adv. Appl. Math."},{"key":"2023022002394474700_","first-page":"1","article-title":"Multivariate adaptive regression splines","volume":"19","author":"Friedman","year":"1991","journal-title":"Ann. Statist."},{"key":"2023022002394474700_","doi-asserted-by":"crossref","first-page":"310","DOI":"10.1198\/000313001753272268","article-title":"Randomization inference and bias of standard errors","volume":"55","author":"Gadbury","year":"2001","journal-title":"Am. Statistn"},{"key":"2023022002394474700_","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1093\/biostatistics\/5.2.263","article-title":"Optimal multivariate matching before randomization","volume":"5","author":"Greevy","year":"2004","journal-title":"Biostatistics"},{"key":"2023022002394474700_","doi-asserted-by":"crossref","first-page":"609","DOI":"10.1198\/016214504000000647","article-title":"Full matching in an observational study of coaching for the SAT","volume":"99","author":"Hansen","year":"2004","journal-title":"J. Am. Statist. Ass."},{"key":"2023022002394474700_","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1214\/08-STS254","article-title":"Covariate balance in simple, stratified and clustered comparative studies","volume":"23","author":"Hansen","year":"2008","journal-title":"Statist. Sci."},{"key":"2023022002394474700_","doi-asserted-by":"crossref","first-page":"7369","DOI":"10.1073\/pnas.1510504113","article-title":"Improving massive experiments with threshold blocking","volume":"113","author":"Higgins","year":"2016","journal-title":"Proc. Natn. Acad. Sci. USA"},{"key":"2023022002394474700_","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1080\/00031305.1978.10479237","article-title":"The hat matrix in regression and ANOVA","volume":"32","author":"Hoaglin","year":"1978","journal-title":"Am. Statistn"},{"key":"2023022002394474700_","doi-asserted-by":"crossref","first-page":"4857","DOI":"10.1002\/sim.3337","article-title":"Variance identification and efficiency analysis in randomized experiments under the matched-pair design","volume":"27","author":"Imai","year":"2008","journal-title":"Statist. Med."},{"key":"2023022002394474700_","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1214\/08-STS274","article-title":"The essential role of pair matching in cluster-randomized experiments, with application to the Mexican universal health insurance evaluation","volume":"24","author":"Imai","year":"2009","journal-title":"Statist. Sci."},{"key":"2023022002394474700_","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1162\/003465304323023651","article-title":"Nonparametric estimation of average treatment effects under exogeneity: a review","volume":"86","author":"Imbens","year":"2004","journal-title":"Rev. Econ. Statist."},{"key":"2023022002394474700_","volume-title":"Conf. International Initiative for Impact Evaluation, Cuernavaca","author":"Imbens","year":"2011"},{"key":"2023022002394474700_","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9781139025751","volume-title":"Causal Inference in Statistics, Social, and Biomedical Sciences","author":"Imbens","year":"2015"},{"key":"2023022002394474700_","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1111\/rssb.12240","article-title":"Optimal a priori balance in the design of controlled experiments","volume":"80","author":"Kallus","year":"2018","journal-title":"J. R. Statist. Soc."},{"key":"2023022002394474700_","doi-asserted-by":"crossref","first-page":"118","DOI":"10.2307\/2985676","article-title":"Standardization: a technique to control for extraneous variables","volume":"17","author":"Kalton","year":"1968","journal-title":"Appl. Statist."},{"key":"2023022002394474700_","doi-asserted-by":"crossref","first-page":"1753","DOI":"10.1002\/(SICI)1097-0258(19970815)16:15<1753::AID-SIM597>3.0.CO;2-E","article-title":"The merits of matching in community intervention trials: a cautionary tale","volume":"16","author":"Klar","year":"1997","journal-title":"Statist. Med."},{"key":"2023022002394474700_","first-page":"113","article-title":"Adaptive matching in randomized trials and observational studies","volume":"46","author":"van der Laan","year":"2012","journal-title":"J. Statist. Res."},{"key":"2023022002394474700_","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1214\/12-AOAS583","article-title":"Agnostic notes on regression adjustments to experimental data: reexamining Freedman\u2019s critique","volume":"7","author":"Lin","year":"2013","journal-title":"Ann. Appl. Statist."},{"key":"2023022002394474700_","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1080\/00031305.2000.10474549","article-title":"Using heteroscedasticity consistent standard errors in the linear regression model","volume":"54","author":"Long","year":"2000","journal-title":"Am. Statistn"},{"key":"2023022002394474700_","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.spl.2016.07.010","article-title":"Covariate adjustment in randomization-based causal inference for 2k factorial designs","volume":"119","author":"Lu","year":"2016","journal-title":"Statist. Probab. Lett."},{"key":"2023022002394474700_","doi-asserted-by":"crossref","first-page":"305","DOI":"10.1016\/0304-4076(85)90158-7","article-title":"Some heteroskedasticity-consistent covariance matrix estimators with improved finite sample properties","volume":"29","author":"MacKinnon","year":"1985","journal-title":"J. Econmetr."},{"key":"2023022002394474700_","doi-asserted-by":"crossref","first-page":"118","DOI":"10.1111\/j.0006-341X.2000.00118.x","article-title":"Substantial gains in bias reduction from matching with a variable number of controls","volume":"56","author":"Ming","year":"2000","journal-title":"Biometrics"},{"key":"2023022002394474700_","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1111\/j.1467-9868.2012.01048.x","article-title":"Adjusting treatment effect estimates by post-stratification in randomized experiments","volume":"75","author":"Miratrix","year":"2013","journal-title":"J. R. Statist. Soc."},{"key":"2023022002394474700_","first-page":"463","article-title":"On the application of probability theory to agricultural experiments: Essay on principles (section 9) (Engl. transl.)","volume":"5","author":"Neyman","year":"1990","journal-title":"Statist. Sci."},{"key":"2023022002394474700_","volume-title":"Insights on variance estimation for blocked and matched pairs designs","author":"Pashley","year":"2017"},{"key":"2023022002394474700_","volume-title":"Improved precision in estimating average treatment effects","author":"Pitkin","year":"2013"},{"key":"2023022002394474700_","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4757-3692-2","volume-title":"Observational Studies","author":"Rosenbaum","year":"2002"},{"key":"2023022002394474700_","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4419-1213-8","volume-title":"Design of Observational Studies","author":"Rosenbaum","year":"2010"},{"key":"2023022002394474700_","doi-asserted-by":"crossref","first-page":"688","DOI":"10.1037\/h0037350","article-title":"Estimating causal effects of treatments in randomized and nonrandomized studies","volume":"66","author":"Rubin","year":"1974","journal-title":"J. Educ. Psychol."},{"key":"2023022002394474700_","volume-title":"The performance and efficiency of threshold blocking","author":"S\u00e4vje","year":"2015"},{"key":"2023022002394474700_","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1214\/09-STS313","article-title":"Matching methods for causal inference: a review and a look forward","volume":"25","author":"Stuart","year":"2010","journal-title":"Statist. Sci."},{"key":"2023022002394474700_","doi-asserted-by":"crossref","first-page":"395","DOI":"10.1037\/0012-1649.44.2.395","article-title":"Using full matching to estimate causal effects in nonexperimental studies: examining the relationship between adolescent marijuana use and adult outcomes","volume":"44","author":"Stuart","year":"2008","journal-title":"Devlpmntl Psychol."},{"key":"2023022002394474700_","volume-title":"Introduction to Variance Estimation","author":"Wolter","year":"2007"}],"container-title":["Journal of the Royal Statistical Society Series B: Statistical Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1111%2Frssb.12290","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/80\/5\/1035\/49269533\/jrsssb_80_5_1035.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/80\/5\/1035\/49269533\/jrsssb_80_5_1035.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,9]],"date-time":"2024-07-09T02:01:10Z","timestamp":1720490470000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jrsssb\/article\/80\/5\/1035\/7048348"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,8,13]]},"references-count":49,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2018,8,13]]},"published-print":{"date-parts":[[2018,11,1]]}},"URL":"https:\/\/doi.org\/10.1111\/rssb.12290","relation":{},"ISSN":["1369-7412","1467-9868"],"issn-type":[{"value":"1369-7412","type":"print"},{"value":"1467-9868","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2018,11]]},"published":{"date-parts":[[2018,8,13]]}}}