{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T04:40:43Z","timestamp":1777351243860,"version":"3.51.4"},"reference-count":22,"publisher":"MIT Press","issue":"4","content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2025,7,17]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>I introduce a simple permutation procedure to test conventional (nonsharp) hypotheses about the effect of a binary treatment in the presence of a finite number of large, heterogeneous clusters when the treatment effect is identified by comparisons across clusters. The procedure asymptotically controls size by applying a level-adjusted permutation test to a suitable statistic. The adjusted permutation test is easy to implement in practice and performs well at conventional levels of significance with at least four treated clusters and a similar number of control clusters. It is particularly robust to situations where some clusters are much more variable than others.<\/jats:p>","DOI":"10.1162\/rest_a_01300","type":"journal-article","created":{"date-parts":[[2023,3,29]],"date-time":"2023-03-29T18:56:50Z","timestamp":1680116210000},"page":"1115-1122","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":7,"title":["Permutation Inference with a Finite Number of Heterogeneous Clusters"],"prefix":"10.1162","volume":"107","author":[{"given":"Andreas","family":"Hagemann","sequence":"first","affiliation":[{"name":"University of Michigan umich.edu\/~hagem"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","published-online":{"date-parts":[[2025,7,17]]},"reference":[{"issue":"4","key":"2025072212203212500_bib1","doi-asserted-by":"publisher","first-page":"301","DOI":"10.1257\/aer.99.4.1384","article-title":"The Effects of High Stakes High School Achievement Awards: Evidence from a Randomized Trial","volume":"99","author":"Angrist","year":"2009","journal-title":"American Economic Review"},{"issue":"1","key":"2025072212203212500_bib2","doi-asserted-by":"publisher","first-page":"249","DOI":"10.1162\/003355304772839588","article-title":"How Much Should We Trust Differences-in-Differences Estimates?","volume":"119","author":"Bertrand","year":"2004","journal-title":"Quarterly Journal of Economics"},{"issue":"2","key":"2025072212203212500_bib3","doi-asserted-by":"publisher","first-page":"137","DOI":"10.1016\/j.jeconom.2011.01.007","article-title":"Inference with Dependent Data Using Cluster Covariance Estimators","volume":"165","author":"Bester","year":"2011","journal-title":"Journal of Econometrics"},{"issue":"3","key":"2025072212203212500_bib4","first-page":"414","article-title":"Bootstrap-Based Improvements for Inference with Clustered Errors","volume":"90","author":"Cameron","year":"2008"},{"issue":"3","key":"2025072212203212500_bib5","doi-asserted-by":"publisher","first-page":"1013","DOI":"10.3982\/ECTA13081","article-title":"Randomization Tests under an Approximate Symmetry Assumption","volume":"85","author":"Canay","year":"2017","journal-title":"Econometrica"},{"issue":"2","key":"2025072212203212500_bib6","first-page":"346","article-title":"The Wild Bootstrap with a \u2018Small\u2019 Number of \u2018Large\u2019 Clusters","volume":"103","author":"Canay","year":"2021"},{"issue":"1","key":"2025072212203212500_bib7","first-page":"113","article-title":"Inference with \u2018Difference in Differences\u2019 with a Small Number of Policy Changes","volume":"93","author":"Conley","year":"2011"},{"issue":"9","key":"2025072212203212500_bib8","doi-asserted-by":"publisher","first-page":"2964","DOI":"10.1257\/aer.20181169","article-title":"Two-Way Fixed Effects Estimators with Heterogeneous Treatment Effects","volume":"110","author":"de Chaisemartin","year":"2020","journal-title":"American Economic Review"},{"issue":"2","key":"2025072212203212500_bib9","doi-asserted-by":"publisher","first-page":"393","DOI":"10.1016\/j.jeconom.2019.04.035","article-title":"Asymptotic Theory and Wild Bootstrap Inference with Clustered Errors","volume":"212","author":"Djogbenou","year":"2019","journal-title":"Journal of Econometrics"},{"issue":"2","key":"2025072212203212500_bib10","first-page":"221","article-title":"Inference with Difference-in-Differences and Other Panel Data","volume":"89","author":"Donald","year":"2007"},{"issue":"1","key":"2025072212203212500_bib11","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.spa.2012.08.014","article-title":"A Central Limit Theorem for Stationary Random Fields","volume":"123","author":"El Machkouri","year":"2013","journal-title":"Stochastic Processes and Their Applications"},{"issue":"3","key":"2025072212203212500_bib12","doi-asserted-by":"publisher","first-page":"607","DOI":"10.1086\/260061","article-title":"Risk, Return, and Equilibrium: Empirical Tests","volume":"81","author":"Fama","year":"1973","journal-title":"Journal of Political Economy"},{"issue":"4","key":"2025072212203212500_bib13","doi-asserted-by":"publisher","first-page":"375","DOI":"10.1080\/00949659708811867","article-title":"Normal Scale Mixtures and Dual Probability Densities","volume":"59","author":"Gneiting","year":"1997","journal-title":"Journal of Statistical Computation and Simulation"},{"issue":"1","key":"2025072212203212500_bib14","doi-asserted-by":"publisher","first-page":"190","DOI":"10.1016\/j.jeconom.2019.04.011","article-title":"Placebo Inference on Treatment Effects When the Number of Clusters Is Small","volume":"213","author":"Hagemann","year":"2019","journal-title":"Journal of Econometrics"},{"issue":"2","key":"2025072212203212500_bib15","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1214\/aoms\/1177729436","article-title":"The Large-Sample Power of Tests Based on Permutations of Observations","volume":"23","author":"Hoeffding","year":"1952","journal-title":"Annals of Mathematical Statistics"},{"issue":"4","key":"2025072212203212500_bib16","doi-asserted-by":"crossref","first-page":"453","DOI":"10.1198\/jbes.2009.08046","article-title":"t-Statistic Based Correlation and Heterogeneity Robust Inference","volume":"28","author":"Ibragimov","year":"2010","journal-title":"Journal of Business & Economic Statistics"},{"issue":"1","key":"2025072212203212500_bib17","first-page":"83","article-title":"Inference with Few Heterogenous Clusters","volume":"98","author":"Ibragimov","year":"2016"},{"issue":"1","key":"2025072212203212500_bib18","doi-asserted-by":"publisher","first-page":"86","DOI":"10.1016\/j.jeconom.2009.02.009","article-title":"Central Limit Theorems and Uniform Laws of Large Numbers for Arrays of Random Fields","volume":"150","author":"Jenish","year":"2009","journal-title":"Journal of Econometrics"},{"issue":"2","key":"2025072212203212500_bib19","doi-asserted-by":"publisher","first-page":"233","DOI":"10.1002\/jae.2508","article-title":"Wild Bootstrap Inference for Wildly Different Cluster Sizes","volume":"32","author":"MacKinnon","year":"2017","journal-title":"Journal of Applied Econometrics"},{"key":"2025072212203212500_bib20","article-title":"What\u2019s Trending in Difference-in-Differences? A Synthesis of the Recent Econometrics Literature","author":"Roth","year":"2022"},{"key":"2025072212203212500_bib21","first-page":"9","article-title":"Student\u2019s t-Test for Scale Mixture Errors","volume-title":"IMS Lecture Notes\u2013Monograph Series","author":"Sz\u00e9kely","year":"2006"},{"key":"2025072212203212500_bib22","volume-title":"Asymptotic Theory for Econometricians","author":"White","year":"2001","edition":"rev."}],"container-title":["Review of Economics and Statistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/rest\/article-pdf\/107\/4\/1115\/2074995\/rest_a_01300.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/rest\/article-pdf\/107\/4\/1115\/2074995\/rest_a_01300.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,22]],"date-time":"2025-07-22T16:20:37Z","timestamp":1753201237000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/rest\/article\/107\/4\/1115\/115263\/Permutation-Inference-with-a-Finite-Number-of"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7]]},"references-count":22,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2025,7,17]]},"published-print":{"date-parts":[[2025,7,17]]}},"URL":"https:\/\/doi.org\/10.1162\/rest_a_01300","relation":{},"ISSN":["0034-6535","1530-9142"],"issn-type":[{"value":"0034-6535","type":"print"},{"value":"1530-9142","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,7]]},"published":{"date-parts":[[2025,7]]}}}