{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,4]],"date-time":"2026-09-04T22:42:27Z","timestamp":1788561747277,"version":"build-2803163510"},"reference-count":46,"publisher":"Oxford University Press (OUP)","issue":"2","license":[{"start":{"date-parts":[[2021,5,1]],"date-time":"2021-05-01T00:00:00Z","timestamp":1619827200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,6,29]]},"abstract":"<jats:title>Summary<\/jats:title>\n               <jats:p>This paper presents a simple decision-theoretic economic approach for analysing social experiments with compromised random assignment protocols that are only partially documented. We model administratively constrained experimenters who satisfice in seeking covariate balance. We develop design-based small-sample hypothesis tests that use worst-case (least favourable) randomization null distributions. Our approach accommodates a variety of compromised experiments, including imperfectly documented rerandomization designs. To make our analysis concrete, we focus much of our discussion on the influential Perry Preschool Project. We reexamine previous estimates of programme effectiveness using our methods. The choice of how to model reassignment vitally affects inference.<\/jats:p>","DOI":"10.1093\/ectj\/utab009","type":"journal-article","created":{"date-parts":[[2021,4,1]],"date-time":"2021-04-01T13:15:50Z","timestamp":1617282950000},"page":"C1-C39","source":"Crossref","is-referenced-by-count":9,"title":["Using a satisficing model of experimenter decision-making to guide finite-sample inference for compromised experiments"],"prefix":"10.1093","volume":"24","author":[{"given":"James J","family":"Heckman","sequence":"first","affiliation":[{"name":"Center for the Economics of Human Development, The University of Chicago , Chicago, IL 60637, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ganesh","family":"Karapakula","sequence":"additional","affiliation":[{"name":"Yale University , New Haven, CT 06511, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2021,6,29]]},"reference":[{"issue":"1)","key":"2022110207134246900_bib1","doi-asserted-by":"crossref","first-page":"265","DOI":"10.3982\/ECTA12675","article-title":"Sampling-based versus design-based uncertainty in regression analysis","volume":"88","author":"Abadie","year":"2020","journal-title":"Econometrica"},{"key":"2022110207134246900_bib2","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1016\/bs.hefe.2016.10.003","article-title":"The econometrics of randomized experiments","author":"Athey","year":"2017","journal-title":"Handbook of Economic Field Experiments Volume 1"},{"issue":"4)","key":"2022110207134246900_bib3","first-page":"200","article-title":"In pursuit of balance: randomization in practice in development field experiments","volume":"1","author":"Bruhn","year":"2009","journal-title":"American Economic Journal: Applied Economics"},{"issue":"1)","key":"2022110207134246900_bib4","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1207\/S1532480XADS0601_05","article-title":"Early childhood education: young adult outcomes from the Abecedarian Project","volume":"6","author":"Campbell","year":"2002","journal-title":"Applied Developmental Science"},{"issue":"2)","key":"2022110207134246900_bib5","doi-asserted-by":"crossref","first-page":"138","DOI":"10.1016\/j.jeconom.2009.09.023","article-title":"Efficient semiparametric estimation of multi-valued treatment effects under ignorability","volume":"155","author":"Cattaneo","year":"2010","journal-title":"Journal of Econometrics"},{"issue":"2)","key":"2022110207134246900_bib6","first-page":"484","article-title":"Exact and asymptotically robust permutation tests","volume":"41","author":"Chung","year":"2013","journal-title":"The Annals of Statistics"},{"issue":"1)","key":"2022110207134246900_bib7","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1016\/j.jeconom.2016.01.003","article-title":"Multivariate and multiple permutation tests","volume":"193","author":"Chung","year":"2016","journal-title":"Journal of Econometrics"},{"key":"2022110207134246900_bib8","doi-asserted-by":"crossref","first-page":"697","DOI":"10.1016\/S1574-0692(06)01012-9","article-title":"Interpreting the evidence on life cycle skill formation","volume":"1","author":"Cunha","year":"2006","journal-title":"Handbook of the Economics of Education"},{"issue":"374)","key":"2022110207134246900_bib9","doi-asserted-by":"crossref","first-page":"467","DOI":"10.1080\/01621459.1981.10477669","article-title":"Estimation of the minimum of a function using order statistics","volume":"76","author":"de\u00a0Haan","year":"1981","journal-title":"Journal of the American Statistical Association"},{"key":"2022110207134246900_bib10","first-page":"235","article-title":"Early childhood education","author":"Elango","year":"2015","journal-title":"Economics of Means-Tested Transfer Programs in the United States Volume 2"},{"key":"2022110207134246900_bib11","volume-title":"Statistical Methods for Research Workers","author":"Fisher","year":"1925"},{"key":"2022110207134246900_bib12","volume-title":"The Design of Experiments","author":"Fisher","year":"1935"},{"issue":"23)","key":"2022110207134246900_bib13","doi-asserted-by":"crossref","first-page":"3133","DOI":"10.1002\/sim.6537","article-title":"Penalization, bias reduction, and default priors in logistic and related categorical and survival regressions","volume":"34","author":"Greenland","year":"2015","journal-title":"Statistics in Medicine"},{"issue":"1)","key":"2022110207134246900_bib14","first-page":"1","article-title":"Analyzing social experiments as implemented: a reexamination of the evidence from the HighScope Perry Preschool Program","volume":"1","author":"Heckman","year":"2010","journal-title":"Quantitative Economics"},{"issue":"1\u20132)","key":"2022110207134246900_bib15","doi-asserted-by":"crossref","first-page":"114","DOI":"10.1016\/j.jpubeco.2009.11.001","article-title":"The rate of return to the Highscope Perry Preschool Program","volume":"94","author":"Heckman","year":"2010","journal-title":"Journal of Public Economics"},{"key":"2022110207134246900_bib16","volume-title":"Inference with imperfect randomization: the case of the Perry Preschool Program","author":"Heckman","year":"2020"},{"issue":"2","key":"2022110207134246900_bib17","first-page":"65","article-title":"A simple sequentially rejective multiple test procedure","volume":"6","author":"Holm","year":"1979","journal-title":"Scandinavian Journal of Statistics"},{"issue":"4)","key":"2022110207134246900_bib18","first-page":"523","article-title":"Demystifying double robustness: a comparison of alternative strategies for estimating a population mean from incomplete data","volume":"22","author":"Kang","year":"2007","journal-title":"Statistical Science"},{"issue":"424)","key":"2022110207134246900_bib19","doi-asserted-by":"crossref","first-page":"1242","DOI":"10.1080\/01621459.1993.10476404","article-title":"The Fisher, Neyman-Pearson theories of testing hypotheses: one theory or two?","volume":"88","author":"Lehmann","year":"1993","journal-title":"Journal of the American Statistical Association"},{"key":"2022110207134246900_bib20","volume-title":"Testing Statistical Hypotheses","author":"Lehmann","year":"2005"},{"issue":"6)","key":"2022110207134246900_bib21","doi-asserted-by":"crossref","first-page":"957","DOI":"10.1002\/sim.6764","article-title":"Exact confidence intervals for the average causal effect on a binary outcome","volume":"35","author":"Li","year":"2016","journal-title":"Statistics in Medicine"},{"issue":"37)","key":"2022110207134246900_bib22","doi-asserted-by":"crossref","first-page":"9157","DOI":"10.1073\/pnas.1808191115","article-title":"Asymptotic theory of rerandomization in treatment\u2013control experiments","volume":"115","author":"Li","year":"2018","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"19)","key":"2022110207134246900_bib23","doi-asserted-by":"crossref","first-page":"2937","DOI":"10.1002\/sim.1903","article-title":"Stratification and weighting via the propensity score in estimation of causal treatment effects: a comparative study","volume":"23","author":"Lunceford","year":"2004","journal-title":"Statistics in Medicine"},{"issue":"2)","key":"2022110207134246900_bib24","first-page":"1263","article-title":"Rerandomization to improve covariate balance in experiments","volume":"40","author":"Morgan","year":"2012","journal-title":"The Annals of Statistics"},{"issue":"512)","key":"2022110207134246900_bib25","doi-asserted-by":"crossref","first-page":"1412","DOI":"10.1080\/01621459.2015.1079528","article-title":"Rerandomization to balance tiers of covariates","volume":"110","author":"Morgan","year":"2015","journal-title":"Journal of the American Statistical Association"},{"key":"2022110207134246900_bib26","first-page":"1","article-title":"Pr\u00f3ba uzasadnienia zastosowa\u0144 rachunku prawdopodobie\u0144stwa do doswiadcze\u0144 polowych (On the application of probability theory to agricultural experiments: essay on principles)","volume":"10","author":"Neyman","year":"1923","journal-title":"Roczniki Nauk Rolniczych (Annals of Agricultural Sciences)"},{"key":"2022110207134246900_bib27","volume-title":"The 2013 State of the Union Address","author":"Obama","year":"2013"},{"issue":"284)","key":"2022110207134246900_bib28","doi-asserted-by":"crossref","first-page":"873","DOI":"10.1080\/01621459.1958.10501484","article-title":"The estimation of the parameters of a linear regression system obeying two separate regimes","volume":"53","author":"Quandt","year":"1958","journal-title":"Journal of the American Statistical Association"},{"issue":"338)","key":"2022110207134246900_bib29","doi-asserted-by":"crossref","first-page":"306","DOI":"10.1080\/01621459.1972.10482378","article-title":"A new approach to estimating switching regressions","volume":"67","author":"Quandt","year":"1972","journal-title":"Journal of the American Statistical Association"},{"issue":"6)","key":"2022110207134246900_bib30","doi-asserted-by":"crossref","first-page":"924","DOI":"10.1002\/sim.6384","article-title":"Randomization inference for treatment effects on a binary outcome","volume":"34","author":"Rigdon","year":"2015","journal-title":"Statistics in Medicine"},{"issue":"427)","key":"2022110207134246900_bib31","doi-asserted-by":"crossref","first-page":"846","DOI":"10.1080\/01621459.1994.10476818","article-title":"Estimation of regression coefficients when some regressors are not always observed","volume":"89","author":"Robins","year":"1994","journal-title":"Journal of the American Statistical Association"},{"issue":"1)","key":"2022110207134246900_bib32","first-page":"141","article-title":"Bootstrap and randomization tests of some nonparametric hypotheses","volume":"17","author":"Romano","year":"1989","journal-title":"The Annals of Statistics"},{"issue":"1)","key":"2022110207134246900_bib33","first-page":"169","article-title":"Inference for the identified set in partially identified econometric models","volume":"78","author":"Romano","year":"2010","journal-title":"Econometrica"},{"issue":"469)","key":"2022110207134246900_bib35","doi-asserted-by":"crossref","first-page":"94","DOI":"10.1198\/016214504000000539","article-title":"Exact and approximate stepdown methods for multiple hypothesis testing","volume":"100","author":"Romano","year":"2005","journal-title":"Journal of the American Statistical Association"},{"issue":"1)","key":"2022110207134246900_bib34","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1146\/annurev.economics.102308.124342","article-title":"Hypothesis testing in econometrics","volume":"2","author":"Romano","year":"2010","journal-title":"Annual Review of Economics"},{"issue":"4)","key":"2022110207134246900_bib36","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1007\/s11292-013-9190-3","article-title":"Long-term follow-up of a preschool experiment","volume":"9","author":"Schweinhart","year":"2013","journal-title":"Journal of Experimental Criminology"},{"key":"2022110207134246900_bib40","volume-title":"Young Children Grow Up: The Effects of the Perry Preschool Program on Youths Through Age 15","author":"Schweinhart","year":"1980"},{"key":"2022110207134246900_bib37","volume-title":"Significant Benefits: The High\/Scope Perry Preschool Study Through Age 27 (Monographs of the High\/Scope Educational Research Foundation, 10)","author":"Schweinhart","year":"1993"},{"issue":"8)","key":"2022110207134246900_bib38","first-page":"548","article-title":"The promise of early childhood education","volume":"66","author":"Schweinhart","year":"1985","journal-title":"The Phi Delta Kappan"},{"key":"2022110207134246900_bib39","volume-title":"Lifetime Effects: The High\/Scope Perry Preschool Study Through Age 40 (Monographs of the High\/Scope Educational Research Foundation, 14)","author":"Schweinhart","year":"2005"},{"issue":"1)","key":"2022110207134246900_bib41","doi-asserted-by":"crossref","first-page":"99","DOI":"10.2307\/1884852","article-title":"A behavioral model of rational choice","volume":"69","author":"Simon","year":"1955","journal-title":"The Quarterly Journal of Economics"},{"key":"2022110207134246900_bib42","volume-title":"The Ypsilanti Perry Preschool Project: Preschool Years and Longitudinal Results Through Fourth Grade","author":"Weikart","year":"1978"},{"key":"2022110207134246900_bib43","article-title":"Perry Preschool Project progress report","volume-title":"Technical report","author":"Weikart","year":"1964"},{"key":"2022110207134246900_bib44","article-title":"Randomization tests for weak null hypotheses in randomized experiments","author":"Wu","year":"2020","journal-title":"Journal of the American Statistical Association"},{"issue":"2)","key":"2022110207134246900_bib45","doi-asserted-by":"crossref","first-page":"557","DOI":"10.1093\/qje\/qjy029","article-title":"Channeling Fisher: randomization tests and the statistical insignificance of seemingly significant experimental results","volume":"134","author":"Young","year":"2019","journal-title":"The Quarterly Journal of Economics"},{"issue":"8)","key":"2022110207134246900_bib46","doi-asserted-by":"crossref","first-page":"915","DOI":"10.1037\/0003-066X.48.8.915.b","article-title":"Reply to Spitz\u2019s comments","volume":"48","author":"Zigler","year":"1993","journal-title":"American Psychologist"}],"container-title":["The Econometrics Journal"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/ectj\/advance-article-pdf\/doi\/10.1093\/ectj\/utab009\/36853960\/utab009.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/ectj\/article-pdf\/24\/2\/C1\/46748116\/utab009.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/ectj\/article-pdf\/24\/2\/C1\/46748116\/utab009.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,11,2]],"date-time":"2022-11-02T07:14:40Z","timestamp":1667373280000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/ectj\/article\/24\/2\/C1\/6207935"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,5]]},"references-count":46,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2021,6,29]]},"published-print":{"date-parts":[[2021,6,29]]}},"URL":"https:\/\/doi.org\/10.1093\/ectj\/utab009","relation":{},"ISSN":["1368-4221","1368-423X"],"issn-type":[{"value":"1368-4221","type":"print"},{"value":"1368-423X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,5]]},"published":{"date-parts":[[2021,5]]}}}