{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,16]],"date-time":"2026-05-16T03:45:33Z","timestamp":1778903133697,"version":"3.51.4"},"reference-count":58,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2019,11,8]],"date-time":"2019-11-08T00:00:00Z","timestamp":1573171200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"DOI":"10.13039\/100010663","name":"European Research Council","doi-asserted-by":"publisher","award":["ERC-2014-CoG-646917-ROMIA"],"award-info":[{"award-number":["ERC-2014-CoG-646917-ROMIA"]}],"id":[{"id":"10.13039\/100010663","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100010663","name":"European Research Council","doi-asserted-by":"publisher","award":["ERC-2015-CoG-682349"],"award-info":[{"award-number":["ERC-2015-CoG-682349"]}],"id":[{"id":"10.13039\/100010663","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000269","name":"UK Economic and Social Research Council","doi-asserted-by":"crossref","award":["ES\/P008909\/1"],"award-info":[{"award-number":["ES\/P008909\/1"]}],"id":[{"id":"10.13039\/501100000269","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2020,1,1]]},"abstract":"<jats:title>Summary<\/jats:title>\n               <jats:p>In a randomized control trial, the precision of an average treatment effect estimator and the power of the corresponding t-test can be improved either by collecting data on additional individuals, or by collecting additional covariates that predict the outcome variable. To design the experiment, a researcher needs to solve this trade-off subject to her budget constraint. We show that this optimization problem is equivalent to optimally predicting outcomes by the covariates, which in turn can be solved using existing machine learning techniques using pre-experimental data such as other similar studies, a census, or a household survey. In two empirical applications, we show that our procedure can lead to reductions of up to 58% in the costs of data collection, or improvements of the same magnitude in the precision of the treatment effect estimator.<\/jats:p>","DOI":"10.1093\/ectj\/utz020","type":"journal-article","created":{"date-parts":[[2019,11,8]],"date-time":"2019-11-08T17:43:06Z","timestamp":1573234986000},"page":"1-31","source":"Crossref","is-referenced-by-count":3,"title":["Optimal data collection for randomized control trials"],"prefix":"10.1093","volume":"23","author":[{"given":"Pedro","family":"Carneiro","sequence":"first","affiliation":[{"name":"University College London, Gower Street, London WC1E 6BT, UK"},{"name":"Institute for Fiscal Studies, 7 Ridgmount Street, London WC1E 7AE, UK"},{"name":"Centre for Microdata Methods and Practice, 7 Ridgmount Street, London WC1E 7AE, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sokbae","family":"Lee","sequence":"additional","affiliation":[{"name":"Columbia University, 420 West 118th Street, New York, NY 10027, USA"},{"name":"Institute for Fiscal Studies, 7 Ridgmount Street, London WC1E 7AE, UK"},{"name":"Centre for Microdata Methods and Practice, 7 Ridgmount Street, London WC1E 7AE, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Daniel","family":"Wilhelm","sequence":"additional","affiliation":[{"name":"University College London, Gower Street, London WC1E 6BT, UK"},{"name":"Institute for Fiscal Studies, 7 Ridgmount Street, London WC1E 7AE, UK"},{"name":"Centre for Microdata Methods and Practice, 7 Ridgmount Street, London WC1E 7AE, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2019,11,8]]},"reference":[{"key":"2021051415474475700_bib1","first-page":"151","article-title":"Microcredit impacts: Evidence from a randomized microcredit program placement experiment by Compartamos Banco","volume":"7","author":"Angelucci","year":"2015","journal-title":"American Economic Journal: Applied Economics"},{"key":"2021051415474475700_bib2","first-page":"90","article-title":"The impacts of microfinance: Evidence from joint-liability lending in Mongolia","volume":"7","author":"Attanasio","year":"2015","journal-title":"American Economic Journal: Applied Economics"},{"key":"2021051415474475700_bib3","article-title":"Free access to child care, labor supply, and child development","author":"Attanasio","year":"2014"},{"key":"2021051415474475700_bib4","first-page":"183","article-title":"The impacts of microcredit: Evidence from Bosnia and Herzegovina","volume":"7","author":"Augsburg","year":"2015","journal-title":"American Economic Journal: Applied Economics"},{"issue":"3","key":"2021051415474475700_bib5","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1257\/jep.25.3.63","article-title":"Field experiments with firms","volume":"25","author":"Bandiera","year":"2011","journal-title":"Journal of Economic Perspectives"},{"key":"2021051415474475700_bib6","doi-asserted-by":"crossref","DOI":"10.3386\/w23867","article-title":"A theory of experimenters","author":"Banerjee","year":"2017"},{"key":"2021051415474475700_bib7","first-page":"22","article-title":"The miracle of microfinance? Evidence from a randomized evaluation","volume":"7","author":"Banerjee","year":"2015","journal-title":"American Economic Journal: Applied Economics"},{"key":"2021051415474475700_bib8","first-page":"1","article-title":"Six randomized evaluations of microcredit: Introduction and further steps","volume":"7","author":"Banerjee","year":"2015","journal-title":"American Economic Journal: Applied Economics"},{"key":"2021051415474475700_bib9","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1146\/annurev.economics.050708.143235","article-title":"The experimental approach to development economics","volume":"1","author":"Banerjee","year":"2009","journal-title":"Annual Review of Economics"},{"key":"2021051415474475700_bib10","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1214\/009053607000000631","article-title":"Approximation and learning by greedy algorithms","volume":"36","author":"Barron","year":"2008","journal-title":"Annals of Statistics"},{"key":"2021051415474475700_bib11","doi-asserted-by":"crossref","first-page":"521","DOI":"10.3150\/11-BEJ410","article-title":"Least squares after model selection in high-dimensional sparse models","volume":"19","author":"Belloni","year":"2013","journal-title":"Bernoulli"},{"key":"2021051415474475700_bib12","doi-asserted-by":"crossref","first-page":"168","DOI":"10.1016\/j.jeconom.2011.11.007","article-title":"Inferring welfare maximizing treatment assignment under budget constraints","volume":"167","author":"Bhattacharya","year":"2012","journal-title":"Journal of Econometrics"},{"key":"2021051415474475700_bib13","first-page":"200","article-title":"In pursuit of balance: Randomization in practice in development field experiments","volume":"1","author":"Bruhn","year":"2009","journal-title":"American Economic Journal: Applied Economics"},{"key":"2021051415474475700_bib14","doi-asserted-by":"crossref","DOI":"10.3386\/w21063","article-title":"Decentralizing education resources: School grants in Senegal","author":"Carneiro","year":"2015"},{"key":"2021051415474475700_bib15","doi-asserted-by":"crossref","first-page":"C1","DOI":"10.1111\/ectj.12097","article-title":"Double\/debiased machine learning for treatment and structural parameters","volume":"21","author":"Chernozhukov","year":"2018","journal-title":"Econometrics Journal"},{"issue":"3","key":"2021051415474475700_bib16","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1257\/jep.29.3.81","article-title":"Pre-analysis plans have limited upside, especially where replications are feasible","volume":"29","author":"Coffman","year":"2015","journal-title":"Journal of Economic Perspectives"},{"key":"2021051415474475700_bib17","first-page":"123","article-title":"Estimating the impact of microcredit on those who take it up: Evidence from a randomized experiment in Morocco","volume":"7","author":"Cr\u00e9pon","year":"2015","journal-title":"American Economic Journal: Applied Economics"},{"key":"2021051415474475700_bib18","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1007\/BF02678430","article-title":"Adaptive greedy approximations","volume":"13","author":"Davis","year":"1997","journal-title":"Constructive Approximation"},{"key":"2021051415474475700_bib19","doi-asserted-by":"crossref","first-page":"1583","DOI":"10.1093\/restud\/rdx005","article-title":"More data or better data? A statistical decision problem","volume":"84","author":"Dominitz","year":"2017","journal-title":"Review of Economic Studies"},{"issue":"5","key":"2021051415474475700_bib20","doi-asserted-by":"crossref","first-page":"1739","DOI":"10.1257\/aer.101.5.1739","article-title":"Peer effects, teacher incentives, and the impact of tracking: Evidence from a randomized evaluation in Kenya","volume":"101","author":"Duflo","year":"2011","journal-title":"American Economic Review"},{"key":"2021051415474475700_bib21","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1016\/j.jpubeco.2014.11.008","article-title":"School governance, teacher incentives, and pupil\u2013teacher ratios: Experimental evidence from Kenyan primary schools","volume":"123","author":"Duflo","year":"2015","journal-title":"Journal of Public Economics"},{"key":"2021051415474475700_bib22","first-page":"3895","article-title":"Using randomization in development economics research: A toolkit","volume-title":"Handbook of Development Economics","author":"Duflo","year":"2007"},{"key":"2021051415474475700_bib23","doi-asserted-by":"crossref","first-page":"1057","DOI":"10.1093\/qje\/qjs020","article-title":"The Oregon health insurance experiment: Evidence from the first year","volume":"127","author":"Finkelstein","year":"2012","journal-title":"Quarterly Journal of Economics"},{"key":"2021051415474475700_bib24","volume-title":"Statistical Methods for Research Workers","author":"Fisher","year":"1925","edition":"1st ed."},{"key":"2021051415474475700_bib25","volume-title":"The Design of Experiments","author":"Fisher","year":"1935"},{"key":"2021051415474475700_bib26","doi-asserted-by":"crossref","first-page":"1685","DOI":"10.1002\/sim.4780111304","article-title":"Repeated measures in clinical trials: Analysis using mean summary statistics and its implications for design","volume":"11","author":"Frison","year":"1992","journal-title":"Statistics in Medicine"},{"key":"2021051415474475700_bib27","first-page":"205","article-title":"Teacher incentives","volume":"2","author":"Glewwe","year":"2010","journal-title":"American Economic Journal: Applied Economics"},{"key":"2021051415474475700_bib28","first-page":"112","article-title":"Many children left behind? Textbooks and test scores in Kenya","volume":"1","author":"Glewwe","year":"2009","journal-title":"American Economic Journal: Applied Economics"},{"key":"2021051415474475700_bib29","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1198\/jbes.2009.08161","article-title":"Adaptive experimental design using the propensity score","volume":"29","author":"Hahn","year":"2011","journal-title":"Journal of Business and Economic Statistics"},{"key":"2021051415474475700_bib30","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1257\/jel.51.1.162","article-title":"Six decades of top economics publishing: Who and how?","volume":"51","author":"Hamermesh","year":"2013","journal-title":"Journal of Economic Literature"},{"key":"2021051415474475700_bib31","first-page":"3371","article-title":"Learning with structured sparsity","volume":"12","author":"Huang","year":"2011","journal-title":"Journal of Machine Learning Research"},{"key":"2021051415474475700_bib32","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9781139025751","volume-title":"Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction","author":"Imbens","year":"2015"},{"key":"2021051415474475700_bib33","doi-asserted-by":"crossref","first-page":"1473","DOI":"10.5705\/ss.2010.081","article-title":"A stepwise regression method and consistent model selection for high-dimensional sparse linear models","volume":"21","author":"Ing","year":"2011","journal-title":"Statistica Sinica"},{"key":"2021051415474475700_bib34","doi-asserted-by":"crossref","first-page":"324","DOI":"10.1093\/pan\/mpw012","article-title":"Why experimenters might not always want to randomize, and what they could do instead","volume":"24","author":"Kasy","year":"2016","journal-title":"Political Analysis"},{"issue":"5","key":"2021051415474475700_bib35","doi-asserted-by":"crossref","first-page":"491","DOI":"10.1257\/aer.p20151023","article-title":"Prediction policy problems","volume":"105","author":"Kleinberg","year":"2015","journal-title":"American Economic Review"},{"key":"2021051415474475700_bib36","doi-asserted-by":"crossref","first-page":"437","DOI":"10.1162\/rest.91.3.437","article-title":"Incentives to learn","volume":"91","author":"Kremer","year":"2009","journal-title":"Review of Economics and Statistics"},{"key":"2021051415474475700_bib37","doi-asserted-by":"crossref","first-page":"9157","DOI":"10.1073\/pnas.1808191115","article-title":"Asymptotic theory of rerandomization in treatment\u2013control experiments","volume":"115","author":"Li","year":"2018","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"3","key":"2021051415474475700_bib38","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1257\/jep.25.3.3","article-title":"Why economists should conduct field experiments and 14 tips for pulling one off","volume":"25","author":"List","year":"2011","journal-title":"Journal of Economic Perspectives"},{"key":"2021051415474475700_bib39","first-page":"103","article-title":"Field experiments in labor economics","volume-title":"Handbook of Labor Economics","author":"List","year":"2011"},{"key":"2021051415474475700_bib40","doi-asserted-by":"crossref","first-page":"439","DOI":"10.1007\/s10683-011-9275-7","article-title":"So you want to run an experiment, now what? Some simple rules of thumb for optimal experimental design","volume":"14","author":"List","year":"2011","journal-title":"Experimental Economics"},{"key":"2021051415474475700_bib41","doi-asserted-by":"crossref","DOI":"10.1920\/wp.ifs.2015.1517","article-title":"Going beyond simple sample size calculations: A practioner\u2019s guide","author":"McConnell","year":"2015"},{"key":"2021051415474475700_bib42","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1016\/j.jdeveco.2012.01.002","article-title":"Beyond baseline and follow-up: The case for more T in experiments","volume":"99","author":"McKenzie","year":"2012","journal-title":"Journal of Development Economics"},{"key":"2021051415474475700_bib43","first-page":"57","article-title":"Understanding the average impact of microcredit expansions: A bayesian hierarchical analysis of seven randomized experiments","volume":"11","author":"Meager","year":"2019","journal-title":"American Economic Journal: Applied Economics"},{"key":"2021051415474475700_bib44","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1111\/j.1468-0262.2004.00481.x","article-title":"Worms: Identifying impacts on education and health in the presence of treatment externalities","volume":"72","author":"Miguel","year":"2004","journal-title":"Econometrica"},{"key":"2021051415474475700_bib45","doi-asserted-by":"crossref","first-page":"1263","DOI":"10.1214\/12-AOS1008","article-title":"Rerandomization to improve covariate balance in experiments","volume":"40","author":"Morgan","year":"2012","journal-title":"Annals of Statistics"},{"key":"2021051415474475700_bib46","doi-asserted-by":"crossref","first-page":"1412","DOI":"10.1080\/01621459.2015.1079528","article-title":"Rerandomization to balance tiers of covariates","volume":"110","author":"Morgan","year":"2015","journal-title":"Journal of the American Statistical Association"},{"key":"2021051415474475700_bib47","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1137\/S0097539792240406","article-title":"Sparse approximate solutions to linear systems","volume":"24","author":"Natarajan","year":"1995","journal-title":"SIAM Journal on Computing"},{"key":"2021051415474475700_bib48","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1257\/jep.29.3.61","article-title":"Promises and perils of pre-analysis plans","volume":"29","author":"Olken","year":"2015","journal-title":"Journal of Economic Perspectives"},{"key":"2021051415474475700_bib49","doi-asserted-by":"crossref","first-page":"1227","DOI":"10.3150\/14-BEJ691","article-title":"Greedy algorithms for prediction","volume":"22","author":"Sancetta","year":"2016","journal-title":"Bernoulli"},{"key":"2021051415474475700_bib50","article-title":"Stratification trees for adaptive randomization in randomized controlled trials","author":"Tabord-Mehan","year":"2018"},{"key":"2021051415474475700_bib51","first-page":"54","article-title":"The impacts of microcredit: Evidence from Ethiopia","volume":"7","author":"Tarozzi","year":"2015","journal-title":"American Economic Journal: Applied Economics"},{"key":"2021051415474475700_bib52","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511762291","volume-title":"Greedy Approximation","author":"Temlyakov","year":"2011"},{"key":"2021051415474475700_bib53","article-title":"An economic theory of statistical testing","author":"Tetenov","year":"2016","journal-title":"Technical report"},{"key":"2021051415474475700_bib54","doi-asserted-by":"crossref","first-page":"2231","DOI":"10.1109\/TIT.2004.834793","article-title":"Greed is good: Algorithmic results for sparse approximation","volume":"50","author":"Tropp","year":"2004","journal-title":"IEEE Transactions on Information Theory"},{"key":"2021051415474475700_bib55","doi-asserted-by":"crossref","first-page":"4655","DOI":"10.1109\/TIT.2007.909108","article-title":"Signal recovery from random measurements via orthogonal matching pursuit","volume":"53","author":"Tropp","year":"2007","journal-title":"IEEE Transactions on Information Theory"},{"key":"2021051415474475700_bib56","doi-asserted-by":"crossref","DOI":"10.2139\/ssrn.667881","article-title":"Schools meals, educational achievement and school competition: Evidence from a randomized evaluation","author":"Vermeersch","year":"2004"},{"key":"2021051415474475700_bib57","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1111\/ecca.12168","article-title":"Education as liberation?","volume":"83","author":"Willa","year":"2016","journal-title":"Economica"},{"key":"2021051415474475700_bib58","first-page":"555","article-title":"On the consistency of feature selection using greedy least squares regression","volume":"10","author":"Zhang","year":"2009","journal-title":"Journal of Machine Learning Research"}],"container-title":["The Econometrics Journal"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/ectj\/advance-article-pdf\/doi\/10.1093\/ectj\/utz020\/30493673\/utz020.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/ectj\/article-pdf\/23\/1\/1\/37935306\/utz020.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/ectj\/article-pdf\/23\/1\/1\/37935306\/utz020.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,5,14]],"date-time":"2021-05-14T15:48:48Z","timestamp":1621007328000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/ectj\/article\/23\/1\/1\/5614972"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,11,8]]},"references-count":58,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2019,11,8]]},"published-print":{"date-parts":[[2020,1,1]]}},"URL":"https:\/\/doi.org\/10.1093\/ectj\/utz020","relation":{},"ISSN":["1368-4221","1368-423X"],"issn-type":[{"value":"1368-4221","type":"print"},{"value":"1368-423X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2020,1]]},"published":{"date-parts":[[2019,11,8]]}}}