{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,1]],"date-time":"2026-10-01T03:22:30Z","timestamp":1790824950625,"version":"4.1.0"},"reference-count":40,"publisher":"The Econometric Society","issue":"2","funder":[{"DOI":"10.13039\/100000935","name":"Russell Sage Foundation","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000935","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["ECTA"],"abstract":"<jats:p>This paper develops methods for detecting discrimination by individual employers using correspondence experiments that send fictitious resumes to real job openings. We establish identification of higher moments of the distribution of job\u2010level callback rates as a function of the number of resumes sent to each job and propose shape\u2010constrained estimators of these moments. Applying our methods to three experimental data sets, we find striking job\u2010level heterogeneity in the extent to which callback probabilities differ by race or sex. Estimates of higher moments reveal that while most jobs barely discriminate, a few discriminate heavily. These moment estimates are then used to bound the share of jobs that discriminate and the posterior probability that each individual job is engaged in discrimination. In a recent experiment manipulating racially distinctive names, we find that at least 85% of jobs that contact both of two white applications and neither of two black applications are engaged in discrimination. To assess the potential value of our methods for regulators, we consider the accuracy of decision rules for investigating suspicious callback behavior in various experimental designs under a simple two\u2010type model that rationalizes the experimental data. Though we estimate that only 17% of employers discriminate on the basis of race, we find that an experiment sending 10 applications to each job would enable detection of 7\u201310% of discriminatory jobs while yielding Type I error rates below 0.2%. A minimax decision rule acknowledging partial identification of the distribution of callback rates yields only slightly fewer investigations than a Bayes decision rule based on the two\u2010type model. These findings suggest illegal labor market discrimination can be reliably monitored with relatively small modifications to existing correspondence designs.<\/jats:p>","DOI":"10.3982\/ecta17489","type":"journal-article","created":{"date-parts":[[2021,3,15]],"date-time":"2021-03-15T05:42:01Z","timestamp":1615786921000},"page":"765-792","source":"Crossref","is-referenced-by-count":44,"title":["Reasonable Doubt: Experimental Detection of Job\u2010Level Employment Discrimination"],"prefix":"10.3982","volume":"89","author":[{"given":"Patrick","family":"Kline","sequence":"first","affiliation":[{"name":"Department of Economics, UC Berkeley"},{"name":"NBER"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christopher","family":"Walters","sequence":"additional","affiliation":[{"name":"Department of Economics, UC Berkeley"},{"name":"NBER"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"2359","published-online":{"date-parts":[[2021]]},"reference":[{"key":"ecta200246-bib-0001","unstructured":"7th Circuit Court of Appeals (2006): \u201cEEOC v. Target Corp.\u201d 460 (F. 3d), 946."},{"key":"ecta200246-bib-0002","first-page":"3143","volume-title":"Handbook of Labor Economics","volume":"3C, Chapter 48","author":"Altonji","year":"1999"},{"key":"ecta200246-bib-0003","doi-asserted-by":"crossref","first-page":"871","DOI":"10.1093\/qje\/qjx001","volume":"132","author":"Angrist","year":"2017","journal-title":"Quarterly Journal of Economics"},{"key":"ecta200246-bib-0004","doi-asserted-by":"crossref","first-page":"376","DOI":"10.1257\/aer.104.5.376","volume":"104","author":"Arceo-Gomez","year":"2014","journal-title":"American Economic Review: Papers & Proceedings"},{"key":"ecta200246-bib-0005","doi-asserted-by":"crossref","unstructured":"Arnold, D., W. Dobbie, and P. Hull (2020): \u201cMeasuring Racial Discrimination in Bail Decisions,\u201d Working paper.","DOI":"10.3386\/w26999"},{"key":"ecta200246-bib-0006","volume-title":"The Economics of Discrimination","author":"Becker","year":"1957"},{"key":"ecta200246-bib-0007","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1111\/j.2517-6161.1995.tb02031.x","volume":"57","author":"Benjamini","year":"1995","journal-title":"Journal of the Royal Statistical Society"},{"key":"ecta200246-bib-0008","doi-asserted-by":"crossref","DOI":"10.1016\/bs.hefe.2016.08.004","volume-title":"Handbook of Field Experiments","volume":"1","author":"Bertrand","year":"2017"},{"key":"ecta200246-bib-0009","doi-asserted-by":"crossref","first-page":"991","DOI":"10.1257\/0002828042002561","volume":"94","author":"Bertrand","year":"2004","journal-title":"American Economic Review"},{"key":"ecta200246-bib-0010","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1214\/07-AOAS138","volume":"2","author":"Brown","year":"2008","journal-title":"The Annals of Applied Statistics"},{"key":"ecta200246-bib-0011","doi-asserted-by":"crossref","unstructured":"Chernozhukov, V., W. K. Newey, and A. Santos (2015): \u201cConstrained Conditional Moment Restriction Models,\u201d arXiv:1509.06311. ArXiv preprint.","DOI":"10.1920\/wp.cem.2015.5915"},{"key":"ecta200246-bib-0012","doi-asserted-by":"crossref","unstructured":"Chetty, R., J. N. Friedman, N. Hendren, M. R. Jones, and S. R. Porter (2018): \u201cThe Opportunity Atlas: Mapping the Childhood Roots of Social Mobility,\u201d Working paper.","DOI":"10.3386\/w25147"},{"key":"ecta200246-bib-0013","doi-asserted-by":"crossref","first-page":"2593","DOI":"10.1257\/aer.104.9.2593","volume":"104","author":"Chetty","year":"2014","journal-title":"American Economic Review"},{"key":"ecta200246-bib-0014","doi-asserted-by":"crossref","first-page":"2633","DOI":"10.1257\/aer.104.9.2633","volume":"104","author":"Chetty","year":"2014","journal-title":"American Economic Review"},{"key":"ecta200246-bib-0015","doi-asserted-by":"crossref","first-page":"1163","DOI":"10.1093\/qje\/qjy006","volume":"133","author":"Chetty","year":"2018","journal-title":"Quarterly Journal of Economics"},{"key":"ecta200246-bib-0016","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1146\/annurev-economics-080217-053417","volume":"10","author":"Chetverikov","year":"2018","journal-title":"Annual Review of Economics"},{"key":"ecta200246-bib-0017","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1214\/09-STS308","volume":"25","author":"Efron","year":"2010","journal-title":"Statistical Science"},{"key":"ecta200246-bib-0018","volume-title":"Large-Scale Inference: Empirical Bayes Methods for Estimation, Testing, and Prediction","volume":"1","author":"Efron","year":"2012"},{"key":"ecta200246-bib-0019","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1093\/biomet\/asv068","volume":"103","author":"Efron","year":"2016","journal-title":"Biometrika"},{"key":"ecta200246-bib-0020","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1080\/01621459.1975.10479864","volume":"70","author":"Efron","year":"1975","journal-title":"Journal of the American Statistical Association"},{"key":"ecta200246-bib-0021","doi-asserted-by":"crossref","first-page":"1151","DOI":"10.1198\/016214501753382129","volume":"96","author":"Efron","year":"2001","journal-title":"Journal of the American Statistical Association"},{"key":"ecta200246-bib-0022","first-page":"377","volume":"86","author":"Fang","year":"2018","journal-title":"The Review of Economic Studies"},{"key":"ecta200246-bib-0023","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1257\/aer.p20161010","volume":"106","author":"Farber","year":"2016","journal-title":"American Economic Review: Papers & Proceedings"},{"key":"ecta200246-bib-0024","doi-asserted-by":"crossref","first-page":"767","DOI":"10.1162\/0033553041502180","volume":"119","author":"Fryer","year":"2004","journal-title":"Quarterly Journal of Economics"},{"key":"ecta200246-bib-0025","doi-asserted-by":"crossref","first-page":"F417","DOI":"10.1111\/ecoj.12080","volume":"123","author":"Guryan","year":"2013","journal-title":"The Economic Journal"},{"key":"ecta200246-bib-0026","doi-asserted-by":"crossref","first-page":"1558","DOI":"10.1086\/697741","volume":"126","author":"Halevy","year":"2018","journal-title":"Journal of Political Economy"},{"key":"ecta200246-bib-0027","doi-asserted-by":"crossref","first-page":"397","DOI":"10.1214\/19-AOS1812","volume":"48","author":"Hong","year":"2020","journal-title":"Annals of Statistics"},{"key":"ecta200246-bib-0028","doi-asserted-by":"crossref","unstructured":"Hull, P. D. (2018): \u201cEstimating Hospital Quality With Quasi-Experimental Data,\u201d Working paper.","DOI":"10.2139\/ssrn.3118358"},{"key":"ecta200246-bib-0029","doi-asserted-by":"crossref","unstructured":"Kline, P., and C. Walters (2021): \u201cSupplement to \u2018Reasonable Doubt: Experimental Detection of Job-Level Employment Discrimination\u2019,\u201d Econometrica Supplemental Material, 89, https:\/\/doi.org\/10.3982\/ECTA17489.","DOI":"10.3982\/ECTA17489"},{"key":"ecta200246-bib-0030","doi-asserted-by":"crossref","first-page":"907","DOI":"10.1126\/science.1215191","volume":"336","author":"Levine","year":"2012","journal-title":"Science"},{"key":"ecta200246-bib-0031","doi-asserted-by":"crossref","first-page":"415","DOI":"10.1016\/S0304-4076(99)00045-7","volume":"95","author":"Manski","year":"2000","journal-title":"Journal of Econometrics"},{"key":"ecta200246-bib-0032","doi-asserted-by":"crossref","first-page":"1094","DOI":"10.1214\/aos\/1013699995","volume":"29","author":"Noubiap","year":"2001","journal-title":"The Annals of Statistics"},{"key":"ecta200246-bib-0033","doi-asserted-by":"crossref","first-page":"1093","DOI":"10.1515\/bejeap-2014-0082","volume":"15","author":"Nunley","year":"2015","journal-title":"B.E. Journal of Economic Analysis and Policy"},{"key":"ecta200246-bib-0034","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1080\/14786440009463897","volume":"50","author":"Pearson","year":"1900","journal-title":"Philosophical Magazine, Series 5"},{"key":"ecta200246-bib-0035","first-page":"591","volume":"75","author":"Rubin","year":"1980","journal-title":"Journal of the American Statistical Association"},{"key":"ecta200246-bib-0036","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1080\/01621459.1951.10500768","volume":"46","author":"Savage","year":"1951","journal-title":"Journal of the American Statistical association"},{"key":"ecta200246-bib-0037","doi-asserted-by":"crossref","first-page":"479","DOI":"10.1111\/1467-9868.00346","volume":"64","author":"Storey","year":"2002","journal-title":"Journal of the Royal Statistical Society: Series B (Statistical Methodology)"},{"key":"ecta200246-bib-0038","doi-asserted-by":"crossref","first-page":"2013","DOI":"10.1214\/aos\/1074290335","volume":"31","author":"Storey","year":"2003","journal-title":"The Annals of Statistics"},{"key":"ecta200246-bib-0039","unstructured":"US Equal Employment Opportunity Commission (2016): \u201cAdvancing opportunity: a review of the systematic program of the US Equal Employment Opportunity Commission,\u201d Technical report."},{"key":"ecta200246-bib-0040","doi-asserted-by":"crossref","first-page":"265","DOI":"10.2307\/1969022","volume":"46","author":"Wald","year":"1945","journal-title":"Annals of Mathematics"}],"container-title":["Econometrica"],"original-title":[],"language":"en","deposited":{"date-parts":[[2024,8,25]],"date-time":"2024-08-25T20:25:27Z","timestamp":1724617527000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.econometricsociety.org\/doi\/10.3982\/ECTA17489"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021]]},"references-count":40,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2021]]}},"alternative-id":["10.3982\/ECTA17489"],"URL":"https:\/\/doi.org\/10.3982\/ecta17489","relation":{},"ISSN":["0012-9682"],"issn-type":[{"value":"0012-9682","type":"print"}],"subject":[],"published":{"date-parts":[[2021]]}}}