{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T07:18:58Z","timestamp":1791875938722,"version":"4.3.4"},"reference-count":56,"publisher":"The Econometric Society","issue":"1","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["ECTA"],"abstract":"<jats:p>We study deep neural networks and their use in semiparametric inference. We establish novel nonasymptotic high probability bounds for deep feedforward neural nets. These deliver rates of convergence that are sufficiently fast (in some cases minimax optimal) to allow us to establish valid second\u2010step inference after first\u2010step estimation with deep learning, a result also new to the literature. Our nonasymptotic high probability bounds, and the subsequent semiparametric inference, treat the current standard architecture: fully connected feedforward neural networks (multilayer perceptrons), with the now\u2010common rectified linear unit activation function, unbounded weights, and a depth explicitly diverging with the sample size. We discuss other architectures as well, including fixed\u2010width, very deep networks. We establish the nonasymptotic bounds for these deep nets for a general class of nonparametric regression\u2010type loss functions, which includes as special cases least squares, logistic regression, and other generalized linear models. We then apply our theory to develop semiparametric inference, focusing on causal parameters for concreteness, and demonstrate the effectiveness of deep learning with an empirical application to direct mail marketing.<\/jats:p>","DOI":"10.3982\/ecta16901","type":"journal-article","created":{"date-parts":[[2021,1,12]],"date-time":"2021-01-12T03:05:05Z","timestamp":1610420705000},"page":"181-213","source":"Crossref","is-referenced-by-count":334,"title":["Deep Neural Networks for Estimation and Inference"],"prefix":"10.3982","volume":"89","author":[{"given":"Max H.","family":"Farrell","sequence":"first","affiliation":[{"name":"Booth School of Business, University of Chicago"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tengyuan","family":"Liang","sequence":"additional","affiliation":[{"name":"Booth School of Business, University of Chicago"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sanjog","family":"Misra","sequence":"additional","affiliation":[{"name":"Booth School of Business, University of Chicago"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"2359","published-online":{"date-parts":[[2021]]},"reference":[{"key":"ecta200232-bib-0001","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511624216","volume-title":"Neural Network Learning: Theoretical Foundations","author":"Anthony","year":"1999"},{"key":"ecta200232-bib-0002","doi-asserted-by":"crossref","unstructured":"Athey, S., G. W. Imbens, J. Metzger, and E. M. Munro (2019): \u201cUsing Wasserstein Generative Adversarial Networks for the Design of Monte Carlo Simulations,\u201d arXiv:1909.02210. Preprint.","DOI":"10.3386\/w26566"},{"key":"ecta200232-bib-0003","doi-asserted-by":"crossref","first-page":"597","DOI":"10.1111\/rssb.12268","volume":"80","author":"Athey","year":"2018","journal-title":"Journal of the Royal Statistical Society, Series B"},{"key":"ecta200232-bib-0004","first-page":"629","volume":"18","author":"Bach","year":"2017","journal-title":"The Journal of Machine Learning Research"},{"key":"ecta200232-bib-0005","doi-asserted-by":"crossref","first-page":"1497","DOI":"10.1214\/009053605000000282","volume":"33","author":"Bartlett","year":"2005","journal-title":"The Annals of Statistics"},{"key":"ecta200232-bib-0006","volume-title":"Proceedings of the 22nd Annual Conference on Learning Theory (COLT 2017)","author":"Bartlett","year":"2017"},{"key":"ecta200232-bib-0007","doi-asserted-by":"crossref","first-page":"2261","DOI":"10.1214\/18-AOS1747","volume":"47","author":"Bauer","year":"2019","journal-title":"Annals of Statistics"},{"key":"ecta200232-bib-0008","doi-asserted-by":"crossref","first-page":"233","DOI":"10.3982\/ECTA12723","volume":"85","author":"Belloni","year":"2017","journal-title":"Econometrica"},{"key":"ecta200232-bib-0009","doi-asserted-by":"crossref","first-page":"608","DOI":"10.1093\/restud\/rdt044","volume":"81","author":"Belloni","year":"2014","journal-title":"Review of Economic Studies"},{"key":"ecta200232-bib-0010","first-page":"381","volume":"50","author":"Bickel","year":"1988","journal-title":"Sankhya"},{"key":"ecta200232-bib-0011","doi-asserted-by":"crossref","first-page":"1705","DOI":"10.1214\/08-AOS620","volume":"37","author":"Bickel","year":"2009","journal-title":"The Annals of Statistics"},{"key":"ecta200232-bib-0012","doi-asserted-by":"crossref","first-page":"1095","DOI":"10.1093\/restud\/rdy053","volume":"86","author":"Cattaneo","year":"2019","journal-title":"Review of Economic Studies"},{"key":"ecta200232-bib-0013","doi-asserted-by":"crossref","first-page":"682","DOI":"10.1109\/18.749011","volume":"45","author":"Chen","year":"1999","journal-title":"IEEE Transactions on Information Theory"},{"key":"ecta200232-bib-0014","doi-asserted-by":"crossref","first-page":"808","DOI":"10.1214\/009053607000000947","volume":"36","author":"Chen","year":"2008","journal-title":"The Annals of Statistics"},{"key":"ecta200232-bib-0015","doi-asserted-by":"crossref","first-page":"C1","DOI":"10.1111\/ectj.12097","volume":"21","author":"Chernozhukov","year":"2018","journal-title":"The Econometrics Journal"},{"key":"ecta200232-bib-0016","volume-title":"Proceedings of the 22nd Annual Conference on Learning Theory (COLT 2017)","author":"Daniely","year":"2017"},{"key":"ecta200232-bib-0017","author":"Dou","year":"2020","journal-title":"Journal of the American Statistical Association"},{"key":"ecta200232-bib-0018","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.jeconom.2015.06.017","volume":"189","author":"Farrell","year":"2015","journal-title":"Journal of Econometrics"},{"key":"ecta200232-bib-0019","unstructured":"Farrell, M. H., T. Liang, and S. Misra (2019a): \u201cDeep Neural Networks for Estimation and Inference,\u201d arXiv:1809.09953."},{"key":"ecta200232-bib-0020","doi-asserted-by":"crossref","unstructured":"Farrell, M. H., T. Liang, and S. Misra (2021): \u201cSupplement to \u2018Deep Neural Networks for Estimation and Inference\u2019,\u201d Econometrica Supplemental Material, 89, https:\/\/doi.org\/10.3982\/ECTA16901.","DOI":"10.3982\/ECTA16901"},{"key":"ecta200232-bib-0021","author":"Ghorbani","year":"2019","journal-title":"Annals of Statistics"},{"key":"ecta200232-bib-0022","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9781107337862","volume-title":"Mathematical Foundations of Infinite-Dimensional Models","author":"Gine","year":"2016"},{"key":"ecta200232-bib-0023","volume-title":"Deep Learning","author":"Goodfellow","year":"2016"},{"key":"ecta200232-bib-0024","doi-asserted-by":"crossref","first-page":"315","DOI":"10.2307\/2998560","volume":"66","author":"Hahn","year":"1998","journal-title":"Econometrica"},{"key":"ecta200232-bib-0025","doi-asserted-by":"crossref","first-page":"992","DOI":"10.3390\/math7100992","volume":"7","author":"Hanin","year":"2019","journal-title":"Mathematics"},{"key":"ecta200232-bib-0026","first-page":"1414","volume-title":"International Conference on Machine Learning","author":"Hartford","year":"2017"},{"key":"ecta200232-bib-0027","doi-asserted-by":"crossref","unstructured":"Hitsch, G. J., and S. Misra (2018): \u201cHeterogeneous Treatment Effects and Optimal Targeting Policy Evaluation,\u201d SSRN preprint 3111957.","DOI":"10.2139\/ssrn.3111957"},{"key":"ecta200232-bib-0028","first-page":"2869","volume":"15","author":"Javanmard","year":"2014","journal-title":"The Journal of Machine Learning Research"},{"key":"ecta200232-bib-0029","unstructured":"Kingma, D. P., and J. Ba (2014): \u201cAdam: A Method for Stochastic Optimization,\u201d arXiv:1412.6980. Preprint."},{"key":"ecta200232-bib-0030","doi-asserted-by":"crossref","first-page":"2593","DOI":"10.1214\/009053606000001019","volume":"34","author":"Koltchinskii","year":"2006","journal-title":"The Annals of Statistics"},{"key":"ecta200232-bib-0031","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-642-22147-7","volume-title":"Oracle Inequalities in Empirical Risk Minimization and Sparse Recovery Problems","author":"Koltchinskii","year":"2011"},{"key":"ecta200232-bib-0032","doi-asserted-by":"crossref","first-page":"443","DOI":"10.1007\/978-1-4612-1358-1_29","volume-title":"High Dimensional Probability II","author":"Koltchinskii","year":"2000"},{"key":"ecta200232-bib-0033","author":"Kutyniok","year":"2019","journal-title":"Annals of Statistics"},{"key":"ecta200232-bib-0034","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","volume":"86","author":"LeCun","year":"1998","journal-title":"Proceedings of the IEEE"},{"key":"ecta200232-bib-0035","unstructured":"Liang, T. (2018): \u201cOn How Well Generative Adversarial Networks Learn Densities: Nonparametric and Parametric Results,\u201d arXiv:1811.03179."},{"key":"ecta200232-bib-0036","first-page":"1260","volume-title":"Conference on Learning Theory","author":"Liang","year":"2015"},{"key":"ecta200232-bib-0037","unstructured":"Ma, X., and J. Wang (2018): \u201cRobust Inference Using Inverse Probability Weighting,\u201d arXiv:1810.11397. Preprint."},{"key":"ecta200232-bib-0038","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/3-540-36434-X","volume-title":"Advanced Lectures on Machine Learning","author":"Mendelson","year":"2003"},{"key":"ecta200232-bib-0039","first-page":"25","volume-title":"Conference on Learning Theory","author":"Mendelson","year":"2014"},{"key":"ecta200232-bib-0040","doi-asserted-by":"crossref","first-page":"829","DOI":"10.1142\/S0219530516400042","volume":"14","author":"Mhaskar","year":"2016","journal-title":"Analysis and Applications"},{"key":"ecta200232-bib-0041","first-page":"807","volume-title":"Proceedings of the 27th International Conference on Machine Learning (ICML-10)","author":"Nair","year":"2010"},{"key":"ecta200232-bib-0042","first-page":"930","volume-title":"Advances in Neural Information Processing Systems","author":"Polson","year":"2018"},{"key":"ecta200232-bib-0043","doi-asserted-by":"crossref","first-page":"1305","DOI":"10.1214\/09-EJS479","volume":"3","author":"Robins","year":"2009","journal-title":"Electronic Journal of Statistics"},{"key":"ecta200232-bib-0044","author":"Schmidt-Hieber","year":"2019","journal-title":"Annals of Statistics"},{"key":"ecta200232-bib-0045","first-page":"3076","volume-title":"Proceedings of the 34th International Conference on Machine Learning","author":"Precup","year":"2017"},{"key":"ecta200232-bib-0046","author":"Shamir","year":"2019","journal-title":"Annals of Statistics"},{"key":"ecta200232-bib-0047","doi-asserted-by":"crossref","first-page":"1040","DOI":"10.1214\/aos\/1176345969","volume":"10","author":"Stone","year":"1982","journal-title":"The Annals of Statistics"},{"key":"ecta200232-bib-0048","doi-asserted-by":"crossref","first-page":"811","DOI":"10.1214\/19-AOS1824","volume":"48","author":"Tan","year":"2020","journal-title":"The Annals of Statistics"},{"key":"ecta200232-bib-0049","first-page":"1517","volume-title":"29th Annual Conference on Learning Theory","author":"Telgarsky","year":"2017"},{"key":"ecta200232-bib-0050","doi-asserted-by":"crossref","first-page":"1166","DOI":"10.1214\/14-AOS1221","volume":"42","author":"Geer","year":"2014","journal-title":"The Annals of Statistics"},{"key":"ecta200232-bib-0051","doi-asserted-by":"crossref","first-page":"1228","DOI":"10.1080\/01621459.2017.1319839","volume":"113","author":"Wager","year":"2018","journal-title":"Journal of the American Statistical Association"},{"key":"ecta200232-bib-0052","doi-asserted-by":"crossref","first-page":"826","DOI":"10.1016\/j.jclinepi.2009.11.020","volume":"63","author":"Westreich","year":"2010","journal-title":"Journal of Clinical Epidemiology"},{"key":"ecta200232-bib-0053","volume-title":"Artificial Neural Networks: Approximation and Learning Theory","author":"White","year":"1992"},{"key":"ecta200232-bib-0054","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1016\/j.neunet.2017.07.002","volume":"94","author":"Yarotsky","year":"2017","journal-title":"Neural Networks"},{"key":"ecta200232-bib-0055","first-page":"639","volume-title":"31st Annual Conference on Learning Theory","author":"Yarotsky","year":"2018"},{"key":"ecta200232-bib-0056","unstructured":"Zhang, C., S. Bengio, M. Hardt, B. Recht, and O. Vinyals (2016): \u201cUnderstanding Deep Learning Requires Rethinking Generalization,\u201d arXiv:1611.03530. Preprint."}],"container-title":["Econometrica"],"original-title":[],"language":"en","deposited":{"date-parts":[[2021,1,12]],"date-time":"2021-01-12T03:30:45Z","timestamp":1610422245000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.econometricsociety.org\/doi\/10.3982\/ECTA16901"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021]]},"references-count":56,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2021]]}},"alternative-id":["10.3982\/ECTA16901"],"URL":"https:\/\/doi.org\/10.3982\/ecta16901","relation":{},"ISSN":["0012-9682"],"issn-type":[{"value":"0012-9682","type":"print"}],"subject":[],"published":{"date-parts":[[2021]]}}}