{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,2]],"date-time":"2026-09-02T01:06:50Z","timestamp":1788311210495,"version":"build-2803163510"},"reference-count":71,"publisher":"Oxford University Press (OUP)","license":[{"start":{"date-parts":[[2026,7,29]],"date-time":"2026-07-29T00:00:00Z","timestamp":1785283200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"European Research Council Advanced","award":["101019498"],"award-info":[{"award-number":["101019498"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>In the context of multivariate nonparametric regression with missing covariates, we propose pattern embedded neural networks (PENNs), which can be applied in conjunction with any existing imputation technique. In addition to a neural network trained on the imputed data, PENNs pass the vectors of observation indicators through a second neural network to provide a compact representation. The outputs are then combined in a third neural network to produce final predictions. Our main theoretical result exploits an assumption that the observation patterns can be partitioned into cells on which the Bayes regression function behaves similarly, and belongs to a compositional H\u00f6lder class. It provides a finite-sample excess risk bound that holds for an arbitrary missingness mechanism, and in combination with a complementary minimax lower bound, demonstrates that our PENN estimator attains in typical cases the minimax rate of convergence as if the cells of the partition were known in advance, up to a poly-logarithmic factor in the sample size. Numerical experiments on simulated, semi-synthetic, and real data confirm that the PENN estimator consistently improves, often dramatically, on standard neural networks without pattern embedding. Code to reproduce our experiments, as well as a tutorial on how to apply our method, is publicly available.<\/jats:p>","DOI":"10.1093\/jrsssb\/qkag114","type":"journal-article","created":{"date-parts":[[2026,7,2]],"date-time":"2026-07-02T11:34:36Z","timestamp":1782992076000},"source":"Crossref","is-referenced-by-count":1,"title":["Deep learning with missing data"],"prefix":"10.1093","author":[{"given":"Tianyi","family":"Ma","sequence":"first","affiliation":[{"name":"Statistical Laboratory, University of Cambridge , Wilberforce Road, Cambridge CB3 0WB ,","place":["UK"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tengyao","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Statistics, London School of Economics and Political Science , Houghton Street, London WC2A 2AE ,","place":["UK"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Richard J","family":"Samworth","sequence":"additional","affiliation":[{"name":"Statistical Laboratory, University of Cambridge , Wilberforce Road, Cambridge CB3 0WB ,","place":["UK"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,7,29]]},"reference":[{"key":"2026072913382216700_qkag114-B1","article-title":"On exact computation with an infinitely wide neural net","volume":"32","author":"Arora","year":"2019","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2026072913382216700_qkag114-B2","author":"Bao","year":"2022"},{"issue":"4","key":"2026072913382216700_qkag114-B3","doi-asserted-by":"publisher","first-page":"2261","DOI":"10.1214\/18-AOS1747","article-title":"On deep learning as a remedy for the curse of dimensionality in nonparametric regression","volume":"47","author":"Bauer","year":"2019","journal-title":"Annals of Statistics"},{"key":"2026072913382216700_qkag114-B4","doi-asserted-by":"publisher","first-page":"2170","DOI":"10.1214\/23-AOS2326","article-title":"Optimal nonparametric testing of missing completely at random, and its connections to compatibility","volume":"51","author":"Berrett","year":"2023","journal-title":"Annals of Statistics"},{"issue":"6","key":"2026072913382216700_qkag114-B5","doi-asserted-by":"publisher","first-page":"2738","DOI":"10.1214\/24-AOS2442","article-title":"Deep neural networks for nonparametric interaction models with diverging dimension","volume":"52","author":"Bhattacharya","year":"2024","journal-title":"Annals of Statistics"},{"issue":"1","key":"2026072913382216700_qkag114-B6","doi-asserted-by":"publisher","first-page":"2724","DOI":"10.1214\/22-EJS2011","article-title":"Convergence rates of deep ReLU networks for multiclass classification","volume":"16","author":"Bos","year":"2022","journal-title":"Electronic Journal of Statistics"},{"issue":"2","key":"2026072913382216700_qkag114-B7","doi-asserted-by":"publisher","first-page":"5601","DOI":"10.1214\/24-EJS2332","article-title":"A supervised deep learning method for nonparametric density estimation","volume":"18","author":"Bos","year":"2024","journal-title":"Electronic Journal of Statistics"},{"issue":"1","key":"2026072913382216700_qkag114-B8","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Machine Learning"},{"issue":"4","key":"2026072913382216700_qkag114-B9","doi-asserted-by":"publisher","first-page":"675","DOI":"10.1111\/rssb.12326","article-title":"High dimensional linear discriminant analysis: Optimality, adaptive algorithm and missing data","volume":"81","author":"Cai","year":"2019","journal-title":"Journal of the Royal Statistical Society: Series B, Statistical Methodology"},{"key":"2026072913382216700_qkag114-B10","author":"Chandrasekher","year":"2020"},{"key":"2026072913382216700_qkag114-B11","author":"Chen","year":"2016"},{"key":"2026072913382216700_qkag114-B12","article-title":"Efficient and minimax-optimal in-context nonparametric regression with transformers","author":"Ching","year":"2026","journal-title":"International Conference on Machine Learning"},{"key":"2026072913382216700_qkag114-B13","article-title":"On the global convergence of gradient descent for over-parameterized models using optimal transport","volume":"31","author":"Chizat","year":"2018","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2026072913382216700_qkag114-B14","author":"Devlin","year":"2019"},{"key":"2026072913382216700_qkag114-B15","author":"Dosovitskiy","year":"2021"},{"key":"2026072913382216700_qkag114-B16","author":"Du","year":"2019"},{"issue":"493","key":"2026072913382216700_qkag114-B17","doi-asserted-by":"publisher","first-page":"306","DOI":"10.1198\/jasa.2011.tm09506","article-title":"Nonparametric regression with predictors missing at random","volume":"106","author":"Efromovich","year":"2011","journal-title":"Journal of the American Statistical Association"},{"issue":"1","key":"2026072913382216700_qkag114-B18","doi-asserted-by":"publisher","first-page":"e229","DOI":"10.1002\/sta4.229","article-title":"Sparse spectral estimation with missing and corrupted measurements","volume":"8","author":"Elsener","year":"2019","journal-title":"Stat"},{"issue":"548","key":"2026072913382216700_qkag114-B19","doi-asserted-by":"publisher","first-page":"2680","DOI":"10.1080\/01621459.2023.2271605","article-title":"Factor augmented sparse throughput deep ReLU neural networks for high dimensional regression","volume":"119","author":"Fan","year":"2024","journal-title":"Journal of the American Statistical Association"},{"issue":"2","key":"2026072913382216700_qkag114-B20","doi-asserted-by":"publisher","first-page":"264","DOI":"10.1214\/20-sts783","article-title":"A selective overview of deep learning","volume":"36","author":"Fan","year":"2020","journal-title":"Statistical Science"},{"issue":"3","key":"2026072913382216700_qkag114-B21","doi-asserted-by":"publisher","first-page":"1023","DOI":"10.1111\/rssb.12540","article-title":"High-dimensional changepoint estimation with heterogeneous missingness","volume":"84","author":"Follain","year":"2022","journal-title":"Journal of the Royal Statistical Society: Series B, Statistical Methodology"},{"key":"2026072913382216700_qkag114-B22","author":"Guo","year":"2016"},{"key":"2026072913382216700_qkag114-B23","author":"He","year":"2022"},{"key":"2026072913382216700_qkag114-B24","author":"He","year":"2016"},{"issue":"5786","key":"2026072913382216700_qkag114-B25","doi-asserted-by":"publisher","first-page":"504","DOI":"10.1126\/science.1127647","article-title":"Reducing the dimensionality of data with neural networks","volume":"313","author":"Hinton","year":"2006","journal-title":"Science"},{"issue":"8045","key":"2026072913382216700_qkag114-B26","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1038\/s41586-024-08328-6","article-title":"Accurate predictions on small data with a tabular foundation model","volume":"637","author":"Hollmann","year":"2025","journal-title":"Nature"},{"key":"2026072913382216700_qkag114-B27","author":"Imaizumi","year":"2019"},{"key":"2026072913382216700_qkag114-B28","first-page":"8580","article-title":"Neural tangent kernel: Convergence and generalization in neural networks","volume":"31","author":"Jacot","year":"2018","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"2","key":"2026072913382216700_qkag114-B29","doi-asserted-by":"publisher","first-page":"691","DOI":"10.1214\/23-AOS2266","article-title":"Deep nonparametric regression on approximate manifolds: Nonasymptotic error bounds with polynomial prefactors","volume":"51","author":"Jiao","year":"2023","journal-title":"Annals of Statistics"},{"issue":"9","key":"2026072913382216700_qkag114-B30","doi-asserted-by":"publisher","first-page":"5447","DOI":"10.1007\/s00362-024-01550-4","article-title":"On the consistency of supervised learning with missing values","volume":"65","author":"Josse","year":"2024","journal-title":"Statistical Papers"},{"issue":"7873","key":"2026072913382216700_qkag114-B31","doi-asserted-by":"publisher","first-page":"583","DOI":"10.1038\/s41586-021-03819-2","article-title":"Highly accurate protein structure prediction with AlphaFold","volume":"596","author":"Jumper","year":"2021","journal-title":"Nature"},{"key":"2026072913382216700_qkag114-B32","doi-asserted-by":"publisher","first-page":"106667","DOI":"10.52202\/079017-3387","article-title":"Transformers are minimax optimal nonparametric in-context learners","volume":"37","author":"Kim","year":"2024","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2026072913382216700_qkag114-B33","author":"Kingma","year":"2015"},{"issue":"4","key":"2026072913382216700_qkag114-B34","doi-asserted-by":"publisher","first-page":"2231","DOI":"10.1214\/20-AOS2034","article-title":"On the rate of convergence of fully connected deep neural network regression estimates","volume":"49","author":"Kohler","year":"2021","journal-title":"Annals of Statistics"},{"key":"2026072913382216700_qkag114-B35","first-page":"1106","article-title":"Imagenet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky","year":"2012","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"11","key":"2026072913382216700_qkag114-B36","doi-asserted-by":"publisher","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proceedings of the IEEE"},{"key":"2026072913382216700_qkag114-B37","author":"LeCun","year":"1998"},{"key":"2026072913382216700_qkag114-B38","first-page":"11530","article-title":"What\u2019s a good imputation to predict with missing values?","volume":"34","author":"Le Morvan","year":"2021","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2026072913382216700_qkag114-B39","author":"Li","year":"2019"},{"key":"2026072913382216700_qkag114-B40","author":"Littwin","year":"2023"},{"key":"2026072913382216700_qkag114-B41","author":"Liu","year":"2019"},{"issue":"3","key":"2026072913382216700_qkag114-B42","doi-asserted-by":"publisher","first-page":"1637","DOI":"10.1214\/12-AOS1018","article-title":"High-dimensional regression with noisy and missing data: Provable guarantees with nonconvexity","volume":"40","author":"Loh","year":"2012","journal-title":"Annals of Statistics"},{"key":"2026072913382216700_qkag114-B43","author":"Loshchilov","year":"2019"},{"issue":"5","key":"2026072913382216700_qkag114-B44","doi-asserted-by":"publisher","first-page":"5465","DOI":"10.1137\/20M134695X","article-title":"Deep network approximation for smooth functions","volume":"53","author":"Lu","year":"2021","journal-title":"SIAM Journal on Mathematical Analysis"},{"key":"2026072913382216700_qkag114-B45","article-title":"Estimation beyond missing (completely) at random","author":"Ma","year":"2026+","journal-title":"Annals of Statistics (to Appear)"},{"key":"2026072913382216700_qkag114-B46","author":"Ma","year":"2025"},{"key":"2026072913382216700_qkag114-B47","author":"Ma","year":"2026"},{"key":"2026072913382216700_qkag114-B48","author":"Mattei","year":"2019"},{"issue":"4","key":"2026072913382216700_qkag114-B49","doi-asserted-by":"publisher","first-page":"667","DOI":"10.1002\/cpa.22008","article-title":"The generalization error of random features regression: Precise asymptotics and the double descent curve","volume":"75","author":"Mei","year":"2022","journal-title":"Communications on Pure and Applied Mathematics"},{"key":"2026072913382216700_qkag114-B50","doi-asserted-by":"publisher","first-page":"E7665","DOI":"10.1002\/cpa.22008","article-title":"A mean field view of the landscape of two-layer neural networks","volume":"115","author":"Mei","year":"2018","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"2026072913382216700_qkag114-B51","author":"Mikolov","year":"2013"},{"key":"2026072913382216700_qkag114-B52","doi-asserted-by":"publisher","first-page":"Article 107501","DOI":"10.1016\/j.patcog.2020.107501","article-title":"Handling incomplete heterogeneous data using VAEs","volume":"107","author":"Nazabal","year":"2020","journal-title":"Pattern Recognition"},{"key":"2026072913382216700_qkag114-B53","first-page":"8024","article-title":"Pytorch: An imperative style, high-performance deep learning library","volume":"32","author":"Paszke","year":"2019","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2026072913382216700_qkag114-B54","doi-asserted-by":"publisher","first-page":"2825","DOI":"10.5555\/1953048.2078195","article-title":"Scikit-learn: Machine learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"Journal of Machine Learning Research"},{"key":"2026072913382216700_qkag114-B55","first-page":"55","volume-title":"Early stopping \u2013 but when?","author":"Prechelt","year":"2002"},{"key":"2026072913382216700_qkag114-B56","doi-asserted-by":"publisher","first-page":"1875","DOI":"10.1214\/19-AOS1875","article-title":"Nonparametric regression using deep neural networks with ReLU activation function","volume":"48","author":"Schmidt-Hieber","year":"2020","journal-title":"Annals of Statistics"},{"issue":"3","key":"2026072913382216700_qkag114-B57","doi-asserted-by":"publisher","first-page":"1178","DOI":"10.1214\/24-AOS2389","article-title":"Nonparametric classification with missing data","volume":"52","author":"Sell","year":"2024","journal-title":"Annals of Statistics"},{"key":"2026072913382216700_qkag114-B58","first-page":"2724","article-title":"Processing of missing data by neural networks","volume":"31","author":"\u015amieja","year":"2018","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"1","key":"2026072913382216700_qkag114-B59","doi-asserted-by":"publisher","first-page":"112","DOI":"10.1093\/bioinformatics\/btr597","article-title":"MissForest\u2013non-parametric missing value imputation for mixed-type data","volume":"28","author":"Stekhoven","year":"2012","journal-title":"Bioinformatics"},{"issue":"1","key":"2026072913382216700_qkag114-B60","doi-asserted-by":"publisher","first-page":"177","DOI":"10.1146\/annurev-statistics-040522-013920","article-title":"A survey on statistical theory of deep learning: Approximation, training dynamics, and generative models","volume":"12","author":"Suh","year":"2024","journal-title":"Annual Review of Statistics and Its Application"},{"key":"2026072913382216700_qkag114-B61","doi-asserted-by":"publisher","first-page":"15545","DOI":"10.52202\/075280-0684","article-title":"Convergence of mean-field langevin dynamics: Time-space discretization, stochastic gradient, and variance reduction","volume":"36","author":"Suzuki","year":"2023","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"7","key":"2026072913382216700_qkag114-B62","doi-asserted-by":"publisher","first-page":"950","DOI":"10.1016\/j.patrec.2008.01.010","article-title":"Good methods for coping with missing data in decision trees","volume":"29","author":"Twala","year":"2008","journal-title":"Pattern Recognition Letters"},{"key":"2026072913382216700_qkag114-B63","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18637\/jss.v045.i03","article-title":"mice: Multivariate imputation by chained equations in R","volume":"45","author":"van Buuren","year":"2011","journal-title":"Journal of Statistical Software"},{"key":"2026072913382216700_qkag114-B64","first-page":"5998","article-title":"Attention is all you need","volume":"30","author":"Vaswani","year":"2017","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2026072913382216700_qkag114-B65","author":"Vincent","year":"2008"},{"key":"2026072913382216700_qkag114-B66","author":"Yang","year":"2021"},{"key":"2026072913382216700_qkag114-B67","author":"Zhang","year":"2025"},{"key":"2026072913382216700_qkag114-B68","author":"Zhang","year":"1988"},{"key":"2026072913382216700_qkag114-B69","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1111\/rssb.12550","article-title":"Classification with deep neural networks and logistic loss","volume":"25","author":"Zhang","year":"2024","journal-title":"Journal of Machine Learning Research"},{"issue":"3","key":"2026072913382216700_qkag114-B70","doi-asserted-by":"publisher","first-page":"1348","DOI":"10.1214\/21-AOS2153","article-title":"Deep learning for the partially linear Cox model","volume":"50","author":"Zhong","year":"2022","journal-title":"Annals of Statistics"},{"issue":"5","key":"2026072913382216700_qkag114-B71","doi-asserted-by":"publisher","first-page":"2000","DOI":"10.1111\/rssb.12550","article-title":"High-dimensional principal component analysis with heterogeneous missingness","volume":"84","author":"Zhu","year":"2022","journal-title":"Journal of the Royal Statistical Society: Series B, Statistical Methodology"}],"container-title":["Journal of the Royal Statistical Society Series B: Statistical Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jrsssb\/advance-article-pdf\/doi\/10.1093\/jrsssb\/qkag114\/70362425\/qkag114.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/advance-article-pdf\/doi\/10.1093\/jrsssb\/qkag114\/70362425\/qkag114.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,29]],"date-time":"2026-07-29T17:38:28Z","timestamp":1785346708000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jrsssb\/advance-article\/doi\/10.1093\/jrsssb\/qkag114\/8747034"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,29]]},"references-count":71,"URL":"https:\/\/doi.org\/10.1093\/jrsssb\/qkag114","relation":{},"ISSN":["1369-7412","1467-9868"],"issn-type":[{"value":"1369-7412","type":"print"},{"value":"1467-9868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,29]]},"article-number":"qkag114"}}