{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,30]],"date-time":"2026-10-30T11:46:49Z","timestamp":1793360809286,"version":"4.5.0"},"reference-count":84,"publisher":"Oxford University Press (OUP)","issue":"5","license":[{"start":{"date-parts":[[2023,7,17]],"date-time":"2023-07-17T00:00:00Z","timestamp":1689552000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"DOI":"10.13039\/100000121","name":"DMS","doi-asserted-by":"publisher","award":["2046874"],"award-info":[{"award-number":["2046874"]}],"id":[{"id":"10.13039\/100000121","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"NIH","doi-asserted-by":"publisher","award":["grants R01AI27271, R01CA222147, R01AG065276, R01GM139926"],"award-info":[{"award-number":["grants R01AI27271, R01CA222147, R01AG065276, R01GM139926"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,2,2]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Predicting sets of outcomes\u2014instead of unique outcomes\u2014is a promising solution to uncertainty quantification in statistical learning. Despite a rich literature on constructing prediction sets with statistical guarantees, adapting to unknown covariate shift\u2014a prevalent issue in practice\u2014poses a serious unsolved challenge. In this article, we show that prediction sets with finite-sample coverage guarantee are uninformative and propose a novel flexible distribution-free method, PredSet-1Step, to efficiently construct prediction sets with an asymptotic coverage guarantee under unknown covariate shift. We formally show that our method is asymptotically probably approximately correct, having well-calibrated coverage error with high confidence for large samples. We illustrate that it achieves nominal coverage in a number of experiments and a data set concerning HIV risk prediction in a South African cohort study. Our theory hinges on a new bound for the convergence rate of the coverage of Wald confidence intervals based on general asymptotically linear estimators.<\/jats:p>","DOI":"10.1093\/jrsssb\/qkad069","type":"journal-article","created":{"date-parts":[[2023,6,28]],"date-time":"2023-06-28T15:42:22Z","timestamp":1687966942000},"page":"1680-1705","source":"Crossref","is-referenced-by-count":12,"title":["Prediction sets adaptive to unknown covariate shift"],"prefix":"10.1093","volume":"85","author":[{"given":"Hongxiang","family":"Qiu","sequence":"first","affiliation":[{"name":"Department of Statistics, The Wharton School, University of Pennsylvania , Philadelphia, PA 19104 , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Edgar","family":"Dobriban","sequence":"additional","affiliation":[{"name":"Department of Statistics, The Wharton School, University of Pennsylvania , Philadelphia, PA 19104 , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Eric","family":"Tchetgen Tchetgen","sequence":"additional","affiliation":[{"name":"Department of Statistics, The Wharton School, University of Pennsylvania , Philadelphia, PA 19104 , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2023,7,17]]},"reference":[{"key":"2024071701001358900_qkad069-B1","author":"Angelopoulos","year":"2021"},{"issue":"4","key":"2024071701001358900_qkad069-B2","doi-asserted-by":"crossref","first-page":"962","DOI":"10.1111\/j.1541-0420.2005.00377.x","article-title":"Doubly robust estimation in missing data and causal inference models","volume":"61","author":"Bang","year":"2005","journal-title":"Biometrics"},{"key":"2024071701001358900_qkad069-B3","author":"Bates","year":"2021"},{"key":"2024071701001358900_qkad069-B4","first-page":"909","article-title":"Safe model-based reinforcement learning with stability guarantees","volume":"30","author":"Berkenkamp","year":"2017","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"3","key":"2024071701001358900_qkad069-B5","doi-asserted-by":"crossref","first-page":"647","DOI":"10.1214\/aos\/1176345863","article-title":"On adaptive estimation","volume":"10","author":"Bickel","year":"1982","journal-title":"The Annals of Statistics"},{"key":"2024071701001358900_qkad069-B6","volume-title":"Mathematical statistics: Basic ideas and selected topics","author":"Bickel","year":"2015","edition":"2nd ed."},{"key":"2024071701001358900_qkad069-B7","volume-title":"Efficient and adaptive estimation for semiparametric models","author":"Bickel","year":"1993"},{"issue":"4","key":"2024071701001358900_qkad069-B8","doi-asserted-by":"crossref","first-page":"1033","DOI":"10.1214\/aos\/1059655904","article-title":"Nonparametric estimators which can be \u201cplugged-in\u201d","volume":"31","author":"Bickel","year":"2003","journal-title":"Annals of Statistics"},{"key":"2024071701001358900_qkad069-B9","article-title":"Discriminative learning for differing training and test distributions","author":"Bickel","year":"2007","journal-title":"ACM International Conference Proceeding Series"},{"key":"2024071701001358900_qkad069-B10","author":"Bojarski","year":"2016"},{"key":"2024071701001358900_qkad069-B11","author":"Cauchois","year":"2020"},{"key":"2024071701001358900_qkad069-B12","doi-asserted-by":"crossref","first-page":"5549","DOI":"10.1016\/S1573-4412(07)06076-X","article-title":"Chapter 76: Large sample sieve estimation of semi-nonparametric models","volume":"6","author":"Chen","year":"2007","journal-title":"Handbook of Econometrics"},{"issue":"1","key":"2024071701001358900_qkad069-B13","doi-asserted-by":"crossref","first-page":"C1","DOI":"10.1111\/ectj.12097","article-title":"Double\/debiased machine learning for treatment and structural parameters","volume":"21","author":"Chernozhukov","year":"2018","journal-title":"Econometrics Journal"},{"key":"2024071701001358900_qkad069-B14","author":"Chernozhukov","year":"2018"},{"key":"2024071701001358900_qkad069-B15","author":"Dunn","year":"2018"},{"key":"2024071701001358900_qkad069-B16","author":"Friedman","year":"2001"},{"issue":"4","key":"2024071701001358900_qkad069-B17","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1016\/S0167-9473(01)00065-2","article-title":"Stochastic gradient boosting","volume":"38","author":"Friedman","year":"2002","journal-title":"Computational Statistics and Data Analysis"},{"key":"2024071701001358900_qkad069-B18","author":"Friedman","year":"2004"},{"key":"2024071701001358900_qkad069-B19","author":"Gal","year":"2017"},{"issue":"11","key":"2024071701001358900_qkad069-B20","doi-asserted-by":"crossref","first-page":"1265","DOI":"10.1503\/cmaj.110977","article-title":"Missing covariate data in clinical research: When and when not to use the missing-indicator method for analysis","volume":"184","author":"Groenwold","year":"2012","journal-title":"Canadian Medical Association Journal"},{"issue":"2","key":"2024071701001358900_qkad069-B21","doi-asserted-by":"crossref","first-page":"315","DOI":"10.2307\/2998560","article-title":"On the role of the propensity score in efficient semiparametric estimation of average treatment effects","volume":"66","author":"Hahn","year":"1998","journal-title":"Econometrica"},{"issue":"3","key":"2024071701001358900_qkad069-B22","doi-asserted-by":"crossref","first-page":"1124","DOI":"10.1214\/aoms\/1177704476","article-title":"Asymptotically most powerful rank-order tests","volume":"33","author":"Hajek","year":"1962","journal-title":"The Annals of Mathematical Statistics"},{"key":"2024071701001358900_qkad069-B23","volume-title":"The bootstrap and Edgeworth expansion","author":"Hall","year":"2013"},{"key":"2024071701001358900_qkad069-B24","author":"Han","year":"2019"},{"issue":"1","key":"2024071701001358900_qkad069-B25","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1214\/aos\/1176324456","article-title":"Penalized discriminant analysis","volume":"23","author":"Hastie","year":"1995","journal-title":"The Annals of Statistics"},{"key":"2024071701001358900_qkad069-B26","volume-title":"Generalized additive models","author":"Hastie","year":"1990"},{"key":"2024071701001358900_qkad069-B27","author":"Hendrycks","year":"2019"},{"key":"2024071701001358900_qkad069-B28","author":"Kennedy","year":"2022"},{"key":"2024071701001358900_qkad069-B29","author":"Kitani","year":"2012"},{"key":"2024071701001358900_qkad069-B30","volume-title":"Th\u00e9orie asymptotique de la d\u00e9cision statistique","author":"Le Cam","year":"1969"},{"issue":"523","key":"2024071701001358900_qkad069-B31","doi-asserted-by":"crossref","first-page":"1094","DOI":"10.1080\/01621459.2017.1307116","article-title":"Distribution-free predictive inference for regression","volume":"113","author":"Lei","year":"2018","journal-title":"Journal of the American Statistical Association"},{"issue":"1-2","key":"2024071701001358900_qkad069-B32","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1007\/s10472-013-9366-6","article-title":"A conformal prediction approach to explore functional data","volume":"74","author":"Lei","year":"2015","journal-title":"Annals of Mathematics and Artificial Intelligence"},{"issue":"501","key":"2024071701001358900_qkad069-B33","doi-asserted-by":"crossref","first-page":"278","DOI":"10.1080\/01621459.2012.751873","article-title":"Distribution-free prediction sets","volume":"108","author":"Lei","year":"2013","journal-title":"Journal of the American Statistical Association"},{"issue":"1","key":"2024071701001358900_qkad069-B34","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1111\/rssb.12021","article-title":"Distribution-free prediction bands for non-parametric regression","volume":"76","author":"Lei","year":"2014","journal-title":"Journal of the Royal Statistical Society. Series B: Statistical Methodology"},{"issue":"5","key":"2024071701001358900_qkad069-B35","doi-asserted-by":"crossref","first-page":"911","DOI":"10.1111\/rssb.12445","article-title":"Conformal inference of counterfactuals and individual treatment effects","volume":"83","author":"Lei","year":"2021","journal-title":"Journal of the Royal Statistical Society. Series B: Statistical Methodology"},{"key":"2024071701001358900_qkad069-B36","author":"Levit","year":"1974"},{"key":"2024071701001358900_qkad069-B37","volume-title":"Statistical analysis with missing data","author":"Little","year":"2019"},{"key":"2024071701001358900_qkad069-B38","author":"Malik","year":"2019"},{"key":"2024071701001358900_qkad069-B39","author":"Mason","year":"1999"},{"key":"2024071701001358900_qkad069-B40","author":"Mason","year":"2000"},{"key":"2024071701001358900_qkad069-B41","author":"Menon","year":"2016"},{"issue":"12","key":"2024071701001358900_qkad069-B42","doi-asserted-by":"crossref","first-page":"e12","DOI":"10.2105\/AJPH.2014.302164","article-title":"Effectiveness of computerized decision support systems linked to electronic health records: A systematic review and meta-analysis","volume":"104","author":"Moja","year":"2014","journal-title":"American Journal of Public Health"},{"issue":"2","key":"2024071701001358900_qkad069-B43","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1002\/jae.3950050202","article-title":"Semiparametric efficiency bounds","volume":"5","author":"Newey","year":"1990","journal-title":"Journal of Applied Econometrics"},{"key":"2024071701001358900_qkad069-B44","author":"Newey","year":"1998"},{"issue":"3","key":"2024071701001358900_qkad069-B45","doi-asserted-by":"crossref","first-page":"947","DOI":"10.1111\/j.1468-0262.2004.00518.x","article-title":"Twicing kernels and a small bias property of semiparametric estimators","volume":"72","author":"Newey","year":"2004","journal-title":"Econometrica"},{"key":"2024071701001358900_qkad069-B46","author":"Papadopoulos","year":"2002"},{"key":"2024071701001358900_qkad069-B47","author":"Park","year":"2022"},{"key":"2024071701001358900_qkad069-B48","author":"Park","year":"2020"},{"key":"2024071701001358900_qkad069-B49","volume-title":"Contributions to a general asymptotic statistical theory","author":"Pfanzagl","year":"1985"},{"key":"2024071701001358900_qkad069-B50","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4612-3396-1","volume-title":"Estimation in semiparametric models","author":"Pfanzagl","year":"1990"},{"issue":"4","key":"2024071701001358900_qkad069-B51","doi-asserted-by":"crossref","first-page":"2300","DOI":"10.3150\/20-BEJ1309","article-title":"Universal sieve-based strategies for efficient estimation using machine learning tools","volume":"27","author":"Qiu","year":"2021","journal-title":"Bernoulli"},{"key":"2024071701001358900_qkad069-B52","volume-title":"Dataset shift in machine learning","author":"Qui\u00f1onero-Candela","year":"2009"},{"issue":"6","key":"2024071701001358900_qkad069-B53","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.1109\/TPAMI.2016.2577031","article-title":"Faster R-CNN: Towards real-time object detection with region proposal networks","volume":"39","author":"Ren","year":"2017","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"issue":"1","key":"2024071701001358900_qkad069-B55","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1093\/biomet\/70.1.41","article-title":"The central role of the propensity score in observational studies for causal effects","volume":"70","author":"Rosenbaum","year":"1983","journal-title":"Biometrika"},{"issue":"1","key":"2024071701001358900_qkad069-B56","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1093\/biomet\/asaa054","article-title":"Characterization of parameters with a mixed bias property","volume":"108","author":"Rotnitzky","year":"2021","journal-title":"Biometrika"},{"issue":"525","key":"2024071701001358900_qkad069-B57","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1080\/01621459.2017.1395341","article-title":"Least ambiguous set-valued classifiers with bounded error levels","volume":"114","author":"Sadinle","year":"2019","journal-title":"Journal of the American Statistical Association"},{"key":"2024071701001358900_qkad069-B58","author":"Saunders","year":"1999"},{"issue":"448","key":"2024071701001358900_qkad069-B59","doi-asserted-by":"crossref","first-page":"1096","DOI":"10.1080\/01621459.1999.10473862","article-title":"Adjusting for nonignorable drop-out using semiparametric nonresponse models","volume":"94","author":"Scharfstein","year":"1999","journal-title":"Journal of the American Statistical Association"},{"issue":"2","key":"2024071701001358900_qkad069-B60","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1214\/aoms\/1177731119","article-title":"Non-parametric estimation. I. Validation of order statistics","volume":"16","author":"Scheffe","year":"1945","journal-title":"The Annals of Mathematical Statistics"},{"issue":"3","key":"2024071701001358900_qkad069-B61","doi-asserted-by":"crossref","first-page":"1139","DOI":"10.1214\/aos\/1176350055","article-title":"On asymptotically efficient estimation in semiparametric models","volume":"14","author":"Schick","year":"1986","journal-title":"The Annals of Statistics"},{"issue":"3","key":"2024071701001358900_qkad069-B62","doi-asserted-by":"crossref","first-page":"1514","DOI":"10.1214\/19-AOS1857","article-title":"The hardness of conditional independence testing and the generalised covariance measure","volume":"48","author":"Shah","year":"2020","journal-title":"The Annals of Statistics"},{"issue":"6","key":"2024071701001358900_qkad069-B63","doi-asserted-by":"crossref","first-page":"2555","DOI":"10.1214\/aos\/1030741085","article-title":"On methods of sieves and penalization","volume":"25","author":"Shen","year":"1997","journal-title":"The Annals of Statistics"},{"issue":"2","key":"2024071701001358900_qkad069-B64","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1016\/S0378-3758(00)00115-4","article-title":"Improving predictive inference under covariate shift by weighting the log-likelihood function","volume":"90","author":"Shimodaira","year":"2000","journal-title":"Journal of Statistical Planning and Inference"},{"key":"2024071701001358900_qkad069-B65","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/9780262017091.001.0001","volume-title":"Machine learning in non-stationary environments: Introduction to covariate shift adaptation","author":"Sugiyama","year":"2012"},{"issue":"4","key":"2024071701001358900_qkad069-B66","doi-asserted-by":"crossref","first-page":"699","DOI":"10.1007\/s10463-008-0197-x","article-title":"Direct importance estimation for covariate shift adaptation","volume":"60","author":"Sugiyama","year":"2008","journal-title":"Annals of the Institute of Statistical Mathematics"},{"key":"2024071701001358900_qkad069-B67","author":"Szegedy","year":"2014"},{"issue":"6122","key":"2024071701001358900_qkad069-B68","doi-asserted-by":"crossref","first-page":"966","DOI":"10.1126\/science.1228160","article-title":"High coverage of ART associated with decline in risk of HIV acquisition in rural KwaZulu-Natal, South Africa","volume":"339","author":"Tanser","year":"2013","journal-title":"Science"},{"issue":"1","key":"2024071701001358900_qkad069-B69","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","article-title":"Regression shrinkage and selection via the lasso","volume":"58","author":"Tibshirani","year":"1996","journal-title":"Journal of the Royal Statistical Society: Series B (Methodological)"},{"key":"2024071701001358900_qkad069-B70","article-title":"Conformal prediction under covariate shift","author":"Tibshirani","year":"2019","journal-title":"Advances in Neural Information Processing Systems 32 (NIPS 2019)"},{"issue":"4","key":"2024071701001358900_qkad069-B71","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1214\/aoms\/1177730343","article-title":"Non-parametric estimation II. Statistically equivalent blocks and tolerance regions\u2013the continuous case","volume":"18","author":"Tukey","year":"1947","journal-title":"The Annals of Mathematical Statistics"},{"issue":"1","key":"2024071701001358900_qkad069-B72","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1214\/aoms\/1177730287","article-title":"Nonparametric estimation, III. Statistically equivalent blocks and multivariate tolerance regions\u2013the discontinuous case","volume":"19","author":"Tukey","year":"1948","journal-title":"The Annals of Mathematical Statistics"},{"issue":"1","key":"2024071701001358900_qkad069-B73","doi-asserted-by":"crossref","DOI":"10.2202\/1544-6115.1309","article-title":"Super learner","volume":"6","author":"van der Laan","year":"2007","journal-title":"Statistical Applications in Genetics and Molecular Biology"},{"key":"2024071701001358900_qkad069-B74","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-65304-4","volume-title":"Targeted learning in data science: Causal inference for complex longitudinal studies","author":"Van der Laan","year":"2018"},{"issue":"1","key":"2024071701001358900_qkad069-B75","doi-asserted-by":"crossref","DOI":"10.2202\/1557-4679.1043","article-title":"Targeted maximum likelihood learning","volume":"2","author":"Van der Laan","year":"2006","journal-title":"The International Journal of Biostatistics"},{"key":"2024071701001358900_qkad069-B76","first-page":"178","article-title":"On differentiable functionals","volume":"19","author":"Van Der Vaart","year":"1991","journal-title":"The Annals of Statistics"},{"key":"2024071701001358900_qkad069-B77","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511802256","volume-title":"Asymptotic statistics","author":"van der Vaart","year":"1998"},{"key":"2024071701001358900_qkad069-B78","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4757-2545-2","volume-title":"Weak convergence and empirical processes: With applications to statistics","author":"van der Vaart","year":"1996"},{"key":"2024071701001358900_qkad069-B79","author":"Vovk","year":"2013"},{"key":"2024071701001358900_qkad069-B80","author":"Vovk","year":"1999"},{"key":"2024071701001358900_qkad069-B81","volume-title":"Algorithmic learning in a random world","author":"Vovk","year":"2005"},{"issue":"1","key":"2024071701001358900_qkad069-B82","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1214\/aoms\/1177731491","article-title":"An extension of Wilks\u2019 method for setting tolerance limits","volume":"14","author":"Wald","year":"1943","journal-title":"The Annals of Mathematical Statistics"},{"issue":"1","key":"2024071701001358900_qkad069-B83","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1214\/aoms\/1177731788","article-title":"Determination of sample sizes for setting tolerance limits","volume":"12","author":"Wilks","year":"1941","journal-title":"The Annals of Mathematical Statistics"},{"key":"2024071701001358900_qkad069-B84","author":"Yang","year":"2022"},{"issue":"11","key":"2024071701001358900_qkad069-B85","doi-asserted-by":"crossref","first-page":"3447","DOI":"10.1016\/j.jspi.2011.05.001","article-title":"Berry-Esseen type bounds in heteroscedastic semi-parametric model","volume":"141","author":"Zhang","year":"2011","journal-title":"Journal of Statistical Planning and Inference"}],"container-title":["Journal of the Royal Statistical Society Series B: Statistical Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/85\/5\/1680\/56555182\/qkad069.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/85\/5\/1680\/56555182\/qkad069.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,16]],"date-time":"2024-07-16T21:00:48Z","timestamp":1721163648000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jrsssb\/article\/85\/5\/1680\/7225361"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,17]]},"references-count":84,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2023,7,17]]},"published-print":{"date-parts":[[2024,2,2]]}},"URL":"https:\/\/doi.org\/10.1093\/jrsssb\/qkad069","relation":{},"ISSN":["1369-7412","1467-9868"],"issn-type":[{"value":"1369-7412","type":"print"},{"value":"1467-9868","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,11]]},"published":{"date-parts":[[2023,7,17]]}}}