{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,10]],"date-time":"2026-09-10T15:22:00Z","timestamp":1789053720209,"version":"build-2803163510"},"reference-count":26,"publisher":"Oxford University Press (OUP)","issue":"2","license":[{"start":{"date-parts":[[2023,2,24]],"date-time":"2023-02-24T00:00:00Z","timestamp":1677196800000},"content-version":"vor","delay-in-days":374,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["5R01 EB001988-16"],"award-info":[{"award-number":["5R01 EB001988-16"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["19 DMS1208164"],"award-info":[{"award-number":["19 DMS1208164"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,4,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We consider the multi-class classification problem when the training data and the out-of-sample test data may have different distributions and propose a method called BCOPS (balanced and conformal optimized prediction sets). BCOPS constructs a prediction set C(x) as a subset of class labels, possibly empty. It tries to optimize the out-of-sample performance, aiming to include the correct class and to detect outliers x as often as possible. BCOPS returns no prediction (corresponding to C(x) equal to the empty set) if it infers x to be an outlier. The proposed method combines supervised learning algorithms with conformal prediction to minimize a misclassification loss averaged over the out-of-sample distribution. The constructed prediction sets have a finite sample coverage guarantee without distributional assumptions. We also propose a method to estimate the outlier detection rate of a given procedure. We prove asymptotic consistency and optimality of our proposals under suitable assumptions and illustrate our methods on real data examples.<\/jats:p>","DOI":"10.1111\/rssb.12443","type":"journal-article","created":{"date-parts":[[2022,2,16]],"date-time":"2022-02-16T06:39:36Z","timestamp":1644993576000},"page":"524-546","source":"Crossref","is-referenced-by-count":29,"title":["Prediction and Outlier Detection in Classification Problems"],"prefix":"10.1093","volume":"84","author":[{"given":"Leying","family":"Guan","sequence":"first","affiliation":[{"name":"Yale University , New Haven, CT , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Robert","family":"Tibshirani","sequence":"additional","affiliation":[{"name":"Stanford University , Stanford, CA , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2022,2,15]]},"reference":[{"key":"2023022408355574800_","volume-title":"Conformal prediction under covariate shift","author":"Barber","year":"2019"},{"key":"2023022408355574800_","first-page":"1823","article-title":"Classification with a reject option using a hinge loss","volume":"9","author":"Bartlett","year":"2008","journal-title":"Journal of Machine Learning Research"},{"key":"2023022408355574800_","doi-asserted-by":"crossref","first-page":"999","DOI":"10.1016\/j.jmva.2005.05.004","article-title":"Kernel estimation of density level sets","volume":"97","author":"Cadre","year":"2006","journal-title":"Journal of multivariate analysis"},{"key":"2023022408355574800_","article-title":"Clustering by estimation of density level sets at a fixed probability","author":"Cadre","year":"2009"},{"key":"2023022408355574800_","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1145\/1541880.1541882","article-title":"Anomaly detection: A survey","volume":"41","author":"Chandola","year":"2009","journal-title":"ACM computing surveys (CSUR)"},{"key":"2023022408355574800_","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1177\/0008068319800106","article-title":"Asymptotically minimal multivariate tolerance sets","volume":"29","author":"Chatterjee","year":"1980","journal-title":"Calcutta Statistical Association Bulletin"},{"key":"2023022408355574800_","article-title":"Clustering algorithms","author":"Hartigan","year":"1975"},{"key":"2023022408355574800_","volume-title":"Cautious deep learning","author":"Hechtlinger","year":"2018"},{"key":"2023022408355574800_","doi-asserted-by":"crossref","first-page":"709","DOI":"10.1002\/cjs.5550340410","article-title":"Classification with reject option","volume":"34","author":"Herbei","year":"2006","journal-title":"Canadian Journal of Statistics"},{"key":"2023022408355574800_","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1023\/B:AIRE.0000045502.10941.a9","article-title":"A survey of outlier detection methodologies","volume":"22","author":"Hodge","year":"2004","journal-title":"Artificial Intelligence Review"},{"key":"2023022408355574800_","article-title":"MNIST handwritten digit database","author":"LeCun","year":"2010"},{"key":"2023022408355574800_","doi-asserted-by":"crossref","first-page":"755","DOI":"10.1093\/biomet\/asu038","article-title":"Classification with confidence","volume":"101","author":"Lei","year":"2014","journal-title":"Biometrika"},{"key":"2023022408355574800_","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1111\/rssb.12021","article-title":"Distribution-free prediction bands for non-parametric regression","volume":"76","author":"Lei","year":"2014","journal-title":"Journal of the Royal Statistical Society: Series B (Statistical Methodology)"},{"key":"2023022408355574800_","doi-asserted-by":"crossref","first-page":"278","DOI":"10.1080\/01621459.2012.751873","article-title":"Distribution-free prediction sets","volume":"108","author":"Lei","year":"2013","journal-title":"Journal of the American Statistical Association"},{"key":"2023022408355574800_","doi-asserted-by":"crossref","first-page":"1299","DOI":"10.1214\/07-AOS505","article-title":"Multivariate spacings based on data depth: I. construction of nonparametric multivariate tolerance regions","volume":"36","author":"Li","year":"2008","journal-title":"The Annals of Statistics"},{"key":"2023022408355574800_","article-title":"Detecting and correcting for label shift with black box predictors","author":"Lipton","year":"2018"},{"key":"2023022408355574800_","first-page":"289","article-title":"Ix. on the problem of the most efficient tests of statistical hypotheses","volume":"231","author":"Neyman","year":"1933","journal-title":"Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character"},{"key":"2023022408355574800_","doi-asserted-by":"crossref","first-page":"1154","DOI":"10.3150\/09-BEJ184","article-title":"Optimal rates for plug-in estimators of density level sets","volume":"15","author":"Rigollet","year":"2009","journal-title":"Bernoulli"},{"key":"2023022408355574800_","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1080\/01621459.2017.1395341","article-title":"Least ambiguous set-valued classifiers with bounded error levels","volume":"114","author":"Sadinle","year":"2019","journal-title":"Journal of the American Statistical Association"},{"key":"2023022408355574800_","first-page":"130","article-title":"Proceedings DARPA Information Survivability Conference and Exposition. DISCEX\u201900","volume":"2","author":"Stolfo","year":"2000","journal-title":"Cost-based modeling for fraud and intrusion detection: Results from the jam project"},{"key":"2023022408355574800_","doi-asserted-by":"crossref","first-page":"614","DOI":"10.1214\/009053607000000929","article-title":"High-dimensional generalized linear models and the lasso","volume":"36","author":"Van de Geer","year":"2008","journal-title":"The Annals of Statistics"},{"key":"2023022408355574800_","volume-title":"Algorithmic learning in a random world","author":"Vovk","year":"2005"},{"key":"2023022408355574800_","doi-asserted-by":"crossref","first-page":"1566","DOI":"10.1214\/08-AOS622","article-title":"On-line predictive linear regression","volume":"37","author":"Vovk","year":"2009","journal-title":"The Annals of Statistics"},{"key":"2023022408355574800_","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1214\/aoms\/1177731491","article-title":"An extension of wilks\u2019 method for setting tolerance limits","volume":"14","author":"Wald","year":"1943","journal-title":"The Annals of Mathematical Statistics"},{"key":"2023022408355574800_","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1214\/aoms\/1177731788","article-title":"Determination of sample sizes for setting tolerance limits","volume":"12","author":"Wilks","year":"1941","journal-title":"The Annals of Mathematical Statistics"},{"key":"2023022408355574800_","first-page":"819","article-title":"Domain adaptation under target and conditional shift","author":"Zhang","year":"2013","journal-title":"International Conference on Machine Learning"}],"container-title":["Journal of the Royal Statistical Society Series B: Statistical Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/rssb.12443","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1111\/rssb.12443","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/84\/2\/524\/49321342\/jrsssb_84_2_524.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/84\/2\/524\/49321342\/jrsssb_84_2_524.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,17]],"date-time":"2023-11-17T15:18:37Z","timestamp":1700234317000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jrsssb\/article\/84\/2\/524\/7056080"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,2,15]]},"references-count":26,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2022,2,15]]},"published-print":{"date-parts":[[2022,4,1]]}},"URL":"https:\/\/doi.org\/10.1111\/rssb.12443","relation":{},"ISSN":["1369-7412","1467-9868"],"issn-type":[{"value":"1369-7412","type":"print"},{"value":"1467-9868","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022,4]]},"published":{"date-parts":[[2022,2,15]]}}}