{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,27]],"date-time":"2026-09-27T06:44:28Z","timestamp":1790491468465,"version":"4.1.0"},"reference-count":55,"publisher":"Oxford University Press (OUP)","issue":"2","license":[{"start":{"date-parts":[[2020,4,22]],"date-time":"2020-04-22T00:00:00Z","timestamp":1587513600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/501100008982","name":"National Science Foundation","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100008982","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000275","name":"Leverhulme Trust","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100000275","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2020,6,1]]},"abstract":"<jats:title>Summary<\/jats:title>\n               <jats:p>We study the effect of imperfect training data labels on the performance of classification methods. In a general setting, where the probability that an observation in the training dataset is mislabelled may depend on both the feature vector and the true label, we bound the excess risk of an arbitrary classifier trained with imperfect labels in terms of its excess risk for predicting a noisy label. This reveals conditions under which a classifier trained with imperfect labels remains consistent for classifying uncorrupted test data points. Furthermore, under stronger conditions, we derive detailed asymptotic properties for the popular $k$-nearest neighbour, support vector machine and linear discriminant analysis classifiers. One consequence of these results is that the $k$-nearest neighbour and support vector machine classifiers are robust to imperfect training labels, in the sense that the rate of convergence of the excess risk of these classifiers remains unchanged; in fact, our theoretical and empirical results even show that in some cases, imperfect labels may improve the performance of these methods. The linear discriminant analysis classifier is shown to be typically inconsistent in the presence of label noise unless the prior probabilities of the classes are equal. Our theoretical results are supported by a simulation study.<\/jats:p>","DOI":"10.1093\/biomet\/asaa011","type":"journal-article","created":{"date-parts":[[2020,2,11]],"date-time":"2020-02-11T12:09:34Z","timestamp":1581422974000},"page":"311-330","source":"Crossref","is-referenced-by-count":32,"title":["Classification with imperfect training labels"],"prefix":"10.1093","volume":"107","author":[{"given":"Timothy I","family":"Cannings","sequence":"first","affiliation":[{"name":"School of Mathematics, University of Edinburgh, James Clerk Maxwell Building, Edinburgh EH9 3FD, U.K"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yingying","family":"Fan","sequence":"additional","affiliation":[{"name":"Department of Data Science and Operations, University of Southern California, 3670 Trousdale Pkwy, Los Angeles, California 90089,U.S.A"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Richard J","family":"Samworth","sequence":"additional","affiliation":[{"name":"Statistical Laboratory, University of Cambridge, Wilberforce Road, Cambridge CB3 0WB, U.K"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2020,4,22]]},"reference":[{"key":"2020071700350454700_B1","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1007\/BF00116829","article-title":"Learning from noisy examples","volume":"2","author":"Angluin,","year":"1988","journal-title":"Mach. Learn."},{"key":"2020071700350454700_B2","doi-asserted-by":"crossref","first-page":"608","DOI":"10.1214\/009053606000001217","article-title":"Fast learning rates for plug-in classifiers","volume":"35","author":"Audibert,","year":"2007","journal-title":"Ann. Statist."},{"key":"2020071700350454700_B3","first-page":"167","article-title":"Efficient learning of linear separators under bounded noise","volume":"4","author":"Awasthi,","year":"2015","journal-title":"Proc. Mach. Learn. Res."},{"key":"2020071700350454700_B4","first-page":"687","article-title":"On the rate of convergence of the bagged nearest neighbor estimate","volume":"11","author":"Biau,","year":"2010","journal-title":"J. Mach. Learn. Res."},{"key":"2020071700350454700_B5","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1214\/009053607000000839","article-title":"Statistical performance of support vector machines","volume":"36","author":"Blanchard,","year":"2008","journal-title":"Ann. Statist."},{"key":"2020071700350454700_B6","doi-asserted-by":"crossref","first-page":"2780","DOI":"10.1214\/16-EJS1193","article-title":"Classification with asymmetric label noise: Consistency and maximal denoising","volume":"10","author":"Blanchard,","year":"2016","journal-title":"Electron. J. Statist."},{"key":"2020071700350454700_B7","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1214\/ss\/1042727940","article-title":"Statistical fraud detection: A review (with Discussion)","volume":"17","author":"Bolton,","year":"2002","journal-title":"Statist. Sci."},{"key":"2020071700350454700_B8","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1016\/j.neucom.2015.12.106","article-title":"A generalised label noise model for classification in the presence of annotation errors","volume":"192","author":"Bootkrajang,","year":"2016","journal-title":"Neurocomp."},{"key":"2020071700350454700_B9","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1007\/978-3-642-33460-3_15","volume-title":"Machine Learning and Knowledge Discovery in Databases","author":"Bootkrajang,","year":"2012"},{"key":"2020071700350454700_B10","doi-asserted-by":"crossref","first-page":"3641","DOI":"10.1016\/j.patcog.2014.05.007","article-title":"Learning kernel logistic regression in the presence of label noise","volume":"47","author":"Bootkrajang,","year":"2014","journal-title":"Pat. Recog."},{"key":"2020071700350454700_B11","first-page":"62","article-title":"Learning probabilistically consistent linear threshold functions","volume-title":"Proc. 10th Annual Conf. Computational Learning Theory (COLT\u201997)","author":"Bylander,","year":"1997"},{"key":"2020071700350454700_B12","author":"Cannings,","year":"2019","journal-title":"Local nearest neighbour classification with applications to semi-supervised learning"},{"key":"2020071700350454700_B13","first-page":"1","article-title":"Theoretical analysis of cross-validation for estimating the risk of the $k$-nearest neighbor classifier","volume":"19","author":"Celisse,","year":"2018","journal-title":"J. Mach. Learn. Res."},{"key":"2020071700350454700_B14","first-page":"3437","article-title":"Rates of convergence for nearest neighbor classification","volume-title":"Proc. 27th Int. Conf. Neural Information Processing Systems (NIPS\u201914)","author":"Chaudhuri,","year":"2014"},{"key":"2020071700350454700_B15","first-page":"1","article-title":"Statistical decision making for optimal budget allocation in crowd labeling","volume":"16","author":"Chen,","year":"2015","journal-title":"J. Mach. Learn. Res."},{"key":"2020071700350454700_B16","author":"Cheng,","year":"2019","journal-title":"Learning with bounded instance- and label-dependent label noise"},{"key":"2020071700350454700_B17","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1007\/BF00994018","article-title":"Support vector networks","volume":"20","author":"Cortes,","year":"1995","journal-title":"Mach. Learn."},{"key":"2020071700350454700_B18","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4612-0711-5","volume-title":"A Probabilistic Theory of Pattern Recognition","author":"Devroye,","year":"1996"},{"key":"2020071700350454700_B19","volume-title":"Discriminatory analysis \u2013 nonparametric discrimination: Consistency properties","author":"Fix,","year":"1951"},{"key":"2020071700350454700_B20","doi-asserted-by":"crossref","first-page":"238","DOI":"10.2307\/1403797","article-title":"Discriminatory analysis \u2013 nonparametric discrimination: Consistency properties","volume":"57","author":"Fix,","year":"1989","journal-title":"Int. Statist. Rev."},{"key":"2020071700350454700_B21","first-page":"667","article-title":"A comprehensive introduction to label noise","volume-title":"Proc. Euro. Symp. Artificial Neural Networks (ESANN 2014)","author":"Fr\u00e9nay,","year":"2014"},{"key":"2020071700350454700_B22","doi-asserted-by":"crossref","first-page":"845","DOI":"10.1109\/TNNLS.2013.2292894","article-title":"Classification in the presence of label noise: A survey","volume":"25","author":"Fr\u00e9nay,","year":"2014","journal-title":"IEEE Trans. Neural Networks Learn. Syst."},{"key":"2020071700350454700_B23","doi-asserted-by":"crossref","first-page":"982","DOI":"10.1214\/15-AOS1395","article-title":"Classification with the nearest neighbour rule in general finite dimensional spaces","volume":"44","author":"Gadat,","year":"2016","journal-title":"Ann. Statist."},{"key":"2020071700350454700_B24","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1016\/j.neucom.2014.09.081","article-title":"Making risk minimization tolerant to label noise","volume":"160","author":"Ghosh,","year":"2015","journal-title":"Neurocomp."},{"key":"2020071700350454700_B25","doi-asserted-by":"crossref","first-page":"2135","DOI":"10.1214\/07-AOS537","article-title":"Choice of neighbour order in nearest-neighbour classification","volume":"36","author":"Hall,","year":"2008","journal-title":"Ann. Statist."},{"key":"2020071700350454700_B26","author":"Inouye,","year":"2017","journal-title":"Hyperparameter selection under localized label noise via corrupt validation"},{"key":"2020071700350454700_B27","doi-asserted-by":"crossref","first-page":"1028","DOI":"10.1109\/18.391248","article-title":"Rates of convergence of nearest neighbor estimation under arbitrary sampling","volume":"41","author":"Kulkarni,","year":"1995","journal-title":"IEEE Trans. Info. Theory"},{"key":"2020071700350454700_B28","doi-asserted-by":"crossref","first-page":"657","DOI":"10.2307\/1266637","article-title":"Discriminant analysis when the initial samples are misclassified","volume":"8","author":"Lachenbruch,","year":"1966","journal-title":"Technometrics"},{"key":"2020071700350454700_B29","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1080\/00401706.1974.10489211","article-title":"Discriminant analysis when the initial samples are misclassified II: Non-random misclassification models","volume":"16","author":"Lachenbruch,","year":"1974","journal-title":"Technometrics"},{"key":"2020071700350454700_B30","doi-asserted-by":"crossref","first-page":"1192","DOI":"10.1109\/SURV.2012.110112.00192","article-title":"A survey on human activity recognition using wearable sensors","volume":"15","author":"Lara,","year":"2013","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"2020071700350454700_B31","first-page":"1910","article-title":"Learning from noisy labels with distillation","volume-title":"2017 IEEE Int. Conf. Computer Vision (ICCV)","author":"Li,","year":"2017"},{"key":"2020071700350454700_B32","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1023\/A:1015469627679","article-title":"Support vector machines and the Bayes rule in classification","volume":"6","author":"Lin,","year":"2002","journal-title":"Data Mining Know. Disc."},{"key":"2020071700350454700_B33","doi-asserted-by":"crossref","first-page":"447","DOI":"10.1109\/TPAMI.2015.2456899","article-title":"Classification with noisy labels by importance reweighting","volume":"38","author":"Liu,","year":"2016","journal-title":"IEEE Trans. Pat. Anal. Mach. Intel."},{"key":"2020071700350454700_B34","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1007\/s10994-009-5165-z","article-title":"Random classification noise defeats all convex potential boosters","volume":"78","author":"Long,","year":"2010","journal-title":"Mach. Learn."},{"key":"2020071700350454700_B35","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1016\/0031-3203(92)90008-7","article-title":"Learning with an unreliable teacher","volume":"25","author":"Lugosi,","year":"1992","journal-title":"Pat. Recog."},{"key":"2020071700350454700_B36","doi-asserted-by":"crossref","first-page":"1146","DOI":"10.1109\/TSMCB.2012.2223460","article-title":"Noise tolerance under risk minimization","volume":"43","author":"Manwani,","year":"2013","journal-title":"IEEE Trans. Cybernetics"},{"key":"2020071700350454700_B37","author":"Menon,","year":"2016","journal-title":"Learning from binary labels with instance-dependent corruption"},{"key":"2020071700350454700_B38","first-page":"1196","article-title":"Learning with noisy labels","volume-title":"Advances in Neural Information Processing Systems 26 (NIPS 2013)","author":"Natarajan,","year":"2013"},{"key":"2020071700350454700_B39","author":"Northcutt,","year":"2017","journal-title":"Learning with confident examples: Rank pruning for robust classification with noisy labels"},{"key":"2020071700350454700_B40","first-page":"238","article-title":"An average-case analysis of the $k$-nearest neighbor classifier for noisy domains","volume-title":"Proc. 15th Int. Joint Conf. Artif. Intel. (IJCAI\u201997)","author":"Okamoto,","year":"1997"},{"key":"2020071700350454700_B41","first-page":"708","article-title":"Loss factorization, weakly supervised learning and label noise robustness","volume-title":"Proc. 33rd Int. Conf. Machine Learning (ICML\u201916)","author":"Patrini,","year":"2016"},{"key":"2020071700350454700_B42","first-page":"1944","article-title":"Making deep neural networks robust to label noise: A loss correction approach","volume-title":"Proc. IEEE Conf. Computer Vision and Pattern Recognition","author":"Patrini,","year":"2017"},{"key":"2020071700350454700_B43","author":"Rolnick,","year":"2018","journal-title":"Deep learning is robust to massive label noise"},{"key":"2020071700350454700_B44","doi-asserted-by":"crossref","first-page":"2733","DOI":"10.1214\/12-AOS1049","article-title":"Optimal weighted nearest neighbour classifiers","volume":"40","author":"Samworth,","year":"2012","journal-title":"Ann. Statist."},{"key":"2020071700350454700_B45","first-page":"1","article-title":"Classification with asymmetric label noise: Consistency and maximal denoising","volume":"30","author":"Scott,","year":"2013","journal-title":"J. Mach. Learn. Res. W&CP"},{"key":"2020071700350454700_B46","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1109\/TIT.2004.839514","article-title":"Consistency of support vector machines and other regularized kernel classifiers","volume":"51","author":"Steinwart,","year":"2005","journal-title":"IEEE Trans. Info. Theory"},{"key":"2020071700350454700_B47","volume-title":"Support Vector Machines","author":"Steinwart,","year":"2008"},{"key":"2020071700350454700_B48","doi-asserted-by":"crossref","first-page":"575","DOI":"10.1214\/009053606000001226","article-title":"Fast rates for support vector machines using Gaussian kernels","volume":"35","author":"Steinwart,","year":"2007","journal-title":"Ann. Statist."},{"key":"2020071700350454700_B49","doi-asserted-by":"crossref","first-page":"884","DOI":"10.1007\/978-3-642-04274-4_91","article-title":"Learning SVMs from sloppily labeled data","volume-title":"Proc. 19th Int. Conf. Artificial Neural Networks (ICANN 2009)","author":"Stempfel,","year":"2009"},{"key":"2020071700350454700_B50","doi-asserted-by":"crossref","first-page":"595","DOI":"10.1214\/aos\/1176343886","article-title":"Consistent nonparametric regression","volume":"5","author":"Stone,","year":"1977","journal-title":"Ann. Statist."},{"key":"2020071700350454700_B51","first-page":"10","article-title":"Learning with symmetric label noise: The importance of being unhinged","volume-title":"Proc. 28th Int. Conf. Neural Information Processing Systems (NIPS\u201915)","author":"van Rooyen,","year":"2015"},{"key":"2020071700350454700_B52","doi-asserted-by":"crossref","first-page":"408","DOI":"10.1109\/TSMC.1972.4309137","article-title":"Asymptotic properties of nearest neighbour rules using edited data","volume":"2","author":"Wilson,","year":"1972","journal-title":"IEEE Trans. Syst. Man. Cybernetics"},{"key":"2020071700350454700_B53","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1023\/A:1007626913721","article-title":"Reduction techniques for instance based learning algorithms","volume":"38","author":"Wilson,","year":"2000","journal-title":"Mach. Learn."},{"key":"2020071700350454700_B54","doi-asserted-by":"crossref","first-page":"1305","DOI":"10.1016\/S0140-6736(14)61705-0","article-title":"Genetic diagnosis of developmental disorders in the DDD study: A scalable analysis of genome-wide research data","volume":"385","author":"Wright,","year":"2015","journal-title":"Lancet"},{"key":"2020071700350454700_B55","first-page":"1","article-title":"Spectral methods meet EM: A provably optimal algorithm for crowdsourcing","volume":"17","author":"Zhang,","year":"2016","journal-title":"J. Mach. Learn. Res."}],"container-title":["Biometrika"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/biomet\/article-pdf\/107\/2\/311\/33218140\/asaa011.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/biomet\/article-pdf\/107\/2\/311\/33218140\/asaa011.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,7,17]],"date-time":"2020-07-17T14:59:40Z","timestamp":1594997980000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/biomet\/article\/107\/2\/311\/5823574"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,4,22]]},"references-count":55,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2020,4,22]]},"published-print":{"date-parts":[[2020,6,1]]}},"URL":"https:\/\/doi.org\/10.1093\/biomet\/asaa011","relation":{},"ISSN":["0006-3444","1464-3510"],"issn-type":[{"value":"0006-3444","type":"print"},{"value":"1464-3510","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2020,6]]},"published":{"date-parts":[[2020,4,22]]}}}