{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,1]],"date-time":"2026-10-01T10:34:49Z","timestamp":1790850889934,"version":"4.1.0"},"reference-count":46,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2005,5,24]],"date-time":"2005-05-24T00:00:00Z","timestamp":1116892800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2005,6,1]]},"abstract":"<jats:title>Summary<\/jats:title><jats:p>It is shown that bagging, a computationally intensive method, asymptotically improves the performance of nearest neighbour classifiers provided that the resample size is less than 69% of the actual sample size, in the case of with-replacement bagging, or less than 50% of the sample size, for without-replacement bagging. However, for larger sampling fractions there is no asymptotic difference between the risk of the regular nearest neighbour classifier and its bagged version. In particular, neither achieves the large sample performance of the Bayes classifier. In contrast, when the sampling fractions converge to 0, but the resample sizes diverge to \u221e, the bagged classifier converges to the optimal Bayes rule and its risk converges to the risk of the latter. These results are most readily seen when the two populations have well-defined densities, but they may also be derived in other cases, where densities exist in only a relative sense. Cross-validation can be used effectively to choose the sampling fraction. Numerical calculation is used to illustrate these theoretical properties.<\/jats:p>","DOI":"10.1111\/j.1467-9868.2005.00506.x","type":"journal-article","created":{"date-parts":[[2005,5,24]],"date-time":"2005-05-24T23:30:01Z","timestamp":1116977401000},"page":"363-379","source":"Crossref","is-referenced-by-count":38,"title":["Properties of Bagged Nearest Neighbour Classifiers"],"prefix":"10.1093","volume":"67","author":[{"given":"Peter","family":"Hall","sequence":"first","affiliation":[{"name":"Australian National University , Canberra , Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Richard J.","family":"Samworth","sequence":"additional","affiliation":[{"name":"Australian National University , Canberra , Australia"},{"name":"University of Cambridge , UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2005,5,24]]},"reference":[{"key":"2023040705113371000_","first-page":"37","volume-title":"Proc. 15th Int. Conf. Machine Learning","author":"Bay","year":"1998"},{"key":"2023040705113371000_","doi-asserted-by":"crossref","first-page":"191","DOI":"10.3233\/IDA-1999-3304","article-title":"Nearest neighbor classification from multiple feature subsets","volume":"3","author":"Bay","year":"1999","journal-title":"Intell. Data Anal."},{"key":"2023040705113371000_","first-page":"1","article-title":"Resampling fewer than n observations: gains, losses, and remedies for losses","volume":"7","author":"Bickel","year":"1997","journal-title":"Statist. Sin."},{"key":"2023040705113371000_","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1007\/BF00058655","article-title":"Bagging predictors","volume":"24","author":"Breiman","year":"1996","journal-title":"Mach. Learn."},{"key":"2023040705113371000_","volume-title":"Using adaptive bagging to debias regressions","author":"Breiman","year":"1999"},{"key":"2023040705113371000_","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn."},{"key":"2023040705113371000_","doi-asserted-by":"crossref","first-page":"927","DOI":"10.1214\/aos\/1031689014","article-title":"Analyzing bagging","volume":"30","author":"B\u00fchlmann","year":"2002","journal-title":"Ann. Statist."},{"key":"2023040705113371000_","volume-title":"The effect of bagging on variance, bias, and mean squared error","author":"Buja","year":"2000"},{"key":"2023040705113371000_","volume-title":"Smoothing effects of bagging","author":"Buja","year":"2000"},{"key":"2023040705113371000_","first-page":"413","volume-title":"Proc. Hawaii Int. Conf. System Sciences","author":"Cover","year":"1968"},{"key":"2023040705113371000_","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1109\/TIT.1967.1053964","article-title":"Nearest neighbor pattern classification","volume":"13","author":"Cover","year":"1967","journal-title":"IEEE Trans. Inform. Theory"},{"key":"2023040705113371000_","volume-title":"An Introduction to the Theory of Point Processes","author":"Daley","year":"1988"},{"key":"2023040705113371000_","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1109\/TPAMI.1982.4767222","article-title":"Any discrimination rule can have an arbitrarily bad probability of error for finite sample size","volume":"4","author":"Devroye","year":"1982","journal-title":"IEEE Trans. Pattn Anal. Mach. Intell."},{"key":"2023040705113371000_","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4612-0711-5","volume-title":"A Probabilistic Theory of Pattern Recognition","author":"Devroye","year":"1996"},{"key":"2023040705113371000_","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1198\/016214502753479248","article-title":"Comparison of discrimination methods for the classification of tumors using gene expression data","volume":"97","author":"Dudoit","year":"2002","journal-title":"J. Am. Statist. Ass."},{"key":"2023040705113371000_","doi-asserted-by":"crossref","first-page":"316","DOI":"10.1080\/01621459.1983.10477973","article-title":"Estimating the error rate of a prediction rule: improvement on cross-validation","volume":"78","author":"Efron","year":"1983","journal-title":"J. Am. Statist. Ass."},{"key":"2023040705113371000_","first-page":"548","article-title":"Improvements on cross-validation: the .632+ bootstrap method","volume":"92","author":"Efron","year":"1997","journal-title":"J. Am. Statist. Ass."},{"key":"2023040705113371000_","volume-title":"Discriminatory analysis, nonparametric discrimination: consistency properties","author":"Fix","year":"1951"},{"key":"2023040705113371000_","first-page":"111","volume-title":"Information Processing and Management of Uncertainty in Knowledge-based Systems","author":"Francois","year":"2001"},{"key":"2023040705113371000_","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1006\/jcss.1997.1504","article-title":"A decision-theoretic generalization of online learning and an application to boosting","volume":"55","author":"Freund","year":"1997","journal-title":"J. Comput. Syst. Sci."},{"key":"2023040705113371000_","volume-title":"On bagging and nonlinear estimation","author":"Friedman","year":"2000"},{"key":"2023040705113371000_","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1214\/aos\/1016218223","article-title":"Additive logistic regression: a statistical view of boosting","volume":"28","author":"Friedman","year":"2000","journal-title":"Ann. Statist."},{"key":"2023040705113371000_","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1109\/TPAMI.1987.4767875","article-title":"Bias of nearest neighbor error estimates","volume":"9","author":"Fukunaga","year":"1987","journal-title":"IEEE Trans. Pattn Anal. Mach. Intell."},{"key":"2023040705113371000_","first-page":"236","volume-title":"Proc. Genetic and Evolutionary Computation Conf","author":"Guerra-Salcedo","year":"1999"},{"key":"2023040705113371000_","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1016\/0167-7152(89)90002-3","article-title":"A local cross-validation algorithm","volume":"8","author":"Hall","year":"1989","journal-title":"Statist. Probab. Lett."},{"key":"2023040705113371000_","volume-title":"Discrimination and Classification","author":"Hand","year":"1981"},{"key":"2023040705113371000_","first-page":"545","volume-title":"Proc. 14th Int. Conf. Pattern Recognition, Brisbane,","author":"Ho","year":"1998"},{"key":"2023040705113371000_","doi-asserted-by":"crossref","first-page":"832","DOI":"10.1109\/34.709601","article-title":"The random subspace method for constructing decision forests","volume":"20","author":"Ho","year":"1998","journal-title":"IEEE Trans. Pattn Anal. Mach. Intell."},{"key":"2023040705113371000_","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1214\/aos\/1015362184","article-title":"On weak base hypotheses and their implications for boosting regression and classification","volume":"30","author":"Jiang","year":"2002","journal-title":"Ann. Statist."},{"key":"2023040705113371000_","doi-asserted-by":"crossref","first-page":"589","DOI":"10.1198\/016214501753168271","article-title":"Classification trees with unbiased multiway splits","volume":"96","author":"Kim","year":"2001","journal-title":"J. Am. Statist. Ass."},{"key":"2023040705113371000_","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1109\/5326.661099","article-title":"Nearest prototype classification: clustering, genetic algorithms, or random search","volume":"28","author":"Kuncheva","year":"1998","journal-title":"IEEE Trans. Syst. Man Cyber."},{"key":"2023040705113371000_","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1016\/S1566-2535(02)00093-3","article-title":"An experimental study on diversity for bagging and boosting with linear classifiers","author":"Kuncheva","year":"2002","journal-title":"Inform. Fusion"},{"key":"2023040705113371000_","first-page":"289","volume-title":"Proc. 19th A. Int. Conf. Research and Development in Information Retrieval","author":"Larkey","year":"1996"},{"key":"2023040705113371000_","doi-asserted-by":"crossref","first-page":"687","DOI":"10.1214\/aos\/1032894460","article-title":"Consistency of data-driven histogram methods for density estimation and classification","volume":"24","author":"Lugosi","year":"1996","journal-title":"Ann. Statist."},{"key":"2023040705113371000_","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4612-2950-6","article-title":"When does bootstrap work?: asymptotic results and simulations","volume":"77","author":"Mammen","year":"1992","journal-title":"Lect. Notes Statist."},{"key":"2023040705113371000_","doi-asserted-by":"crossref","first-page":"1142","DOI":"10.1214\/aos\/1176346328","article-title":"Optimal rates on convergence to Bayes risk in nonparametric discrimination","volume":"11","author":"Marron","year":"1983","journal-title":"Ann. Statist."},{"key":"2023040705113371000_","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1016\/0378-3758(89)90039-6","article-title":"Local data-driven bandwidth choice for density estimation","volume":"23","author":"Mielniczuk","year":"1989","journal-title":"J. Statist. Planng Inf."},{"key":"2023040705113371000_","first-page":"640","volume-title":"Proc. 4th Wrld Multiconf. Systemics, Cybernetics and Informatics","author":"Mollineda","year":"2000"},{"key":"2023040705113371000_","doi-asserted-by":"crossref","first-page":"820","DOI":"10.1109\/18.335893","article-title":"On the finite sample performance of the nearest neighbour classifier","volume":"40","author":"Psaltis","year":"1994","journal-title":"IEEE Trans. Inform. Theory"},{"key":"2023040705113371000_","first-page":"1651","article-title":"Boosting the margin: a new explanation for the effectiveness of voting methods","volume":"26","author":"Schapire","year":"1998","journal-title":"Ann. Statist."},{"key":"2023040705113371000_","doi-asserted-by":"crossref","first-page":"909","DOI":"10.1016\/S0031-3203(97)00110-6","article-title":"Bagging for linear classifiers","volume":"31","author":"Skurichina","year":"1998","journal-title":"Pattn Recogn"},{"key":"2023040705113371000_","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1007\/3-540-45428-4_6","article-title":"Bagging and boosting for the nearest mean classifier: effects of sample size on diversity and accuracy","volume":"2364","author":"Skurichina","year":"2002","journal-title":"Lect. Notes Comput. Sci."},{"key":"2023040705113371000_","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1023\/A:1008933626919","article-title":"Ideal bootstrap estimation of expected prediction error for k-nearest neighbor classifiers: applications for classification and error assessment","volume":"10","author":"Steele","year":"2000","journal-title":"Statist. Comput."},{"key":"2023040705113371000_","doi-asserted-by":"crossref","first-page":"770","DOI":"10.1080\/01621459.1954.10501233","article-title":"Univariate two-population distribution-free discrimination","volume":"49","author":"Stoller","year":"1954","journal-title":"J. Am. Statist. Ass."},{"key":"2023040705113371000_","doi-asserted-by":"crossref","first-page":"2271","DOI":"10.1109\/18.796368","article-title":"Minimax nonparametric classification\u2014part I: rates of convergence","volume":"45","author":"Yang","year":"1999","journal-title":"IEEE Trans. Inform. Theory"},{"key":"2023040705113371000_","first-page":"1067","volume-title":"Proc. Artificial Neural Networks in Engineering","author":"Zemke","year":"1999"}],"container-title":["Journal of the Royal Statistical Society Series B: Statistical Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1111%2Fj.1467-9868.2005.00506.x","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/67\/3\/363\/49795139\/jrsssb_67_3_363.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/67\/3\/363\/49795139\/jrsssb_67_3_363.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,3]],"date-time":"2023-05-03T07:57:18Z","timestamp":1683100638000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jrsssb\/article\/67\/3\/363\/7109474"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2005,5,24]]},"references-count":46,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2005,5,24]]},"published-print":{"date-parts":[[2005,6,1]]}},"URL":"https:\/\/doi.org\/10.1111\/j.1467-9868.2005.00506.x","relation":{},"ISSN":["1369-7412","1467-9868"],"issn-type":[{"value":"1369-7412","type":"print"},{"value":"1467-9868","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2005,6]]},"published":{"date-parts":[[2005,5,24]]}}}