{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,14]],"date-time":"2026-10-14T04:13:00Z","timestamp":1791951180077,"version":"4.3.4"},"reference-count":27,"publisher":"Oxford University Press (OUP)","issue":"4","license":[{"start":{"date-parts":[[2023,3,13]],"date-time":"2023-03-13T00:00:00Z","timestamp":1678665600000},"content-version":"vor","delay-in-days":3283,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/100006754","name":"US Army Research Laboratory","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100006754","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000183","name":"US Army Research Office","doi-asserted-by":"publisher","award":["W911NF-11-1-0391"],"award-info":[{"award-number":["W911NF-11-1-0391"]}],"id":[{"id":"10.13039\/100000183","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1122732"],"award-info":[{"award-number":["1122732"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2014,9,1]]},"abstract":"<jats:title>Summary<\/jats:title><jats:p>The bootstrap provides a simple and powerful means of assessing the quality of estimators. However, in settings involving large data sets\u2014which are increasingly prevalent\u2014the calculation of bootstrap-based quantities can be prohibitively demanding computationally. Although variants such as subsampling and the m out of n bootstrap can be used in principle to reduce the cost of bootstrap computations, these methods are generally not robust to specification of tuning parameters (such as the number of subsampled data points), and they often require knowledge of the estimator's convergence rate, in contrast with the bootstrap. As an alternative, we introduce the \u2018bag of little bootstraps\u2019 (BLB), which is a new procedure which incorporates features of both the bootstrap and subsampling to yield a robust, computationally efficient means of assessing the quality of estimators. The BLB is well suited to modern parallel and distributed computing architectures and furthermore retains the generic applicability and statistical efficiency of the bootstrap. We demonstrate the BLB's favourable statistical performance via a theoretical analysis elucidating the procedure's properties, as well as a simulation study comparing the BLB with the bootstrap, the m out of n bootstrap and subsampling. In addition, we present results from a large-scale distributed implementation of the BLB demonstrating its computational superiority on massive data, a method for adaptively selecting the BLB's tuning parameters, an empirical study applying the BLB to several real data sets and an extension of the BLB to time series data.<\/jats:p>","DOI":"10.1111\/rssb.12050","type":"journal-article","created":{"date-parts":[[2014,3,17]],"date-time":"2014-03-17T11:35:50Z","timestamp":1395056150000},"page":"795-816","source":"Crossref","is-referenced-by-count":292,"title":["A Scalable Bootstrap for Massive Data"],"prefix":"10.1093","volume":"76","author":[{"given":"Ariel","family":"Kleiner","sequence":"first","affiliation":[{"name":"University of California , Berkeley , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ameet","family":"Talwalkar","sequence":"additional","affiliation":[{"name":"University of California , Berkeley , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Purnamrita","family":"Sarkar","sequence":"additional","affiliation":[{"name":"University of California , Berkeley , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael I.","family":"Jordan","sequence":"additional","affiliation":[{"name":"University of California , Berkeley , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2014,3,17]]},"reference":[{"key":"2023031400063468700_","doi-asserted-by":"crossref","first-page":"1196","DOI":"10.1214\/aos\/1176345637","article-title":"Some asymptotic theory for the bootstrap","volume":"9","author":"Bickel","year":"1981","journal-title":"Ann. Statist."},{"key":"2023031400063468700_","first-page":"1","article-title":"Resampling fewer than n observations: gains, losses, and remedies for losses","volume":"7","author":"Bickel","year":"1997","journal-title":"Statist. Sin."},{"key":"2023031400063468700_","first-page":"640","article-title":"Extrapolation and the bootstrap","volume":"64","author":"Bickel","year":"2002","journal-title":"Sankhya A"},{"key":"2023031400063468700_","first-page":"967","article-title":"On the choice of m in the m out of n bootstrap and confidence bounds for extrema","volume":"18","author":"Bickel","year":"2008","journal-title":"Statist. Sin."},{"key":"2023031400063468700_","doi-asserted-by":"crossref","first-page":"387","DOI":"10.1080\/01621459.1988.10478609","article-title":"Richardson extrapolation and the bootstrap","volume":"83","author":"Bickel","year":"1988","journal-title":"J. Am. Statist. Ass."},{"key":"2023031400063468700_","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1038\/scientificamerican0583-116","article-title":"Computer-intensive methods in statistics","volume":"248","author":"Diaconis","year":"1983","journal-title":"Scient. Am."},{"key":"2023031400063468700_","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1214\/aos\/1176344552","article-title":"Bootstrap methods: another look at the jackknife","volume":"7","author":"Efron","year":"1979","journal-title":"Ann. Statist."},{"key":"2023031400063468700_","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1080\/01621459.1990.10475309","article-title":"More efficient bootstrap computations","volume":"85","author":"Efron","year":"1988","journal-title":"J. Am. Statist. Ass."},{"key":"2023031400063468700_","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4899-4541-9","volume-title":"An Introduction to the Bootstrap","author":"Efron","year":"1993"},{"key":"2023031400063468700_","volume-title":"UCI machine learning repository","author":"Frank","year":"2010"},{"key":"2023031400063468700_","doi-asserted-by":"crossref","first-page":"851","DOI":"10.1214\/aop\/1176990862","article-title":"Bootstrapping general empirical measures","volume":"18","author":"Gin\u00e9","year":"1990","journal-title":"Ann. Probab."},{"key":"2023031400063468700_","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1017\/S0266466600009051","article-title":"Bootstrapping quantile regression estimators","volume":"11","author":"Hahn","year":"1995","journal-title":"Econmetr. Theor."},{"key":"2023031400063468700_","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4612-4384-7","volume-title":"The Bootstrap and Edgeworth Expansion","author":"Hall","year":"1992"},{"key":"2023031400063468700_","doi-asserted-by":"crossref","first-page":"2011","DOI":"10.1214\/aos\/1176325769","article-title":"On general resampling algorithms and their performance in distribution estimation","volume":"22","author":"Hall","year":"1994","journal-title":"Ann. Statist."},{"key":"2023031400063468700_","doi-asserted-by":"crossref","first-page":"1217","DOI":"10.1214\/aos\/1176347265","article-title":"The jacknife and the bootstrap for general stationary observations","volume":"17","author":"Kunsch","year":"1989","journal-title":"Ann. Statist."},{"key":"2023031400063468700_","doi-asserted-by":"crossref","first-page":"1552","DOI":"10.1214\/12-AOAS587","article-title":"Gap bootstrap methods for massive data sets with an application to transportation engineering","volume":"6","author":"Lahiri","year":"2012","journal-title":"Ann. Appl. Statist."},{"key":"2023031400063468700_","first-page":"225","volume-title":"Exploring the Limits of the Bootstrap","author":"Liu","year":"1992"},{"key":"2023031400063468700_","volume-title":"Numerical Optimization","author":"Nocedal","year":"2006"},{"key":"2023031400063468700_","doi-asserted-by":"crossref","first-page":"1303","DOI":"10.1080\/01621459.1994.10476870","article-title":"The stationary bootstrap","volume":"89","author":"Politis","year":"1994","journal-title":"J. Am. Statist. Ass."},{"key":"2023031400063468700_","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4612-1554-7","article-title":"Subsampling","author":"Politis","year":"1999"},{"key":"2023031400063468700_","doi-asserted-by":"crossref","first-page":"2297","DOI":"10.1214\/aos\/1032181156","article-title":"Resampling: consistency of substitution estimators","volume":"24","author":"Putter","year":"1996","journal-title":"Ann. Statist."},{"key":"2023031400063468700_","doi-asserted-by":"crossref","first-page":"985","DOI":"10.1093\/biomet\/90.4.985","article-title":"A note on methods of restoring consistency to the bootstrap","volume":"90","author":"Samworth","year":"2003","journal-title":"Biometrika"},{"key":"2023031400063468700_","doi-asserted-by":"crossref","DOI":"10.1007\/b97553","volume-title":"Mathematical Statistics","author":"Shao","year":"2003"},{"key":"2023031400063468700_","volume-title":"How many bootstraps? Technical Report","author":"Tibshirani","year":"1985"},{"key":"2023031400063468700_","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511802256","volume-title":"Asymptotic Statistics","author":"van der Vaart","year":"1998"},{"key":"2023031400063468700_","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4757-2545-2","volume-title":"Weak Convergence and Empirical Processes","author":"van der Vaart","year":"1996"},{"key":"2023031400063468700_","volume-title":"Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing","author":"Zaharia","year":"2012"}],"container-title":["Journal of the Royal Statistical Society Series B: Statistical Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1111%2Frssb.12050","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/76\/4\/795\/49514215\/jrsssb_76_4_795.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/76\/4\/795\/49514215\/jrsssb_76_4_795.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,7,11]],"date-time":"2023-07-11T06:48:22Z","timestamp":1689058102000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jrsssb\/article\/76\/4\/795\/7076037"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,3,17]]},"references-count":27,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2014,3,17]]},"published-print":{"date-parts":[[2014,9,1]]}},"URL":"https:\/\/doi.org\/10.1111\/rssb.12050","relation":{},"ISSN":["1369-7412","1467-9868"],"issn-type":[{"value":"1369-7412","type":"print"},{"value":"1467-9868","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2014,9]]},"published":{"date-parts":[[2014,3,17]]}}}