{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T16:40:09Z","timestamp":1787330409153,"version":"3.56.0"},"reference-count":64,"publisher":"Society for Industrial & Applied Mathematics (SIAM)","issue":"4","funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1613218"],"award-info":[{"award-number":["1613218"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1915786"],"award-info":[{"award-number":["1915786"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1811405"],"award-info":[{"award-number":["1811405"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1811661"],"award-info":[{"award-number":["1811661"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1916125"],"award-info":[{"award-number":["1916125"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["SIAM Journal on Mathematics of Data Science"],"published-print":{"date-parts":[[2020,1]]},"abstract":"<jats:p>When randomized ensemble methods such as bagging and random forests are implemented, a basic question arises: Is the ensemble large enough? In particular, the practitioner desires a rigorous guarantee that a given ensemble will perform nearly as well as an ideal infinite ensemble (trained on the same data). The purpose of the current paper is to develop a bootstrap method for solving this problem in the context of regression---which complements our companion paper in the context of classification [Lopes, Ann. Statist., 47 (2019), 1088--1112]. In contrast to the classification setting, the current paper shows that theoretical guarantees for the proposed bootstrap can be established under much weaker assumptions. In addition, we illustrate the flexibility of the method by showing how it can be adapted to measure algorithmic convergence for variable selection. Lastly, we provide numerical results demonstrating that the method works well in a range of situations.<\/jats:p>","DOI":"10.1137\/20m1343300","type":"journal-article","created":{"date-parts":[[2020,10,7]],"date-time":"2020-10-07T13:16:07Z","timestamp":1602076567000},"page":"921-943","source":"Crossref","is-referenced-by-count":4,"title":["Measuring the Algorithmic Convergence of Randomized Ensembles: The Regression Setting"],"prefix":"10.1137","volume":"2","author":[{"given":"Miles E.","family":"Lopes","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Suofei","family":"Wu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thomas C. M.","family":"Lee","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"351","published-online":{"date-parts":[[2020,10,7]]},"reference":[{"key":"atypb1","unstructured":"S. Arlot and R. Genuer,\n                      Analysis of Purely Random Forests Bias\n                      , preprint,https:\/\/arxiv.org\/abs\/1407.3939, 2014."},{"key":"atypb2","first-page":"41","author":"Basilico J.","year":"2011","journal-title":"Washington, DC"},{"key":"atypb3","doi-asserted-by":"crossref","first-page":"149","DOI":"10.2307\/3318585","volume":"3","author":"Bertail P.","year":"1997","journal-title":"Bernoulli"},{"key":"atypb4","doi-asserted-by":"crossref","first-page":"667","DOI":"10.2307\/3316014","volume":"29","author":"Bertail P.","year":"2001","journal-title":"Canad. J. Statist."},{"key":"atypb5","first-page":"1063","volume":"13","author":"Biau G.","year":"2012","journal-title":"J. Mach. Learn. Res."},{"key":"atypb6","first-page":"2015","volume":"9","author":"Biau G.","year":"2008","journal-title":"J. Mach. Learn. Res."},{"key":"atypb7","first-page":"640","volume":"64","author":"Bickel P. J.","year":"2002","journal-title":"Sankhy\u0101 Ser. A"},{"key":"atypb8","doi-asserted-by":"crossref","first-page":"387","DOI":"10.1080\/01621459.1988.10478609","volume":"83","author":"Bickel P. J.","year":"1988","journal-title":"J. Amer. Statist. Assoc."},{"key":"atypb9","first-page":"126","volume":"17","author":"Blaser R.","year":"2016","journal-title":"J. Mach. Learn. Res."},{"key":"atypb10","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1023\/A:1018054314350","volume":"24","author":"Breiman L.","year":"1996","journal-title":"Mach. Learn."},{"key":"atypb11","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","volume":"45","author":"Breiman L.","year":"2001","journal-title":"Mach. Learn."},{"key":"atypb12","unstructured":"L. Breiman, J. Friedman, C. J. Stone, and R. A. Olshen,\n                      Classification and Regression Trees\n                      , CRC, Boca Raton, FL, 1984."},{"key":"atypb13","unstructured":"C. Brezinski and M. R. Zaglia,\n                      Extrapolation Methods: Theory and Practice\n                      , Elsevier, New York, 2013."},{"key":"atypb14","first-page":"927","volume":"30","author":"B\u00fchlmann P.","year":"2002","journal-title":"Ann. Statist."},{"key":"atypb15","doi-asserted-by":"crossref","first-page":"959","DOI":"10.1111\/rssb.12228","volume":"79","author":"Cannings T. I.","year":"2017","journal-title":"J. R. Stat. Soc. Ser. B Stat. Methodol."},{"key":"atypb16","unstructured":"V. Chernozhukov, D. Chetverikov, K. Kato, and Y. Koike,\n                      Improved Central Limit Theorem and Bootstrap Approximations in High Dimensions\n                      , preprint,https:\/\/arxiv.org\/abs\/1912.10529, 2019."},{"key":"atypb17","doi-asserted-by":"crossref","first-page":"266","DOI":"10.1214\/09-AOAS285","volume":"4","author":"Chipman H. A.","year":"2010","journal-title":"Ann. Appl. Stat."},{"key":"atypb18","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1186\/1471-2105-7-3","volume":"7","author":"D\u00edaz-Uriarte R.","year":"2006","journal-title":"BMC Bioinformatics"},{"key":"atypb19","unstructured":"D. Dua and C. Graff,\n                      UCI Machine Learning Repository\n                      ,http:\/\/archive.ics.uci.edu\/ml, 2017."},{"key":"atypb20","doi-asserted-by":"crossref","first-page":"987","DOI":"10.1111\/sjos.12378","volume":"46","author":"Fang Y.","year":"2019","journal-title":"Scand. J. Stat."},{"key":"atypb21","first-page":"3053","volume":"19","author":"Fang Y.","year":"2018","journal-title":"J. Mach. Learn. Res."},{"key":"atypb22","unstructured":"J. Friedman, T. Hastie, and R. Tibshirani,\n                      The Elements of Statistical Learning\n                      , Springer, New York, 2001."},{"key":"atypb23","doi-asserted-by":"crossref","first-page":"2225","DOI":"10.1016\/j.patrec.2010.03.014","volume":"31","author":"Genuer R.","year":"2010","journal-title":"Pattern Recognition Lett."},{"key":"atypb24","doi-asserted-by":"crossref","first-page":"19","DOI":"10.32614\/RJ-2015-018","volume":"7","author":"Genuer R.","year":"2015","journal-title":"The R Journal"},{"key":"atypb25","unstructured":"A. G\u00e9ron,\n                      Hands-on Machine Learning with Scikit-Learn and TensorFlow\n                      , O'Reilly Media,https:\/\/github.com\/ageron\/handson-ml\/tree\/master\/datasets, 2017."},{"key":"atypb26","unstructured":"R. L. Graham, D. E. Knuth, O. Patashnik, and S. Liu,\n                      Concrete Mathematics: A Foundation for Computer Science\n                      , Addison & Wesley, Reading, MA, 1994."},{"key":"atypb27","doi-asserted-by":"crossref","first-page":"659","DOI":"10.1007\/s11222-016-9646-1","volume":"27","author":"Gregorutti B.","year":"2017","journal-title":"Stat. Comput."},{"key":"atypb28","doi-asserted-by":"crossref","first-page":"363","DOI":"10.1111\/j.1467-9868.2005.00506.x","volume":"67","author":"Hall P.","year":"2005","journal-title":"J. R. Stat. Soc. Ser. B Stat. Methodol."},{"key":"atypb29","doi-asserted-by":"crossref","first-page":"1323","DOI":"10.1016\/j.patcog.2012.10.021","volume":"46","author":"Hern\u00e1ndez-Lobato D.","year":"2013","journal-title":"Pattern Recognition"},{"key":"atypb30","doi-asserted-by":"crossref","first-page":"519","DOI":"10.1214\/07-EJS039","volume":"1","author":"Ishwaran H.","year":"2007","journal-title":"Electron. J. Stat."},{"key":"atypb31","first-page":"178","author":"Latinne P.","year":"2001","journal-title":"Berlin"},{"key":"atypb32","first-page":"16619","author":"Li T.","year":"2018","journal-title":"AAAI"},{"key":"atypb33","first-page":"18","volume":"2","author":"Liaw A.","year":"2002","journal-title":"R News"},{"key":"atypb34","doi-asserted-by":"crossref","first-page":"578","DOI":"10.1198\/016214505000001230","volume":"101","author":"Lin Y.","year":"2006","journal-title":"J. Amer. Statist. Assoc."},{"key":"atypb35","unstructured":"M. E. Lopes,\n                      Estimating a Sharp Convergence Bound for Randomized Ensembles\n                      , preprint,https:\/\/arxiv.org\/abs\/1303.0727, 2016."},{"key":"atypb36","doi-asserted-by":"crossref","first-page":"1088","DOI":"10.1214\/18-AOS1707","volume":"47","author":"Lopes M. E.","year":"2019","journal-title":"Ann. Statist."},{"key":"atypb37","first-page":"2978","author":"Lopes M. E.","year":"2020","journal-title":"Austria"},{"key":"atypb38","doi-asserted-by":"crossref","first-page":"1214","DOI":"10.1214\/19-AOS1844","volume":"48","author":"Lopes M. E.","year":"2020","journal-title":"Ann. Statist."},{"key":"atypb39","first-page":"3223","author":"Lopes M. E.","year":"2018","journal-title":"Sweden"},{"key":"atypb40","first-page":"1","volume":"20","author":"Lopes M. E.","year":"2019","journal-title":"J. Mach. Learn. Res."},{"key":"atypb41","unstructured":"G. Louppe, L. Wehenkel, A. Sutera, and P. Geurts,\n                      Understanding variable importances in forests of randomized trees\n                      , in Advances in Neural Information Processing Systems, NeurIPS, San Diego, CA, 2013, pp. 431-439."},{"key":"atypb42","doi-asserted-by":"crossref","unstructured":"P.G. Martinsson and J. Tropp,\n                      Randomized numerical linear algebra: Foundations & algorithms\n                      , Acta Numer., to appear; preprint,https:\/\/arxiv.org\/abs\/2002.01387, 2020.","DOI":"10.1017\/S0962492920000021"},{"key":"atypb43","first-page":"1","volume":"17","author":"Mentch L.","year":"2016","journal-title":"J. Mach. Learn. Res."},{"key":"atypb44","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1002\/prot.23200","volume":"79","author":"Moult J.","year":"2011","journal-title":"Proteins Struct. Funct. Bioinform."},{"key":"atypb45","doi-asserted-by":"crossref","first-page":"3711","DOI":"10.1093\/bioinformatics\/bty373","volume":"34","author":"Nembrini S.","year":"2018","journal-title":"Bioinformatics"},{"key":"atypb46","first-page":"377","author":"Ng A. Y.","year":"2001","journal-title":"MA"},{"key":"atypb47","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1186\/1471-2105-11-110","volume":"11","author":"Nicodemus K. K.","year":"2010","journal-title":"BMC Bioinformatics"},{"key":"atypb48","first-page":"154","author":"Oshiro T. M.","year":"2012","journal-title":"New York"},{"key":"atypb49","first-page":"1","volume":"18","author":"Probst P.","year":"2018","journal-title":"J. Mach. Learn. Res."},{"key":"atypb50","first-page":"2733","volume":"40","author":"Samworth R. J.","year":"2012","journal-title":"Ann. Statist."},{"key":"atypb51","doi-asserted-by":"crossref","first-page":"611","DOI":"10.1198\/106186008X344522","volume":"17","author":"Sandri M.","year":"2008","journal-title":"J. Comput. Graph. Statist."},{"key":"atypb52","first-page":"1377","author":"Schwing A.","year":"2011","journal-title":", in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Washington, DC"},{"key":"atypb53","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1016\/j.jmva.2015.06.009","volume":"146","author":"Scornet E.","year":"2016","journal-title":"J. Multivariate Anal."},{"key":"atypb54","doi-asserted-by":"crossref","first-page":"1485","DOI":"10.1109\/TIT.2016.2514489","volume":"62","author":"Scornet E.","year":"2016","journal-title":"IEEE Trans. Inform. Theory"},{"key":"atypb55","doi-asserted-by":"crossref","first-page":"1716","DOI":"10.1214\/15-AOS1321","volume":"43","author":"Scornet E.","year":"2015","journal-title":"Ann. Statist."},{"key":"atypb56","doi-asserted-by":"crossref","first-page":"801","DOI":"10.1016\/j.csda.2008.08.007","volume":"53","author":"Sexton J.","year":"2009","journal-title":"Comput. Statist. Data Anal."},{"key":"atypb57","doi-asserted-by":"crossref","unstructured":"A. Sidi,\n                      Practical Extrapolation Methods: Theory and Applications\n                      , Cambridge University Press, Cambridge, UK, 2003.","DOI":"10.1017\/CBO9780511546815"},{"key":"atypb58","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1186\/1471-2105-9-307","volume":"9","author":"Strobl C.","year":"2008","journal-title":"BMC Bioinformatics"},{"key":"atypb59","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1186\/1471-2105-8-25","volume":"8","author":"Strobl C.","year":"2007","journal-title":"BMC Bioinformatics"},{"key":"atypb60","unstructured":"W. J. Su and Y. Zhu,\n                      Uncertainty Quantification for Online Learning and Stochastic Approximation via Hierarchical Incremental Gradient Descent\n                      , preprint,https:\/\/arxiv.org\/abs\/1802.04876, 2018."},{"key":"atypb61","doi-asserted-by":"crossref","first-page":"1546","DOI":"10.1214\/aop\/1176991174","volume":"17","author":"Talagrand M.","year":"1989","journal-title":"Ann. Probab."},{"key":"atypb62","first-page":"1625","volume":"15","author":"Wager S.","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"atypb63","doi-asserted-by":"crossref","unstructured":"H. Wickham,\n                      ggplot2: Elegant Graphics for Data Analysis\n                      , 2nd ed., Springer, New York, 2016.","DOI":"10.1007\/978-3-319-24277-4_9"},{"key":"atypb64","first-page":"1115","author":"Zhou F.","year":"2014","journal-title":"Washington, DC"}],"container-title":["SIAM Journal on Mathematics of Data Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/epubs.siam.org\/doi\/pdf\/10.1137\/20M1343300","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T16:03:23Z","timestamp":1787328203000},"score":1,"resource":{"primary":{"URL":"https:\/\/epubs.siam.org\/doi\/10.1137\/20M1343300"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,1]]},"references-count":64,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2020,1]]}},"alternative-id":["10.1137\/20M1343300"],"URL":"https:\/\/doi.org\/10.1137\/20m1343300","relation":{},"ISSN":["2577-0187"],"issn-type":[{"value":"2577-0187","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,1]]}}}