{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,8,23]],"date-time":"2024-08-23T02:40:13Z","timestamp":1724380813770},"reference-count":16,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2021,1,27]],"date-time":"2021-01-27T00:00:00Z","timestamp":1611705600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,1,27]],"date-time":"2021-01-27T00:00:00Z","timestamp":1611705600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Stat Comput"],"published-print":{"date-parts":[[2021,3]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>To overcome the inherent limitations of axis-aligned base learners in ensemble learning, several methods of rotating the feature space have been discussed in the literature. In particular, smoother decision boundaries can often be obtained from axis-aligned ensembles by rotating the feature space. In the present paper, we introduce a low-cost regularization technique that favors rotations which produce compact base learners. The restated problem adds a shrinkage term to the loss function that explicitly accounts for the complexity of the base learners. For example, for tree-based ensembles, we apply a penalty based on the median number of nodes and the median depth of the trees in the forest. Rather than jointly minimizing prediction error and model complexity, which is computationally infeasible, we first generate a prioritized weighting of the available feature rotations that promotes lower model complexity and subsequently minimize prediction errors on each of the selected rotations. We show that the resulting ensembles tend to be significantly more dense, faster to evaluate, and competitive at generalizing in out-of-sample predictions.<\/jats:p>","DOI":"10.1007\/s11222-020-09973-3","type":"journal-article","created":{"date-parts":[[2021,1,27]],"date-time":"2021-01-27T07:02:36Z","timestamp":1611730956000},"update-policy":"http:\/\/dx.doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Regularizing axis-aligned ensembles via data rotations that favor simpler learners"],"prefix":"10.1007","volume":"31","author":[{"given":"Rico","family":"Blaser","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Piotr","family":"Fryzlewicz","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,1,27]]},"reference":[{"key":"9973_CR1","doi-asserted-by":"publisher","first-page":"625","DOI":"10.1137\/0908055","volume":"8","author":"T Anderson","year":"1987","unstructured":"Anderson, T., Olkin, I., Underhill, L.: Generation of random orthogonal matrices. SIAM J. Sci. Stat. Comput. 8, 625\u2013629 (1987)","journal-title":"SIAM J. Sci. Stat. Comput."},{"key":"9973_CR2","first-page":"1","volume":"17","author":"R Blaser","year":"2016","unstructured":"Blaser, R., Fryzlewicz, P.: Random rotation ensembles. J. Mach. Learn. Res. 17, 1\u201326 (2016)","journal-title":"J. Mach. Learn. Res."},{"key":"9973_CR3","unstructured":"Breiman, L. Random forests random features. Technical report (1999)"},{"key":"9973_CR4","volume-title":"Classification and Regression Trees","author":"L Breiman","year":"1984","unstructured":"Breiman, L., Friedman, J., Stone, C., Olshen, R.: Classification and Regression Trees. Wadsworth, Belmont (1984)"},{"key":"9973_CR5","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1111\/rssb.12228","volume":"79","author":"T Cannings","year":"2017","unstructured":"Cannings, T., Samworth, R.: Random-projection ensemble classification. J. R. Stat. Soc. B 79, 1\u201338 (2017)","journal-title":"J. R. Stat. Soc. B"},{"key":"9973_CR6","unstructured":"Dheeru, D., Taniskidou, E.: UCI machine learning repository (2017). http:\/\/archive.ics.uci.edu\/ml"},{"key":"9973_CR7","unstructured":"Durrant, R., Kaban, A.: Random projections as regularizers: learning a linear discriminant ensemble from fewer observations than dimensions. JMLR: Workshop and Conference Proceedings, vol. 29, pp. 17\u201332 (2013)"},{"key":"9973_CR8","doi-asserted-by":"publisher","first-page":"1189","DOI":"10.1214\/aos\/1013203451","volume":"5","author":"J Friedman","year":"2001","unstructured":"Friedman, J.: Greedy function approximation: a gradient boosting machine. Ann. Stat. 5, 1189\u20131232 (2001)","journal-title":"Ann. Stat."},{"key":"9973_CR9","unstructured":"Goodrich, M., Mirelli, V., Orletsky, M., Salowe, J.: Decision tree construction in fixed dimensions: being global is hard but local greed is good. Technical Report TR95-1 (1995)"},{"key":"9973_CR10","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-84858-7","volume-title":"The Elements of Statistical Learning: Data Mining, Inference, and Prediction","author":"T Hastie","year":"2009","unstructured":"Hastie, T., Friedman, J., Tibshirani, R.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, New York (2009)"},{"key":"9973_CR11","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1016\/0020-0190(76)90095-8","volume":"5","author":"L Hyafil","year":"1976","unstructured":"Hyafil, L., Rivest, R.: Constructing optimal binary decision trees is NP-complete. Inf. Process. Lett. 5, 15\u201317 (1976)","journal-title":"Inf. Process. Lett."},{"key":"9973_CR12","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4614-7138-7","volume-title":"An Introduction to Statistical Learning","author":"G James","year":"2013","unstructured":"James, G., Witten, D., Hastie, T.: An Introduction to Statistical Learning. Springer, New York (2013)"},{"key":"9973_CR13","doi-asserted-by":"publisher","first-page":"1619","DOI":"10.1109\/TPAMI.2006.211","volume":"28","author":"J Rodriguez","year":"2006","unstructured":"Rodriguez, J., Kuncheva, L., Alonso, C.: Rotation forest: a new classifier ensemble method. IEEE Trans. Pattern Anal. Mach. Intell. 28, 1619\u20131630 (2006)","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"9973_CR14","doi-asserted-by":"publisher","first-page":"2970","DOI":"10.1109\/TIP.2012.2230011","volume":"22","author":"G Takacs","year":"2013","unstructured":"Takacs, G., Chandrasekhar, V., Tsai, S.: Fast computation of rotation-invariant image features by approximate radial gradient transform. IEEE Trans. Image Process. 22, 2970\u20132982 (2013)","journal-title":"IEEE Trans. Image Process."},{"key":"9973_CR15","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","volume":"58","author":"R Tibshirani","year":"1996","unstructured":"Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. B 58, 267\u201388 (1996)","journal-title":"J. R. Stat. Soc. B"},{"key":"9973_CR16","doi-asserted-by":"crossref","unstructured":"Tomita, T., Maggioni, M., Vogelstein, J.: Roflmao: robust oblique forests with linear matrix operations. In: Proceedings of the 2017 SIAM International Conference on Data Mining, vol. 1, pp. 498\u2013506 (2017)","DOI":"10.1137\/1.9781611974973.56"}],"container-title":["Statistics and Computing"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s11222-020-09973-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s11222-020-09973-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s11222-020-09973-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,23]],"date-time":"2024-08-23T01:30:44Z","timestamp":1724376644000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s11222-020-09973-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,1,27]]},"references-count":16,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2021,3]]}},"alternative-id":["9973"],"URL":"https:\/\/doi.org\/10.1007\/s11222-020-09973-3","relation":{},"ISSN":["0960-3174","1573-1375"],"issn-type":[{"type":"print","value":"0960-3174"},{"type":"electronic","value":"1573-1375"}],"subject":[],"published":{"date-parts":[[2021,1,27]]},"assertion":[{"value":"25 January 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 December 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 January 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"15"}}