{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,29]],"date-time":"2026-09-29T18:01:03Z","timestamp":1790704863917,"version":"4.1.0"},"reference-count":46,"publisher":"Public Library of Science (PLoS)","issue":"9","license":[{"start":{"date-parts":[[2026,9,29]],"date-time":"2026-09-29T00:00:00Z","timestamp":1790640000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS One"],"abstract":"<jats:p>Given the continuous increase in dataset sizes and the complexity of forecasting models, the trade-off between forecast accuracy and computational cost is emerging as an extremely relevant topic, especially in the context of ensemble learning for time series forecasting. To assess it, we evaluated ten base models and eight ensemble configurations across two large-scale retail datasets, considering both point and probabilistic accuracy under varying retraining frequencies. We showed that ensembles consistently improve forecasting performance, particularly in probabilistic settings. However, these gains come at a substantial computational cost, especially for larger, accuracy-driven ensembles. We found that reducing retraining frequency significantly lowers costs, with minimal impact on accuracy, particularly for point forecasts. Moreover, efficiency-driven ensembles offer a strong balance, achieving competitive accuracy with considerably lower costs compared to accuracy-optimized combinations. Most importantly, small ensembles of two or three models are often sufficient to achieve near-optimal results. These findings provide practical guidelines for deploying scalable and cost-efficient forecasting systems, supporting the broader goals of sustainable artificial intelligence in forecasting.<\/jats:p>","DOI":"10.1371\/journal.pone.0336281","type":"journal-article","created":{"date-parts":[[2026,9,29]],"date-time":"2026-09-29T17:31:46Z","timestamp":1790703106000},"page":"e0336281","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":0,"title":["The cost of ensembling: Is it always worth combining global retail forecasting models?"],"prefix":"10.1371","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-8242-0402","authenticated-orcid":true,"given":"Marco","family":"Zanotti","sequence":"first","affiliation":[{"name":"University of Milano-Bicocca","place":["Milan, Italy"],"department":["Department of Economics, Management and Statistics"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2026,9,29]]},"reference":[{"issue":"3","key":"pone.0336281.ref001","doi-asserted-by":"crossref","first-page":"754","DOI":"10.1016\/j.ijforecast.2015.12.005","article-title":"The forecast combination puzzle: A simple theoretical explanation","volume":"32","author":"G Claeskens","year":"2016","journal-title":"Int J Forecast"},{"issue":"4","key":"pone.0336281.ref002","doi-asserted-by":"crossref","first-page":"1518","DOI":"10.1016\/j.ijforecast.2022.11.005","article-title":"Forecast combinations: An over 50-year review","volume":"39","author":"X Wang","year":"2023","journal-title":"Int J Forecast"},{"issue":"2","key":"pone.0336281.ref003","doi-asserted-by":"crossref","first-page":"238","DOI":"10.1016\/j.ijforecast.2010.04.006","article-title":"Combining exponential smoothing forecasts using Akaike weights","volume":"27","author":"S Kolassa","year":"2011","journal-title":"Int J Forecast"},{"issue":"8","key":"pone.0336281.ref004","doi-asserted-by":"crossref","first-page":"1564","DOI":"10.1080\/01605682.2024.2421339","article-title":"Wielding Occam\u2019s razor: Fast and frugal retail forecasting","volume":"76","author":"F Petropoulos","year":"2024","journal-title":"J Oper Res Soc"},{"issue":"12","key":"pone.0336281.ref005","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1145\/3381831","article-title":"Green AI","volume":"63","author":"R Schwartz","year":"2020","journal-title":"Commun ACM"},{"issue":"3","key":"pone.0336281.ref006","doi-asserted-by":"crossref","first-page":"1072","DOI":"10.1016\/j.ijforecast.2020.11.009","article-title":"Investigating the accuracy of cross-learning time series forecasting methods","volume":"37","author":"A-A Semenoglou","year":"2021","journal-title":"Int J Forecast"},{"issue":"4","key":"pone.0336281.ref007","doi-asserted-by":"crossref","first-page":"1632","DOI":"10.1016\/j.ijforecast.2021.03.004","article-title":"Principles and algorithms for forecasting groups of time series: Locality and globality","volume":"37","author":"P Montero-Manso","year":"2021","journal-title":"Int J Forecast"},{"key":"pone.0336281.ref008","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1007\/978-3-031-35879-1_6","article-title":"How to Leverage Data for Time Series Forecasting with Artificial Intelligence Models: Illustrations and Guidelines for Cross-Learning.","volume-title":"How to Leverage Data for Time Series Forecasting with Artificial Intelligence Models: Illustrations and Guidelines for Cross-Learning","author":"P. Montero-Manso","year":"2023"},{"issue":"4","key":"pone.0336281.ref009","doi-asserted-by":"crossref","first-page":"1346","DOI":"10.1016\/j.ijforecast.2021.11.013","article-title":"M5 accuracy competition: Results, findings, and conclusions","volume":"38","author":"S Makridakis","year":"2022","journal-title":"Int J Forecast"},{"key":"pone.0336281.ref010","doi-asserted-by":"crossref","first-page":"107518","DOI":"10.1016\/j.knosys.2021.107518","article-title":"Ensembles of localised models for time series forecasting","volume":"233","author":"R Godahewa","year":"2021","journal-title":"Knowl-Based Syst"},{"key":"pone.0336281.ref011","doi-asserted-by":"crossref","first-page":"112896","DOI":"10.1016\/j.eswa.2019.112896","article-title":"Forecasting across time series databases using recurrent neural networks on groups of similar series: A clustering approach","volume":"140","author":"K Bandara","year":"2020","journal-title":"Expert Syst Appl"},{"key":"pone.0336281.ref012","doi-asserted-by":"crossref","first-page":"108148","DOI":"10.1016\/j.patcog.2021.108148","article-title":"Improving the accuracy of global forecasting models using time series data augmentation","volume":"120","author":"K Bandara","year":"2021","journal-title":"Pattern Recogn"},{"issue":"2","key":"pone.0336281.ref013","doi-asserted-by":"crossref","first-page":"131","DOI":"10.2307\/2344546","article-title":"Experience with forecasting univariate time series and the combination of forecasts","volume":"137","author":"P Newbold","year":"1974","journal-title":"J R Stat Soc Series A (General)"},{"issue":"1","key":"pone.0336281.ref014","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1016\/j.ijforecast.2019.03.019","article-title":"Weighted ensemble of statistical models","volume":"36","author":"M Pawlikowski","year":"2020","journal-title":"Int J Forecast"},{"issue":"4","key":"pone.0336281.ref015","doi-asserted-by":"crossref","first-page":"1325","DOI":"10.1016\/j.ijforecast.2021.07.007","article-title":"The M5 competition: Background, organization, and implementation","volume":"38","author":"S Makridakis","year":"2022","journal-title":"Int J Forecast"},{"issue":"4","key":"pone.0336281.ref016","doi-asserted-by":"crossref","first-page":"1365","DOI":"10.1016\/j.ijforecast.2021.10.009","article-title":"The M5 uncertainty competition: Results, findings and conclusions","volume":"38","author":"S Makridakis","year":"2022","journal-title":"Int J Forecast"},{"issue":"4","key":"pone.0336281.ref017","doi-asserted-by":"crossref","first-page":"1319","DOI":"10.1016\/j.ijforecast.2021.09.012","article-title":"Post-script-Retail forecasting: Research and practice","volume":"38","author":"R Fildes","year":"2022","journal-title":"Int J Forecast"},{"issue":"4","key":"pone.0336281.ref018","doi-asserted-by":"crossref","first-page":"1424","DOI":"10.1016\/j.ijforecast.2019.02.002","article-title":"Machine learning methods for GEFCom2017 probabilistic load forecasting","volume":"35","author":"S Smyl","year":"2019","journal-title":"Int J Forecast"},{"issue":"4","key":"pone.0336281.ref019","doi-asserted-by":"crossref","first-page":"495","DOI":"10.1111\/obes.12163","article-title":"Quantile aggregation of density forecasts","volume":"79","author":"F Busetti","year":"2017","journal-title":"Oxford Bull Econ Stat"},{"key":"pone.0336281.ref020","first-page":"100769","article-title":"On the retraining frequency of global models in retail demand forecasting","volume":"22","author":"M Zanotti","year":"2025","journal-title":"Mach Learn Appl"},{"issue":"1","key":"pone.0336281.ref021","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1016\/j.ejor.2023.08.056","article-title":"On the update frequency of univariate forecasting models","volume":"314","author":"E Spiliotis","year":"2024","journal-title":"Eur J Oper Res"},{"issue":"4","key":"pone.0336281.ref022","doi-asserted-by":"crossref","first-page":"1420","DOI":"10.1016\/j.ijforecast.2020.02.005","article-title":"Daily retail demand forecasting using machine learning with emphasis on calendric special days","volume":"36","author":"J Huber","year":"2020","journal-title":"Int J Forecast"},{"key":"pone.0336281.ref023","unstructured":"Getzner J, Charpentier B, G\u00fcnnemann S. Accuracy is not the only Metric that matters: Estimating the Energy Consumption of Deep Learning Models. arXiv. 2023. Available from: https:\/\/arxiv.org\/abs\/2304.00897"},{"key":"pone.0336281.ref024","unstructured":"Howard A, Makridakis S. M5 Forecasting - Accuracy. Kaggle. 2020. Available from: https:\/\/kaggle.com\/competitions\/m5-forecasting-accuracy"},{"key":"pone.0336281.ref025","unstructured":"Vandeput N. VN1 Forecasting - Accuracy Challenge. DataSource.ai. 2024. https:\/\/www.datasource.ai\/en\/home\/data-science-competitions-for-startups\/phase-2-vn1-forecasting-accuracy-challenge\/description"},{"key":"pone.0336281.ref026","unstructured":"Godahewa RW, Bergmeir C, Webb GI, Hyndman R, Montero-Manso P. Monash Time Series Forecasting Archive. In: Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2). 2021. pp. 1\u201314. Available from: https:\/\/openreview.net\/forum?id=wEc1mgAjU-"},{"issue":"1","key":"pone.0336281.ref027","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random Forests","volume":"45","author":"L Breiman","year":"2001","journal-title":"Mach Learn"},{"key":"pone.0336281.ref028","doi-asserted-by":"crossref","first-page":"785","DOI":"10.1145\/2939672.2939785","article-title":"XGBoost: A Scalable Tree Boosting System.","volume-title":"Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. KDD \u201916","author":"T Chen","year":"2016"},{"key":"pone.0336281.ref029","first-page":"1","article-title":"LightGBM: A Highly Efficient Gradient Boosting Decision Tree.","volume-title":"Advances in Neural Information Processing Systems","author":"G Ke","year":"2017"},{"key":"pone.0336281.ref030","first-page":"6639","article-title":"CatBoost: unbiased boosting with categorical features.","volume-title":"Proceedings of the 32nd International Conference on Neural Information Processing Systems. NIPS\u201918","author":"L Prokhorenkova","year":"2018"},{"issue":"6","key":"pone.0336281.ref031","doi-asserted-by":"crossref","first-page":"386","DOI":"10.1037\/h0042519","article-title":"The perceptron: a probabilistic model for information storage and organization in the brain","volume":"65","author":"F Rosenblatt","year":"1958","journal-title":"Psychol Rev"},{"issue":"8","key":"pone.0336281.ref032","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"S Hochreiter","year":"1997","journal-title":"Neural Comput"},{"key":"pone.0336281.ref033","unstructured":"Van den Oord A, Dieleman S, Zen H, Simonyan K, Vinyals O, Graves A, et al. WaveNet: A Generative Model for Raw Audio. CoRR. 2016. Available from: http:\/\/dblp.uni-trier.de\/db\/journals\/corr\/corr1609.html#OordDZSVGKSK16"},{"key":"pone.0336281.ref034","unstructured":"Oreshkin BN, Carpov D, Chapados N, Bengio Y. N-BEATS: Neural basis expansion analysis for interpretable time series forecasting. In: International Conference on Learning Representations. 2020. pp. 1\u201331. Available from: https:\/\/openreview.net\/forum?id=r1ecqn4YwB"},{"key":"pone.0336281.ref035","unstructured":"Challu C, Olivares KG, Oreshkin BN, Garza F, Mergenthaler-Canseco M, Dubrawski AW. N-HiTS: Neural Hierarchical Interpolation for Time Series Forecasting. ArXiv. 2022. Available from: https:\/\/api.semanticscholar.org\/CorpusID:246430557"},{"key":"pone.0336281.ref036","doi-asserted-by":"crossref","first-page":"226","DOI":"10.1016\/j.ijpe.2018.05.019","article-title":"Another look at forecast selection and combination: Evidence from forecast pooling","volume":"209","author":"N Kourentzes","year":"2019","journal-title":"Int J Prod Econ"},{"key":"pone.0336281.ref037","unstructured":"Nixtla. Nixtla - State-of-the-art time series and forecasting software. Nixtla; 2022. https:\/\/nixtlaverse.nixtla.io\/"},{"issue":"4","key":"pone.0336281.ref038","doi-asserted-by":"crossref","first-page":"437","DOI":"10.1016\/S0169-2070(00)00065-0","article-title":"Out-of-sample tests of forecasting accuracy: an analysis and review","volume":"16","author":"LJ Tashman","year":"2000","journal-title":"Int J Forecast"},{"key":"pone.0336281.ref039","doi-asserted-by":"crossref","first-page":"192","DOI":"10.1016\/j.ins.2011.12.028","article-title":"On the use of cross-validation for time series predictor evaluation","volume":"191","author":"C Bergmeir","year":"2012","journal-title":"Inform Sci"},{"issue":"2","key":"pone.0336281.ref040","doi-asserted-by":"crossref","first-page":"788","DOI":"10.1007\/s10618-022-00894-5","article-title":"Forecast evaluation for data scientists: Common pitfalls and best practices","volume":"37","author":"H Hewamalage","year":"2023","journal-title":"Data Min Knowl Discov"},{"issue":"1","key":"pone.0336281.ref041","doi-asserted-by":"crossref","first-page":"208","DOI":"10.1016\/j.ijforecast.2019.02.017","article-title":"Why the \u201cbest\u201d point forecast depends on the error or accuracy measure","volume":"36","author":"S Kolassa","year":"2020","journal-title":"Int J Forecast"},{"issue":"4","key":"pone.0336281.ref042","doi-asserted-by":"crossref","first-page":"679","DOI":"10.1016\/j.ijforecast.2006.03.001","article-title":"Another look at measures of forecast accuracy","volume":"22","author":"RJ Hyndman","year":"2006","journal-title":"Int J Forecast"},{"issue":"3","key":"pone.0336281.ref043","doi-asserted-by":"crossref","first-page":"788","DOI":"10.1016\/j.ijforecast.2015.12.004","article-title":"Evaluating predictive count data distributions in retail sales forecasting","volume":"32","author":"S Kolassa","year":"2016","journal-title":"Int J Forecast"},{"key":"pone.0336281.ref044","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1016\/j.ijpe.2016.03.017","article-title":"Distributions of forecasting errors of forecast combinations: Implications for inventory management","volume":"177","author":"DK Barrow","year":"2016","journal-title":"Int J Prod Econ"},{"issue":"1","key":"pone.0336281.ref045","first-page":"1","article-title":"Statistical comparisons of classifiers over multiple data sets","volume":"7","author":"J Dem\u0161ar","year":"2006","journal-title":"J Mach Learn Res"},{"key":"pone.0336281.ref046","doi-asserted-by":"crossref","first-page":"322","DOI":"10.1016\/j.cor.2017.05.007","article-title":"Forecasting for big data: Does suboptimality matter?","volume":"98","author":"K Nikolopoulos","year":"2018","journal-title":"Comput Oper Res"}],"container-title":["PLOS One"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0336281","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,29]],"date-time":"2026-09-29T17:32:09Z","timestamp":1790703129000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0336281"}},"subtitle":[],"editor":[{"given":"Flavio A.","family":"Ziegelmann","sequence":"first","affiliation":[{"name":"Universidade Federal do Rio Grande do Sul","place":["BRAZIL"]}],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026,9,29]]},"references-count":46,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2026,9,29]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0336281","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,29]]}}}