{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,14]],"date-time":"2025-10-14T00:34:07Z","timestamp":1760402047528,"version":"build-2065373602"},"reference-count":33,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2020,4,22]],"date-time":"2020-04-22T00:00:00Z","timestamp":1587513600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Data"],"abstract":"<jats:p>As forecasting becomes more and more appreciated in situations and activities of everyday life that involve prediction and risk assessment, more methods and solutions make their appearance in this exciting arena of uncertainty. However, less is known about what makes a promising or a poor forecast. In this article, we provide a multi-factor analysis on the forecasting methods that participated and stood out in the M4 competition, by focusing on Error (predictive performance), Correlation (among different methods), and Complexity (computational performance). The main goal of this study is to recognize the key elements of the contemporary forecasting methods, reveal what made them excel in the M4 competition, and eventually provide insights towards better understanding the forecasting task.<\/jats:p>","DOI":"10.3390\/data5020041","type":"journal-article","created":{"date-parts":[[2020,4,23]],"date-time":"2020-04-23T10:46:22Z","timestamp":1587638782000},"page":"41","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["A Multi-Factor Analysis of Forecasting Methods: A Study on the M4 Competition"],"prefix":"10.3390","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9897-4879","authenticated-orcid":false,"given":"Pantelis","family":"Agathangelou","sequence":"first","affiliation":[{"name":"Department of Computer Science, School of Sciences and Engineering, University of Nicosia, Nicosia CY-2417, Cyprus"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9540-7342","authenticated-orcid":false,"given":"Demetris","family":"Trihinas","sequence":"additional","affiliation":[{"name":"Department of Computer Science, School of Sciences and Engineering, University of Nicosia, Nicosia CY-2417, Cyprus"}]},{"given":"Ioannis","family":"Katakis","sequence":"additional","affiliation":[{"name":"Department of Computer Science, School of Sciences and Engineering, University of Nicosia, Nicosia CY-2417, Cyprus"}]}],"member":"1968","published-online":{"date-parts":[[2020,4,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1145\/3236644.3236646","article-title":"AI Education Matters: Lessons from a Kaggle Click-through Rate Prediction Competition","volume":"4","author":"Neller","year":"2018","journal-title":"AI Matters"},{"key":"ref_2","unstructured":"Jordon, J., Yoon, J., and van der Schaar, M. (2018). Measuring the quality of Synthetic data for use in competitions. arXiv."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Witkowski, J., Freeman, R., Vaughan, J., Pennock, D., and Krause, A. (2018, January 2\u20137). Incentive-Compatible Forecasting Competitions. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11471"},{"key":"ref_4","first-page":"1006","article-title":"The Ladder: A Reliable Leaderboard for Machine Learning Competitions","volume":"Volume 37","author":"Bach","year":"2015","journal-title":"Proceedings of the 32nd International Conference on Machine Learning"},{"key":"ref_5","first-page":"1","article-title":"Automatic Time Series Forecasting: The forecast Package for R","volume":"27","author":"Hyndman","year":"2008","journal-title":"J. Stat. Softw. Artic."},{"key":"ref_6","unstructured":"Makridakis, S., Wheelwright, S.C., and Hyndman, R.J. (2008). Forecasting Methods and Applications, John Wiley & Sons."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1016\/j.jmarsys.2016.08.006","article-title":"Influence of climate variability on anchovy reproductive timing off northern Chile","volume":"164","author":"Canales","year":"2016","journal-title":"J. Mar. Syst."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"679","DOI":"10.1016\/j.ijforecast.2006.03.001","article-title":"Another look at measures of forecast accuracy","volume":"22","author":"Hyndman","year":"2006","journal-title":"Int. J. Forecast."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"451","DOI":"10.1016\/S0169-2070(00)00057-1","article-title":"The M3-Competition: Results, conclusions and implications","volume":"16","author":"Makridakis","year":"2000","journal-title":"Int. J. Forecast."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Hyndman, R., Koehler, A.B., Ord, J.K., and Snyder, R.D. (2008). Forecasting with Exponential Smoothing: The State Space Approach, Springer Science & Business Media.","DOI":"10.1007\/978-3-540-71918-2"},{"key":"ref_11","unstructured":"M4 Competition (2020, April 21). Forecast. Compete. Excel. Competitor\u2019s Guide: Prizes and Rules; 1 January\u201331 May 2018. Available online: https:\/\/mofc.unic.ac.cy\/m4\/."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1016\/j.ijforecast.2019.04.014","article-title":"The M4 Competition: 100,000 time series and 61 forecasting methods","volume":"36","author":"Makridakis","year":"2020","journal-title":"Int. J. Forecast."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1016\/j.ijforecast.2019.03.017","article-title":"A hybrid method of exponential smoothing and recurrent neural networks for time series forecasting","volume":"36","author":"Smyl","year":"2020","journal-title":"Int. J. Forecast."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1016\/j.ijforecast.2019.02.011","article-title":"FFORMA: Feature-based forecast model averaging","volume":"36","author":"Athanasopoulos","year":"2020","journal-title":"Int. J. Forecast."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1016\/j.ijforecast.2019.03.019","article-title":"Weighted ensemble of statistical models","volume":"36","author":"Pawlikowski","year":"2020","journal-title":"Int. J. Forecast."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1016\/j.ijforecast.2019.03.030","article-title":"A combination-based forecasting method for the M4-competition","volume":"36","author":"Jaganathan","year":"2020","journal-title":"Int. J. Forecast."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1016\/j.ijforecast.2019.04.013","article-title":"GROEC: Combination method via Generalized Rolling Origin Evaluation","volume":"36","author":"Fiorucci","year":"2020","journal-title":"Int. J. Forecast."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1016\/j.ijforecast.2019.01.006","article-title":"A simple combination of univariate models","volume":"36","author":"Petropoulos","year":"2020","journal-title":"Int. J. Forecast."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"116","DOI":"10.1016\/j.ijforecast.2019.03.032","article-title":"Fast and accurate yearly time series forecasting with forecast combinations","volume":"36","author":"Shaub","year":"2020","journal-title":"Int. J. Forecast."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1016\/j.ijforecast.2019.03.012","article-title":"Card forecasts for M4","volume":"36","author":"Doornik","year":"2020","journal-title":"Int. J. Forecast."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"802","DOI":"10.1016\/j.ijforecast.2018.06.001","article-title":"The M4 Competition: Results, findings, conclusion and way forward","volume":"34","author":"Makridakis","year":"2018","journal-title":"Int. J. Forecast."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"212","DOI":"10.1016\/j.ijforecast.2019.03.020","article-title":"Correlation analysis of forecasting methods: The case of the M4 competition","volume":"36","author":"Agathangelou","year":"2020","journal-title":"Int. J. Forecast."},{"key":"ref_23","unstructured":"Hyndman, R.J., and Athanasopoulos, G. (2013). Forecasting: Principles and Practice, OTexts, Monash University."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1016\/j.ijforecast.2003.09.015","article-title":"Forecasting seasonals and trends by exponentially weighted moving averages","volume":"20","author":"Holt","year":"2004","journal-title":"Int. J. Forecast."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1237","DOI":"10.1287\/mnsc.31.10.1237","article-title":"Forecasting Trends in Time Series","volume":"31","author":"Gardner","year":"1985","journal-title":"Manag. Sci."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"800","DOI":"10.1109\/TCC.2015.2511760","article-title":"Monitoring Elastically Adaptive Multi-Cloud Services","volume":"6","author":"Trihinas","year":"2018","journal-title":"IEEE Trans. Cloud Comput."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Matsubara, Y., and Sakurai, Y. (2019, January 4\u20138). Dynamic Modeling and Forecasting of Time-Evolving Data Streams. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA.","DOI":"10.1145\/3292500.3330947"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Ben Taieb, S., and Koo, B. (2019, January 4\u20138). Regularized Regression for Hierarchical Forecasting without Unbiasedness Conditions. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA.","DOI":"10.1145\/3292500.3330976"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Adhikari, B., Xu, X., Ramakrishnan, N., and Prakash, B.A. (2019, January 4\u20138). EpiDeep: Exploiting Embeddings for Epidemic Forecasting. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA.","DOI":"10.1145\/3292500.3330917"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Deshpande, P., and Sarawagi, S. (2019, January 4\u20138). Streaming Adaptation of Deep Forecasting Models Using Adaptive Recurrent Units. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA.","DOI":"10.1145\/3292500.3330996"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Wang, B., Lu, J., Yan, Z., Luo, H., Li, T., Zheng, Y., and Zhang, G. (2019, January 4\u20138). Deep Uncertainty Quantification: A Machine Learning Approach for Weather Forecasting. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA.","DOI":"10.1145\/3292500.3330704"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Fan, C., Zhang, Y., Pan, Y., Li, X., Zhang, C., Yuan, R., Wu, D., Wang, W., Pei, J., and Huang, H. (2019, January 4\u20138). Multi-Horizon Time Series Forecasting with Temporal Attention Learning. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA.","DOI":"10.1145\/3292500.3330662"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Sch\u00f6n, C., Dittrich, J., and M\u00fcller, R. (2019, January 4\u20138). The Error is the Feature: How to Forecast Lightning Using a Model Prediction Error. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA.","DOI":"10.1145\/3292500.3330682"}],"container-title":["Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2306-5729\/5\/2\/41\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T13:23:31Z","timestamp":1760361811000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2306-5729\/5\/2\/41"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,4,22]]},"references-count":33,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2020,6]]}},"alternative-id":["data5020041"],"URL":"https:\/\/doi.org\/10.3390\/data5020041","relation":{},"ISSN":["2306-5729"],"issn-type":[{"type":"electronic","value":"2306-5729"}],"subject":[],"published":{"date-parts":[[2020,4,22]]}}}