{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,21]],"date-time":"2026-03-21T07:37:56Z","timestamp":1774078676493,"version":"3.50.1"},"reference-count":52,"publisher":"Association for Computing Machinery (ACM)","issue":"3","funder":[{"DOI":"10.13039\/501100000923","name":"Australian Research Council","doi-asserted-by":"crossref","award":["210101347"],"award-info":[{"award-number":["210101347"]}],"id":[{"id":"10.13039\/501100000923","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Australian Research Council (ARC) Discovery Early Career Researcher Award","award":["DE190100626"],"award-info":[{"award-number":["DE190100626"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2026,6,30]]},"abstract":"<jats:p>Long-term time series forecasting (LTSF) is fundamental to various real-world applications, where Transformer-based models have become the dominant framework due to their ability to capture long-range dependencies. However, these models often experience overfitting due to data redundancy in rolling forecasting settings, limiting their generalization ability particularly evident in longer sequences with highly similar adjacent data. In this work, we introduce CLMFormer, a novel framework that mitigates redundancy through curriculum learning and a memory-driven decoder. Specifically, we progressively introduce Bernoulli noise to the training samples, which effectively breaks the high similarity between adjacent data points. This curriculum-driven noise introduction aids the memory-driven decoder by supplying more diverse and representative training data, enhancing the decoder\u2019s ability to model seasonal tendencies and dependencies in the time series data. To further enhance forecasting accuracy, we introduce a memory-driven decoder. This component enables the model to capture seasonal tendencies and dependencies in the time series data and leverages temporal relationships to facilitate the forecasting process. Extensive experiments on six real-world LTSF benchmarks show that CLMFormer consistently improves Transformer-based models by up to 30%, demonstrating its effectiveness in long-horizon forecasting.<\/jats:p>","DOI":"10.1145\/3735651","type":"journal-article","created":{"date-parts":[[2025,5,16]],"date-time":"2025-05-16T12:48:39Z","timestamp":1747399719000},"page":"1-21","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Mitigating Data Redundancy to Revitalize Transformer-Based Long-Term Time Series Forecasting System"],"prefix":"10.1145","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6096-9858","authenticated-orcid":false,"given":"Mingjie","family":"Li","sequence":"first","affiliation":[{"name":"Department of Radiation Oncology, Stanford University, Stanford, California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-5177-5466","authenticated-orcid":false,"given":"Rui","family":"Liu","sequence":"additional","affiliation":[{"name":"FEIT, University of Technology Sydney, Sydney, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0313-6379","authenticated-orcid":false,"given":"Guangsi","family":"Shi","sequence":"additional","affiliation":[{"name":"FIT, Monash University, Melbourne, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0040-6177","authenticated-orcid":false,"given":"Mingfei","family":"Han","sequence":"additional","affiliation":[{"name":"University of Technology Sydney, Sydney, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5477-5918","authenticated-orcid":false,"given":"Changlin","family":"Li","sequence":"additional","affiliation":[{"name":"University of Technology Sydney, Sydney, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4149-839X","authenticated-orcid":false,"given":"Lina","family":"Yao","sequence":"additional","affiliation":[{"name":"Computer Science and Engineering, University of New South Wales, Sydney, Australia and CSIRO Data61 Business Unit, Eveleigh, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7778-8807","authenticated-orcid":false,"given":"Xiaojun","family":"Chang","sequence":"additional","affiliation":[{"name":"University of Science and Technology of China, Hefei, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6468-5729","authenticated-orcid":false,"given":"Ling","family":"Chen","sequence":"additional","affiliation":[{"name":"University of Technology Sydney, Sydney, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2026,3,20]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"D. Bahdanau. 2014. Neural machine translation by jointly learning to align and translate. arXiv:1409.0473. Retrieved from https:\/\/arxiv.org\/abs\/1409.0473"},{"key":"e_1_3_2_3_2","volume-title":"ICLR","author":"Banino A.","year":"2020","unstructured":"A. Banino, A. P. Badia, R. K\u00f6ster, M. J. Chadwick, V. F. Zambaldi, D. Hassabis, C. Barry, M. Botvinick, D. Kumaran, and C. Blundell. 2020. MEMO: A deep network for flexible combination of episodic memories. In ICLR."},{"key":"e_1_3_2_4_2","article-title":"Scheduled sampling for sequence prediction with recurrent neural networks","volume":"28","author":"Bengio S.","year":"2015","unstructured":"S. Bengio, O. Vinyals, N. Jaitly, and N. Shazeer. 2015. Scheduled sampling for sequence prediction with recurrent neural networks. In Advances in Neural Information Processing Systems, Vol. 28.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/3533382"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-023-06185-3"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1995.7.1.108"},{"issue":"2","key":"e_1_3_2_8_2","first-page":"91","article-title":"Some recent advances in forecasting and control","volume":"17","author":"Box G. E.","year":"1968","unstructured":"G. E. Box and G. M. Jenkins. 1968. Some recent advances in forecasting and control. Journal of the Royal Statistical Society. Series C. Applied Statistics 17, 2 (1968), 91\u2013109.","journal-title":"Journal of the Royal Statistical Society. Series C. Applied Statistics"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v37i6.25845"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.112"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2021.3094908"},{"key":"e_1_3_2_12_2","doi-asserted-by":"crossref","unstructured":"K. Cho B. Van Merri\u00ebnboer D. Bahdanau and Y. Bengio. 2014. On the properties of neural machine translation: Encoder-decoder approaches. arXiv:1409.1259. Retrieved from https:\/\/arxiv.org\/abs\/1409.1259","DOI":"10.3115\/v1\/W14-4012"},{"issue":"1","key":"e_1_3_2_13_2","first-page":"3","article-title":"Stl: A seasonal-trend decomposition","volume":"6","author":"Cleveland R. B.","year":"1990","unstructured":"R. B. Cleveland, W. S. Cleveland, J. E. McRae, and I. Terpenning. 1990. Stl: A seasonal-trend decomposition. Journal of Official Statistics 6, 1 (1990), 3\u201373.","journal-title":"Journal of Official Statistics"},{"key":"e_1_3_2_14_2","volume-title":"ICML","author":"Das A.","year":"2024","unstructured":"A. Das, W. Kong, R. Sen, and Y. Zhou. 2024. A decoder-only foundation model for time-series forecasting. In ICML. Retrieved from https:\/\/openreview.net\/forum?id=jn2iTJas6h"},{"key":"e_1_3_2_15_2","volume-title":"NAACL","author":"Devlin J.","year":"2018","unstructured":"J. Devlin, M. W. Chang, K. Lee, and K. Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL."},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i2.16220"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3649448"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1002\/for.3980040103"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2957192"},{"key":"e_1_3_2_21_2","unstructured":"M. Jin G. Shi Y. F. Li Q. Wen B. Xiong T. Zhou and S. Pan. 2023. How expressive are spectral-temporal graph neural networks for time series forecasting?. arXiv:2305.06587. Retrieved from https:\/\/arxiv.org\/abs\/2305.06587"},{"key":"e_1_3_2_22_2","volume-title":"ICML","author":"Jin M.","year":"2024","unstructured":"M. Jin, Y. Zhang, W. Chen, K. Zhang, Y. Liang, B. Yang, J. Wang, S. Pan, and Q. Wen. 2024. Position: What can large language models tell us about time series analysis. In ICML. Retrieved from https:\/\/openreview.net\/forum?id=iroZNDxFJZ"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2022.3221989"},{"issue":"2","key":"e_1_3_2_24_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3630637","article-title":"Ts-fastformer: Fast transformer for time-series forecasting","volume":"15","author":"Lee S.","year":"2024","unstructured":"S. Lee, J. Hong, L. Liu, and W. Choi. 2024. Ts-fastformer: Fast transformer for time-series forecasting. ACM Transactions on Intelligent Systems and Technology 15, 2 (2024), 1\u201320.","journal-title":"ACM Transactions on Intelligent Systems and Technology"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2022.3181116"},{"key":"e_1_3_2_26_2","doi-asserted-by":"crossref","unstructured":"M. Li B. Lin Z. Chen H. Lin X. Liang and X. Chang. 2023. Dynamic graph enhanced contrastive learning for chest x-ray report generation. arXiv:2303.10323. Retrieved from https:\/\/arxiv.org\/abs\/2303.10323","DOI":"10.1109\/CVPR52729.2023.00325"},{"key":"e_1_3_2_27_2","first-page":"162","volume-title":"ECCV","author":"Li M.","year":"2024","unstructured":"M. Li, H. Lin, L. Qiu, X. Liang, L. Chen, A. Elsaddik, and X. Chang. 2024. Contrastive learning with counterfactual explanations for radiology report generation. In ECCV. Springer, 162\u2013180."},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00729"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i04.5945"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP39728.2021.9414897"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.383"},{"key":"e_1_3_2_32_2","volume-title":"ICLR","author":"Nie Y.","year":"2023","unstructured":"Y. Nie, N. H. Nguyen, P. Sinthong, and J. Kalagnanam. 2023. A time series is worth 64 words: Long-term forecasting with transformers. In ICLR. Retrieved from https:\/\/openreview.net\/pdf?id=Jbdc0vTOcol"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.5555\/3172077.3172254"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-023-02132-4"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijforecast.2019.07.001"},{"key":"e_1_3_2_36_2","unstructured":"I. Sutskever. 2014. Sequence to sequence learning with neural networks. arXiv:1409.3215. Retrieved from https:\/\/arxiv.org\/abs\/1409.3215"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2022.09.003"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2021.3128770"},{"key":"e_1_3_2_39_2","volume-title":"NeurIPS","author":"Vaswani A.","year":"2017","unstructured":"A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, \u0141. Kaiser, and I. Polosukhin. 2017. Attention is all you need. In NeurIPS."},{"key":"e_1_3_2_40_2","volume-title":"NeurIPS","author":"Wager S.","year":"2013","unstructured":"S. Wager, S. I. Wang, and P. Liang. 2013. Dropout training as adaptive regularization. In NeurIPS."},{"key":"e_1_3_2_41_2","unstructured":"S. Wang J. Li X. Shi Z. Ye B. Mo W. Lin S. Ju Z. Chu and M. Jin. 2024. Timemixer++: A general time series pattern machine for universal predictive analysis. arXiv:2410.16032. Retrieved from https:\/\/arxiv.org\/abs\/2410.16032"},{"key":"e_1_3_2_42_2","volume-title":"ICLR","author":"Wang S.","year":"2024","unstructured":"S. Wang, H. Wu, X. Shi, T. Hu, H. Luo, L. Ma, J. Y. Zhang, and J. Zhou. 2024. Timemixer: Decomposable multiscale mixing for time series forecasting. In ICLR. Retrieved from https:\/\/openreview.net\/forum?id=7oLshfEIC2"},{"key":"e_1_3_2_43_2","unstructured":"Q. Wen T. Zhou C. Zhang W. Chen Z. Ma J. Yan and L. Sun. 2022. Transformers in time series: A survey. arXiv:2202.07125. Retrieved from https:\/\/arxiv.org\/abs\/2202.07125"},{"key":"e_1_3_2_44_2","unstructured":"R. Wen K. Torkkola B. Narayanaswamy and D. Madeka. 2018. A multi-horizon quantile recurrent forecaster. arXiv:1711.11053. Retrieved from https:\/\/arxiv.org\/abs\/1711.11053"},{"key":"e_1_3_2_45_2","first-page":"22419","article-title":"Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting","volume":"34","author":"Wu H.","year":"2021","unstructured":"H. Wu, J. Xu, J. Wang, and M. Long. 2021. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. In Advances in Neural Information Processing Systems, Vol. 34, 22419\u201322430.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_46_2","volume-title":"NeurIPS","author":"Wu S.","year":"2020","unstructured":"S. Wu, X. Xiao, Q. Ding, P. Zhao, Y. Wei, and J. Huang. 2020. Adversarial sparse transformer for time series forecasting. In NeurIPS."},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i02.5496"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2020.3003420"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v37i9.26317"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611974010.51"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i12.17325"},{"key":"e_1_3_2_52_2","volume-title":"NeurIPS","author":"Zhou T.","year":"2022","unstructured":"T. Zhou, Z. Ma, X. Wang, Q. Wen, L. Sun, T. Yao, W. Yin, and R. Jin. 2022. Film: Frequency improved legendre memory model for long-term time series forecasting. In NeurIPS."},{"key":"e_1_3_2_53_2","first-page":"27268","volume-title":"ICML","author":"Zhou T.","year":"2022","unstructured":"T. Zhou, Z. Ma, Q. Wen, X. Wang, L. Sun, and R. Jin. 2022. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In ICML. PMLR, 27268\u201327286."}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3735651","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,21]],"date-time":"2026-03-21T04:48:40Z","timestamp":1774068520000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3735651"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,20]]},"references-count":52,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2026,6,30]]}},"alternative-id":["10.1145\/3735651"],"URL":"https:\/\/doi.org\/10.1145\/3735651","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"value":"2157-6904","type":"print"},{"value":"2157-6912","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,20]]},"assertion":[{"value":"2024-11-18","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-04-06","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-03-20","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}