{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,2]],"date-time":"2026-03-02T22:49:16Z","timestamp":1772491756135,"version":"3.50.1"},"reference-count":50,"publisher":"Association for Computing Machinery (ACM)","issue":"2","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2024,10]]},"abstract":"<jats:p>Accurate long-term forecasting from multivariate time series has important real-world applications. However, achieving this so is challenging. Thus, analyses reveal that time series that span long durations often exhibit dynamic and disrupted correlations. State-of-the-art methods employ attention mechanisms to capture dynamic correlations, but they often do not contend well with disrupted correlations, which reduces prediction accuracy. We introduce local and global information concepts and then leverage these in a Memory Guided Transformer, called the Memformer. By integrating patch-wise recurrent graph learning and global attention, the Memformer aims to capture dynamic correlations and take disrupted correlations into account. We also integrate a so-called Alternating Memory Enhancer into the Memformer to capture correlations between local and global information. We report on experiments that offer insight into the effectiveness of the Memformer at capturing dynamic correlations and its robustness to disrupted correlations. The experiments offer evidence that the new method is capable of advancing the state-of-the-art in forecasting accuracy on real-world datasets.<\/jats:p>","DOI":"10.14778\/3705829.3705842","type":"journal-article","created":{"date-parts":[[2025,2,28]],"date-time":"2025-02-28T23:21:06Z","timestamp":1740784866000},"page":"239-252","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["A Memory Guided Transformer for Time Series Forecasting"],"prefix":"10.14778","volume":"18","author":[{"given":"Yunyao","family":"Cheng","sequence":"first","affiliation":[{"name":"Aalborg University"}]},{"given":"Chenjuan","family":"Guo","sequence":"additional","affiliation":[{"name":"East China Normal University"}]},{"given":"Bin","family":"Yang","sequence":"additional","affiliation":[{"name":"East China Normal University"}]},{"given":"Haomin","family":"Yu","sequence":"additional","affiliation":[{"name":"Aalborg University"}]},{"given":"Kai","family":"Zhao","sequence":"additional","affiliation":[{"name":"Aalborg University"}]},{"given":"Christian S.","family":"Jensen","sequence":"additional","affiliation":[{"name":"Aalborg University"}]}],"member":"320","published-online":{"date-parts":[[2025,2,28]]},"reference":[{"key":"e_1_2_1_1_1","first-page":"17804","article-title":"Adaptive graph convolutional recurrent network for traffic forecasting","volume":"33","author":"Bai Lei","year":"2020","unstructured":"Lei Bai, Lina Yao, Can Li, Xianzhi Wang, and Can Wang. 2020. Adaptive graph convolutional recurrent network for traffic forecasting. Advances in Neural Information Processing Systems 33 (2020), 17804--17815.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_1_2_1","volume-title":"Quantizable transformers: Removing outliers by helping attention heads do nothing. Advances in Neural Information Processing Systems 36","author":"Bondarenko Yelysei","year":"2024","unstructured":"Yelysei Bondarenko, Markus Nagel, and Tijmen Blankevoort. 2024. Quantizable transformers: Removing outliers by helping attention heads do nothing. Advances in Neural Information Processing Systems 36 (2024)."},{"key":"e_1_2_1_3_1","volume-title":"Memory Matching Networks for One-Shot Image Recognition. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018","author":"Cai Qi","year":"2018","unstructured":"Qi Cai, Yingwei Pan, Ting Yao, Chenggang Yan, and Tao Mei. 2018. Memory Matching Networks for One-Shot Image Recognition. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18--22, 2018. 4080--4088."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.14778\/3681954.3681957"},{"key":"e_1_2_1_5_1","first-page":"2","volume-title":"Proc. ACM Manag. Data 1","author":"Campos David","year":"2023","unstructured":"David Campos, Miao Zhang, Bin Yang, Tung Kieu, Chenjuan Guo, and Christian S. Jensen. 2023. LightTS: Lightweight Time Series Classification with Adaptive Ensemble Distillation. Proc. ACM Manag. Data 1, 2 (2023), 171:1--171:27."},{"key":"e_1_2_1_6_1","volume-title":"The Twelfth International Conference on Learning Representations.","author":"Chen Peng","year":"2024","unstructured":"Peng Chen, Yingying Zhang, Yunyao Cheng, Yang Shu, Yihang Wang, Qingsong Wen, Bin Yang, and Chenjuan Guo. 2024. Pathformer: Multi-scale transformers with Adaptive Pathways for Time Series Forecasting. In The Twelfth International Conference on Learning Representations."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.14778\/3636218.3636231"},{"key":"e_1_2_1_8_1","volume-title":"ADMA 2019, Dalian, China, November 21--23, 2019","volume":"11888","author":"Cheng Yunyao","year":"2019","unstructured":"Yunyao Cheng, Bin Wu, Li Song, and Chuan Shi. 2019. Spatial-Temporal Recurrent Neural Network for Anomalous Trajectories Detection. In Advanced Data Mining and Applications - 15th International Conference, ADMA 2019, Dalian, China, November 21--23, 2019, Proceedings, Vol. 11888. 565--578."},{"key":"e_1_2_1_9_1","volume-title":"Variable-Specific Attentions for Long Sequence Multivariate Time Series Forecasting. International Joint Conference on Artificial Intelligence","author":"Cirstea Razvan-Gabriel","year":"2022","unstructured":"Razvan-Gabriel Cirstea, Chenjuan Guo, Bin Yang, Tung Kieu, Xuanyi Dong, and Shirui Pan. 2022. Triformer: Triangular, Variable-Specific Attentions for Long Sequence Multivariate Time Series Forecasting. International Joint Conference on Artificial Intelligence, 1994--2001."},{"key":"e_1_2_1_10_1","unstructured":"Razvan-Gabriel Cirstea Bin Yang and Chenjuan Guo. 2019. Graph Attention Recurrent Neural Networks for Correlated Time Series Forecasting.. In MileTS19@KDD."},{"key":"e_1_2_1_11_1","volume-title":"Long-term Forecasting with","author":"Das Abhimanyu","year":"2023","unstructured":"Abhimanyu Das, Weihao Kong, Andrew Leach, Shaan K Mathur, Rajat Sen, and Rose Yu. 2023. Long-term Forecasting with TiDE: Time-series Dense Encoder. Transactions on Machine Learning Research (2023)."},{"key":"e_1_2_1_12_1","volume-title":"International Conference on Learning Representations.","author":"Dosovitskiy Alexey","year":"2020","unstructured":"Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. 2020. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. International Conference on Learning Representations."},{"key":"e_1_2_1_13_1","volume-title":"Neural turing machines. arXiv preprint arXiv:1410.5401","author":"Graves Alex","year":"2014","unstructured":"Alex Graves, Greg Wayne, and Ivo Danihelka. 2014. Neural turing machines. arXiv preprint arXiv:1410.5401 (2014)."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58580-8_19"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v37i7.25976"},{"key":"e_1_2_1_16_1","volume-title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. NAACL-HLT, 4171--4186.","author":"Ming-Wei Chang Jacob Devlin","year":"2019","unstructured":"Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. NAACL-HLT, 4171--4186."},{"key":"e_1_2_1_17_1","volume-title":"Proc. VLDB Endow. 18","author":"Kieu Duc","year":"2024","unstructured":"Duc Kieu, Tung Kieu, Peng Han, Bin Yang, Christian S. Jensen, and Bac Le. 2024. TEAM: Topological Evolution-aware Framework for Traffic Forecasting. Proc. VLDB Endow. 18 (2024)."},{"key":"e_1_2_1_18_1","volume-title":"International Conference on Machine Learning, 1378--1387","author":"Kumar Ankit","year":"2016","unstructured":"Ankit Kumar, Ozan Irsoy, Peter Ondruska, Mohit Iyyer, James Bradbury, Ishaan Gulrajani, Victor Zhong, Romain Paulus, and Richard Socher. 2016. Ask me anything: Dynamic memory networks for natural language processing. International Conference on Machine Learning, 1378--1387."},{"key":"e_1_2_1_19_1","volume-title":"Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. Advances in Neural Information Processing Systems 32","author":"Li Shiyang","year":"2019","unstructured":"Shiyang Li, Xiaoyong Jin, Yao Xuan, Xiyou Zhou, Wenhu Chen, Yu-Xiang Wang, and Xifeng Yan. 2019. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. Advances in Neural Information Processing Systems 32 (2019)."},{"key":"e_1_2_1_20_1","volume-title":"Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting. International Conference on Learning Representations.","author":"Li Yaguang","year":"2018","unstructured":"Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. 2018. Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting. International Conference on Learning Representations."},{"key":"e_1_2_1_21_1","volume-title":"Revisiting long-term time series forecasting: An investigation on linear mapping. arXiv preprint arXiv:2305.10721","author":"Li Zhe","year":"2023","unstructured":"Zhe Li, Shiyi Qi, Yiduo Li, and Zenglin Xu. 2023. Revisiting long-term time series forecasting: An investigation on linear mapping. arXiv preprint arXiv:2305.10721 (2023)."},{"key":"e_1_2_1_22_1","volume-title":"International Conference on Learning Representations.","author":"Liu Shizhan","year":"2021","unstructured":"Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X Liu, and Schahram Dustdar. 2021. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. International Conference on Learning Representations."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3511808.3557638"},{"key":"e_1_2_1_24_1","volume-title":"International Conference on Learning Representations","author":"Liu Yong","year":"2024","unstructured":"Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. 2024. itransformer: Inverted transformers are effective for time series forecasting. International Conference on Learning Representations (2024)."},{"key":"e_1_2_1_25_1","volume-title":"International Conference on Learning Representations.","author":"Luo Donghao","year":"2024","unstructured":"Donghao Luo and Xue Wang. 2024. ModernTCN: A modern pure convolution structure for general time series analysis. International Conference on Learning Representations."},{"key":"e_1_2_1_26_1","volume-title":"Proc. VLDB Endow. 18","author":"Miao Hao","year":"2024","unstructured":"Hao Miao, Ziqiao Liu, Yan Zhao, Chenjuan Guo, Bin Yang, Kai Zheng, and Christian S. Jensen. 2024. Less is More: Efficient Time Series Dataset Condensation via Two-fold Modal Matching. Proc. VLDB Endow. 18 (2024)."},{"key":"e_1_2_1_27_1","volume-title":"International Conference on Learning Representations","author":"Nie Yuqi","year":"2023","unstructured":"Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. 2023. A time series is worth 64 words: Long-term forecasting with transformers. International Conference on Learning Representations (2023)."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.14778\/3397230.3397248"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.14778\/3665844.3665863"},{"key":"e_1_2_1_30_1","volume-title":"International Conference on Machine Learning","author":"Santoro Adam","year":"2016","unstructured":"Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. 2016. Meta-learning with memory-augmented neural networks. International Conference on Machine Learning, 1842--1850."},{"key":"e_1_2_1_31_1","volume-title":"Discrete Graph Structure Learning for Forecasting Multiple Time Series. International Conference on Learning Representations.","author":"Shang Chao","year":"2021","unstructured":"Chao Shang and Jie Chen. 2021. Discrete Graph Structure Learning for Forecasting Multiple Time Series. International Conference on Learning Representations."},{"key":"e_1_2_1_32_1","volume-title":"ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 1567--1577","author":"Shao Zezhi","year":"2022","unstructured":"Zezhi Shao, Zhao Zhang, Fei Wang, and Yongjun Xu. 2022. Pre-training enhanced spatial-temporal graph neural network for multivariate time series forecasting. ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 1567--1577."},{"key":"e_1_2_1_33_1","volume-title":"Memto: Memory-guided transformer for multivariate time series anomaly detection. Advances in Neural Information Processing Systems 36","author":"Song Junho","year":"2024","unstructured":"Junho Song, Keonwoo Kim, Jeonglyul Oh, and Sungzoon Cho. 2024. Memto: Memory-guided transformer for multivariate time series anomaly detection. Advances in Neural Information Processing Systems 36 (2024)."},{"key":"e_1_2_1_34_1","unstructured":"Sainbayar Sukhbaatar Jason Weston Rob Fergus et al. 2015. End-to-end memory networks. Advances in Neural Information Processing Systems 28 (2015)."},{"key":"e_1_2_1_35_1","volume-title":"Attention is all you need. Advances in Neural Information Processing Systems 30","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in Neural Information Processing Systems 30 (2017)."},{"key":"e_1_2_1_36_1","volume-title":"CARD: Channel Aligned Robust Blend Transformer for Time Series Forecasting. International Conference on Learning Representations.","author":"Wang Xue","year":"2024","unstructured":"Xue Wang, Tian Zhou, Qingsong Wen, Jinyang Gao, Bolin Ding, and Rong Jin. 2024. CARD: Channel Aligned Robust Blend Transformer for Time Series Forecasting. International Conference on Learning Representations."},{"key":"e_1_2_1_37_1","volume-title":"AAAI Conference on Artificial Intelligence 36","author":"Wang Zhaonan","year":"2022","unstructured":"Zhaonan Wang, Renhe Jiang, Hao Xue, Flora D Salim, Xuan Song, and Ryosuke Shibasaki. 2022. Event-aware multimodal mobility nowcasting. AAAI Conference on Artificial Intelligence 36, 4, 4228--4236."},{"key":"e_1_2_1_38_1","volume-title":"ACM International Conference on Information & Knowledge Management, 3523--3527","author":"Wang Zhen","year":"2021","unstructured":"Zhen Wang, Yang Zhang, Ai Jiang, Ji Zhang, Zhao Li, Jun Gao, Ke Li, Chenhao Lu, and Zujie Ren. 2021. Improving irregularly sampled time series learning with time-aware dual-attention memory-augmented networks. ACM International Conference on Information & Knowledge Management, 3523--3527."},{"key":"e_1_2_1_39_1","first-page":"22419","article-title":"Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting","volume":"34","author":"Wu Haixu","year":"2021","unstructured":"Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. 2021. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Advances in Neural Information Processing Systems 34 (2021), 22419--22430.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-024-00872-x"},{"key":"e_1_2_1_41_1","volume-title":"Proc. VLDB Endow. 18","author":"Wu Xinle","year":"2024","unstructured":"Xinle Wu, Xingjian Wu, Dalin Zhang, Miao Zhang, Chenjuan Guo, Bin Yang, and Christian S. Jensen. 2024. Fully Automated Correlated Time Series Forecasting in Minutes. Proc. VLDB Endow. 18 (2024)."},{"key":"e_1_2_1_42_1","volume-title":"ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 753--763","author":"Wu Zonghan","year":"2020","unstructured":"Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang. 2020. Connecting the dots: Multivariate time series forecasting with graph neural networks. ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 753--763."},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2019\/264"},{"key":"e_1_2_1_44_1","volume-title":"International Conference on Machine Learning, 2397--2406","author":"Xiong Caiming","year":"2016","unstructured":"Caiming Xiong, Stephen Merity, and Richard Socher. 2016. Dynamic memory networks for visual and textual question answering. International Conference on Machine Learning, 2397--2406."},{"key":"e_1_2_1_45_1","volume-title":"AAAI Conference on Artificial Intelligence 35","author":"Ye Junchen","year":"2021","unstructured":"Junchen Ye, Leilei Sun, Bowen Du, Yanjie Fu, and Hui Xiong. 2021. Coupled layer-wise graph convolution for transportation demand prediction. AAAI Conference on Artificial Intelligence 35, 5, 4617--4625."},{"key":"e_1_2_1_46_1","volume-title":"AAAI Conference on Artificial Intelligence 37","author":"Zeng Ailing","year":"2023","unstructured":"Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. 2023. Are transformers effective for time series forecasting? AAAI Conference on Artificial Intelligence 37, 9, 11121--11128."},{"key":"e_1_2_1_47_1","volume-title":"International Conference on Learning Representations.","author":"Zhang Yunhao","year":"2023","unstructured":"Yunhao Zhang and Junchi Yan. 2023. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. International Conference on Learning Representations."},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.14778\/3636218.3636230"},{"key":"e_1_2_1_49_1","volume-title":"AAAI Conference on Artificial Intelligence 35","author":"Zhou Haoyi","year":"2021","unstructured":"Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. 2021. Informer: Beyond efficient transformer for long sequence time-series forecasting. AAAI Conference on Artificial Intelligence 35, 12, 11106--11115."},{"key":"e_1_2_1_50_1","volume-title":"International Conference on Machine Learning, 27268--27286","author":"Zhou Tian","year":"2022","unstructured":"Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. 2022. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. International Conference on Machine Learning, 27268--27286."}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3705829.3705842","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,2,28]],"date-time":"2025-02-28T23:21:57Z","timestamp":1740784917000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3705829.3705842"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10]]},"references-count":50,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,10]]}},"alternative-id":["10.14778\/3705829.3705842"],"URL":"https:\/\/doi.org\/10.14778\/3705829.3705842","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2024,10]]},"assertion":[{"value":"2025-02-28","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}