{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,4]],"date-time":"2025-12-04T14:47:05Z","timestamp":1764859625307,"version":"3.37.3"},"reference-count":38,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2024,3,1]],"date-time":"2024-03-01T00:00:00Z","timestamp":1709251200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,3,15]],"date-time":"2024-03-15T00:00:00Z","timestamp":1710460800000},"content-version":"vor","delay-in-days":14,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Appl Intell"],"published-print":{"date-parts":[[2024,3]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Multivariate time series forecasting is essential in various fields, including healthcare and traffic management, but it is a challenging task due to the strong dynamics in both <jats:italic>intra-channel relations<\/jats:italic> (temporal patterns within individual variables) and <jats:italic>inter-channel relations<\/jats:italic> (the relationships between variables), which can evolve over time with abrupt changes. This paper proposes ERAN (Evolving Relational Attention Network), a framework for multivariate time series forecasting, that is capable to capture such dynamics of these relations. On the one hand, ERAN represents inter-channel relations with a graph which evolves over time, modeled using a recurrent neural network. On the other hand, ERAN represents the intra-channel relations using a temporal attentional convolution, which captures the local temporal dependencies adaptively with the input data. The elvoving graph structure and the temporal attentional convolution are intergrated in a unified model to capture both types of relations. The model is experimented on a large number of real-life datasets including traffic flows, energy consumption, and COVID-19 transmission data. The experimental results show a significant improvement over the state-of-the-art methods in multivariate time series forecasting particularly for non-stationary data.<\/jats:p>","DOI":"10.1007\/s10489-023-05220-0","type":"journal-article","created":{"date-parts":[[2024,3,15]],"date-time":"2024-03-15T13:01:41Z","timestamp":1710507701000},"page":"3918-3932","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Learning evolving relations for multivariate time series forecasting"],"prefix":"10.1007","volume":"54","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8243-4894","authenticated-orcid":false,"given":"Binh","family":"Nguyen-Thai","sequence":"first","affiliation":[]},{"given":"Vuong","family":"Le","sequence":"additional","affiliation":[]},{"given":"Ngoc-Dung T.","family":"Tieu","sequence":"additional","affiliation":[]},{"given":"Truyen","family":"Tran","sequence":"additional","affiliation":[]},{"given":"Svetha","family":"Venkatesh","sequence":"additional","affiliation":[]},{"given":"Naeem","family":"Ramzan","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,3,15]]},"reference":[{"key":"5220_CR1","unstructured":"Alexandrov A, Benidis K, Bohlke-Schneider M et\u00a0al (2020) Gluonts: probabilistic and neural time series modeling in python. J Mach Learn Res 21(116):1\u20136. http:\/\/jmlr.org\/papers\/v21\/19-820.html"},{"key":"5220_CR2","unstructured":"Bai L, Yao L, Li C et\u00a0al (2020) Adaptive graph convolutional recurrent network for traffic forecasting. In: Larochelle H, Ranzato M, Hadsell R et\u00a0al (eds) Advances in neural information processing systems, vol\u00a033. Curran Associates, Inc., pp 17,804\u201317,815"},{"key":"5220_CR3","unstructured":"Bai S, Kolter JZ, Koltun V (2018) An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv:1803.01271"},{"key":"5220_CR4","unstructured":"Bruna J, Zaremba W, Szlam A et\u00a0al (2014) Spectral networks and locally connected networks on graphs. International conference on learning representations (ICLR 2014)"},{"key":"5220_CR5","unstructured":"Cao D, Wang Y, Duan J et\u00a0al (2020) Spectral temporal graph neural network for multivariate time-series forecasting. Adv Neural Inf Process Syst 33"},{"key":"5220_CR6","doi-asserted-by":"crossref","unstructured":"Cho K, van Merri\u00ebnboer B, Gulcehre C et\u00a0al (2014) Learning phrase representations using RNN encoder\u2013decoder for statistical machine translation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1724\u20131734","DOI":"10.3115\/v1\/D14-1179"},{"issue":"3","key":"5220_CR7","first-page":"299","volume":"2","author":"M Conti","year":"1994","unstructured":"Conti M, Turchetti C (1994) Approximation of dynamical systems by continuous-time recurrent approximate identity neural networks. Neural Parallel Sci Comput 2(3):299\u2013320","journal-title":"Neural Parallel Sci Comput"},{"key":"5220_CR8","unstructured":"Defferrard M, Bresson X, Vandergheynst P (2016) Convolutional neural networks on graphs with fast localized spectral filtering. In: Proceedings of the 30th international conference on neural information processing systems, NIPS\u201916, pp 3844\u20133852"},{"issue":"366","key":"5220_CR9","doi-asserted-by":"publisher","first-page":"427","DOI":"10.2307\/2286348","volume":"74","author":"DA Dickey","year":"1979","unstructured":"Dickey DA, Fuller WA (1979) Distribution of the estimators for autoregressive time series with a unit root. J Am Stat Assoc 74(366):427\u2013431","journal-title":"J Am Stat Assoc"},{"key":"5220_CR10","first-page":"821","volume-title":"Intelligent systems\u20192014","author":"G Dudek","year":"2015","unstructured":"Dudek G (2015) Short-term load forecasting using random forests. In: Filev D, Jab\u0142kowski J, Kacprzyk J et al (eds) Intelligent systems\u20192014. Springer International Publishing, Cham, pp 821\u2013828"},{"key":"5220_CR11","unstructured":"Gilmer J, Schoenholz SS, Riley PF et\u00a0al (2017) Neural message passing for quantum chemistry. In: Proceedings of the 34th international conference on machine learning, ICML\u201917, vol 70. pp 1263\u20131272"},{"key":"5220_CR12","doi-asserted-by":"crossref","unstructured":"Guo S, Lin Y, Feng N et\u00a0al (2019) Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. In: Proceedings of the AAAI conference on artificial intelligence, pp 922\u2013929","DOI":"10.1609\/aaai.v33i01.3301922"},{"key":"5220_CR13","doi-asserted-by":"publisher","unstructured":"He K, Zhang X, Ren S et\u00a0al (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 770\u2013778. https:\/\/doi.org\/10.1109\/CVPR.2016.90","DOI":"10.1109\/CVPR.2016.90"},{"issue":"8","key":"5220_CR14","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","volume":"9","author":"S Hochreiter","year":"1997","unstructured":"Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735\u20131780","journal-title":"Neural Comput"},{"key":"5220_CR15","volume-title":"Forecasting: principles and practice","author":"R Hyndman","year":"2021","unstructured":"Hyndman R, Athanasopoulos G (2021) Forecasting: principles and practice, 3rd edn. OTexts, Australia","edition":"3"},{"key":"5220_CR16","unstructured":"Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: Proceedings of the 5th international conference on learning representations, ICLR \u201917"},{"key":"5220_CR17","unstructured":"Kitaev N, Kaiser L, Levskaya A (2020) Reformer: the efficient transformer. In: International conference on learning representations. https:\/\/openreview.net\/forum?id=rkgNKkHtvB"},{"key":"5220_CR18","doi-asserted-by":"crossref","unstructured":"Lai G, Chang WC, Yang Y et\u00a0al (2018) Modeling long- and short-term temporal patterns with deep neural networks. In: The 41st international ACM SIGIR conference on research and development in information retrieval, SIGIR\u201918. New York, pp 95\u2013104","DOI":"10.1145\/3209978.3210006"},{"key":"5220_CR19","unstructured":"Li S, Jin X, Xuan Y et\u00a0al (2019) Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. In: Advances in neural information processing systems, pp 5243\u20135253"},{"key":"5220_CR20","unstructured":"Li Y, Yu R, Shahabi C et\u00a0al (2018) Diffusion convolutional recurrent neural network: data-driven traffic forecasting. In: International conference on learning representations (ICLR \u201918)"},{"key":"5220_CR21","doi-asserted-by":"crossref","unstructured":"Lim B, Ar\u0131k S\u00d6, Loeff N et al (2021) Temporal fusion transformers for interpretable multi-horizon time series forecasting. Int J Forecasting 37(4):1748\u20131764","DOI":"10.1016\/j.ijforecast.2021.03.012"},{"key":"5220_CR22","volume-title":"Time series techniques for economists","author":"TC Mills","year":"1990","unstructured":"Mills TC (1990) Time series techniques for economists. Cambridge University Press"},{"key":"5220_CR23","unstructured":"Nguyen D, Nguyen B, Nguyen P et\u00a0al (2021) High-order representation learning for multivariate time series forecasting. In: Time series workshop@ICML 2021"},{"key":"5220_CR24","unstructured":"Oord Avd, Dieleman S, Zen H et\u00a0al (2016) Wavenet: a generative model for raw audio. arXiv:1609.03499"},{"key":"5220_CR25","doi-asserted-by":"publisher","unstructured":"Pai PF, Lin KP, Lin CS et al (2010) Time series forecasting by a seasonal support vector regression model. Expert Syst Appl 37(6):4261\u20134265. https:\/\/doi.org\/10.1016\/j.eswa.2009.11.076, https:\/\/www.sciencedirect.com\/science\/article\/pii\/S0957417409010185","DOI":"10.1016\/j.eswa.2009.11.076"},{"key":"5220_CR26","doi-asserted-by":"crossref","unstructured":"Pareja A, Domeniconi G, Chen J et\u00a0al (2020) EvolveGCN: evolving graph convolutional networks for dynamic graphs. In: Proceedings of the thirty-fourth AAAI conference on artificial intelligence","DOI":"10.1609\/aaai.v34i04.5984"},{"key":"5220_CR27","doi-asserted-by":"crossref","unstructured":"Pham T, Tran T, Phung D et\u00a0al (2017) Column networks for collective classification. In: Proceedings of AAAI conference on artificial intelligence","DOI":"10.1609\/aaai.v31i1.10851"},{"key":"5220_CR28","unstructured":"Rangapuram SS, Seeger MW, Gasthaus J et\u00a0al (2018) Deep state space models for time series forecasting. In: Bengio S, Wallach H, Larochelle H et\u00a0al (eds) Advances in neural information processing systems, vol\u00a031. Curran Associates, Inc., https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2018file\/5cf68969fb67aa6082363a6d4e6468e2-Paper.pdf"},{"key":"5220_CR29","unstructured":"Rasul K, Seward C, Schuster I et\u00a0al (2021) Autoregressive denoising diffusion models for multivariate probabilistic time series forecasting. In: International conference on machine learning. https:\/\/api.semanticscholar.org\/CorpusID:231719657"},{"key":"5220_CR30","doi-asserted-by":"crossref","unstructured":"Salinas D, Flunkert V, Gasthaus J et al (2020) Deepar: probabilistic forecasting with autoregressive recurrent networks. Int J Forecasting 36(3):1181\u20131191. https:\/\/www.sciencedirect.com\/science\/article\/pii\/S0169207019301888","DOI":"10.1016\/j.ijforecast.2019.07.001"},{"key":"5220_CR31","doi-asserted-by":"publisher","unstructured":"Shaw P, Uszkoreit J, Vaswani A (2018) Self-attention with relative position representations. In: Proceedings of the 2018 conference of the North American chapter of the association for computational linguistics: human language technologies, vol 2 (Short Papers). Association for Computational Linguistics, New Orleans, pp 464\u2013468. https:\/\/doi.org\/10.18653\/v1\/N18-2074, https:\/\/aclanthology.org\/N18-2074","DOI":"10.18653\/v1\/N18-2074"},{"key":"5220_CR32","unstructured":"Vaswani A, Shazeer N, Parmar N et\u00a0al (2017) Attention is all you need. In: Proceedings of the 31st international conference on neural information processing systems, NIPS\u201917, pp 6000\u20136010"},{"key":"5220_CR33","unstructured":"Wu N, Green B, Ben X et\u00a0al (2020a) Deep transformer models for time series forecasting: the influenza prevalence case. arXiv:2001.08317"},{"key":"5220_CR34","doi-asserted-by":"crossref","unstructured":"Wu Z, Pan S, Long G et\u00a0al (2019) Graph WaveNet for deep spatial-temporal graph modeling. In: Proceedings of the twenty-eighth international joint conference on artificial intelligence, IJCAI-19. International Joint Conferences on Artificial Intelligence Organization, pp 1907\u20131913","DOI":"10.24963\/ijcai.2019\/264"},{"key":"5220_CR35","doi-asserted-by":"crossref","unstructured":"Wu Z, Pan S, Long G et\u00a0al (2020b) Connecting the dots: multivariate time series forecasting with graph neural networks. In: Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery and data mining. Association for Computing Machinery, New York, pp 753\u2013763","DOI":"10.1145\/3394486.3403118"},{"key":"5220_CR36","doi-asserted-by":"crossref","unstructured":"Yu B, Yin H, Zhu Z (2018) Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting. In: Proceedings of the 27th international joint conference on artificial intelligence (IJCAI)","DOI":"10.24963\/ijcai.2018\/505"},{"key":"5220_CR37","unstructured":"Zhang Y, Yan J (2023) Crossformer: transformer utilizing cross-dimension dependency for multivariate time series forecasting. In: International conference on learning representations"},{"key":"5220_CR38","doi-asserted-by":"crossref","unstructured":"Zhou H, Zhang S, Peng J et\u00a0al (2021) Informer: beyond efficient transformer for long sequence time-series forecasting. In: The thirty-fifth AAAI conference on artificial intelligence, AAAI 2021. AAAI Press, p online","DOI":"10.1609\/aaai.v35i12.17325"}],"container-title":["Applied Intelligence"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10489-023-05220-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10489-023-05220-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10489-023-05220-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,29]],"date-time":"2024-04-29T13:33:52Z","timestamp":1714397632000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10489-023-05220-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3]]},"references-count":38,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2024,3]]}},"alternative-id":["5220"],"URL":"https:\/\/doi.org\/10.1007\/s10489-023-05220-0","relation":{},"ISSN":["0924-669X","1573-7497"],"issn-type":[{"type":"print","value":"0924-669X"},{"type":"electronic","value":"1573-7497"}],"subject":[],"published":{"date-parts":[[2024,3]]},"assertion":[{"value":"6 December 2023","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 March 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"All datasets used in this paper are publicly available and no consent was required for their use.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical standard"}},{"value":"The authors have no conflicts of interest to declare that are relevant to the content of this article.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing Interests"}}]}}