{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,8]],"date-time":"2026-08-08T17:19:04Z","timestamp":1786209544088,"version":"3.56.0"},"reference-count":47,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2022,1,22]],"date-time":"2022-01-22T00:00:00Z","timestamp":1642809600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Key R&amp;D Program of China","award":["2018YFB0105100"],"award-info":[{"award-number":["2018YFB0105100"]}]},{"name":"Science and Technology Innovation Committee of Shenzhen","award":["CJGJZD2020061710281005"],"award-info":[{"award-number":["CJGJZD2020061710281005"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Multivariate time series forecasting has long been a research hotspot because of its wide range of application scenarios. However, the dynamics and multiple patterns of spatiotemporal dependencies make this problem challenging. Most existing methods suffer from two major shortcomings: (1) They ignore the local context semantics when modeling temporal dependencies. (2) They lack the ability to capture the spatial dependencies of multiple patterns. To tackle such issues, we propose a novel Transformer-based model for multivariate time series forecasting, called the spatial\u2013temporal convolutional Transformer network (STCTN). STCTN mainly consists of two novel attention mechanisms to respectively model temporal and spatial dependencies. Local-range convolutional attention mechanism is proposed in STCTN to simultaneously focus on both global and local context temporal dependencies at the sequence level, which addresses the first shortcoming. Group-range convolutional attention mechanism is designed to model multiple spatial dependency patterns at graph level, as well as reduce the computation and memory complexity, which addresses the second shortcoming. Continuous positional encoding is proposed to link the historical observations and predicted future values in positional encoding, which also improves the forecasting performance. Extensive experiments on six real-world datasets show that the proposed STCTN outperforms the start-of-the-art methods and is more robust to nonsmooth time series data.<\/jats:p>","DOI":"10.3390\/s22030841","type":"journal-article","created":{"date-parts":[[2022,1,23]],"date-time":"2022-01-23T20:36:27Z","timestamp":1642970187000},"page":"841","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":45,"title":["Spatial-Temporal Convolutional Transformer Network for Multivariate Time Series Forecasting"],"prefix":"10.3390","volume":"22","author":[{"given":"Lei","family":"Huang","sequence":"first","affiliation":[{"name":"Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China"},{"name":"Department of Automation, Tsinghua University, Beijing 100086, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Feng","family":"Mao","sequence":"additional","affiliation":[{"name":"Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China"},{"name":"Department of Automation, Tsinghua University, Beijing 100086, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kai","family":"Zhang","sequence":"additional","affiliation":[{"name":"Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China"},{"name":"Research Institute of Tsinghua, Pearl River Delta, Guangzhou 510530, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhiheng","family":"Li","sequence":"additional","affiliation":[{"name":"Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,1,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"106181","DOI":"10.1016\/j.asoc.2020.106181","article-title":"Financial time series forecasting with deep learning: A systematic literature review: 2005\u20132019","volume":"90","author":"Sezer","year":"2020","journal-title":"Appl. Soft Comput."},{"key":"ref_2","first-page":"20200209","article-title":"Time-series forecasting with deep learning: A survey","volume":"379","author":"Lim","year":"2021","journal-title":"Philos. Trans. R. Soc. London. Ser. A Math. Phys. Eng. Sci."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"334","DOI":"10.1016\/j.fcij.2018.10.003","article-title":"Time series forecasting using artificial neural networks methodologies: A systematic review","volume":"3","author":"Tealab","year":"2018","journal-title":"Futur. Comput. Inform. J."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Wu, Z., Pan, S., Long, G., Jiang, J., and Zhang, C. (2019, January 10\u201316). Graph WaveNet for Deep Spatial-Temporal Graph Modeling. Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19), Macao, China.","DOI":"10.24963\/ijcai.2019\/264"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Lu, B., Gan, X., Jin, H., Fu, L., and Zhang, H. (2020, January 19\u201323). Spatiotemporal Adaptive Gated Graph Convolution Network for Urban Traffic Flow Forecasting. Proceedings of the 29th ACM International Conference on Information & Knowledge Management, Virtual Event, Ireland.","DOI":"10.1145\/3340531.3411894"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1016\/S0925-2312(01)00702-0","article-title":"Time series forecasting using a hybrid ARIMA and neural network model","volume":"50","author":"Zhang","year":"2003","journal-title":"Neurocomputing"},{"key":"ref_7","unstructured":"Zivot, E., and Wang, J. (2006). Vector Autoregressive Models for Multivariate Time Series. Modeling Financial Time Series with S-Plus\u00ae, Springer."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Yang, H., Jiang, Z., and Lu, H. (2017). A Hybrid Wind Speed Forecasting System Based on a \u2018Decomposition and Ensemble\u2019 Strategy and Fuzzy Time Series. Energies, 10.","DOI":"10.3390\/en10091422"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"276","DOI":"10.1109\/TITS.2004.837813","article-title":"Travel-Time Prediction with Support Vector Regression","volume":"5","author":"Wu","year":"2004","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1049\/iet-its.2016.0208","article-title":"LSTM network: A deep learning approach for short-term traffic forecast","volume":"11","author":"Zhao","year":"2017","journal-title":"IET Intell. Transp. Syst."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Lai, G., Chang, W.-C., Yang, Y., and Liu, H. (2018, January 8\u201312). Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks. Proceedings of the SIGIR \u201918: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, Ann Arbor, MI, USA.","DOI":"10.1145\/3209978.3210006"},{"key":"ref_12","first-page":"79","article-title":"A novel dbn model for time series forecasting","volume":"44","author":"Ren","year":"2017","journal-title":"IAENG Int. J. Comput. Sci."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1016\/j.artint.2018.03.002","article-title":"Predicting citywide crowd flows using deep spatio-temporal residual networks","volume":"259","author":"Zhang","year":"2018","journal-title":"Artif. Intell."},{"key":"ref_14","unstructured":"Li, Y., Yu, R., Shahabi, C., and Liu, Y. (May, January 30). Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Yu, B., Yin, H., and Zhu, Z. (2018, January 13\u201319). Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting. Proceedings of the 27th International Joint Conference on Artificial Intelligence, Stockholm, Sweden.","DOI":"10.24963\/ijcai.2018\/505"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Guo, S., Lin, Y., Feng, N., Song, C., and Wan, H. (2019, January 27). Attention Based Spatial-Temporal Graph Convolutional Networks for Traffic Flow Forecasting. Proceedings of the 33rd AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.","DOI":"10.1609\/aaai.v33i01.3301922"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Wu, Z., Pan, S., Long, G., Jiang, J., Chang, X., and Zhang, C. (2020, January 6\u201310). Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Virtual.","DOI":"10.1145\/3394486.3403118"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Song, C., Lin, Y., Guo, S., and Wan, H. (2020, January 7\u201312). Spatial-Temporal Synchronous Graph Convolutional Networks: A New Framework for Spatial-Temporal Network Data Forecasting. Proceedings of the 34th AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i01.5438"},{"key":"ref_19","unstructured":"Cao, D., Wang, Y., Duan, J., Zhang, C., Zhu, X., Huang, C., Tong, Y., Xu, B., Bai, J., and Tong, J. (2021, January 6\u201314). Spectral temporal graph neural network for multivariate time-series forecasting. Proceedings of the Thirty-fifth Conference on Neural Information Processing Systems, Virtual Event, Ireland."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"3430","DOI":"10.1109\/TITS.2015.2457240","article-title":"Trend Modeling for Traffic Time Series Analysis: An Integrated Study","volume":"16","author":"Li","year":"2015","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1016\/j.trc.2011.12.006","article-title":"The retrieval of intra-day trend and its influence on traffic prediction","volume":"22","author":"Chen","year":"2012","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_22","unstructured":"Shi, X., Chen, Z., Wang, H., Yeung, D.Y., Wong, W.K., and Woo, W.C. (2015, January 7\u201312). Convolutional LSTM network: A machine learning approach for precipitation nowcasting. Proceedings of the Twenty-ninth Conference on Neural Information Processing Systems Montr\u00e9al, Montreal, QC, Canada."},{"key":"ref_23","first-page":"2627","article-title":"A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction","volume":"366","author":"Qin","year":"2017","journal-title":"IJCAI"},{"key":"ref_24","unstructured":"Yu, R., Zheng, S., Anandkumar, A., and Yue, Y. (2017). Long-term forecasting using higher order tensor RNNs. arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Zhang, L., Aggarwal, C., and Qi, G.-J. (2017, January 13\u201317). Stock Price Prediction via Discovering Multi-Frequency Trading Patterns. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada.","DOI":"10.1145\/3097983.3098117"},{"key":"ref_26","first-page":"847","article-title":"Temporal regularized matrix factorization for high-dimensional time series prediction","volume":"29","author":"Yu","year":"2016","journal-title":"Adv. Neural Inf. Processing Syst."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1145\/3065386","article-title":"Imagenet classification with deep convolutional neural networks","volume":"60","author":"Krizhevsky","year":"2017","journal-title":"Commun. ACM"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1526","DOI":"10.1587\/transinf.2018EDP7330","article-title":"MF-CNN: Traffic Flow Prediction Using Convolutional Neural Network and Multi-Features Fusion","volume":"102","author":"Yang","year":"2019","journal-title":"IEICE Trans. Inf. Syst."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Yu, H., Wu, Z., Wang, S., Wang, Y., and Ma, X. (2017). Spatiotemporal Recurrent Convolutional Networks for Traffic Prediction in Transportation Networks. Sensors, 17.","DOI":"10.3390\/s17071501"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Lv, Z., Xu, J., Zheng, K., Yin, H., Zhao, P., and Zhou, X. (2018, January 13\u201319). LC-RNN: A Deep Learning Model for Traffic Speed Prediction. Proceedings of the 27 International Joint Conference on Artificial Intelligence, Stockholm, Sweden.","DOI":"10.24963\/ijcai.2018\/482"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Ma, X., Dai, Z., He, Z., Ma, J., Wang, Y., and Wang, Y. (2017). Learning Traffic as Images: A Deep Convolutional Neural Network for Large-Scale Transportation Network Speed Prediction. Sensors, 17.","DOI":"10.3390\/s17040818"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Chen, W., Chen, L., Xie, Y., Cao, W., Gao, Y., and Feng, X. (2020, January 7\u201312). Multi-Range Attentive Bicomponent Graph Convolutional Network for Traffic Forecasting. Proceedings of the 34 th AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i04.5758"},{"key":"ref_33","unstructured":"Kipf, T.N., and Welling, M. (2017). Semi-Supervised Classification with Graph Convolutional Networks. arXiv."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Zheng, C., Fan, X., Wang, C., and Qi, J. (2020, January 7\u201312). GMAN: A Graph Multi-Attention Network for Traffic Prediction. Proceedings of the 34th AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i01.5477"},{"key":"ref_35","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017, January 4\u20139). Attention is all you need. Proceedings of the Thirty-first Conference on Neural Information Processing Systems, Long Beach, LA, USA."},{"key":"ref_36","first-page":"2232","article-title":"A tensorized transformer for language modeling","volume":"32","author":"Ma","year":"2019","journal-title":"Adv. Neural Inf. Processing Syst."},{"key":"ref_37","unstructured":"Parmar, N., Vaswani, A., Uszkoreit, J., Kaiser, L., Shazeer, N., Ku, A., and Tran, D. (2018, January 10\u201315). Image transformer. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_38","first-page":"5243","article-title":"Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting","volume":"32","author":"Li","year":"2019","journal-title":"Adv. Neural Inf. Processing Syst."},{"key":"ref_39","unstructured":"Wu, S., Xiao, X., Ding, Q., Zhao, P., Ying WE, I., and Huang, J. (2020, January 6\u201312). Adversarial Sparse Transformer for Time Series Forecasting. Proceedings of the Thirty-fourth Conference on Neural Information Processing Systems, Virtual Event, Ireland."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"2596","DOI":"10.1109\/TMI.2019.2905990","article-title":"TeTrIS: Template Transformer Networks for Image Segmentation with Shape Priors","volume":"38","author":"Lee","year":"2019","journal-title":"IEEE Trans. Med Imaging"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"736","DOI":"10.1111\/tgis.12644","article-title":"Traffic transformer: Capturing the continuity and periodicity of time series for traffic forecasting","volume":"24","author":"Cai","year":"2020","journal-title":"Trans. GIS"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Zhou, K., Wang, W., Hu, T., and Deng, K. (2020). Time Series Forecasting and Classification Models Based on Recurrent with Attention Mechanism and Generative Adversarial Networks. Sensors, 20.","DOI":"10.3390\/s20247211"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., and Zhang, W. (2021, January 2\u20139). Informer: Beyond efficient transformer for long sequence time-series forecasting. Proceedings of the 35th AAAI Conference on Artificial Intelligence, Virtual Event, Ireland.","DOI":"10.1609\/aaai.v35i12.17325"},{"key":"ref_44","unstructured":"Liu, Z., Luo, S., Li, W., Lu, J., Wu, Y., Sun, S., Li, C., and Yang, L. (2020). Convtransformer: A convolutional transformer network for video frame synthesis. arXiv."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"96","DOI":"10.3141\/1748-12","article-title":"Freeway Performance Measurement System: Mining Loop Detector Data","volume":"1748","author":"Chen","year":"2001","journal-title":"Transp. Res. Rec. J. Transp. Res. Board"},{"key":"ref_46","first-page":"7785","article-title":"Deep state space models for time series forecasting","volume":"31","author":"Rangapuram","year":"2018","journal-title":"Adv. Neural Inf. Processing Syst."},{"key":"ref_47","unstructured":"Oreshkin, B.N., Carpov, D., Chapados, N., and Bengio, Y. (2019, January 6\u20139). N-BEATS: Neural basis expansion analysis for interpretable time series forecasting. Proceedings of the International Conference on Learning Representations, New Orleans, LA, USA."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/3\/841\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:05:59Z","timestamp":1760133959000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/3\/841"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,22]]},"references-count":47,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2022,2]]}},"alternative-id":["s22030841"],"URL":"https:\/\/doi.org\/10.3390\/s22030841","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1,22]]}}}