{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,26]],"date-time":"2026-03-26T03:22:00Z","timestamp":1774495320738,"version":"3.50.1"},"reference-count":48,"publisher":"MDPI AG","issue":"24","license":[{"start":{"date-parts":[[2020,12,16]],"date-time":"2020-12-16T00:00:00Z","timestamp":1608076800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100002851","name":"China Academy of Engineering Physics","doi-asserted-by":"publisher","award":["SJ2019A05"],"award-info":[{"award-number":["SJ2019A05"]}],"id":[{"id":"10.13039\/501100002851","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Time series classification and forecasting have long been studied with the traditional statistical methods. Recently, deep learning achieved remarkable successes in areas such as image, text, video, audio processing, etc. However, research studies conducted with deep neural networks in these fields are not abundant. Therefore, in this paper, we aim to propose and evaluate several state-of-the-art neural network models in these fields. We first review the basics of representative models, namely long short-term memory and its variants, the temporal convolutional network and the generative adversarial network. Then, long short-term memory with autoencoder and attention-based models, the temporal convolutional network and the generative adversarial model are proposed and applied to time series classification and forecasting. Gaussian sliding window weights are proposed to speed the training process up. Finally, the performances of the proposed methods are assessed using five optimizers and loss functions with the public benchmark datasets, and comparisons between the proposed temporal convolutional network and several classical models are conducted. Experiments show the proposed models\u2019 effectiveness and confirm that the temporal convolutional network is superior to long short-term memory models in sequence modeling. We conclude that the proposed temporal convolutional network reduces time consumption to around 80% compared to others while retaining the same accuracy. The unstable training process for generative adversarial network is circumvented by tuning hyperparameters and carefully choosing the appropriate optimizer of \u201cAdam\u201d. The proposed generative adversarial network also achieves comparable forecasting accuracy with traditional methods.<\/jats:p>","DOI":"10.3390\/s20247211","type":"journal-article","created":{"date-parts":[[2020,12,16]],"date-time":"2020-12-16T09:21:15Z","timestamp":1608110475000},"page":"7211","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":52,"title":["Time Series Forecasting and Classification Models Based on Recurrent with Attention Mechanism and Generative Adversarial Networks"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7926-6017","authenticated-orcid":false,"given":"Kun","family":"Zhou","sequence":"first","affiliation":[{"name":"School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China"},{"name":"Institute of Computer Application, China Academy of Engineering Physics, Mianyang 621900, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4095-547X","authenticated-orcid":false,"given":"Wenyong","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8624-0210","authenticated-orcid":false,"given":"Teng","family":"Hu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China"},{"name":"Institute of Computer Application, China Academy of Engineering Physics, Mianyang 621900, China"}]},{"given":"Kai","family":"Deng","sequence":"additional","affiliation":[{"name":"Institute of Computer Application, China Academy of Engineering Physics, Mianyang 621900, China"}]}],"member":"1968","published-online":{"date-parts":[[2020,12,16]]},"reference":[{"key":"ref_1","unstructured":"Kleinbaum, D.G., Dietz, K., Gail, M., Klein, M., and Klein, M. (2002). Logistic Regression, Springer."},{"key":"ref_2","unstructured":"Rish, I. (2001, January 4\u20136). An empirical study of the naive Bayes classifier. Proceedings of the IJCAI 2001 Workshop on Empirical Methods in Artificial Intelligence, Seattle, WA, USA."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/5254.708428","article-title":"Support vector machines","volume":"13","author":"Hearst","year":"1998","journal-title":"IEEE Intell. Syst. Their Appl."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1883","DOI":"10.4249\/scholarpedia.1883","article-title":"K-nearest neighbor","volume":"4","author":"Peterson","year":"2009","journal-title":"Scholarpedia"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Adam, S.P., Alexandropoulos, S.A.N., Pardalos, P.M., and Vrahatis, M.N. (2019). No free lunch theorem: A review. Approximation and Optimization, Springer.","DOI":"10.1007\/978-3-030-12767-1_5"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1717","DOI":"10.1016\/j.jbusres.2015.03.031","article-title":"Golden rule of forecasting: Be conservative","volume":"68","author":"Armstrong","year":"2015","journal-title":"J. Bus. Res."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"917","DOI":"10.1007\/s10618-019-00619-1","article-title":"Deep learning for time series classification: A review","volume":"33","author":"Forestier","year":"2019","journal-title":"Data Min. Knowl. Discov."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Wang, Z., Yan, W., and Oates, T. (2017, January 14\u201319). Time series classification from scratch with deep neural networks: A strong baseline. Proceedings of the International Joint Conference on Neural Networks, Anchorage, AK, USA.","DOI":"10.1109\/IJCNN.2017.7966039"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1936","DOI":"10.1007\/s10618-020-00710-y","article-title":"InceptionTime: Finding AlexNet for Time Series Classification","volume":"34","author":"Fawaz","year":"2020","journal-title":"Data Min. Knowl. Discov."},{"key":"ref_10","unstructured":"(2020, December 15). Time Series Classification Repository. Available online: http:\/\/timeseriesclassification.com\/index.php."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1016\/S0925-2312(01)00702-0","article-title":"Time series forecasting using a hybrid ARIMA and neural network model","volume":"50","author":"Zhang","year":"2003","journal-title":"Neurocomputing"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Koehler, J., and Kuenzer, C. (2020). Forecasting Spatio-Temporal Dynamics on the Land Surface Using Earth Observation Data\u2014A Review. Remote Sens., 12.","DOI":"10.3390\/rs12213513"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ghaderpour, E., and Vujadinovic, T. (2020). The Potential of the Least-Squares Spectral and Cross-Wavelet Analyses for Near-Real-Time Disturbance Detection within Unequally Spaced Satellite Image Time Series. Remote Sens., 12.","DOI":"10.3390\/rs12152446"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1875","DOI":"10.1109\/TMM.2015.2477044","article-title":"Describing Multimedia Content Using Attention-Based Encoder-Decoder Networks","volume":"17","author":"Cho","year":"2015","journal-title":"IEEE Trans. Multimed."},{"key":"ref_15","unstructured":"Bahdanau, D., Cho, K.H., and Bengio, Y. (2015, January 7\u20139). Neural machine translation by jointly learning to align and translate. Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015\u2014Conference Track Proceedings, San Diego, CA, USA."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Cheng, J., Dong, L., and Lapata, M. (2016, January 1\u20135). Long short-term memory-networks for machine reading. Proceedings of the EMNLP 2016\u2014Conference on Empirical Methods in Natural Language Processing, Austin, TX, USA.","DOI":"10.18653\/v1\/D16-1053"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Graves, A., Mohamed, A.R., and Hinton, G. (2013, January 26\u201331). Speech recognition with deep recurrent neural networks. Proceedings of the ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing\u2014Proceedings, Vancouver, BC, Canada.","DOI":"10.1109\/ICASSP.2013.6638947"},{"key":"ref_18","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017, January 4\u20139). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"28309","DOI":"10.1109\/ACCESS.2019.2901920","article-title":"A Model Combining Convolutional Neural Network and LightGBM Algorithm for Ultra-Short-Term Wind Power Forecasting","volume":"7","author":"Ju","year":"2019","journal-title":"IEEE Access"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"668","DOI":"10.1109\/TNN.2003.810601","article-title":"An empirical investigation of bias and variance in time series forecasting: Modeling considerations and error evaluation","volume":"14","author":"Berardi","year":"2003","journal-title":"IEEE Trans. Neural Networks"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"3026","DOI":"10.1109\/TKDE.2014.2316504","article-title":"Highly Comparative Feature-Based Time-Series Classification","volume":"26","author":"Fulcher","year":"2014","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1363","DOI":"10.1109\/TCYB.2015.2426723","article-title":"Learning a Mahalanobis Distance-Based Dynamic Time Warping Measure for Multivariate Time Series Classification","volume":"46","author":"Mei","year":"2016","journal-title":"IEEE Trans. Cybern."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"18940","DOI":"10.1109\/ACCESS.2019.2895688","article-title":"Multilevel Weighted Feature Fusion Using Convolutional Neural Networks for EEG Motor Imagery Classification","volume":"7","author":"Amin","year":"2019","journal-title":"IEEE Access"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1109\/TKDE.2015.2462369","article-title":"Similarity Measure Selection for Clustering Time Series Databases","volume":"28","author":"Mori","year":"2016","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"4788","DOI":"10.1109\/TIE.2018.2864702","article-title":"Time Series Classification With Multivariate Convolutional Neural Network","volume":"66","author":"Liu","year":"2019","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_26","first-page":"2672","article-title":"Generative adversarial nets","volume":"3","author":"Goodfellow","year":"2014","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_27","unstructured":"Arjovsky, M., Chintala, S., and Bottou, L. (2017, January 6\u201311). Wasserstein generative adversarial networks. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Yu, L., Zhang, W., Wang, J., and Yu, Y. (2017, January 4\u201310). SeqGAN: Sequence generative adversarial nets with policy gradient. Proceedings of the 31st AAAI Conference on Artificial Intelligence (AAAI 2017), San Francisco, CA, USA.","DOI":"10.1609\/aaai.v31i1.10804"},{"key":"ref_29","unstructured":"Gong, X., Chang, S., Jiang, Y., and Wang, Z. (November, January 27). Autogan: Neural architecture search for generative adversarial networks. Proceedings of the IEEE International Conference on Computer Vision, Seoul, Korea."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1947","DOI":"10.1109\/TPAMI.2018.2856256","article-title":"StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks","volume":"41","author":"Zhang","year":"2019","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_31","first-page":"5768","article-title":"Improved training of wasserstein GANs","volume":"30","author":"Gulrajani","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_32","first-page":"2234","article-title":"Improved techniques for training GANs","volume":"29","author":"Salimans","year":"2016","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_33","unstructured":"Che, T., Li, Y., Jacob, A.P., Bengio, Y., and Li, W. (2016). Mode regularized generative adversarial networks. arXiv."},{"key":"ref_34","unstructured":"Mescheder, L., Geiger, A., and Nowozin, S. (2018, January 10\u201315). Which training methods for GANs do actually converge?. Proceedings of the 35th International Conference on Machine Learning (ICML 2018), Stockholm, Sweden."},{"key":"ref_35","unstructured":"Arora, S., Ge, R., Liang, Y., Ma, T., and Zhang, Y. (2017, January 6\u201311). Generalization and equilibrium in generative adversarial nets (GANs). Proceedings of the 34th International Conference on Machine Learning (ICML 2017), Sydney, Australia."},{"key":"ref_36","unstructured":"Che, T., Li, Y., Zhang, R., Hjelm, R.D., Li, W., Song, Y., and Bengio, Y. (2017). Maximum-likelihood augmented discrete generative adversarial networks. arXiv."},{"key":"ref_37","unstructured":"Kusner, M.J., and Hern\u00e1ndez-Lobato, J.M. (2016). Gans for sequences of discrete elements with the gumbel-softmax distribution. arXiv."},{"key":"ref_38","unstructured":"Bai, S., Kolter, J.Z., and Koltun, V. (2018). An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long Short-term Memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_40","unstructured":"(2020, December 15). Kaggle Web Traffic Competition. Available online: https:\/\/www.kaggle.com\/c\/web-traffic-time-series-forecasting."},{"key":"ref_41","unstructured":"(2020, December 15). VPN-nonVPN Dataset from Canadian Institute of Cyber-Security. Available online: https:\/\/www.unb.ca\/cic\/datasets\/vpn.html."},{"key":"ref_42","unstructured":"Le, Q.V., Jaitly, N., and Hinton, G.E. (2015). A simple way to initialize recurrent networks of rectified linear units. arXiv."},{"key":"ref_43","unstructured":"Zhang, S., Wu, Y., Che, T., Lin, Z., Memisevic, R., Salakhutdinov, R.R., and Bengio, Y. (2016, January 5\u201310). Architectural complexity measures of recurrent neural networks. Proceedings of the Advances in Neural Information Processing Systems, Barcelona, Spain."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Press, O., Smith, N.A., and Levy, O. (2019). Improving Transformer Models by Reordering their Sublayers. arXiv.","DOI":"10.18653\/v1\/2020.acl-main.270"},{"key":"ref_45","unstructured":"Ba, J.L., Kiros, J.R., and Hinton, G.E. (2016). Layer normalization. arXiv."},{"key":"ref_46","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_47","unstructured":"Dehghani, M., Gouws, S., Vinyals, O., Uszkoreit, J., and Kaiser, \u0141. (2018). Universal transformers. arXiv."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Draguns, A., Ozoli\u0146\u0161, E., \u0160ostaks, A., Apinis, M., and Freivalds, K. (2020). Residual Shuffle-Exchange Networks for Fast Processing of Long Sequences. arXiv.","DOI":"10.1609\/aaai.v35i8.16890"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/24\/7211\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:45:56Z","timestamp":1760179556000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/24\/7211"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12,16]]},"references-count":48,"journal-issue":{"issue":"24","published-online":{"date-parts":[[2020,12]]}},"alternative-id":["s20247211"],"URL":"https:\/\/doi.org\/10.3390\/s20247211","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,12,16]]}}}