{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,17]],"date-time":"2026-03-17T13:23:00Z","timestamp":1773753780699,"version":"3.50.1"},"reference-count":37,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2023,12,26]],"date-time":"2023-12-26T00:00:00Z","timestamp":1703548800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Institute of Information &amp; communications Technology Planning &amp; Evaluation (IITP)","award":["2021-0-01806"],"award-info":[{"award-number":["2021-0-01806"]}]},{"name":"Institute of Information &amp; communications Technology Planning &amp; Evaluation (IITP)","award":["20224B10100140"],"award-info":[{"award-number":["20224B10100140"]}]},{"DOI":"10.13039\/501100007053","name":"Korea Institute of Energy Technology Evaluation and Planning (KETEP)","doi-asserted-by":"publisher","award":["2021-0-01806"],"award-info":[{"award-number":["2021-0-01806"]}],"id":[{"id":"10.13039\/501100007053","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100007053","name":"Korea Institute of Energy Technology Evaluation and Planning (KETEP)","doi-asserted-by":"publisher","award":["20224B10100140"],"award-info":[{"award-number":["20224B10100140"]}],"id":[{"id":"10.13039\/501100007053","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Data scarcity is a significant obstacle for modern data science and artificial intelligence research communities. The fact that abundant data are a key element of a powerful prediction model is well known through various past studies. However, industrial control systems (ICS) are operated in a closed environment due to security and privacy issues, so collected data are generally not disclosed. In this environment, synthetic data generation can be a good alternative. However, ICS datasets have time-series characteristics and include features with short- and long-term temporal dependencies. In this paper, we propose the attention-based variational recurrent autoencoder (AVRAE) for generating time-series ICS data. We first extend the evidence lower bound of the variational inference to time-series data. Then, a recurrent neural-network-based autoencoder is designed to take this as the objective. AVRAE employs the attention mechanism to effectively learn the long-term and short-term temporal dependencies ICS data implies. Finally, we present an algorithm for generating synthetic ICS time-series data using learned AVRAE. In a comprehensive evaluation using the ICS dataset HAI and various performance indicators, AVRAE successfully generated visually and statistically plausible synthetic ICS data.<\/jats:p>","DOI":"10.3390\/s24010128","type":"journal-article","created":{"date-parts":[[2023,12,26]],"date-time":"2023-12-26T04:40:44Z","timestamp":1703565644000},"page":"128","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["A Synthetic Time-Series Generation Using a Variational Recurrent Autoencoder with an Attention Mechanism in an Industrial Control System"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7116-6062","authenticated-orcid":false,"given":"Seungho","family":"Jeon","sequence":"first","affiliation":[{"name":"Department of Computer Engineering (Smart Security), Gachon University, Seongnam-si 1342, Republic of Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0971-8548","authenticated-orcid":false,"given":"Jung Taek","family":"Seo","sequence":"additional","affiliation":[{"name":"Department of Computer Engineering, Gachon University, Seongnam-si 1342, Republic of Korea"}]}],"member":"1968","published-online":{"date-parts":[[2023,12,26]]},"reference":[{"key":"ref_1","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017). Attention is all you need. arXiv."},{"key":"ref_2","unstructured":"Radford, A., Narasimhan, K., Salimans, T., and Sutskever, I. (2023, December 25). Improving Language Understanding by Generative Pre-Training. OpenAI.Com. Available online: https:\/\/openai.com\/research\/language-unsupervised."},{"key":"ref_3","unstructured":"Radford, A., Narasimhan, K., Salimans, T., and Sutskever, I. (2023, December 25). Language Models Are Unsupervised Multitask Learners. OpenAI Blog. Available online: https:\/\/cdn.openai.com\/better-language-models\/language_models_are_unsupervised_multitask_learners.pdf."},{"key":"ref_4","unstructured":"Kingma, D.P., and Welling, M. (2014, January 14\u201316). Auto-encoding variational bayes. Proceedings of the 2nd International Conference on Learning Representations, ICLR 2014\u2014Conference Track Proceedings, Banff, AB, Canada."},{"key":"ref_5","unstructured":"Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014, January 8\u201313). Generative adversarial nets. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Wan, Z., Zhang, T., and He, H. (December, January 27). Variational Autoencoder Based Synthetic Data Generation for Imbalanced Learning. Proceedings of the 2017 IEEE Symposium Series on Computational Intelligence (SSCI), Honolulu, HI, USA.","DOI":"10.1109\/SSCI.2017.8285168"},{"key":"ref_7","unstructured":"Yoon, J., Jarrett, D., and van der Schaar, M. (2019, January 8\u201314). Time-series generative adversarial networks. Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, QC, Canada."},{"key":"ref_8","unstructured":"Xu, L., Skoularidou, M., Cuesta-Infante, A., and Veeramachaneni, K. (2019, January 8\u201314). Modeling tabular data using conditional GAN. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, Canada."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Zhu, J.Y., Park, T., Isola, P., and Efros, A.A. (2017, January 22\u201329). Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.244"},{"key":"ref_10","unstructured":"Rebuffi, S.A., Gowal, S., Calian, D., Stimberg, F., Wiles, O., and Mann, T. (2021). Data Augmentation Can Improve Robustness. arXiv."},{"key":"ref_11","unstructured":"Zhong, Z., Zheng, L., Kang, G., Li, S., and Yang, Y. (2020, January 7\u201312). Random Erasing Data Augmentation. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA."},{"key":"ref_12","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019, January 2\u20137). BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the NAACL HLT 2019\u20142019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies-Proceedings of the Conference, Minneapolis, MN, USA."},{"key":"ref_13","unstructured":"Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C.L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., and Ray, A. (December, January 28). Training language models to follow instructions with human feedback. Proceedings of the Advances in Neural Information Processing Systems, New Orleans, LA, USA."},{"key":"ref_14","unstructured":"Hannun, A., Case, C., Casper, J., Catanzaro, B., Diamos, G., Elsen, E., Prenger, R., Satheesh, S., Sengupta, S., and Coates, A. (2014). Deep Speech: Scaling up end-to-end speech recognition. arXiv."},{"key":"ref_15","unstructured":"Chorowski, J., Bahdanau, D., Serdyuk, D., Cho, K., and Bengio, Y. (2015, January 10\u201316). Attention-based models for speech recognition. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"677","DOI":"10.1109\/TPAMI.2016.2599174","article-title":"Long-Term Recurrent Convolutional Networks for Visual Recognition and Description","volume":"39","author":"Donahue","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_17","unstructured":"Ng, J.Y.-H., Hausknecht, M., Vijayanarasimhan, S., Vinyals, O., Monga, R., and Toderici, G. (2015, January 7\u201312). Beyond short snippets: Deep networks for video classification. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Boston, MA, USA."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Cho, K., van Merri\u00ebnboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014, January 25\u201329). Learning Phrase Representations Using RNN Encoder-Decoder for Statistical Machine Translation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.","DOI":"10.3115\/v1\/D14-1179"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Luong, M., Pham, H., and Manning, C.D. (2015, January 17\u201321). Effective Approaches to Attention-based Neural Machine Translation. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP), Lisbon, Portugal.","DOI":"10.18653\/v1\/D15-1166"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Woo, S., Park, J., Lee, J.Y., and Kweon, I.S. (2018, January 8\u201314). CBAM: Convolutional block attention module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01234-2_1"},{"key":"ref_22","unstructured":"Fabius, O., and van Amersfoort, J.R. (2015, January 7\u20139). Variational recurrent auto-encoders. Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015-Workshop Track Proceedings, San Diego, CA, USA."},{"key":"ref_23","unstructured":"Sutskever, I., Vinyals, O., and Le, Q.V. (2014, January 8\u201313). Sequence to sequence learning with neural networks. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_24","unstructured":"Shin, H.K., Lee, W., Yun, J.H., and Kim, H.C. (2020, January 10). HAI 1.0: HIL-based augmented ICS security dataset. Proceedings of the CSET 2020\u201413th USENIX Workshop on Cyber Security Experimentation and Test, Co-Located with USENIX Security, Online."},{"key":"ref_25","first-page":"7402","article-title":"Data augmentation for spoken language understanding via joint variational generation","volume":"33","author":"Yoo","year":"2019","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Xia, C., Xiong, C., Yu, P., and Socher, R. (2020, January 16\u201320). Composed variational natural language generation for few-shot intents. Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2020, Online.","DOI":"10.18653\/v1\/2020.findings-emnlp.303"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Arroyo, D.M., Postels, J., and Tombari, F. (2021, January 20\u201325). Variational transformer networks for layout generation. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.01343"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Zhang, C., Kuppannagari, S.R., Kannan, R., and Prasanna, V.K. (2018, January 29\u201331). Generative Adversarial Network for Synthetic Time Series Data Generation in Smart Grids. Proceedings of the 2018 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids, SmartGridComm, Aalborg, Denmark.","DOI":"10.1109\/SmartGridComm.2018.8587464"},{"key":"ref_29","unstructured":"Smith, K., and Smith, A.O. (2020). Conditional GAN for Timeseries Generation. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Marcheggiani, D., and Perez-Beltrachini, L. (2018, January 5\u20138). Deep graph convolutional encoders for structured data to text generation. Proceedings of the 11th International Natural Language Generation Conference, Tilburg, The Netherlands.","DOI":"10.18653\/v1\/W18-6501"},{"key":"ref_31","unstructured":"Kipf, T., Fetaya, E., Wang, K.C., Welling, M., and Zemel, R. (2018, January 10\u201315). Neural relational inference for Interacting systems. Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Yu, B., Yin, H., and Zhu, Z. (2018, January 13\u201319). Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. Proceedings of the IJCAI International Joint Conference on Artificial Intelligence, Stockholm, Sweden.","DOI":"10.24963\/ijcai.2018\/505"},{"key":"ref_33","unstructured":"Chung, J., Kastner, K., Dinh, L., Goel, K., Courville, A., and Bengio, Y. (2015, January 7\u201312). A recurrent latent variable model for sequential data. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_34","unstructured":"Ba, J.L., Kiros, J.R., and Hiton, G.E. (2016). Layer Normalization. arXiv."},{"key":"ref_35","unstructured":"Kingma, D.P., and Ba, J.L. (2015, January 7\u20139). Adam: A method for stochastic optimization. Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015-Conference Track Proceedings, San Diego, CA, USA."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1109\/TASSP.1978.1163055","article-title":"Dynamic programming algorithm optimization for spoken word recognition","volume":"Volume 26","author":"Sakoe","year":"1978","journal-title":"IEEE Transactions on Acoustics, Speech, and Signal Processing"},{"key":"ref_37","first-page":"2579","article-title":"Visualizing data using t-SNE","volume":"9","author":"Hinton","year":"2008","journal-title":"J. Mach. Learn. Res."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/1\/128\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:42:06Z","timestamp":1760132526000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/1\/128"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,26]]},"references-count":37,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,1]]}},"alternative-id":["s24010128"],"URL":"https:\/\/doi.org\/10.3390\/s24010128","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,26]]}}}