{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,30]],"date-time":"2026-04-30T07:26:41Z","timestamp":1777534001330,"version":"3.51.4"},"reference-count":101,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,5,7]],"date-time":"2024-05-07T00:00:00Z","timestamp":1715040000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,5,7]],"date-time":"2024-05-07T00:00:00Z","timestamp":1715040000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100008769","name":"Julius-Maximilians-Universit\u00e4t W\u00fcrzburg","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100008769","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Big Data"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Synthetic data generation describes the process of learning the underlying distribution of a given real dataset in a model, which is, in turn, sampled to produce new data objects still adhering to the original distribution. This approach often finds application where circumstances limit the availability or usability of real-world datasets, for instance, in health care due to privacy concerns. While image synthesis has received much attention in the past, time series are key for many practical (e.g., industrial) applications. To date, numerous different generative models and measures to evaluate time series syntheses have been proposed. However, regarding the defining features of high-quality synthetic time series and how to quantify quality, no consensus has yet been reached among researchers. Hence, we propose a comprehensive survey on evaluation measures for time series generation to assist users in evaluating synthetic time series. For one, we provide brief descriptions or - where applicable - precise definitions. Further, we order the measures in a taxonomy and examine applicability and usage. To assist in the selection of the most appropriate measures, we provide a concise guide for fast lookup. Notably, our findings reveal a lack of a universally accepted approach for an evaluation procedure, including the selection of appropriate measures. We believe this situation hinders progress and may even erode evaluation standards to a \u201cdo as you like\u201d-approach to synthetic data evaluation. Therefore, this survey is a preliminary step to advance the field of synthetic data evaluation.<\/jats:p>","DOI":"10.1186\/s40537-024-00924-7","type":"journal-article","created":{"date-parts":[[2024,5,7]],"date-time":"2024-05-07T08:02:04Z","timestamp":1715068924000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":23,"title":["Evaluation is key: a survey on evaluation measures for synthetic time series"],"prefix":"10.1186","volume":"11","author":[{"given":"Michael","family":"Stenger","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Robert","family":"Leppich","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ian","family":"Foster","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Samuel","family":"Kounev","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Andr\u00e9","family":"Bauer","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,5,7]]},"reference":[{"key":"924_CR1","doi-asserted-by":"publisher","DOI":"10.1098\/rsta.2020.0209","author":"B Lim","year":"2021","unstructured":"Lim B, Zohren S. Time-series forecasting with deep learning: a survey. Philos Trans Royal Soc A. 2021. https:\/\/doi.org\/10.1098\/rsta.2020.0209.","journal-title":"Philos Trans Royal Soc A"},{"issue":"4","key":"924_CR2","doi-asserted-by":"publisher","first-page":"917","DOI":"10.1007\/s10618-019-00619-1","volume":"33","author":"H Ismail Fawaz","year":"2019","unstructured":"Ismail Fawaz H, Forestier G, Weber J, Idoumghar L, Muller PA. Deep learning for time series classification: a review. Data Mining Knowl Discov. 2019;33(4):917\u201363. https:\/\/doi.org\/10.1007\/s10618-019-00619-1.","journal-title":"Data Mining Knowl Discov"},{"key":"924_CR3","doi-asserted-by":"publisher","DOI":"10.1145\/3444690","author":"A Bl\u00e1zquez-Garc\u00eda","year":"2021","unstructured":"Bl\u00e1zquez-Garc\u00eda A, Conde A, Mori U, Lozano JA. A review on outlier\/anomaly detection in time series data. ACM Comput Surv. 2021. https:\/\/doi.org\/10.1145\/3444690.","journal-title":"ACM Comput Surv"},{"key":"924_CR4","doi-asserted-by":"publisher","DOI":"10.1145\/3559540","author":"E Brophy","year":"2023","unstructured":"Brophy E, Wang Z, She Q, Ward T. Generative adversarial networks in time series: a systematic literature review. ACM Comput Surv. 2023. https:\/\/doi.org\/10.1145\/3559540.","journal-title":"ACM Comput Surv"},{"key":"924_CR5","doi-asserted-by":"publisher","DOI":"10.1161\/CIRCOUTCOMES.118.005122","author":"BK Beaulieu-Jones","year":"2019","unstructured":"Beaulieu-Jones BK, Wu ZS, Williams C, Lee R, Bhavnani SP, Byrd JB, et al. Privacy-preserving generative deep neural networks support clinical data sharing. Circ: Cardiovasc Q Outcomes. 2019. https:\/\/doi.org\/10.1161\/CIRCOUTCOMES.118.005122.","journal-title":"Circ: Cardiovasc Q Outcomes"},{"key":"924_CR6","doi-asserted-by":"crossref","unstructured":"Petitjean F, Forestier G, Webb GI, Nicholson AE, Chen Y, Keogh E. Dynamic Time Warping Averaging of Time Series Allows Faster and More Accurate Classification. In: 2014 IEEE International Conference on Data Mining; 2014;470\u2013479.","DOI":"10.1109\/ICDM.2014.27"},{"key":"924_CR7","doi-asserted-by":"publisher","first-page":"1074","DOI":"10.1007\/s10618-018-0565-y","volume":"32","author":"HA Dau","year":"2018","unstructured":"Dau HA, Silva DF, Petitjean F, Forestier G, Bagnall A, Mueen A, et al. Optimizing dynamic time warping\u2019s window width for time series data mining applications. Data Mining Knowl Discov. 2018;32:1074\u2013120. https:\/\/doi.org\/10.1007\/s10618-018-0565-y.","journal-title":"Data Mining Knowl Discov"},{"key":"924_CR8","doi-asserted-by":"publisher","DOI":"10.4855\/ARXIV.2401.02524","author":"A Bauer","year":"2024","unstructured":"Bauer A, Trapp S, Stenger M, Leppich R, Kounev S, Leznik M, et al. Comprehensive exploration of synthetic data generation: a survey. arXiv preprint. 2024. https:\/\/doi.org\/10.4855\/ARXIV.2401.02524.","journal-title":"arXiv preprint"},{"key":"924_CR9","doi-asserted-by":"publisher","DOI":"10.5070\/SR32159045","author":"M Leznik","year":"2022","unstructured":"Leznik M, Lochner A, Wesner S, Domaschka J. [SoK] The great GAN bake Off, an extensive systematic evaluation of generative adversarial network architectures for time series synthesis. J Syst Res. 2022. https:\/\/doi.org\/10.5070\/SR32159045.","journal-title":"J Syst Res"},{"key":"924_CR10","first-page":"28968","volume-title":"Advances in neural information processing systems","author":"D Jarrett","year":"2021","unstructured":"Jarrett D, Bica I, van der Schaar M. Time-series generation by contrastive imitation. In: Ranzato M, Beygelzimer A, Dauphin Y, Liang PS, Vaughan JW, editors. Advances in neural information processing systems. New York: Curran Associates, Inc.; 2021. p. 28968\u201382."},{"key":"924_CR11","doi-asserted-by":"publisher","DOI":"10.1016\/j.cviu.2021.103329","volume":"215","author":"A Borji","year":"2022","unstructured":"Borji A. Pros and cons of GAN evaluation measures: new developments. Comput Vis Image Underst. 2022;215: 103329. https:\/\/doi.org\/10.1016\/j.cviu.2021.103329.","journal-title":"Comput Vis Image Underst"},{"issue":"6","key":"924_CR12","doi-asserted-by":"publisher","first-page":"2515","DOI":"10.1016\/j.jksuci.2020.04.001","volume":"34","author":"T Iqbal","year":"2022","unstructured":"Iqbal T, Qureshi S. The survey: text generation models in deep learning. J King Saud Univ - Comput Inform Sci. 2022;34(6):2515\u201328. https:\/\/doi.org\/10.1016\/j.jksuci.2020.04.001.","journal-title":"J King Saud Univ - Comput Inform Sci"},{"issue":"15","key":"924_CR13","doi-asserted-by":"publisher","first-page":"2733","DOI":"10.3390\/math10152733","volume":"10","author":"A Figueira","year":"2022","unstructured":"Figueira A, Vaz B. Survey on synthetic data generation, evaluation methods and GANs. Mathematics. 2022;10(15):2733.","journal-title":"Mathematics"},{"key":"924_CR14","doi-asserted-by":"publisher","DOI":"10.4855\/ARXIV.1806.07755","author":"Q Xu","year":"2018","unstructured":"Xu Q, Huang G, Yuan Y, Guo C, Sun Y, Wu F, et al. An empirical study on evaluation metrics of generative adversarial networks. arXiv preprint. 2018. https:\/\/doi.org\/10.4855\/ARXIV.1806.07755.","journal-title":"arXiv preprint"},{"key":"924_CR15","doi-asserted-by":"publisher","first-page":"47304","DOI":"10.1109\/ACCESS.2023.3275134","volume":"11","author":"P Eigenschink","year":"2023","unstructured":"Eigenschink P, Reutterer T, Vamosi S, Vamosi R, Sun C, Kalcher K. Deep generative models for synthetic data: a survey. IEEE Access. 2023;11:47304\u201320. https:\/\/doi.org\/10.1109\/ACCESS.2023.3275134.","journal-title":"IEEE Access"},{"key":"924_CR16","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1016\/j.cviu.2018.10.009","volume":"179","author":"A Borji","year":"2019","unstructured":"Borji A. Pros and cons of GAN evaluation measures. Comput Vis Image Underst. 2019;179:41\u201365. https:\/\/doi.org\/10.1016\/j.cviu.2018.10.009.","journal-title":"Comput Vis Image Underst"},{"key":"924_CR17","doi-asserted-by":"publisher","first-page":"11147","DOI":"10.1109\/ACCESS.2022.3144765","volume":"10","author":"FK Dankar","year":"2022","unstructured":"Dankar FK, Ibrahim MK, Ismail L. A multi-dimensional evaluation of synthetic data generators. IEEE Access. 2022;10:11147\u201358. https:\/\/doi.org\/10.1109\/ACCESS.2022.3144765.","journal-title":"IEEE Access"},{"key":"924_CR18","doi-asserted-by":"publisher","DOI":"10.4855\/ARXIV.2011.06801","author":"S Ji","year":"2020","unstructured":"Ji S, Luo J, Yang X. A comprehensive survey on deep music generation: multi-level representations, algorithms, evaluations, and future directions. arXiv preprint. 2020. https:\/\/doi.org\/10.4855\/ARXIV.2011.06801.","journal-title":"arXiv preprint"},{"key":"924_CR19","doi-asserted-by":"publisher","first-page":"53490","DOI":"10.1109\/ACCESS.2022.3174108","volume":"10","author":"N Fatima","year":"2022","unstructured":"Fatima N, Imran AS, Kastrati Z, Daudpota SM, Soomro A. A systematic literature review on text generation using deep neural network models. IEEE Access. 2022;10:53490\u2013503. https:\/\/doi.org\/10.1109\/ACCESS.2022.3174108.","journal-title":"IEEE Access"},{"key":"924_CR20","doi-asserted-by":"crossref","unstructured":"Assefa SA, Dervovic D, Mahfouz M, Tillman RE, Reddy P, Veloso M. Generating synthetic data in finance: opportunities, challenges and pitfalls. In: Proceedings of the First ACM International Conference on AI in Finance; 2020. p. 1\u20138.","DOI":"10.1145\/3383455.3422554"},{"key":"924_CR21","unstructured":"Theis L, van\u00a0den Oord A, Bethge M. A note on the evaluation of generative models. In: International Conference on Learning Representations (ICLR 2016); 2016. ."},{"issue":"1","key":"924_CR22","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1146\/annurev-statistics-040720-031848","volume":"8","author":"TE Raghunathan","year":"2021","unstructured":"Raghunathan TE. Synthetic data. Ann Rev Statist Appl. 2021;8(1):129\u201340. https:\/\/doi.org\/10.1146\/annurev-statistics-040720-031848.","journal-title":"Ann Rev Statist Appl"},{"key":"924_CR23","volume-title":"Springer optimization and its applications","author":"SI Nikolenko","year":"2021","unstructured":"Nikolenko SI. Synthetic data for deep learning. In: Gaw N, Pardalos PM, Gahrooei MR, editors. Springer optimization and its applications. Cham: Springer; 2021."},{"key":"924_CR24","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-52452-8","volume-title":"Time series analysis and its applications","author":"RH Shumway","year":"2017","unstructured":"Shumway RH, Stoffer DS. Time series analysis and its applications. Berlin: Springer; 2017."},{"key":"924_CR25","volume-title":"Advances in neural information processing systems","author":"M Berglund","year":"2015","unstructured":"Berglund M, Raiko T, Honkala M, Karkkainen L, Vetek A, Karhunen JT. Bidirectional recurrent neural networks as generative models. In: Cortes C, Lawrence N, Lee D, Sugiyama M, Garnett R, editors. Advances in neural information processing systems. New York: Curran Associates, Inc.; 2015."},{"key":"924_CR26","volume-title":"Advances in neural information processing systems","author":"J Yoon","year":"2019","unstructured":"Yoon J, Jarrett D, van der Schaar M. Time-series generative adversarial networks. In: Wallach H, Larochelle H, Beygelzimer A, d\u2019Alch\u2019e-Buc F, Fox E, Garnett R, editors. Advances in neural information processing systems. New York: Curran Associates Inc.; 2019."},{"key":"924_CR27","doi-asserted-by":"publisher","DOI":"10.4855\/ARXIV.2111.08095","author":"A Desai","year":"2021","unstructured":"Desai A, Freeman C, Wang Z, Beaver I. TimeVAE: a variational auto-encoder for multivariate time series generation. arXiv preprint. 2021. https:\/\/doi.org\/10.4855\/ARXIV.2111.08095.","journal-title":"arXiv preprint"},{"key":"924_CR28","doi-asserted-by":"publisher","first-page":"1505","DOI":"10.1007\/s10618-014-0377-7","volume":"29","author":"P Sch\u00e4fer","year":"2015","unstructured":"Sch\u00e4fer P. The BOSS is concerned with time series classification in the presence of noise. Data Mining Knowl Discov. 2015;29:1505\u201330.","journal-title":"Data Mining Knowl Discov"},{"key":"924_CR29","doi-asserted-by":"crossref","unstructured":"Ye L, Keogh E. Time series shapelets: a new primitive for data mining. In: Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining; 2009. p. 947\u2013956.","DOI":"10.1145\/1557019.1557122"},{"key":"924_CR30","first-page":"699","volume-title":"A complexity-invariant distance measure for time series","author":"GEAPA Batista","year":"2011","unstructured":"Batista GEAPA, Wang X, Keogh EJ. A complexity-invariant distance measure for time series. New Delhi: SIAM; 2011. p. 699\u2013710."},{"key":"924_CR31","doi-asserted-by":"publisher","first-page":"58","DOI":"10.1016\/j.patrec.2018.11.013","volume":"117","author":"G Jiang","year":"2019","unstructured":"Jiang G, Wang W, Zhang W. A novel distance measure for time series: Maximum shifting correlation distance. Pattern Recognit Lett. 2019;117:58\u201365. https:\/\/doi.org\/10.1016\/j.patrec.2018.11.013.","journal-title":"Pattern Recognit Lett"},{"key":"924_CR32","unstructured":"Webster J, Watson RT. Analyzing the past to prepare for the future: Writing a literature review. MIS quarterly. 2002; p. xiii\u2013xxiii."},{"key":"924_CR33","doi-asserted-by":"crossref","unstructured":"Bindschaedler V, Shokri R, Gunter CA. Plausible Deniability for Privacy-Preserving Data Synthesis. Proceedings of the VLDB Endowment. 2017;10(5).","DOI":"10.14778\/3055540.3055542"},{"key":"924_CR34","doi-asserted-by":"publisher","DOI":"10.4855\/ARXIV.1909.13403","author":"Z Lin","year":"2019","unstructured":"Lin Z, Jain A, Wang C, Fanti GC, Sekar V. Generating high-fidelity, synthetic time series datasets with DoppelGANger. arXiv preprint. 2019. https:\/\/doi.org\/10.4855\/ARXIV.1909.13403.","journal-title":"arXiv preprint"},{"key":"924_CR35","unstructured":"Wu Y, Burda Y, Salakhutdinov R, Grosse R. On the Quantitative Analysis of Decoder-Based Generative Models. In: International Conference on Learning Representations; 2017. Available from: https:\/\/openreview.net\/forum?id=B1M8JF9xx."},{"key":"924_CR36","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1023\/A:1008923215028","volume":"11","author":"RM Neal","year":"2001","unstructured":"Neal RM. Annealed importance sampling. Stat Comput. 2001;11:125\u201339.","journal-title":"Stat Comput"},{"issue":"1","key":"924_CR37","doi-asserted-by":"publisher","first-page":"110","DOI":"10.1063\/1.166092","volume":"5","author":"S Pincus","year":"1995","unstructured":"Pincus S. Approximate entropy (ApEn) as a complexity measure. Chaos: Interdiscip J Nonlinear Sci. 1995;5(1):110\u20137. https:\/\/doi.org\/10.1063\/1.166092.","journal-title":"Chaos: Interdiscip J Nonlinear Sci"},{"key":"924_CR38","doi-asserted-by":"crossref","unstructured":"Leznik M, Michalsky P, Willis P, Schanzel B, \u00d6stberg PO, Domaschka J. Multivariate Time Series Synthesis Using Generative Adversarial Networks. In: Proceedings of the ACM\/SPEC International Conference on Performance Engineering. ICPE \u201921. New York, NY, USA: Association for Computing Machinery; 2021. p. 43-50.","DOI":"10.1145\/3427921.3450257"},{"key":"924_CR39","doi-asserted-by":"publisher","first-page":"36","DOI":"10.1007\/978-3-030-18579-4_3","volume-title":"Database systems for advanced applications","author":"L Wang","year":"2019","unstructured":"Wang L, Zhang W, He X. Continuous patient-centric sequence generation via sequentially coupled adversarial learning. In: Li G, Yang J, Gama J, Natwichai J, Tong Y, editors. Database systems for advanced applications. Cham: Springer International Publishing; 2019. p. 36\u201352."},{"key":"924_CR40","unstructured":"Jeha P, Bohlke-Schneider M, Mercado P, Kapoor S, Nirwan RS, Flunkert V, et\u00a0al. PSA-GAN: Progressive Self Attention GANs for Synthetic Time Series. In: International Conference on Learning Representations; 2022. Available from: https:\/\/openreview.net\/forum?id=Ix_mh42xq5w."},{"key":"924_CR41","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1007\/978-3-031-09342-5_13","volume-title":"Artificial intelligence in medicine","author":"X Li","year":"2022","unstructured":"Li X, Metsis V, Wang H, Ngu AHH. TTS-GAN: a transformer-based time-series generative adversarial network. In: Michalowski M, Abidi SSR, Abidi S, editors. Artificial intelligence in medicine. Cham: Springer International Publishing; 2022. p. 133\u201343."},{"key":"924_CR42","first-page":"32777","volume":"35","author":"A Seyfi","year":"2022","unstructured":"Seyfi A, Rajotte JF, Ng R. Generating multivariate time series with COmmon source CoordInated GAN (COSCI-GAN). Advances in neural information processing systems. 2022;35:32777\u201388.","journal-title":"Advances in neural information processing systems"},{"key":"924_CR43","volume-title":"Advances in neural information processing systems","author":"S Ravuri","year":"2019","unstructured":"Ravuri S, Vinyals O. Classification accuracy score for conditional generative models. In: Wallach H, Larochelle H, Beygelzimer A, d\u2019Alch\u2019e-Buc F, Fox E, Garnett R, editors. Advances in neural information processing systems. New York: Curran Associates Inc.; 2019."},{"key":"924_CR44","unstructured":"Lopez-Paz D, Oquab M. Revisiting classifier two-sample tests. In: International Conference on Learning Representations. Toulon, France; 2017. Available from: https:\/\/hal.inria.fr\/hal-01862834."},{"key":"924_CR45","doi-asserted-by":"publisher","DOI":"10.4855\/ARXIV.1811.12801","author":"V Kulkarni","year":"2018","unstructured":"Kulkarni V, Tagasovska N, Vatter T, Garbinato B. Generative models for simulating mobility trajectories. arXiv preprint. 2018. https:\/\/doi.org\/10.4855\/ARXIV.1811.12801.","journal-title":"arXiv preprint"},{"key":"924_CR46","doi-asserted-by":"crossref","unstructured":"Norgaard S, Saeedi R, Sasani K, Gebremedhin AH. Synthetic Sensor Data Generation for Health Applications: A Supervised Deep Learning Approach. In: 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC); 2018. p. 1164\u20131167.","DOI":"10.1109\/EMBC.2018.8512470"},{"key":"924_CR47","doi-asserted-by":"crossref","unstructured":"Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2016. p. 2818\u20132826.","DOI":"10.1109\/CVPR.2016.308"},{"key":"924_CR48","unstructured":"Franceschi JY, Dieuleveut A, Jaggi M. Unsupervised scalable representation learning for multivariate time series. In: Advances in neural information processing systems, vol. 32; 2019. p. 4627\u20134638."},{"key":"924_CR49","unstructured":"Fr\u00e9chet M. Sur la distance de deux lois de probabilit\u00e9. In: Annales de l\u2019ISUP. vol.\u00a06; 1957. p. 183\u2013198."},{"issue":"7","key":"924_CR50","doi-asserted-by":"publisher","first-page":"8098","DOI":"10.1609\/aaai.v36i7.20782","volume":"36","author":"C Remlinger","year":"2022","unstructured":"Remlinger C, Mikael J, Elie R. Conditional loss and deep euler scheme for time series generation. Proc AAAI Conf Artif Intell. 2022;36(7):8098\u2013105. https:\/\/doi.org\/10.1609\/aaai.v36i7.20782.","journal-title":"Proc AAAI Conf Artif Intell"},{"issue":"1","key":"924_CR51","doi-asserted-by":"publisher","first-page":"7","DOI":"10.3390\/risks11010007","volume":"11","author":"N Boursin","year":"2022","unstructured":"Boursin N, Remlinger C, Mikael J. Deep generators on commodity markets application to deep hedging. Risks. 2022;11(1):7.","journal-title":"Risks"},{"key":"924_CR52","unstructured":"Naeem MF, Oh SJ, Uh Y, Choi Y, Yoo J. Reliable Fidelity and Diversity Metrics for Generative Models. In: III HD, Singh A, editors. Proceedings of the 37th International Conference on Machine Learning. vol. 119 of Proceedings of Machine Learning Research. PMLR; 2020. p. 7176\u20137185."},{"key":"924_CR53","unstructured":"Meehan C, Chaudhuri K, Dasgupta S. A non-parametric test to detect data-copying in generative models. In: Chiappa S, Calandra R, editors. Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics. vol. 108 of Proceedings of Machine Learning Research. PMLR; 2020. p. 3546\u20133556. Available from: https:\/\/proceedings.mlr.press\/v108\/meehan20a.html."},{"issue":"9","key":"924_CR54","doi-asserted-by":"publisher","first-page":"1419","DOI":"10.1080\/14697688.2020.1730426","volume":"20","author":"M Wiese","year":"2020","unstructured":"Wiese M, Knobloch R, Korn R, Kretschmer P. Quant GANs: deep generation of financial time series. Quantit Finance. 2020;20(9):1419\u201340. https:\/\/doi.org\/10.1080\/14697688.2020.1730426.","journal-title":"Quantit Finance"},{"key":"924_CR55","doi-asserted-by":"publisher","DOI":"10.4855\/ARXIV.1911.01700","author":"M Wiese","year":"2019","unstructured":"Wiese M, Bai L, Wood B, Buehler H. Deep hedging: learning to simulate equity option markets. arXiv preprint. 2019. https:\/\/doi.org\/10.4855\/ARXIV.1911.01700.","journal-title":"arXiv preprint"},{"key":"924_CR56","doi-asserted-by":"publisher","DOI":"10.4855\/ARXIV.1706.02633","author":"C Esteban","year":"2017","unstructured":"Esteban C, Hyland SL, R\u00e4tsch G. Real-valued (Medical) time series generation with recurrent conditional GANs. arXiv preprint. 2017. https:\/\/doi.org\/10.4855\/ARXIV.1706.02633.","journal-title":"arXiv preprint"},{"key":"924_CR57","doi-asserted-by":"crossref","unstructured":"Ouyang K, Shokri R, Rosenblum DS, Yang W. A Non-Parametric Generative Model for Human Trajectories. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence. IJCAI\u201918. AAAI Press; 2018. p. 3812-3817.","DOI":"10.24963\/ijcai.2018\/530"},{"key":"924_CR58","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2021.106312","volume":"209","author":"O Pastor-Serrano","year":"2021","unstructured":"Pastor-Serrano O, Lathouwers D, Perk\u00f3 Z. A semi-supervised autoencoder framework for joint generation and classification of breathing. Comput Methods Programs Biomed. 2021;209: 106312. https:\/\/doi.org\/10.1016\/j.cmpb.2021.106312.","journal-title":"Comput Methods Programs Biomed"},{"issue":"3","key":"924_CR59","doi-asserted-by":"publisher","first-page":"757","DOI":"10.1007\/s00477-016-1239-1","volume":"31","author":"C Camera","year":"2017","unstructured":"Camera C, Bruggeman A, Hadjinicolaou P, Michaelides S, Lange MA. Evaluation of a spatial rainfall generator for generating high resolution precipitation projections over orographically complex terrain. Stoch Environ Res Risk Assess. 2017;31(3):757\u201373.","journal-title":"Stoch Environ Res Risk Assess"},{"key":"924_CR60","doi-asserted-by":"publisher","DOI":"10.3390\/en12050849","author":"Z Pan","year":"2019","unstructured":"Pan Z, Wang J, Liao W, Chen H, Yuan D, Zhu W, et al. Data-driven EV load profiles generation using a variational auto-encoder. Energies. 2019. https:\/\/doi.org\/10.3390\/en12050849.","journal-title":"Energies"},{"key":"924_CR61","first-page":"8798","volume-title":"Advances in neural information processing systems","author":"T Xu","year":"2020","unstructured":"Xu T, Wenliang LK, Munn M, Acciaio B. COT-GAN: generating sequential data via causal optimal transport. In: Larochelle H, Ranzato M, Hadsell R, Balcan MF, Lin H, editors. Advances in neural information processing systems. New York: Curran Associates, Inc.; 2020. p. 8798\u2013809."},{"key":"924_CR62","volume-title":"Advances in neural information processing systems","author":"P Grnarova","year":"2019","unstructured":"Grnarova P, Levy KY, Lucchi A, Perraudin N, Goodfellow I, Hofmann T, et al. A domain agnostic measure for monitoring and evaluating GANs. In: Wallach H, Larochelle H, Beygelzimer A, d\u2019Alch\u00e9-Buc F, Fox E, Garnett R, editors., et al., Advances in neural information processing systems. New York p: Curran Associates, Inc.; 2019."},{"key":"924_CR63","doi-asserted-by":"crossref","unstructured":"Sidheekh S, Aimen A, Madan V, Krishnan NC. On Duality Gap as a Measure for Monitoring GAN Training. In: 2021 International Joint Conference on Neural Networks (IJCNN); 2021. p. 1\u20138.","DOI":"10.1109\/IJCNN52387.2021.9534186"},{"key":"924_CR64","unstructured":"Sidheekh S, Aimen A, Krishnan NC. On Characterizing GAN Convergence Through Proximal Duality Gap. In: Meila M, Zhang T, editors. Proceedings of the 38th International Conference on Machine Learning. vol. 139 of Proceedings of Machine Learning Research. PMLR; 2021. p. 9660\u20139670."},{"issue":"6","key":"924_CR65","doi-asserted-by":"publisher","first-page":"1821","DOI":"10.1007\/s10618-019-00647-x","volume":"33","author":"CH Lubba","year":"2019","unstructured":"Lubba CH, Sethi SS, Knaute P, Schultz SR, Fulcher BD, Jones NS. catch22: CAnonical time-series CHaracteristics: selected through highly comparative time-series analysis. Data Mining Knowl Discov. 2019;33(6):1821\u201352.","journal-title":"Data Mining Knowl Discov"},{"key":"924_CR66","volume-title":"Advances in neural information processing systems","author":"M Heusel","year":"2017","unstructured":"Heusel M, Ramsauer H, Unterthiner T, Nessler B, Hochreiter S, et al. GANs trained by a two time-scale update rule converge to a local nash equilibrium. In: Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, et al., editors. Advances in neural information processing systems. New york: Curran Associates, Inc.; 2017."},{"key":"924_CR67","doi-asserted-by":"publisher","DOI":"10.4855\/ARXIV.1902.05624","author":"E Brophy","year":"2019","unstructured":"Brophy E, Wang Z, Ward TE. Quick and easy time series generation with established image-based GANs. arXiv preprint. 2019. https:\/\/doi.org\/10.4855\/ARXIV.1902.05624.","journal-title":"arXiv preprint"},{"key":"924_CR68","volume-title":"Advances in neural information processing systems","author":"MSM Sajjadi","year":"2018","unstructured":"Sajjadi MSM, Bachem O, Lucic M, Bousquet O, Gelly S. Assessing generative models via precision and recall. In: Bengio S, Wallach H, Larochelle H, Grauman K, Cesa-Bianchi N, Garnett R, editors. Advances in neural information processing systems. New York: Curran Associates, Inc.; 2018."},{"key":"924_CR69","volume-title":"Advances in neural information processing systems","author":"T Kynk\u00e4\u00e4nniemi","year":"2019","unstructured":"Kynk\u00e4\u00e4nniemi T, Karras T, Laine S, Lehtinen J, Aila T. Improved precision and recall metric for assessing generative models. In: Wallach H, Larochelle H, Beygelzimer A, d\u2019Alch\u00e9-Buc F, Fox E, Garnett R, editors. Advances in neural information processing systems. New York: Curran Associates, Inc.; 2019."},{"key":"924_CR70","first-page":"7294","volume-title":"Advances in neural information processing systems","author":"S Barannikov","year":"2021","unstructured":"Barannikov S, Trofimov I, Sotnikov G, Trimbach E, Korotin A, Filippov A, et al. Manifold topology divergence: a framework for comparing data manifolds. In: Ranzato M, Beygelzimer A, Dauphin Y, Liang PS, Vaughan JW, editors., et al., Advances in neural information processing systems. New York p: Curran Associates, Inc.; 2021. p. 7294\u2013305."},{"key":"924_CR71","volume-title":"Advances in neural information processing systems","author":"A Gretton","year":"2006","unstructured":"Gretton A, Borgwardt K, Rasch M, Sch\u00f6lkopf B, Smola A. A kernel method for the two-sample-problem. In: Sch\u00f6lkopf B, Platt J, Hoffman T, editors. Advances in neural information processing systems. Cambridge: MIT Press; 2006."},{"key":"924_CR72","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1109\/SP.2017.41","volume-title":"2017 IEEE symposium on security and privacy (SP)","author":"R Shokri","year":"2017","unstructured":"Shokri R, Stronati M, Song C, Shmatikov V. Membership inference attacks against machine learning models. In: Chattopadhyay Ankur, Schulz Michael J, Rettler Clinton, Turkiewicz Katie, Fernandez Laleah, editors. 2017 IEEE symposium on security and privacy (SP). Piscataway: IEEE; 2017. p. 3\u201318."},{"key":"924_CR73","doi-asserted-by":"crossref","unstructured":"Brophy E. Synthesis of Dependent Multichannel ECG Using Generative Adversarial Networks. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management. CIKM \u201920. New York, NY, USA: Association for Computing Machinery; 2020. p. 3229-3232.","DOI":"10.1145\/3340531.3418509"},{"key":"924_CR74","doi-asserted-by":"crossref","unstructured":"Shokri R, Theodorakopoulos G, Le\u00a0Boudec JY, Hubaux JP. Quantifying Location Privacy. In: 2011 IEEE Symposium on Security and Privacy; 2011. p. 247\u2013262.","DOI":"10.1109\/SP.2011.18"},{"key":"924_CR75","doi-asserted-by":"crossref","unstructured":"Bai CY, Lin HT, Raffel C, Kan WCw. On Training Sample Memorization: Lessons from Benchmarking Generative Modeling with a Large-Scale Competition. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. KDD \u201921. New York, NY, USA: Association for Computing Machinery; 2021. p. 2534-2542.","DOI":"10.1145\/3447548.3467198"},{"key":"924_CR76","doi-asserted-by":"publisher","DOI":"10.4855\/ARXIV.2207.14406","author":"K Zhang","year":"2022","unstructured":"Zhang K, Patki N, Veeramachaneni K. Sequential models in the synthetic data vault. arXiv preprint. 2022. https:\/\/doi.org\/10.4855\/ARXIV.2207.14406.","journal-title":"arXiv preprint"},{"key":"924_CR77","unstructured":"Gulrajani I, Raffel C, Metz L. Towards GAN Benchmarks Which Require Generalization. In: International Conference on Learning Representations; 2019. Available from: https:\/\/openreview.net\/forum?id=HkxKH2AcFm."},{"key":"924_CR78","volume-title":"Advances in neural information processing systems","author":"E Richardson","year":"2018","unstructured":"Richardson E, Weiss Y. On GANs and GMMs. In: Bengio S, Wallach H, Larochelle H, Grauman K, Cesa-Bianchi N, Garnett R, editors. Advances in neural information processing systems. New York: Curran Associates, Inc.; 2018."},{"key":"924_CR79","doi-asserted-by":"publisher","DOI":"10.4855\/ARXIV.2001.00062","author":"H Arnout","year":"2019","unstructured":"Arnout H, Kehrer J, Bronner J, Runkler T. Visual evaluation of generative adversarial networks for time series data. arXiv preprint. 2019. https:\/\/doi.org\/10.4855\/ARXIV.2001.00062.","journal-title":"arXiv preprint"},{"key":"924_CR80","unstructured":"Simon L, Webster R, Rabin J. Revisiting precision recall definition for generative modeling. In: Chaudhuri K, Salakhutdinov R, editors. Proceedings of the 36th International Conference on Machine Learning. vol.\u00a097 of Proceedings of Machine Learning Research. PMLR; 2019. p. 5799\u20135808. Available from: https:\/\/proceedings.mlr.press\/v97\/simon19a.html."},{"key":"924_CR81","unstructured":"Bounliphone W, Belilovsky E, Blaschko MB, Antonoglou I, Gretton A. A Test of Relative Similarity For Model Selection in Generative Models. In: International Conference on Learning Representations; 2016. Available from: https:\/\/arxiv.org\/pdf\/1511.04581.pdf."},{"key":"924_CR82","unstructured":"Arjovsky M, Chintala S, Bottou L. Wasserstein Generative Adversarial Networks. In: Precup D, Teh YW, editors. Proceedings of the 34th International Conference on Machine Learning. vol.\u00a070 of Proceedings of Machine Learning Research. PMLR; 2017. p. 214\u2013223. Available from: https:\/\/proceedings.mlr.press\/v70\/arjovsky17a.html."},{"key":"924_CR83","first-page":"93","volume-title":"The wasserstein distances","author":"C Villani","year":"2009","unstructured":"Villani C. The wasserstein distances. Berlin: Springer; 2009. p. 93\u2013111."},{"key":"924_CR84","doi-asserted-by":"publisher","DOI":"10.4855\/ARXIV.2009.12682","author":"H Sun","year":"2020","unstructured":"Sun H, Deng Z, Chen H, Parkes DC. Decision-aware conditional GANs for time series data. arXiv preprint. 2020. https:\/\/doi.org\/10.4855\/ARXIV.2009.12682.","journal-title":"arXiv preprint"},{"key":"924_CR85","doi-asserted-by":"publisher","DOI":"10.4855\/ARXIV.2206.13676","author":"X Li","year":"2022","unstructured":"Li X, Ngu AHH, Metsis V. TTS-CGAN: a transformer time-series conditional GAN for biosignal data augmentation. arXiv preprint. 2022. https:\/\/doi.org\/10.4855\/ARXIV.2206.13676.","journal-title":"arXiv preprint"},{"issue":"5\/6","key":"924_CR86","doi-asserted-by":"publisher","first-page":"561","DOI":"10.5194\/npg-11-561-2004","volume":"11","author":"A Grinsted","year":"2004","unstructured":"Grinsted A, Moore JC, Jevrejeva S. Application of the cross wavelet transform and wavelet coherence to geophysical time series. Nonlinear Process Geophys. 2004;11(5\/6):561\u20136. https:\/\/doi.org\/10.5194\/npg-11-561-2004.","journal-title":"Nonlinear Process Geophys"},{"key":"924_CR87","unstructured":"Alaa A, Van\u00a0Breugel B, Saveliev ES, van\u00a0der Schaar M. How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Models. In: Chaudhuri K, Jegelka S, Song L, Szepesvari C, Niu G, Sabato S, editors. Proceedings of the 39th International Conference on Machine Learning. vol. 162 of Proceedings of Machine Learning Research. PMLR; 2022. p. 290\u2013306. Available from: https:\/\/proceedings.mlr.press\/v162\/alaa22a.html."},{"key":"924_CR88","doi-asserted-by":"publisher","DOI":"10.1007\/s10489-022-03742-7","author":"B Heidrich","year":"2022","unstructured":"Heidrich B, Turowski M, Phipps K, Schmieder K, S\u00fc\u00df W, Mikut R, et al. Controlling non-stationarity and periodicities in time series generation using conditional invertible neural networks. Appl Intell. 2022. https:\/\/doi.org\/10.1007\/s10489-022-03742-7.","journal-title":"Appl Intell"},{"key":"924_CR89","doi-asserted-by":"publisher","DOI":"10.4855\/ARXIV.2205.11164","author":"P Srinivasan","year":"2022","unstructured":"Srinivasan P, Knottenbelt WJ. Time-series transformer generative adversarial networks. arXiv preprint. 2022. https:\/\/doi.org\/10.4855\/ARXIV.2205.11164.","journal-title":"arXiv preprint"},{"key":"924_CR90","doi-asserted-by":"crossref","unstructured":"Pei H, Ren K, Yang Y, Liu C, Qin T, Li D. Towards Generating Real-World Time Series Data. In: 2021 IEEE International Conference on Data Mining (ICDM); 2021. p. 469\u2013478.","DOI":"10.1109\/ICDM51629.2021.00058"},{"key":"924_CR91","doi-asserted-by":"publisher","DOI":"10.4855\/ARXIV.2208.05836","author":"E Fons","year":"2022","unstructured":"Fons E, Sztrajman A, El-laham Y, Iosifidis A, Vyetrenko S. HyperTime: implicit neural representation for time series. arXiv preprint. 2022. https:\/\/doi.org\/10.4855\/ARXIV.2208.05836.","journal-title":"arXiv preprint"},{"key":"924_CR92","unstructured":"Alaa A, Chan AJ, van\u00a0der Schaar M. Generative Time-series Modeling with Fourier Flows. In: International Conference on Learning Representations; 2021. Available from: https:\/\/openreview.net\/forum?id=PpshD0AXfA."},{"key":"924_CR93","doi-asserted-by":"publisher","DOI":"10.4855\/ARXIV.2006.05421","author":"H Ni","year":"2020","unstructured":"Ni H, Szpruch L, Wiese M, Liao S, Xiao B. Conditional sig-wasserstein GANs for time series generation. arXiv preprint. 2020. https:\/\/doi.org\/10.4855\/ARXIV.2006.05421.","journal-title":"arXiv preprint"},{"key":"924_CR94","doi-asserted-by":"publisher","DOI":"10.3390\/biology9120441","author":"D Hazra","year":"2020","unstructured":"Hazra D, Byun YC. SynSigGAN: generative adversarial networks for synthetic biomedical signal generation. Biology. 2020. https:\/\/doi.org\/10.3390\/biology9120441.","journal-title":"Biology"},{"key":"924_CR95","doi-asserted-by":"crossref","unstructured":"Kegel L, Hahmann M, Lehner W. Feature-Based Comparison and Generation of Time Series. In: Proceedings of the 30th International Conference on Scientific and Statistical Database Management. SSDBM \u201918. New York, NY, USA: Association for Computing Machinery; 2018. .","DOI":"10.1145\/3221269.3221293"},{"key":"924_CR96","doi-asserted-by":"publisher","DOI":"10.4855\/ARXIV.2102.10231","author":"A Shifaz","year":"2021","unstructured":"Shifaz A, Pelletier C, Petitjean F, Webb GI. Elastic similarity measures for multivariate time series classification. arXiv preprint. 2021. https:\/\/doi.org\/10.4855\/ARXIV.2102.10231.","journal-title":"arXiv preprint"},{"key":"924_CR97","doi-asserted-by":"crossref","unstructured":"Li J, Wang X, Lin Y, Sinha A, Wellman M. Generating realistic stock market order streams. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol.\u00a034; 2020. p. 727\u2013734.","DOI":"10.1609\/aaai.v34i01.5415"},{"key":"924_CR98","doi-asserted-by":"crossref","DOI":"10.1137\/1.9781611970104","volume-title":"Lectures on wavelets","author":"Daubechies I Ten","year":"1992","unstructured":"Ten Daubechies I. Lectures on wavelets. Philadelphia: SIAM; 1992."},{"key":"924_CR99","doi-asserted-by":"crossref","unstructured":"Lynn PA, Lynn PA. The Laplace Transform and the z-transform. Electronic Signals and Systems. 1986;p. 225\u2013272.","DOI":"10.1007\/978-1-349-18461-3_6"},{"key":"924_CR100","unstructured":"Kosara R, Bendix F, Hauser H. Time Histograms for Large, Time-Dependent Data. In: Proceedings of the Sixth Joint Eurographics - IEEE TCVG Conference on Visualization. VISSYM\u201904. Goslar, DEU: Eurographics Association; 2004. p. 45-54."},{"issue":"1","key":"924_CR101","doi-asserted-by":"publisher","first-page":"523","DOI":"10.1109\/TVCG.2018.2865077","volume":"25","author":"A Gogolou","year":"2019","unstructured":"Gogolou A, Tsandilas T, Palpanas T, Bezerianos A. Comparing similarity perception in time series visualizations. IEEE Trans Vis Comput Graphics. 2019;25(1):523\u201333. https:\/\/doi.org\/10.1109\/TVCG.2018.2865077.","journal-title":"IEEE Trans Vis Comput Graphics"}],"container-title":["Journal of Big Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s40537-024-00924-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s40537-024-00924-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s40537-024-00924-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,5,7]],"date-time":"2024-05-07T08:11:36Z","timestamp":1715069496000},"score":1,"resource":{"primary":{"URL":"https:\/\/journalofbigdata.springeropen.com\/articles\/10.1186\/s40537-024-00924-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,7]]},"references-count":101,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["924"],"URL":"https:\/\/doi.org\/10.1186\/s40537-024-00924-7","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-3331381\/v1","asserted-by":"object"}]},"ISSN":["2196-1115"],"issn-type":[{"value":"2196-1115","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,7]]},"assertion":[{"value":"1 November 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 April 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 May 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"66"}}