{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,23]],"date-time":"2026-07-23T21:06:26Z","timestamp":1784840786350,"version":"3.55.0"},"reference-count":47,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2025,9,13]],"date-time":"2025-09-13T00:00:00Z","timestamp":1757721600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>Analyzing electroencephalographic (EEG) time series can be challenging, especially with deep neural networks, due to the large variability among human subjects and often small datasets. To address these challenges, various strategies, such as self-supervised learning, have been suggested, but they typically rely on extensive empirical datasets. Inspired by recent advances in computer vision, we propose a pretraining task termed \u201cfrequency pretraining\u201d to pretrain a neural network for sleep staging by predicting the frequency content of randomly generated synthetic time series. Our experiments demonstrate that our method surpasses fully supervised learning in scenarios with limited data and few subjects, and matches its performance in regimes with many subjects. Furthermore, our results underline the relevance of frequency information for sleep stage scoring, while also demonstrating that deep neural networks utilize information beyond frequencies to enhance sleep staging performance, which is consistent with previous research. We anticipate that our approach will be advantageous across a broad spectrum of applications where EEG data is limited or derived from a small number of subjects, including the domain of brain-computer interfaces.<\/jats:p>","DOI":"10.3390\/a18090580","type":"journal-article","created":{"date-parts":[[2025,9,15]],"date-time":"2025-09-15T11:51:43Z","timestamp":1757937103000},"page":"580","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Data-Efficient Sleep Staging with Synthetic Time Series Pretraining"],"prefix":"10.3390","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-3774-5418","authenticated-orcid":false,"given":"Niklas","family":"Grieger","sequence":"first","affiliation":[{"name":"Department of Medical Engineering and Technomathematics, FH Aachen University of Applied Sciences, 52428 J\u00fclich, Germany"},{"name":"Department of Information and Computing Sciences, Utrecht University, 3584 CS Utrecht, The Netherlands"},{"name":"Institute for Data-Driven Technologies, FH Aachen University of Applied Sciences, 52428 J\u00fclich, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0516-0391","authenticated-orcid":false,"given":"Siamak","family":"Mehrkanoon","sequence":"additional","affiliation":[{"name":"Department of Information and Computing Sciences, Utrecht University, 3584 CS Utrecht, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1150-8080","authenticated-orcid":false,"given":"Stephan","family":"Bialonski","sequence":"additional","affiliation":[{"name":"Department of Medical Engineering and Technomathematics, FH Aachen University of Applied Sciences, 52428 J\u00fclich, Germany"},{"name":"Institute for Data-Driven Technologies, FH Aachen University of Applied Sciences, 52428 J\u00fclich, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,9,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"051001","DOI":"10.1088\/1741-2552\/ab260c","article-title":"Deep learning-based electroencephalography analysis: A systematic review","volume":"16","author":"Roy","year":"2019","journal-title":"J. Neural Eng."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Ko, W., Jeon, E., Jeong, S., Phyo, J., and Suk, H.I. (2021). A Survey on Deep Learning-Based Short\/Zero-Calibration Approaches for EEG-Based Brain\u2013Computer Interfaces. Front. Hum. Neurosci., 15.","DOI":"10.3389\/fnhum.2021.643386"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"04TR01","DOI":"10.1088\/1361-6579\/ac6049","article-title":"Automatic sleep staging of EEG signals: Recent development, challenges, and future directions","volume":"43","author":"Phan","year":"2022","journal-title":"Physiol. Meas."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"101204","DOI":"10.1016\/j.smrv.2019.07.007","article-title":"Automated sleep scoring: A review of the latest approaches","volume":"48","author":"Fiorillo","year":"2019","journal-title":"Sleep Med. Rev."},{"key":"ref_5","unstructured":"Bronstein, M.M., Bruna, J., Cohen, T., and Velickovic, P. (2021). Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges. arXiv."},{"key":"ref_6","unstructured":"Goodfellow, I.J., Bengio, Y., and Courville, A.C. (2016). Deep Learning, MIT Press. Adaptive Computation and Machine Learning."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"108885","DOI":"10.1016\/j.jneumeth.2020.108885","article-title":"Data augmentation for deep-learning-based electroencephalography","volume":"346","author":"Lashgari","year":"2020","journal-title":"J. Neurosci. Meth."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"He, C., Liu, J., Zhu, Y., and Du, W. (2021). Data Augmentation for Deep Neural Networks Model in EEG Classification Task: A Review. Front. Hum. Neurosci., 15.","DOI":"10.3389\/fnhum.2021.765525"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Ebbehoj, A., Thunbo, M.\u00d8., Andersen, O.E., Glindtvad, M.V., and Hulman, A. (2022). Transfer learning for non-image data in clinical research: A scoping review. PLoS Digit. Health, 1.","DOI":"10.1371\/journal.pdig.0000014"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1007\/s11633-022-1382-8","article-title":"Pre-training in Medical Data: A Survey","volume":"20","author":"Qiu","year":"2023","journal-title":"Mach. Intell. Res."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Liu, Z., Alavi, A., Li, M., and Zhang, X. (2023). Self-Supervised Contrastive Learning for Medical Time Series: A Systematic Review. Sensors, 23.","DOI":"10.3390\/s23094221"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"046020","DOI":"10.1088\/1741-2552\/abca18","article-title":"Uncovering the structure of clinical EEG signals with self-supervised learning","volume":"18","author":"Banville","year":"2021","journal-title":"J. Neural Eng."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1186\/s12984-023-01169-w","article-title":"Generative adversarial networks in EEG analysis: An overview","volume":"20","author":"Habashi","year":"2023","journal-title":"J. NeuroEng. Rehabil."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Carrle, F.P., Hollenbenders, Y., and Reichenbach, A. (2023). Generation of synthetic EEG data for training algorithms supporting the diagnosis of major depressive disorder. Front. Neurosci., 17.","DOI":"10.3389\/fnins.2023.1219133"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"You, Z., Guo, Y., Zhang, X., and Zhao, Y. (2025). Virtual Electroencephalogram Acquisition: A Review on Electroencephalogram Generative Methods. Sensors, 25.","DOI":"10.3390\/s25103178"},{"key":"ref_16","unstructured":"Ranzato, M., Beygelzimer, A., Dauphin, Y.N., Liang, P., and Vaughan, J.W. (, January 6\u201314). Learning to See by Looking at Noise. Proceedings of the Annual Conference on Neural Information Processing Systems, NeurIPS, Virtual."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"990","DOI":"10.1007\/s11263-021-01555-8","article-title":"Pre-Training Without Natural Images","volume":"130","author":"Kataoka","year":"2022","journal-title":"Int. J. Comput. Vis."},{"key":"ref_18","unstructured":"Berry, R.B., Brooks, R., Gamaldo, C.E., Harding, S.M., Lloyd, R.M., Marcus, C.L., and Vaughn, B.V. (2020). The AASM Manual for the Scoring of Sleep and Associated Events: Rules, Terminology and Technical Specifications, Version 2.6, American Academy of Sleep Medicine."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1016\/j.bspc.2013.12.003","article-title":"Signal processing techniques applied to human sleep EEG signals\u2014A review","volume":"10","author":"Hill","year":"2014","journal-title":"Biomed. Signal Process. Control."},{"key":"ref_20","unstructured":"Grieger, N. (2025, August 29). Source Code of the Model Presented in Grieger et al., Data-Efficient Sleep Staging with Synthetic Time Series Pretraining. Available online: https:\/\/github.com\/dslaborg\/frequency-pretraining."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2015, January 7\u201313). Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. Proceedings of the 2015 IEEE International Conference on Computer Vision, ICCV 2015, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.123"},{"key":"ref_22","unstructured":"Chen, T., Kornblith, S., Norouzi, M., and Hinton, G.E. (2020, January 13\u201318). A Simple Framework for Contrastive Learning of Visual Representations. Proceedings of the 37th International Conference on Machine Learning, ICML 2020. PMLR, Virtual."},{"key":"ref_23","unstructured":"Bardes, A., Ponce, J., and LeCun, Y. (2022, January 25\u201329). VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning. Proceedings of the Tenth International Conference on Learning Representations, ICLR, Virtual."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Obeid, I., and Picone, J. (2016). The Temple University Hospital EEG Data Corpus. Front. Neurosci., 10.","DOI":"10.3389\/fnins.2016.00196"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1351","DOI":"10.1093\/jamia\/ocy064","article-title":"The National Sleep Research Resource: Towards a Sleep Data Commons","volume":"25","author":"Zhang","year":"2018","journal-title":"J. Am. Med. Inform. Assn."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1333","DOI":"10.1109\/TNSRE.2023.3245285","article-title":"Self-Supervised Learning for Label-Efficient Sleep Stage Classification: A Comprehensive Evaluation","volume":"31","author":"Eldele","year":"2023","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1016\/j.jsmc.2023.05.002","article-title":"Challenges of Applying Automated Polysomnography Scoring at Scale","volume":"18","year":"2023","journal-title":"Sleep Med. Clin."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1038\/s41746-023-00784-0","article-title":"U-Sleep\u2019s resilience to AASM guidelines","volume":"6","author":"Fiorillo","year":"2023","journal-title":"npj Digit. Med."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1007\/s13534-023-00299-3","article-title":"Current status and prospects of automatic sleep stages scoring: Review","volume":"13","author":"Gaiduk","year":"2023","journal-title":"Biomed. Eng. Lett."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"e70092","DOI":"10.7554\/eLife.70092","article-title":"An open-source, high-performance tool for automated sleep staging","volume":"10","author":"Vallat","year":"2021","journal-title":"eLife"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Donckt, M.J.V.D., Donckt, J.V.D., Deprost, E., Vandenbussche, N., Rademaker, M., Vandewiele, G., and Hoecke, S.V. (2023). Do not sleep on traditional machine learning: Simple and interpretable techniques are competitive to deep learning for sleep scoring. Biomed. Signal Process. Control., 81.","DOI":"10.1016\/j.bspc.2022.104429"},{"key":"ref_32","unstructured":"Guyon, I., von Luxburg, U., Bengio, S., Wallach, H.M., Fergus, R., Vishwanathan, S.V.N., and Garnett, R. (2017). Attention is All you Need. Proceedings of the Annual Conference Neural Information Processing Systems, NeurIPS, Long Beach, CA, USA, 4\u20139 December 2017, Curran Associates Inc."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"310","DOI":"10.1016\/j.neunet.2022.07.020","article-title":"Relational local electroencephalography representations for sleep scoring","volume":"154","author":"Brandmayr","year":"2022","journal-title":"Neural Netw."},{"key":"ref_34","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2021, January 3\u20137). An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. Proceedings of the International Conference on Learning Representations, ICLR, Vienna, Austria."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"333","DOI":"10.1038\/s41597-022-01409-z","article-title":"The two decades brainclinics research archive for insights in neurophysiology (TDBRAIN) database","volume":"9","author":"Denys","year":"2022","journal-title":"Sci. Data"},{"key":"ref_36","unstructured":"Grieger, N., Mehrkanoon, S., and Bialonski, S. (2024, January 7\u201311). Pretraining Sleep Staging Models without Patient Data. Proceedings of the International Conference on Learning Representations (ICLR), Workshop on Learning from Time Series for Health, Vienna, Austria."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Supratak, A., and Guo, Y. (2020, January 20\u201324). TinySleepNet: An Efficient Deep Learning Model for Sleep Stage Scoring based on Raw Single-Channel EEG. Proceedings of the 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society, EMBC 2020, Montreal, QC, Canada.","DOI":"10.1109\/EMBC44109.2020.9176741"},{"key":"ref_38","unstructured":"Bengio, Y., and LeCun, Y. (2015, January 7\u20139). Adam: A Method for Stochastic Optimization. Proceedings of the 3rd International Conference Learning Representations, ICLR, San Diego, CA, USA."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"427","DOI":"10.1016\/j.ipm.2009.03.002","article-title":"A systematic analysis of performance measures for classification tasks","volume":"45","author":"Sokolova","year":"2009","journal-title":"Inf. Process. Manag."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1955","DOI":"10.1109\/TNSRE.2020.3011181","article-title":"Dreem Open Datasets: Multi-Scored Sleep Datasets to Compare Human and Automated Sleep Staging","volume":"28","author":"Guillot","year":"2020","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1185","DOI":"10.1109\/10.867928","article-title":"Analysis of a sleep-dependent neuronal feedback loop: The slow-wave microcontinuity of the EEG","volume":"47","author":"Kemp","year":"2000","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"557","DOI":"10.1093\/sleep\/18.7.557","article-title":"Age and Gender Affect Different Characteristics of Slow Waves in the Sleep EEG","volume":"18","author":"Mourtazaev","year":"1995","journal-title":"Sleep"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"180","DOI":"10.1016\/j.cmpb.2015.10.013","article-title":"ISRUC-Sleep: A comprehensive public dataset for sleep researchers","volume":"124","author":"Khalighi","year":"2016","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1161\/01.CIR.101.23.e215","article-title":"PhysioBank, PhysioToolkit, and PhysioNet: Components of a New Research Resource for Complex Physiologic Signals","volume":"101","author":"Goldberger","year":"2000","journal-title":"Circulation"},{"key":"ref_45","unstructured":"Rechtschaffen, A., Kales, A., Berger, R., Dement, W., Jacobson, A., Johnson, L., Jouvet, M., Monroe, L., Oswald, I., and Roffwarg, H. (1968). A Manual of Standardized Terminology, Techniques and Scoring System for Sleep Stages of Human Subjects."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long Short-Term Memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"168","DOI":"10.1016\/j.aci.2018.08.003","article-title":"Classification assessment methods","volume":"17","author":"Tharwat","year":"2021","journal-title":"Appl. Comput. Inform."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/18\/9\/580\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:45:10Z","timestamp":1760035510000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/18\/9\/580"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,13]]},"references-count":47,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2025,9]]}},"alternative-id":["a18090580"],"URL":"https:\/\/doi.org\/10.3390\/a18090580","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,9,13]]}}}