{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T23:47:56Z","timestamp":1781912876858,"version":"3.54.5"},"reference-count":55,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2023,2,21]],"date-time":"2023-02-21T00:00:00Z","timestamp":1676937600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"FWF, Austrian Science Fund","award":["W 1233-B"],"award-info":[{"award-number":["W 1233-B"]}]},{"name":"FWF, Austrian Science Fund","award":["20102-F2002176-FPR"],"award-info":[{"award-number":["20102-F2002176-FPR"]}]},{"name":"county of Salzburg","award":["W 1233-B"],"award-info":[{"award-number":["W 1233-B"]}]},{"name":"county of Salzburg","award":["20102-F2002176-FPR"],"award-info":[{"award-number":["20102-F2002176-FPR"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Sleep staging based on polysomnography (PSG) performed by human experts is the de facto \u201cgold standard\u201d for the objective measurement of sleep. PSG and manual sleep staging is, however, personnel-intensive and time-consuming and it is thus impractical to monitor a person\u2019s sleep architecture over extended periods. Here, we present a novel, low-cost, automatized, deep learning alternative to PSG sleep staging that provides a reliable epoch-by-epoch four-class sleep staging approach (Wake, Light [N1 + N2], Deep, REM) based solely on inter-beat-interval (IBI) data. Having trained a multi-resolution convolutional neural network (MCNN) on the IBIs of 8898 full-night manually sleep-staged recordings, we tested the MCNN on sleep classification using the IBIs of two low-cost (&lt;EUR 100) consumer wearables: an optical heart rate sensor (VS) and a breast belt (H10), both produced by POLAR\u00ae. The overall classification accuracy reached levels comparable to expert inter-rater reliability for both devices (VS: 81%, \u03ba = 0.69; H10: 80.3%, \u03ba = 0.69). In addition, we used the H10 and recorded daily ECG data from 49 participants with sleep complaints over the course of a digital CBT-I-based sleep training program implemented in the App NUKKUAA\u2122. As proof of principle, we classified the IBIs extracted from H10 using the MCNN over the course of the training program and captured sleep-related changes. At the end of the program, participants reported significant improvements in subjective sleep quality and sleep onset latency. Similarly, objective sleep onset latency showed a trend toward improvement. Weekly sleep onset latency, wake time during sleep, and total sleep time also correlated significantly with the subjective reports. The combination of state-of-the-art machine learning with suitable wearables allows continuous and accurate monitoring of sleep in naturalistic settings with profound implications for answering basic and clinical research questions.<\/jats:p>","DOI":"10.3390\/s23052390","type":"journal-article","created":{"date-parts":[[2023,2,22]],"date-time":"2023-02-22T02:08:34Z","timestamp":1677031714000},"page":"2390","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":20,"title":["The Virtual Sleep Lab\u2014A Novel Method for Accurate Four-Class Sleep Staging Using Heart-Rate Variability from Low-Cost Wearables"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6235-060X","authenticated-orcid":false,"given":"Pavlos","family":"Topalidis","sequence":"first","affiliation":[{"name":"Laboratory for Sleep, Cognition and Consciousness Research, Department of Psychology and Centre for Cognitive Neuroscience Salzburg (CCNS), Paris-Lodron University of Salzburg, 5020 Salzburg, Austria"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7977-8557","authenticated-orcid":false,"given":"Dominik P. J.","family":"Heib","sequence":"additional","affiliation":[{"name":"Laboratory for Sleep, Cognition and Consciousness Research, Department of Psychology and Centre for Cognitive Neuroscience Salzburg (CCNS), Paris-Lodron University of Salzburg, 5020 Salzburg, Austria"},{"name":"Institut Proschlaf, 5020 Salzburg, Austria"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2747-7728","authenticated-orcid":false,"given":"Sebastian","family":"Baron","sequence":"additional","affiliation":[{"name":"Department of Mathematics, Paris-Lodron University of Salzburg, 5020 Salzburg, Austria"},{"name":"Department of Artificial Intelligence and Human Interfaces (AIHI), Paris-Lodron University of Salzburg, 5020 Salzburg, Austria"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0018-7632","authenticated-orcid":false,"given":"Esther-Sevil","family":"Eigl","sequence":"additional","affiliation":[{"name":"Laboratory for Sleep, Cognition and Consciousness Research, Department of Psychology and Centre for Cognitive Neuroscience Salzburg (CCNS), Paris-Lodron University of Salzburg, 5020 Salzburg, Austria"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5043-9909","authenticated-orcid":false,"given":"Alexandra","family":"Hinterberger","sequence":"additional","affiliation":[{"name":"Laboratory for Sleep, Cognition and Consciousness Research, Department of Psychology and Centre for Cognitive Neuroscience Salzburg (CCNS), Paris-Lodron University of Salzburg, 5020 Salzburg, Austria"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5899-8772","authenticated-orcid":false,"given":"Manuel","family":"Schabus","sequence":"additional","affiliation":[{"name":"Laboratory for Sleep, Cognition and Consciousness Research, Department of Psychology and Centre for Cognitive Neuroscience Salzburg (CCNS), Paris-Lodron University of Salzburg, 5020 Salzburg, Austria"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,2,21]]},"reference":[{"key":"ref_1","unstructured":"Grandner, M.A. (2019). Sleep and Health, Elsevier."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3390\/healthcare7010001","article-title":"The global problem of insufficient sleep and its serious public health implications","volume":"7","author":"Chattu","year":"2018","journal-title":"Healthcare"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"240","DOI":"10.3390\/medicina44030031","article-title":"Factors associated with poor sleep and health-related quality of life","volume":"44","author":"Varoneckas","year":"2008","journal-title":"Medicina"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Garbarino, S., Lanteri, P., Durando, P., Magnavita, N., and Sannita, W.G. (2016). Co-morbidity, mortality, quality of life and the healthcare\/welfare\/social costs of disordered sleep: A rapid review. Int. J. Environ. Res. Public Health, 13.","DOI":"10.3390\/ijerph13080831"},{"key":"ref_5","unstructured":"Precedence Research (2021, December 31). Sleep Aids Market. Available online: https:\/\/www.precedenceresearch.com\/sleep-aids-market."},{"key":"ref_6","first-page":"2012","article-title":"The AASM manual for the scoring of sleep and associated events","volume":"Volume 176","author":"Berry","year":"2012","journal-title":"Rules, Terminology and Technical Specifications"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"681","DOI":"10.1152\/physrev.00032.2012","article-title":"About sleep\u2019s role in memory","volume":"93","author":"Rasch","year":"2013","journal-title":"Physiol. Rev."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.cophys.2019.10.020","article-title":"Cleaning the sleeping brain\u2013the potential restorative function of the glymphatic system","volume":"15","author":"Hauglund","year":"2020","journal-title":"Curr. Opin. Physiol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1016\/j.smrv.2017.12.005","article-title":"Sleep and emotional processing","volume":"40","author":"Tempesta","year":"2018","journal-title":"Sleep Med. Rev."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1016\/0301-0511(82)90045-X","article-title":"Energy expenditure and restorative sleep","volume":"15","author":"Shapiro","year":"1982","journal-title":"Biol. Psychol."},{"key":"ref_11","first-page":"1","article-title":"Deep learning for automated sleep staging using instantaneous heart rate","volume":"3","author":"Sridhar","year":"2020","journal-title":"NPJ Digit. Med."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1016\/0013-4694(73)90176-4","article-title":"Variations of heart-rate during sleep as a function of the sleep cycle","volume":"35","author":"Aldredge","year":"1973","journal-title":"Electroencephalogr. Clin. Neurophysiol."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"402","DOI":"10.3389\/fnins.2014.00402","article-title":"heart-rate variability: A tool to explore the sleeping brain?","volume":"8","author":"Chouchou","year":"2014","journal-title":"Front. Neurosci."},{"key":"ref_14","first-page":"369","article-title":"Spectral analysis of heart-rate variability in sleep","volume":"54","author":"Salinger","year":"2005","journal-title":"Physiol. Res."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1016\/S0301-0511(03)00052-8","article-title":"heart-rate variability during sleep as a function of the sleep cycle","volume":"63","author":"Versace","year":"2003","journal-title":"Biol. Psychol."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"246","DOI":"10.1504\/IJBET.2010.032695","article-title":"Sleep staging from heart-rate variability: Time-varying spectral features and hidden Markov models","volume":"3","author":"Mendez","year":"2010","journal-title":"Int. J. Biomed. Eng. Technol."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Loh, H.W., Ooi, C.P., Vicnesh, J., Oh, S.L., Faust, O., Gertych, A., and Acharya, U.R. (2020). Automated detection of sleep stages using deep learning techniques: A systematic review of the last decade (2010\u20132020). Appl. Sci., 10.","DOI":"10.3390\/app10248963"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"08TR02","DOI":"10.1088\/1361-6579\/ac826e","article-title":"A systematic review of deep learning methods for modeling electrocardiograms during sleep","volume":"43","author":"Sun","year":"2022","journal-title":"Physiol. Meas."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"124005","DOI":"10.1088\/1361-6579\/aaf339","article-title":"Deep learning in the cross-time frequency domain for sleep staging from a single-lead electrocardiogram","volume":"39","author":"Li","year":"2018","journal-title":"Physiol. Meas."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Imtiaz, S.A. (2021). A systematic review of sensing technologies for wearable sleep staging. Sensors, 21.","DOI":"10.3390\/s21051562"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"301","DOI":"10.5664\/JCSM.1078","article-title":"Sleep staging based on autonomic signals: A multi-center validation study","volume":"7","author":"Hedner","year":"2011","journal-title":"J. Clin. Sleep Med."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1968","DOI":"10.1088\/1361-6579\/aa9047","article-title":"Estimation of sleep stages in a healthy adult population from optical plethysmography and accelerometer signals","volume":"38","author":"Beattie","year":"2017","journal-title":"Physiol. Meas."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Gasmi, A., Augusto, V., Beaudet, P.A., Faucheu, J., Morin, C., Serpaggi, X., and Vassel, F. (2020, January 20\u201321). Sleep stages classification using cardio-respiratory variables. Proceedings of the 2020 IEEE 16th International Conference on Automation Science and Engineering (CASE), Hong Kong, China.","DOI":"10.1109\/CASE48305.2020.9217045"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Fedorin, I., Slyusarenko, K., Lee, W., and Sakhnenko, N. (2019, January 2\u20136). Sleep stages classification in a healthy people based on optical plethysmography and accelerometer signals via wearable devices. Proceedings of the 2019 IEEE 2nd Ukraine Conference on Electrical and Computer Engineering (UKRCON), Lviv, Ukraine.","DOI":"10.1109\/UKRCON.2019.8879875"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"2631","DOI":"10.1080\/02640414.2020.1797448","article-title":"A validation study of the WHOOP strap against polysomnography to assess sleep","volume":"38","author":"Miller","year":"2020","journal-title":"J. Sport. Sci."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"e24704","DOI":"10.2196\/24704","article-title":"Heart rate variability and firstbeat method for detecting sleep stages in healthy young adults: Feasibility study","volume":"9","author":"Kuula","year":"2021","journal-title":"JMIR MHealth UHealth"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Widasari, E.R., Tanno, K., and Tamura, H. (2018, January 7\u201310). Automatic sleep stage detection based on HRV spectrum analysis. Proceedings of the 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Miyazaki, Japan.","DOI":"10.1109\/SMC.2018.00155"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1016\/0020-7101(93)90059-F","article-title":"Interpretation of low kappa values","volume":"33","author":"Donker","year":"1993","journal-title":"Int. J. Bio-Med. Comput."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"675","DOI":"10.1111\/jsr.12594","article-title":"European guideline for the diagnosis and treatment of insomnia","volume":"26","author":"Riemann","year":"2017","journal-title":"J. Sleep Res."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"S76","DOI":"10.37765\/ajmc.2020.42769","article-title":"Insomnia overview: Epidemiology, pathophysiology, diagnosis and monitoring, and nonpharmacologic therapy","volume":"26","author":"Dopheide","year":"2020","journal-title":"Am. J. Manag. Care"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1016\/j.sleep.2020.08.020","article-title":"Efficacy of digital cognitive behavioural therapy for insomnia: A meta-analysis of randomised controlled trials","volume":"75","author":"Soh","year":"2020","journal-title":"Sleep Med."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1016\/j.smrv.2019.06.002","article-title":"The impact of cognitive behavioural therapy for insomnia on objective sleep parameters: A meta-analysis and systematic review","volume":"47","author":"Mitchell","year":"2019","journal-title":"Sleep Med. Rev."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Schaffarczyk, M., Rogers, B., Reer, R., and Gronwald, T. (2022). Validity of the polar H10 sensor for heart-rate variability analysis during resting state and incremental exercise in recreational men and women. Sensors, 22.","DOI":"10.3390\/s22176536"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1525","DOI":"10.1007\/s00421-019-04142-5","article-title":"RR interval signal quality of a heart rate monitor and an ECG Holter at rest and during exercise","volume":"119","author":"Schweizer","year":"2019","journal-title":"Eur. J. Appl. Physiol."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Hettiarachchi, I.T., Hanoun, S., Nahavandi, D., and Nahavandi, S. (2019). Validation of Polar OH1 optical heart rate sensor for moderate and high intensity physical activities. PLoS ONE, 14.","DOI":"10.1371\/journal.pone.0217288"},{"key":"ref_36","unstructured":"R Core Team (2017). R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"105004","DOI":"10.1088\/1361-6579\/aae021","article-title":"An open source benchmarked toolbox for cardiovascular waveform and interval analysis","volume":"39","author":"Vest","year":"2018","journal-title":"Physiol. Meas."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Hu, J., Shen, L., and Sun, G. (2018, January 18\u201323). Squeeze-and-excitation networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00745"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"809","DOI":"10.1109\/TNSRE.2021.3076234","article-title":"An attention-based deep learning approach for sleep stage classification with single-channel eeg","volume":"29","author":"Eldele","year":"2021","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Shaw, P., Uszkoreit, J., and Vaswani, A. (2018). Self-attention with relative position representations. arXiv.","DOI":"10.18653\/v1\/N18-2074"},{"key":"ref_41","unstructured":"Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., and Devin, M. (2022, June 01). TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. Available online: https:\/\/www.tensorflow.org\/."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"zsac154","DOI":"10.1093\/sleep\/zsac154","article-title":"Scoring sleep with artificial intelligence enables quantification of sleep stage ambiguity: Hypnodensity based on multiple expert scorers and auto-scoring","volume":"46","author":"Bakker","year":"2022","journal-title":"Sleep"},{"key":"ref_43","first-page":"360","article-title":"Understanding interobserver agreement: The kappa statistic","volume":"37","author":"Viera","year":"2005","journal-title":"Fam. Med."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1177\/096228029900800204","article-title":"Measuring agreement in method comparison studies","volume":"8","author":"Bland","year":"1999","journal-title":"Stat. Methods Med. Res."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"81","DOI":"10.5664\/jcsm.2350","article-title":"The American Academy of Sleep Medicine inter-scorer reliability program: Sleep stage scoring","volume":"9","author":"Rosenberg","year":"2013","journal-title":"J. Clin. Sleep Med."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1643","DOI":"10.1093\/jamia\/ocy131","article-title":"Expert-level sleep scoring with deep neural networks","volume":"25","author":"Biswal","year":"2018","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"193","DOI":"10.5664\/jcsm.9538","article-title":"Interrater reliability of sleep stage scoring: A meta-analysis","volume":"18","author":"Lee","year":"2022","journal-title":"J. Clin. Sleep Med."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"141","DOI":"10.11613\/BM.2015.015","article-title":"Understanding bland altman analysis","volume":"25","author":"Giavarina","year":"2015","journal-title":"Biochem. Medica"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1088\/0967-3334\/28\/2\/002","article-title":"Detecting REM sleep from the finger: An automatic REM sleep algorithm based on peripheral arterial tone (PAT) and actigraphy","volume":"28","author":"Herscovici","year":"2006","journal-title":"Physiol. Meas."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"571","DOI":"10.1088\/0967-3334\/29\/5\/004","article-title":"Differentiating between light and deep sleep stages using an ambulatory device based on peripheral arterial tonometry","volume":"29","author":"Bresler","year":"2008","journal-title":"Physiol. Meas."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"14149","DOI":"10.1038\/s41598-019-49703-y","article-title":"Sleep stage classification from heart-rate variability using long short-term memory neural networks","volume":"9","author":"Radha","year":"2019","journal-title":"Sci. Rep."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Habib, A., Motin, M.A., Penzel, T., Palaniswami, M., Yearwood, J., and Karmakar, C. (2022). Performance of a Convolutional Neural Network Derived from PPG Signal in Classifying Sleep Stages. IEEE Trans. Biomed. Eng., 1\u201315.","DOI":"10.1109\/TBME.2022.3219863"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"2027","DOI":"10.1088\/0967-3334\/36\/10\/2027","article-title":"Sleep stage classification with ECG and respiratory effort","volume":"36","author":"Fonseca","year":"2015","journal-title":"Physiol. Meas."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"zsaa048","DOI":"10.1093\/sleep\/zsaa048","article-title":"Automatic sleep staging using heart rate variability, body movements, and recurrent neural networks in a sleep disordered population","volume":"43","author":"Fonseca","year":"2020","journal-title":"Sleep"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"196","DOI":"10.1016\/j.smrv.2018.01.002","article-title":"Paradoxical insomnia and subjective\u2013objective sleep discrepancy: A review","volume":"40","author":"Rezaie","year":"2018","journal-title":"Sleep Med. Rev."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/5\/2390\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:38:32Z","timestamp":1760121512000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/5\/2390"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,21]]},"references-count":55,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2023,3]]}},"alternative-id":["s23052390"],"URL":"https:\/\/doi.org\/10.3390\/s23052390","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,21]]}}}