{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,10]],"date-time":"2026-02-10T01:17:43Z","timestamp":1770686263545,"version":"3.49.0"},"reference-count":42,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2020,12,23]],"date-time":"2020-12-23T00:00:00Z","timestamp":1608681600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"This research was sponsored by the Robert and Prudie Leibrock Professorship in Engineering at the University of Texas at Austin.","award":["-"],"award-info":[{"award-number":["-"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Background: Performance of wrist actigraphy in assessing sleep not only depends on the sensor technology of the actigraph hardware but also on the attributes of the interpretative algorithm (IA). The objective of our research was to improve assessment of sleep quality, relative to existing IAs, through development of a novel IA using deep learning methods, utilizing as input activity count and heart rate variability (HRV) metrics of different window length (number of epochs of data). Methods: Simultaneously recorded polysomnography (PSG) and wrist actigraphy data of 222 participants were utilized. Classic deep learning models were applied to: (a) activity count alone (without HRV), (b) activity count + HRV (30-s window), (c) activity count + HRV (3-min window), and (d) activity count + HRV (5-min window) to ascertain the best set of inputs. A novel deep learning model (Haghayegh Algorithm, HA), founded on best set of inputs, was developed, and its sleep scoring performance was then compared with the most popular University of California San Diego (UCSD) and Actiwatch proprietary IAs. Results: Activity count combined with HRV metrics calculated per 5-min window produced highest agreement with PSG. HA showed 84.5% accuracy (5.3\u20136.2% higher than comparator IAs), 89.5% sensitivity (6.2% higher than UCSD IA and 6% lower than Actiwatch proprietary IA), 70.0% specificity (8.2\u201334.3% higher than comparator IAs), and 58.7% Kappa agreement (16\u201323% higher than comparator IAs) in detecting sleep epochs. HA did not differ significantly from PSG in deriving sleep parameters\u2014sleep efficiency, total sleep time, sleep onset latency, and wake after sleep onset; moreover, bias and mean absolute error of the HA model in estimating them was less than the comparator IAs. HA showed, respectively, 40.9% and 54.0% Kappa agreement with PSG in detecting rapid and non-rapid eye movement (REM and NREM) epochs. Conclusions: The HA model simultaneously incorporating activity count and HRV metrics calculated per 5-min window demonstrates significantly better sleep scoring performance than existing popular IAs.<\/jats:p>","DOI":"10.3390\/s21010025","type":"journal-article","created":{"date-parts":[[2020,12,23]],"date-time":"2020-12-23T12:19:51Z","timestamp":1608725991000},"page":"25","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Deep Neural Network Sleep Scoring Using Combined Motion and Heart Rate Variability Data"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7232-2637","authenticated-orcid":false,"given":"Shahab","family":"Haghayegh","sequence":"first","affiliation":[{"name":"Department of Biostatics, T.H. Chan School of Public Health, Harvard University, Boston, MA 02115, USA"},{"name":"Department of Biomedical Engineering, Cockrell School of Engineering, The University of Texas at Austin, Austin, TX 78712, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4811-8078","authenticated-orcid":false,"given":"Sepideh","family":"Khoshnevis","sequence":"additional","affiliation":[{"name":"Department of Biomedical Engineering, Cockrell School of Engineering, The University of Texas at Austin, Austin, TX 78712, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michael H.","family":"Smolensky","sequence":"additional","affiliation":[{"name":"Department of Biomedical Engineering, Cockrell School of Engineering, The University of Texas at Austin, Austin, TX 78712, USA"},{"name":"Department of Internal Medicine, Division of Cardiology McGovern School of Medicine, The University of Texas Health Science Center at Houston, Houston, TX 77030, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kenneth R.","family":"Diller","sequence":"additional","affiliation":[{"name":"Department of Biomedical Engineering, Cockrell School of Engineering, The University of Texas at Austin, Austin, TX 78712, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Richard J.","family":"Castriotta","sequence":"additional","affiliation":[{"name":"Division of Pulmonary, Critical Care and Sleep Medicine, Keck School of Medicine, University of Southern California, Los Angeles, CA 90033, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,12,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1111\/1469-8986.3620233","article-title":"How many nights are enough? The short-term stability of sleep parameters in elderly insomniacs and normal sleepers","volume":"36","author":"Wohlgemuth","year":"1999","journal-title":"Psychophysiology"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1111\/j.1469-8986.1972.tb00745.x","article-title":"Reliability of sleep measures","volume":"9","author":"Moses","year":"1972","journal-title":"Psychophysiology"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1007\/s11325-008-0214-6","article-title":"Assessment of the test\u2013retest reliability of laboratory polysomnography","volume":"13","author":"Levendowski","year":"2009","journal-title":"Sleep Breath."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1519","DOI":"10.1016\/S0277-9536(97)00088-9","article-title":"Collecting retrospective data: Accuracy of recall after 50 years judged against historical records","volume":"45","author":"Berney","year":"1997","journal-title":"Soc. Sci. Med."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1093\/sleep\/26.2.213","article-title":"Evidence for the validity of a sleep habits survey for adolescents","volume":"26","author":"Wolfson","year":"2003","journal-title":"Sleep"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Casaccia, S., Braccili, E., Scalise, L., and Revel, G.M. (2019). Experimental assessment of sleep-related parameters by passive infrared sensors: Measurement setup, feature extraction, and uncertainty analysis. Sensors, 19.","DOI":"10.3390\/s19173773"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"519","DOI":"10.1093\/sleep\/30.4.519","article-title":"Practice parameters for the use of actigraphy in the assessment of sleep and sleep disorders: An update for 2007","volume":"30","author":"Morgenthaler","year":"2007","journal-title":"Sleep"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1111\/j.1365-2869.2009.00814.x","article-title":"Objective measurements of sleep for non-laboratory settings as alternatives to polysomnography\u2014A systematic review","volume":"20","author":"Holmes","year":"2011","journal-title":"J. Sleep Res."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1093\/sleep\/17.3.201","article-title":"Activity-based sleep-wake identification: An empirical test of methodological issues","volume":"17","author":"Sadeh","year":"1994","journal-title":"Sleep"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"461","DOI":"10.1093\/sleep\/15.5.461","article-title":"Automatic Sleep\/Wake Identification from Wrist Activity","volume":"15","author":"Cole","year":"1992","journal-title":"Sleep"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1093\/sleep\/5.4.389","article-title":"An activity-based sleep monitor system for ambulatory use","volume":"5","author":"Webster","year":"1982","journal-title":"Sleep"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/S0165-0270(00)00364-2","article-title":"Sleep estimation from wrist movement quantified by different actigraphic modalities","volume":"105","author":"Kripke","year":"2001","journal-title":"J. Neurosci. Methods"},{"key":"ref_13","unstructured":"Hersen, M. (2011). Clinician\u2019s Handbook of Child Behavioral Assessment, Elsevier."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1016\/j.smrv.2011.02.005","article-title":"Heart rate variability, sleep and sleep disorders","volume":"16","author":"Stein","year":"2012","journal-title":"Sleep Med. Rev."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"290","DOI":"10.1111\/j.1469-8986.1984.tb02936.x","article-title":"Heart Rhythm Control During Sleep in Ischemic Heart Disease","volume":"21","author":"Varoneckas","year":"1984","journal-title":"Psychophysiology"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1037","DOI":"10.1016\/S0031-9384(97)00234-5","article-title":"Vegetative background of sleep: Spectral analysis of the heart rate variability","volume":"62","author":"Scholz","year":"1997","journal-title":"Physiol. Behav."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1752","DOI":"10.1080\/07420528.2019.1679826","article-title":"Performance comparison of different interpretative algorithms utilized to derive sleep parameters from wrist actigraphy data","volume":"36","author":"Haghayegh","year":"2019","journal-title":"Chronobiol. Int."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"A457","DOI":"10.1093\/sleep\/zsaa056.1190","article-title":"1196 Machine learning derived-interpretative algorithm better differentiates sleep and wake epochs and estimates sleep parameters from wrist actigraphy data","volume":"43","author":"Haghayegh","year":"2020","journal-title":"Sleep"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1080\/07420528.2019.1682006","article-title":"Performance assessment of new-generation Fitbit technology in deriving sleep parameters and stages","volume":"37","author":"Haghayegh","year":"2020","journal-title":"Chronobiol. Int."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Haghayegh, S., Khoshnevis, S., Smolensky, M.H., Diller, K.R., and Castriotta, R.J. (2019). Accuracy of wristband fitbit models in assessing sleep: Systematic review and meta-analysis. J. Med. Internet Res., 21.","DOI":"10.2196\/preprints.16273"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1016\/j.sleep.2020.05.008","article-title":"Application of deep learning to improve sleep scoring of wrist actigraphy","volume":"74","author":"Haghayegh","year":"2020","journal-title":"Sleep Med."},{"key":"ref_22","unstructured":"(2016). ANSI\/CTA Standard: Definitions and Characteristics for Wearable Sleep Monitors, Consumer Technology Association. ANSI\/CTA\/NSF-2052.1."},{"key":"ref_23","first-page":"877","article-title":"Racial\/Ethnic Differences in Sleep Disturbances: The Multi-Ethnic Study of Atherosclerosis (MESA)","volume":"38","author":"Chen","year":"2015","journal-title":"Sleep"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1351","DOI":"10.1093\/jamia\/ocy064","article-title":"The national rleep research resource: Towards a sleep data commons","volume":"25","author":"Zhang","year":"2018","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1016\/j.cmpb.2013.07.024","article-title":"Kubios HRV\u2014Heart rate variability analysis software","volume":"113","author":"Tarvainen","year":"2014","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_26","first-page":"1","article-title":"An overview of heart rate variability metrics and norms","volume":"5","author":"Shaffer","year":"2017","journal-title":"Front. Public Heal."},{"key":"ref_27","unstructured":"Tarvainen, M.P., Lipponen, J., Niskanen, J.-P., and Ranta-aho, P.O. (2020, August 02). Kubios HRV User Guide, Kubios Oy. Available online: https:\/\/www.kubios.com\/downloads\/Kubios_HRV_Users_Guide.pdf."},{"key":"ref_28","unstructured":"Baevsky, R.M., and Berseneva, A.P. (2020, August 02). Methodical Recommendations Use Kardivar System for Determination of the Stress Level and Estimation of the Body Adaptability Standards of Measurements and Physiological Interpretation, Available online: https:\/\/www.academia.edu\/35296847\/Methodical_recommendations_USE_KARDiVAR_SYSTEM_FOR_DETERMINATION_OF_THE_STRESS_LEVEL_AND_ESTIMATION_OF_THE_BODY_ADAPTABILITY_Standards_of_measurements_and_physiological_interpretation_Moscow_Prague_2008."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"917","DOI":"10.1007\/s10618-019-00619-1","article-title":"Deep learning for time series classification: A review","volume":"33","author":"Fawaz","year":"2019","journal-title":"Data Min. Knowl. Discov."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Wang, Z., Yan, W., and Oates, T. (2017, January 14\u201319). Time series classification from scratch with deep neural networks: A strong baseline. Proceedings of the 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, USA.","DOI":"10.1109\/IJCNN.2017.7966039"},{"key":"ref_31","unstructured":"Serr\u00e0, J., Pascual, S., and Karatzoglou, A. (2018, January 8\u201310). Towards a universal neural network encoder for time series. Proceedings of the CCIA 2018, 21st International Conference of the Catalan Association for Artificial Intelligence, Alt Empord\u00e0, Catalonia, Spain."},{"key":"ref_32","unstructured":"Le Guennec, A., Malinowski, S., and Tavenard, R. (2016, January 19\u201323). Data augmentation for time series classification using convolutional neural networks. Proceedings of the ECML\/PKDD Workshop on Advanced Analytics andLearning on Temporal Data, Riva Del Garda, Italy. Available online: https:\/\/halshs.archives-ouvertes.fr\/halshs-01357973."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"100548","DOI":"10.1016\/j.softx.2020.100548","article-title":"Mcfly: Automated deep learning on time series","volume":"12","author":"Meijer","year":"2020","journal-title":"SoftwareX"},{"key":"ref_34","unstructured":"LeCun, Y., and Bengio, Y. (1998). Convolutional networks for images, speech, and time series. The Handbook of Brain Theory and Neural Networks, MIT Press."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Ord\u00f3\u00f1ez, F.J., and Roggen, D. (2016). Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors, 16.","DOI":"10.3390\/s16010115"},{"key":"ref_36","unstructured":"Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., and Isard, M. (2016, January 2\u20134). Tensorflow: A system for large-scale machine learning. Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u2019 16), Savannah, GA, USA."},{"key":"ref_37","unstructured":"Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., and Devin, M. (2020, January 30). TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems. Available online: www.tensorflow.org."},{"key":"ref_38","unstructured":"Chollet, F. (2020, January 30). Keras. Available online: https:\/\/keras.io."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1177\/096228029900800204","article-title":"Measuring agreement in method comparison studies","volume":"8","author":"Bland","year":"1999","journal-title":"Stat. Methods Med. Res."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Haghayegh, S., Kang, H.-A., Khoshnevis, S., Smolensky, M.H., and Diller, K.R. (2020). A comprehensive guideline for bland-altman and intra class correlation calculations to properly compare two methods of measurement and interpret findings. Physiol. Meas.","DOI":"10.1088\/1361-6579\/ab86d6"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1016\/j.smrv.2019.05.001","article-title":"Agreement between actigraphic and polysomnographic measures of sleep in adults with and without chronic conditions: A systematic review and meta-analysis","volume":"46","author":"Conley","year":"2019","journal-title":"Sleep Med. Rev."},{"key":"ref_42","first-page":"1","article-title":"Sleep stage prediction with raw acceleration and photoplethysmography heart rate data derived from a consumer wearable device","volume":"42","author":"Walch","year":"2019","journal-title":"Sleep"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/1\/25\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:48:37Z","timestamp":1760179717000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/1\/25"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12,23]]},"references-count":42,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2021,1]]}},"alternative-id":["s21010025"],"URL":"https:\/\/doi.org\/10.3390\/s21010025","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,12,23]]}}}