{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,7]],"date-time":"2026-04-07T16:09:59Z","timestamp":1775578199866,"version":"3.50.1"},"reference-count":45,"publisher":"Oxford University Press (OUP)","issue":"12","license":[{"start":{"date-parts":[[2018,11,17]],"date-time":"2018-11-17T00:00:00Z","timestamp":1542412800000},"content-version":"vor","delay-in-days":1,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"funder":[{"DOI":"10.13039\/100008503","name":"Center for Integration of Medicine and Innovative Technology","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100008503","id-type":"DOI","asserted-by":"crossref"}]},{"name":"NIH-NINDS","award":["1K23NS090900"],"award-info":[{"award-number":["1K23NS090900"]}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["IIS-1418511"],"award-info":[{"award-number":["IIS-1418511"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["CCF-1533768"],"award-info":[{"award-number":["CCF-1533768"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"NIH","doi-asserted-by":"publisher","award":["1R01MD011682-01"],"award-info":[{"award-number":["1R01MD011682-01"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"NIH","doi-asserted-by":"publisher","award":["R56HL138415"],"award-info":[{"award-number":["R56HL138415"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2018,12,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objectives<\/jats:title>\n                  <jats:p>Scoring laboratory polysomnography (PSG) data remains a manual task of visually annotating 3 primary categories: sleep stages, sleep disordered breathing, and limb movements. Attempts to automate this process have been hampered by the complexity of PSG signals and physiological heterogeneity between patients. Deep neural networks, which have recently achieved expert-level performance for other complex medical tasks, are ideally suited to PSG scoring, given sufficient training data.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Methods<\/jats:title>\n                  <jats:p>We used a combination of deep recurrent and convolutional neural networks (RCNN) for supervised learning of clinical labels designating sleep stages, sleep apnea events, and limb movements. The data for testing and training were derived from 10\u00a0000 clinical PSGs and 5804 research PSGs.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>When trained on the clinical dataset, the RCNN reproduces PSG diagnostic scoring for sleep staging, sleep apnea, and limb movements with accuracies of 87.6%, 88.2% and 84.7% on held-out test data, a level of performance comparable to human experts. The RCNN model performs equally well when tested on the independent research PSG database. Only small reductions in accuracy were noted when training on limited channels to mimic at-home monitoring devices: frontal leads only for sleep staging, and thoracic belt signals only for the apnea-hypopnea index.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusions<\/jats:title>\n                  <jats:p>By creating accurate deep learning models for sleep scoring, our work opens the path toward broader and more timely access to sleep diagnostics. Accurate scoring automation can improve the utility and efficiency of in-lab and at-home approaches to sleep diagnostics, potentially extending the reach of sleep expertise beyond specialty clinics.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocy131","type":"journal-article","created":{"date-parts":[[2018,9,22]],"date-time":"2018-09-22T06:42:07Z","timestamp":1537598527000},"page":"1643-1650","source":"Crossref","is-referenced-by-count":249,"title":["Expert-level sleep scoring with deep neural networks"],"prefix":"10.1093","volume":"25","author":[{"given":"Siddharth","family":"Biswal","sequence":"first","affiliation":[{"name":"School of Computational Science and Engineering, Georgia Institute of Technology, Atlanta, GA, USA"}]},{"given":"Haoqi","family":"Sun","sequence":"additional","affiliation":[{"name":"Neurology Department, Massachusetts General Hospital, Wang 720, Boston, MA, USA"}]},{"given":"Balaji","family":"Goparaju","sequence":"additional","affiliation":[{"name":"Neurology Department, Massachusetts General Hospital, Wang 720, Boston, MA, USA"},{"name":"Division of Sleep Medicine, Harvard Medical School, Boston, MA, USA"}]},{"given":"M Brandon","family":"Westover","sequence":"additional","affiliation":[{"name":"Neurology Department, Massachusetts General Hospital, Wang 720, Boston, MA, USA"}]},{"given":"Jimeng","family":"Sun","sequence":"additional","affiliation":[{"name":"School of Computational Science and Engineering, Georgia Institute of Technology, Atlanta, GA, USA"}]},{"given":"Matt T","family":"Bianchi","sequence":"additional","affiliation":[{"name":"Neurology Department, Massachusetts General Hospital, Wang 720, Boston, MA, USA"},{"name":"Division of Sleep Medicine, Harvard Medical School, Boston, MA, USA"}]}],"member":"286","published-online":{"date-parts":[[2018,11,16]]},"reference":[{"issue":"7","key":"2020110613012688700_ocy131-B1","doi-asserted-by":"crossref","first-page":"706","DOI":"10.1001\/jama.2013.193","article-title":"Insomnia","volume":"309","author":"Buysse","year":"2013","journal-title":"JAMA"},{"issue":"1","key":"2020110613012688700_ocy131-B2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.jsmc.2015.10.011","article-title":"Sleep in neurodegenerative diseases","volume":"11","author":"Iranzo","year":"2016","journal-title":"Sleep Med Clin"},{"issue":"9","key":"2020110613012688700_ocy131-B3","first-page":"1155","article-title":"Obstructive sleep apnea: diagnosis, epidemiology, and economics","volume":"55","author":"Kapur","year":"2010","journal-title":"Respir Care"},{"issue":"10","key":"2020110613012688700_ocy131-B4","first-page":"1322","article-title":"Sleep-disordered breathing and cardiovascular disorders","volume":"55","author":"Budhiraja","year":"2010","journal-title":"Respir Care"},{"key":"2020110613012688700_ocy131-B5","first-page":"573","article-title":"Obstructive sleep apnea and risk of motor vehicle crash: systematic review and meta-analysis. J Clin Sleep Med","author":"Tregear","year":"2009"},{"issue":"2","key":"2020110613012688700_ocy131-B6","doi-asserted-by":"crossref","first-page":"533","DOI":"10.1016\/j.aap.2009.12.004","article-title":"Sleep disorders, medical conditions, and road accident risk","volume":"43","author":"Smolensky","year":"2011","journal-title":"Accid Anal Prev"},{"issue":"11","key":"2020110613012688700_ocy131-B7","doi-asserted-by":"crossref","first-page":"1015","DOI":"10.2165\/11537390-000000000-00000","article-title":"Economic implications of sleep disorders","volume":"28","author":"Skaer","year":"2010","journal-title":"Pharmacoeconomics"},{"issue":"6","key":"2020110613012688700_ocy131-B8","first-page":"695","article-title":"An integrated health-economic analysis of diagnostic and therapeutic strategies in the treatment of moderate-to-severe obstructive sleep apnea","volume":"34","author":"Pietzsch","year":"2011","journal-title":"Sleep"},{"issue":"6","key":"2020110613012688700_ocy131-B9","doi-asserted-by":"crossref","first-page":"427","DOI":"10.1016\/j.smrv.2009.02.004","article-title":"A systematic review of continuous positive airway pressure for obstructive sleep apnoea-hypopnoea syndrome","volume":"13","author":"McDaid","year":"2009","journal-title":"Sleep Med Rev"},{"issue":"1049","key":"2020110613012688700_ocy131-B10","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1136\/postgradmedj-2012-131340","article-title":"Obstructive sleep apnoea in adults","volume":"89","author":"Usmani","year":"2013","journal-title":"Postgrad Med J"},{"issue":"6","key":"2020110613012688700_ocy131-B11","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1016\/j.smrv.2010.01.003","article-title":"Societal costs of insomnia","volume":"14","author":"Leger","year":"2010","journal-title":"Sleep Med Rev"},{"issue":"10","key":"2020110613012688700_ocy131-B12","doi-asserted-by":"crossref","DOI":"10.1093\/sleep\/zsx139","article-title":"Large-scale automated sleep staging","volume":"40","author":"Sun","year":"2017","journal-title":"Sleep"},{"issue":"7639","key":"2020110613012688700_ocy131-B13","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1038\/nature21056","article-title":"Dermatologist-level classification of skin cancer with deep neural networks","volume":"542","author":"Esteva","year":"2017","journal-title":"Nature"},{"issue":"22","key":"2020110613012688700_ocy131-B14","doi-asserted-by":"crossref","first-page":"2402","DOI":"10.1001\/jama.2016.17216","article-title":"Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs","volume":"316","author":"Gulshan","year":"2016","journal-title":"JAMA"},{"issue":"3","key":"2020110613012688700_ocy131-B15","doi-asserted-by":"crossref","first-page":"230","DOI":"10.3414\/ME09-01-0054","article-title":"Classification of sleep stages using multi-wavelet time frequency entropy and LDA","volume":"49","author":"Fraiwan","year":"2010","journal-title":"Methods Inf Med"},{"key":"2020110613012688700_ocy131-B16","doi-asserted-by":"crossref","first-page":"94","DOI":"10.1016\/j.jneumeth.2015.01.022","article-title":"Learning machines and sleeping brains: automatic sleep stage classification using decision-tree multi-class support vector machines","volume":"250","author":"Lajnef","year":"2015","journal-title":"J Neurosci Methods"},{"issue":"1","key":"2020110613012688700_ocy131-B17","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1016\/j.jneumeth.2011.12.022","article-title":"A rule-based automatic sleep staging method","volume":"205","author":"Liang","year":"2012","journal-title":"J Neurosci Methods"},{"issue":"3","key":"2020110613012688700_ocy131-B18","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1159\/000085205","article-title":"health solution for automatic sleep classification according to Rechtschaffen and Kales: validation study of the Somnolyzer 24 x 7 utilizing the Siesta database","volume":"51","author":"Anderer","year":"2005","journal-title":"Neuropsychobiology"},{"issue":"11","key":"2020110613012688700_ocy131-B19","doi-asserted-by":"crossref","first-page":"1587","DOI":"10.1093\/sleep\/30.11.1587","article-title":"Automatic analysis of single-channel sleep EEG: validation in healthy individuals","volume":"30","author":"Berthomier","year":"2007","journal-title":"Sleep"},{"key":"2020110613012688700_ocy131-B20","first-page":"101","article-title":"Evaluation of an automated single-channel sleep staging algorithm","volume":"7","author":"Wang","year":"2015","journal-title":"Nat Sci Sleep"},{"key":"2020110613012688700_ocy131-B21","doi-asserted-by":"crossref","first-page":"107.","DOI":"10.1016\/j.jneumeth.2016.07.012","article-title":"A decision support system for automatic sleep staging from EEG signals using tunable Q-factor wavelet transform and spectral features","volume":"271","author":"Hassan","year":"2016","journal-title":"J Neurosci Methods"},{"issue":"10","key":"2020110613012688700_ocy131-B22","doi-asserted-by":"crossref","first-page":"1555","DOI":"10.5665\/sleep.5046","article-title":"Computer-assisted automated scoring of polysomnograms using the somnolyzer system","volume":"38","author":"Punjabi","year":"2015","journal-title":"Sleep"},{"issue":"4","key":"2020110613012688700_ocy131-B23","doi-asserted-by":"crossref","first-page":"573","DOI":"10.5665\/sleep.2548","article-title":"Performance of an automated polysomnography scoring system versus computer-assisted manual scoring","volume":"36","author":"Malhotra","year":"2013","journal-title":"Sleep"},{"issue":"4","key":"2020110613012688700_ocy131-B24","doi-asserted-by":"crossref","first-page":"250","DOI":"10.1159\/000320864","article-title":"Computer-assisted sleep classification according to the standard of the American Academy of Sleep Medicine: validation study of the AASM version of the Somnolyzer 24 x 7","volume":"62","author":"Anderer","year":"2010","journal-title":"Neuropsychobiology"},{"issue":"1","key":"2020110613012688700_ocy131-B25","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1093\/sleep\/19.1.26","article-title":"Sleep stage scoring using the neural network model: comparison between visual and automatic analysis in normal subjects and patients","volume":"19","author":"Schaltenbrand","year":"1996","journal-title":"Sleep"},{"issue":"05","key":"2020110613012688700_ocy131-B26","doi-asserted-by":"crossref","first-page":"735","DOI":"10.5664\/jcsm.5808","article-title":"Accuracy of automatic polysomnography scoring using frontal electrodes","volume":"12","author":"Younes","year":"2016","journal-title":"J Clin Sleep Med"},{"issue":"02","key":"2020110613012688700_ocy131-B27","doi-asserted-by":"crossref","first-page":"245","DOI":"10.5664\/jcsm.6456","article-title":"Performance of a new portable wireless sleep monitor","volume":"13","author":"Younes","year":"2017","journal-title":"J Clin Sleep Med"},{"issue":"2","key":"2020110613012688700_ocy131-B28","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1111\/j.1365-2869.2011.00944.x","article-title":"Validation of an automated wireless system to monitor sleep in healthy adults","volume":"21","author":"Shambroom","year":"2012","journal-title":"J Sleep Res"},{"key":"2020110613012688700_ocy131-B29","first-page":"00633 2017","article-title":"Deep convolutional neural networks for interpretable analysis of EEG sleep stage scoring","volume":"1710","author":"Vilamala","journal-title":"arXiv"},{"issue":"4","key":"2020110613012688700_ocy131-B30","doi-asserted-by":"crossref","first-page":"435","DOI":"10.1177\/0142331215587568","article-title":"Automatic sleep stage classification based on sparse deep belief net and combination of multiple classifiers","volume":"38","author":"Zhang","year":"2016","journal-title":"Trans Inst Meas Control"},{"issue":"10","key":"2020110613012688700_ocy131-B31","doi-asserted-by":"crossref","first-page":"1419","DOI":"10.1093\/jamia\/ocy068","article-title":"Opportunities and challenges in developing deep learning models using electronic health records data: a systematic review","volume":"25","author":"Xiao","year":"2018","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613012688700_ocy131-B32","doi-asserted-by":"crossref","first-page":"1055","DOI":"10.1109\/PROC.1982.12433","article-title":"Spectrum estimation and harmonic analysis","volume":"70","author":"Thomson","year":"1982","journal-title":"Proc IEEE"},{"issue":"1","key":"2020110613012688700_ocy131-B33","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1016\/j.jneumeth.2010.06.020","article-title":"Chronux: a platform for analyzing neural signals","volume":"192","author":"Bokil","year":"2010","journal-title":"J Neurosci Methods"},{"issue":"2","key":"2020110613012688700_ocy131-B34","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1016\/j.jneumeth.2006.07.011","article-title":"Comparing spectra and coherences for groups of unequal size","volume":"159","author":"Bokil","year":"2007","journal-title":"J Neurosci Methods"},{"key":"2020110613012688700_ocy131-B35","article-title":"Batch normalization: accelerating deep network training by reducing internal covariate shift","volume":"1502","author":"Ioffe","year":"2015","journal-title":"ArXiv"},{"key":"2020110613012688700_ocy131-B36","first-page":"249","article-title":"Understanding the difficulty of training deep feedforward neural networks","volume":"9","author":"Glorot","year":"2010","journal-title":"Proc Mach Learn Res"},{"issue":"2","key":"2020110613012688700_ocy131-B37","doi-asserted-by":"crossref","first-page":"121","DOI":"10.5664\/jcsm.26814","article-title":"The visual scoring of sleep in adults","volume":"3","author":"Silber","year":"2007","journal-title":"J Clin Sleep Med"},{"issue":"1","key":"2020110613012688700_ocy131-B38","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1111\/j.1365-2869.2008.00700.x","article-title":"Interrater reliability for sleep scoring according to the Rechtschaffen & Kales and the new AASM standard","volume":"18","author":"Danker-Hopfe","year":"2009","journal-title":"J Sleep Res"},{"issue":"4","key":"2020110613012688700_ocy131-B39","doi-asserted-by":"crossref","first-page":"591","DOI":"10.5665\/sleep.2552","article-title":"Agreement in the scoring of respiratory events and sleep among international sleep centers","volume":"36","author":"Magalang","year":"2013","journal-title":"Sleep"},{"issue":"2","key":"2020110613012688700_ocy131-B40","doi-asserted-by":"crossref","first-page":"169","DOI":"10.5664\/jcsm.26818","article-title":"The scoring of respiratory events in sleep: reliability and validity","volume":"3","author":"Redline","year":"2007","journal-title":"J Clin Sleep Med"},{"issue":"1","key":"2020110613012688700_ocy131-B41","doi-asserted-by":"crossref","first-page":"42.","DOI":"10.1186\/s12883-017-0821-6","article-title":"Validation of a leg movements count and periodic leg movements analysis in a custom polysomnography system","volume":"17","author":"Stefani","year":"2017","journal-title":"BMC Neurol"},{"key":"2020110613012688700_ocy131-B42","first-page":"2579","article-title":"Visualizing data using t-SNE","volume":"9","author":"van der Maaten","year":"2008","journal-title":"J Mach Learn Res"},{"issue":"5","key":"2020110613012688700_ocy131-B43","doi-asserted-by":"crossref","first-page":"531","DOI":"10.5664\/JCSM.1328","article-title":"Obstructive sleep apnea devices for out-of-center (OOC) testing: technology evaluation","volume":"7","author":"Collop","year":"2011","journal-title":"J Clin Sleep Med"},{"key":"2020110613012688700_ocy131-B44","article-title":"Sleep devices: wearables and nearables, informational and interventional, consumer and clinical","author":"Bianchi","year":"2017","journal-title":"Metabolism"},{"key":"2020110613012688700_ocy131-B45","article-title":"FDA Announces New Steps to Empower Consumers and Advance Digital Healthcare","author":"Gottlieb","year":"2017","journal-title":"Secondary FDA Announces New Steps to Empower Consumers and Advance Digital Healthcare"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/25\/12\/1643\/34150943\/ocy131.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/25\/12\/1643\/34150943\/ocy131.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,11,6]],"date-time":"2020-11-06T18:49:37Z","timestamp":1604688577000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/25\/12\/1643\/5185596"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,11,16]]},"references-count":45,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2018,11,16]]},"published-print":{"date-parts":[[2018,12,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocy131","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2018,12]]},"published":{"date-parts":[[2018,11,16]]}}}