{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T18:46:54Z","timestamp":1775069214349,"version":"3.50.1"},"reference-count":27,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2023,3,26]],"date-time":"2023-03-26T00:00:00Z","timestamp":1679788800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Australian Government Research Training Program"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Sleep scoring involves the inspection of multimodal recordings of sleep data to detect potential sleep disorders. Given that symptoms of sleep disorders may be correlated with specific sleep stages, the diagnosis is typically supported by the simultaneous identification of a sleep stage and a sleep disorder. This paper investigates the automatic recognition of sleep stages and disorders from multimodal sensory data (EEG, ECG, and EMG). We propose a new distributed multimodal and multilabel decision-making system (MML-DMS). It comprises several interconnected classifier modules, including deep convolutional neural networks (CNNs) and shallow perceptron neural networks (NNs). Each module works with a different data modality and data label. The flow of information between the MML-DMS modules provides the final identification of the sleep stage and sleep disorder. We show that the fused multilabel and multimodal method improves the diagnostic performance compared to single-label and single-modality approaches. We tested the proposed MML-DMS on the PhysioNet CAP Sleep Database, with VGG16 CNN structures, achieving an average classification accuracy of 94.34% and F1 score of 0.92 for sleep stage detection (six stages) and an average classification accuracy of 99.09% and F1 score of 0.99 for sleep disorder detection (eight disorders). A comparison with related studies indicates that the proposed approach significantly improves upon the existing state-of-the-art approaches.<\/jats:p>","DOI":"10.3390\/s23073468","type":"journal-article","created":{"date-parts":[[2023,3,27]],"date-time":"2023-03-27T03:01:14Z","timestamp":1679886074000},"page":"3468","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":32,"title":["Simultaneous Sleep Stage and Sleep Disorder Detection from Multimodal Sensors Using Deep Learning"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5239-3465","authenticated-orcid":false,"given":"Yi-Hsuan","family":"Cheng","sequence":"first","affiliation":[{"name":"School of Engineering, RMIT University, Melbourne, VIC 3000, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7860-7289","authenticated-orcid":false,"given":"Margaret","family":"Lech","sequence":"additional","affiliation":[{"name":"School of Engineering, RMIT University, Melbourne, VIC 3000, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1318-8447","authenticated-orcid":false,"given":"Richardt Howard","family":"Wilkinson","sequence":"additional","affiliation":[{"name":"School of Engineering, RMIT University, Melbourne, VIC 3000, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,3,26]]},"reference":[{"key":"ref_1","unstructured":"Walker, M. (2017). Why We Sleep: The New Science of Sleep and Dreams, Penguin Random House."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1016\/j.sleep.2021.05.022","article-title":"The association between sleep quality and quality of life: A population-based study","volume":"84","author":"Lee","year":"2021","journal-title":"Sleep Med."},{"key":"ref_3","first-page":"2012","article-title":"The AASM manual for the scoring of sleep and associated events","volume":"176","author":"Berry","year":"2012","journal-title":"Rules Terminol. Tech. Specif. Darien Illinois Am. Acad. Sleep Med."},{"key":"ref_4","unstructured":"Allan, R., and Anthony, K. (1968). A Manual of Standardized Terminology, Techniques and Scoring System for Sleep Stages of Human Subjects, NIH Publication, U.S. National Institute of Neurological Diseases and Blindness, Neurological Information Network."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"622","DOI":"10.1108\/SASBE-09-2020-0139","article-title":"Artificial intelligence-based automatic visual inspection system for built heritage","volume":"11","author":"Mansuri","year":"2021","journal-title":"Smart Sustain. Built Environ."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"312","DOI":"10.1016\/j.job.2022.03.003","article-title":"Deep learning models in medical image analysis","volume":"64","author":"Tsuneki","year":"2022","journal-title":"J. Oral Biosci."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1016\/j.smrv.2011.06.003","article-title":"Sleep scoring using artificial neural networks","volume":"16","author":"Ronzhina","year":"2012","journal-title":"Sleep Med. Rev."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1007\/s10916-014-0018-0","article-title":"A Comparative Study on Classification of Sleep Stage Based on EEG Signals Using Feature Selection and Classification Algorithms","volume":"38","author":"Peker","year":"2014","journal-title":"J. Med. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Radha, M., Garcia-Molina, G., Poel, M., and Tononi, G. (2014, January 26\u201330). Comparison of feature and classifier algorithms for online automatic sleep staging based on a single EEG signal. Proceedings of the 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Chicago, IL, USA.","DOI":"10.1109\/EMBC.2014.6943976"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"79419","DOI":"10.1109\/ACCESS.2022.3194145","article-title":"A Systematic Review of Literature on Automated Sleep Scoring","volume":"10","author":"Alsolai","year":"2022","journal-title":"IEEE Access"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"101204","DOI":"10.1016\/j.smrv.2019.07.007","article-title":"Automated sleep scoring: A review of the latest approaches","volume":"48","author":"Fiorillo","year":"2019","journal-title":"Sleep Med. Rev."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"435","DOI":"10.1016\/j.procs.2017.10.026","article-title":"Sleep stage classification based on noise-reduced fractal property of heart rate variability","volume":"116","author":"Kim","year":"2017","journal-title":"Procedia Comput. Sci."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Fern\u00e1ndez-Varela, I., Hern\u00e1ndez-Pereira, E., and Moret-Bonillo, V. (2018). A convolutional network for the classification of sleep stages. Proceedings, 2.","DOI":"10.3390\/proceedings2181174"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1351","DOI":"10.1093\/jamia\/ocy064","article-title":"The National Sleep Research Resource: Towards a sleep data commons","volume":"25","author":"Zhang","year":"2018","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_15","first-page":"1077","article-title":"The sleep heart health study: Design, rationale, and methods","volume":"20","author":"Quan","year":"1997","journal-title":"Sleep"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1285","DOI":"10.1109\/TBME.2018.2872652","article-title":"Joint Classification and Prediction CNN Framework for Automatic Sleep Stage Classification","volume":"66","author":"Phan","year":"2019","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1185","DOI":"10.1109\/10.867928","article-title":"Analysis of a sleep-dependent neuronal feedback loop: The slow-wave microcontinuity of the EEG","volume":"47","author":"Kemp","year":"2000","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"e215","DOI":"10.1161\/01.CIR.101.23.e215","article-title":"PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals","volume":"101","author":"Goldberger","year":"2000","journal-title":"Circulation"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Yan, R., Li, F., Zhou, D., Ristaniemi, T., and Cong, F. (2021, January 18\u201321). A Deep Learning Model for Automatic Sleep Scoring using Multimodality Time Series. Proceedings of the 2020 28th European Signal Processing Conference (EUSIPCO), Amsterdam, The Netherlands.","DOI":"10.23919\/Eusipco47968.2020.9287518"},{"key":"ref_20","unstructured":"Zhuang, D., Rao, I., and Ibrahim, A.K. (2022). A Machine Learning Approach to Automatic Classification of Eight Sleep Disorders. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"537","DOI":"10.1016\/S1389-9457(01)00149-6","article-title":"Atlas, rules, and recording techniques for the scoring of cyclic alternating pattern (CAP) in human sleep","volume":"2","author":"Terzano","year":"2001","journal-title":"Sleep Med."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"105224","DOI":"10.1016\/j.compbiomed.2022.105224","article-title":"Automated identification of sleep disorders using wavelet-based features extracted from electrooculogram and electromyogram signals","volume":"143","author":"Sharma","year":"2022","journal-title":"Comput. Biol. Med."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Cheng, Y.H., Lech, M., and Wilkinson, R. (2021, January 13\u201315). Sleep Stage Recognition from EEG Using a Distributed Multi-Channel Decision-Making System. Proceedings of the 2021 15th International Conference on Signal Processing and Communication Systems (ICSPCS), Sydney, Australia.","DOI":"10.1109\/ICSPCS53099.2021.9660265"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Vakharia, V., Kiran, M.B., Dave, N.J., and Kagathara, U. (2017, January 22\u201325). Feature extraction and classification of machined component texture images using wavelet and artificial intelligence techniques. Proceedings of the 2017 8th International Conference on Mechanical and Aerospace Engineering (ICMAE), Prague, Czech Republic.","DOI":"10.1109\/ICMAE.2017.8038631"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Alturki, F.A., AlSharabi, K., Abdurraqeeb, A.M., and Aljalal, M. (2020). EEG signal analysis for diagnosing neurological disorders using discrete wavelet transform and intelligent techniques. Sensors, 20.","DOI":"10.3390\/s20092505"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Qassim, H., Verma, A., and Feinzimer, D. (2018, January 8\u201310). Compressed residual-VGG16 CNN model for big data places image recognition. Proceedings of the 2018 IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC), Las Vegas, NV, USA.","DOI":"10.1109\/CCWC.2018.8301729"},{"key":"ref_27","unstructured":"Simonyan, K., and Zisserman, A. (2015). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/7\/3468\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:03:25Z","timestamp":1760123005000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/7\/3468"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,26]]},"references-count":27,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2023,4]]}},"alternative-id":["s23073468"],"URL":"https:\/\/doi.org\/10.3390\/s23073468","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,26]]}}}