{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,1]],"date-time":"2026-06-01T23:15:07Z","timestamp":1780355707158,"version":"3.54.1"},"reference-count":39,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,12,18]],"date-time":"2024-12-18T00:00:00Z","timestamp":1734480000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2020YFC1522500"],"award-info":[{"award-number":["2020YFC1522500"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Comput. Neurosci."],"abstract":"<jats:sec><jats:title>Background<\/jats:title><jats:p>Automatic sleep staging is essential for assessing sleep quality and diagnosing sleep disorders. While previous research has achieved high classification performance, most current sleep staging networks have only been validated in healthy populations, ignoring the impact of Obstructive Sleep Apnea (OSA) on sleep stage classification. In addition, it remains challenging to effectively improve the fine-grained detection of polysomnography (PSG) and capture multi-scale transitions between sleep stages. Therefore, a more widely applicable network is needed for sleep staging.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>This paper introduces MSDC-SSNet, a novel deep learning network for automatic sleep stage classification. MSDC-SSNet transforms two channels of electroencephalogram (EEG) and one channel of electrooculogram (EOG) signals into time-frequency representations to obtain feature sequences at different temporal and frequency scales. An improved Transformer encoder architecture ensures temporal consistency and effectively captures long-term dependencies in EEG and EOG signals. The Multi-Scale Feature Extraction Module (MFEM) employs convolutional layers with varying dilation rates to capture spatial patterns from fine to coarse granularity. It adaptively fuses the weights of features to enhance the robustness of the model. Finally, multiple channel data are integrated to address the heterogeneity between different modalities effectively and alleviate the impact of OSA on sleep stages.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>We evaluated MSDC-SSNet on three public datasets and our collection of PSG records of 17 OSA patients. It achieved an accuracy of 80.4% on the OSA dataset. It also outperformed the state-of-the-art methods in terms of accuracy, F1 score, and Cohen's Kappa coefficient on the remaining three datasets.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p>The MSDC-SSRNet multi-channel sleep staging architecture proposed in this study enhances widespread system applicability by supplementing inter-channel features. It employs multi-scale attention to extract transition rules between sleep stages and effectively integrates multimodal information. Our method address the limitations of single-channel approaches, enhancing interpretability for clinical applications.<\/jats:p><\/jats:sec>","DOI":"10.3389\/fncom.2024.1505746","type":"journal-article","created":{"date-parts":[[2024,12,18]],"date-time":"2024-12-18T06:47:10Z","timestamp":1734504430000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["Multimodal sleep staging network based on obstructive sleep apnea"],"prefix":"10.3389","volume":"18","author":[{"given":"Jingxin","family":"Fan","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mingfu","family":"Zhao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Li","family":"Huang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bin","family":"Tang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lurui","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhong","family":"He","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaoling","family":"Peng","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2024,12,18]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"70","DOI":"10.1111\/j.1365-2869.1993.tb00065.x","article-title":"All-night dynamics of the human sleep EEG","volume":"2","author":"Aeschbach","year":"1993","journal-title":"J. Sleep Res"},{"key":"B2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12911-020-01332-6","article-title":"Explainability for artificial intelligence in healthcare: a multidisciplinary perspective","volume":"20","author":"Amann","year":"2020","journal-title":"BMC Med. Inform. Decis. Mak"},{"key":"B3","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1109\/EMBC.2018.8512214","article-title":"\u201cMultichannel sleep stage classification and transfer learning using convolutional neural networks,\u201d","volume-title":"2018 40th annual international conference of the IEEE engineering in medicine and biology society (EMBC)","author":"Andreotti","year":"2018"},{"key":"B4","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1186\/s40537-023-00731-6","article-title":"Research in computing-intensive simulations for nature-oriented civil-engineering and related scientific fields, using machine learning and big data: an overview of open problems","volume":"10","author":"Babovi\u0107","year":"2023","journal-title":"J. Big Data"},{"key":"B5","doi-asserted-by":"publisher","first-page":"665","DOI":"10.5664\/jcsm.6576","article-title":"Aasm scoring manual updates for 2017 (version 2.4)","volume":"13","author":"Berry","year":"2017","journal-title":"J. Clin. Sleep Med"},{"key":"B6","first-page":"176","author":"Berry","year":"2012"},{"key":"B7","doi-asserted-by":"publisher","first-page":"758","DOI":"10.1109\/TNSRE.2018.2813138","article-title":"A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time series","volume":"26","author":"Chambon","year":"2018","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng"},{"key":"B8","first-page":"126","article-title":"Overview of sleep & sleep disorders","volume":"131","author":"Chokroverty","year":"2010","journal-title":"Indian J. Med. Res"},{"key":"B9","doi-asserted-by":"publisher","first-page":"4204","DOI":"10.1109\/JBHI.2023.3284160","article-title":"Multichannelsleepnet: a transformer-based model for automatic sleep stage classification with PSG","volume":"27","author":"Dai","year":"2023","journal-title":"IEEE J. Biomed. Health Inform"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1810.04805","article-title":"Bert: pre-training of deep bidirectional transformers for language understanding","author":"Devlin","year":"2018","journal-title":"arXiv"},{"key":"B11","doi-asserted-by":"publisher","first-page":"809","DOI":"10.1109\/TNSRE.2021.3076234","article-title":"An attention-based deep learning approach for sleep stage classification with single-channel EEG","volume":"29","author":"Eldele","year":"2021","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng"},{"key":"B12","doi-asserted-by":"publisher","first-page":"e215","DOI":"10.1161\/01.CIR.101.23.e215","article-title":"Physiobank, physiotoolkit, and physionet: components of a new research resource for complex physiologic signals","volume":"101","author":"Goldberger","year":"2000","journal-title":"Circulation"},{"key":"B13","doi-asserted-by":"publisher","first-page":"1441","DOI":"10.1109\/TNSRE.2021.3098968","article-title":"Robustsleepnet: transfer learning for automated sleep staging at scale","volume":"29","author":"Guillot","year":"2021","journal-title":"IEEE Trans. Neural Syst.Rehabil. Eng"},{"key":"B14","doi-asserted-by":"publisher","first-page":"3028","DOI":"10.3390\/app12063028","article-title":"Classification of sleep stage with biosignal images using convolutional neural networks","volume":"12","author":"Joe","year":"2022","journal-title":"Appl. Sci"},{"key":"B15","doi-asserted-by":"publisher","first-page":"1185","DOI":"10.1109\/10.867928","article-title":"Analysis of a sleep-dependent neuronal feedback loop: the slow-wave microcontinuity of the EEG","volume":"47","author":"Kemp","year":"2000","journal-title":"IEEE Trans. Biomed. Eng"},{"key":"B16","doi-asserted-by":"publisher","first-page":"102981","DOI":"10.1016\/j.bspc.2021.102981","article-title":"An EEG spectrogram-based automatic sleep stage scoring method via data augmentation, ensemble convolution neural network, and expert knowledge","volume":"70","author":"Kuo","year":"2021","journal-title":"Biomed. Signal Process. Control"},{"key":"B17","doi-asserted-by":"publisher","first-page":"122551","DOI":"10.1016\/j.eswa.2023.122551","article-title":"Sleepyco: automatic sleep scoring with feature pyramid and contrastive learning","volume":"240","author":"Lee","year":"2024","journal-title":"Expert Syst. Appl"},{"key":"B18","doi-asserted-by":"publisher","first-page":"102203","DOI":"10.1016\/j.bspc.2020.102203","article-title":"End-to-end sleep staging using convolutional neural network in raw single-channel EEG","volume":"63","author":"Li","year":"2021","journal-title":"Biomed. Signal Process. Control"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2023\/439","article-title":"\u201cTeacher assistant-based knowledge distillation extracting multi-level features on single channel sleep EEG,\u201d","author":"Liang","year":"2023","journal-title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence (IJCAI-23)"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1312.4400","article-title":"Network in network","author":"Lin","year":"2013","journal-title":"arXiv"},{"key":"B21","first-page":"2117","article-title":"\u201cFeature pyramid networks for object detection,\u201d","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Lin","year":"2017"},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1711.05101","article-title":"Decoupled weight decay regularization","author":"Loshchilov","year":"2017","journal-title":"arXiv"},{"key":"B23","doi-asserted-by":"publisher","first-page":"573","DOI":"10.5665\/sleep.2548","article-title":"Performance of an automated polysomnography scoring system versus computer-assisted manual scoring","volume":"36","author":"Malhotra","year":"2013","journal-title":"Sleep"},{"key":"B24","doi-asserted-by":"publisher","first-page":"zsaa072","DOI":"10.1093\/sleep\/zsaa072","article-title":"The prevalence of common sleep disorders in young adults: a descriptive population-based study","volume":"43","author":"McArdle","year":"2020","journal-title":"Sleep"},{"key":"B25","doi-asserted-by":"publisher","first-page":"71","DOI":"10.1016\/j.compbiomed.2019.01.013","article-title":"Cascaded lstm recurrent neural network for automated sleep stage classification using single-channel EEG signals","volume":"106","author":"Michielli","year":"2019","journal-title":"Comput. Biol. Med"},{"key":"B26","doi-asserted-by":"publisher","first-page":"e0216456","DOI":"10.1371\/journal.pone.0216456","article-title":"Sleepeegnet: automated sleep stage scoring with sequence to sequence deep learning approach","volume":"14","author":"Mousavi","year":"2019","journal-title":"PLoS ONE"},{"key":"B27","doi-asserted-by":"publisher","first-page":"105203","DOI":"10.1016\/j.bspc.2023.105203","article-title":"Sleepvitransformer: patch-based sleep spectrogram transformer for automatic sleep staging","volume":"86","author":"Peng","year":"2023","journal-title":"Biomed. Signal Process. Control"},{"key":"B28","doi-asserted-by":"publisher","first-page":"1285","DOI":"10.1109\/TBME.2018.2872652","article-title":"Joint classification and prediction cnn framework for automatic sleep stage classification","volume":"66","author":"Phan","year":"","journal-title":"IEEE Trans. Biomed. Eng"},{"key":"B29","doi-asserted-by":"publisher","first-page":"400","DOI":"10.1109\/TNSRE.2019.2896659","article-title":"Seqsleepnet: End-to-end hierarchical recurrent neural network for sequence-to-sequence automatic sleep staging","volume":"27","author":"Phan","year":"","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng"},{"key":"B30","doi-asserted-by":"publisher","first-page":"2456","DOI":"10.1109\/TBME.2022.3147187","article-title":"Sleeptransformer: automatic sleep staging with interpretability and uncertainty quantification","volume":"69","author":"Phan","year":"2022","journal-title":"IEEE Trans. Biomed. Eng"},{"key":"B31","doi-asserted-by":"publisher","first-page":"1835","DOI":"10.1109\/TCBB.2019.2912955","article-title":"Deep learning for automated feature discovery and classification of sleep stages","volume":"17","author":"Sokolovsky","year":"2019","journal-title":"IEEE\/ACM Trans. Comput. Biol. Bioinform"},{"key":"B32","doi-asserted-by":"publisher","first-page":"1998","DOI":"10.1109\/TNSRE.2017.2721116","article-title":"Deepsleepnet: a model for automatic sleep stage scoring based on raw single-channel EEG","volume":"25","author":"Supratak","year":"2017","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng"},{"key":"B33","first-page":"641","article-title":"\u201cTinysleepnet: an efficient deep learning model for sleep stage scoring based on raw single-channel EEG,\u201d","volume-title":"2020 42nd Annual International Conference of the IEEE Engineering in Medicine &Biology Society (EMBC)","author":"Supratak","year":"2020"},{"key":"B34","doi-asserted-by":"publisher","first-page":"103548","DOI":"10.1016\/j.bspc.2022.103548","article-title":"Deep adaptation network for subject-specific sleep stage classification based on a single-lead ECG","volume":"75","author":"Tang","year":"2022","journal-title":"Biomed. Signal Process. Control"},{"key":"B35","first-page":"52","article-title":"\u201cCrossnorm and selfnorm for generalization under distribution shifts,\u201d","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision","author":"Tang","year":"2021"},{"key":"B36","doi-asserted-by":"publisher","first-page":"1074","DOI":"10.3390\/bioengineering10091074","article-title":"Rem sleep stage identification with raw single-channel EEG","volume":"10","author":"Toban","year":"2023","journal-title":"Bioengineering"},{"key":"B37","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1038\/nature19773","article-title":"Circuit-based interrogation of sleep control","volume":"538","author":"Weber","year":"2016","journal-title":"Nature"},{"key":"B38","doi-asserted-by":"publisher","first-page":"105089","DOI":"10.1016\/j.cmpb.2019.105089","article-title":"Orthogonal convolutional neural networks for automatic sleep stage classification based on single-channel EEG","volume":"183","author":"Zhang","year":"2020","journal-title":"Comput. Methods Programs Biomed"},{"key":"B39","doi-asserted-by":"publisher","first-page":"106806","DOI":"10.1016\/j.cmpb.2022.106806","article-title":"Sleepcontextnet: a temporal context network for automatic sleep staging based single-channel EEG","volume":"220","author":"Zhao","year":"2022","journal-title":"Comput. Methods Programs Biomed"}],"container-title":["Frontiers in Computational Neuroscience"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fncom.2024.1505746\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,18]],"date-time":"2024-12-18T06:47:16Z","timestamp":1734504436000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fncom.2024.1505746\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,18]]},"references-count":39,"alternative-id":["10.3389\/fncom.2024.1505746"],"URL":"https:\/\/doi.org\/10.3389\/fncom.2024.1505746","relation":{},"ISSN":["1662-5188"],"issn-type":[{"value":"1662-5188","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,18]]},"article-number":"1505746"}}