{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,4]],"date-time":"2026-03-04T17:18:10Z","timestamp":1772644690425,"version":"3.50.1"},"reference-count":48,"publisher":"MDPI AG","issue":"16","license":[{"start":{"date-parts":[[2021,8,11]],"date-time":"2021-08-11T00:00:00Z","timestamp":1628640000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Sleep Apnea is a breathing disorder occurring during sleep. Older people suffer most from this disease. In-time diagnosis of apnea is needed which can be observed by the application of a proper health monitoring system. In this work, we focus on Obstructive Sleep Apnea (OSA) detection from the Electrocardiogram (ECG) signals obtained through the body sensors. Our work mainly consists of an experimental study of different ensemble techniques applied on three deep learning models\u2014two Convolutional Neural Network (CNN) based models, and a combination of CNN and Long Short-Term Memory (LSTM) models, which were previously proposed in the OSA detection domain. We have chosen four ensemble techniques\u2014majority voting, sum rule and Choquet integral based fuzzy fusion and trainable ensemble using Multi-Layer Perceptron (MLP) for our case study. All the experiments are conducted on the benchmark PhysioNet Apnea-ECG Database. Finally, we have achieved highest OSA detection accuracy of 85.58% using the MLP based ensemble approach. Our best result is also able to surpass many of state-of-the-art methods.<\/jats:p>","DOI":"10.3390\/s21165425","type":"journal-article","created":{"date-parts":[[2021,8,11]],"date-time":"2021-08-11T08:35:52Z","timestamp":1628670952000},"page":"5425","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":31,"title":["Ensemble of Deep Learning Models for Sleep Apnea Detection: An Experimental Study"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6013-1067","authenticated-orcid":false,"given":"Debadyuti","family":"Mukherjee","sequence":"first","affiliation":[{"name":"Department of Computer Science and Engineering, Jadavpur University, Kolkata 700032, India"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4846-3410","authenticated-orcid":false,"given":"Koustav","family":"Dhar","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, Jadavpur University, Kolkata 700032, India"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5118-0812","authenticated-orcid":false,"given":"Friedhelm","family":"Schwenker","sequence":"additional","affiliation":[{"name":"Institute of Neural Information Processing, University of Ulm, 89069 Ulm, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8813-4086","authenticated-orcid":false,"given":"Ram","family":"Sarkar","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, Jadavpur University, Kolkata 700032, India"}]}],"member":"1968","published-online":{"date-parts":[[2021,8,11]]},"reference":[{"key":"ref_1","first-page":"3","article-title":"The internet of things in healthcare: An overview","volume":"1","author":"Yin","year":"2016","journal-title":"J. Ind. Inf. Integr."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"469","DOI":"10.1109\/TITB.2012.2188299","article-title":"Real-time sleep apnea detection by classifier combination","volume":"16","author":"Xie","year":"2012","journal-title":"IEEE Trans. Inf. Technol. Biomed."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"136","DOI":"10.1513\/pats.200709-155MG","article-title":"The epidemiology of adult obstructive sleep apnea","volume":"5","author":"Punjabi","year":"2008","journal-title":"Proc. Am. Thorac. Soc."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1016\/j.pcad.2008.08.001","article-title":"Epidemiology, risk factors, and consequences of obstructive sleep apnea and short sleep duration","volume":"51","author":"Patel","year":"2009","journal-title":"Prog. Cardiovasc. Dis."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Almazaydeh, L., Elleithy, K., and Faezipour, M. (2012, January 6\u20138). Detection of obstructive sleep apnea through ECG signal features. Proceedings of the 2012 IEEE International Conference on Electro\/Information Technology, Indianapolis, IN, USA.","DOI":"10.1109\/EIT.2012.6220730"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"162","DOI":"10.21629\/JSEE.2017.01.18","article-title":"Convolutional neural networks for time series classification","volume":"28","author":"Zhao","year":"2017","journal-title":"J. Syst. Eng. Electron."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"15813","DOI":"10.1007\/s11042-018-6161-8","article-title":"Obstructive sleep apnea detection using ecg-sensor with convolutional neural networks","volume":"79","author":"Wang","year":"2020","journal-title":"Multimed. Tools Appl."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Sharan, R.V., Berkovsky, S., Xiong, H., and Coiera, E. (2020, January 20\u201324). ECG-Derived Heart Rate Variability Interpolation and 1-D Convolutional Neural Networks for Detecting Sleep Apnea. Proceedings of the 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Montreal, QC, Canada.","DOI":"10.1109\/EMBC44109.2020.9175998"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long Short-Term Memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Almutairi, H., Hassan, G.M., and Datta, A. (2021, January 18\u201321). Detection of Obstructive Sleep Apnoea by ECG signals using Deep Learning Architectures. Proceedings of the 2020 28th European Signal Processing Conference (EUSIPCO), Amsterdam, The Netherlands.","DOI":"10.23919\/Eusipco47968.2020.9287360"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Dietterich, T.G. (2000). Ensemble methods in machine learning. Lecture Notes in Computer Science, Proceedings of the International Workshop on Multiple Classifier Systems, Cagliari, Italy, 21\u201323 June 2000, Springer.","DOI":"10.1007\/3-540-45014-9_1"},{"key":"ref_12","unstructured":"Penzel, T., Moody, G.B., Mark, R.G., Goldberger, A.L., and Peter, J.H. (2000, January 24\u201327). The apnea-ECG database. Proceedings of the Computers in Cardiology 2000. Vol. 27 (Cat. 00CH37163), Cambridge, MA, USA."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1007\/BF00994018","article-title":"Support-vector networks","volume":"20","author":"Cortes","year":"1995","journal-title":"Mach. Learn."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Guo, G., Wang, H., Bell, D., Bi, Y., and Greer, K. (2003). KNN model-based approach in classification. Lecture Notes in Computer Science, Proceedings of the OTM Confederated International Conferences \u201cOn the Move to Meaningful Internet Systems\u201d, Catania, Italy, 3\u20137 November 2003, Springer.","DOI":"10.1007\/978-3-540-39964-3_62"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2360","DOI":"10.1111\/j.1365-2702.2008.02323.x","article-title":"Evaluation of the performance of using mean absolute amplitude analysis of thoracic and abdominal signals for immediate indication of sleep apnoea events","volume":"17","author":"Ng","year":"2008","journal-title":"J. Clin. Nurs."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"399","DOI":"10.1088\/0967-3334\/27\/4\/006","article-title":"Nonlinear characteristics of blood oxygen saturation from nocturnal oximetry for obstructive sleep apnoea detection","volume":"27","author":"Alvarez","year":"2006","journal-title":"Physiol. Meas."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"zsab018","DOI":"10.1093\/sleep\/zsab018","article-title":"Heart rate variability during wakefulness as a marker of obstructive sleep apnea severity","volume":"44","author":"Qin","year":"2021","journal-title":"Sleep"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Cheng, M., Sori, W.J., Jiang, F., Khan, A., and Liu, S. (2017, January 21\u201324). Recurrent neural network based classification of ECG signal features for obstruction of sleep apnea detection. Proceedings of the 2017 IEEE International Conference on Computational Science and Engineering (CSE) and IEEE International Conference on Embedded and Ubiquitous Computing (EUC), Guangzhou, China.","DOI":"10.1109\/CSE-EUC.2017.220"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Marhon, S.A., Cameron, C.J., and Kremer, S.C. (2013). Recurrent neural networks. Handbook on Neural Information Processing, Springer.","DOI":"10.1007\/978-3-642-36657-4_2"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1285","DOI":"10.1109\/JBHI.2013.2292928","article-title":"An online sleep apnea detection method based on recurrence quantification analysis","volume":"18","author":"Nguyen","year":"2013","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1016\/j.bspc.2016.05.009","article-title":"Computer-aided obstructive sleep apnea detection using normal inverse Gaussian parameters and adaptive boosting","volume":"29","author":"Hassan","year":"2016","journal-title":"Biomed. Signal Process. Control"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1006\/jcss.1997.1504","article-title":"A decision-theoretic generalization of on-line learning and an application to boosting","volume":"55","author":"Freund","year":"1997","journal-title":"J. Comput. Syst. Sci."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Wang, T., Lu, C., and Shen, G. (2019). Detection of sleep apnea from single-lead ECG signal using a time window artificial neural network. BioMed Res. Int., 2019.","DOI":"10.1155\/2019\/9768072"},{"key":"ref_25","first-page":"1","article-title":"Multiscale Deep Neural Network for Obstructive Sleep Apnea Detection Using RR Interval From Single-Lead ECG Signal","volume":"70","author":"Shen","year":"2021","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Chang, H.Y., Yeh, C.Y., Lee, C.T., and Lin, C.C. (2020). A sleep apnea detection system based on a one-dimensional deep convolution neural network model using single-lead electrocardiogram. Sensors, 20.","DOI":"10.3390\/s20154157"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Thompson, S., Fergus, P., Chalmers, C., and Reilly, D. (2020, January 19\u201324). Detection of Obstructive Sleep Apnoea Using Features Extracted from Segmented Time-Series ECG Signals Using a One Dimensional Convolutional Neural Network. Proceedings of the 2020 International Joint Conference on Neural Networks (IJCNN), Glasgow, UK.","DOI":"10.1109\/IJCNN48605.2020.9207470"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"104532","DOI":"10.1016\/j.compbiomed.2021.104532","article-title":"SCNN: Scalogram-based Convolutional Neural Network to Detect Obstructive Sleep Apnea using Single-lead Electrocardiogram Signals","volume":"134","author":"Mashrur","year":"2021","journal-title":"Comput. Biol. Med."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Fau\u00dfer, S., and Schwenker, F. (2011). Ensemble methods for reinforcement learning with function approximation. Lecture Notes in Computer Science, Proceedings of the International Workshop on Multiple Classifier Systems, Naples, Italy, 15\u201317 June 2011, Springer.","DOI":"10.1007\/978-3-642-21557-5_8"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1007\/s00180-012-0374-5","article-title":"Ensemble Gaussian mixture models for probability density estimation","volume":"28","author":"Glodek","year":"2013","journal-title":"Comput. Stat."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Chakraborty, A., De, R., Chatterjee, A., Schwenker, F., and Sarkar, R. (2019). Filter Method Ensemble with Neural Networks. Lecture Notes in Computer Science, Proceedings of the International Conference on Artificial Neural Networks, Munich, Germany, 17\u201319 September 2019, Springer.","DOI":"10.1007\/978-3-030-30484-3_59"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1007\/s12530-016-9158-4","article-title":"Adaptive confidence learning for the personalization of pain intensity estimation systems","volume":"8","author":"Amirian","year":"2017","journal-title":"Evol. Syst."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Dey, S., Bhattacharya, R., Schwenker, F., and Sarkar, R. (2021). Median Filter Aided CNN Based Image Denoising: An Ensemble Aprroach. Algorithms, 14.","DOI":"10.3390\/a14040109"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Bellmann, P., Thiam, P., and Schwenker, F. (2018). Multi-classifier-systems: Architectures, algorithms and applications. Computational Intelligence for Pattern Recognition, Springer.","DOI":"10.1007\/978-3-319-89629-8_4"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41598-021-93658-y","article-title":"Fuzzy rank-based fusion of CNN models using Gompertz function for screening COVID-19 CT-scans","volume":"11","author":"Kundu","year":"2021","journal-title":"Sci. Rep."},{"key":"ref_36","unstructured":"Angelov, P., and Almeida Soares, E. (2020). SARS-CoV-2 CT-scan dataset: A large dataset of real patients CT scans for SARS-CoV-2 identification. MedRxiv."},{"key":"ref_37","unstructured":"Soares, E., and Angelov, P. (2020). A large dataset of real patients CT scans for COVID-19 identification. Harv. Dataverse, 1."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Sedghamiz, H., and Santonocito, D. (2015, January 19\u201321). Unsupervised detection and classification of motor unit action potentials in intramuscular electromyography signals. Proceedings of the 2015 E-Health and Bioengineering Conference (EHB), Iasi, Romania.","DOI":"10.1109\/EHB.2015.7391510"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1016\/0165-0114(89)90194-2","article-title":"An interpretation of fuzzy measures and the Choquet integral as an integral with respect to a fuzzy measure","volume":"29","author":"Murofushi","year":"1989","journal-title":"Fuzzy Sets Syst."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1016\/j.neucom.2018.03.002","article-title":"Aggregation of neural classifiers using Choquet integral with respect to a fuzzy measure","volume":"292","author":"Pacheco","year":"2018","journal-title":"Neurocomputing"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"733","DOI":"10.1109\/21.57289","article-title":"Information fusion in computer vision using the fuzzy integral","volume":"20","author":"Tahani","year":"1990","journal-title":"IEEE Trans. Syst. Man Cybern."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Narukawa, Y., and Murofushi, T. (2004). Decision modelling using the Choquet integral. Lecture Notes in Computer Science, Proceedings of the International Conference on Modeling Decisions for Artificial Intelligence, Barcelona, Spain, 1\u20133 August 2004, Springer.","DOI":"10.1007\/978-3-540-27774-3_18"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Mishra, M., and Srivastava, M. (2014, January 1\u20132). A view of artificial neural network. Proceedings of the 2014 International Conference on Advances in Engineering & Technology Research (ICAETR-2014), Unnao, India.","DOI":"10.1109\/ICAETR.2014.7012785"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1109\/45.329294","article-title":"Feed-forward neural networks","volume":"13","author":"Bebis","year":"1994","journal-title":"IEEE Potentials"},{"key":"ref_45","unstructured":"Wilson, E., and Tufts, D.W. (1994, January 6\u20138). Multilayer perceptron design algorithm. Proceedings of the IEEE Workshop on Neural Networks for Signal Processing, Ermioni, Greece."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s40537-020-00316-7","article-title":"Multi-layered deep learning perceptron approach for health risk prediction","volume":"7","author":"Bikku","year":"2020","journal-title":"J. Big Data"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"136","DOI":"10.1016\/j.bbe.2017.11.003","article-title":"Application of intrinsic band function technique for automated detection of sleep apnea using HRV and EDR signals","volume":"38","author":"Tripathy","year":"2018","journal-title":"Biocybern. Biomed. Eng."},{"key":"ref_48","unstructured":"Feng, K., and Liu, G. (2019, January 20\u201322). Obstructive sleep apnea detection based on unsupervised feature learning and hidden markov model. Proceedings of the BIBE 2019; The Third International Conference on Biological Information and Biomedical Engineering, Hangzhou, China."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/16\/5425\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:44:01Z","timestamp":1760165041000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/16\/5425"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,11]]},"references-count":48,"journal-issue":{"issue":"16","published-online":{"date-parts":[[2021,8]]}},"alternative-id":["s21165425"],"URL":"https:\/\/doi.org\/10.3390\/s21165425","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,8,11]]}}}