{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,13]],"date-time":"2026-07-13T10:18:50Z","timestamp":1783937930476,"version":"3.55.0"},"reference-count":61,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2022,2,16]],"date-time":"2022-02-16T00:00:00Z","timestamp":1644969600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Lung or heart sound classification is challenging due to the complex nature of audio data, its dynamic properties of time, and frequency domains. It is also very difficult to detect lung or heart conditions with small amounts of data or unbalanced and high noise in data. Furthermore, the quality of data is a considerable pitfall for improving the performance of deep learning. In this paper, we propose a novel feature-based fusion network called FDC-FS for classifying heart and lung sounds. The FDC-FS framework aims to effectively transfer learning from three different deep neural network models built from audio datasets. The innovation of the proposed transfer learning relies on the transformation from audio data to image vectors and from three specific models to one fused model that would be more suitable for deep learning. We used two publicly available datasets for this study, i.e., lung sound data from ICHBI 2017 challenge and heart challenge data. We applied data augmentation techniques, such as noise distortion, pitch shift, and time stretching, dealing with some data issues in these datasets. Importantly, we extracted three unique features from the audio samples, i.e., Spectrogram, MFCC, and Chromagram. Finally, we built a fusion of three optimal convolutional neural network models by feeding the image feature vectors transformed from audio features. We confirmed the superiority of the proposed fusion model compared to the state-of-the-art works. The highest accuracy we achieved with FDC-FS is 99.1% with Spectrogram-based lung sound classification while 97% for Spectrogram and Chromagram based heart sound classification.<\/jats:p>","DOI":"10.3390\/s22041521","type":"journal-article","created":{"date-parts":[[2022,2,16]],"date-time":"2022-02-16T21:36:24Z","timestamp":1645047384000},"page":"1521","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":91,"title":["Feature-Based Fusion Using CNN for Lung and Heart Sound Classification"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7129-4267","authenticated-orcid":false,"given":"Zeenat","family":"Tariq","sequence":"first","affiliation":[{"name":"Department of Computer Science and Electrical Engineering, University of Missouri-Kansas City, Kansas City, MO 64110, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sayed Khushal","family":"Shah","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Electrical Engineering, University of Missouri-Kansas City, Kansas City, MO 64110, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yugyung","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Electrical Engineering, University of Missouri-Kansas City, Kansas City, MO 64110, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,2,16]]},"reference":[{"key":"ref_1","unstructured":"WHO (2021, July 02). WHO\u2019s Global Health Estimates: The Top 10 Causes of Death. Available online: https:\/\/www.who.int\/news-room\/fact-sheets\/detail\/the-top-10-causes-of-death."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"100156","DOI":"10.1016\/j.ajpc.2021.100156","article-title":"US population at increased risk of severe illness from COVID-19","volume":"6","author":"Ajufo","year":"2021","journal-title":"Am. J. Prev. Cardiol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1377\/hlthaff.2019.01451","article-title":"National Health Care Spending In 2018: Growth Driven By Accelerations In Medicare And Private Insurance Spending: US health care spending increased 4.6 percent to reach $3.6 trillion in 2018, a faster growth rate than that of 4.2 percent in 2017 but the same rate as in 2016","volume":"39","author":"Hartman","year":"2020","journal-title":"Health Affairs"},{"key":"ref_4","unstructured":"Kahya, Y.P., Guler, E.C., and Sahin, S. (November, January 30). Respiratory disease diagnosis using lung sounds. Proceedings of the 19th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. \u2018Magnificent Milestones and Emerging Opportunities in Medical Engineering\u2019 (Cat. No. 97CH36136), Chicago, IL, USA."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"7347","DOI":"10.1038\/s41598-020-64405-6","article-title":"The diagnostic accuracy of lung auscultation in adult patients with acute pulmonary pathologies: A meta-analysis","volume":"10","author":"Arts","year":"2020","journal-title":"Sci. Rep."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1119","DOI":"10.1164\/ajrccm.159.4.9806083","article-title":"Pulmonary auscultatory skills during training in internal medicine and family practice","volume":"159","author":"Mangione","year":"1999","journal-title":"Am. J. Respir. Crit. Care Med."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"e20171154","DOI":"10.1542\/peds.2017-1154","article-title":"Pulse oximetry and auscultation for congenital heart disease detection","volume":"140","author":"Hu","year":"2017","journal-title":"Pediatrics"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Mishra, M., Singh, A., Dutta, M.K., Burget, R., and Masek, J. (2017, January 5\u20137). Classification of normal and abnormal heart sounds for automatic diagnosis. Proceedings of the 2017 40th International Conference on Telecommunications and Signal Processing (TSP), Barcelona, Spain.","DOI":"10.1109\/TSP.2017.8076089"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Brown, C., Chauhan, J., Grammenos, A., Han, J., Hasthanasombat, A., Spathis, D., Xia, T., Cicuta, P., and Mascolo, C. (2020, January 6\u201310). Exploring automatic diagnosis of COVID-19 from crowdsourced respiratory sound data. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Virtual Event.","DOI":"10.1145\/3394486.3412865"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Nogueira, D.M., Zarmehri, M.N., Ferreira, C.A., Jorge, A.M., and Antunes, L. (2019, January 3\u20136). Heart sounds classification using images from wavelet transformation. Proceedings of the EPIA Conference on Artificial Intelligence, Vila Real, Portugal.","DOI":"10.1007\/978-3-030-30241-2_27"},{"key":"ref_11","unstructured":"Cobos, M., Perez-Solano, J., and Berger, L. (2016). Acoustic-based technologies for ambient assisted living. Introd. Smart Ehealth Ecare Technol., 159\u2013180."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Dimitrievski, A., Zdravevski, E., Lameski, P., and Trajkovik, V. (2016, January 8\u201310). A survey of Ambient Assisted Living systems: Challenges and opportunities. Proceedings of the IEEE 12th International Conference on Intelligent Computer Communication and Processing (ICCP), Cluj-Napoca, Romania.","DOI":"10.1109\/ICCP.2016.7737121"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Doukas, C., and Maglogiannis, I. (2008, January 1). Advanced patient or elder fall detection based on movement and sound data. Proceedings of the 2008 Second International Conference on Pervasive Computing Technologies for Healthcare, Tampere, Finland.","DOI":"10.1109\/PCTHEALTH.2008.4571042"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"947.e11","DOI":"10.1016\/j.jvoice.2018.07.014","article-title":"A survey on machine learning approaches for automatic detection of voice disorders","volume":"33","author":"Hegde","year":"2019","journal-title":"J. Voice"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"8316","DOI":"10.1109\/ACCESS.2018.2889437","article-title":"Algorithms for automatic analysis and classification of heart sounds\u2014A systematic review","volume":"7","author":"Dwivedi","year":"2018","journal-title":"IEEE Access"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1561\/2000000039","article-title":"Deep learning: Methods and applications","volume":"7","author":"Deng","year":"2014","journal-title":"Found. Trends\u00ae Signal Process."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"27781","DOI":"10.1109\/ACCESS.2019.2901672","article-title":"EEG pathology detection based on deep learning","volume":"7","author":"Alhussein","year":"2019","journal-title":"IEEE Access"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Fattah, S.A., Rahman, N.M., Maksud, A., Foysal, S.I., Chowdhury, R.I., Chowdhury, S.S., and Shahanaz, C. (2017, January 19\u201322). Stetho-phone: Low-cost digital stethoscope for remote personalized healthcare. Proceedings of the 2017 IEEE Global Humanitarian Technology Conference (GHTC), San Jose, CA, USA.","DOI":"10.1109\/GHTC.2017.8239325"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1969","DOI":"10.1109\/TMM.2016.2594148","article-title":"Audiovisual spatial-audio analysis by means of sound localization and imaging: A multimedia healthcare framework in abdominal sound mapping","volume":"18","author":"Dimoulas","year":"2016","journal-title":"IEEE Trans. Multimed."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"351","DOI":"10.1007\/s11633-021-1293-0","article-title":"Deep audio-visual learning: A survey","volume":"18","author":"Zhu","year":"2021","journal-title":"Int. J. Autom. Comput."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s10916-019-1286-5","article-title":"Classifying heart sounds using images of motifs, mfcc and temporal features","volume":"43","author":"Nogueira","year":"2019","journal-title":"J. Med. Syst."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"516","DOI":"10.1016\/j.ins.2017.09.010","article-title":"A novel multi-modality image fusion method based on image decomposition and sparse representation","volume":"432","author":"Zhu","year":"2018","journal-title":"Inf. Sci."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"107819","DOI":"10.1016\/j.apacoust.2020.107819","article-title":"Ensemble of handcrafted and deep features for urban sound classification","volume":"175","author":"Luz","year":"2021","journal-title":"Appl. Acoust."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"841","DOI":"10.1109\/TCBB.2018.2806438","article-title":"A multimodal deep neural network for human breast cancer prognosis prediction by integrating multi-dimensional data","volume":"16","author":"Sun","year":"2018","journal-title":"IEEE\/ACM Trans. Comput. Biol. Bioinform."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Nanni, L., Maguolo, G., Brahnam, S., and Paci, M. (2020). An Ensemble of Convolutional Neural Networks for Audio Classification. arXiv.","DOI":"10.1186\/s13636-020-00175-3"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"743","DOI":"10.1038\/nmeth.4304","article-title":"Fused cerebral organoids model interactions between brain regions","volume":"14","author":"Bagley","year":"2017","journal-title":"Nat. Methods"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Rocha, B., Filos, D., Mendes, L., Vogiatzis, I., Perantoni, E., Kaimakamis, E., Natsiavas, P., Oliveira, A., J\u00e1come, C., and Marques, A. (2017, January 18\u201321). A respiratory sound database for the development of automated classification. Proceedings of the International Conference on Biomedical and Health Informatics, Thessaloniki, Greece.","DOI":"10.1007\/978-981-10-7419-6_6"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Miko\u0142ajczyk, A., and Grochowski, M. (2018, January 12). Data augmentation for improving deep learning in image classification problem. Proceedings of the 2018 International Interdisciplinary PhD Workshop (IIPhDW), Swinoujscie, Poland.","DOI":"10.1109\/IIPHDW.2018.8388338"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Nguyen, T., and Pernkopf, F. (2020, January 20\u201324). Lung Sound Classification Using Snapshot Ensemble of Convolutional Neural Networks. Proceedings of the 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Montreal, QC, Canada.","DOI":"10.1109\/EMBC44109.2020.9176076"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"240","DOI":"10.3934\/publichealth.2021019","article-title":"Automatic COVID-19 disease diagnosis using 1D convolutional neural network and augmentation with human respiratory sound based on parameters: Cough, breath, and voice","volume":"8","author":"Lella","year":"2021","journal-title":"AIMS Public Health"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Kochetov, K., and Filchenkov, A. (2020, January 27\u201329). Generative Adversarial Networks for Respiratory Sound Augmentation. Proceedings of the 2020 International Conference on Control, Robotics and Intelligent System, Xiamen, China.","DOI":"10.1145\/3437802.3437821"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1016\/j.artmed.2018.04.008","article-title":"Lung sounds classification using convolutional neural networks","volume":"88","author":"Bardou","year":"2018","journal-title":"Artif. Intell. Med."},{"key":"ref_33","first-page":"1385","article-title":"R.A.L.E Lung Sounds 3.1 Profesional Edition","volume":"50","author":"Ward","year":"2005","journal-title":"Respir. Care"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Dubey, R., and M Bodade, R. (2019, January 14\u201317). A Review of Classification Techniques Based on Neural Networks for Pulmonary Obstructive Diseases. Proceedings of the Recent Advances in Interdisciplinary Trends in Engineering & Applications (RAITEA), Indore, Inde.","DOI":"10.2139\/ssrn.3363485"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"32845","DOI":"10.1109\/ACCESS.2019.2903859","article-title":"Triple-classification of respiratory sounds using optimized s-transform and deep residual networks","volume":"7","author":"Chen","year":"2019","journal-title":"IEEE Access"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"105376","DOI":"10.1109\/ACCESS.2020.3000111","article-title":"Classification of Lung Sounds with CNN Model Using Parallel Pooling Structure","volume":"8","author":"Demir","year":"2020","journal-title":"IEEE Access"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s13755-019-0091-3","article-title":"Convolutional neural networks based efficient approach for classification of lung diseases","volume":"8","author":"Demir","year":"2020","journal-title":"Health Inf. Sci. Syst."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"2595","DOI":"10.1109\/JBHI.2020.3048006","article-title":"A lightweight cnn model for detecting respiratory diseases from lung auscultation sounds using emd-cwt-based hybrid scalogram","volume":"25","author":"Shuvo","year":"2020","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"103831","DOI":"10.1016\/j.compbiomed.2020.103831","article-title":"Multi-channel lung sound classification with convolutional recurrent neural networks","volume":"122","author":"Messner","year":"2020","journal-title":"Comput. Biol. Med."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.bbe.2020.11.003","article-title":"Automatic identification of respiratory diseases from stethoscopic lung sound signals using ensemble classifiers","volume":"41","author":"Fraiwan","year":"2021","journal-title":"Biocybern. Biomed. Eng."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Potes, C., Parvaneh, S., Rahman, A., and Conroy, B. (2016, January 11\u201314). Ensemble of feature-based and deep learning-based classifiers for detection of abnormal heart sounds. Proceedings of the 2016 Computing in Cardiology Conference (CinC), Vancouver, BC, Canada.","DOI":"10.22489\/CinC.2016.182-399"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Zhang, W., and Han, J. (2017, January 24\u201327). Towards heart sound classification without segmentation using convolutional neural network. Proceedings of the 2017 Computing in Cardiology (CinC), Rennes, France.","DOI":"10.22489\/CinC.2017.254-164"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"132","DOI":"10.1016\/j.compbiomed.2018.06.026","article-title":"A study of time-frequency features for CNN-based automatic heart sound classification for pathology detection","volume":"100","author":"Bozkurt","year":"2018","journal-title":"Comput. Biol. Med."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Clifford, G.D., Liu, C., Moody, B., Springer, D., Silva, I., Li, Q., and Mark, R.G. (2016, January 11\u201314). Classification of normal\/abnormal heart sound recordings: The PhysioNet\/Computing in Cardiology Challenge 2016. Proceedings of the 2016 Computing in cardiology conference (CinC), Vancouver, BC, Canada.","DOI":"10.22489\/CinC.2016.179-154"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"105604","DOI":"10.1016\/j.cmpb.2020.105604","article-title":"Classification of heart sound signals using a novel deep wavenet model","volume":"196","author":"Oh","year":"2020","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1016\/j.neunet.2020.06.015","article-title":"Heart sound classification based on improved MFCC features and convolutional recurrent neural networks","volume":"130","author":"Deng","year":"2020","journal-title":"Neural Netw."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Chen, W., Sun, Q., Chen, X., Xie, G., Wu, H., and Xu, C. (2021). Deep Learning Methods for Heart Sounds Classification: A Systematic Review. Entropy, 23.","DOI":"10.3390\/e23060667"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"190","DOI":"10.1016\/j.ins.2017.06.027","article-title":"Application of deep convolutional neural network for automated detection of myocardial infarction using ECG signals","volume":"415","author":"Acharya","year":"2017","journal-title":"Inf. Sci."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"278","DOI":"10.1016\/j.compbiomed.2018.06.002","article-title":"Automated diagnosis of arrhythmia using combination of CNN and LSTM techniques with variable length heart beats","volume":"102","author":"Oh","year":"2018","journal-title":"Comput. Biol. Med."},{"key":"ref_50","unstructured":"Rajpurkar, P., Hannun, A.Y., Haghpanahi, M., Bourn, C., and Ng, A.Y. (2017). Cardiologist-level arrhythmia detection with convolutional neural networks. arXiv."},{"key":"ref_51","unstructured":"Wyse, L. (2017). Audio spectrogram representations for processing with convolutional neural networks. arXiv."},{"key":"ref_52","unstructured":"McFee, B., Humphrey, E.J., and Bello, J.P. (2015). A Software Framework for Musical Data Augmentation, ISMIR."},{"key":"ref_53","unstructured":"Wei, S., Xu, K., Wang, D., Liao, F., Wang, H., and Kong, Q. (2018). Sample mixed-based data augmentation for domestic audio tagging. arXiv."},{"key":"ref_54","unstructured":"Cohen, L. (1995). Time-Frequency Analysis, Prentice Hall."},{"key":"ref_55","unstructured":"Semmlow, J.L., and Griffel, B. (2014). Biosignal and Medical Image Processing, CRC Press."},{"key":"ref_56","unstructured":"Molau, S., Pitz, M., Schluter, R., and Ney, H. (2001, January 7\u201311). Computing mel-frequency cepstral coefficients on the power spectrum. Proceedings of the 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing, Proceedings (Cat. No. 01CH37221), Salt Lake City, UT, USA."},{"key":"ref_57","unstructured":"Bentley, P., Nordehn, G., Coimbra, M., and Mannor, S. (2022, February 13). The PASCAL Classifying Heart Sounds Challenge 2011 (CHSC2011) Results. Available online: http:\/\/www.peterjbentley.com\/heartchallenge\/index.html."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1016\/j.asoc.2019.01.019","article-title":"Applying an ensemble convolutional neural network with Savitzky\u2013Golay filter to construct a phonocardiogram prediction model","volume":"78","author":"Wu","year":"2019","journal-title":"Appl. Soft Comput."},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1016\/j.neucom.2018.09.101","article-title":"Heart sounds classification using a novel 1-D convolutional neural network with extremely low parameter consumption","volume":"392","author":"Xiao","year":"2020","journal-title":"Neurocomputing"},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"108152","DOI":"10.1016\/j.apacoust.2021.108152","article-title":"Heart sounds classification using convolutional neural network with 1D-local binary pattern and 1D-local ternary pattern features","volume":"180","author":"Bilal","year":"2021","journal-title":"Appl. Acoust."},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Koike, T., Qian, K., Kong, Q., Plumbley, M.D., Schuller, B.W., and Yamamoto, Y. (2020, January 20\u201324). Audio for audio is better? an investigation on transfer learning models for heart sound classification. Proceedings of the 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Montreal, QC, Canada.","DOI":"10.1109\/EMBC44109.2020.9175450"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/4\/1521\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:20:38Z","timestamp":1760134838000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/4\/1521"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,2,16]]},"references-count":61,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2022,2]]}},"alternative-id":["s22041521"],"URL":"https:\/\/doi.org\/10.3390\/s22041521","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,2,16]]}}}