{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,20]],"date-time":"2026-10-20T01:01:52Z","timestamp":1792458112842,"version":"4.5.0"},"reference-count":33,"publisher":"IOP Publishing","issue":"5","license":[{"start":{"date-parts":[[2026,5,7]],"date-time":"2026-05-07T00:00:00Z","timestamp":1778112000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/publishingsupport.iopscience.iop.org\/iop-standard\/v1"},{"start":{"date-parts":[[2026,5,7]],"date-time":"2026-05-07T00:00:00Z","timestamp":1778112000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Physiol. Meas."],"published-print":{"date-parts":[[2026,5,29]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    <jats:italic>Objective.<\/jats:italic>\n                    Accurate physiological assessment of cardiac function from heart sounds remains challenging due to background noise, variable heart rates, and the need for reliable cardiac-cycle segmentation. This study aimed to develop a fully E2E deep learning framework that extracts diagnostic information directly from raw heart sound recordings for cardiac abnormality detection and classification.\n                    <jats:italic>Approach.<\/jats:italic>\n                    We propose HS-MMNet, an E2E multi-modal deep learning framework designed for physiological heart sound analysis. Recordings are preprocessed (normalization and 25\u2013400\u2009Hz bandpass filtering) and divided into fixed-length 2.5\u2009s segments. A Convolution Head with multi-atrous spatial pyramid and channel-spatial attention extracts fine-grained local temporal patterns from the filtered 1-D waveform. A Transformer Head captures long-range spectro-temporal dependencies from Log-Mel spectrograms. These hypotheses are iteratively fused by a novel multi-hypothesis cross-attention module with cyclic query-key-value assignment and a hypothesis-mixing MLP, enabling rich cross-site interaction and effective suppression of noise and non-informative regions. Recording-level classification is obtained via a fully connected layer.\n                    <jats:italic>Main results.<\/jats:italic>\n                    On the PhysioNet\/CinC Challenge 2016 dataset, HS-MMNet achieved 94.80% accuracy, 92.10% sensitivity, 96.85% specificity, 87.50% precision, and 89.74%\n                    <jats:italic>F<\/jats:italic>\n                    1-score, outperforming all previously reported methods. On the balanced five-class Yaseen dataset (normal, aortic stenosis, mitral regurgitation, mitral stenosis, mitral valve prolapse), it attained 99.60% macro-averaged precision, recall, and\n                    <jats:italic>F<\/jats:italic>\n                    1-score with only four misclassifications in 1000 recordings, establishing new state-of-the-art (SOTA) benchmarks.\n                    <jats:italic>Significance.<\/jats:italic>\n                    HS-MMNet represents an advance in automated physiological measurement from heart sounds. By eliminating cardiac cycle detection and multi-channel requirements while achieving SOTA diagnostic performance, it provides a practical, scalable solution for accurate cardiovascular screening with primary-care and low-resource settings.\n                  <\/jats:p>","DOI":"10.1088\/1361-6579\/ae6415","type":"journal-article","created":{"date-parts":[[2026,4,23]],"date-time":"2026-04-23T22:52:58Z","timestamp":1776984778000},"page":"055004","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["A multimodal fusion network for heart sound abnormality detection and classification"],"prefix":"10.1088","volume":"47","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8253-4046","authenticated-orcid":true,"given":"Hong","family":"Duc Nguyen","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1042-5032","authenticated-orcid":true,"given":"Duc","family":"Tri Phan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"266","published-online":{"date-parts":[[2026,5,7]]},"reference":[{"key":"pmeaae6415bib1","first-page":"pp 1","type":"conference-proceedings","article-title":"Hybrid deep neural network combining CNN, BiLSTM and GRU for heart sound classification","author":"Ait Ichou","year":"2025"},{"key":"pmeaae6415bib2","doi-asserted-by":"publisher","first-page":"224852","DOI":"10.1109\/ACCESS.2020.3043290","type":"journal-article","article-title":"A framework classification of heart sound signals in PhysioNet challenge 2016 using high order statistics and adaptive neuro-fuzzy inference system","volume":"8","author":"Al-Naami","year":"2020","journal-title":"IEEE Access"},{"key":"pmeaae6415bib3","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2021.105940","type":"journal-article","article-title":"Convolutional and recurrent neural networks for the detection of valvular heart diseases in phonocardiogram recordings","volume":"200","author":"Alkhodari","year":"2021","journal-title":"Comput. Methods Programs Biomed."},{"key":"pmeaae6415bib4","doi-asserted-by":"publisher","first-page":"66","DOI":"10.1007\/s13721-020-00272-5","type":"journal-article","article-title":"Classification of heart sound short records using bispectrum analysis approach images and deep learning","volume":"9","author":"Alqudah","year":"2020","journal-title":"Netw. Modeling Anal. Health Inform. Bioinf."},{"key":"pmeaae6415bib5","doi-asserted-by":"publisher","first-page":"7347","DOI":"10.1038\/s41598-020-64405-6","type":"journal-article","article-title":"The diagnostic accuracy of lung auscultation in adult patients with acute pulmonary pathologies: a meta-analysis","volume":"10","author":"Arts","year":"2020","journal-title":"Sci. Rep."},{"key":"pmeaae6415bib6","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2020.105750","type":"journal-article","article-title":"Automatic diagnosis of multiple cardiac diseases from pcg signals using convolutional neural network","volume":"197","author":"Baghel","year":"2020","journal-title":"Comput. Methods Programs Biomed."},{"key":"pmeaae6415bib7","doi-asserted-by":"publisher","DOI":"10.1016\/j.bspc.2020.102019","type":"journal-article","article-title":"Analysis of heart sound anomalies using ensemble learning","volume":"62","author":"Baydoun","year":"2020","journal-title":"Biomed. Signal Process. Control"},{"key":"pmeaae6415bib8","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TIM.2023.3246513","type":"journal-article","article-title":"Le-LWTNet: a learnable lifting wavelet convolutional neural network for heart sound abnormality detection","volume":"72","author":"Fan","year":"2023","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"pmeaae6415bib9","doi-asserted-by":"publisher","DOI":"10.1016\/j.cpcardiol.2023.101622","type":"journal-article","article-title":"Global cardiovascular diseases death rate prediction","volume":"48","author":"Gaidai","year":"2023","journal-title":"Curr. Problems Cardiol."},{"key":"pmeaae6415bib10","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2020.103632","type":"journal-article","article-title":"Automated detection of heart valve diseases using chirplet transform and multiclass composite classifier with pcg signals","volume":"118","author":"Ghosh","year":"2020","journal-title":"Comput. Biol. Med."},{"key":"pmeaae6415bib11","first-page":"pp 817","type":"conference-proceedings","article-title":"Automatic heart sound recording classification using a nested set of ensemble algorithms","author":"Homsi","year":"2016"},{"key":"pmeaae6415bib12","doi-asserted-by":"publisher","first-page":"1645","DOI":"10.1088\/1361-6579\/aa6a3d","type":"journal-article","article-title":"Dropconnected neural networks trained on time-frequency and inter-beat features for classifying heart sounds","volume":"38","author":"Kay","year":"2017","journal-title":"Physiol. Meas."},{"key":"pmeaae6415bib13","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s42235-025-00714-8","type":"journal-article","article-title":"Detection of abnormal cardiac rhythms using feature fusion technique with heart sound spectrograms","volume":"22","author":"Khan","year":"2025","journal-title":"J. Bionic Eng."},{"key":"pmeaae6415bib14","doi-asserted-by":"publisher","DOI":"10.1016\/j.bspc.2021.102893","type":"journal-article","article-title":"Heart sound classification based on log Mel-frequency spectral coefficients features and convolutional neural networks","volume":"69","author":"Kui","year":"2021","journal-title":"Biomed. Signal Process. Control"},{"key":"pmeaae6415bib15","doi-asserted-by":"publisher","first-page":"3956","DOI":"10.3390\/app10113956","type":"journal-article","article-title":"Classification of heart sounds using convolutional neural network","volume":"10","author":"Li","year":"2020","journal-title":"Appl. Sci."},{"key":"pmeaae6415bib16","doi-asserted-by":"publisher","first-page":"179339","DOI":"10.1109\/ACCESS.2019.2959081","type":"journal-article","article-title":"Heart sound signal classification algorithm: a combination of wavelet scattering transform and twin support vector machine","volume":"7","author":"Li","year":"2019","journal-title":"IEEE Access"},{"key":"pmeaae6415bib17","doi-asserted-by":"publisher","first-page":"2181","DOI":"10.1088\/0967-3334\/37\/12\/2181","type":"journal-article","article-title":"An open access database for the evaluation of heart sound algorithms","volume":"37","author":"Liu","year":"2016","journal-title":"Physiol. Meas."},{"key":"pmeaae6415bib18","doi-asserted-by":"publisher","first-page":"642","DOI":"10.3390\/e23060642","type":"journal-article","article-title":"Detection of coronary artery disease using multi-domain feature fusion of multi-channel heart sound signals","volume":"23","author":"Liu","year":"2021","journal-title":"Entropy"},{"key":"pmeaae6415bib19","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2023.107906","type":"journal-article","article-title":"An edge-device-compatible algorithm for valvular heart diseases screening using phonocardiogram signals with a lightweight convolutional neural network and self-supervised learning","volume":"243","author":"Ma","year":"2024","journal-title":"Comput. Methods Programs Biomed."},{"key":"pmeaae6415bib20","doi-asserted-by":"publisher","first-page":"576","DOI":"10.1002\/mp.13940","type":"journal-article","article-title":"Cardiac substructure segmentation with deep learning for improved cardiac sparing","volume":"47","author":"Morris","year":"2020","journal-title":"Med. Phys."},{"key":"pmeaae6415bib21","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2020.105604","type":"journal-article","article-title":"Classification of heart sound signals using a novel deep WaveNet model","volume":"196","author":"Oh","year":"2020","journal-title":"Comput. Methods Programs Biomed."},{"key":"pmeaae6415bib22","first-page":"pp 1989","type":"conference-proceedings","article-title":"The robustness of random forest and support vector machine algorithms to a faulty heart sound segmentation","author":"Oliveira","year":"2022"},{"key":"pmeaae6415bib23","doi-asserted-by":"publisher","first-page":"1515","DOI":"10.1016\/j.pcl.2004.08.004","type":"journal-article","article-title":"The physiology of cardiac auscultation","volume":"51","author":"Pelech","year":"2004","journal-title":"Pedia. Clin."},{"key":"pmeaae6415bib24","first-page":"pp 621","type":"conference-proceedings","article-title":"Ensemble of feature-based and deep learning-based classifiers for detection of abnormal heart sounds","author":"Potes","year":"2016"},{"key":"pmeaae6415bib25","doi-asserted-by":"publisher","first-page":"4819","DOI":"10.3390\/s19214819","type":"journal-article","article-title":"Heartbeat sound signal classification using deep learning","volume":"19","author":"Raza","year":"2019","journal-title":"Sensors"},{"key":"pmeaae6415bib26","first-page":"pp 1","type":"conference-proceedings","article-title":"Mathematical morphology-based envelope detection for cardiac disease classification from phonocardiogram signals","author":"Rebiai","year":"2024"},{"key":"pmeaae6415bib27","first-page":"pp 565","type":"conference-proceedings","article-title":"Heart sound classification using deep structured features","author":"Tschannen","year":"2016"},{"key":"pmeaae6415bib28","first-page":"pp 215","type":"book","article-title":"Accurate classification of heart sounds for disease diagnosis by using spectral analysis and deep learning methods","author":"Upretee","year":"2021"},{"key":"pmeaae6415bib29","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2025.127238","type":"journal-article","article-title":"WCFormer: an interpretable deep learning framework for heart sound signal analysis and automated diagnosis of cardiovascular diseases","volume":"276","author":"Wang","year":"2025","journal-title":"Expert Syst. Appl."},{"key":"pmeaae6415bib30","doi-asserted-by":"publisher","first-page":"650","DOI":"10.1109\/TBME.2020.3010241","type":"journal-article","article-title":"Temporal-framing adaptive network for heart sound segmentation without prior knowledge of state duration","volume":"68","author":"Wang","year":"2020","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"pmeaae6415bib31","doi-asserted-by":"publisher","first-page":"153","DOI":"10.1016\/j.neucom.2018.09.101","type":"journal-article","article-title":"Heart sounds classification using a novel 1-d convolutional neural network with extremely low parameter consumption","volume":"392","author":"Xiao","year":"2020","journal-title":"Neurocomputing"},{"key":"pmeaae6415bib32","doi-asserted-by":"publisher","first-page":"4752","DOI":"10.1007\/s00034-025-03000-4","type":"journal-article","article-title":"Segmentation of heart sound signals using improved Hilbert transform and wavelet packet transform","volume":"44","author":"Xiao","year":"2025","journal-title":"Circuits Syst. Signal Process."},{"key":"pmeaae6415bib33","doi-asserted-by":"publisher","first-page":"2344","DOI":"10.3390\/app8122344","type":"journal-article","article-title":"Classification of heart sound signal using multiple features","volume":"8","author":"Yaseen","year":"2018","journal-title":"Appl. Sci."}],"container-title":["Physiological Measurement"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6579\/ae6415","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6579\/ae6415\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6579\/ae6415","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6579\/ae6415\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6579\/ae6415\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6579\/ae6415\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6579\/ae6415\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6579\/ae6415\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,7]],"date-time":"2026-05-07T06:49:39Z","timestamp":1778136579000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6579\/ae6415"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,7]]},"references-count":33,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2026,5,7]]},"published-print":{"date-parts":[[2026,5,29]]}},"URL":"https:\/\/doi.org\/10.1088\/1361-6579\/ae6415","relation":{},"ISSN":["0967-3334","1361-6579"],"issn-type":[{"value":"0967-3334","type":"print"},{"value":"1361-6579","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,7]]},"assertion":[{"value":"A multimodal fusion network for heart sound abnormality detection and classification","name":"article_title","label":"Article Title"},{"value":"Physiological Measurement","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2026 Institute of Physics and Engineering in Medicine. All rights, including for text and data mining, AI training, and similar technologies, are reserved.","name":"copyright_information","label":"Copyright Information"},{"value":"2025-12-10","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2026-04-23","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2026-05-07","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}