{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,13]],"date-time":"2026-01-13T03:22:18Z","timestamp":1768274538119,"version":"3.49.0"},"reference-count":35,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,9,12]],"date-time":"2025-09-12T00:00:00Z","timestamp":1757635200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,9,12]],"date-time":"2025-09-12T00:00:00Z","timestamp":1757635200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Italian Ministry of Research","award":["2022A49KR3"],"award-info":[{"award-number":["2022A49KR3"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Med Syst"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Early detection of cardiovascular diseases (CVDs) is crucial for improving patient outcomes and alleviating healthcare burdens. Electrocardiograms (ECGs) and phonocardiograms (PCGs) offer low-cost, non-invasive, and easily integrable solutions for preventive care settings. In this work, we propose a novel bimodal deep learning model that combines ECG and PCG signals to enhance the early detection of CVDs. To address the challenge of limited bimodal data, we fine-tuned a Convolutional Neural Network (CNN) pre-trained on large-scale audio recordings, leveraging all publicly available unimodal PCG datasets. This PCG branch was then integrated with a 1D-CNN ECG branch via late fusion. Evaluated on an augmented version of MITHSDB, currently the only publicly available bimodal dataset, our approach achieved an AUROC of 96.4%, significantly outperforming ECG-only and PCG-only models by approximately 3%pts and 11%pts, respectively. To interpret the model\u2019s decisions, we applied three explainability techniques, quantifying the relative contributions of the electrical and acoustic features. Furthermore, by projecting the learned embeddings into two dimensions using UMAP, we revealed clear separation between normal and pathological samples. Our results conclusively demonstrate that combining ECG and PCG modalities yields substantial performance gains, with explainability and visualization providing critical insights into model behavior. These findings underscore the importance of multimodal approaches for CVDs diagnosis and prevention, and strongly motivate the collection of larger, more diverse bimodal datasets for future research.<\/jats:p>","DOI":"10.1007\/s10916-025-02245-5","type":"journal-article","created":{"date-parts":[[2025,9,12]],"date-time":"2025-09-12T06:36:08Z","timestamp":1757658968000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Bimodal ECG and PCG Cardiovascular Disease Detection: Exploring the Potential and Modality Contribution"],"prefix":"10.1007","volume":"49","author":[{"given":"Alessia","family":"Calzoni","sequence":"first","affiliation":[]},{"given":"Mattia","family":"Savardi","sequence":"additional","affiliation":[]},{"given":"Marco","family":"Silvestri","sequence":"additional","affiliation":[]},{"given":"Sergio","family":"Benini","sequence":"additional","affiliation":[]},{"given":"Alberto","family":"Signoroni","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2025,9,12]]},"reference":[{"key":"2245_CR1","unstructured":"World Health Organization, Cardiovascular diseases. https:\/\/www.who.int\/health-topics\/cardiovascular-diseases. Accessed 20 Aug 2024, 2024."},{"key":"2245_CR2","doi-asserted-by":"publisher","unstructured":"Joshi, R. C., Khan, J. S., Pathak, V. K., and Dutta, M. K., AI-CardioCare: Artificial intelligence based device for cardiac health monitoring. IEEE Trans. Hum.-Mach. Syst. 52(6):1292\u20131302, 2022. https:\/\/doi.org\/10.1109\/THMS.2022.3211460","DOI":"10.1109\/THMS.2022.3211460"},{"key":"2245_CR3","doi-asserted-by":"publisher","unstructured":"Li, P., Hu, Y., and Liu, Z. -P., Prediction of cardiovascular diseases by integrating multi-modal features with machine learning methods. Biomed. Signal Process. Control 66:102474, 2021. https:\/\/doi.org\/10.1016\/j.bspc.2021.102474","DOI":"10.1016\/j.bspc.2021.102474"},{"key":"2245_CR4","doi-asserted-by":"publisher","unstructured":"Liu, C., Springer, D., Li, Q., Moody, B., Juan, R.A., Chorro, F. J., Castells, F., Roig, J. M., Silva, I., Johnson, A. E. W., Syed, Z., Schmidt, S. E., Papadaniil, C. D., Hadjileontiadis, L., Naseri, H., Moukadem, A., Dieterlen, A., Brandt, C., Tang, H., Samieinasab, M., Samieinasab, M. R., Sameni, R., Mark, R. G., and Clifford, G. D., An open access database for the evaluation of heart sound algorithms. Physiol. Meas. 37(12):2181, 2016. https:\/\/doi.org\/10.1088\/0967-3334\/37\/12\/2181","DOI":"10.1088\/0967-3334\/37\/12\/2181"},{"key":"2245_CR5","doi-asserted-by":"publisher","unstructured":"Baloglu, U. B., Talo, M., Yildirim, O., Tan, R. S., and Acharya, U. R., Classification of myocardial infarction with multi-lead ECG signals and deep CNN. Pattern Recognit. Lett. 122:23\u201330, 2019. https:\/\/doi.org\/10.1016\/j.patrec.2019.02.016","DOI":"10.1016\/j.patrec.2019.02.016"},{"key":"2245_CR6","doi-asserted-by":"publisher","unstructured":"Ribeiro, A. H., Ribeiro, M. H., Paix\u00e3o, G. M. M., Oliveira, D. M., Gomes, P. R., Canazart, J. A., Ferreira, M. P. S., Andersson, C. R., Macfarlane, P. W., Jr., W. M., Sch\u00f6n, T. B., and Ribeiro, A. L. P.: Automatic diagnosis of the 12-lead ECG using a deep neural network. Nat. Commun. 11(1):1760, 2020. https:\/\/doi.org\/10.1038\/s41467-020-15432-4","DOI":"10.1038\/s41467-020-15432-4"},{"key":"2245_CR7","doi-asserted-by":"publisher","unstructured":"Haimovich, J. S., Diamant, N., Khurshid, S., Di Achille, P., Reeder, C., Friedman, S., Singh, P., Spurlock, W., Ellinor, P.T., Philippakis, A., Batra, P., Ho, J. E., and Lubitz, S. A., Artificial intelligence\u2013enabled classification of hypertrophic heart diseases using electrocardiograms. Cardiovasc. Digit. Health J. 4(2):48\u201359, 2023. https:\/\/doi.org\/10.1016\/j.cvdhj.2023.03.001","DOI":"10.1016\/j.cvdhj.2023.03.001"},{"key":"2245_CR8","doi-asserted-by":"publisher","unstructured":"De\u00a0Marco, F., Ferrucci, F., Risi, M., and Tortora, G., Classification of QRS complexes to detect Premature Ventricular Contraction using machine learning techniques. PLoS One 17(8):0268555, 2022. https:\/\/doi.org\/10.1371\/journal.pone.0268555","DOI":"10.1371\/journal.pone.0268555"},{"key":"2245_CR9","doi-asserted-by":"publisher","unstructured":"Khan, F.A., Abid, A., Khan, M.S.: Automatic heart sound classification from segmented\/unsegmented phonocardiogram signals using time and frequency features. Physiol. Meas. 41(5):055006, 2020. https:\/\/doi.org\/10.1088\/1361-6579\/ab8770","DOI":"10.1088\/1361-6579\/ab8770"},{"key":"2245_CR10","doi-asserted-by":"publisher","unstructured":"Al-Issa, Y., and Alqudah, A. M., A lightweight hybrid deep learning system for cardiac valvular disease classification. Sci. Rep. 12(1):14297, 2022. https:\/\/doi.org\/10.1038\/s41598-022-18293-7","DOI":"10.1038\/s41598-022-18293-7"},{"key":"2245_CR11","doi-asserted-by":"publisher","unstructured":"Nguyen, M. T., Lin, W. W., and Huang, J. H., Heart sound classification using deep learning techniques based on log-mel spectrogram. Circ. Syst. Signal Process. 42(1):344\u2013360, 2023. https:\/\/doi.org\/10.1007\/s00034-022-02124-1","DOI":"10.1007\/s00034-022-02124-1"},{"key":"2245_CR12","doi-asserted-by":"publisher","unstructured":"Li, H., Wang, X., Liu, C., Wang, Y., Li, P., Tang, H., Yao, L., and Zhang, H., Dual-input neural network integrating feature extraction and deep learning for coronary artery disease detection using electrocardiogram and phonocardiogram. IEEE Access 7:146457\u2013146469, 2019. https:\/\/doi.org\/10.1109\/ACCESS.2019.2943197","DOI":"10.1109\/ACCESS.2019.2943197"},{"key":"2245_CR13","doi-asserted-by":"publisher","unstructured":"Chakir, F., Jilbab, A., Nacir, C., and Hammouch, A., Recognition of cardiac abnormalities from synchronized ECG and PCG signals. Phys. Eng. Sci. Med. 43(2):673\u2013677, 2020. https:\/\/doi.org\/10.1007\/s13246-020-00875-2","DOI":"10.1007\/s13246-020-00875-2"},{"key":"2245_CR14","doi-asserted-by":"publisher","unstructured":"Ajitkumar\u00a0Singh, S., Ashinikumar\u00a0Singh, S., Dinita\u00a0Devi, N., and Majumder, S., Heart Abnormality classification using PCG and ECG recordings. Comput. Sist. 25(2):381\u2013391, 2021. https:\/\/doi.org\/10.13053\/cys-25-2-3447","DOI":"10.13053\/cys-25-2-3447"},{"key":"2245_CR15","doi-asserted-by":"publisher","unstructured":"Li, J., Ke, L., Du, Q., Chen, X., and Ding, X., Multi-modal cardiac function signals classification algorithm based on improved D-S evidence theory. Biomed. Signal Process. Control 71:103078, 2022. https:\/\/doi.org\/10.1016\/j.bspc.2021.103078","DOI":"10.1016\/j.bspc.2021.103078"},{"key":"2245_CR16","doi-asserted-by":"publisher","unstructured":"Li, H., Wang, X., Liu, C., Li, P., and Jiao, Y., Integrating multi-domain deep features of electrocardiogram and phonocardiogram for coronary artery disease detection. Comput. Biol. Med. 138:104914, 2021. https:\/\/doi.org\/10.1016\/j.compbiomed.2021.104914","DOI":"10.1016\/j.compbiomed.2021.104914"},{"key":"2245_CR17","doi-asserted-by":"publisher","unstructured":"Li, J., Ke, L., Du, Q., Ding, X., and Chen, X., Research on the classification of ECG and PCG signals based on BiLSTM-GoogLeNet-DS. Appl. Sci. 12(22), 2022. https:\/\/doi.org\/10.3390\/app122211762","DOI":"10.3390\/app122211762"},{"key":"2245_CR18","doi-asserted-by":"publisher","unstructured":"Morshed, M., and Fattah, S. A., A deep neural network for heart valve defect classification from synchronously recorded ECG and PCG. IEEE Sens. Lett. 7(9):1\u20134, 2023. https:\/\/doi.org\/10.1109\/LSENS.2023.3307053","DOI":"10.1109\/LSENS.2023.3307053"},{"key":"2245_CR19","doi-asserted-by":"publisher","unstructured":"Jyothi, P., and Pradeepini, G., Heart disease detection system based on ECG and PCG signals with the aid of GKVDLNN classifier. Multimed. Tools Appl. 83(10):30587\u201330612, 2024. https:\/\/doi.org\/10.1007\/s11042-023-16562-9","DOI":"10.1007\/s11042-023-16562-9"},{"key":"2245_CR20","doi-asserted-by":"publisher","unstructured":"Hettiarachchi, R., Haputhanthri, U., Herath, K., Kariyawasam, H., Munasinghe, S., Wickramasinghe, K., Samarasinghe, D., De\u00a0Silva, A., and Edussooriya, C. U. S., A novel transfer learning-based approach for screening pre-existing heart diseases using synchronized ECG signals and heart sounds. In: 2021 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1\u20135, 2021. https:\/\/doi.org\/10.1109\/ISCAS51556.2021.9401093","DOI":"10.1109\/ISCAS51556.2021.9401093"},{"key":"2245_CR21","unstructured":"Vieira, H. M. C., Multimodal deep learning for heart sound and electrocardiogram classification. Dissertation. Master\u2019s thesis, University of Porto: Porto, Portugal, 2023."},{"key":"2245_CR22","doi-asserted-by":"publisher","unstructured":"Mathew, G., Barbosa, D., Prince, J., and Venkatraman, S., Foundation models for cardiovascular disease detection via biosignals from digital stethoscopes. Npj Cardiovasc. Health 1(1):25, 2024. https:\/\/doi.org\/10.1038\/s44325-024-00027-5","DOI":"10.1038\/s44325-024-00027-5"},{"key":"2245_CR23","unstructured":"Calzoni, A., Savardi, M., and Signoroni, A., Bimodal ECG-PCG cardiovascular disease detection: A close look at transfer learning and data collection issues. In: CEUR workshop proceedings - 3rd AIxIA workshop on artificial intelligence for healthcare, HC@AIxIA 2024, vol. 3880, pp. 93\u2013107, 2024."},{"key":"2245_CR24","doi-asserted-by":"publisher","unstructured":"Goldberger, A. L., Amaral, L. A. N., Glass, L., Hausdorff, J. M., Ivanov, P. C., Mark, R. G., Mietus, J. E., Moody, G. B., Peng, C. -K., and Stanley, H. E., PhysioBank, physiotoolkit, and physionet. Circulation 101(23):215\u2013220, 2000. https:\/\/doi.org\/10.1161\/01.CIR.101.23.e215","DOI":"10.1161\/01.CIR.101.23.e215"},{"key":"2245_CR25","doi-asserted-by":"publisher","unstructured":"Yaseen, Son, G.-Y., and Kwon, S., Classification of heart sound signal using multiple features. Appl. Sci. 8(12), 2018. https:\/\/doi.org\/10.3390\/app8122344","DOI":"10.3390\/app8122344"},{"key":"2245_CR26","doi-asserted-by":"publisher","unstructured":"Oliveira, J., Renna, F., Costa, P., Nogueira, M., Oliveira, A. C., Elola, A., Ferreira, C., Jorge, A., Rad, A. B., Reyna, M., Sameni, R., Clifford, G., and Coimbra, M., The CirCor DigiScope phonocardiogram dataset (version 1.0.3). PhysioNet, 2022. https:\/\/doi.org\/10.13026\/tshs-mw03","DOI":"10.13026\/tshs-mw03"},{"key":"2245_CR27","doi-asserted-by":"publisher","unstructured":"Oliveira, J., Renna, F., Costa, P. D., Nogueira, M., Oliveira, C., Ferreira, C., Jorge, A., Mattos, S., Hatem, T., Tavares, T., Elola, A., Rad, A. B., Sameni, R., Clifford, G. D., and Coimbra, M. T., The CirCor DigiScope dataset: From murmur detection to murmur classification. IEEE J. Biomed. Health Inform. 26(6):2524\u20132535, 2022. https:\/\/doi.org\/10.1109\/JBHI.2021.3137048","DOI":"10.1109\/JBHI.2021.3137048"},{"key":"2245_CR28","doi-asserted-by":"publisher","unstructured":"Li, P., Prediction of cardiovascular diseases by integrating multi-modal features with machine learning methods. Zenodo (2020). https:\/\/doi.org\/10.5281\/zenodo.4263528","DOI":"10.5281\/zenodo.4263528"},{"key":"2245_CR29","doi-asserted-by":"publisher","unstructured":"Koike, T., Qian, K., Kong, Q., Plumbley, M. D., Schuller, B. W., and Yamamoto, Y., Audio for audio is better? An investigation on transfer learning models for heart sound classification. In: 2020 42nd Annual international conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pp. 74\u201377, 2020. https:\/\/doi.org\/10.1109\/EMBC44109.2020.9175450","DOI":"10.1109\/EMBC44109.2020.9175450"},{"key":"2245_CR30","doi-asserted-by":"publisher","unstructured":"Kong, Q., Cao, Y., Iqbal, T., Wang, Y., Wang, W., and Plumbley, M. D., PANNs: Large-scale pretrained audio neural networks for audio pattern recognition. IEEE\/ACM Trans. Audio Speech Lang. Process. 28:2880\u20132894, 2020. https:\/\/doi.org\/10.1109\/TASLP.2020.3030497","DOI":"10.1109\/TASLP.2020.3030497"},{"key":"2245_CR31","doi-asserted-by":"publisher","unstructured":"Gemmeke, J. F., Ellis, D. P. W., Freedman, D., Jansen, A., Lawrence, W., Moore, R. C., Plakal, M., and Ritter, M., Audio set: An ontology and human-labeled dataset for audio events. In: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 776\u2013780, 2017. https:\/\/doi.org\/10.1109\/ICASSP.2017.7952261","DOI":"10.1109\/ICASSP.2017.7952261"},{"key":"2245_CR32","doi-asserted-by":"publisher","unstructured":"Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D., Grad-CAM: Visual explanations from deep networks via gradient-based localization. Int. J. Comput. Vis. 128(2):336\u2013359, 2020. https:\/\/doi.org\/10.1007\/s11263-019-01228-7","DOI":"10.1007\/s11263-019-01228-7"},{"key":"2245_CR33","doi-asserted-by":"publisher","unstructured":"Li, S., Li, T., Sun, C., Yan, R., and Chen, X., Multilayer grad-CAM: An effective tool towards explainable deep neural networks for intelligent fault diagnosis. J. Manuf. Syst. 69:20\u201330, 2023. https:\/\/doi.org\/10.1016\/j.jmsy.2023.05.027","DOI":"10.1016\/j.jmsy.2023.05.027"},{"key":"2245_CR34","doi-asserted-by":"publisher","unstructured":"Ribeiro, M., Singh, S., and Guestrin, C., \"Why should i trust you?\": Explaining the predictions of any classifier. In: KDD \u201916: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135\u20131144, 2016. https:\/\/doi.org\/10.1145\/2939672.2939778","DOI":"10.1145\/2939672.2939778"},{"key":"2245_CR35","doi-asserted-by":"publisher","unstructured":"McInnes, L., Healy, J., Saul, N., and Gro\u00dfberger, L., UMAP: uniform manifold approximation and projection. J. Open Source Softw. 3(29):861, 2018. https:\/\/doi.org\/10.21105\/joss.00861","DOI":"10.21105\/joss.00861"}],"container-title":["Journal of Medical Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10916-025-02245-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10916-025-02245-5","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10916-025-02245-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,22]],"date-time":"2025-12-22T03:57:15Z","timestamp":1766375835000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10916-025-02245-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,12]]},"references-count":35,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["2245"],"URL":"https:\/\/doi.org\/10.1007\/s10916-025-02245-5","relation":{},"ISSN":["1573-689X"],"issn-type":[{"value":"1573-689X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,9,12]]},"assertion":[{"value":"29 May 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 August 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 September 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing Interests"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Clinical Trial Number"}},{"value":"No ethical approval and consent form is required in this study as it utilizes publicly available datasets.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics Approval and Consent to Participate"}}],"article-number":"113"}}