{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,29]],"date-time":"2026-08-29T05:19:16Z","timestamp":1787980756164,"version":"build-2784847793"},"reference-count":31,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2026,8,27]],"date-time":"2026-08-27T00:00:00Z","timestamp":1787788800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"publisher","award":["RGPIN-2024-05287"],"award-info":[{"award-number":["RGPIN-2024-05287"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]},{"award":["RGPIN-2024-05287"],"award-info":[{"award-number":["RGPIN-2024-05287"]}],"id":[{"id":"https:\/\/ror.org\/01h531d29","id-type":"ROR","asserted-by":"publisher"}]},{"DOI":"10.13039\/100012973","name":"Universit\u00e9 de Moncton","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100012973","id-type":"DOI","asserted-by":"crossref"}]},{"id":[{"id":"https:\/\/ror.org\/029tnqt29","id-type":"ROR","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Bioengineering"],"abstract":"<jats:p>Cardiovascular deterioration is a clinically important event in intensive care units, but early prediction remains challenging because risk may be reflected across structured clinical variables, physiological time series, and clinical text. In this work, we present MultiCardioNet, a multimodal deep learning framework for early prediction of a composite ICU deterioration endpoint using MIMIC-IV data. The prediction target was defined as vasopressor initiation and\/or early death during the 24\u201372 h outcome window, making the task broader than mortality prediction alone but also related to treatment escalation and circulatory support. The model combines structured clinical variables, 24-h vital sign time series, and timestamp-filtered radiology reports. Structured information was represented using enriched first-24-h clinical features, while physiological dynamics were modeled using a transformer-based time series encoder and radiology reports were represented using CXR-BERT-specialized embeddings. On the held-out test set, MultiCardioNet achieved an AUROC of 0.9014, AUPRC of 0.8481, and F1-score of 0.7723. These findings suggest that the three modalities provide complementary information for this composite deterioration endpoint in the internal test setting.<\/jats:p>","DOI":"10.3390\/bioengineering13090993","type":"journal-article","created":{"date-parts":[[2026,8,27]],"date-time":"2026-08-27T12:03:43Z","timestamp":1787832223000},"page":"993","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["MultiCardioNet: A Multimodal Deep Learning Model for Early Cardiovascular Deterioration Prediction in ICU Patients"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-9541-3013","authenticated-orcid":false,"given":"Bassem","family":"Jandoubi","sequence":"first","affiliation":[{"name":"Perception, Robotics, and Intelligent Machines (PRIME), Department of Computer Science, Universit\u00e9 de Moncton, Moncton, NB E1A3E9, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4378-2669","authenticated-orcid":false,"given":"Moulay A.","family":"Akhloufi","sequence":"additional","affiliation":[{"name":"Perception, Robotics, and Intelligent Machines (PRIME), Department of Computer Science, Universit\u00e9 de Moncton, Moncton, NB E1A3E9, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,8,27]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41597-022-01899-x","article-title":"MIMIC-IV, a Freely Accessible Electronic Health Record Dataset","volume":"10","author":"Johnson","year":"2023","journal-title":"Sci. Data"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"e179","DOI":"10.1016\/S2589-7500(20)30018-2","article-title":"Dynamic and Explainable Machine Learning Prediction of Mortality in Patients in the Intensive Care Unit: A Retrospective Study of High-Frequency Data in Electronic Patient Records","volume":"2","author":"Nielsen","year":"2020","journal-title":"Lancet Digit. Health"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1016\/j.jbi.2018.02.008","article-title":"Mortality Prediction in Intensive Care Units (ICUs) Using a Deep Rule-Based Fuzzy Classifier","volume":"79","author":"Davoodi","year":"2018","journal-title":"J. Biomed. Inform."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1038\/s41746-018-0029-1","article-title":"Scalable and Accurate Deep Learning with Electronic Health Records","volume":"1","author":"Rajkomar","year":"2018","journal-title":"npj Digit. Med."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"103876","DOI":"10.1016\/j.jbi.2021.103876","article-title":"Interpretability of Time-Series Deep Learning Models: A Study in Cardiovascular Patients Admitted to Intensive Care Unit","volume":"121","author":"Gandin","year":"2021","journal-title":"J. Biomed. Inform."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long Short-Term Memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_7","unstructured":"Bai, S., Kolter, J.Z., and Koltun, V. (2018). An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling. arXiv."},{"key":"ref_8","first-page":"5998","article-title":"Attention Is All You Need","volume":"Volume 30","author":"Vaswani","year":"2017","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Kaji, D.A., Zech, J.R., Kim, J.S., Cho, S.K., Dangayach, N.S., Costa, A.B., and Oermann, E.K. (2019). An Attention Based Deep Learning Model of Clinical Events in the Intensive Care Unit. PLoS ONE, 14.","DOI":"10.1371\/journal.pone.0211057"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"110406","DOI":"10.1016\/j.compbiomed.2025.110406","article-title":"Multivariate Multi-Horizon Time-Series Forecasting for Real-Time Patient Monitoring Based on Cascaded Fine Tuning of Attention-Based Models","volume":"194","author":"Saleh","year":"2025","journal-title":"Comput. Biol. Med."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"20552076251412651","DOI":"10.1177\/20552076251412651","article-title":"Transformer-Based Deep Learning for Early Detection of Intensive Care Unit-Acquired Bloodstream Infection Through Multivariate Time-Series Analysis","volume":"12","author":"Peng","year":"2026","journal-title":"Digit. Health"},{"key":"ref_12","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019, January 2\u20137). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), Minneapolis, MN, USA."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Boecking, B., Usuyama, N., Bannur, S., Castro, D.C., Schwaighofer, A., Hyland, S., Wetscherek, M., Naumann, T., Nori, A., and Alvarez-Valle, J. (2022). Making the Most of Text Semantics to Improve Biomedical Vision\u2013Language Processing. Computer Vision\u2014ECCV 2022, Springer Nature.","DOI":"10.1007\/978-3-031-20059-5_1"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1038\/s41746-022-00712-8","article-title":"Multimodal Machine Learning in Precision Health: A Scoping Review","volume":"5","author":"Kline","year":"2022","journal-title":"npj Digit. Med."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"105001","DOI":"10.1016\/j.jbi.2026.105001","article-title":"Explainable Multimodal Deep Learning Models for Variable-Length Sequences in Critically Ill Patients","volume":"177","author":"Martin","year":"2026","journal-title":"J. Biomed. Inform."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Jandoubi, B., and Akhloufi, M.A. (2025). Multimodal Artificial Intelligence in Medical Diagnostics. Information, 16.","DOI":"10.3390\/info16070591"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"161","DOI":"10.12788\/jhm.3151","article-title":"Statistical Modeling and Aggregate-Weighted Scoring Systems in Prediction of Mortality and ICU Transfer: A Systematic Review","volume":"14","author":"Linnen","year":"2019","journal-title":"J. Hosp. Med."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"465","DOI":"10.1016\/j.resuscitation.2012.12.016","article-title":"The Ability of the National Early Warning Score (NEWS) to Discriminate Patients at Risk of Early Cardiac Arrest, Unanticipated Intensive Care Unit Admission, and Death","volume":"84","author":"Smith","year":"2013","journal-title":"Resuscitation"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Thiele, D., Rodseth, R., Friedland, R., Berger, F., Mathew, C., Maslo, C., Moll, V., Leithner, C., Storm, C., and Krannich, A. (2025). Machine Learning Models for the Early Real-Time Prediction of Deterioration in Intensive Care Units\u2014A Novel Approach to the Early Identification of High-Risk Patients. J. Clin. Med., 14.","DOI":"10.3390\/jcm14020350"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"762","DOI":"10.1001\/jama.2016.0288","article-title":"Assessment of Clinical Criteria for Sepsis: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3)","volume":"315","author":"Seymour","year":"2016","journal-title":"JAMA"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1038\/s41746-023-00960-2","article-title":"Real-Time Machine Learning Model to Predict In-Hospital Cardiac Arrest Using Heart Rate Variability in ICU","volume":"6","author":"Lee","year":"2023","journal-title":"npj Digit. Med."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"e49142","DOI":"10.2196\/49142","article-title":"Prediction of In-Hospital Cardiac Arrest in the Intensive Care Unit: Machine Learning-Based Multimodal Approach","volume":"12","author":"Lee","year":"2024","journal-title":"JMIR Med. Inform."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Gao, S., Zhang, Y., Deng, M., Liu, H., Xu, W., Luo, M., Tian, Y., and Zhang, B. (2026). Machine Learning-Based Early Warning System for Hemodynamic Deterioration in Cardiovascular ICU Patients: A Bidirectional Cross-Validation Study. Front. Cardiovasc. Med., 12.","DOI":"10.3389\/fcvm.2025.1694001"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"707","DOI":"10.1007\/BF01709751","article-title":"The SOFA (Sepsis-related Organ Failure Assessment) Score to Describe Organ Dysfunction\/Failure","volume":"22","author":"Vincent","year":"1996","journal-title":"Intensive Care Med."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"818","DOI":"10.1097\/00003246-198510000-00009","article-title":"APACHE II: A Severity of Disease Classification System","volume":"13","author":"Knaus","year":"1985","journal-title":"Crit. Care Med."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"E1027","DOI":"10.1503\/cmaj.240132","article-title":"Clinical Evaluation of a Machine Learning\u2013Based Early Warning System for Patient Deterioration","volume":"196","author":"Verma","year":"2024","journal-title":"Can. Med. Assoc. J."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"e3743","DOI":"10.7717\/peerj-cs.3743","article-title":"Predicting Heart Failure Using MIMIC-IV and MIMIC-IV-ED: A Comparative Study of Machine Learning and Deep Learning Models","volume":"12","author":"Teoh","year":"2026","journal-title":"PeerJ Comput. Sci."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"513","DOI":"10.1016\/0306-4573(88)90021-0","article-title":"Term-Weighting Approaches in Automatic Text Retrieval","volume":"24","author":"Salton","year":"1988","journal-title":"Inf. Process. Manag."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random Forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Chen, T., and Guestrin, C. (2016, January 13\u201317). XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939785"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Saito, T., and Rehmsmeier, M. (2015). The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets. PLoS ONE, 10.","DOI":"10.1371\/journal.pone.0118432"}],"container-title":["Bioengineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2306-5354\/13\/9\/993\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,29]],"date-time":"2026-08-29T04:51:17Z","timestamp":1787979077000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2306-5354\/13\/9\/993"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,27]]},"references-count":31,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2026,9]]}},"alternative-id":["bioengineering13090993"],"URL":"https:\/\/doi.org\/10.3390\/bioengineering13090993","relation":{},"ISSN":["2306-5354"],"issn-type":[{"value":"2306-5354","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,27]]}}}