{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,6]],"date-time":"2026-06-06T16:53:16Z","timestamp":1780764796915,"version":"3.54.1"},"reference-count":40,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,12,12]],"date-time":"2024-12-12T00:00:00Z","timestamp":1733961600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Comput. Sci."],"abstract":"<jats:sec><jats:title>Introduction<\/jats:title><jats:p>An automated computerized approach can aid radiologists in the early diagnosis of lung disease from video modalities. This study focuses on the difficulties associated with identifying and categorizing respiratory diseases, including COVID-19, influenza, and pneumonia.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>We propose a novel method that combines three dimensional (3D) models, model explainability (XAI), and a Decision Support System (DSS) that utilizes lung ultrasound (LUS) videos. The objective of the study is to improve the quality of video frames, boost the diversity of the dataset, maintain the sequence of frames, and create a hybrid 3D model [Three-Dimensional Time Distributed Convolutional Neural Network-Long short-term memory (TD-CNNLSTM-LungNet)] for precise classification. The proposed methodology involves applying morphological opening and contour detection to improve frame quality, utilizing geometrical augmentation for dataset balance, introducing a graph-based approach for frame sequencing, and implementing a hybrid 3D model combining time-distributed CNN and LSTM networks utilizing vast ablation study. Model explainability is ensured through heatmap generation, region of interest segmentation, and Probability Density Function (PDF) graphs illustrating feature distribution.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>Our model TD-CNN-LSTM-LungNet attained a remarkable accuracy of 96.57% in classifying LUS videos into pneumonia, COVID-19, normal, and other lung disease classes, which is above compared to ten traditional transfer learning models experimented with in this study. The eleven-ablation case study reduced training costs and redundancy. K-fold cross-validation and accuracy-loss curves demonstrated model generalization. The DSS, incorporating Layer Class Activation Mapping (LayerCAM) and heatmaps, improved interpretability and reliability, and PDF graphs facilitated precise decision-making by identifying feature boundaries. The DSS facilitates clinical marker analysis, and the validation by using the proposed algorithms highlights its impact on a reliable diagnosis outcome.<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussion<\/jats:title><jats:p>Our proposed methodology could assist radiologists in accurately detecting and comprehending the patterns of respiratory disorders.<\/jats:p><\/jats:sec>","DOI":"10.3389\/fcomp.2024.1438126","type":"journal-article","created":{"date-parts":[[2024,12,12]],"date-time":"2024-12-12T06:22:38Z","timestamp":1733984558000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":12,"title":["Automated diagnosis of respiratory diseases from lung ultrasound videos ensuring XAI: an innovative hybrid model approach"],"prefix":"10.3389","volume":"6","author":[{"given":"Arefin Ittesafun","family":"Abian","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mohaimenul Azam","family":"Khan Raiaan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Asif","family":"Karim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sami","family":"Azam","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nur Mohammad","family":"Fahad","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Niusha","family":"Shafiabady","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kheng Cher","family":"Yeo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Friso","family":"De Boer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2024,12,12]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","first-page":"105399","DOI":"10.1016\/j.jcv.2023.105399","article-title":"Focusing on severe infections with the respiratory syncytial virus (RSV) in adults: risk factors, symptomatology and clinical course compared to influenza A\/B and the original SARS-CoV-2 strain","volume":"161","author":"Ambrosch","year":"2023","journal-title":"J. Clin. Virol."},{"key":"ref2","year":"2022"},{"key":"ref3","doi-asserted-by":"publisher","first-page":"42","DOI":"10.3390\/fi11020042","article-title":"3D-CNN-based fused feature maps with LSTM applied to action recognition","volume":"11","author":"Arif","year":"2019","journal-title":"Fut. Internet"},{"key":"ref4","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1007\/s10278-023-00925-7","article-title":"An automated decision support system to analyze malignancy patterns of breast masses employing medically relevant features of ultrasound images","volume":"37","author":"Azam","year":"2024","journal-title":"J. Imaging Inform. Med."},{"key":"ref5","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1007\/978-3-030-66614-9_8","article-title":"Multimodality 3D lung imaging","volume-title":"Image-guided management of COVID-19 lung disease","author":"Bard","year":"2021"},{"key":"ref6","doi-asserted-by":"publisher","first-page":"5486","DOI":"10.3390\/s21165486","article-title":"Pulmonary COVID-19: learning spatiotemporal features combining CNN and LSTM networks for LUS video classification","volume":"21","author":"Barros","year":"2021","journal-title":"Sensors"},{"key":"ref7","doi-asserted-by":"publisher","first-page":"106156","DOI":"10.1016\/j.compbiomed.2022.106156","article-title":"Explanatory classification of CXR images into COVID-19, pneumonia and Tuberculosis using deep learning and XAI","volume":"150","author":"Bhandari","year":"2022","journal-title":"Comput. Biol. Med."},{"key":"ref8","doi-asserted-by":"publisher","first-page":"12084","DOI":"10.48550\/arXiv.2004.12084","article-title":"POCOVID-net: automatic detection of COVID-19 from a new lung ultrasound imaging dataset (POCUS)","volume":"2004","author":"Born","year":"2020","journal-title":"arXiv [Preprint]"},{"key":"ref9","first-page":"14572","article-title":"Seqtrack: Sequence to sequence learning for visual object tracking","author":"Chen","year":"2023"},{"key":"ref10","doi-asserted-by":"publisher","first-page":"104296","DOI":"10.1016\/j.compbiomed.2021.104296","article-title":"An integrated autoencoder-based hybrid CNN-LSTM model for COVID-19 severity prediction from LUS","volume":"132","author":"Dastider","year":"2021","journal-title":"Comput. Biol. Med."},{"key":"ref11","doi-asserted-by":"publisher","first-page":"431","DOI":"10.1053\/j.ro.2023.06.002","article-title":"Pleural effusions: clues for diagnosis and characterization","volume":"58","author":"De Groot","year":"2023","journal-title":"Semin. Roentgenol."},{"key":"ref12","doi-asserted-by":"publisher","first-page":"e0255886","DOI":"10.1371\/journal.pone.0255886","article-title":"Deep-learning based detection of COVID-19 using lung ultrasound imagery","volume":"16","author":"Diaz-Escobar","year":"2021","journal-title":"PLoS One"},{"key":"ref13","doi-asserted-by":"publisher","first-page":"220","DOI":"10.1177\/08850666221120221","article-title":"Lung ultrasonographic signs in pulmonary disease\u2013a video review","volume":"38","author":"Dugar","year":"2023","journal-title":"J. Intensive Care Med."},{"key":"ref14","doi-asserted-by":"publisher","first-page":"100687","DOI":"10.1016\/j.imu.2021.100687","article-title":"Automated detection of pneumonia in lung ultrasound using deep video classification for COVID-19","volume":"25","author":"Ebadi","year":"2021","journal-title":"Inform. Med. Unlocked"},{"key":"ref15","doi-asserted-by":"publisher","first-page":"198","DOI":"10.31083\/j.fbl2707198","article-title":"COVIDx-US: An open-access benchmark dataset of ultrasound imaging data for AI-driven COVID-19 analytics","volume":"27","author":"Ebadi","year":"2022","journal-title":"Frontiers in Bioscience (Landmark Edition)"},{"key":"ref16","doi-asserted-by":"publisher","first-page":"e230147","DOI":"10.1148\/ryai.230147","article-title":"Curated and annotated dataset of lung US images in Zambian children with clinical pneumonia","volume":"6","author":"Etter","year":"2024","journal-title":"Radiol. Artif. Intell."},{"key":"ref7001","doi-asserted-by":"crossref","first-page":"5875","DOI":"10.1109\/TIP.2021.3089943","article-title":"Layercam: Exploring hierarchical class activation maps for localization","volume":"30","author":"Jiang","year":"2021","journal-title":"IEEE Transactions on Image Processing"},{"key":"ref17","doi-asserted-by":"publisher","first-page":"143","DOI":"10.1016\/j.jiph.2022.12.008","article-title":"Do selected lifestyle parameters affect the severity and symptoms of COVID-19 among elderly patients? The retrospective evaluation of individuals from the STOP-COVID registry of the PoLoCOV study","volume":"16","author":"Kapusta","year":"2023","journal-title":"J. Infect. Public Health"},{"key":"ref18","doi-asserted-by":"publisher","first-page":"619","DOI":"10.2147\/IDR.S392844","article-title":"Outcomes and predictors of severe community-acquired pneumonia among adults admitted to the University of Gondar Comprehensive Specialized Hospital: a prospective follow-up study","volume":"16","author":"Kassaw","year":"2023","journal-title":"Infection Drug Resistance"},{"key":"ref19","doi-asserted-by":"publisher","first-page":"1566","DOI":"10.3390\/biomedicines11061566","article-title":"A computer-aided diagnostic system to identify diabetic retinopathy, utilizing a modified compact convolutional transformer and low-resolution images to reduce computation time","volume":"11","author":"Khan","year":"2023","journal-title":"Biomedicines"},{"key":"ref20","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s41479-023-00110-y","article-title":"Acute organ injury and long-term sequelae of severe pneumococcal infections","volume":"15","author":"Kruckow","year":"2023","journal-title":"Pneumonia"},{"key":"ref21","first-page":"2483","article-title":"Weakly semi-supervised detector-based video classification with temporal context for LUS","author":"Li","year":"2023"},{"key":"ref22","doi-asserted-by":"publisher","first-page":"693448","DOI":"10.3389\/fphys.2021.693448","article-title":"Classification of lung disease in children by using LUS images and deep convolutional neural network","volume":"12","author":"Magrelli","year":"2021","journal-title":"Front. Physiol."},{"key":"ref23","doi-asserted-by":"publisher","first-page":"60039","DOI":"10.1109\/ACCESS.2022.3179577","article-title":"Timedistributed-cnn-lstm: a hybrid approach combining CNN and LSTM to classify brain tumor on 3d mri scans performing ablation study","volume":"10","author":"Montaha","year":"2022","journal-title":"IEEE Access"},{"key":"ref24","doi-asserted-by":"publisher","first-page":"80","DOI":"10.1016\/j.inffus.2021.02.013","article-title":"COVID-19 and non-COVID-19 classification using multi-layers fusion from LUS images","volume":"72","author":"Muhammad","year":"2021","journal-title":"Inform. Fusion"},{"key":"ref25","first-page":"1","article-title":"Lung ultrasound image classification using deep learning and histogram of oriented gradients features for COVID-19 detection","author":"Nehary","year":"2023"},{"key":"ref26","doi-asserted-by":"publisher","first-page":"3904","DOI":"10.1121\/10.0007273","article-title":"Diagnostic ultrasound imaging of the lung: a simulation approach based on propagation and reverberation in the human body","volume":"150","author":"Ostras","year":"2021","journal-title":"J. Acoust. Soc. Am."},{"key":"ref27","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3587271","article-title":"Co-design of human-centered, explainable AI for clinical decision support","volume":"13","author":"Panigutti","year":"2023","journal-title":"ACM Trans. Interact. Intell. Syst."},{"key":"ref28","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1007\/s12055-022-01430-2","article-title":"Current investigative modalities for detecting and staging lung cancers: a comprehensive summary","volume":"39","author":"Philip","year":"2023","journal-title":"Indian J. Thoracic Cardiovasc. Surgery"},{"key":"ref29","doi-asserted-by":"publisher","first-page":"2932","DOI":"10.3390\/s18092932","article-title":"A time-distributed spatiotemporal feature learning method for machine health monitoring with multi-sensor time series","volume":"18","author":"Qiao","year":"2018","journal-title":"Sensors"},{"key":"ref30","doi-asserted-by":"publisher","first-page":"372","DOI":"10.3390\/fi15120372","article-title":"IoT-based object-detection system to safeguard endangered animals and bolster agricultural farm security","volume":"15","author":"Raiaan","year":"2023","journal-title":"Future Internet"},{"key":"ref31","doi-asserted-by":"publisher","first-page":"106279","DOI":"10.1016\/j.bspc.2024.106279","article-title":"Mammo-light: a lightweight convolutional neural network for diagnosing breast cancer from mammography images","volume":"94","author":"Raiaan","year":"2024","journal-title":"Biomed. Signal Proc. Control"},{"key":"ref32","doi-asserted-by":"publisher","first-page":"105567","DOI":"10.1016\/j.bspc.2023.105567","article-title":"Hybrid framework for respiratory lung diseases detection based on classical CNN and quantum classifiers from chest X-rays","volume":"88","author":"Rao","year":"2024","journal-title":"Biomed. Signal Process. Control"},{"key":"ref33","doi-asserted-by":"publisher","first-page":"2676","DOI":"10.1109\/TMI.2020.2994459","article-title":"Deep learning for classification and localization of COVID-19 markers in point-of-care LUS","volume":"39","author":"Roy","year":"2020","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref34","doi-asserted-by":"publisher","first-page":"680","DOI":"10.3390\/jpm12050680","article-title":"LungNet22: a fine-tuned model for multiclass classification and prediction of lung disease using X-ray images","volume":"12","author":"Shamrat","year":"2022","journal-title":"J. Pers. Med."},{"key":"ref35","first-page":"265","article-title":"Improved processing of ultrasound tongue videos by combining convlstm and 3d convolutional networks","author":"Shandiz","year":"2022"},{"key":"ref36","first-page":"3103","article-title":"Deep learning video classification of LUS features associated with pneumonia","author":"Shea","year":"2023"},{"key":"ref37","doi-asserted-by":"publisher","first-page":"1459","DOI":"10.1002\/jum.15284","article-title":"Is there a role for lung ultrasound during theCOVID\u201019 pandemic?","volume":"39","author":"Soldati","year":"2020","journal-title":"J. Ultrasound Med."},{"key":"ref38","doi-asserted-by":"publisher","first-page":"38","DOI":"10.1016\/j.ejmp.2021.02.023","article-title":"Automatic deep learning-based pleural effusion classification in LUS images for respiratory pathology diagnosis","volume":"83","author":"Tsai","year":"2021","journal-title":"Phys. Med."},{"key":"ref39","doi-asserted-by":"publisher","first-page":"104162","DOI":"10.1016\/j.bspc.2022.104162","article-title":"Deep CNN for COPD identification by multi-view snapshot integration of 3D airway tree and lung field","volume":"79","author":"Wu","year":"2023","journal-title":"Biomed. Signal Process. Control"}],"container-title":["Frontiers in Computer Science"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcomp.2024.1438126\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,12]],"date-time":"2024-12-12T06:22:41Z","timestamp":1733984561000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcomp.2024.1438126\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,12]]},"references-count":40,"alternative-id":["10.3389\/fcomp.2024.1438126"],"URL":"https:\/\/doi.org\/10.3389\/fcomp.2024.1438126","relation":{},"ISSN":["2624-9898"],"issn-type":[{"value":"2624-9898","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,12]]},"article-number":"1438126"}}