{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,2]],"date-time":"2026-09-02T20:51:07Z","timestamp":1788382267779,"version":"build-2803163510"},"reference-count":30,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2023,11,20]],"date-time":"2023-11-20T00:00:00Z","timestamp":1700438400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diseases"],"abstract":"<jats:p>Background: Considering the large number of patients with pulmonary symptoms admitted to the emergency department daily, it is essential to diagnose them correctly. It is necessary to quickly solve the differential diagnosis between COVID-19 and typical bacterial pneumonia to address them with the best management possible. In this setting, an artificial intelligence (AI) system can help radiologists detect pneumonia more quickly. Methods: We aimed to test the diagnostic performance of an AI system in detecting COVID-19 pneumonia and typical bacterial pneumonia in patients who underwent a chest X-ray (CXR) and were admitted to the emergency department. The final dataset was composed of three sub-datasets: the first included all patients positive for COVID-19 pneumonia (n = 1140, namely \u201cCOVID-19+\u201d), the second one included all patients with typical bacterial pneumonia (n = 500, \u201cpneumonia+\u201d), and the third one was composed of healthy subjects (n = 1000). Two radiologists were blinded to demographic, clinical, and laboratory data. The developed AI system was used to evaluate all CXRs randomly and was asked to classify them into three classes. Cohen\u2019s \u03ba was used for interrater reliability analysis. The AI system\u2019s diagnostic accuracy was evaluated using a confusion matrix, and 95%CIs were reported as appropriate. Results: The interrater reliability analysis between the most experienced radiologist and the AI system reported an almost perfect agreement for COVID-19+ (\u03ba = 0.822) and pneumonia+ (\u03ba = 0.913). We found 96% sensitivity (95% CIs = 94.9\u201396.9) and 79.8% specificity (76.4\u201382.9) for the radiologist and 94.7% sensitivity (93.4\u201395.8) and 80.2% specificity (76.9\u201383.2) for the AI system in the detection of COVID-19+. Moreover, we found 97.9% sensitivity (98\u201399.3) and 88% specificity (83.5\u201391.7) for the radiologist and 97.5% sensitivity (96.5\u201398.3) and 83.9% specificity (79\u201387.9) for the AI system in the detection of pneumonia+ patients. Finally, the AI system reached an accuracy of 93.8%, with a misclassification rate of 6.2% and weighted-F1 of 93.8% in detecting COVID+, pneumonia+, and healthy subjects. Conclusions: The AI system demonstrated excellent diagnostic performance in identifying COVID-19 and typical bacterial pneumonia in CXRs acquired in the emergency setting.<\/jats:p>","DOI":"10.3390\/diseases11040171","type":"journal-article","created":{"date-parts":[[2023,11,20]],"date-time":"2023-11-20T10:32:32Z","timestamp":1700476352000},"page":"171","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Artificial Intelligence Applied to Chest X-ray: A Reliable Tool to Assess the Differential Diagnosis of Lung Pneumonia in the Emergency Department"],"prefix":"10.3390","volume":"11","author":[{"given":"Davide","family":"Ippolito","sequence":"first","affiliation":[{"name":"Department of Diagnostic Radiology, Fondazione IRCCS San Gerardo dei Tintori, Via Pergolesi 33, 20900 Monza, Italy"},{"name":"School of Medicine, University of Milano-Bicocca, Via Cadore 48, 20900 Monza, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5742-802X","authenticated-orcid":false,"given":"Cesare","family":"Maino","sequence":"additional","affiliation":[{"name":"Department of Diagnostic Radiology, Fondazione IRCCS San Gerardo dei Tintori, Via Pergolesi 33, 20900 Monza, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Davide","family":"Gandola","sequence":"additional","affiliation":[{"name":"Department of Diagnostic Radiology, Fondazione IRCCS San Gerardo dei Tintori, Via Pergolesi 33, 20900 Monza, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8280-3787","authenticated-orcid":false,"given":"Paolo Niccol\u00f2","family":"Franco","sequence":"additional","affiliation":[{"name":"Department of Diagnostic Radiology, Fondazione IRCCS San Gerardo dei Tintori, Via Pergolesi 33, 20900 Monza, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Radu","family":"Miron","sequence":"additional","affiliation":[{"name":"Sentic Lab, Strada Elena Doamna 20, 700398 Ia\u0219i, Romania"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Vlad","family":"Barbu","sequence":"additional","affiliation":[{"name":"Sentic Lab, Strada Elena Doamna 20, 700398 Ia\u0219i, Romania"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marco","family":"Bologna","sequence":"additional","affiliation":[{"name":"Synbrain, Corso Milano 23, 20900 Milan, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rocco","family":"Corso","sequence":"additional","affiliation":[{"name":"Department of Diagnostic Radiology, Fondazione IRCCS San Gerardo dei Tintori, Via Pergolesi 33, 20900 Monza, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mihaela Elena","family":"Breaban","sequence":"additional","affiliation":[{"name":"Faculty of Computer Science, \u201cAlexandru Ioan Cuza\u201d University of Iasi, Strada General Henri Mathias Berthelot 16, 700483 Ia\u0219i, Romania"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,11,20]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"500","DOI":"10.1038\/s41568-018-0016-5","article-title":"Artificial Intelligence in Radiology","volume":"18","author":"Hosny","year":"2018","journal-title":"Nat. Rev. Cancer"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"540","DOI":"10.1055\/s-0038-1673383","article-title":"Artificial Intelligence in Radiology: Current Technology and Future Directions","volume":"22","author":"Syed","year":"2018","journal-title":"Semin. Musculoskelet. Radiol."},{"key":"ref_3","first-page":"101596","article-title":"Deep Learning Framework for Rapid and Accurate Respiratory COVID-19 Prediction Using Chest X-ray Images","volume":"35","author":"Ukwuoma","year":"2023","journal-title":"J. King Saud. Univ. Comput. Inf. Sci."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"100497","DOI":"10.1016\/j.ejro.2023.100497","article-title":"Performance of an AI Algorithm during the Different Phases of the COVID Pandemics: What Can We Learn from the AI and Vice Versa","volume":"11","author":"Catalano","year":"2023","journal-title":"Eur. J. Radiol. Open"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"4887","DOI":"10.1038\/s41598-023-31910-3","article-title":"Evidence of a Cognitive Bias in the Quantification of COVID-19 with CT: An Artificial Intelligence Randomised Clinical Trial","volume":"13","author":"Bercean","year":"2023","journal-title":"Sci. Rep."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"704256","DOI":"10.3389\/fmed.2021.704256","article-title":"Artificial Intelligence for COVID-19: A Systematic Review","volume":"8","author":"Wang","year":"2021","journal-title":"Front. Med."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1109\/RBME.2020.2987975","article-title":"Review of Artificial Intelligence Techniques in Imaging Data Acquisition, Segmentation, and Diagnosis for COVID-19","volume":"14","author":"Shi","year":"2021","journal-title":"IEEE Rev. Biomed. Eng."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Bouchareb, Y., Moradi Khaniabadi, P., Al Kindi, F., Al Dhuhli, H., Shiri, I., Zaidi, H., and Rahmim, A. (2021). Artificial Intelligence-Driven Assessment of Radiological Images for COVID-19. Comput. Biol. Med., 136.","DOI":"10.1016\/j.compbiomed.2021.104665"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"E72","DOI":"10.1148\/radiol.2020201160","article-title":"Frequency and Distribution of Chest Radiographic Findings in Patients Positive for COVID-19","volume":"296","author":"Wong","year":"2020","journal-title":"Radiology"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Bukhari, S.U.K., Bukhari, S.S.K., Syed, A., and Shah, S.S.H. (2020). The Diagnostic Evaluation of Convolutional Neural Network (CNN) for the Assessment of Chest X-ray of Patients Infected with COVID-19. medRxiv.","DOI":"10.1101\/2020.03.26.20044610"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"462","DOI":"10.1007\/s40846-020-00529-4","article-title":"Extracting Possibly Representative COVID-19 Biomarkers from X-ray Images with Deep Learning Approach and Image Data Related to Pulmonary Diseases","volume":"40","author":"Apostolopoulos","year":"2020","journal-title":"J. Med. Biol. Eng."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"W105","DOI":"10.1097\/RTI.0000000000000533","article-title":"Diagnostic Performance of Chest X-ray for COVID-19 Pneumonia During the SARS-CoV-2 Pandemic in Lombardy, Italy","volume":"35","author":"Schiaffino","year":"2020","journal-title":"J. Thorac. Imaging"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"209","DOI":"10.1016\/j.ejrad.2009.05.039","article-title":"Dose and Perceived Image Quality in Chest Radiography","volume":"72","author":"Veldkamp","year":"2009","journal-title":"Eur. J. Radiol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"19549","DOI":"10.1038\/s41598-020-76550-z","article-title":"COVID-Net: A Tailored Deep Convolutional Neural Network Design for Detection of COVID-19 Cases from Chest X-ray Images","volume":"10","author":"Wang","year":"2020","journal-title":"Sci. Rep."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"e210217","DOI":"10.1148\/ryai.210217","article-title":"Performance of a Chest Radiograph AI Diagnostic Tool for COVID-19: A Prospective Observational Study","volume":"4","author":"Sun","year":"2022","journal-title":"Radiol. Artif. Intell."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"610","DOI":"10.1038\/s42256-021-00338-7","article-title":"AI for Radiographic COVID-19 Detection Selects Shortcuts over Signal","volume":"3","author":"DeGrave","year":"2021","journal-title":"Nat. Mach. Intell."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"109278","DOI":"10.1016\/j.knosys.2022.109278","article-title":"Explainable Multi-Instance and Multi-Task Learning for COVID-19 Diagnosis and Lesion Segmentation in CT Images","volume":"252","author":"Li","year":"2022","journal-title":"Knowl.-Based Syst."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"He, K., Gkioxari, G., Doll\u00e1r, P., and Girshick, R. (2017). Mask R-CNN. arXiv.","DOI":"10.1109\/ICCV.2017.322"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2015). Deep Residual Learning for Image Recognition. arXiv.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Lin, T.-Y., Doll\u00e1r, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2016). Feature Pyramid Networks for Object Detection. arXiv.","DOI":"10.1109\/CVPR.2017.106"},{"key":"ref_21","unstructured":"Ren, S., He, K., Girshick, R., and Sun, J. (2015). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1078","DOI":"10.1016\/j.radi.2021.04.010","article-title":"Radiographers and COVID-19 Pneumonia: Diagnostic Performance Using CO-RADS","volume":"27","author":"Vicini","year":"2021","journal-title":"Radiography"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Baltazar, L.R., Manzanillo, M.G., Gaudillo, J., Viray, E.D., Domingo, M., Tiangco, B., and Albia, J. (2021). Artificial Intelligence on COVID-19 Pneumonia Detection Using Chest Xray Images. PLoS ONE, 16.","DOI":"10.1371\/journal.pone.0257884"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"6039","DOI":"10.1007\/s00330-020-07628-5","article-title":"Artificial Intelligence\u2013Assisted Chest X-ray Assessment Scheme for COVID-19","volume":"31","author":"Rangarajan","year":"2021","journal-title":"Eur. Radiol."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"572","DOI":"10.1016\/j.acra.2021.01.016","article-title":"Multi-Radiologist User Study for Artificial Intelligence-Guided Grading of COVID-19 Lung Disease Severity on Chest Radiographs","volume":"28","author":"Li","year":"2021","journal-title":"Acad. Radiol."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Salvatore, C., Interlenghi, M., Monti, C.B., Ippolito, D., Capra, D., Cozzi, A., Schiaffino, S., Polidori, A., Gandola, D., and Al\u00ec, M. (2021). Artificial Intelligence Applied to Chest X-ray for Differential Diagnosis of COVID-19 Pneumonia. Diagnostics, 11.","DOI":"10.3390\/diagnostics11030530"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1186\/s41747-020-00203-z","article-title":"Machine Learning Applied on Chest X-ray Can Aid in the Diagnosis of COVID-19: A First Experience from Lombardy, Italy","volume":"5","author":"Castiglioni","year":"2021","journal-title":"Eur. Radiol. Exp."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"E88","DOI":"10.1148\/radiol.2020202944","article-title":"Diagnosis of Coronavirus Disease 2019 Pneumonia by Using Chest Radiography: Value of Artificial Intelligence","volume":"298","author":"Zhang","year":"2021","journal-title":"Radiology"},{"key":"ref_29","first-page":"E167","article-title":"DeepCOVID-XR: An Artificial Intelligence Algorithm to Detect COVID-19 on Chest Radiographs Trained and Tested on a Large U","volume":"299","author":"Wehbe","year":"2021","journal-title":"S. Clin. Data Set. Radiol."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Islam, N., Ebrahimzadeh, S., Salameh, J.-P., Kazi, S., Fabiano, N., Treanor, L., Absi, M., Hallgrimson, Z., Leeflang, M.M., and Hooft, L. (2021). Thoracic Imaging Tests for the Diagnosis of COVID-19. Cochrane Database Syst. Rev., 2021.","DOI":"10.1002\/14651858.CD013639.pub4"}],"container-title":["Diseases"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2079-9721\/11\/4\/171\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:26:15Z","timestamp":1760131575000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2079-9721\/11\/4\/171"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,20]]},"references-count":30,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["diseases11040171"],"URL":"https:\/\/doi.org\/10.3390\/diseases11040171","relation":{},"ISSN":["2079-9721"],"issn-type":[{"value":"2079-9721","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,20]]}}}