{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,22]],"date-time":"2026-10-22T07:46:08Z","timestamp":1792655168849,"version":"4.5.0"},"reference-count":38,"publisher":"MDPI AG","issue":"22","license":[{"start":{"date-parts":[[2024,11,18]],"date-time":"2024-11-18T00:00:00Z","timestamp":1731888000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>Background: The increasing integration of AI in chest X-ray evaluation holds promise for enhancing diagnostic accuracy and optimizing clinical workflows. However, understanding its performance in real-world clinical settings is essential. Objectives: In this study, we evaluated the sensitivity (Se) and specificity (Sp) of an AI-based software (Arterys MICA v29.4.0) alongside a radiology resident in interpreting chest X-rays referred from the emergency department (ED), using a senior radiologist\u2019s assessment as the gold standard (GS). We assessed the concordance between the AI system and the resident, noted the frequency of doubtful cases for each category, identified how many were considered positive by the GS, and assessed variables that AI was not trained to detect. Methods: We conducted a retrospective observational study analyzing chest X-rays from a sample of 784 patients referred from the ED at our hospital. The AI system was trained to detect five categorical variables\u2014pulmonary nodule, pulmonary opacity, pleural effusion, pneumothorax, and fracture\u2014and assign each a confidence label (\u201cpositive\u201d, \u201cdoubtful\u201d, or \u201cnegative\u201d). Results: Sensitivity in detecting fractures and pneumothorax was high (100%) for both AI and the resident, moderate for pulmonary opacity (AI = 76%, resident = 71%), and acceptable for pleural effusion (AI = 60%, resident = 67%), with negative predictive values (NPV) above 95% and areas under the curve (AUC) exceeding 0.8. The resident showed moderate sensitivity (75%) for pulmonary nodules, while AI\u2019s sensitivity was low (33%). AI assigned a \u201cdoubtful\u201d label to some diagnoses, most of which were deemed negative by the GS; the resident expressed doubt less frequently. The Kappa coefficient between the resident and AI was fair (0.3) across most categories, except for pleural effusion, where concordance was moderate (0.5). Our study highlighted additional findings not detected by AI, including 16% prevalence of mediastinal abnormalities, 20% surgical materials, and 20% other pulmonary findings. Conclusions: Although AI demonstrated utility in identifying most primary findings\u2014except for pulmonary nodules\u2014its high NPV suggests it may be valuable for screening. Further training of the AI software and broadening its scope to identify additional findings could enhance its detection capabilities and increase its applicability in clinical practice.<\/jats:p>","DOI":"10.3390\/diagnostics14222592","type":"journal-article","created":{"date-parts":[[2024,11,18]],"date-time":"2024-11-18T13:03:14Z","timestamp":1731934994000},"page":"2592","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":20,"title":["Diagnostic Performance of Artificial Intelligence in Chest Radiographs Referred from the Emergency Department"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-2474-9858","authenticated-orcid":false,"given":"Julia","family":"L\u00f3pez Alcolea","sequence":"first","affiliation":[{"name":"Hospital Universitario QuironSalud Madrid, 28223 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ana","family":"Fern\u00e1ndez Alfonso","sequence":"additional","affiliation":[{"name":"Hospital Universitario QuironSalud Madrid, 28223 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Raquel","family":"Cano Alonso","sequence":"additional","affiliation":[{"name":"Hospital Universitario QuironSalud Madrid, 28223 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ana","family":"\u00c1lvarez V\u00e1zquez","sequence":"additional","affiliation":[{"name":"Hospital Universitario QuironSalud Madrid, 28223 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-9802-6756","authenticated-orcid":false,"given":"Alejandro","family":"D\u00edaz Moreno","sequence":"additional","affiliation":[{"name":"Hospital Universitario QuironSalud Madrid, 28223 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-4760-5357","authenticated-orcid":false,"given":"David","family":"Garc\u00eda Castellanos","sequence":"additional","affiliation":[{"name":"Hospital Universitario QuironSalud Madrid, 28223 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Luc\u00eda","family":"Sanabria Greciano","sequence":"additional","affiliation":[{"name":"Hospital Universitario QuironSalud Madrid, 28223 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chawar","family":"Hayoun","sequence":"additional","affiliation":[{"name":"Hospital Universitario QuironSalud Madrid, 28223 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Manuel","family":"Recio Rodr\u00edguez","sequence":"additional","affiliation":[{"name":"Hospital Universitario QuironSalud Madrid, 28223 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7558-4325","authenticated-orcid":false,"given":"Cristina","family":"Andreu V\u00e1zquez","sequence":"additional","affiliation":[{"name":"Faculty of Biomedical and Health Science, Universidad Europea de Madrid, 28670 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6990-0619","authenticated-orcid":false,"given":"Israel John","family":"Thuissard Vasallo","sequence":"additional","affiliation":[{"name":"Faculty of Biomedical and Health Science, Universidad Europea de Madrid, 28670 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Vicente","family":"Mart\u00ednez de Vega","sequence":"additional","affiliation":[{"name":"Hospital Universitario QuironSalud Madrid, 28223 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,11,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"110913","DOI":"10.1016\/j.ejrad.2023.110913","article-title":"Breast cancer missed at screening; hindsight or mistakes?","volume":"165","author":"Hovda","year":"2023","journal-title":"Eur. J. Radiol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"e232479","DOI":"10.1148\/radiol.232479","article-title":"Early Indicators of the Impact of Using AI in Mammography Screening for Breast Cancer","volume":"311","author":"Lauritzen","year":"2024","journal-title":"Radiology"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"111638","DOI":"10.1016\/j.ejrad.2024.111638","article-title":"AI supported detection of cerebral multiple sclerosis lesions decreases radiologic reporting times","volume":"178","author":"Peters","year":"2024","journal-title":"Eur. J. Radiol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"110934","DOI":"10.1016\/j.ejrad.2023.110934","article-title":"Artificial intelligence-based diagnosis of Alzheimer\u2019s disease with brain MRI images","volume":"165","author":"Yao","year":"2023","journal-title":"Eur. J. Radiol."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Wang, D., Jin, R., Shieh, C.C., Ng, A.Y., Pham, H., Dugal, T., Barnett, M., Winoto, L., Wang, C., and Barnett, Y. (2023). Real world validation of an AI-based CT hemorrhage detection tool. Front. Neurol., 14.","DOI":"10.3389\/fneur.2023.1177723"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1072","DOI":"10.1002\/acn3.51790","article-title":"Accuracy of artificial intelligence software for CT angiography in stroke","volume":"10","author":"Mair","year":"2023","journal-title":"Ann. Clin. Transl. Neurol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"594","DOI":"10.1016\/j.diii.2022.06.004","article-title":"Added value of an artificial intelligence solution for fracture detection in the radiologist\u2019s daily trauma emergencies workflow","volume":"103","author":"Verdot","year":"2022","journal-title":"Diagn. Interv. Imaging"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1148\/radiol.211785","article-title":"Artificial Intelligence in Fracture Detection: A Systematic Review and Meta-Analysis","volume":"304","author":"Kuo","year":"2022","journal-title":"Radiology"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1148\/radiol.2021203886","article-title":"Assessment of an AI Aid in Detection of Adult Appendicular Skeletal Fractures by Emergency Physicians and Radiologists: A Multicenter Cross-sectional Diagnostic Study","volume":"300","author":"Duron","year":"2021","journal-title":"Radiology"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2129","DOI":"10.1007\/s00256-022-04070-0","article-title":"Automated detection of acute appendicular skeletal fractures in pediatric patients using deep learning","volume":"51","author":"Hayashi","year":"2022","journal-title":"Skelet. Radiol."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Siouras, A., Moustakidis, S., Giannakidis, A., Chalatsis, G., Liampas, I., Vlychou, M., Hantes, M., Tasoulis, S., and Tsaopoulos, D. (2022). Knee Injury Detection Using Deep Learning on MRI Studies: A Systematic Review. Diagnostics, 12.","DOI":"10.3390\/diagnostics12020537"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1007\/s00256-021-03830-8","article-title":"Artificial intelligence for MRI diagnosis of joints: A scoping review of the current state-of-the-art of deep learning-based approaches","volume":"51","author":"Fritz","year":"2022","journal-title":"Skelet. Radiol."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.diii.2022.11.007","article-title":"Artificial intelligence: A critical review of applications for lung nodule and lung cancer","volume":"104","author":"Chassagnon","year":"2023","journal-title":"Diagn. Interv. Imaging"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Irmici, G., C\u00e8, M., Caloro, E., Khenkina, N., Della Pepa, G., Ascenti, V., Martinenghi, C., Papa, S., Oliva, G., and Cellina, M. (2023). Chest X-ray in Emergency Radiology: What Artificial Intelligence Applications Are Available?. Diagnostics, 13.","DOI":"10.3390\/diagnostics13020216"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Li, D., Pehrson, L.M., Lauridsen, C.A., T\u00f8ttrup, L., Fraccaro, M., Elliott, D., Zaj\u0105c, H.D., Darkner, S., Carlsen, J.F., and Nielsen, M.B. (2021). The Added Effect of Artificial Intelligence on Physicians\u2019 Performance in Detecting Thoracic Pathologies on CT and Chest X-ray: A Systematic Review. Diagnostics, 11.","DOI":"10.3390\/diagnostics11122206"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"110887","DOI":"10.1016\/j.ejrad.2023.110887","article-title":"The role of AI in prostate MRI quality and interpretation: Opportunities and challenges","volume":"165","author":"Kim","year":"2023","journal-title":"Eur. J. Radiol."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"110928","DOI":"10.1016\/j.ejrad.2023.110928","article-title":"Radiomics based automated quality assessment for T2W prostate MR images","volume":"165","author":"Thijssen","year":"2023","journal-title":"Eur. J. Radiol."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"110924","DOI":"10.1016\/j.ejrad.2023.110924","article-title":"Deep learning for assessing image quality in bi-parametric prostate MRI: A feasibility study","volume":"165","author":"Alis","year":"2023","journal-title":"Eur. J. Radiol."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"445","DOI":"10.1016\/j.diii.2022.08.001","article-title":"Does artificial intelligence surpass the radiologist?","volume":"103","author":"Soyer","year":"2022","journal-title":"Diagn. Interv. Imaging"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"810","DOI":"10.1126\/science.aaw0029","article-title":"Regulation of predictive analytics in medicine","volume":"363","author":"Parikh","year":"2019","journal-title":"Science"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1007\/s13312-011-0055-4","article-title":"Kumar-2011-Receiver operating characteristic 1","volume":"48","author":"Kumar","year":"2011","journal-title":"Indian Pediatr."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"e20200186","DOI":"10.1590\/1677-5449.200186","article-title":"Use of roc curves in clinical and experimental studies","volume":"19","author":"Polo","year":"2020","journal-title":"J. Vasc. Bras."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1315","DOI":"10.1097\/JTO.0b013e3181ec173d","article-title":"Receiver Operating Characteristic Curve in Diagnostic Test Assessment","volume":"5","author":"Mandrekar","year":"2010","journal-title":"J. Thorac. Oncol."},{"key":"ref_24","first-page":"627","article-title":"Receiver Operating Characteristic (ROC) Curve Analysis for Medical Diagnostic Test Evaluation","volume":"4","year":"2013","journal-title":"Casp. J. Intern. Med."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"e191095","DOI":"10.1001\/jamanetworkopen.2019.1095","article-title":"Development and Validation of a Deep Learning-Based Automated Detection Algorithm for Major Thoracic Diseases on Chest Radiographs","volume":"2","author":"Hwang","year":"2019","journal-title":"JAMA Netw. Open"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"e230860","DOI":"10.1148\/radiol.230860","article-title":"Using AI to Improve Radiologist Performance in Detection of Abnormalities on Chest Radiographs","volume":"309","author":"Bennani","year":"2023","journal-title":"Radiology"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"218","DOI":"10.1148\/radiol.2018180237","article-title":"Development and validation of deep learning-based automatic detection algorithm for malignant pulmonary nodules on chest radiographs","volume":"290","author":"Nam","year":"2019","journal-title":"Radiology"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"214","DOI":"10.1136\/pmj.79.930.214","article-title":"Chest radiographs in the emergency department: Is the radiologist really necessary?","volume":"79","author":"Gatt","year":"2003","journal-title":"Postgrad. Med. J."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"j3855","DOI":"10.1136\/bmj.j3855","article-title":"Radiologist shortage leaves patient care at risk, warns royal college","volume":"358","author":"Kmietowicz","year":"2017","journal-title":"BMJ"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Kim, E.Y., Kim, Y.J., Choi, W.J., Jeon, J.S., Kim, M.Y., Oh, D.H., Jin, K.N., and Cho, Y.J. (2022). Concordance rate of radiologists and a commercialized deep-learning solution for chest X-ray: Real-world experience with a multicenter health screening cohort. PLoS ONE, 17.","DOI":"10.1371\/journal.pone.0264383"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.crad.2022.08.129","article-title":"Validation study of machine-learning chest radiograph software in primary and emergency medicine","volume":"78","author":"Ahn","year":"2023","journal-title":"Clin. Radiol."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Lee, K., Lee, S., Kwak, J.S., Park, H., Oh, H., and Koh, J.C. (2024). Development and Validation of an Artificial Intelligence Model for Detecting Rib Fractures on Chest Radiographs. J. Clin. Med., 13.","DOI":"10.3390\/jcm13133850"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"2138","DOI":"10.1007\/s10278-023-00868-z","article-title":"Automated Rib Fracture Detection on Chest X-Ray Using Contrastive Learning","volume":"36","author":"Sun","year":"2023","journal-title":"J. Digit. Imaging"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"E2229289","DOI":"10.1001\/jamanetworkopen.2022.29289","article-title":"Association of Artificial Intelligence-Aided Chest Radiograph Interpretation with Reader Performance and Efficiency","volume":"5","author":"Ahn","year":"2022","journal-title":"JAMA Netw. Open"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"e2022779","DOI":"10.1001\/jamanetworkopen.2020.22779","article-title":"Comparison of Chest Radiograph Interpretations by Artificial Intelligence Algorithm vs. Radiology Residents","volume":"3","author":"Wu","year":"2020","journal-title":"JAMA Netw. Open"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"9664","DOI":"10.1007\/s00330-021-08074-7","article-title":"AI-based improvement in lung cancer detection on chest radiographs: Results of a multi-reader study in NLST dataset","volume":"31","author":"Yoo","year":"2021","journal-title":"Eur. Radiol."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"E25663","DOI":"10.1097\/MD.0000000000025663","article-title":"Evaluation of a deep learning-based computer-aided detection algorithm on chest radiographs: Case-control study","volume":"100","author":"Choi","year":"2021","journal-title":"Medicine"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"450","DOI":"10.1148\/radiol.2021202818","article-title":"Added value of deep learning-based detection system for multiple major findings on chest radiographs: A randomized crossover study","volume":"299","author":"Sung","year":"2021","journal-title":"Radiology"}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/14\/22\/2592\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:34:40Z","timestamp":1760114080000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/14\/22\/2592"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,18]]},"references-count":38,"journal-issue":{"issue":"22","published-online":{"date-parts":[[2024,11]]}},"alternative-id":["diagnostics14222592"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics14222592","relation":{},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,18]]}}}