{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,23]],"date-time":"2026-04-23T08:19:20Z","timestamp":1776932360744,"version":"3.51.2"},"reference-count":31,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2025,2,18]],"date-time":"2025-02-18T00:00:00Z","timestamp":1739836800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>Background\/Objectives: The growing use of artificial intelligence (AI) in musculoskeletal radiographs presents significant potential to improve diagnostic accuracy and optimize clinical workflow. However, assessing its performance in clinical environments is essential for successful implementation. We hypothesized that our AI applied to urgent bone X-rays could detect fractures, joint dislocations, and effusion with high sensitivity (Sens) and specificity (Spec). The specific objectives of our study were as follows: 1. To determine the Sens and Spec rates of AI in detecting bone fractures, dislocations, and elbow joint effusion compared to the gold standard (GS). 2. To evaluate the concordance rate between AI and radiology residents (RR). 3. To compare the proportion of doubtful results identified by AI and the RR, and the rates confirmed by GS. Methods: We conducted an observational, double-blind, retrospective study on adult bone X-rays (BXRs) referred from the emergency department at our center between October and November 2022, with a final sample of 792 BXRs, categorized into three groups: large joints, small joints, and long-flat bones. Our AI system detects fractures, dislocations, and elbow effusions, providing results as positive, negative, or doubtful. We compared the diagnostic performance of AI and the RR against a senior radiologist (GS). Results: The study population\u2019s median age was 48 years; 48.6% were male. Statistical analysis showed Sens = 90.6% and Spec = 98% for fracture detection by the RR, and 95.8% and 97.6% by AI. The RR achieved higher Sens (77.8%) and Spec (100%) for dislocation detection compared to AI. The Kappa coefficient between RR and AI was 0.797 for fractures in large joints, and concordance was considered acceptable for all other variables. We also analyzed doubtful cases and their confirmation by GS. Additionally, we analyzed findings not detected by AI, such as chronic fractures, arthropathy, focal lesions, and anatomical variants. Conclusions: This study assessed the impact of AI in a real-world clinical setting, comparing its performance with that of radiologists (both in training and senior). AI achieved high Sens, Spec, and AUC in bone fracture detection and showed strong concordance with the RR. In conclusion, AI has the potential to be a valuable screening tool, helping reduce missed diagnoses in clinical practice.<\/jats:p>","DOI":"10.3390\/diagnostics15040491","type":"journal-article","created":{"date-parts":[[2025,2,18]],"date-time":"2025-02-18T03:47:06Z","timestamp":1739850426000},"page":"491","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Diagnostic Performance of an Artificial Intelligence Software for the Evaluation of Bone X-Ray Examinations Referred from the Emergency Department"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-9802-6756","authenticated-orcid":false,"given":"Alejandro","family":"D\u00edaz Moreno","sequence":"first","affiliation":[{"name":"Hospital Universitario QuironSalud Madrid, 28223 Madrid, Spain"},{"name":"Department of Medicine Faculty of Medicine, Health and Sports Universidad Europea de Madrid, 28670 Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Raquel","family":"Cano Alonso","sequence":"additional","affiliation":[{"name":"Hospital Universitario QuironSalud Madrid, 28223 Madrid, Spain"},{"name":"Department of Medicine Faculty of Medicine, Health and Sports Universidad Europea de Madrid, 28670 Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1526-9136","authenticated-orcid":false,"given":"Ana","family":"Fern\u00e1ndez Alfonso","sequence":"additional","affiliation":[{"name":"Hospital Universitario QuironSalud Madrid, 28223 Madrid, Spain"},{"name":"Department of Medicine Faculty of Medicine, Health and Sports Universidad Europea de Madrid, 28670 Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9554-1541","authenticated-orcid":false,"given":"Ana","family":"\u00c1lvarez V\u00e1zquez","sequence":"additional","affiliation":[{"name":"Hospital Universitario QuironSalud Madrid, 28223 Madrid, Spain"},{"name":"Department of Medicine Faculty of Medicine, Health and Sports Universidad Europea de Madrid, 28670 Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-9424-2292","authenticated-orcid":false,"given":"Javier","family":"Carrascoso Arranz","sequence":"additional","affiliation":[{"name":"Hospital Universitario QuironSalud Madrid, 28223 Madrid, Spain"},{"name":"Department of Medicine Faculty of Medicine, Health and Sports Universidad Europea de Madrid, 28670 Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-2474-9858","authenticated-orcid":false,"given":"Julia","family":"L\u00f3pez Alcolea","sequence":"additional","affiliation":[{"name":"Hospital Universitario QuironSalud Madrid, 28223 Madrid, Spain"},{"name":"Department of Medicine Faculty of Medicine, Health and Sports Universidad Europea de Madrid, 28670 Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-4760-5357","authenticated-orcid":false,"given":"David","family":"Garc\u00eda Castellanos","sequence":"additional","affiliation":[{"name":"Hospital Universitario QuironSalud Madrid, 28223 Madrid, Spain"},{"name":"Department of Medicine Faculty of Medicine, Health and Sports Universidad Europea de Madrid, 28670 Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Luc\u00eda","family":"Sanabria Greciano","sequence":"additional","affiliation":[{"name":"Hospital Universitario QuironSalud Madrid, 28223 Madrid, Spain"},{"name":"Department of Medicine Faculty of Medicine, Health and Sports Universidad Europea de Madrid, 28670 Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7082-7869","authenticated-orcid":false,"given":"Manuel","family":"Recio Rodr\u00edguez","sequence":"additional","affiliation":[{"name":"Hospital Universitario QuironSalud Madrid, 28223 Madrid, Spain"},{"name":"Department of Medicine Faculty of Medicine, Health and Sports Universidad Europea de Madrid, 28670 Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Cristina","family":"Andreu-V\u00e1zquez","sequence":"additional","affiliation":[{"name":"Faculty of Biomedical and Health Science, Universidad Europea de Madrid, 28670 Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6990-0619","authenticated-orcid":false,"given":"Israel John","family":"Thuissard Vasallo","sequence":"additional","affiliation":[{"name":"Faculty of Biomedical and Health Science, Universidad Europea de Madrid, 28670 Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Vicente","family":"Mart\u00ednez De Vega","sequence":"additional","affiliation":[{"name":"Hospital Universitario QuironSalud Madrid, 28223 Madrid, Spain"},{"name":"Department of Medicine Faculty of Medicine, Health and Sports Universidad Europea de Madrid, 28670 Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,2,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1148\/radiol.2021203886","article-title":"Assessment of an AI Aid in Detection of Adult Appendicular Skeletal Fractures by Emergency Physicians and Radiologists: A Multicenter Cross-sectional Diagnostic Study","volume":"300","author":"Duron","year":"2021","journal-title":"Radiology"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1080\/13814788.2019.1625886","article-title":"Diagnostic errors reported in primary healthcare and emergency departments: A retrospective and descriptive cohort study of 4830 reported cases of preventable harm in Sweden","volume":"25","author":"Fernholm","year":"2019","journal-title":"Eur. J. Gen. Pract."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1136\/emj.18.4.263","article-title":"Diagnostic errors in an accident and emergency department","volume":"18","author":"Guly","year":"2001","journal-title":"Emerg. Med. J."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1148\/radiol.211785","article-title":"Artificial Intelligence in Fracture Detection: A Systematic Review and Meta-Analysis","volume":"304","author":"Kuo","year":"2022","journal-title":"Radiology"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"A1","DOI":"10.1016\/j.mri.2019.12.006","article-title":"Artificial intelligence in medical imaging","volume":"68","author":"Gore","year":"2020","journal-title":"Magn. Reson. Imaging"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"3797","DOI":"10.1007\/s00330-021-07892-z","article-title":"Artificial intelligence in radiology: 100 commercially available products and their scientific evidence","volume":"31","author":"Schalekamp","year":"2021","journal-title":"Eur. Radiol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1204","DOI":"10.2214\/AJR.19.21346","article-title":"Global trend in artificial intelligence-based publications in radiology from 2000 to 2018","volume":"213","author":"West","year":"2019","journal-title":"Am. J. Roentgenol."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"L\u00f3pez Alcolea, J., Fern\u00e1ndez Alfonso, A., Cano Alonso, R., \u00c1lvarez V\u00e1zquez, A., D\u00edaz Moreno, A., Garc\u00eda Castellanos, D., Greciano, L.S., Hayoun, C., Rodr\u00edguez, M.R., and V\u00e1zquez, C.A. (2024). Diagnostic Performance of Artificial Intelligence in Chest Radiographs Referred from the Emergency Department. Diagnostics, 14.","DOI":"10.3390\/diagnostics14222592"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Cellina, M., C\u00e8, M., Irmici, G., Ascenti, V., Caloro, E., Bianchi, L., Pellegrino, G., D\u2019amico, N., Papa, S., and Carrafiello, G. (2022). Artificial Intelligence in Emergency Radiology: Where Are We Going?. Diagnostics, 12.","DOI":"10.3390\/diagnostics12123223"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1016\/j.diii.2022.07.005","article-title":"Artificial intelligence in emergency radiology: A review of applications and possibilities","volume":"104","author":"Katzman","year":"2023","journal-title":"Diagn. Interv. Imaging"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Debs, P., and Fayad, L.M. (2023). The promise and limitations of artificial intelligence in musculoskeletal imaging. Front. Radiol., 3.","DOI":"10.3389\/fradi.2023.1242902"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2129","DOI":"10.1007\/s00256-022-04070-0","article-title":"Automated detection of acute appendicular skeletal fractures in pediatric patients using deep learning","volume":"51","author":"Hayashi","year":"2022","journal-title":"Skelet. Radiol."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"869","DOI":"10.2214\/AJR.22.27873","article-title":"Artificial Intelligence (AI) for Fracture Diagnosis: An Overview of Current Products and Considerations for Clinical Adoption, From the AJR Special Series on AI Applications","volume":"219","author":"Zech","year":"2022","journal-title":"Am. J. Roentgenol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"110447","DOI":"10.1016\/j.ejrad.2022.110447","article-title":"Assessment of performances of a deep learning algorithm for the detection of limbs and pelvic fractures, dislocations, focal bone lesions, and elbow effusions on trauma X-rays","volume":"154","author":"Regnard","year":"2022","journal-title":"Eur. J. Radiol."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"e24.00106","DOI":"10.2106\/JBJS.RVW.24.00106","article-title":"The Accuracy of Artificial Intelligence Models in Hand\/Wrist Fracture and Dislocation Diagnosis","volume":"12","author":"Wong","year":"2024","journal-title":"JBJS Rev."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Zhao, Z., Pi, Y., Jiang, L., Xiang, Y., Wei, J., Yang, P., Zhang, W., Zhong, X., Zhou, K., and Li, Y. (2020). Deep neural network based artificial intelligence assisted diagnosis of bone scintigraphy for cancer bone metastasis. Sci. Rep., 10.","DOI":"10.1038\/s41598-020-74135-4"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Huhtanen, J.T., Nyman, M., Doncenco, D., Hamedian, M., Kawalya, D., Salminen, L., Sequeiros, R.B., Koskinen, S.K., Pudas, T.K., and Kajander, S. (2022). Deep learning accurately classifies elbow joint effusion in adult and pediatric radiographs. Sci. Rep., 12.","DOI":"10.1038\/s41598-022-16154-x"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1361","DOI":"10.2214\/AJR.18.19974","article-title":"Detection of Traumatic Pediatric Elbow Joint Effusion Using a Deep Convolutional Neural Network","volume":"211","author":"England","year":"2018","journal-title":"Am. J. Roentgenol."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Wang, X., Zhou, B., Gong, P., Zhang, T., Mo, Y., Tang, J., Shi, X., Wang, J., Yuan, X., and Bai, F. (2022). Artificial Intelligence\u2013Assisted Bone Age Assessment to Improve the Accuracy and Consistency of Physicians With Different Levels of Experience. Front. Pediatr., 10.","DOI":"10.3389\/fped.2022.818061"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"3664","DOI":"10.1007\/s12325-024-02944-4","article-title":"Validation of an AI-Powered Automated X-ray Bone Age Analyzer in Chinese Children and Adolescents: A Comparison with the Tanner\u2013Whitehouse 3 Method","volume":"41","author":"Liang","year":"2024","journal-title":"Adv. Ther."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Yang, J., Wang, J., and Meng, M.Q.-H. (2024). A Landmark-aware Network for Automated Cobb Angle Estimation Using X-ray Images. arXiv.","DOI":"10.2139\/ssrn.5039753"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"e50631","DOI":"10.2196\/50631","article-title":"Use of Artificial Intelligence in Cobb Angle Measurement for Scoliosis: Retrospective Reliability and Accuracy Study of a Mobile App","volume":"26","author":"Li","year":"2024","journal-title":"J. Med. Internet Res."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1007\/s00256-021-03909-2","article-title":"AI MSK clinical applications: Cartilage and osteoarthritis","volume":"51","author":"Joseph","year":"2022","journal-title":"Skelet. Radiol."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"20170344","DOI":"10.1259\/dmfr.20170344","article-title":"Osteoporosis detection in panoramic radiographs using a deep convolutional neural network-based computer-assisted diagnosis system: A preliminary study","volume":"48","author":"Lee","year":"2019","journal-title":"Dentomaxillofacial Radiol."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Meetschen, M., Salh\u00f6fer, L., Beck, N., Kroll, L., Ziegenfu\u00df, C.D., Schaarschmidt, B.M., Forsting, M., Mizan, S., Umutlu, L., and Hosch, R. (2024). AI-Assisted X-ray Fracture Detection in Residency Training: Evaluation in Pediatric and Adult Trauma Patients. Diagnostics, 14.","DOI":"10.3390\/diagnostics14060596"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"627","DOI":"10.1148\/radiol.210937","article-title":"Improving Radiographic Fracture Recognition Performance and Efficiency Using Artificial Intelligence","volume":"302","author":"Guermazi","year":"2022","journal-title":"Radiology"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Oppenheimer, J., L\u00fcken, S., Hamm, B., and Niehues, S.M. (2023). A Prospective Approach to Integration of AI Fracture Detection Software in Radiographs into Clinical Workflow. Life, 13.","DOI":"10.3390\/life13010223"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1007\/s00256-012-1503-5","article-title":"Visual expertise in detecting and diagnosing skeletal fractures","volume":"42","author":"Wood","year":"2013","journal-title":"Skelet. Radiol."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"111637","DOI":"10.1016\/j.ejrad.2024.111637","article-title":"Diagnostic performance of an AI algorithm for the detection of appendicular bone fractures in pediatric patients","volume":"178","author":"Franco","year":"2024","journal-title":"Eur. J. Radiol."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1930","DOI":"10.21037\/qims-23-878","article-title":"Artificial intelligence diagnostic model for multi-site fracture X-ray images of extremities based on deep convolutional neural networks","volume":"14","author":"Xie","year":"2024","journal-title":"Quant. Imaging Med. Surg."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"104","DOI":"10.1016\/j.diii.2023.09.008","article-title":"External validation of an artificial intelligence solution for the detection of elbow fractures and joint effusions in children","volume":"105","author":"Dupuis","year":"2024","journal-title":"Diagn. Interv. Imaging"}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/15\/4\/491\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T16:36:33Z","timestamp":1760027793000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/15\/4\/491"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,18]]},"references-count":31,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2025,2]]}},"alternative-id":["diagnostics15040491"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics15040491","relation":{},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,2,18]]}}}