{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T16:37:27Z","timestamp":1791909447904,"version":"4.3.4"},"reference-count":29,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2022,8,28]],"date-time":"2022-08-28T00:00:00Z","timestamp":1661644800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>Purpose: We assessed whether a CXR AI algorithm was able to detect missed or mislabeled chest radiograph (CXR) findings in radiology reports. Methods: We queried a multi-institutional radiology reports search database of 13 million reports to identify all CXR reports with addendums from 1999\u20132021. Of the 3469 CXR reports with an addendum, a thoracic radiologist excluded reports where addenda were created for typographic errors, wrong report template, missing sections, or uninterpreted signoffs. The remaining reports contained addenda (279 patients) with errors related to side-discrepancies or missed findings such as pulmonary nodules, consolidation, pleural effusions, pneumothorax, and rib fractures. All CXRs were processed with an AI algorithm. Descriptive statistics were performed to determine the sensitivity, specificity, and accuracy of the AI in detecting missed or mislabeled findings. Results: The AI had high sensitivity (96%), specificity (100%), and accuracy (96%) for detecting all missed and mislabeled CXR findings. The corresponding finding-specific statistics for the AI were nodules (96%, 100%, 96%), pneumothorax (84%, 100%, 85%), pleural effusion (100%, 17%, 67%), consolidation (98%, 100%, 98%), and rib fractures (87%, 100%, 94%). Conclusions: The CXR AI could accurately detect mislabeled and missed findings. Clinical Relevance: The CXR AI can reduce the frequency of errors in detection and side-labeling of radiographic findings.<\/jats:p>","DOI":"10.3390\/diagnostics12092086","type":"journal-article","created":{"date-parts":[[2022,8,29]],"date-time":"2022-08-29T01:39:19Z","timestamp":1661737159000},"page":"2086","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":20,"title":["Performance of a Chest Radiography AI Algorithm for Detection of Missed or Mislabeled Findings: A Multicenter Study"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0911-7507","authenticated-orcid":false,"given":"Parisa","family":"Kaviani","sequence":"first","affiliation":[{"name":"Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA 02114, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Subba R.","family":"Digumarthy","sequence":"additional","affiliation":[{"name":"Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA 02114, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bernardo C.","family":"Bizzo","sequence":"additional","affiliation":[{"name":"Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA 02114, USA"},{"name":"MGH & BWH Center for Clinical Data Science, Boston, MA 02114, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7643-7346","authenticated-orcid":false,"given":"Bhargava","family":"Reddy","sequence":"additional","affiliation":[{"name":"Qure.ai, Mumbai 400063, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Manoj","family":"Tadepalli","sequence":"additional","affiliation":[{"name":"Qure.ai, Mumbai 400063, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Preetham","family":"Putha","sequence":"additional","affiliation":[{"name":"Qure.ai, Mumbai 400063, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ammar","family":"Jagirdar","sequence":"additional","affiliation":[{"name":"Qure.ai, Mumbai 400063, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shadi","family":"Ebrahimian","sequence":"additional","affiliation":[{"name":"Internal Medicine, Icahn School of Medicine at Mount Sinai, Elmhurst Hospital Center, Elmhurst, NY 11373, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mannudeep K.","family":"Kalra","sequence":"additional","affiliation":[{"name":"Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA 02114, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Keith J.","family":"Dreyer","sequence":"additional","affiliation":[{"name":"Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA 02114, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,8,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"20150023","DOI":"10.1259\/bjr.20150023","article-title":"Radiographers\u2019 performance in chest X-ray interpretation: The Nigerian experience","volume":"88","author":"Ekpo","year":"2015","journal-title":"Br. J. Radiol."},{"key":"ref_2","first-page":"574","article-title":"Chest radiography in general practice: Indications, diagnostic yield and consequences for patient management","volume":"56","author":"Speets","year":"2006","journal-title":"Br. J. Gen. Pract."},{"key":"ref_3","first-page":"485","article-title":"Radiologic errors in patients with lung cancer","volume":"134","author":"Forrest","year":"1981","journal-title":"West J. Med."},{"key":"ref_4","first-page":"143","article-title":"The chest radiograph","volume":"81","author":"Kelly","year":"2012","journal-title":"Ulster Med. J."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1818","DOI":"10.1007\/s00330-008-0948-3","article-title":"Digital chest radiography: An update on modern technology, dose containment and control of image quality","volume":"18","author":"Neitzel","year":"2008","journal-title":"Eur. Radiol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"349","DOI":"10.7861\/clinmedicine.13-4-349","article-title":"Assessing the accuracy and certainty in interpreting chest X-rays in the medical division","volume":"13","author":"Satia","year":"2013","journal-title":"Clin. Med."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"S253","DOI":"10.1093\/cid\/cix082","article-title":"Standardized Interpretation of Chest Radiographs in Cases of Pediatric Pneumonia From the PERCH Study","volume":"64","author":"Fancourt","year":"2017","journal-title":"Clin. Infect. Dis."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1148\/radiology.192.1.8208934","article-title":"Reporting the \u201cmissed\u201d radiologic diagnosis: Medicolegal and ethical considerations","volume":"192","author":"Berlin","year":"1994","journal-title":"Radiology"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"720","DOI":"10.1378\/chest.115.3.720","article-title":"Miss rate of lung cancer on the chest radiograph in clinical practice","volume":"115","author":"Quekel","year":"1999","journal-title":"Chest"},{"key":"ref_10","unstructured":"Institute of Medicine (US) Committee on Quality of Health Care in America, Kohn, L.T., Corrigan, J.M., and Donaldson, M.S. (2000). To Err Is Human: Building a Safer Health System, National Academies Press."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Ebrahimian, S., Kalra, M.K., Agarwal, S., Bizzo, B.C., Elkholy, M., Wald, C., Allen, B., and Dreyer, K.J. (Acad. Radiol., 2021). FDA-regulated AI algorithms: Trends, strengths, and gaps of validation studies, Acad. Radiol., in press.","DOI":"10.1016\/j.acra.2021.09.002"},{"key":"ref_12","first-page":"4851","article-title":"Attention-Guided Convolutional Neural Network for Detecting Pneumonia on Chest X-rays","volume":"2019","author":"Li","year":"2019","journal-title":"Annu. Int. Conf. IEEE Eng. Med. Biol. Soc."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"e2022779","DOI":"10.1001\/jamanetworkopen.2020.22779","article-title":"Comparison of Chest Radiograph Interpretations by Artificial Intelligence Algorithm vs Radiology Residents","volume":"3","author":"Wu","year":"2020","journal-title":"JAMA Netw. Open"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"101744","DOI":"10.1016\/j.artmed.2019.101744","article-title":"Multi-resolution convolutional networks for chest X-ray radiograph based lung nodule detection","volume":"103","author":"Li","year":"2020","journal-title":"Artif. Intell. Med."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"490","DOI":"10.7150\/ijms.69400","article-title":"Deep Learning-based Artificial Intelligence Improves Accuracy of Error-prone Lung Nodules","volume":"19","author":"Lan","year":"2022","journal-title":"Int. J. Med. Sci."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"327","DOI":"10.2174\/1573405617666210806125953","article-title":"Lung nodule detectability of artificial intelligence-assisted CT image reading in lung cancer screening","volume":"18","author":"Zhang","year":"2022","journal-title":"Curr. Med. Imaging"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1097\/RLI.0000000000000813","article-title":"Artificial Intelligence in Chest Radiography Reporting Accuracy: Added Clinical Value in the Emergency Unit Setting Without 24\/7 Radiology Coverage","volume":"57","author":"Rudolph","year":"2022","journal-title":"Investig. Radiol."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"890759","DOI":"10.3389\/fdgth.2022.890759","article-title":"Deployment and validation of an AI system for detecting abnormal chest radiographs in clinical settings","volume":"4","author":"Nguyen","year":"2022","journal-title":"Front. Digit. Health"},{"key":"ref_19","first-page":"20584601221107345","article-title":"Observer performance evaluation of the feasibility of a deep learning model to detect cardiomegaly on chest radiographs","volume":"11","author":"Ajmera","year":"2022","journal-title":"Acta Radiol. Open"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"e2141096","DOI":"10.1001\/jamanetworkopen.2021.41096","article-title":"An Artificial Intelligence\u2013Based Chest X-ray Model on Human Nodule Detection Accuracy From a Multicenter Study","volume":"4","author":"Homayounieh","year":"2021","journal-title":"JAMA Netw. Open"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Engle, E., Gabrielian, A., Long, A., Hurt, D.E., and Rosenthal, A. (2020). Performance of Qure. ai automatic classifiers against a large annotated database of patients with diverse forms of tuberculosis. PLoS ONE, 15.","DOI":"10.1371\/journal.pone.0224445"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"e2017135","DOI":"10.1001\/jamanetworkopen.2020.17135","article-title":"Validation of a deep learning algorithm for the detection of malignant pulmonary nodules in chest radiographs","volume":"3","author":"Yoo","year":"2020","journal-title":"JAMA Netw. Open"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"105162","DOI":"10.1016\/j.cmpb.2019.105162","article-title":"Deep learning, reusable and problem-based architectures for detection of consolidation on chest X-ray images","volume":"185","author":"Rostami","year":"2020","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1845","DOI":"10.1148\/rg.2018180021","article-title":"Fundamentals of diagnostic error in imaging","volume":"38","author":"Itri","year":"2018","journal-title":"Radiographics"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"e200190","DOI":"10.1148\/ryai.2021200190","article-title":"Deep Learning Systems for Pneumothorax Detection on Chest Radiographs: A Multicenter External Validation Study","volume":"3","author":"Thian","year":"2021","journal-title":"Radiol. Artif. Intell."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1257","DOI":"10.1007\/s13246-021-01060-9","article-title":"AI-based diagnosis of COVID-19 patients using X-ray scans with stochastic ensemble of CNNs","volume":"44","author":"Arora","year":"2021","journal-title":"Phys. Eng. Sci. Med."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41598-021-93967-2","article-title":"Deep learning for distinguishing normal versus abnormal chest radiographs and generalization to two unseen diseases tuberculosis and COVID-19","volume":"11","author":"Nabulsi","year":"2021","journal-title":"Sci. Rep."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"3837","DOI":"10.1007\/s00330-020-07480-7","article-title":"Smart chest X-ray worklist prioritization using artificial intelligence: A clinical workflow simulation","volume":"31","author":"Baltruschat","year":"2021","journal-title":"Eur. Radiol."},{"key":"ref_29","first-page":"e200024","article-title":"Active reprioritization of the reading worklist using artificial intelligence has a beneficial effect on the turnaround time for interpretation of head CT with intracranial hemorrhage","volume":"3","author":"Xi","year":"2020","journal-title":"Radiol. Artif. Intell."}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/12\/9\/2086\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:19:15Z","timestamp":1760141955000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/12\/9\/2086"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,28]]},"references-count":29,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2022,9]]}},"alternative-id":["diagnostics12092086"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics12092086","relation":{"has-preprint":[{"id-type":"doi","id":"10.20944\/preprints202208.0189.v1","asserted-by":"object"}]},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,8,28]]}}}