{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,20]],"date-time":"2026-08-20T16:17:50Z","timestamp":1787242670481,"version":"build-2736575974"},"reference-count":18,"publisher":"Public Library of Science (PLoS)","issue":"2","license":[{"start":{"date-parts":[[2021,2,19]],"date-time":"2021-02-19T00:00:00Z","timestamp":1613692800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003710","name":"Korea Health Industry Development Institute","doi-asserted-by":"crossref","award":["HI19C0847"],"award-info":[{"award-number":["HI19C0847"]}],"id":[{"id":"10.13039\/501100003710","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS ONE"],"abstract":"\n Purpose<\/jats:title>\n This study evaluated the performance of a commercially available deep-learning algorithm (DLA) (Insight CXR, Lunit, Seoul, South Korea) for referable thoracic abnormalities on chest X-ray (CXR) using a consecutively collected multicenter health screening cohort.<\/jats:p>\n <\/jats:sec>\n \n Methods and materials<\/jats:title>\n A consecutive health screening cohort of participants who underwent both CXR and chest computed tomography (CT) within 1 month was retrospectively collected from three institutions\u2019 health care clinics (n = 5,887). Referable thoracic abnormalities were defined as any radiologic findings requiring further diagnostic evaluation or management, including DLA-target lesions of nodule\/mass, consolidation, or pneumothorax. We evaluated the diagnostic performance of the DLA for referable thoracic abnormalities using the area under the receiver operating characteristic (ROC) curve (AUC), sensitivity, and specificity using ground truth based on chest CT (CT-GT). In addition, for CT-GT-positive cases, three independent radiologist readings were performed on CXR and clear visible (when more than two radiologists called) and visible (at least one radiologist called) abnormalities were defined as CXR-GTs (clear visible CXR-GT and visible CXR-GT, respectively) to evaluate the performance of the DLA.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n Among 5,887 subjects (4,329 males; mean age 54\u00b111 years), referable thoracic abnormalities were found in 618 (10.5%) based on CT-GT. DLA-target lesions were observed in 223 (4.0%), nodule\/mass in 202 (3.4%), consolidation in 31 (0.5%), pneumothorax in one 1 (<0.1%), and DLA-non-target lesions in 409 (6.9%). For referable thoracic abnormalities based on CT-GT, the DLA showed an AUC of 0.771 (95% confidence interval [CI], 0.751\u20130.791), a sensitivity of 69.6%, and a specificity of 74.0%. Based on CXR-GT, the prevalence of referable thoracic abnormalities decreased, with visible and clear visible abnormalities found in 405 (6.9%) and 227 (3.9%) cases, respectively. The performance of the DLA increased significantly when using CXR-GTs, with an AUC of 0.839 (95% CI, 0.829\u20130.848), a sensitivity of 82.7%, and s specificity of 73.2% based on visible CXR-GT and an AUC of 0.872 (95% CI, 0.863\u20130.880,\n P<\/jats:italic>\n <0.001 for the AUC comparison of GT-CT vs. clear visible CXR-GT), a sensitivity of 83.3%, and a specificity of 78.8% based on clear visible CXR-GT.\n <\/jats:p>\n <\/jats:sec>\n \n Conclusion<\/jats:title>\n The DLA provided fair-to-good stand-alone performance for the detection of referable thoracic abnormalities in a multicenter consecutive health screening cohort. The DLA showed varied performance according to the different methods of ground truth.<\/jats:p>\n <\/jats:sec>","DOI":"10.1371\/journal.pone.0246472","type":"journal-article","created":{"date-parts":[[2021,2,19]],"date-time":"2021-02-19T14:57:16Z","timestamp":1613746636000},"page":"e0246472","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":20,"title":["Performance of a deep-learning algorithm for referable thoracic abnormalities on chest radiographs: A multicenter study of a health screening cohort"],"prefix":"10.1371","volume":"16","author":[{"given":"Eun Young","family":"Kim","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Young Jae","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8096-7542","authenticated-orcid":true,"given":"Won-Jun","family":"Choi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9785-3776","authenticated-orcid":true,"given":"Gi Pyo","family":"Lee","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ye Ra","family":"Choi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5494-9113","authenticated-orcid":true,"given":"Kwang Nam","family":"Jin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Young Jun","family":"Cho","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2021,2,19]]},"reference":[{"issue":"2","key":"pone.0246472.ref001","doi-asserted-by":"crossref","first-page":"575","DOI":"10.1148\/radiol.2332031796","article-title":"Routine chest radiography in a primary care setting","volume":"233","author":"S Tigges","year":"2004","journal-title":"Radiology"},{"issue":"1","key":"pone.0246472.ref002","doi-asserted-by":"crossref","first-page":"138","DOI":"10.1001\/jamainternmed.2015.7110","article-title":"Korean National Health Insurance Database","volume":"176","author":"DW Shin","year":"2016","journal-title":"JAMA Intern Med"},{"issue":"2","key":"pone.0246472.ref003","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1111\/j.1754-9485.2012.02348.x","article-title":"Common patterns in 558 diagnostic radiology errors","volume":"56","author":"JJ Donald","year":"2012","journal-title":"J Med Imaging Radiat Oncol"},{"issue":"10","key":"pone.0246472.ref004","doi-asserted-by":"crossref","first-page":"4202","DOI":"10.1166\/jctn.2019.8501","article-title":"Computer Aided Diagnosis of Pneumonia from Chest Radiographs","volume":"16","author":"P Malhotra","year":"2019","journal-title":"Journal of Computational and Theoretical Nanoscience"},{"issue":"8","key":"pone.0246472.ref005","doi-asserted-by":"crossref","first-page":"555","DOI":"10.1016\/j.ijmedinf.2007.10.010","article-title":"Computer-aided diagnosis in chest radiography for detection of childhood pneumonia","volume":"77","author":"LL Oliveira","year":"2008","journal-title":"Int J Med Inform"},{"key":"pone.0246472.ref006","volume-title":"Detection of Pneumonia from X-Ray Images using Convolutional Neural Network","author":"H Omar","year":"2019"},{"issue":"3","key":"pone.0246472.ref007","doi-asserted-by":"crossref","first-page":"e191095","DOI":"10.1001\/jamanetworkopen.2019.1095","article-title":"Development and Validation of a Deep Learning-Based Automated Detection Algorithm for Major Thoracic Diseases on Chest Radiographs","volume":"2","author":"EJ Hwang","year":"2019","journal-title":"JAMA Netw Open"},{"issue":"11","key":"pone.0246472.ref008","doi-asserted-by":"crossref","first-page":"e1002686","DOI":"10.1371\/journal.pmed.1002686","article-title":"Deep learning for chest radiograph diagnosis: A retrospective comparison of the CheXNeXt algorithm to practicing radiologists","volume":"15","author":"P Rajpurkar","year":"2018","journal-title":"PLoS Med"},{"key":"pone.0246472.ref009","first-page":"1278","article-title":"Deep Learning Based Lung Region Segmentation with Data Preprocessing by Generative Adversarial Nets","volume":"2020","author":"J Nitta","year":"2020","journal-title":"Annu Int Conf IEEE Eng Med Biol Soc"},{"key":"pone.0246472.ref010","first-page":"1246","article-title":"Lung Region Segmentation in Chest X-Ray Images using Deep Convolutional Neural Networks","volume":"2020","author":"RDS Portela","year":"2020","journal-title":"Annu Int Conf IEEE Eng Med Biol Soc"},{"issue":"1","key":"pone.0246472.ref011","doi-asserted-by":"crossref","first-page":"218","DOI":"10.1148\/radiol.2018180237","article-title":"Development and Validation of Deep Learning-based Automatic Detection Algorithm for Malignant Pulmonary Nodules on Chest Radiographs","volume":"290","author":"JG Nam","year":"2019","journal-title":"Radiology"},{"issue":"1","key":"pone.0246472.ref012","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1038\/s41597-019-0322-0","article-title":"MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports","volume":"6","author":"AEW Johnson","year":"2019","journal-title":"Sci Data"},{"issue":"3","key":"pone.0246472.ref013","doi-asserted-by":"crossref","first-page":"697","DOI":"10.1148\/radiol.2462070712","article-title":"Fleischner Society: glossary of terms for thoracic imaging","volume":"246","author":"DM Hansell","year":"2008","journal-title":"Radiology"},{"issue":"4","key":"pone.0246472.ref014","first-page":"204","article-title":"Strengthening of Epidemiological and Statistical Services Unit","volume":"43","author":"The international conference for the tenth revision of the International Classification of Diseases","year":"1990","journal-title":"World Health Organization, Geneva. World Health Stat Q"},{"issue":"5","key":"pone.0246472.ref015","doi-asserted-by":"crossref","first-page":"739","DOI":"10.1093\/cid\/ciy967","article-title":"Development and Validation of a Deep Learning-based Automatic Detection Algorithm for Active Pulmonary Tuberculosis on Chest Radiographs","volume":"69","author":"EJ Hwang","year":"2019","journal-title":"Clin Infect Dis"},{"key":"pone.0246472.ref016","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2016; 2016. p. 770\u2013778.","DOI":"10.1109\/CVPR.2016.90"},{"key":"pone.0246472.ref017","article-title":"Learning Visual Context by Comparison","author":"M Kim","year":"2020","journal-title":"arXiv preprint arXiv"},{"key":"pone.0246472.ref018","doi-asserted-by":"crossref","unstructured":"Cubuk ED, Zoph B, Mane D, Vasudevan V, Le QV. Autoaugment: Learning augmentation strategies from data. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2019; 2019. p. 113\u2013123.","DOI":"10.1109\/CVPR.2019.00020"}],"updated-by":[{"DOI":"10.1371\/journal.pone.0251045","type":"correction","label":"Correction","source":"publisher","updated":{"date-parts":[[2021,4,28]],"date-time":"2021-04-28T00:00:00Z","timestamp":1619568000000}}],"container-title":["PLOS ONE"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0246472","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,2,19]],"date-time":"2021-02-19T14:57:38Z","timestamp":1613746658000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0246472"}},"subtitle":[],"editor":[{"given":"Pierpaolo","family":"Alongi","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2021,2,19]]},"references-count":18,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2021,2,19]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0246472","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,2,19]]}}}