{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,3]],"date-time":"2026-10-03T17:57:15Z","timestamp":1791050235216,"version":"4.1.0"},"reference-count":24,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2020,10,15]],"date-time":"2020-10-15T00:00:00Z","timestamp":1602720000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,10,15]],"date-time":"2020-10-15T00:00:00Z","timestamp":1602720000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We evaluated the reproducibility of computer-aided detections (CADs) with a convolutional neural network (CNN) on chest radiographs (CXRs) of abnormal pulmonary patterns in patients, acquired within a short-term interval. Anonymized CXRs (n\u2009=\u20099792) obtained from 2010 to 2016 and comprising five types of disease patterns, including the nodule (N), consolidation (C), interstitial opacity (IO), pleural effusion (PLE), and pneumothorax (PN), were included. The number of normal and abnormal CXRs was 6068 and 3724, respectively. The number of CXRs (region of interests, ROIs) of N, C, IO, PLE, and PN was 944 (1092), 550 (721), 280 (538), 1361 (1661), and 589 (622), respectively. CXRs were randomly allocated to training, tuning, and test sets in 70:10:20 ratios. Two thoracic radiologists labeled and delineated the ROIs of each disease pattern. The CAD system was developed using eDenseYOLO. For the reproducibility evaluation of developed CAD, paired CXRs of various diseases (N\u2009=\u2009121, C\u2009=\u200928, IO\u2009=\u200912, PLE\u2009=\u200967, and PN\u2009=\u200920), acquired within a short-term interval from the test sets without any changes confirmed by thoracic radiologists, were used to evaluate CAD reproducibility. Percent positive agreement (PPAs) and Chamberlain\u2019s percent positive agreement (CPPAs) were used to evaluate CAD reproducibility. The figure of merit (FOM) of five classes based on eDenseYOLO showed N-0.72 (0.68\u20130.75), C-0.41 (0.33\u20130.43), IO-0.97 (0.96\u20130.98), PLE-0.94 (0.92\u201395), and PN-0.87 (0.76\u20130.93). The PPAs of the five disease patterns including N, C, IO, PLE, and PN were 83.39%, 74.14%, 95.12%, 96.84%, and 84.58%, respectively, whereas the values of CPPAs were 71.70%, 59.13%, 91.16%, 93.91%, and 74.17%, respectively. The reproducibility of abnormal pulmonary patterns from CXRs, based on deep learning-based CAD, showed different results; this is important for assessing the reproducible performance of CAD in clinical settings.<\/jats:p>","DOI":"10.1038\/s41598-020-74626-4","type":"journal-article","created":{"date-parts":[[2020,10,15]],"date-time":"2020-10-15T10:04:27Z","timestamp":1602756267000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Reproducibility of abnormality detection on chest radiographs using convolutional neural network in paired radiographs obtained within a short-term interval"],"prefix":"10.1038","volume":"10","author":[{"given":"Yongwon","family":"Cho","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Young-Gon","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sang Min","family":"Lee","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joon Beom","family":"Seo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Namkug","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2020,10,15]]},"reference":[{"issue":"1","key":"74626_CR1","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1148\/radiology.182.1.1727272","volume":"182","author":"JH Austin","year":"1992","unstructured":"Austin, J. H., Romney, B. M. & Goldsmith, L. S. Missed bronchogenic carcinoma: radiographic findings in 27 patients with a potentially resectable lesion evident in retrospect. Radiology 182(1), 115\u2013122 (1992).","journal-title":"Radiology"},{"issue":"3","key":"74626_CR2","doi-asserted-by":"publisher","first-page":"720","DOI":"10.1378\/chest.115.3.720","volume":"115","author":"LG Quekel","year":"1999","unstructured":"Quekel, L. G., Kessels, A. G., Goei, R. & van Engelshoven, J. M. Miss rate of lung cancer on the chest radiograph in clinical practice. Chest 115(3), 720\u2013724 (1999).","journal-title":"Chest"},{"issue":"4","key":"74626_CR3","doi-asserted-by":"publisher","first-page":"597","DOI":"10.1007\/s003300000595","volume":"11","author":"L Monnier-Cholley","year":"2001","unstructured":"Monnier-Cholley, L. et al. Characteristics of missed lung cancer on chest radiographs: a French experience. Eur. Radiol. 11(4), 597\u2013605 (2001).","journal-title":"Eur. Radiol."},{"issue":"1","key":"74626_CR4","doi-asserted-by":"publisher","first-page":"273","DOI":"10.1148\/radiol.2461061848","volume":"246","author":"F Li","year":"2008","unstructured":"Li, F., Engelmann, R., Metz, C. E., Doi, K. & MacMahon, H. Lung cancers missed on chest radiographs: results obtained with a commercial computer-aided detection program. Radiology 246(1), 273\u2013280 (2008).","journal-title":"Radiology"},{"issue":"1","key":"74626_CR5","doi-asserted-by":"publisher","first-page":"273","DOI":"10.1148\/radiol.2522081319","volume":"252","author":"CS White","year":"2009","unstructured":"White, C. S., Flukinger, T., Jeudy, J. & Chen, J. J. Use of a computer-aided detection system to detect missed lung cancer at chest radiography. Radiology 252(1), 273\u2013281 (2009).","journal-title":"Radiology"},{"issue":"1","key":"74626_CR6","doi-asserted-by":"publisher","first-page":"252","DOI":"10.1148\/radiol.14131315","volume":"272","author":"S Schalekamp","year":"2014","unstructured":"Schalekamp, S. et al. Computer-aided detection improves detection of pulmonary nodules in chest radiographs beyond the support by bone-suppressed images. Radiology 272(1), 252\u2013261 (2014).","journal-title":"Radiology"},{"issue":"2","key":"74626_CR7","doi-asserted-by":"publisher","first-page":"532","DOI":"10.1148\/radiol.10092437","volume":"257","author":"B de Hoop","year":"2010","unstructured":"de Hoop, B. et al. Computer-aided detection of lung cancer on chest radiographs: effect on observer performance. Radiology 257(2), 532\u2013540 (2010).","journal-title":"Radiology"},{"issue":"4","key":"74626_CR8","doi-asserted-by":"publisher","first-page":"475","DOI":"10.1016\/j.acra.2014.11.008","volume":"22","author":"F Li","year":"2015","unstructured":"Li, F., Engelmann, R., Armato, S. G. III. & MacMahon, H. Computer-aided nodule detection system: results in an unselected series of consecutive chest radiographs. Acad. Radiol. 22(4), 475\u2013480 (2015).","journal-title":"Acad. Radiol."},{"issue":"2","key":"74626_CR9","doi-asserted-by":"publisher","first-page":"574","DOI":"10.1148\/radiol.2017162326","volume":"284","author":"P Lakhani","year":"2017","unstructured":"Lakhani, P. & Sundaram, B. Deep learning at chest radiography: automated classification of pulmonary tuberculosis by using convolutional neural networks. Radiology 284(2), 574\u2013582 (2017).","journal-title":"Radiology"},{"issue":"2","key":"74626_CR10","doi-asserted-by":"publisher","first-page":"233","DOI":"10.1109\/TMI.2013.2284099","volume":"33","author":"S Jaeger","year":"2014","unstructured":"Jaeger, S. et al. Automatic tuberculosis screening using chest radiographs. IEEE Trans. Med. Imaging 33(2), 233\u2013245 (2014).","journal-title":"IEEE Trans. Med. Imaging"},{"key":"74626_CR11","unstructured":"Mohammad, T. I., Abdul, A., Ahmed, T. M. & Khalid, A. Abnormality detection and localization in chest X-rays using deep convolutional neural networks. arXiv:1705.098503v3 (2017)."},{"key":"74626_CR12","doi-asserted-by":"publisher","first-page":"281","DOI":"10.1097\/RLI.0000000000000341","volume":"52","author":"M Ciero","year":"2017","unstructured":"Ciero, M., Bilbily, A. & Colak, E. Training and validating a deep convolutional neural network for computer-aided detection and classification of abnormalities on frontal chest radiographs. Investig. Radiol. 52, 281\u2013287 (2017).","journal-title":"Investig. Radiol."},{"issue":"5","key":"74626_CR13","doi-asserted-by":"publisher","first-page":"1160","DOI":"10.1109\/TMI.2016.2536809","volume":"35","author":"A Setio","year":"2016","unstructured":"Setio, A. et al. Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks. IEEE Trans. Med. Imaging 35(5), 1160\u20131169 (2016).","journal-title":"IEEE Trans. Med. Imaging"},{"key":"74626_CR14","first-page":"1","volume":"17","author":"H Shin","year":"2016","unstructured":"Shin, H. et al. Interleaved text\/image deep mining on a large-scale radiology database for automated image interpretation. J. Mach. Learn. Res. 17, 1\u201331 (2016).","journal-title":"J. Mach. Learn. Res."},{"issue":"10","key":"74626_CR15","doi-asserted-by":"publisher","first-page":"e0204155","DOI":"10.1371\/journal.pone.0204155","volume":"13","author":"R Singh","year":"2018","unstructured":"Singh, R. et al. Deep learning in chest radiography: detection of findings and presence of change. PLoS ONE 13(10), e0204155 (2018).","journal-title":"PLoS ONE"},{"key":"74626_CR16","doi-asserted-by":"publisher","first-page":"18738","DOI":"10.1038\/s41598-019-55373-7","volume":"9","author":"Y-G Kim","year":"2019","unstructured":"Kim, Y.-G. et al. Short-term reproducibility of pulmonary nodule and mass detection in chest radiographs: comparison among radiologists and four different computer-aided detections with convolutional neural net. Sci. Rep. 9, 18738 (2019).","journal-title":"Sci. Rep."},{"key":"74626_CR17","unstructured":"Joseph, R. & Ali, F. YOLO9000: better, faster, stronger. arXiv:1612.08242 (2016)."},{"key":"74626_CR18","doi-asserted-by":"publisher","first-page":"4943","DOI":"10.1007\/s00330-020-06892-9","volume":"30","author":"Y-G Kim","year":"2020","unstructured":"Kim, Y.-G. et al. Optimal matrix size of chest radiographs for computer-aided detection on lung nodule or mass with deep learning. Eur. Radiol. 30, 4943\u20134951 (2020).","journal-title":"Eur. Radiol."},{"key":"74626_CR19","doi-asserted-by":"publisher","first-page":"466","DOI":"10.1002\/uog.5256","volume":"31","author":"JW Bartlett","year":"2008","unstructured":"Bartlett, J. W. & Frost, C. Reliability, repeatability, and reproducibility: analysis of measurement errors in continuous variables. Ultrasound. Obstet. Gynecol. 31, 466\u2013475 (2008).","journal-title":"Ultrasound. Obstet. Gynecol."},{"issue":"1","key":"74626_CR20","doi-asserted-by":"publisher","first-page":"9","DOI":"10.12771\/emj.2017.40.1.9","volume":"40","author":"KA Kong","year":"2017","unstructured":"Kong, K. A. Statistical methods: reliability assessment and method comparison. Ewha Med. J. 40(1), 9\u201316 (2017).","journal-title":"Ewha Med. J."},{"issue":"1","key":"74626_CR21","doi-asserted-by":"publisher","first-page":"15352","DOI":"10.1038\/s41598-019-51832-3","volume":"9","author":"B Park","year":"2019","unstructured":"Park, B. et al. A curriculum learning strategy to enhance the accuracy of classification of various lesions in chest-PA X-ray screening for pulmonary abnormalities. Sci. Rep. 9(1), 15352 (2019).","journal-title":"Sci. Rep."},{"key":"74626_CR22","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pmed.1002686","author":"P Rajpurkar","year":"2018","unstructured":"Rajpurkar, P. et al. Deep learning for chest radiograph diagnosis: a retrospective comparison of the CheXNeXt algorithm to practicing radiologists. PLoS Med. https:\/\/doi.org\/10.1371\/journal.pmed.1002686 (2018).","journal-title":"PLoS Med."},{"key":"74626_CR23","doi-asserted-by":"publisher","first-page":"5341","DOI":"10.1007\/s00330-019-06130-x","volume":"29","author":"S Park","year":"2019","unstructured":"Park, S. et al. Application of deep learning-based computer-aided detection system: detecting pneumothorax on chest radiograph after biopsy. Eur. Radiol. 29, 5341\u20135348 (2019).","journal-title":"Eur. Radiol."},{"key":"74626_CR24","doi-asserted-by":"publisher","first-page":"3660","DOI":"10.1007\/s00330-020-06771-3","volume":"7","author":"EJ Hwang","year":"2020","unstructured":"Hwang, E. J., Hong, J. H., Lee, K. H., Kim, J. I. & Nam, J. G. Deep learning algorithm for surveillance of pneumothorax after lung biopsy: a multicenter diagnostic cohort study. Eur. Radiol. 7, 3660\u20134367 (2020).","journal-title":"Eur. Radiol."}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-020-74626-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-020-74626-4","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-020-74626-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,6]],"date-time":"2022-12-06T21:38:55Z","timestamp":1670362735000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-020-74626-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,15]]},"references-count":24,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,12]]}},"alternative-id":["74626"],"URL":"https:\/\/doi.org\/10.1038\/s41598-020-74626-4","relation":{},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,10,15]]},"assertion":[{"value":"8 January 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 September 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 October 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"17417"}}