{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T17:22:43Z","timestamp":1791566563493,"version":"4.3.4"},"reference-count":29,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2020,5,30]],"date-time":"2020-05-30T00:00:00Z","timestamp":1590796800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>The novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused a pandemic resulting in over 2.7 million infected individuals and over 190,000 deaths and growing. Assertions in the literature suggest that respiratory disorders due to COVID-19 commonly present with pneumonia-like symptoms which are radiologically confirmed as opacities. Radiology serves as an adjunct to the reverse transcription-polymerase chain reaction test for confirmation and evaluating disease progression. While computed tomography (CT) imaging is more specific than chest X-rays (CXR), its use is limited due to cross-contamination concerns. CXR imaging is commonly used in high-demand situations, placing a significant burden on radiology services. The use of artificial intelligence (AI) has been suggested to alleviate this burden. However, there is a dearth of sufficient training data for developing image-based AI tools. We propose increasing training data for recognizing COVID-19 pneumonia opacities using weakly labeled data augmentation. This follows from a hypothesis that the COVID-19 manifestation would be similar to that caused by other viral pathogens affecting the lungs. We expand the training data distribution for supervised learning through the use of weakly labeled CXR images, automatically pooled from publicly available pneumonia datasets, to classify them into those with bacterial or viral pneumonia opacities. Next, we use these selected images in a stage-wise, strategic approach to train convolutional neural network-based algorithms and compare against those trained with non-augmented data. Weakly labeled data augmentation expands the learned feature space in an attempt to encompass variability in unseen test distributions, enhance inter-class discrimination, and reduce the generalization error. Empirical evaluations demonstrate that simple weakly labeled data augmentation (Acc: 0.5555 and Acc: 0.6536) is better than baseline non-augmented training (Acc: 0.2885 and Acc: 0.5028) in identifying COVID-19 manifestations as viral pneumonia. Interestingly, adding COVID-19 CXRs to simple weakly labeled augmented training data significantly improves the performance (Acc: 0.7095 and Acc: 0.8889), suggesting that COVID-19, though viral in origin, creates a uniquely different presentation in CXRs compared with other viral pneumonia manifestations.<\/jats:p>","DOI":"10.3390\/diagnostics10060358","type":"journal-article","created":{"date-parts":[[2020,6,1]],"date-time":"2020-06-01T11:49:21Z","timestamp":1591012161000},"page":"358","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":72,"title":["Weakly Labeled Data Augmentation for Deep Learning: A Study on COVID-19 Detection in Chest X-Rays"],"prefix":"10.3390","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0871-8634","authenticated-orcid":false,"given":"Sivaramakrishnan","family":"Rajaraman","sequence":"first","affiliation":[{"name":"Lister Hill National Center for Biomedical Communications, National Library of Medicine, 8600 Rockville Pike, Bethesda, MD 20894, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0040-1387","authenticated-orcid":false,"given":"Sameer","family":"Antani","sequence":"additional","affiliation":[{"name":"Lister Hill National Center for Biomedical Communications, National Library of Medicine, 8600 Rockville Pike, Bethesda, MD 20894, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,5,30]]},"reference":[{"key":"ref_1","unstructured":"World Health Organization (WHO) (2020, March 02). Coronavirus Disease (COVID-2019) Situation Reports. Available online: https:\/\/www.who.int\/emergencies\/diseases\/novel-coronavirus-2019\/situation-reports."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Rubin, G.D., Ryerson, C.J., Haramati, L.B., Sverzellati, N., Kanne, J.P., Raoof, S., Schluger, N.W., Volpi, A., Yim, J.-J., and Martin, I.B.K. (2020). The Role of Chest Imaging in Patient Management during the COVID-19 Pandemic: A Multinational Consensus Statement from the Fleischner Society. Radiology, 201365.","DOI":"10.1148\/radiol.2020201365"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Bai, H.X., Hsieh, B., Xiong, Z., Halsey, K., Choi, J.W., Tran, T.M.L., Pan, I., Shi, L.-B., Wang, D.-C., and Mei, J. (2020). Performance of radiologists in differentiating COVID-19 from viral pneumonia on chest CT. Radiology, 200823.","DOI":"10.1148\/radiol.2020200823"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1122","DOI":"10.1016\/j.cell.2018.02.010","article-title":"Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning","volume":"172","author":"Kermany","year":"2018","journal-title":"Cell"},{"key":"ref_5","unstructured":"(2020, March 12). ACR Recommendations for the use of Chest Radiography and Computed Tomography (CT) for Suspected COVID-19 Infection. Available online: https:\/\/www.acr.org\/Advocacy-and-Economics\/ACR-Position-Statements\/Recommendations-for-Chest-Radiography-and-CT-for-Suspected-COVID19-Infection."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Li, L., Qin, L., Xu, Z., Yin, Y., Wang, X., Kong, B., Bai, J., Lu, Y., Fang, Z., and Song, Q. (2020). Artificial Intelligence Distinguishes COVID-19 from Community Acquired Pneumonia on Chest CT. Radiology, 200905.","DOI":"10.1148\/radiol.2020200905"},{"key":"ref_7","unstructured":"(2020, April 01). RSNA Announces COVID-19 Imaging Data Repository. Available online: https:\/\/press.rsna.org\/timssnet\/media\/pressreleases\/14_pr_target.cfm?ID=2167."},{"key":"ref_8","unstructured":"(2020, April 02). A European Initiative for Automated Diagnosis and Quantitative Analysis of COVID-19 on Imaging. Available online: https:\/\/imagingcovid19ai.eu\/."},{"key":"ref_9","unstructured":"Perez, L., and Wang, J. (2017). The Effectiveness of Data Augmentation in Image Classification using Deep Learning. arXiv."},{"key":"ref_10","first-page":"841","article-title":"Assessment of Data Augmentation Strategies Toward Performance Improvement of Abnormality Classification in Chest Radiographs","volume":"Volume 2019","author":"Ganesan","year":"2019","journal-title":"Proceedings of the 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Takahashi, R., Matsubara, T., and Uehara, K. (2020). Data Augmentation using Random Image Cropping and Patching for Deep CNNs. IEEE Trans. Circuits Syst. Video Technol., 1.","DOI":"10.1109\/TCSVT.2019.2935128"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L., Li, K., and Li, F.-F. (2009, January 20\u201325). ImageNet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ben-Cohen, A., Klang, E., Amitai, M.M., Goldberger, J., and Greenspan, H. (2018, January 4\u20137). Anatomical data augmentation for CNN based pixel-wise classification. Proceedings of the 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), Washington, DC, USA.","DOI":"10.1109\/ISBI.2018.8363762"},{"key":"ref_14","unstructured":"Goodfellow, I. (2016). Nips 2016 tutorial: Generative adversarial networks. arXiv."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"e180041","DOI":"10.1148\/ryai.2019180041","article-title":"Augmenting the National Institutes of Health Chest Radiograph Dataset with Expert Annotations of Possible Pneumonia","volume":"1","author":"Shih","year":"2019","journal-title":"Radiol. Artif. Intell."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., and Summers, R.M. (2017). ChestX-Ray8: Hospital-Scale Chest X-Ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Institute of Electrical and Electronics Engineers (IEEE).","DOI":"10.1109\/CVPR.2017.369"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"590","DOI":"10.1609\/aaai.v33i01.3301590","article-title":"CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison","volume":"Volume 33","author":"Irvin","year":"2019","journal-title":"Proceedings of the Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"ref_18","unstructured":"Cohen, J.P., Morrison, P., and Dao, L. (2020). COVID-19 Image Data Collection 2020. arXiv."},{"key":"ref_19","first-page":"234","article-title":"U-Net: Convolutional Networks for Biomedical Image Segmentation","volume":"9351","author":"Ronneberger","year":"2015","journal-title":"Appl. Evol. Comput."},{"key":"ref_20","unstructured":"Yu, F., and Koltun, V. (2015). Multi-Scale Context Aggregation by Dilated Convolutions. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Candemir, S., Antani, S., Jaeger, S., Browning, R., and Thoma, G.R. (2015, January 21\u201326). Lung boundary detection in pediatric chest x-rays. Proceedings of the Medical Imaging 2015: PACS and Imaging Informatics: Next Generation and Innovations, Orlando, FL, USA.","DOI":"10.1117\/12.2081060"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Zagoruyko, S., Komodakis, N., Wilson, R.C., Hancock, E.R., Smith, W.A.P., Pears, N.E., and Bors, A.G. (2016, January 19\u201322). Wide Residual Networks. Proceedings of the British Machine Vision Conference 2016, York, UK.","DOI":"10.5244\/C.30.87"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Institute of Electrical and Electronics Engineers (IEEE).","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_24","unstructured":"Simonyan, K., and Zisserman, A. (2015, January 7\u20139). Very deep convolutional networks for large-scale image recognition. Proceedings of the International Conference on Learning Representations, San Diego, CA, USA."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (July, January 26). Rethinking the Inception Architecture for Computer Vision. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas Valley, NV, USA.","DOI":"10.1109\/CVPR.2016.308"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Chollet, F. (2017). Xception: Deep Learning with Depthwise Separable Convolutions. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Institute of Electrical and Electronics Engineers (IEEE).","DOI":"10.1109\/CVPR.2017.195"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017). Densely Connected Convolutional Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Institute of Electrical and Electronics Engineers (IEEE).","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_28","unstructured":"Pham, H., Guan, M.Y., Zoph, B., Le, Q.V., and Dean, J. (2018, January 10\u201315). Efficient Neural Architecture Search via Parameter Sharing. Proceedings of the Proc. Int. Conf. Machine Learning (ICML), Stockholm, Sweden."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017). Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Institute of Electrical and Electronics Engineers (IEEE).","DOI":"10.1109\/ICCV.2017.74"}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/10\/6\/358\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:33:59Z","timestamp":1760175239000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/10\/6\/358"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,5,30]]},"references-count":29,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2020,6]]}},"alternative-id":["diagnostics10060358"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics10060358","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2020.05.04.20090803","asserted-by":"object"}]},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,5,30]]}}}