{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,7]],"date-time":"2026-11-07T14:22:49Z","timestamp":1794061369421,"version":"4.8.0"},"reference-count":25,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2019,4,23]],"date-time":"2019-04-23T00:00:00Z","timestamp":1555977600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2019,4,23]],"date-time":"2019-04-23T00:00:00Z","timestamp":1555977600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>The increased availability of labeled X-ray image archives (e.g. ChestX-ray14 dataset) has triggered a growing interest in deep learning techniques. To provide better insight into the different approaches, and their applications to chest X-ray classification, we investigate a powerful network architecture in detail: the ResNet-50. Building on prior work in this domain, we consider transfer learning with and without fine-tuning as well as the training of a dedicated X-ray network from scratch. To leverage the high spatial resolution of X-ray data, we also include an extended ResNet-50 architecture, and a network integrating non-image data (patient age, gender and acquisition type) in the classification process. In a concluding experiment, we also investigate multiple ResNet depths (i.e. ResNet-38 and ResNet-101). In a systematic evaluation, using 5-fold re-sampling and a multi-label loss function, we compare the performance of the different approaches for pathology classification by ROC statistics and analyze differences between the classifiers using rank correlation. Overall, we observe a considerable spread in the achieved performance and conclude that the X-ray-specific ResNet-38, integrating non-image data yields the best overall results. Furthermore, class activation maps are used to understand the classification process, and a detailed analysis of the impact of non-image features is provided.<\/jats:p>","DOI":"10.1038\/s41598-019-42294-8","type":"journal-article","created":{"date-parts":[[2019,4,22]],"date-time":"2019-04-22T19:05:24Z","timestamp":1555959924000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":393,"title":["Comparison of Deep Learning Approaches for Multi-Label Chest X-Ray Classification"],"prefix":"10.1038","volume":"9","author":[{"given":"Ivo M.","family":"Baltruschat","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1604-6647","authenticated-orcid":false,"given":"Hannes","family":"Nickisch","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael","family":"Grass","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1589-8517","authenticated-orcid":false,"given":"Tobias","family":"Knopp","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Axel","family":"Saalbach","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2019,4,23]]},"reference":[{"key":"42294_CR1","unstructured":"Commission, C. Q. Queen Alexandra hospital quality report. Available at, https:\/\/www.cqc.org.uk\/location\/RHU03 (2017)."},{"key":"42294_CR2","unstructured":"Krizhevsky, A., Sutskever, I. & Hinton, G. E. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, 1097\u20131105 (2012)."},{"key":"42294_CR3","doi-asserted-by":"publisher","unstructured":"Szegedy, C. et al. Going deeper with convolutions. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1\u20139, https:\/\/doi.org\/10.1109\/CVPR.2015.7298594 (2015).","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"42294_CR4","unstructured":"Simonyan, K. & Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)."},{"key":"42294_CR5","doi-asserted-by":"publisher","unstructured":"He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770\u2013778, https:\/\/doi.org\/10.1109\/CVPR.2016.90 (2016).","DOI":"10.1109\/CVPR.2016.90"},{"key":"42294_CR6","doi-asserted-by":"crossref","unstructured":"Bar, Y. et al. Chest pathology detection using deep learning with non-medical training. In 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI), 294\u2013297 (Citeseer, 2015).","DOI":"10.1109\/ISBI.2015.7163871"},{"key":"42294_CR7","doi-asserted-by":"publisher","first-page":"304","DOI":"10.1093\/jamia\/ocv080","volume":"23","author":"D Demner-Fushman","year":"2015","unstructured":"Demner-Fushman, D. et al. Preparing a collection of radiology examinations for distribution and retrieval. J. Am. Med. Informatics Assoc. 23, 304\u2013310 (2015).","journal-title":"J. Am. Med. Informatics Assoc."},{"key":"42294_CR8","doi-asserted-by":"crossref","unstructured":"Wang, X. et al. Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 3462\u20133471, 10.1109\/CVPR.2017.369 (2017).","DOI":"10.1109\/CVPR.2017.369"},{"key":"42294_CR9","unstructured":"Yao, L. et al. Learning to diagnose from scratch by exploiting dependencies among labels. arXiv preprint arXiv:1710.10501 (2017)."},{"key":"42294_CR10","doi-asserted-by":"publisher","unstructured":"Huang, G., Liu, Z., v. d. Maaten, L. & Weinberger, K. Q. Densely connected convolutional networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2261\u20132269, https:\/\/doi.org\/10.1109\/CVPR.2017.243 (2017).","DOI":"10.1109\/CVPR.2017.243"},{"key":"42294_CR11","doi-asserted-by":"publisher","unstructured":"Li, Z. et al. Thoracic disease identification and localization with limited supervision. In 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, 8290\u20138299, https:\/\/doi.org\/10.1109\/CVPR.2018.00865 (2018).","DOI":"10.1109\/CVPR.2018.00865"},{"key":"42294_CR12","unstructured":"Rajpurkar, P. et al. Chexnet: Radiologist-level pneumonia detection on chest x-rays with deep learning. arXiv preprint arXiv:1711.05225 (2017)."},{"key":"42294_CR13","unstructured":"Yao, L., Prosky, J., Poblenz, E., Covington, B. & Lyman, K. Weakly supervised medical diagnosis and localization from multiple resolutions. arXiv preprint arXiv:1803.07703 (2018)."},{"key":"42294_CR14","doi-asserted-by":"crossref","unstructured":"Guendel, S. et al. Learning to recognize abnormalities in chest x-rays with location-aware dense networks. arXiv preprint arXiv:1803.04565 (2018).","DOI":"10.1007\/978-3-030-13469-3_88"},{"key":"42294_CR15","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S. & Sun, J. Identity mappings in deep residual networks. In Computer Vision \u2013 ECCV 2016, 630\u2013645 (Springer International Publishing, 2016).","DOI":"10.1007\/978-3-319-46493-0_38"},{"key":"42294_CR16","doi-asserted-by":"publisher","first-page":"1345","DOI":"10.1109\/TKDE.2009.191","volume":"22","author":"SJ Pan","year":"2010","unstructured":"Pan, S. J. & Yang, Q. A survey on transfer learning. IEEE Transactions on Knowl. Data Eng. 22, 1345\u20131359, https:\/\/doi.org\/10.1109\/TKDE.2009.191 (2010).","journal-title":"IEEE Transactions on Knowl. Data Eng."},{"key":"42294_CR17","unstructured":"Yosinski, J., Clune, J., Bengio, Y. & Lipson, H. How transferable are features in deep neural networks? In Advances in neural information processing systems, 3320\u20133328 (2014)."},{"key":"42294_CR18","doi-asserted-by":"publisher","unstructured":"Razavian, A. S., Azizpour, H., Sullivan, J. & Carlsson, S. Cnn features off-the-shelf: An astounding baseline for recognition. In 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops, 512\u2013519, https:\/\/doi.org\/10.1109\/CVPRW.2014.131 (2014).","DOI":"10.1109\/CVPRW.2014.131"},{"key":"42294_CR19","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","volume":"115","author":"O Russakovsky","year":"2015","unstructured":"Russakovsky, O. et al. Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115, 211\u2013252 (2015).","journal-title":"Int. J. Comput. Vis."},{"key":"42294_CR20","doi-asserted-by":"publisher","first-page":"3301","DOI":"10.1093\/bioinformatics\/bti499","volume":"21","author":"AM Molinaro","year":"2005","unstructured":"Molinaro, A. M., Simon, R. & Pfeiffer, R. M. Prediction error estimation: a comparison of resampling methods. Bioinforma. 21, 3301\u20133307, https:\/\/doi.org\/10.1093\/bioinformatics\/bti499 (2005).","journal-title":"Bioinforma."},{"key":"42294_CR21","first-page":"1929","volume":"15","author":"N Srivastava","year":"2014","unstructured":"Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I. & Salakhutdinov, R. Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1929\u20131958 (2014).","journal-title":"J. Mach. Learn. Res."},{"key":"42294_CR22","unstructured":"Kingma, D. P. & Ba, J. L. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR) (2015)."},{"key":"42294_CR23","doi-asserted-by":"publisher","unstructured":"Selvaraju, R. R. et al. Grad-cam: Visual explanations from deep networks via gradient-based localization. In 2017 IEEE International Conference on Computer Vision (ICCV), 618\u2013626, https:\/\/doi.org\/10.1109\/ICCV.2017.74 (2017).","DOI":"10.1109\/ICCV.2017.74"},{"key":"42294_CR24","unstructured":"Oakden-Rayner, L. Exploring the ChestXray14 dataset: Problems. Available at, https:\/\/lukeoakdenrayner.wordpress.com\/2017\/12\/18\/ (2017)."},{"key":"42294_CR25","doi-asserted-by":"publisher","first-page":"251S","DOI":"10.1016\/S0197-2456(00)00097-0","volume":"21","author":"JK Gohagan","year":"2000","unstructured":"Gohagan, J. K., Prorok, P. C., Hayes, R. B. & Kramer, B.-S. The prostate, lung, colorectal and ovarian (PLCO) cancer screening trial of the national cancer institute: history, organization, and status. Control. clinical trials 21, 251S\u2013272S (2000).","journal-title":"Control. clinical trials"}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-019-42294-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-019-42294-8","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-019-42294-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,17]],"date-time":"2022-12-17T03:55:36Z","timestamp":1671249336000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-019-42294-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,4,23]]},"references-count":25,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2019,12]]}},"alternative-id":["42294"],"URL":"https:\/\/doi.org\/10.1038\/s41598-019-42294-8","relation":{},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,4,23]]},"assertion":[{"value":"14 August 2018","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 March 2019","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 April 2019","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"M.G., H.N. and A.S. are employees of Philips Research, Hamburg, Germany. I.M.B. and T.K. declare no potential conflict of interest.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing Interests"}}],"article-number":"6381"}}