{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,29]],"date-time":"2026-08-29T15:34:50Z","timestamp":1788017690446,"version":"build-2784847793"},"reference-count":15,"publisher":"Georg Thieme Verlag KG","issue":"09","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Rofo"],"published-print":{"date-parts":[[2024,9]]},"abstract":"<jats:p>\n            Purpose\u2002The aim of this study was to develop an algorithm to automatically extract annotations from German thoracic radiology reports to train deep learning-based chest X-ray classification models.<\/jats:p><jats:p>\n            Materials and Methods\u2002An automatic label extraction model for German thoracic radiology reports was designed based on the CheXpert architecture. The algorithm can extract labels for twelve common chest pathologies, the presence of support devices, and \u201cno finding\u201d. For iterative improvements and to generate a ground truth, a web-based multi-reader annotation interface was created. With the proposed annotation interface, a radiologist annotated 1086 retrospectively collected radiology reports from 2020\u20132021 (data set 1). The effect of automatically extracted labels on chest radiograph classification performance was evaluated on an additional, in-house pneumothorax data set (data set 2), containing 6434 chest radiographs with corresponding reports, by comparing a DenseNet-121 model trained on extracted labels from the associated reports, image-based pneumothorax labels, and publicly available data, respectively.<\/jats:p><jats:p>\n            Results\u2002Comparing automated to manual labeling on data set 1: \u201cmention extraction\u201d class-wise F1 scores ranged from 0.8 to 0.995, the \u201cnegation detection\u201d F1 scores from 0.624 to 0.981, and F1 scores for \u201cuncertainty detection\u201d from 0.353 to 0.725. Extracted pneumothorax labels on data set 2 had a sensitivity of 0.997 [95\u200a% CI: 0.994, 0.999] and specificity of 0.991 [95\u200a% CI: 0.988, 0.994]. The model trained on publicly available data achieved an area under the receiver operating curve (AUC) for pneumothorax classification of 0.728 [95\u200a% CI: 0.694, 0.760], while the models trained on automatically extracted labels and on manual annotations achieved values of 0.858 [95\u200a% CI: 0.832, 0.882] and 0.934 [95\u200a% CI: 0.918, 0.949], respectively.<\/jats:p><jats:p>\n            Conclusion\u2002Automatic label extraction from German thoracic radiology reports is a promising substitute for manual labeling. By reducing the time required for data annotation, larger training data sets can be created, resulting in improved overall modeling performance. Our results demonstrated that a pneumothorax classifier trained on automatically extracted labels strongly outperformed the model trained on publicly available data, without the need for additional annotation time and performed competitively compared to manually labeled data.<\/jats:p><jats:p>\n            Key Points\u2002<\/jats:p><jats:p>\n            Zitierweise\n          <\/jats:p>","DOI":"10.1055\/a-2234-8268","type":"journal-article","created":{"date-parts":[[2024,1,31]],"date-time":"2024-01-31T23:51:53Z","timestamp":1706745113000},"page":"956-965","source":"Crossref","is-referenced-by-count":5,"title":["German CheXpert Chest X-ray Radiology Report Labeler"],"prefix":"10.1055","volume":"196","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9535-0502","authenticated-orcid":false,"given":"Alessandro","family":"Wollek","sequence":"additional","affiliation":[{"name":"Munich Institute of Biomedical Engineering, Technical University of Munich, Garching b. M\u00fcnchen, Germany"},{"name":"School of Computation, Information and Technology, Technical University of Munich, Garching b. M\u00fcnchen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sardi","family":"Hyska","sequence":"additional","affiliation":[{"name":"Department of Radiology, Ludwig-Maximilians-University Hospital Munich, M\u00fcnchen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thomas","family":"Sedlmeyr","sequence":"additional","affiliation":[{"name":"Munich Institute of Biomedical Engineering, Technical University of Munich, Garching b. M\u00fcnchen, Germany"},{"name":"School of Computation, Information and Technology, Technical University of Munich, Garching b. M\u00fcnchen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Philip","family":"Haitzer","sequence":"additional","affiliation":[{"name":"Munich Institute of Biomedical Engineering, Technical University of Munich, Garching b. M\u00fcnchen, Germany"},{"name":"School of Computation, Information and Technology, Technical University of Munich, Garching b. M\u00fcnchen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Johannes","family":"Rueckel","sequence":"additional","affiliation":[{"name":"Department of Radiology, Ludwig-Maximilians-University Hospital Munich, M\u00fcnchen, Germany"},{"name":"Institute of Neuroradiology, Ludwig-Maximilians-University Hospital Munich, M\u00fcnchen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bastian O.","family":"Sabel","sequence":"additional","affiliation":[{"name":"Institute for Clinical Radiology, Ludwig-Maximilians-University Hospital Munich, Germany, M\u00fcnchen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0268-9078","authenticated-orcid":false,"given":"Michael","family":"Ingrisch","sequence":"additional","affiliation":[{"name":"Department of Radiology, Ludwig-Maximilians-University Hospital Munich, M\u00fcnchen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tobias","family":"Lasser","sequence":"additional","affiliation":[{"name":"Munich Institute of Biomedical Engineering, Technical University of Munich, Garching b. M\u00fcnchen, Germany"},{"name":"School of Computation, Information and Technology, Technical University of Munich, Garching b. M\u00fcnchen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"194","published-online":{"date-parts":[[2024,1,31]]},"reference":[{"key":"ref3","doi-asserted-by":"publisher","DOI":"10.7191\/jgr.2020.1072","article-title":"Diagnostic Radiology in Nigeria: A Country Report","volume":"6","author":"B Idowu","year":"2020","journal-title":"Journal of Global Radiology"},{"key":"ref4","first-page":"359","article-title":"Radiologist shortage leaves patient care at risk, warns royal college","author":"A Rimmer","year":"2017","journal-title":"BMJ: British Medical Journal (Online)"},{"key":"ref5","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1148\/radiol.2015150921","article-title":"The US radiologist workforce: an analysis of temporal and geographic variation by using large national datasets","volume":"279","author":"A B Rosenkrantz","year":"2016","journal-title":"Radiology"},{"key":"ref6","doi-asserted-by":"crossref","first-page":"421","DOI":"10.1148\/radiol.2019191293","article-title":"Chest radiograph interpretation with deep learning models: assessment with radiologist-adjudicated reference standards and population-adjusted evaluation","volume":"294","author":"A Majkowska","year":"2020","journal-title":"Radiology"},{"key":"ref7","doi-asserted-by":"publisher","first-page":"e1002686","DOI":"10.1371\/journal.pmed.1002686","article-title":"Deep learning for chest radiograph diagnosis: A retrospective comparison of the CheXNeXt algorithm to practicing radiologists","volume":"15","author":"P Rajpurkar","year":"2018","journal-title":"PLOS Medicine"},{"key":"ref8","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"ImageNet Large Scale Visual Recognition Challenge","volume":"115","author":"O Russakovsky","year":"2015","journal-title":"International Journal of Computer Vision"},{"key":"ref9","first-page":"1097","article-title":"Imagenet classification with deep convolutional neural networks","volume":"25","author":"A Krizhevsky","year":"2012","journal-title":"Advances in neural information processing systems"},{"key":"ref15","doi-asserted-by":"publisher","first-page":"e220187","DOI":"10.1148\/ryai.220187","article-title":"Attention-based Saliency Maps Improve Interpretability of Pneumothorax Classification","author":"A Wollek","year":"2023","journal-title":"Radiology: Artificial Intelligence"},{"key":"ref17","first-page":"6","article-title":"MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports","author":"A E Johnson","year":"2019","journal-title":"Scientific Data"},{"key":"ref18","doi-asserted-by":"publisher","first-page":"487","DOI":"10.1038\/s41597-022-01608-8","article-title":"BRAX, Brazilian labeled chest x-ray dataset","volume":"9","author":"E P Reis","year":"2022","journal-title":"Sci Data"},{"key":"ref19","doi-asserted-by":"publisher","first-page":"e0276545","DOI":"10.1371\/journal.pone.0276545","article-title":"Learning to diagnose common thorax diseases on chest radiographs from radiology reports in Vietnamese","volume":"17","author":"T TB Nguyen","year":"2022","journal-title":"PLoS ONE"},{"key":"ref20","doi-asserted-by":"publisher","DOI":"10.1007\/s00330-023-09526-y","article-title":"Transformer-based structuring of free-text radiology report databases","author":"S Nowak","year":"2023","journal-title":"Eur Radiol"},{"key":"ref21","first-page":"115","volume-title":"Negation Detection in Clinical Reports Written in German. In: Proceedings of the Fifth Workshop on Building and Evaluating Resources for Biomedical Text Mining (BioTxtM2016)","author":"V Cotik","year":"2016"},{"key":"ref22","doi-asserted-by":"crossref","first-page":"697","DOI":"10.1148\/radiol.2462070712","article-title":"Fleischner Society: glossary of terms for thoracic imaging","volume":"246","author":"D M Hansell","year":"2008","journal-title":"Radiology"},{"key":"ref23","doi-asserted-by":"publisher","first-page":"792","DOI":"10.1097\/RLI.0000000000000707","article-title":"Impact of Confounding Thoracic Tubes and Pleural Dehiscence Extent on Artificial Intelligence Pneumothorax Detection in Chest Radiographs","volume":"55","author":"J Rueckel","year":"2020","journal-title":"Invest Radiol"}],"container-title":["R\u00f6Fo - Fortschritte auf dem Gebiet der R\u00f6ntgenstrahlen und der bildgebenden Verfahren"],"original-title":[],"language":"de","link":[{"URL":"http:\/\/www.thieme-connect.de\/products\/ejournals\/pdf\/10.1055\/a-2234-8268.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,5]],"date-time":"2024-11-05T10:40:07Z","timestamp":1730803207000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.thieme-connect.de\/DOI\/DOI?10.1055\/a-2234-8268"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,31]]},"references-count":15,"journal-issue":{"issue":"09","published-online":{"date-parts":[[2024,8,15]]},"published-print":{"date-parts":[[2024,9]]}},"URL":"https:\/\/doi.org\/10.1055\/a-2234-8268","archive":["Portico","CLOCKSS"],"relation":{},"ISSN":["1438-9029","1438-9010"],"issn-type":[{"value":"1438-9029","type":"print"},{"value":"1438-9010","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,1,31]]}}}