{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,24]],"date-time":"2026-03-24T14:25:36Z","timestamp":1774362336827,"version":"3.50.1"},"reference-count":27,"publisher":"Oxford University Press (OUP)","issue":"21","license":[{"start":{"date-parts":[[2020,7,23]],"date-time":"2020-07-23T00:00:00Z","timestamp":1595462400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"name":"BIH Charit\u00e9 Digital Clinician Scientist Program"},{"name":"Charit\u00e9 \u2013 Universit\u00e4tsmedizin Berlin and the Berlin Institute of Health"},{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["SFB 1340\/1 2018"],"award-info":[{"award-number":["SFB 1340\/1 2018"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["5943\/31\/41\/91"],"award-info":[{"award-number":["5943\/31\/41\/91"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,1,29]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Motivation<\/jats:title>\n                  <jats:p>The development of deep, bidirectional transformers such as Bidirectional Encoder Representations from Transformers (BERT) led to an outperformance of several Natural Language Processing (NLP) benchmarks. Especially in radiology, large amounts of free-text data are generated in daily clinical workflow. These report texts could be of particular use for the generation of labels in machine learning, especially for image classification. However, as report texts are mostly unstructured, advanced NLP methods are needed to enable accurate text classification. While neural networks can be used for this purpose, they must first be trained on large amounts of manually labelled data to achieve good results. In contrast, BERT models can be pre-trained on unlabelled data and then only require fine tuning on a small amount of manually labelled data to achieve even better results.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>Using BERT to identify the most important findings in intensive care chest radiograph reports, we achieve areas under the receiver operation characteristics curve of 0.98 for congestion, 0.97 for effusion, 0.97 for consolidation and 0.99 for pneumothorax, surpassing the accuracy of previous approaches with comparatively little annotation effort. Our approach could therefore help to improve information extraction from free-text medical reports.<\/jats:p>\n                  <jats:p>Availability\u2002 and implementation<\/jats:p>\n                  <jats:p>We make the source code for fine-tuning the BERT-models freely available at https:\/\/github.com\/fast-raidiology\/bert-for-radiology.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Supplementary information<\/jats:title>\n                  <jats:p>Supplementary data are available at Bioinformatics online.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btaa668","type":"journal-article","created":{"date-parts":[[2020,7,17]],"date-time":"2020-07-17T11:10:16Z","timestamp":1594984216000},"page":"5255-5261","source":"Crossref","is-referenced-by-count":60,"title":["Highly accurate classification of chest radiographic reports using a deep learning natural language model pre-trained on 3.8 million text reports"],"prefix":"10.1093","volume":"36","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9249-8624","authenticated-orcid":false,"given":"Keno K","family":"Bressem","sequence":"first","affiliation":[{"name":"Department of Radiology, Charit\u00e9 , Berlin 12203, Germany"},{"name":"Charit\u00e9 \u2013 Universit\u00e4tsmedizin Berlin, Corporate Member of Freie Universit\u00e4t Berlin, Humboldt-Universit\u00e4t zu Berlin, and Berlin Institute of Health , Berlin 10117, Germany"}]},{"given":"Lisa C","family":"Adams","sequence":"additional","affiliation":[{"name":"Department of Radiology, Charit\u00e9 , Berlin 12203, Germany"},{"name":"Charit\u00e9 \u2013 Universit\u00e4tsmedizin Berlin, Corporate Member of Freie Universit\u00e4t Berlin, Humboldt-Universit\u00e4t zu Berlin, and Berlin Institute of Health , Berlin 10117, Germany"}]},{"given":"Robert A","family":"Gaudin","sequence":"additional","affiliation":[{"name":"Department of Oral- and Maxillofacial Surgery, Charit\u00e9 , Berlin 12203, Germany"}]},{"given":"Daniel","family":"Tr\u00f6ltzsch","sequence":"additional","affiliation":[{"name":"Department of Oral- and Maxillofacial Surgery, Charit\u00e9 , Berlin 12203, Germany"}]},{"given":"Bernd","family":"Hamm","sequence":"additional","affiliation":[{"name":"Department of Radiology, Charit\u00e9 , Berlin 12203, Germany"}]},{"given":"Marcus R","family":"Makowski","sequence":"additional","affiliation":[{"name":"Department of Diagnostic and Interventional Radiology, Technical University of Munich, School of Medicine , Munich 81675, Germany"}]},{"given":"Chan-Yong","family":"Sch\u00fcle","sequence":"additional","affiliation":[{"name":"Department of Radiology, Charit\u00e9 , Berlin 12203, Germany"}]},{"given":"Janis L","family":"Vahldiek","sequence":"additional","affiliation":[{"name":"Department of Radiology, Charit\u00e9 , Berlin 12203, Germany"}]},{"given":"Stefan M","family":"Niehues","sequence":"additional","affiliation":[{"name":"Department of Radiology, Charit\u00e9 , Berlin 12203, Germany"}]}],"member":"286","published-online":{"date-parts":[[2020,7,23]]},"reference":[{"key":"2023062408063288000_btaa668-B1","first-page":"265","author":"Abadi","year":"2016"},{"key":"2023062408063288000_btaa668-B2","first-page":"3606","author":"Beltagy","year":"2019"},{"key":"2023062408063288000_btaa668-B3","author":"Bustos","year":"2019"},{"key":"2023062408063288000_btaa668-B4","doi-asserted-by":"crossref","first-page":"176","DOI":"10.1148\/rg.2016150080","article-title":"Natural language processing technologies in radiology research and clinical applications","volume":"36","author":"Cai","year":"2016","journal-title":"RadioGraphics"},{"key":"2023062408063288000_btaa668-B5","doi-asserted-by":"crossref","first-page":"845","DOI":"10.1148\/radiol.2017171115","article-title":"Deep learning to classify radiology free-text reports","volume":"286","author":"Chen","year":"2018","journal-title":"Radiology"},{"key":"2023062408063288000_btaa668-B6","author":"Devlin","year":"2018"},{"key":"2023062408063288000_btaa668-B7","author":"","year":"2017"},{"key":"2023062408063288000_btaa668-B8","doi-asserted-by":"crossref","first-page":"1544","DOI":"10.1001\/jamainternmed.2018.3763","article-title":"Potential biases in machine learning algorithms using electronic health record data","volume":"178","author":"Gianfrancesco","year":"2018","journal-title":"JAMA Int. Med"},{"key":"2023062408063288000_btaa668-B9","author":"Goldberg","year":"2019"},{"key":"2023062408063288000_btaa668-B10","doi-asserted-by":"crossref","first-page":"500","DOI":"10.1038\/s41568-018-0016-5","article-title":"Artificial intelligence in radiology","volume":"18","author":"Hosny","year":"2018","journal-title":"Nat. Rev. Cancer"},{"key":"2023062408063288000_btaa668-B11","author":"Howard","year":"2018"},{"key":"2023062408063288000_btaa668-B12","author":"Irvin","year":"2019"},{"key":"2023062408063288000_btaa668-B13","volume-title":"Bert-Vocab-Builder","author":"Kwon","year":"2019"},{"key":"2023062408063288000_btaa668-B14","volume-title":"RadLex: A New Method for Indexing Online Educational Materials","author":"Langlotz","year":"2006"},{"key":"2023062408063288000_btaa668-B15","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"2023062408063288000_btaa668-B16","author":"Lee","year":"2019"},{"key":"2023062408063288000_btaa668-B17","doi-asserted-by":"crossref","first-page":"276","DOI":"10.11613\/BM.2012.031","article-title":"Interrater reliability: the kappa statistic","volume":"22","author":"McHugh","year":"2012","journal-title":"Biochem. Med. (Zagreb)"},{"key":"2023062408063288000_btaa668-B18","article-title":"Pytorch: An imperative style, high-performance deep learning library. In: Advances in neural information processing systems (NeurIPS, Vancouver, Canada). pp. 8026-8037","author":"Paszke","year":"2019"},{"key":"2023062408063288000_btaa668-B19","author":"Peters","year":"2018"},{"key":"2023062408063288000_btaa668-B20","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1186\/s13244-019-0777-8","article-title":"Structured report data can be used to develop deep learning algorithms: a proof of concept in ankle radiographs","volume":"10","author":"Pinto Dos Santos","year":"2019","journal-title":"Insights Imaging"},{"key":"2023062408063288000_btaa668-B21","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1148\/radiol.16142770","article-title":"Natural language processing in radiology: a systematic review","volume":"279","author":"Pons","year":"2016","journal-title":"Radiology"},{"key":"2023062408063288000_btaa668-B22"},{"key":"2023062408063288000_btaa668-B23","author":"Radford","year":"2018"},{"key":"2023062408063288000_btaa668-B24","author":"Rajapakse","year":"2019"},{"key":"2023062408063288000_btaa668-B25","author":"Tenney","year":"2019"},{"key":"2023062408063288000_btaa668-B26","first-page":"5998","article-title":"Attention is all you need","author":"Vaswani","year":"2017","journal-title":"Advances in Neural Information Processing Systems, NeurIPS"},{"key":"2023062408063288000_btaa668-B27","author":"Wolf","year":"2019"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btaa668\/33862493\/btaa668.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/36\/21\/5255\/50692618\/btaa668.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/36\/21\/5255\/50692618\/btaa668.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,24]],"date-time":"2023-06-24T23:27:25Z","timestamp":1687649245000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/36\/21\/5255\/5875602"}},"subtitle":[],"editor":[{"given":"Jonathan","family":"Wren","sequence":"additional","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2020,7,23]]},"references-count":27,"journal-issue":{"issue":"21","published-print":{"date-parts":[[2021,1,29]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btaa668","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2020,11,1]]},"published":{"date-parts":[[2020,7,23]]}}}