{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,27]],"date-time":"2026-08-27T03:15:31Z","timestamp":1787800531624,"version":"build-2784847793"},"reference-count":32,"publisher":"Springer Science and Business Media LLC","issue":"10","license":[{"start":{"date-parts":[[2021,3,27]],"date-time":"2021-03-27T00:00:00Z","timestamp":1616803200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,3,27]],"date-time":"2021-03-27T00:00:00Z","timestamp":1616803200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100011699","name":"Siemens Healthineers","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100011699","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Eur Radiol"],"published-print":{"date-parts":[[2021,10]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objectives<\/jats:title>\n                    <jats:p>Diagnostic accuracy of artificial intelligence (AI) pneumothorax (PTX) detection in chest radiographs (CXR) is limited by the noisy annotation quality of public training data and confounding thoracic tubes (TT). We hypothesize that in-image annotations of the dehiscent visceral pleura for algorithm training boosts algorithm\u2019s performance and suppresses confounders.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>Our single-center evaluation cohort of 3062 supine CXRs includes 760 PTX-positive cases with radiological annotations of PTX size and inserted TTs. Three step-by-step improved algorithms (differing in algorithm architecture, training data from public datasets\/clinical sites, and in-image annotations included in algorithm training) were characterized by area under the receiver operating characteristics (AUROC) in detailed subgroup analyses and referenced to the well-established \u201cCheXNet\u201d algorithm.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Performances of established algorithms exclusively trained on publicly available data without in-image annotations are limited to AUROCs of 0.778 and strongly biased towards TTs that can completely eliminate algorithm\u2019s discriminative power in individual subgroups. Contrarily, our final \u201calgorithm 2\u201d which was trained on a lower number of images but additionally with in-image annotations of the dehiscent pleura achieved an overall AUROC of 0.877 for unilateral PTX detection with a significantly reduced TT-related confounding bias.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>We demonstrated strong limitations of an established PTX-detecting AI algorithm that can be significantly reduced by designing an AI system capable of learning to both classify and localize PTX. Our results are aimed at drawing attention to the necessity of high-quality in-image localization in training data to reduce the risks of unintentionally biasing the training process of pathology-detecting AI algorithms.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Key Points<\/jats:title>\n                    <jats:p>\n                      <jats:italic>\u2022 Established pneumothorax-detecting artificial intelligence algorithms trained on public training data are strongly limited and biased by confounding thoracic tubes.<\/jats:italic>\n                    <\/jats:p>\n                    <jats:p>\n                      <jats:italic>\u2022 We used high-quality in-image annotated training data to effectively boost algorithm performance and suppress the impact of confounding thoracic tubes.<\/jats:italic>\n                    <\/jats:p>\n                    <jats:p>\n                      <jats:italic>\u2022 Based on our results, we hypothesize that even hidden confounders might be effectively addressed by in-image annotations of pathology-related image features.<\/jats:italic>\n                    <\/jats:p>\n                  <\/jats:sec>","DOI":"10.1007\/s00330-021-07833-w","type":"journal-article","created":{"date-parts":[[2021,3,27]],"date-time":"2021-03-27T18:02:40Z","timestamp":1616868160000},"page":"7888-7900","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":42,"title":["Pneumothorax detection in chest radiographs: optimizing artificial intelligence system for accuracy and confounding bias reduction using in-image annotations in algorithm training"],"prefix":"10.1007","volume":"31","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6695-2958","authenticated-orcid":false,"given":"Johannes","family":"Rueckel","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christian","family":"Huemmer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andreas","family":"Fieselmann","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Florin-Cristian","family":"Ghesu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Awais","family":"Mansoor","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Balthasar","family":"Schachtner","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Philipp","family":"Wesp","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lena","family":"Trappmann","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Basel","family":"Munawwar","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jens","family":"Ricke","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael","family":"Ingrisch","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bastian O.","family":"Sabel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,3,27]]},"reference":[{"key":"7833_CR1","doi-asserted-by":"publisher","first-page":"545","DOI":"10.1378\/chest.10-1302","volume":"141","author":"S Raoof","year":"2012","unstructured":"Raoof S, Feigin D, Sung A et al (2012) Interpretation of plain chest roentgenogram. Chest 141:545\u2013558. https:\/\/doi.org\/10.1378\/chest.10-1302","journal-title":"Chest"},{"key":"7833_CR2","doi-asserted-by":"publisher","unstructured":"Kallianos K, Mongan J, Antani S et al (2019) How far have we come? Artificial intelligence for chest radiograph interpretation. Clin Radiol 74:338\u2013345. https:\/\/doi.org\/10.1016\/j.crad.2018.12.015","DOI":"10.1016\/j.crad.2018.12.015"},{"key":"7833_CR3","doi-asserted-by":"publisher","first-page":"1068","DOI":"10.1097\/SLA.0000000000001073","volume":"261","author":"DJ Roberts","year":"2015","unstructured":"Roberts DJ, Leigh-Smith S, Faris PD et al (2015) Clinical presentation of patients with tension pneumothorax: a systematic review. Ann Surg 261:1068\u20131078. https:\/\/doi.org\/10.1097\/SLA.0000000000001073","journal-title":"Ann Surg"},{"key":"7833_CR4","first-page":"184","volume":"53","author":"CG Ball","year":"2010","unstructured":"Ball CG, Wyrzykowski AD, Kirkpatrick AW et al (2010) Thoracic needle decompression for tension pneumothorax: clinical correlation with catheter length. Can J Surg 53:184\u2013188","journal-title":"Can J Surg"},{"key":"7833_CR5","doi-asserted-by":"publisher","first-page":"974","DOI":"10.1001\/jama.2017.10476","volume":"318","author":"JB Imran","year":"2017","unstructured":"Imran JB, Eastman AL (2017) Pneumothorax. JAMA 318:974. https:\/\/doi.org\/10.1001\/jama.2017.10476","journal-title":"JAMA"},{"key":"7833_CR6","doi-asserted-by":"publisher","first-page":"578","DOI":"10.1016\/S2213-2600(15)00220-9","volume":"3","author":"OJ Bintcliffe","year":"2015","unstructured":"Bintcliffe OJ, Hallifax RJ, Edey A et al (2015) Spontaneous pneumothorax: time to rethink management? Lancet Respir Med 3:578\u2013588. https:\/\/doi.org\/10.1016\/S2213-2600(15)00220-9","journal-title":"Lancet Respir Med"},{"key":"7833_CR7","doi-asserted-by":"publisher","first-page":"1356","DOI":"10.1016\/j.rmed.2005.11.022","volume":"100","author":"A-M Kelly","year":"2006","unstructured":"Kelly A-M, Weldon D, Tsang AYL, Graham CA (2006) Comparison between two methods for estimating pneumothorax size from chest X-rays. Respir Med 100:1356\u20131359. https:\/\/doi.org\/10.1016\/j.rmed.2005.11.022","journal-title":"Respir Med"},{"key":"7833_CR8","doi-asserted-by":"publisher","first-page":"3462","DOI":"10.1109\/CVPR.2017.369","volume":"2017","author":"X Wang","year":"2017","unstructured":"Wang X, Peng Y, Lu L et al (2017) ChestX-ray8: hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017:3462\u20133471. https:\/\/doi.org\/10.1109\/CVPR.2017.369","journal-title":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR)"},{"key":"7833_CR9","unstructured":"Yao L, Poblenz E, Dagunts D, et al (2017) Learning to diagnose from scratch by exploiting dependencies among labels. arXiv:171010501 [cs]"},{"key":"7833_CR10","unstructured":"CheXNet: Radiologist-level pneumonia detection on chest X-rays with deep learning. https:\/\/stanfordmlgroup.github.io\/projects\/chexnet\/. Accessed 29 Oct 2018"},{"key":"7833_CR11","doi-asserted-by":"publisher","first-page":"3660","DOI":"10.1007\/s00330-020-06771-3","volume":"30","author":"EJ Hwang","year":"2020","unstructured":"Hwang EJ, Hong JH, Lee KH et al (2020) Deep learning algorithm for surveillance of pneumothorax after lung biopsy: a multicenter diagnostic cohort study. Eur Radiol 30:3660\u20133671. https:\/\/doi.org\/10.1007\/s00330-020-06771-3","journal-title":"Eur Radiol"},{"key":"7833_CR12","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1016\/j.clinimag.2020.01.008","volume":"61","author":"G Kitamura","year":"2020","unstructured":"Kitamura G, Deible C (2020) Retraining an open-source pneumothorax detecting machine learning algorithm for improved performance to medical images. Clin Imaging 61:15\u201319. https:\/\/doi.org\/10.1016\/j.clinimag.2020.01.008","journal-title":"Clin Imaging"},{"key":"7833_CR13","doi-asserted-by":"publisher","first-page":"490","DOI":"10.1007\/s10278-019-00299-9","volume":"33","author":"RW Filice","year":"2020","unstructured":"Filice RW, Stein A, Wu CC et al (2020) Crowdsourcing pneumothorax annotations using machine learning annotations on the NIH chest X-ray dataset. J Digit Imaging 33:490\u2013496. https:\/\/doi.org\/10.1007\/s10278-019-00299-9","journal-title":"J Digit Imaging"},{"key":"7833_CR14","doi-asserted-by":"crossref","unstructured":"Irvin J, Rajpurkar P, Ko M, et al (2019) CheXpert: a large chest radiograph dataset with uncertainty labels and expert comparison. arXiv:190107031 [cs, eess]","DOI":"10.1609\/aaai.v33i01.3301590"},{"key":"7833_CR15","doi-asserted-by":"publisher","first-page":"251S","DOI":"10.1016\/S0197-2456(00)00097-0","volume":"21","author":"JK Gohagan","year":"2000","unstructured":"Gohagan JK, Prorok PC, Hayes RB et al (2000) The Prostate, Lung, Colorectal and Ovarian (PLCO) Cancer Screening Trial of the National Cancer Institute: history, organization, and status. Control Clin Trials 21:251S\u2013272S","journal-title":"Control Clin Trials"},{"key":"7833_CR16","doi-asserted-by":"publisher","first-page":"792","DOI":"10.1097\/RLI.0000000000000707","volume":"55","author":"J Rueckel","year":"2020","unstructured":"Rueckel J, Trappmann L, Schachtner B et al (2020) Impact of confounding thoracic tubes and pleural dehiscence extent on artificial intelligence pneumothorax detection in chest radiographs. Invest Radiol 55:792\u2013798. https:\/\/doi.org\/10.1097\/RLI.0000000000000707","journal-title":"Invest Radiol"},{"key":"7833_CR17","doi-asserted-by":"crossref","unstructured":"Wang X, Peng Y, Lu L, et al (2017) ChestX-ray8: hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. arXiv:170502315 [cs]","DOI":"10.1109\/CVPR.2017.369"},{"key":"7833_CR18","unstructured":"Rajpurkar P, Irvin J, Zhu K, et al (2017) CheXNet: radiologist-level pneumonia detection on chest X-rays with deep learning. arXiv:171105225 [cs, stat]"},{"key":"7833_CR19","unstructured":"Weng X (2020) CheXNet for classification and localization of thoracic diseases. https:\/\/github.com\/arnoweng\/CheXNet."},{"key":"7833_CR20","unstructured":"Guendel S, Ghesu FC, Grbic S, et al (2019) Multi-task learning for chest X-ray abnormality classification on noisy labels. arXiv:190506362 [cs]"},{"key":"7833_CR21","doi-asserted-by":"crossref","unstructured":"Guendel S, Grbic S, Georgescu B, et al (2018) Learning to recognize abnormalities in chest X-rays with location-aware dense networks. arXiv:180304565 [cs]","DOI":"10.1007\/978-3-030-13469-3_88"},{"key":"7833_CR22","doi-asserted-by":"crossref","unstructured":"Huang G, Liu Z, van der Maaten L, Weinberger KQ (2018) Densely connected convolutional networks. arXiv:160806993 [cs]","DOI":"10.1109\/CVPR.2017.243"},{"key":"7833_CR23","doi-asserted-by":"publisher","first-page":"415","DOI":"10.1097\/HCR.0b013e31818c3c78","volume":"28","author":"S Mandic","year":"2008","unstructured":"Mandic S, Go C, Aggarwal I et al (2008) Relationship of predictive modeling to receiver operating characteristics. J Cardiopulm Rehabil Prev 28:415\u2013419. https:\/\/doi.org\/10.1097\/HCR.0b013e31818c3c78","journal-title":"J Cardiopulm Rehabil Prev"},{"key":"7833_CR24","doi-asserted-by":"publisher","first-page":"1315","DOI":"10.1097\/JTO.0b013e3181ec173d","volume":"5","author":"JN Mandrekar","year":"2010","unstructured":"Mandrekar JN (2010) Receiver operating characteristic curve in diagnostic test assessment. J Thorac Oncol 5:1315\u20131316. https:\/\/doi.org\/10.1097\/JTO.0b013e3181ec173d","journal-title":"J Thorac Oncol"},{"key":"7833_CR25","doi-asserted-by":"publisher","first-page":"837","DOI":"10.2307\/2531595","volume":"44","author":"ER DeLong","year":"1988","unstructured":"DeLong ER, DeLong DM, Clarke-Pearson DL (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44:837\u2013845","journal-title":"Biometrics"},{"key":"7833_CR26","doi-asserted-by":"publisher","first-page":"1389","DOI":"10.1109\/LSP.2014.2337313","volume":"21","author":"X Sun","year":"2014","unstructured":"Sun X, Xu W (2014) Fast implementation of DeLong\u2019s algorithm for comparing the areas under correlated receiver operating characteristic curves. IEEE Signal Process Lett 21:1389\u20131393. https:\/\/doi.org\/10.1109\/LSP.2014.2337313","journal-title":"IEEE Signal Process Lett"},{"key":"7833_CR27","doi-asserted-by":"publisher","unstructured":"Philbrick KA, Weston AD, Akkus Z et al (2019) RIL-Contour: a medical imaging dataset annotation tool for and with deep learning. J Digit Imaging 32:571\u2013581. https:\/\/doi.org\/10.1007\/s10278-019-00232-0","DOI":"10.1007\/s10278-019-00232-0"},{"key":"7833_CR28","doi-asserted-by":"crossref","unstructured":"Wang Z, Qinami K, Karakozis IC, et al (2020) Towards fairness in visual recognition: effective strategies for bias mitigation. arXiv:191111834 [cs]","DOI":"10.1109\/CVPR42600.2020.00894"},{"key":"7833_CR29","doi-asserted-by":"crossref","unstructured":"Wang A, Narayanan A, Russakovsky O (2020) ViBE: a tool for measuring and mitigating bias in image datasets. arXiv:200407999 [cs]","DOI":"10.1007\/978-3-030-58580-8_43"},{"key":"7833_CR30","doi-asserted-by":"crossref","unstructured":"Alvi M, Zisserman A, Nellaker C (2018) Turning a blind eye: explicit removal of biases and variation from deep neural network embeddings. arXiv:180902169 [cs]","DOI":"10.1007\/978-3-030-11009-3_34"},{"key":"7833_CR31","doi-asserted-by":"publisher","first-page":"e1002697","DOI":"10.1371\/journal.pmed.1002697","volume":"15","author":"AG Taylor","year":"2018","unstructured":"Taylor AG, Mielke C, Mongan J (2018) Automated detection of moderate and large pneumothorax on frontal chest X-rays using deep convolutional neural networks: a retrospective study. PLoS Med 15:e1002697. https:\/\/doi.org\/10.1371\/journal.pmed.1002697","journal-title":"PLoS Med"},{"key":"7833_CR32","doi-asserted-by":"publisher","first-page":"2038","DOI":"10.1016\/j.ejrad.2015.07.003","volume":"84","author":"A Thelle","year":"2015","unstructured":"Thelle A, Gjerdevik M, Grydeland T et al (2015) Pneumothorax size measurements on digital chest radiographs: intra- and inter- rater reliability. Eur J Radiol 84:2038\u20132043. https:\/\/doi.org\/10.1016\/j.ejrad.2015.07.003","journal-title":"Eur J Radiol"}],"container-title":["European Radiology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00330-021-07833-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00330-021-07833-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00330-021-07833-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,9,20]],"date-time":"2021-09-20T14:43:38Z","timestamp":1632149018000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00330-021-07833-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,27]]},"references-count":32,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2021,10]]}},"alternative-id":["7833"],"URL":"https:\/\/doi.org\/10.1007\/s00330-021-07833-w","relation":{},"ISSN":["0938-7994","1432-1084"],"issn-type":[{"value":"0938-7994","type":"print"},{"value":"1432-1084","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,3,27]]},"assertion":[{"value":"5 September 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 January 2021","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 February 2021","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 March 2021","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Dr. Johannes R\u00fcckel, B.Sc.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Guarantor"}},{"value":"LMU Department of Radiology received research grant from Siemens Healthineers. Johannes Rueckel and Bastian Sabel received funding for speaker\u2019s activity by Siemens Healthineers.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"No complex statistical methods were necessary for this paper. Several authors have significant expertise for the applied methods.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Statistics and biometry"}},{"value":"Does not apply\u2014retrospective study.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Informed consent"}},{"value":"Approval of the institutional ethics commission was obtained for this study.","order":6,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}},{"value":"Study subjects have been partially reported:Johannes Rueckel, Lena Trappmann, Balthasar Schachtner, et al: Impact of Confounding Thoracic Tubes and Pleural Dehiscence Extent on Artificial Intelligence Pneumothorax Detection in Chest Radiographs. Investigative Radiology 2020","order":7,"name":"Ethics","group":{"name":"EthicsHeading","label":"Study subjects or cohort overlap"}},{"value":"\u2022 retrospective\u2022 diagnostic study\u2022 performed at one institution","order":8,"name":"Ethics","group":{"name":"EthicsHeading","label":"Methodology"}}]}}