{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,4]],"date-time":"2026-07-04T18:02:17Z","timestamp":1783188137864,"version":"3.54.6"},"reference-count":21,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2022,10,25]],"date-time":"2022-10-25T00:00:00Z","timestamp":1666656000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,10,25]],"date-time":"2022-10-25T00:00:00Z","timestamp":1666656000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Eur Radiol"],"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Objectives<\/jats:title>\n                <jats:p>To quantify reader agreement for the British Society of Thoracic Imaging (BSTI) diagnostic and severity classification for COVID-19 on chest radiographs (CXR), in particular agreement for an indeterminate CXR that could instigate CT imaging, from single and paired images.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>Twenty readers (four groups of five individuals)\u2014consultant chest (CCR), general consultant (GCR), and specialist registrar (RSR) radiologists, and infectious diseases clinicians (IDR)\u2014assigned BSTI categories and severity in addition to modified Covid-Radiographic Assessment of Lung Edema Score (Covid-RALES), to 305 CXRs (129 paired; 2 time points) from 176 guideline-defined COVID-19 patients. Percentage agreement with a consensus of two chest radiologists was calculated for (1) categorisation to those needing CT (indeterminate) versus those that did not (classic\/probable, non-COVID-19); (2) severity; and (3) severity change on paired CXRs using the two scoring systems.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>Agreement with consensus for the indeterminate category was low across all groups (28\u201337%). Agreement for other BSTI categories was highest for classic\/probable for the other three reader groups (66\u201376%) compared to GCR (49%). Agreement for normal was similar across all radiologists (54\u201361%) but lower for IDR (31%). Agreement for a severe CXR was lower for GCR (65%), compared to the other three reader groups (84\u201395%). For all groups, agreement for changes across paired CXRs was modest.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusion<\/jats:title>\n                <jats:p>Agreement for the indeterminate BSTI COVID-19 CXR category is low, and generally moderate for the other BSTI categories and for severity change, suggesting that the test, rather than readers, is limited in utility for both deciding disposition and serial monitoring.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Key Points<\/jats:title>\n                <jats:p><jats:italic>\u2022 Across different reader groups, agreement for COVID-19 diagnostic categorisation on CXR varies widely.<\/jats:italic><\/jats:p>\n                <jats:p><jats:italic>\u2022 Agreement varies to a degree that may render CXR alone ineffective for triage, especially for indeterminate cases.<\/jats:italic><\/jats:p>\n                <jats:p><jats:italic>\u2022 Agreement for serial CXR change is moderate, limiting utility in guiding management.<\/jats:italic><\/jats:p>\n              <\/jats:sec>","DOI":"10.1007\/s00330-022-09172-w","type":"journal-article","created":{"date-parts":[[2022,10,25]],"date-time":"2022-10-25T06:03:05Z","timestamp":1666677785000},"page":"2096-2104","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Chest radiograph classification and severity of suspected COVID-19 by different radiologist groups and attending clinicians: multi-reader, multi-case study"],"prefix":"10.1007","volume":"33","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9270-3771","authenticated-orcid":false,"given":"Arjun","family":"Nair","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alexander","family":"Procter","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Steve","family":"Halligan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thomas","family":"Parry","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Asia","family":"Ahmed","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mark","family":"Duncan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Magali","family":"Taylor","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Manil","family":"Chouhan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Trevor","family":"Gaunt","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"James","family":"Roberts","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Niels","family":"van Vucht","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alan","family":"Campbell","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Laura May","family":"Davis","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joseph","family":"Jacob","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rachel","family":"Hubbard","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shankar","family":"Kumar","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ammaarah","family":"Said","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xinhui","family":"Chan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tim","family":"Cutfield","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Akish","family":"Luintel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael","family":"Marks","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Neil","family":"Stone","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sue","family":"Mallet","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,10,25]]},"reference":[{"issue":"5","key":"9172_CR1","doi-asserted-by":"publisher","first-page":"329","DOI":"10.1016\/j.crad.2020.03.008","volume":"75","author":"A Nair","year":"2020","unstructured":"Nair A, Rodrigues JCL, Hare S et al (2020) A British Society of Thoracic Imaging statement: considerations in designing local imaging diagnostic algorithms for the COVID-19 pandemic. Clin Radiol 75(5):329\u2013334","journal-title":"Clin Radiol"},{"issue":"19","key":"9172_CR2","first-page":"1861","volume":"382","author":"WJ Guan","year":"2020","unstructured":"Guan WJ, Zhong NS (2020) Clinical characteristics of Covid-19 in China. Reply N Engl J Med 382(19):1861\u20131862","journal-title":"Reply N Engl J Med"},{"issue":"2","key":"9172_CR3","doi-asserted-by":"publisher","first-page":"E72","DOI":"10.1148\/radiol.2020201160","volume":"296","author":"HYF Wong","year":"2020","unstructured":"Wong HYF, Lam HYS, Fong AH et al (2020) Frequency and distribution of chest radiographic findings in patients positive for COVID-19. Radiology. 296(2):E72\u2013EE8","journal-title":"Radiology."},{"issue":"8","key":"9172_CR4","doi-asserted-by":"publisher","first-page":"1081","DOI":"10.1001\/jamainternmed.2020.2033","volume":"180","author":"W Liang","year":"2020","unstructured":"Liang W, Liang H, Ou L et al (2020) Development and validation of a clinical risk score to predict the occurrence of critical illness in hospitalized patients with COVID-19. JAMA Intern Med 180(8):1081\u20131089","journal-title":"JAMA Intern Med"},{"key":"9172_CR5","unstructured":"Public Health England(2020) COVID-19: investigation and initial clinical management of possible cases [updated 12\/14\/2020. Available from: https:\/\/www.gov.uk\/government\/publications\/wuhan-novel-coronavirus-initial-investigation-of-possible-cases\/investigation-and-initial-clinical-management-of-possible-cases-of-wuhan-novel-coronavirus-wn-cov-infection. Accessed 23 Dec\u00a02021"},{"issue":"4","key":"9172_CR6","doi-asserted-by":"publisher","first-page":"W105","DOI":"10.1097\/RTI.0000000000000533","volume":"35","author":"S Schiaffino","year":"2020","unstructured":"Schiaffino S, Tritella S, Cozzi A et al (2020) Diagnostic performance of chest X-ray for COVID-19 pneumonia during the SARS-CoV-2 pandemic in Lombardy, Italy. J Thorac Imaging 35(4):W105\u2013W1W6","journal-title":"J Thorac Imaging"},{"issue":"12","key":"9172_CR7","doi-asserted-by":"publisher","first-page":"1271","DOI":"10.1007\/s11547-020-01272-1","volume":"125","author":"M Gatti","year":"2020","unstructured":"Gatti M, Calandri M, Barba M et al (2020) Baseline chest X-ray in coronavirus disease 19 (COVID-19) patients: association with clinical and laboratory data. Radiol Med 125(12):1271\u20131279","journal-title":"Radiol Med"},{"issue":"2","key":"9172_CR8","doi-asserted-by":"publisher","first-page":"822","DOI":"10.4269\/ajtmh.20-0535","volume":"103","author":"MA Orsi","year":"2020","unstructured":"Orsi MA, Oliva G, Toluian T, Valenti PC, Panzeri M, Cellina M (2020) Feasibility, reproducibility, and clinical validity of a quantitative Chest X-ray assessment for COVID-19. Am J Trop Med Hyg 103(2):822\u2013827","journal-title":"Am J Trop Med Hyg"},{"issue":"1","key":"9172_CR9","doi-asserted-by":"publisher","first-page":"E197","DOI":"10.1148\/radiol.2020201754","volume":"297","author":"D Toussie","year":"2020","unstructured":"Toussie D, Voutsinas N, Finkelstein M et al (2020) Clinical and chest radiography features determine patient outcomes in young and middle-aged adults with COVID-19. Radiology. 297(1):E197\u2013E206","journal-title":"Radiology."},{"issue":"4","key":"9172_CR10","doi-asserted-by":"publisher","first-page":"1999","DOI":"10.1007\/s00330-020-07270-1","volume":"31","author":"M Balbi","year":"2021","unstructured":"Balbi M, Caroli A, Corsi A et al (2021) Chest X-ray for predicting mortality and the need for ventilatory support in COVID-19 patients presenting to the emergency department. Eur Radiol 31(4):1999\u20132012","journal-title":"Eur Radiol"},{"issue":"3","key":"9172_CR11","doi-asserted-by":"publisher","first-page":"E166","DOI":"10.1148\/radiol.2020201874","volume":"296","author":"K Murphy","year":"2020","unstructured":"Murphy K, Smits H, Knoops AJG et al (2020) COVID-19 on chest radiographs: a multireader evaluation of an artificial intelligence system. Radiology. 296(3):E166\u2013EE72","journal-title":"Radiology."},{"issue":"1","key":"9172_CR12","doi-asserted-by":"publisher","first-page":"858","DOI":"10.1038\/s41598-020-79470-0","volume":"11","author":"S Ebrahimian","year":"2021","unstructured":"Ebrahimian S, Homayounieh F, Rockenbach MABC et al (2021) Artificial intelligence matches subjective severity assessment of pneumonia for prediction of patient outcome and need for mechanical ventilation: a cohort study. Sci Rep 11(1):858","journal-title":"Sci Rep"},{"issue":"11","key":"9172_CR13","doi-asserted-by":"publisher","first-page":"e0242759","DOI":"10.1371\/journal.pone.0242759","volume":"15","author":"SB Jang","year":"2020","unstructured":"Jang SB, Lee SH, Lee DE et al (2020) Deep-learning algorithms for the interpretation of chest radiographs to aid in the triage of COVID-19 patients: A multicenter retrospective study. PLoS One 15(11):e0242759","journal-title":"PLoS One"},{"issue":"9","key":"9172_CR14","doi-asserted-by":"publisher","first-page":"710","DOI":"10.1016\/j.crad.2020.06.005","volume":"75","author":"SS Hare","year":"2020","unstructured":"Hare SS, Tavare AN, Dattani V et al (2020) Validation of the British Society of Thoracic Imaging guidelines for COVID-19 chest radiograph reporting. Clin Radiol 75(9):710\u20137e9","journal-title":"Clin Radiol"},{"key":"9172_CR15","doi-asserted-by":"publisher","first-page":"109272","DOI":"10.1016\/j.ejrad.2020.109272","volume":"132","author":"A Cozzi","year":"2020","unstructured":"Cozzi A, Schiaffino S, Arpaia F et al (2020) Chest x-ray in the COVID-19 pandemic: radiologists' real-world reader performance. Eur J Radiol 132:109272","journal-title":"Eur J Radiol"},{"key":"9172_CR16","doi-asserted-by":"publisher","unstructured":"Reeves RA, Pomeranz C, Gomella AA et al (2021) Performance of a severity score on admission chest radiography in predicting clinical outcomes in hospitalized patients with coronavirus disease (COVID-19). Am J Roentgenol 217(3):623\u2013632. https:\/\/doi.org\/10.2214\/AJR.20.24801","DOI":"10.2214\/AJR.20.24801"},{"issue":"1","key":"9172_CR17","first-page":"20200020","volume":"2","author":"OJ Kemp","year":"2020","unstructured":"Kemp OJ, Watson DJ, Swanson-Low CL, Cameron JA, Von Vopelius-Feldt J (2020) Comparison of chest X-ray interpretation by Emergency Department clinicians and radiologists in suspected COVID-19 infection: a retrospective cohort study. BJR Open 2(1):20200020","journal-title":"BJR Open"},{"key":"9172_CR18","doi-asserted-by":"publisher","unstructured":"Islam N, Ebrahimzadeh S, Salameh J-P et al (2021) Thoracic imaging tests for the diagnosis of COVID\u201019. Cochrane Database Syst Rev 3(3):CD013639. https:\/\/doi.org\/10.1002\/14651858.CD013639.pub4","DOI":"10.1002\/14651858.CD013639.pub4"},{"issue":"1","key":"9172_CR19","doi-asserted-by":"publisher","first-page":"346","DOI":"10.1186\/s12916-020-01810-8","volume":"18","author":"S Mallett","year":"2020","unstructured":"Mallett S, Allen AJ, Graziadio S et al (2020) At what times during infection is SARS-CoV-2 detectable and no longer detectable using RT-PCR-based tests? A systematic review of individual participant data. BMC Med 18(1):346","journal-title":"BMC Med"},{"issue":"1","key":"9172_CR20","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s43055-021-00541-x","volume":"52","author":"A \u00c7inkoo\u011flu","year":"2021","unstructured":"\u00c7inkoo\u011flu A, Bayraktaro\u011flu S, Ceylan N, Sava\u015f R (2021) Efficacy of chest X-ray in the diagnosis of COVID-19 pneumonia: comparison with computed tomography through a simplified scoring system designed for triage. Egypt J Radiol Nucl Med 52(1):1\u20139","journal-title":"Egypt J Radiol Nucl Med"},{"issue":"2","key":"9172_CR21","doi-asserted-by":"publisher","first-page":"2001809","DOI":"10.1183\/13993003.01809-2020","volume":"56","author":"J Jacob","year":"2020","unstructured":"Jacob J, Alexander D, Baillie JK et al (2020) Using imaging to combat a pandemic: rationale for developing the UK National COVID-19 Chest Imaging Database. Eur Respir J 56(2):2001809","journal-title":"Eur Respir J"}],"container-title":["European Radiology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00330-022-09172-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00330-022-09172-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00330-022-09172-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,2,16]],"date-time":"2023-02-16T18:08:12Z","timestamp":1676570892000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00330-022-09172-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,25]]},"references-count":21,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2023,3]]}},"alternative-id":["9172"],"URL":"https:\/\/doi.org\/10.1007\/s00330-022-09172-w","relation":{},"ISSN":["1432-1084"],"issn-type":[{"value":"1432-1084","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,10,25]]},"assertion":[{"value":"28 December 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 July 2022","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 August 2022","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 October 2022","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The scientific guarantor of this publication is Arjun Nair (arjun.nair1@nhs.net)","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Guarantor"}},{"value":"The authors of this manuscript declare relationships with the following companies:AN reports a medical advisory role with Aidence BV, an artificial intelligence company; AN reports a consultation role with Faculty Science Limited, an artificial intelligence company.SH and SM report grants from the National Institute for Health Research (NIHR) outside the submitted work.JJ reports Consultancy fees from Boehringer Ingelheim, F. Hoffmann-La Roche, GlaxoSmithKline, NHSX; is on the Advisory Boards of Boehringer Ingelheim, F. Hoffmann-La Roche; has received lecture fees from Boehringer Ingelheim, F. Hoffmann-La Roche, Takeda; receives grant funding from GlaxoSmithKline; holds a UK patent (application number 2113765.8).The other authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"One of the authors has significant statistical expertise- Professor Sue Mallet is a Professor in diagnostic and prognostic medical statistics, specialising in clinical trial design, methodology, and systematic reviews. She is a senior author of the paper.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Statistics and biometry"}},{"value":"Written informed consent was waived by the Institutional Review Board, as per our ethics approval.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Informed consent"}},{"value":"Institutional Review Board approval was obtained- Our institution granted ethical approval for COVID-19-related imaging studies (Integrated Research Application Service reference IRAS 282063).","order":6,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}},{"value":"\u2022 retrospective\u2022 observational\u2022 performed at one institution","order":7,"name":"Ethics","group":{"name":"EthicsHeading","label":"Methodology"}}]}}