{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,13]],"date-time":"2026-09-13T23:01:02Z","timestamp":1789340462099,"version":"build-2803163510"},"reference-count":33,"publisher":"BMJ","issue":"2","license":[{"start":{"date-parts":[[2024,2,12]],"date-time":"2024-02-12T00:00:00Z","timestamp":1707696000000},"content-version":"unspecified","delay-in-days":11,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"name":"NHSX AI in Health and Care Award","award":["AI_AWARD02354"],"award-info":[{"award-number":["AI_AWARD02354"]}]}],"content-domain":{"domain":["bmj.com"],"crossmark-restriction":true},"short-container-title":["BMJ Open"],"accepted":{"date-parts":[[2024,1,28]]},"published-print":{"date-parts":[[2024,2]]},"abstract":"<jats:sec>\n                    <jats:title>Introduction<\/jats:title>\n                    <jats:p>A non-contrast CT head scan (NCCTH) is the most common cross-sectional imaging investigation requested in the emergency department. Advances in computer vision have led to development of several artificial intelligence (AI) tools to detect abnormalities on NCCTH. These tools are intended to provide clinical decision support for clinicians, rather than stand-alone diagnostic devices. However, validation studies mostly compare AI performance against radiologists, and there is relative paucity of evidence on the impact of AI assistance on other healthcare staff who review NCCTH in their daily clinical practice.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods and analysis<\/jats:title>\n                    <jats:p>A retrospective data set of 150 NCCTH will be compiled, to include 60 control cases and 90 cases with intracranial haemorrhage, hypodensities suggestive of infarct, midline shift, mass effect or skull fracture. The intracranial haemorrhage cases will be subclassified into extradural, subdural, subarachnoid, intraparenchymal and intraventricular. 30 readers will be recruited across four National Health Service (NHS) trusts including 10 general radiologists, 15 emergency medicine clinicians and 5 CT radiographers of varying experience. Readers will interpret each scan first without, then with, the assistance of the qER EU 2.0 AI tool, with an intervening 2-week washout period. Using a panel of neuroradiologists as ground truth, the stand-alone performance of qER will be assessed, and its impact on the readers\u2019 performance will be analysed as change in accuracy (area under the curve), median review time per scan and self-reported diagnostic confidence. Subgroup analyses will be performed by reader professional group, reader seniority, pathological finding, and neuroradiologist-rated difficulty.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Ethics and dissemination<\/jats:title>\n                    <jats:p>The study has been approved by the UK Healthcare Research Authority (IRAS 310995, approved 13 December 2022). The use of anonymised retrospective NCCTH has been authorised by Oxford University Hospitals. The results will be presented at relevant conferences and published in a peer-reviewed journal.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Trial registration number<\/jats:title>\n                    <jats:p>\n                      <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"clintrialgov\" xlink:href=\"NCT06018545\">NCT06018545<\/jats:ext-link>\n                      .\n                    <\/jats:p>\n                  <\/jats:sec>","DOI":"10.1136\/bmjopen-2023-079824","type":"journal-article","created":{"date-parts":[[2024,2,12]],"date-time":"2024-02-12T21:11:34Z","timestamp":1707772294000},"page":"e079824","update-policy":"https:\/\/doi.org\/10.1136\/crossmarkpolicy","source":"Crossref","is-referenced-by-count":11,"title":["AI assisted reader evaluation in acute CT head interpretation (AI-REACT): protocol for a multireader multicase study"],"prefix":"10.1136","volume":"14","clinical-trial-number":[{"clinical-trial-number":"nct06018545","registry":"10.18810\/clinical-trials-gov"}],"author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0582-9518","authenticated-orcid":false,"given":"Howell","family":"Fu","sequence":"first","affiliation":[{"name":"Oxford University Hospitals NHS Foundation Trust, Oxford, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alex","family":"Novak","sequence":"additional","affiliation":[{"name":"Emergency Medicine Research Oxford, Oxford University Hospitals NHS Foundation Trust, Oxford, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dennis","family":"Robert","sequence":"additional","affiliation":[{"name":"Qure.AI, Bangalore, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shamie","family":"Kumar","sequence":"additional","affiliation":[{"name":"Qure.AI, Bangalore, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Swetha","family":"Tanamala","sequence":"additional","affiliation":[{"name":"Qure.AI, Bangalore, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jason","family":"Oke","sequence":"additional","affiliation":[{"name":"Department of Primary Care Health Sciences, University of Oxford, Oxford, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kanika","family":"Bhatia","sequence":"additional","affiliation":[{"name":"Oxford University Hospitals NHS Foundation Trust, Oxford, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ruchir","family":"Shah","sequence":"additional","affiliation":[{"name":"Oxford University Hospitals NHS Foundation Trust, Oxford, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrea","family":"Romsauerova","sequence":"additional","affiliation":[{"name":"Oxford University Hospitals NHS Foundation Trust, Oxford, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tilak","family":"Das","sequence":"additional","affiliation":[{"name":"Department of Clinical Radiology, Cambridge University Hospitals NHS Foundation Trust, Cambridge, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Abdal\u00e1","family":"Espinosa","sequence":"additional","affiliation":[{"name":"Emergency Medicine Research Oxford, Oxford University Hospitals NHS Foundation Trust, Oxford, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mariusz Tadeusz","family":"Grzeda","sequence":"additional","affiliation":[{"name":"School of Biomedical Science, King\u2019s College London, London, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mariapaola","family":"Narbone","sequence":"additional","affiliation":[{"name":"Guy\u2019s and St Thomas\u2019 Hospitals NHS Trust, London, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rahul","family":"Dharmadhikari","sequence":"additional","affiliation":[{"name":"Northumbria Healthcare NHS Foundation Trust, Northumberland, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mark","family":"Harrison","sequence":"additional","affiliation":[{"name":"Emergency Department, Northumbria Specialist Emergency Care Hospital, Cramlington, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2236-7279","authenticated-orcid":false,"given":"Kavitha","family":"Vimalesvaran","sequence":"additional","affiliation":[{"name":"Clinical Scientific Computing, Guy\u2019s and St Thomas\u2019 NHS Foundation Trust, London, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jane","family":"Gooch","sequence":"additional","affiliation":[{"name":"College of Health, Psychology & Social Care, University of Derby, Derby, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9598-189X","authenticated-orcid":false,"given":"Nicholas","family":"Woznitza","sequence":"additional","affiliation":[{"name":"Radiology Department, University College London Hospitals NHS Foundation Trust, London, UK"},{"name":"School of Allied and Public Health Professions, Canterbury Christ Church University, Canterbury, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nabeeha","family":"Salik","sequence":"additional","affiliation":[{"name":"RAIQC Ltd, Oxford, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alan","family":"Campbell","sequence":"additional","affiliation":[{"name":"Radiology Department, University College London Hospitals NHS Foundation Trust, London, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Farhaan","family":"Khan","sequence":"additional","affiliation":[{"name":"Oxford University Hospitals NHS Foundation Trust, Oxford, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4866-2049","authenticated-orcid":false,"given":"David J","family":"Lowe","sequence":"additional","affiliation":[{"name":"NHS Greater Glasgow and Clyde, Glasgow, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Haris","family":"Shuaib","sequence":"additional","affiliation":[{"name":"Clinical Scientific Computing, Guy\u2019s and St Thomas\u2019 NHS Foundation Trust, London, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sarim","family":"Ather","sequence":"additional","affiliation":[{"name":"Oxford University Hospitals NHS Foundation Trust, Oxford, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"239","published-online":{"date-parts":[[2024,2,12]]},"reference":[{"key":"2025101813242383000_14.2.e079824.1","doi-asserted-by":"crossref","first-page":"1085","DOI":"10.1111\/ans.15285","article-title":"Reduction in hospital admissions with an early computed tomography scan: results of an outpatient management protocol for uncomplicated acute diverticulitis","volume":"89","author":"Juszczyk","year":"2019","journal-title":"ANZ J Surg"},{"key":"2025101813242383000_14.2.e079824.2","first-page":"25","article-title":"Pre-operative imaging can reduce negative appendectomy rate in acute appendicitis","volume":"89","author":"Chan","year":"2020","journal-title":"Ulster Med J"},{"key":"2025101813242383000_14.2.e079824.3","doi-asserted-by":"crossref","first-page":"224","DOI":"10.1016\/j.crad.2019.10.025","article-title":"Royal College of Radiologists national audit evaluating the provision of imaging in the severely injured patient and compliance with national guidelines","volume":"75","author":"Greenhalgh","year":"2020","journal-title":"Clin Radiol"},{"key":"2025101813242383000_14.2.e079824.4","unstructured":"Richards M . Diagnostics: recovery and renewal \u2013 report of the independent review of diagnostic services for NHS England [NHS England]. 2022. Available: https:\/\/www.england.nhs.uk\/publication\/diagnostics-recovery-and-renewal-report-of-the-independent-review-of-diagnostic-services-for-nhs-england\/ [Accessed 6 May 2023]."},{"key":"2025101813242383000_14.2.e079824.5","unstructured":"Royal College of Radiologists . Clinical Radiology UK workforce census 2019 report. 2020. Available: https:\/\/www.rcr.ac.uk\/publication\/clinical-radiology-uk-workforce-census-2019-report [Accessed 6 May 2023]."},{"key":"2025101813242383000_14.2.e079824.6","doi-asserted-by":"publisher","DOI":"10.1016\/j.carj.2018.10.004"},{"key":"2025101813242383000_14.2.e079824.7","doi-asserted-by":"publisher","DOI":"10.1016\/j.amjmed.2011.06.023"},{"key":"2025101813242383000_14.2.e079824.8","doi-asserted-by":"crossref","DOI":"10.3389\/fneur.2022.791816","article-title":"Computational Approaches for Acute Traumatic Brain Injury Image Recognition","volume":"13","author":"Lin","year":"2022","journal-title":"Front Neurol"},{"key":"2025101813242383000_14.2.e079824.9","doi-asserted-by":"publisher","DOI":"10.1136\/neurintsurg-2021-018142"},{"key":"2025101813242383000_14.2.e079824.10","doi-asserted-by":"publisher","DOI":"10.1136\/neurintsurg-2020-017099"},{"key":"2025101813242383000_14.2.e079824.11","doi-asserted-by":"publisher","DOI":"10.1016\/s0140-6736(18)31645-3"},{"key":"2025101813242383000_14.2.e079824.12","doi-asserted-by":"crossref","first-page":"e584","DOI":"10.1016\/S2589-7500(22)00090-5","article-title":"Deep learning with weak annotation from diagnosis reports for detection of multiple head disorders: a prospective, multicentre study","volume":"4","author":"Guo","year":"2022","journal-title":"Lancet Digit Health"},{"key":"2025101813242383000_14.2.e079824.13","doi-asserted-by":"crossref","first-page":"20546","DOI":"10.1038\/s41598-020-77441-z","article-title":"Detection and classification of intracranial haemorrhage on CT images using a novel deep-learning algorithm","volume":"10","author":"Lee","year":"2020","journal-title":"Sci Rep"},{"key":"2025101813242383000_14.2.e079824.14","doi-asserted-by":"crossref","DOI":"10.1038\/s41746-017-0015-z","article-title":"Advanced machine learning in action: identification of intracranial hemorrhage on computed tomography scans of the head with clinical workflow integration","volume":"1","author":"Arbabshirani","year":"2018","journal-title":"NPJ Digit Med"},{"key":"2025101813242383000_14.2.e079824.15","doi-asserted-by":"publisher","DOI":"10.1067\/j.cpradiol.2020.10.007"},{"key":"2025101813242383000_14.2.e079824.16","unstructured":"National Institute for Health and Care Excellence . Artificial intelligence for analysing CT brain scans. 2020. Available: https:\/\/www.nice.org.uk\/advice\/mib207 [Accessed 6 May 2023]."},{"key":"2025101813242383000_14.2.e079824.17","doi-asserted-by":"publisher","DOI":"10.1161\/STROKEAHA.121.036204"},{"key":"2025101813242383000_14.2.e079824.18","doi-asserted-by":"crossref","unstructured":"Finck T , Moosbauer J , Probst M , et al . Faster and better: how anomaly detection can accelerate and improve reporting of head computed tomography. Diagnostics;12:452. doi:10.3390\/diagnostics12020452","DOI":"10.3390\/diagnostics12020452"},{"key":"2025101813242383000_14.2.e079824.19","article-title":"Deep learning system boosts radiologist detection of intracranial hemorrhage","volume":"14","author":"Warman","year":"2022","journal-title":"Cureus"},{"key":"2025101813242383000_14.2.e079824.20","doi-asserted-by":"crossref","first-page":"534","DOI":"10.1159\/000339116","article-title":"Ability of emergency physicians to detect early ischemic changes of acute ischemic stroke on cranial computed tomography","volume":"21","author":"\u00dcnl\u00fcer","year":"2012","journal-title":"Med Princ Pract"},{"key":"2025101813242383000_14.2.e079824.21","doi-asserted-by":"crossref","first-page":"86","DOI":"10.5812\/traumamon.12023","article-title":"Interpretation of computed tomography of the head: emergency physicians versus radiologists","volume":"18","author":"Dolatabadi","year":"2013","journal-title":"Trauma Mon"},{"key":"2025101813242383000_14.2.e079824.22","doi-asserted-by":"crossref","unstructured":"Scavasine VC , Ferreti LA , Costa RT , et al . Artificial intelligence in the emergency room: how E-ASPECTS helps emergency physicians evaluate brain CT of patients with acute ischemic stroke. SSRN Journal 2022. doi:10.2139\/ssrn.4062896","DOI":"10.2139\/ssrn.4062896"},{"key":"2025101813242383000_14.2.e079824.23","doi-asserted-by":"publisher","DOI":"10.1007\/s00234-021-02826-4"},{"key":"2025101813242383000_14.2.e079824.24","doi-asserted-by":"crossref","first-page":"106702","DOI":"10.1016\/j.jstrokecerebrovasdis.2022.106702","article-title":"Comparison of automated ASPECTS, large vessel occlusion detection and CTP analysis provided by Brainomix and RapidAI in patients with suspected ischaemic stroke","volume":"31","author":"Mallon","year":"2022","journal-title":"J Stroke Cerebrovasc Dis"},{"key":"2025101813242383000_14.2.e079824.25","doi-asserted-by":"crossref","first-page":"e313","DOI":"10.1016\/j.crad.2022.10.017","article-title":"Diagnostic accuracy of a decision-support software for the detection of intracranial large-vessel occlusion in CT angiography","volume":"78","author":"Andralojc","year":"2023","journal-title":"Clin Radiol"},{"key":"2025101813242383000_14.2.e079824.26","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pmed.1002683","article-title":"Variable generalization performance of A deep learning model to detect pneumonia in chest radiographs: A cross-sectional study","volume":"15","author":"Zech","year":"2018","journal-title":"PLOS Med"},{"key":"2025101813242383000_14.2.e079824.27","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1038\/s41746-023-00811-0","article-title":"Self-supervised learning for medical image classification: a systematic review and implementation guidelines","volume":"6","author":"Huang","year":"2023","journal-title":"NPJ Digit Med"},{"key":"2025101813242383000_14.2.e079824.28","doi-asserted-by":"publisher","DOI":"10.1007\/s00234-022-03075-9"},{"key":"2025101813242383000_14.2.e079824.29","unstructured":"US Food and Drug Administration . FDA marketing approval K200921. 2020. Available: https:\/\/www.accessdata.fda.gov\/cdrh_docs\/pdf20\/K200921.pdf [Accessed 1 Dec 2023]."},{"key":"2025101813242383000_14.2.e079824.30","doi-asserted-by":"publisher","DOI":"10.2214\/ajr.175.3.1750603"},{"key":"2025101813242383000_14.2.e079824.31","doi-asserted-by":"publisher","DOI":"10.1080\/03610919508813243"},{"key":"2025101813242383000_14.2.e079824.32","doi-asserted-by":"publisher","DOI":"10.1016\/S1076-6332(99)80468-1"},{"key":"2025101813242383000_14.2.e079824.33","doi-asserted-by":"publisher","DOI":"10.1002\/sim.2024"}],"container-title":["BMJ Open"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.1136\/bmjopen-2023-079824","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,18]],"date-time":"2025-10-18T20:24:37Z","timestamp":1760819077000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmjopen.bmj.com\/lookup\/doi\/10.1136\/bmjopen-2023-079824"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2]]},"references-count":33,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2024,2,12]]},"published-print":{"date-parts":[[2024,2]]}},"alternative-id":["10.1136\/bmjopen-2023-079824"],"URL":"https:\/\/doi.org\/10.1136\/bmjopen-2023-079824","relation":{},"ISSN":["2044-6055","2044-6055"],"issn-type":[{"value":"2044-6055","type":"print"},{"value":"2044-6055","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2]]}}}