{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T21:01:15Z","timestamp":1783630875206,"version":"3.55.0"},"reference-count":17,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2019,9,9]],"date-time":"2019-09-09T00:00:00Z","timestamp":1567987200000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2019,9,9]],"date-time":"2019-09-09T00:00:00Z","timestamp":1567987200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100012338","name":"Alan Turing Institute","doi-asserted-by":"crossref","award":["EP\/N510129\/1"],"award-info":[{"award-number":["EP\/N510129\/1"]}],"id":[{"id":"10.13039\/100012338","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100012338","name":"Alan Turing Institute","doi-asserted-by":"publisher","award":["EP\/N510129\/1"],"award-info":[{"award-number":["EP\/N510129\/1"]}],"id":[{"id":"10.13039\/100012338","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100004440","name":"Wellcome Trust","doi-asserted-by":"publisher","award":["203769\/Z\/16\/Z"],"award-info":[{"award-number":["203769\/Z\/16\/Z"]}],"id":[{"id":"10.13039\/100004440","id-type":"DOI","asserted-by":"publisher"}]},{"name":"The UK Stroke Association","award":["SA L-SMP 18\\1000"],"award-info":[{"award-number":["SA L-SMP 18\\1000"]}]},{"DOI":"10.13039\/501100000265","name":"Medical Research Council","doi-asserted-by":"publisher","award":["G0902303"],"award-info":[{"award-number":["G0902303"]}],"id":[{"id":"10.13039\/501100000265","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000589","name":"Chief Scientist Office","doi-asserted-by":"publisher","award":["CAF\/17\/01"],"award-info":[{"award-number":["CAF\/17\/01"]}],"id":[{"id":"10.13039\/501100000589","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"published-print":{"date-parts":[[2019,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n              <jats:sec>\n                <jats:title>Background<\/jats:title>\n                <jats:p>Manual coding of phenotypes in brain radiology reports is time consuming. We developed a natural language processing (NLP) algorithm to enable automatic identification of brain imaging in radiology reports performed in routine clinical practice in the UK National Health Service (NHS).<\/jats:p>\n              <\/jats:sec>\n              <jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>We used anonymized text brain imaging reports from a cohort study of stroke\/TIA patients and from a regional hospital to develop and test an NLP algorithm. Two experts marked up text in 1692 reports for 24 cerebrovascular and other neurological phenotypes. We developed and tested a rule-based NLP algorithm first within the cohort study, and further evaluated it in the reports from the regional hospital.<\/jats:p>\n              <\/jats:sec>\n              <jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>The agreement between expert readers was excellent (Cohen\u2019s \u03ba =0.93) in both datasets. In the final test dataset (<jats:italic>n<\/jats:italic>\u00a0=\u2009700) in unseen regional hospital reports, the algorithm had very good performance for a report of any ischaemic stroke [sensitivity 89% (95% CI:81\u201394); positive predictive value (PPV) 85% (76\u201390); specificity 100% (95% CI:0.99\u20131.00)]; any haemorrhagic stroke [sensitivity 96% (95% CI: 80\u201399), PPV 72% (95% CI:55\u201384); specificity 100% (95% CI:0.99\u20131.00)]; brain tumours [sensitivity 96% (CI:87\u201399); PPV 84% (73\u201391); specificity: 100% (95% CI:0.99\u20131.00)] and cerebral small vessel disease and cerebral atrophy (sensitivity, PPV and specificity all &gt;\u200997%). We obtained few reports of subarachnoid haemorrhage, microbleeds or subdural haematomas. In 110,695 reports from NHS Tayside, atrophy (<jats:italic>n<\/jats:italic>\u00a0=\u200928,757, 26%), small vessel disease (15,015, 14%) and old, deep ischaemic strokes (10,636, 10%) were the commonest findings.<\/jats:p>\n              <\/jats:sec>\n              <jats:sec>\n                <jats:title>Conclusions<\/jats:title>\n                <jats:p>An NLP algorithm can be developed in UK NHS radiology records to allow identification of cohorts of patients with important brain imaging phenotypes at a scale that would otherwise not be possible.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1186\/s12911-019-0908-7","type":"journal-article","created":{"date-parts":[[2019,9,9]],"date-time":"2019-09-09T14:03:23Z","timestamp":1568037803000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":34,"title":["A validated natural language processing algorithm for brain imaging phenotypes from radiology reports in UK electronic health records"],"prefix":"10.1186","volume":"19","author":[{"given":"Emily","family":"Wheater","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Grant","family":"Mair","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cathie","family":"Sudlow","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Beatrice","family":"Alex","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Claire","family":"Grover","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4816-8991","authenticated-orcid":false,"given":"William","family":"Whiteley","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2019,9,9]]},"reference":[{"key":"908_CR1","volume-title":"Diagnostic imaging dataset bodysite provider counts 2016\u20132017","author":"NHS Digital","year":"2017","unstructured":"NHS Digital. Diagnostic imaging dataset bodysite provider counts 2016\u20132017. 2017."},{"issue":"3","key":"908_CR2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pmed.1001779","volume":"12","author":"C Sudlow","year":"2015","unstructured":"Sudlow C, Gallacher J, Allen N, Beral V, Burton PP, Danesh J, et al. UK biobank: an open access resource for identifying the causes of a wide range of complex diseases of middle and old age. PLoS Med. 2015;12(3):e1001779.","journal-title":"PLoS Med"},{"issue":"3","key":"908_CR3","doi-asserted-by":"publisher","first-page":"305","DOI":"10.1016\/S0196-0644(96)70264-0","volume":"27","author":"EH Gilbert","year":"1996","unstructured":"Gilbert EH, Lowenstein SR, Koziol-McLain J, Barta DC, Steiner J. Chart reviews in emergency medicine research: where are the methods? Ann Emerg Med. 1996;27(3):305\u20138.","journal-title":"Ann Emerg Med"},{"issue":"2","key":"908_CR4","doi-asserted-by":"publisher","first-page":"329","DOI":"10.1148\/radiol.16142770","volume":"279","author":"E Pons","year":"2016","unstructured":"Pons E, Braun LMM, Hunink MGM, Kors JA. Natural language processing in radiology: a systematic review. Radiology. 2016;279(2):329\u201343.","journal-title":"Radiology."},{"issue":"2","key":"908_CR5","doi-asserted-by":"publisher","first-page":"164","DOI":"10.1212\/WNL.0000000000003490","volume":"88","author":"VM Castro","year":"2017","unstructured":"Castro VM, Dligach D, Finan S, Yu S, Can A, Abd-El-Barr M, et al. Large-scale identification of patients with cerebral aneurysms using natural language processing. Neurology. 2017;88(2):164\u20138.","journal-title":"Neurology."},{"issue":"2","key":"908_CR6","doi-asserted-by":"publisher","DOI":"10.2196\/12109","volume":"7","author":"S Fu","year":"2019","unstructured":"Fu S, Leung LY, Wang Y, Raulli A-O, Kallmes DF, Kinsman KA, et al. Natural language processing for the identification of silent brain infarcts from neuroimaging reports. JMIR Med Informatics. 2019;7(2):e12109.","journal-title":"JMIR Med Informatics"},{"issue":"9","key":"908_CR7","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pmed.1000145","volume":"6","author":"W Whiteley","year":"2009","unstructured":"Whiteley W, Jackson C, Lewis S, Lowe G, Rumley A, Sandercock P, et al. Inflammatory markers and poor outcome after stroke: a prospective cohort study and systematic review of Interleukin-6. PLoS Med. 2009;6(9):e1000145.","journal-title":"PLoS Med"},{"key":"908_CR8","unstructured":"Stenetorp P, Pyysalo S, Topi\u0107 G, Ohta T, Ananiadou S, Tsujii J. BRAT: a web-based tool for NLP-assisted text annotation. Proceedings of the demonstrations at the 13th conference of the European chapter of the association for computational linguistics: Association for Computational Linguistics; 2012. p. 102\u20137."},{"key":"908_CR9","first-page":"144","volume":"2012","author":"L Deleger","year":"2012","unstructured":"Deleger L, Li Q, Lingren T, Kaiser M, Molnar K, Stoutenborough L, et al. Building gold standard corpora for medical natural language processing tasks. AMIA. Annu Symp proceedings AMIA Symp. 2012;2012:144\u201353.","journal-title":"Annu Symp proceedings AMIA Symp"},{"key":"908_CR10","doi-asserted-by":"crossref","unstructured":"Grover C, Matthews M, Tobin R, Grover C, Matthews M, Tobin\u00a1 R. Tools to address the interdependence between tokenisation and standoff annotation. Proc NLPXML. 2006:19\u201326.","DOI":"10.3115\/1621034.1621038"},{"key":"908_CR11","doi-asserted-by":"publisher","first-page":"164","DOI":"10.3115\/1119176.1119200","volume-title":"Proceedings of the seventh conference on natural language learning at HLT-NAACL 2003","author":"JR Curran","year":"2003","unstructured":"Curran JR, Clark S. Language independent NER using a maximum entropy tagger. In: Proceedings of the seventh conference on natural language learning at HLT-NAACL 2003. Morristown, NJ, USA: Association for Computational Linguistics; 2003. p. 164\u20137."},{"issue":"Suppl 1","key":"908_CR12","doi-asserted-by":"publisher","first-page":"i180","DOI":"10.1093\/bioinformatics\/btg1023","volume":"19","author":"J-D Kim","year":"2003","unstructured":"Kim J-D, Ohta T, Tateisi Y, Tsujii J. GENIA corpus--a semantically annotated corpus for bio-textmining. Bioinformatics. 2003;19(Suppl 1):i180\u20132.","journal-title":"Bioinformatics."},{"key":"908_CR13","doi-asserted-by":"publisher","first-page":"201","DOI":"10.3115\/1118253.1118281","volume-title":"Proceedings of the first international conference on natural language generation - INLG \u201800","author":"G Minnen","year":"2000","unstructured":"Minnen G, Carroll J, Pearce D. Robust, applied morphological generation. In: Proceedings of the first international conference on natural language generation - INLG \u201800. Morristown, NJ, USA: Association for Computational Linguistics; 2000. p. 201."},{"key":"908_CR14","doi-asserted-by":"publisher","first-page":"209","DOI":"10.1080\/01621459.1927.10502953","volume":"22","author":"EB Wilson","year":"1927","unstructured":"Wilson EB. Probable inference, the law of succession, and statistical inference. J Am Stat Assoc. 1927;22:209\u201312.","journal-title":"J Am Stat Assoc"},{"key":"908_CR15","unstructured":"Gorinski PJ, Wu H, Grover C, Tobin R, Talbot C, Whalley H, et al. Named Entity Recognition for Electronic Health Records: A Comparison of Rule-based and Machine Learning Approaches. arXiv:190303985v2 [csCL]. 2019 Mar 10;"},{"issue":"10","key":"908_CR16","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0140533","volume":"10","author":"R Woodfield","year":"2015","unstructured":"Woodfield R, Grant I, Sudlow CLM. Accuracy of electronic health record data for identifying stroke cases in large-scale epidemiological studies: a systematic review from the UK biobank stroke outcomes group. Quinn TJ, editor. PLoS One. 2015;10(10):e0140533.","journal-title":"PLoS One"},{"key":"908_CR17","unstructured":"Alex B, Nissim M, Grover C. The impact of annotation on the performance of protein tagging in biomedical text. In: proceedings of the fifth international conference on language resources and evaluation (LREC\u201906). European Language Resources Association (ELRA). 2006:L06\u20131235."}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-019-0908-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1186\/s12911-019-0908-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-019-0908-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,9,7]],"date-time":"2020-09-07T23:12:15Z","timestamp":1599520335000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-019-0908-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,9,9]]},"references-count":17,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2019,12]]}},"alternative-id":["908"],"URL":"https:\/\/doi.org\/10.1186\/s12911-019-0908-7","relation":{},"ISSN":["1472-6947"],"issn-type":[{"value":"1472-6947","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,9,9]]},"assertion":[{"value":"27 May 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 September 2019","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 September 2019","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The Edinburgh Stroke Study received ethical approval from the Lothian Research Ethics Committee (LREC\/2001\/4\/46). We received permission from the NHS Tayside Caldicott Guardian to use anonymized brain imaging reports.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"None.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"184"}}