{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,12]],"date-time":"2026-06-12T07:08:54Z","timestamp":1781248134810,"version":"3.54.1"},"reference-count":38,"publisher":"Cambridge University Press (CUP)","issue":"2","license":[{"start":{"date-parts":[[2020,11,18]],"date-time":"2020-11-18T00:00:00Z","timestamp":1605657600000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/www.cambridge.org\/core\/terms"}],"content-domain":{"domain":["cambridge.org"],"crossmark-restriction":true},"short-container-title":["Nat. Lang. Eng."],"published-print":{"date-parts":[[2021,3]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Using natural language processing, it is possible to extract structured information from raw text in the electronic health record (EHR) at reasonably high accuracy. However, the accurate distinction between negated and non-negated mentions of clinical terms remains a challenge. EHR text includes cases where diseases are stated not to be present or only hypothesised, meaning a disease can be mentioned in a report when it is not being reported as present. This makes tasks such as document classification and summarisation more difficult. We have developed the rule-based EdIE-R-Neg, part of an existing text mining pipeline called EdIE-R (Edinburgh Information Extraction for Radiology reports), developed to process brain imaging reports, (<jats:uri xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" xlink:href=\"https:\/\/www.ltg.ed.ac.uk\/software\/edie-r\/\">https:\/\/www.ltg.ed.ac.uk\/software\/edie-r\/<\/jats:uri>) and two machine learning approaches; one using a bidirectional long short-term memory network and another using a feedforward neural network. These were developed on data from the Edinburgh Stroke Study (ESS) and tested on data from routine reports from NHS Tayside (Tayside). Both datasets consist of written reports from medical scans. These models are compared with two existing rule-based models: pyConText (Harkema <jats:italic>et al<\/jats:italic>. 2009. <jats:italic>Journal of Biomedical Informatics<\/jats:italic><jats:bold>42<\/jats:bold>(5), 839\u2013851), a python implementation of a generalisation of NegEx, and NegBio (Peng <jats:italic>et al<\/jats:italic>. 2017. NegBio: A high-performance tool for negation and uncertainty detection in radiology reports. arXiv e-prints, p. <jats:uri xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" xlink:href=\"https:\/\/arXiv:1712.05898\">arXiv:1712.05898<\/jats:uri>), which identifies negation scopes through patterns applied to a syntactic representation of the sentence. On both the test set of the dataset from which our models were developed, as well as the largely similar Tayside test set, the neural network models and our custom-built rule-based system outperformed the existing methods. EdIE-R-Neg scored highest on F1 score, particularly on the test set of the Tayside dataset, from which no development data were used in these experiments, showing the power of custom-built rule-based systems for negation detection on datasets of this size. The performance gap of the machine learning models to EdIE-R-Neg on the Tayside test set was reduced through adding development Tayside data into the ESS training set, demonstrating the adaptability of the neural network models.<\/jats:p>","DOI":"10.1017\/s1351324920000509","type":"journal-article","created":{"date-parts":[[2020,11,18]],"date-time":"2020-11-18T07:56:55Z","timestamp":1605686215000},"page":"203-224","update-policy":"https:\/\/doi.org\/10.1017\/policypage","source":"Crossref","is-referenced-by-count":32,"title":["Comparison of rule-based and neural network models for negation detection in radiology reports"],"prefix":"10.1017","volume":"27","author":[{"given":"D.","family":"Sykes","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"A.","family":"Grivas","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"C.","family":"Grover","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"R.","family":"Tobin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"C.","family":"Sudlow","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"W.","family":"Whiteley","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"A.","family":"Mcintosh","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"H.","family":"Whalley","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"B.","family":"Alex","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"56","published-online":{"date-parts":[[2020,11,18]]},"reference":[{"key":"S1351324920000509_ref4","doi-asserted-by":"publisher","DOI":"10.1109\/72.279181"},{"key":"S1351324920000509_ref8","doi-asserted-by":"publisher","DOI":"10.1002\/asi.23533"},{"key":"S1351324920000509_ref35","doi-asserted-by":"publisher","DOI":"10.1136\/amiajnl-2011-000203"},{"key":"S1351324920000509_ref2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W19-1909"},{"key":"S1351324920000509_ref9","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P16-1047"},{"key":"S1351324920000509_ref31","unstructured":"Pratt, L.Y. , Mostow, J. and Kamm, C.A. (1991). Direct transfer of learned information among neural networks. In Proceedings of the Ninth National Conference on Artificial Intelligence - Volume 2, AAAI91. AAAI Press, pp. 584\u2013589."},{"key":"S1351324920000509_ref1","doi-asserted-by":"publisher","DOI":"10.1186\/s13326-019-0211-7"},{"key":"S1351324920000509_ref29","doi-asserted-by":"publisher","DOI":"10.1109\/MCAS.2006.1688199"},{"key":"S1351324920000509_ref17","doi-asserted-by":"publisher","DOI":"10.1197\/jamia.M1733"},{"key":"S1351324920000509_ref22","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P14-5010"},{"key":"S1351324920000509_ref16","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0174708"},{"key":"S1351324920000509_ref37","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0112774"},{"key":"S1351324920000509_ref38","doi-asserted-by":"crossref","unstructured":"Grivas, A. , Alex, B. , Grover, C. , Tobin, R. and Whiteley, W. (2020). Not a cute stroke: Analysis of Rule- and Neural Network-Based Information Extraction Systems for Brain Radiology Reports. In Proceedings of the 11th International Workshop on Health Text Mining and Information Analysis (LOUHI 2020) at EMNLP 2020.","DOI":"10.18653\/v1\/2020.louhi-1.4"},{"key":"S1351324920000509_ref25","doi-asserted-by":"publisher","DOI":"10.1136\/jamia.2001.0080598"},{"key":"S1351324920000509_ref18","doi-asserted-by":"publisher","DOI":"10.1197\/jamia.M2284"},{"key":"S1351324920000509_ref32","unstructured":"Stenetorp, P. , Pyysalo, S. , Topi\u0107, G. , Ohta, T. , Ananiadou, S. and Tsujii, J. (2012). brat: A web-based tool for NLP-assisted text annotation. In Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics, Avignon, France. Association for Computational Linguistics, pp. 102\u2013107."},{"key":"S1351324920000509_ref21","unstructured":"Maldonado, R. , Goodwin, T. and Harabagiu, S.M. (2017). Active deep learning-based annotation of electroencephalography reports for cohort identification. In CRI, vol. 2017, pp. 229\u2013238."},{"key":"S1351324920000509_ref14","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2009.05.002"},{"key":"S1351324920000509_ref11","volume-title":"Implementation and Evaluation of Four Different Methods of Negation Detection","author":"Goryachev","year":"2006"},{"key":"S1351324920000509_ref27","unstructured":"Peng, Y. , Wang, X. , Lu, L. , Bagheri, M. , Summers, R. and Lu, Z. (2017). NegBio: A high-performance tool for negation and uncertainty detection in radiology reports. arXiv e-prints, p. arXiv:1712.05898."},{"key":"S1351324920000509_ref6","doi-asserted-by":"publisher","DOI":"10.1177\/001316446002000104"},{"key":"S1351324920000509_ref34","doi-asserted-by":"publisher","DOI":"10.3115\/1118853.1118877"},{"key":"S1351324920000509_ref12","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2005.06.042"},{"key":"S1351324920000509_ref26","unstructured":"Paszke, A. , Gross, S. , Chintala, S. , Chanan, G. , Yang, E. , DeVito, Z. , Lin, Z. , Desmaison, A. , Antiga, L. and Lerer, A. (2017). Automatic differentiation in pytorch. In NIPS-W."},{"key":"S1351324920000509_ref24","doi-asserted-by":"crossref","unstructured":"Mou, L. , Meng, Z. , Yan, R. , Li, G. , Xu, Y. , Zhang, L. and Jin, Z. (2016). How transferable are neural networks in NLP applications? arXiv e-prints, p. arXiv:1603.06111.","DOI":"10.18653\/v1\/D16-1046"},{"key":"S1351324920000509_ref28","doi-asserted-by":"crossref","unstructured":"Peng, Y. , Yan, K. , Sandfort, V. , Summers, R.M. and Lu, Z. (2019). A self-attention based deep learning method for lesion attribute detection from ct reports. arXiv preprint arXiv:1904.13018.","DOI":"10.1109\/ICHI.2019.8904668"},{"key":"S1351324920000509_ref10","unstructured":"Gorinski, P.J. , Wu, H. , Grover, C. , Tobin, R. , Talbot, C. , Whalley, H. , Sudlow, C. , Whiteley, W. and Alex, B. (2019). Named entity recognition for electronic health records: A comparison of rule-based and machine learning approaches. arXiv e-prints, p. arXiv:1903.03985."},{"key":"S1351324920000509_ref30","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.16142770"},{"key":"S1351324920000509_ref23","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2015.02.010"},{"key":"S1351324920000509_ref7","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W16-6103"},{"key":"S1351324920000509_ref3","unstructured":"Ba, J.L. , Kiros, J.R. and Hinton, G.E. (2016). Layer normalization. arXiv e-prints, p. arXiv:1607.06450."},{"key":"S1351324920000509_ref15","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"S1351324920000509_ref5","doi-asserted-by":"publisher","DOI":"10.1006\/jbin.2001.1029"},{"key":"S1351324920000509_ref13","unstructured":"Grover, C. and Tobin, R. (2006). Rule-based chunking and reusability. In Proceedings of LREC 2006, pp. 873\u2013878."},{"key":"S1351324920000509_ref20","unstructured":"Kingma, D.P. and Ba, J. (2015). Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, 7\u20139 May, 2015, Conference Track Proceedings."},{"key":"S1351324920000509_ref33","unstructured":"Taylor, S. and Harabagiu, S. (2018). The role of a deep-learning method for negation detection in patient cohort identification from electroencephalography reports. Proceedings of the AMIA Annual Symposium 2018, 1018\u20131027."},{"key":"S1351324920000509_ref19","doi-asserted-by":"publisher","DOI":"10.1093\/qjmed\/hcm152"},{"key":"S1351324920000509_ref36","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.369"}],"container-title":["Natural Language Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S1351324920000509","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,5,12]],"date-time":"2021-05-12T11:04:44Z","timestamp":1620817484000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S1351324920000509\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,11,18]]},"references-count":38,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2021,3]]}},"alternative-id":["S1351324920000509"],"URL":"https:\/\/doi.org\/10.1017\/s1351324920000509","relation":{},"ISSN":["1351-3249","1469-8110"],"issn-type":[{"value":"1351-3249","type":"print"},{"value":"1469-8110","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,11,18]]},"assertion":[{"value":"\u00a9 The Author(s), 2020. Published by Cambridge University Press","name":"copyright","label":"Copyright","group":{"name":"copyright_and_licensing","label":"Copyright and Licensing"}}]}}