{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,1]],"date-time":"2025-12-01T02:53:40Z","timestamp":1764557620547,"version":"3.37.3"},"reference-count":31,"publisher":"Springer Science and Business Media LLC","issue":"S9","license":[{"start":{"date-parts":[[2021,11,1]],"date-time":"2021-11-01T00:00:00Z","timestamp":1635724800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,11,16]],"date-time":"2021-11-16T00:00:00Z","timestamp":1637020800000},"content-version":"vor","delay-in-days":15,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100005229","name":"Key Science and Technology Foundation of Gansu Province","doi-asserted-by":"publisher","award":["No.17YF1GA016"],"award-info":[{"award-number":["No.17YF1GA016"]}],"id":[{"id":"10.13039\/501100005229","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"published-print":{"date-parts":[[2021,11]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Background<\/jats:title><jats:p>In recent years, with the development of artificial intelligence, the use of deep learning technology for clinical information extraction has become a new trend. Clinical Event Detection (CED) as its subtask has attracted the attention from academia and industry. However, directly applying the advancements in deep learning to CED task often yields unsatisfactory results. The main reasons are due to the following two points: (1) A great number of obscure professional terms in the electronic medical record leads to poor recognition performance of model. (2) The scarcity of datasets required for the task leads to poor model robustness. Therefore, it is urgent to solve these two problems to improve model performance.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>This paper proposes a combining data augmentation and domain information with TENER Model for Clinical Event Detection.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>We use two evaluation metrics to compare the overall performance of the proposed model with the existing model on the 2012 i2b2 challenge dataset. Experimental results demonstrate that our proposed model achieves the best F1-score of 80.26%, type accuracy of 93% and Span F1-score of 90.33%, and outperforms the state-of-the-art approaches.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>This paper proposes a multi-granularity information fusion encoder-decoder framework, which applies the TENER model to the CED task for the first time. It uses the pre-trained language model (BioBERT) to generate word-level features, solving the problem of a great number of obscure professional terms in the electronic medical record lead to poor recognition performance of model. In addition, this paper proposes a new data augmentation method for sequence labeling tasks, solving the problem of the scarcity of datasets required for the task leads to poor model robustness.<\/jats:p><\/jats:sec>","DOI":"10.1186\/s12911-021-01618-3","type":"journal-article","created":{"date-parts":[[2021,11,17]],"date-time":"2021-11-17T11:03:18Z","timestamp":1637146998000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Combining data augmentation and domain information with TENER model for Clinical Event Detection"],"prefix":"10.1186","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3306-8493","authenticated-orcid":false,"given":"Zhichang","family":"Zhang","sequence":"first","affiliation":[]},{"given":"Dan","family":"Liu","sequence":"additional","affiliation":[]},{"given":"Minyu","family":"Zhang","sequence":"additional","affiliation":[]},{"given":"Xiaohui","family":"Qin","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,11,16]]},"reference":[{"issue":"5","key":"1618_CR1","doi-asserted-by":"publisher","first-page":"806","DOI":"10.1136\/amiajnl-2013-001628","volume":"20","author":"W Sun","year":"2013","unstructured":"Sun W, Rumshisky A, Uzuner O. Evaluating temporal relations in clinical text: 2012 i2b2 Challenge. J Am Med Inform Assoc. 2013;20(5):806\u201313. https:\/\/doi.org\/10.1136\/amiajnl-2013-001628.","journal-title":"J Am Med Inform Assoc"},{"issue":"3","key":"1618_CR2","first-page":"273","volume":"20","author":"C Cortes","year":"1995","unstructured":"Cortes C, Vapnik VN. Support-vector networks. Mach Learn. 1995;20(3):273\u201397.","journal-title":"Mach Learn"},{"key":"1618_CR3","unstructured":"Lafferty J, Mccallum A, Pereira FCN. Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of 18th international conference on machine learning (2001)."},{"issue":"5","key":"1618_CR4","doi-asserted-by":"publisher","first-page":"867","DOI":"10.1136\/amiajnl-2013-001619","volume":"20","author":"K Roberts","year":"2013","unstructured":"Roberts K, Rink B, Harabagiu SM. A flexible framework for recognizing events, temporal expressions, and temporal relations in clinical text. J Am Med Inform Assoc. 2013;20(5):867\u201375. https:\/\/doi.org\/10.1136\/amiajnl-2013-001619.","journal-title":"J Am Med Inform Assoc"},{"issue":"5","key":"1618_CR5","doi-asserted-by":"publisher","first-page":"859","DOI":"10.1136\/amiajnl-2013-001625","volume":"20","author":"A Kova\u010devi\u0107","year":"2013","unstructured":"Kova\u010devi\u0107 A, Dehghan A, Filannino M, Keane JA, a Nenadic G. Combining rules and machine learning for extraction of temporal expressions and events from clinical narratives. J Am Med Inform Assoc. 2013;20(5):859\u201366.","journal-title":"J Am Med Inform Assoc"},{"issue":"5","key":"1618_CR6","doi-asserted-by":"publisher","first-page":"820","DOI":"10.1136\/amiajnl-2013-001627","volume":"20","author":"C Grouin","year":"2013","unstructured":"Grouin C, Grabar N, Hamon T, Rosset S, Tannier X, Zweigenbaum P. Eventual situations for timeline extraction from clinical reports. J Am Med Inform Assoc. 2013;20(5):820\u20137. https:\/\/doi.org\/10.1136\/amiajnl-2013-001627.","journal-title":"J Am Med Inform Assoc"},{"issue":"11","key":"1618_CR7","doi-asserted-by":"publisher","first-page":"1297","DOI":"10.1093\/jamia\/ocz096","volume":"26","author":"Y Si","year":"2019","unstructured":"Si Y, Wang J, Xu H, Roberts K. Enhancing clinical concept extraction with contextual embeddings. J Am Med Inform Assoc. 2019;26(11):1297\u2013304. https:\/\/doi.org\/10.1093\/jamia\/ocz096.","journal-title":"J Am Med Inform Assoc"},{"key":"1618_CR8","unstructured":"Adnan A, Dietrich T, Georg G. Sequence labeling: a practical approach. J Am Med Inform Assoc Jamia. 2018. arXiv:1808.03926."},{"issue":"1","key":"1618_CR9","first-page":"6236","volume":"33","author":"H Chen","year":"2019","unstructured":"Chen H, Lin ZJ, Ding GG, Lou JG, Zhang YS, Karlsson B. GRN: gated relation network to enhance convolutional neural network for named entity recognition. Proc AAAI Conf Artif Intell. 2019;33(1):6236\u201343.","journal-title":"Proc AAAI Conf Artif Intell."},{"key":"1618_CR10","doi-asserted-by":"crossref","unstructured":"Liu P, Chang S, Huang X, Tang J, Cheung JCK. Contextualized non-local neural networks for sequence learning. In: AAAI conference on artificial intelligence 2019, 2018;vol. 33. arXiv:1811.08600.","DOI":"10.1609\/aaai.v33i01.33016762"},{"key":"1618_CR11","unstructured":"Huang ZH, Xu W, Yu K. Bidirectional lstm-crf models for sequence tagging. Comput Sci. 2015. arXiv:1508.01991."},{"key":"1618_CR12","unstructured":"Yan H, Deng B, Li X, Qiu X. Tener: adapting transformer encoder for named entity recognition. Comput Res Reposit. 2019. arXiv:1911.04474."},{"key":"1618_CR13","doi-asserted-by":"crossref","unstructured":"Cui XD, Goel V, Kingsbury B. Data augmentation for deep neural network acoustic modeling. IEEE\/ACM Trans. 2015;23(9):1469\u201377.","DOI":"10.1109\/TASLP.2015.2438544"},{"issue":"6","key":"1618_CR14","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1145\/3065386","volume":"60","author":"A Krizhevsky","year":"2017","unstructured":"Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Commun ACM. 2017;60(6):84\u201390.","journal-title":"Commun ACM"},{"key":"1618_CR15","doi-asserted-by":"crossref","unstructured":"Wei JW, Zou K. Eda: Easy data augmentation techniques for boosting performance on text classification tasks, arXiv preprint. 2019. arXiv:1901.11196.","DOI":"10.18653\/v1\/D19-1670"},{"key":"1618_CR16","doi-asserted-by":"publisher","unstructured":"Zhu H, Dong L, Wei F, Wang W, Qin B, Liu T. Learning to ask unanswerable questions for machine reading comprehension. In: Proceedings of the 57th annual meeting of the association for computational linguistics, pp. 4238\u20134248. Association for Computational Linguistics, Florence, Italy. 2019. https:\/\/doi.org\/10.18653\/v1\/P19-1415","DOI":"10.18653\/v1\/P19-1415"},{"key":"1618_CR17","doi-asserted-by":"crossref","unstructured":"Li XY, Sun XF, Meng YX, Liang JJ, Wu F, Li JW. Dice Loss for Data-imbalanced NLP Tasks. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online, 2020, pp. 465\u201376.","DOI":"10.18653\/v1\/2020.acl-main.45"},{"key":"1618_CR18","unstructured":"Yu AW, Dohan D, Luong MT, Zhao R, Chen K, Norouzi M, Le QV. Qanet: Combining local convolution with global self-attention for reading comprehension. arXiv preprint. 2018. arXiv:1804.09541."},{"key":"1618_CR19","unstructured":"Xie Z, Wang SI, Li JW, L\u00e9vy D, Nie A, Jurafsky D, Ng AY. Data noising as smoothing in neural network language models. arXiv preprint. 2017. arXiv:1703.02573."},{"key":"1618_CR20","doi-asserted-by":"crossref","unstructured":"Kobayashi S. Contextual augmentation: Data augmentation by words with paradigmatic relations. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2018; 2 (Short Papers):452\u20137.","DOI":"10.18653\/v1\/N18-2072"},{"key":"1618_CR21","doi-asserted-by":"publisher","unstructured":"Ribeiro MT, Wu T, Guestrin C, Singh S. Beyond accuracy: behavioral testing of NLP models with CheckList. In: Proceedings of the 58th annual meeting of the association for computational linguistics, 2020; pp. 4902\u20134912. Association for Computational Linguistics. https:\/\/doi.org\/10.18653\/v1\/2020.acl-main.442","DOI":"10.18653\/v1\/2020.acl-main.442"},{"issue":"4","key":"1618_CR22","doi-asserted-by":"crossref","first-page":"1234","DOI":"10.1093\/bioinformatics\/btz682","volume":"36","author":"L Jinhyuk","year":"2020","unstructured":"Jinhyuk L, Wonjin Y, Sungdong K, Donghyeon K, Sunkyu K, ChanHo S, Jaewoo K. Biobert: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics. 2020;36(4):1234\u201340.","journal-title":"Bioinformatics"},{"key":"1618_CR23","doi-asserted-by":"crossref","unstructured":"Lample G, Ballesteros M, Subramanian S, Kawakami K, Dyer C. Neural architectures for named entity recognition. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2016.","DOI":"10.18653\/v1\/N16-1030"},{"key":"1618_CR24","doi-asserted-by":"publisher","unstructured":"Ma X, Hovy E. End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1064\u20131074. Association for Computational Linguistics, Berlin. 2016. https:\/\/doi.org\/10.18653\/v1\/P16-1101","DOI":"10.18653\/v1\/P16-1101"},{"key":"1618_CR25","unstructured":"Liyuan L, Jingbo S, Frank X, Xiang R. Empower sequence labeling with task-aware neural language model. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence. New Orleans, 2018;5253\u20135260."},{"key":"1618_CR26","doi-asserted-by":"publisher","unstructured":"Zhang Y, Yang J. Chinese NER using lattice LSTM. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1554\u20131564. Association for Computational Linguistics, Melbourne, Australia. 2018. https:\/\/doi.org\/10.18653\/v1\/P18-1144","DOI":"10.18653\/v1\/P18-1144"},{"key":"1618_CR27","doi-asserted-by":"publisher","unstructured":"Li X, Yan H, Qiu X, Huang X. FLAT: Chinese NER using flat-lattice transformer. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 6836\u20136842. Association for Computational Linguistics, 2020. https:\/\/doi.org\/10.18653\/v1\/2020.acl-main.611","DOI":"10.18653\/v1\/2020.acl-main.611"},{"key":"1618_CR28","doi-asserted-by":"crossref","unstructured":"Chiu JP, Nichols E. Named entity recognition with bidirectional LSTM-CNNs. Trans Assoc Comput Linguist. 2016;4:357\u201370.","DOI":"10.1162\/tacl_a_00104"},{"key":"1618_CR29","doi-asserted-by":"crossref","unstructured":"Smith LN. Cyclical learning rates for training neural networks. In: 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), Santa Rosa, CA, USA, pp. 464\u201372.","DOI":"10.1109\/WACV.2017.58"},{"key":"1618_CR30","doi-asserted-by":"crossref","unstructured":"Pennington J, Socher R, Manning C. Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, 2014; pp. 1532\u201343.","DOI":"10.3115\/v1\/D14-1162"},{"key":"1618_CR31","doi-asserted-by":"crossref","unstructured":"Peters ME, Neumann M, Iyyer M, Gardner M, Clark C, Lee K, Zettlemoyer L. Deep contextualized word representations. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, Louisiana, 2018, pp. 2227\u201337.","DOI":"10.18653\/v1\/N18-1202"}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-021-01618-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12911-021-01618-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-021-01618-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,12]],"date-time":"2023-11-12T19:37:35Z","timestamp":1699817855000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-021-01618-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11]]},"references-count":31,"journal-issue":{"issue":"S9","published-print":{"date-parts":[[2021,11]]}},"alternative-id":["1618"],"URL":"https:\/\/doi.org\/10.1186\/s12911-021-01618-3","relation":{},"ISSN":["1472-6947"],"issn-type":[{"type":"electronic","value":"1472-6947"}],"subject":[],"published":{"date-parts":[[2021,11]]},"assertion":[{"value":"15 August 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 August 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 November 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"261"}}