{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,7]],"date-time":"2026-03-07T05:17:13Z","timestamp":1772860633935,"version":"3.50.1"},"reference-count":79,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2023,2,17]],"date-time":"2023-02-17T00:00:00Z","timestamp":1676592000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"MIUR \u201cFondo Dipartimenti di Eccellenza 2018\u20132022\u201d of the DII Department at the University of Brescia, Italy"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>Biomedical named entity recognition (BioNER) is a preliminary task for many other tasks, e.g., relation extraction and semantic search. Extracting the text of interest from biomedical documents becomes more demanding as the availability of online data is increasing. Deep learning models have been adopted for biomedical named entity recognition (BioNER) as deep learning has been found very successful in many other tasks. Nevertheless, the complex structure of biomedical text data is still a challenging aspect for deep learning models. Limited annotated biomedical text data make it more difficult to train deep learning models with millions of trainable parameters. The single-task model, which focuses on learning a specific task, has issues in learning complex feature representations from a limited quantity of annotated data. Moreover, manually constructing annotated data is a time-consuming job. It is, therefore, vital to exploit other efficient ways to train deep learning models on the available annotated data. This work enhances the performance of the BioNER task by taking advantage of various knowledge transfer techniques: multitask learning and transfer learning. This work presents two multitask models (MTMs), which learn shared features and task-specific features by implementing the shared and task-specific layers. In addition, the presented trained MTM is also fine-tuned for each specific dataset to tailor it from a general features representation to a specialized features representation. The presented empirical results and statistical analysis from this work illustrate that the proposed techniques enhance significantly the performance of the corresponding single-task model (STM).<\/jats:p>","DOI":"10.3390\/fi15020079","type":"journal-article","created":{"date-parts":[[2023,2,17]],"date-time":"2023-02-17T03:27:41Z","timestamp":1676604461000},"page":"79","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["On the Use of Knowledge Transfer Techniques for Biomedical Named Entity Recognition"],"prefix":"10.3390","volume":"15","author":[{"given":"Tahir","family":"Mehmood","sequence":"first","affiliation":[{"name":"Department of Information Engineering, University of Brescia, Via Branze 38, 25121 Brescia, Italy"},{"name":"NLP Research Group, Fondazione Bruno Kessler, Via Sommarive 18, 38123 Trento, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7785-9492","authenticated-orcid":false,"given":"Ivan","family":"Serina","sequence":"additional","affiliation":[{"name":"Department of Information Engineering, University of Brescia, Via Branze 38, 25121 Brescia, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7175-6804","authenticated-orcid":false,"given":"Alberto","family":"Lavelli","sequence":"additional","affiliation":[{"name":"NLP Research Group, Fondazione Bruno Kessler, Via Sommarive 18, 38123 Trento, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Luca","family":"Putelli","sequence":"additional","affiliation":[{"name":"Department of Information Engineering, University of Brescia, Via Branze 38, 25121 Brescia, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alfonso","family":"Gerevini","sequence":"additional","affiliation":[{"name":"Department of Information Engineering, University of Brescia, Via Branze 38, 25121 Brescia, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,2,17]]},"reference":[{"key":"ref_1","unstructured":"Mehmood, T., Gerevini, A.E., Lavelli, A., and Serina, I. (2020, January 16\u201318). Combining Multi-task Learning with Transfer Learning for Biomedical Named Entity Recognition. Proceedings of the Knowledge-Based and Intelligent Information & Engineering Systems: 24th International Conference KES-2020, Virtual Event."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Mehmood, T., Gerevini, A., Lavelli, A., and Serina, I. (2019, January 19\u201322). Leveraging Multi-task Learning for Biomedical Named Entity Recognition. Proceedings of the AI*IA 2019\u2014Advances in Artificial Intelligence\u2014XVIIIth International Conference of the Italian Association for Artificial Intelligence, Rende, Italy.","DOI":"10.1007\/978-3-030-35166-3_31"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Mehmood, T., Gerevini, A., Lavelli, A., and Serina, I. (2019, January 13\u201315). Multi-task Learning Applied to Biomedical Named Entity Recognition Task. Proceedings of the Sixth Italian Conference on Computational Linguistics, Bari, Italy.","DOI":"10.1007\/978-3-030-35166-3_31"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Xu, M., Jiang, H., and Watcharawittayakul, S. (August, January 30). A Local Detection Approach for Named Entity Recognition and Mention Detection. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, BC, Canada.","DOI":"10.18653\/v1\/P17-1114"},{"key":"ref_5","unstructured":"Lin, Y.F., Tsai, T.H., Chou, W.C., Wu, K.P., Sung, T.Y., and Hsu, W.L. (2004, January 22). A maximum entropy approach to biomedical named entity recognition. Proceedings of the 4th International Conference on Data Mining in Bioinformatics, Seattle, WA, USA."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Settles, B. (2004, January 28\u201329). Biomedical named entity recognition using conditional random fields and rich feature sets. Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and its Applications (NLPBA\/BioNLP), Geneva, Switzerland.","DOI":"10.3115\/1567594.1567618"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Alex, B., Haddow, B., and Grover, C. (2007, January 29). Recognising nested named entities in biomedical text. Proceedings of the Biological, Translational, and Clinical Language Processing, Prague, Czech Republic.","DOI":"10.3115\/1572392.1572404"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"158","DOI":"10.1186\/s12938-018-0573-6","article-title":"Comparison of named entity recognition methodologies in biomedical documents","volume":"17","author":"Song","year":"2018","journal-title":"Biomed. Eng. Online"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Ciresan, D.C., Meier, U., Gambardella, L.M., and Schmidhuber, J. (2011, January 18\u201321). Convolutional Neural Network Committees for Handwritten Character Classification. Proceedings of the 2011 International Conference on Document Analysis and Recognition, ICDAR 2011, Beijing, China.","DOI":"10.1109\/ICDAR.2011.229"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Deng, L., Hinton, G.E., and Kingsbury, B. (2013, January 26\u201331). New types of deep neural network learning for speech recognition and related applications: An overview. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2013, Vancouver, BC, Canada.","DOI":"10.1109\/ICASSP.2013.6639344"},{"key":"ref_11","unstructured":"Ramsundar, B., Kearnes, S.M., Riley, P., Webster, D., Konerding, D.E., and Pande, V.S. (2015). Massively Multitask Networks for Drug Discovery. arXiv."},{"key":"ref_12","unstructured":"Mehmood, T., Serina, I., Lavelli, A., and Gerevini, A. (2020, January 25\u201327). Knowledge Distillation Techniques for Biomedical Named Entity Recognition. Proceedings of the 4th Workshop on Natural Language for Artificial Intelligence (NL4AI 2020) Co-Located with the 19th International Conference of the Italian Association for Artificial Intelligence (AI*IA 2020), Anywhere."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1023\/A:1007379606734","article-title":"Multitask learning","volume":"28","author":"Caruana","year":"1997","journal-title":"Mach. Learn."},{"key":"ref_14","unstructured":"Mitchell, T.M. (1980). The Need for Biases in Learning Generalizations, Rutgers University. (Rutgers Computer Science Tech. Rept. CBM-TR-117)."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Mehmood, T., Lavelli, A., Serina, I., and Gerevini, A. (2021, January 14\u201316). Knowledge Distillation with Teacher Multi-task Model for Biomedical Named Entity Recognition. Proceedings of the Innovation in Medicine and Healthcare: Proceedings of 9th KES-InMed, Virtual Event.","DOI":"10.1007\/978-981-16-3013-2_3"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Girshick, R.B. (2015, January 7\u201313). Fast R-CNN. Proceedings of the 2015 IEEE International Conference on Computer Vision, ICCV 2015, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.169"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Collobert, R., and Weston, J. (2008, January 5\u20139). A unified architecture for natural language processing: Deep neural networks with multitask learning. Proceedings of the Twenty-Fifth International Conference on Machine Learning (ICML 2008), Helsinki, Finland.","DOI":"10.1145\/1390156.1390177"},{"key":"ref_18","unstructured":"Bollmann, M., and S\u00f8gaard, A. (2016). Improving historical spelling normalization with bi-directional LSTMs and multi-task learning. arXiv."},{"key":"ref_19","unstructured":"Peng, N., and Dredze, M. (2016). Multi-task multi-domain representation learning for sequence tagging. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Plank, B., S\u00f8gaard, A., and Goldberg, Y. (2016). Multilingual part-of-speech tagging with bidirectional long short-term memory models and auxiliary loss. arXiv.","DOI":"10.18653\/v1\/P16-2067"},{"key":"ref_21","unstructured":"Yang, Z., Salakhutdinov, R., and Cohen, W. (2016). Multi-task cross-lingual sequence tagging from scratch. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Zhang, Y., and Weiss, D. (2016, January 7\u201312). Stack-propagation: Improved Representation Learning for Syntax. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, Berlin, Germany.","DOI":"10.18653\/v1\/P16-1147"},{"key":"ref_23","unstructured":"Johansson, R. (2013, January 9\u201314). Training parsers on incompatible treebanks. Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Atlanta, GA, USA."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"S\u00f8gaard, A., and Goldberg, Y. (2016, January 7\u201312). Deep multi-task learning with low level tasks supervised at lower layers. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, Berlin, Germany.","DOI":"10.18653\/v1\/P16-2038"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Hashimoto, K., Xiong, C., Tsuruoka, Y., and Socher, R. (2017, January 9\u201311). A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark.","DOI":"10.18653\/v1\/D17-1206"},{"key":"ref_26","unstructured":"Luong, M., Le, Q.V., Sutskever, I., Vinyals, O., and Kaiser, L. (2016, January 2\u20134). Multi-task Sequence to Sequence Learning. Proceedings of the 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico."},{"key":"ref_27","unstructured":"Yosinski, J., Clune, J., Bengio, Y., and Lipson, H. (2014, January 8\u201313). How transferable are features in deep neural networks?. Proceedings of the Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, Montreal, QC, Canada."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Savini, E., and Caragea, C. (2022). Intermediate-Task Transfer Learning with BERT for Sarcasm Detection. Mathematics, 10.","DOI":"10.3390\/math10050844"},{"key":"ref_29","unstructured":"Radford, A., Narasimhan, K., Salimans, T., and Sutskever, I. (2023, February 01). Improving Language Understanding by Generative Pre-Training. Available online: https:\/\/www.cs.ubc.ca\/~amuham01\/LING530\/papers\/radford2018improving.pdf."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Oquab, M., Bottou, L., Laptev, I., and Sivic, J. (2014, January 23\u201328). Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.222"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1007\/s10115-015-0870-3","article-title":"Transfer learning for class imbalance problems with inadequate data","volume":"48","author":"Reddy","year":"2016","journal-title":"Knowl. Inf. Syst."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Yang, J., Zhang, Y., and Dong, F. (August, January 30). Neural Word Segmentation with Rich Pretraining. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, BC, Canada.","DOI":"10.18653\/v1\/P17-1078"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Zoph, B., Yuret, D., May, J., and Knight, K. (2016, January 1\u20134). Transfer Learning for Low-Resource Neural Machine Translation. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, TX, USA.","DOI":"10.18653\/v1\/D16-1163"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Crichton, G., Pyysalo, S., Chiu, B., and Korhonen, A. (2017). A neural network multi-task learning approach to biomedical named entity recognition. BMC Bioinform., 18.","DOI":"10.1186\/s12859-017-1776-8"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1745","DOI":"10.1093\/bioinformatics\/bty869","article-title":"Cross-type biomedical named entity recognition with deep multi-task learning","volume":"35","author":"Wang","year":"2019","journal-title":"Bioinformatics"},{"key":"ref_36","unstructured":"Dugas, F., and Nichols, E. (2016, January 11). DeepNNNER: Applying BLSTM-CNNs and Extended Lexicons to Named Entity Recognition in Tweets. Proceedings of the 2nd Workshop on Noisy User-Generated Text (WNUT), Osaka, Japan."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Segura-Bedmar, I., Su\u00e1rez-Paniagua, V., and Mart\u00ednez, P. (2015, January 17). Exploring Word Embedding for Drug Name Recognition. Proceedings of the Sixth International Workshop on Health Text Mining and Information Analysis, Louhi@EMNLP 2015, Lisbon, Portugal.","DOI":"10.18653\/v1\/W15-2608"},{"key":"ref_38","unstructured":"Huang, Z., Xu, W., and Yu, K. (2015). Bidirectional LSTM-CRF Models for Sequence Tagging. arXiv."},{"key":"ref_39","unstructured":"Limsopatham, N., and Collier, N. (2016, January 12). Learning Orthographic Features in Bi-directional LSTM for Biomedical Named Entity Recognition. Proceedings of the Fifth Workshop on Building and Evaluating Resources for Biomedical Text Mining, BioTxtM@COLING 2016, Osaka, Japan."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"dos Santos, C., Guimaraes, V., Niter\u00f3i, R., and de Janeiro, R. (2015, January 31). Boosting Named Entity Recognition with Neural Character Embeddings. Proceedings of the NEWS 2015 The Fifth Named Entities Workshop, Beijing, China.","DOI":"10.18653\/v1\/W15-3904"},{"key":"ref_41","first-page":"2493","article-title":"Natural language processing (almost) from scratch","volume":"12","author":"Collobert","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_42","unstructured":"Yang, J., Liang, S., and Zhang, Y. (2018, January 20\u201326). Design Challenges and Misconceptions in Neural Sequence Labeling. Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018, Santa Fe, NW, USA."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Li, J., Liu, C., and Gong, Y. (2018, January 2\u20136). Layer Trajectory LSTM. Proceedings of the Interspeech 2018, 19th Annual Conference of the International Speech Communication Association, Hyderabad, India.","DOI":"10.21437\/Interspeech.2018-1485"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"262","DOI":"10.1016\/j.neucom.2013.08.044","article-title":"A biologically inspired dual-network memory model for reduction of catastrophic forgetting","volume":"134","author":"Hattori","year":"2014","journal-title":"Neurocomputing"},{"key":"ref_45","unstructured":"Ramasesh, V.V., Dyer, E., and Raghu, M. (2021, January 3\u20137). Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics. Proceedings of the 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1016\/S1364-6613(99)01294-2","article-title":"Catastrophic forgetting in connectionist networks","volume":"3","author":"French","year":"1999","journal-title":"Trends Cogn. Sci."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Ma, X., and Hovy, E.H. (2016, January 7\u201312). End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, Berlin, Germany.","DOI":"10.18653\/v1\/P16-1101"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Sugianto, N., Tjondronegoro, D., Sorwar, G., Chakraborty, P., and Yuwono, E.I. (2019, January 18\u201321). Continuous Learning without Forgetting for Person Re-Identification. Proceedings of the 2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Taipei, Taiwan.","DOI":"10.1109\/AVSS.2019.8909828"},{"key":"ref_49","unstructured":"Mikolov, T., Grave, E., Bojanowski, P., Puhrsch, C., and Joulin, A. (2018, January 7\u201312). Advances in Pre-Training Distributed Word Representations. Proceedings of the Eleventh International Conference on Language Resources and Evaluation, LREC 2018, Miyazaki, Japan."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Pennington, J., Socher, R., and Manning, C. (2014, January 26\u201328). Glove: Global Vectors for Word Representation. Proceedings of the Conference On Empirical Methods In Natural Language Processing (EMNLP), Doha, Qatar.","DOI":"10.3115\/v1\/D14-1162"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Zhai, Z., Nguyen, D.Q., Akhondi, S.A., Thorne, C., Druckenbrodt, C., Cohn, T., Gregory, M., and Verspoor, K. (2019, January 1). Improving Chemical Named Entity Recognition in Patents with Contextualized Word Embeddings. Proceedings of the 18th BioNLP Workshop and Shared Task, BioNLP@ACL 2019, Florence, Italy.","DOI":"10.18653\/v1\/W19-5035"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Lester, B. (2020). iobes: A Library for Span-Level Processing. arXiv.","DOI":"10.18653\/v1\/2020.nlposs-1.16"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Yu, J., Bohnet, B., and Poesio, M. (2020, January 5\u201310). Named Entity Recognition as Dependency Parsing. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online.","DOI":"10.18653\/v1\/2020.acl-main.577"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"4087","DOI":"10.1093\/bioinformatics\/bty449","article-title":"Transfer learning for biomedical named entity recognition with neural networks","volume":"34","author":"Giorgi","year":"2018","journal-title":"Bioinformatics"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1002\/pri.66","article-title":"The use and interpretation of the Friedman test in the analysis of ordinal-scale data in repeated measures designs","volume":"1","author":"Sheldon","year":"1996","journal-title":"Physiother. Res. Int."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1080\/00220973.1993.9943832","article-title":"Relative power of the Wilcoxon test, the Friedman test, and repeated-measures ANOVA on ranks","volume":"62","author":"Zimmerman","year":"1993","journal-title":"J. Exp. Educ."},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Peng, Y., Yan, S., and Lu, Z. (2019, January 1). Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets. Proceedings of the 18th BioNLP Workshop and Shared Task, BioNLP@ACL 2019, Florence, Italy.","DOI":"10.18653\/v1\/W19-5006"},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"1234","DOI":"10.1093\/bioinformatics\/btz682","article-title":"BioBERT: A pre-trained biomedical language representation model for biomedical text mining","volume":"36","author":"Lee","year":"2020","journal-title":"Bioinformatics"},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Tsai, R.T., Wu, S., Chou, W., Lin, Y., He, D., Hsiang, J., Sung, T., and Hsu, W. (2006). Various criteria in the evaluation of biomedical named entity recognition. BMC Bioinform., 7.","DOI":"10.1186\/1471-2105-7-92"},{"key":"ref_60","unstructured":"Sang, E.F.T.K., and Meulder, F.D. (June, January 31). Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition. Proceedings of the Seventh Conference on Natural Language Learning, CoNLL 2003, Edmonton, AB, Canada."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"868","DOI":"10.1093\/bioinformatics\/btt580","article-title":"Anatomical entity mention recognition at literature scale","volume":"30","author":"Pyysalo","year":"2014","journal-title":"Bioinformatics"},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Francis, S., Van Landeghem, J., and Moens, M.F. (2019). Transfer Learning for Named Entity Recognition in Financial and Biomedical Documents. Information, 10.","DOI":"10.3390\/info10080248"},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"S1","DOI":"10.1186\/1758-2946-7-S1-S1","article-title":"CHEMDNER: The drugs and chemical names extraction challenge","volume":"7","author":"Krallinger","year":"2015","journal-title":"J. Cheminform."},{"key":"ref_64","unstructured":"Wei, C.H., Peng, Y., Leaman, R., Davis, A.P., Mattingly, C.J., Li, J., Wiegers, T.C., and Lu, Z. (2015, January 31). Overview of the BioCreative V chemical disease relation (CDR) task. Proceedings of the Fifth BioCreative Challenge Evaluation Workshop, Sevilla, Spain."},{"key":"ref_65","doi-asserted-by":"crossref","unstructured":"Kim, J.D., Ohta, T., Pyysalo, S., Kano, Y., and Tsujii, J. (2009, January 5). Overview of BioNLP\u201909 shared task on event extraction. Proceedings of the BioNLP 2009 Workshop Companion Volume for Shared Task, Boulder, CO, USA.","DOI":"10.3115\/1572340.1572342"},{"key":"ref_66","unstructured":"Ohta, T., Pyysalo, S., and Tsujii, J. (2011, January 24). Overview of the epigenetics and post-translational modifications (EPI) task of BioNLP shared task 2011. Proceedings of the BioNLP Shared Task 2011 Workshop, Portland, OR, USA."},{"key":"ref_67","unstructured":"Pyysalo, S., Ohta, T., Miwa, M., and Tsujii, J. (2011, January 24). Towards exhaustive protein modification event extraction. Proceedings of the BioNLP 2011 Workshop, Portland, OR, USA."},{"key":"ref_68","doi-asserted-by":"crossref","unstructured":"Pyysalo, S., Ohta, T., Rak, R., Sullivan, D., Mao, C., Wang, C., Sobral, B., Tsujii, J., and Ananiadou, S. (2011, January 24). Overview of the infectious diseases (ID) task of BioNLP Shared Task 2011. Proceedings of the BioNLP Shared Task 2011 Workshop, Portland, OR, USA.","DOI":"10.1186\/1471-2105-13-S11-S2"},{"key":"ref_69","unstructured":"N\u00e9dellec, C., Bossy, R., Kim, J.D., Kim, J.J., Ohta, T., Pyysalo, S., and Zweigenbaum, P. (2013, January 8\u20139). Overview of BioNLP shared task 2013. Proceedings of the BioNLP Shared Task 2013 Workshop, Sofia, Bulgaria."},{"key":"ref_70","unstructured":"Pyysalo, S., Ohta, T., and Ananiadou, S. (2013, January 8\u20139). Overview of the cancer genetics (CG) task of BioNLP Shared Task 2013. Proceedings of the BioNLP Shared Task 2013 Workshop, Sofia, Bulgaria."},{"key":"ref_71","doi-asserted-by":"crossref","unstructured":"Kim, J.D., Kim, J.j., Han, X., and Rebholz-Schuhmann, D. (2015). Extending the evaluation of Genia Event task toward knowledge base construction and comparison to Gene Regulation Ontology task. BMC Bioinform., 16.","DOI":"10.1186\/1471-2105-16-S10-S3"},{"key":"ref_72","unstructured":"Kim, J.D., Wang, Y., and Yasunori, Y. (2013, January 8\u20139). The genia event extraction shared task, 2013 edition-overview. Proceedings of the BioNLP Shared Task 2013 Workshop, Sofia, Bulgaria."},{"key":"ref_73","unstructured":"Ohta, T., Pyysalo, S., Rak, R., Rowley, A., Chun, H.W., Jung, S.J., Choi, S.P., Ananiadou, S., and Tsujii, J. (2013, January 8\u20139). Overview of the pathway curation (PC) task of bioNLP shared task 2013. Proceedings of the BioNLP Shared Task 2013 Workshop, Sofia, Bulgaria."},{"key":"ref_74","unstructured":"Basher, A.R.M., Purdy, A.S., and Birol, I. Event extraction from biomedical literature. bioRxiv, 2015."},{"key":"ref_75","unstructured":"Mi, H., and Thomas, P. (2009). Protein Networks and Pathway Analysis, Springer."},{"key":"ref_76","doi-asserted-by":"crossref","unstructured":"Bada, M., Eckert, M., Evans, D., Garcia, K., Shipley, K., Sitnikov, D., Baumgartner, W.A., Cohen, K.B., Verspoor, K., and Blake, J.A. (2012). Concept annotation in the CRAFT corpus. BMC Bioinform., 13.","DOI":"10.1186\/1471-2105-13-161"},{"key":"ref_77","doi-asserted-by":"crossref","unstructured":"Kim, J.D., Ohta, T., Tsuruoka, Y., Tateisi, Y., and Collier, N. (2004, January 28\u201329). Introduction to the bio-entity recognition task at JNLPBA. Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and Its Applications, Geneva, Switzerland.","DOI":"10.3115\/1567594.1567610"},{"key":"ref_78","doi-asserted-by":"crossref","first-page":"e29626","DOI":"10.3897\/BDJ.7.e29626","article-title":"COPIOUS: A gold standard corpus of named entities towards extracting species occurrence from biodiversity literature","volume":"7","author":"Nguyen","year":"2019","journal-title":"Biodivers. Data J."},{"key":"ref_79","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.jbi.2013.12.006","article-title":"NCBI disease corpus: A resource for disease name recognition and concept normalization","volume":"47","author":"Leaman","year":"2014","journal-title":"J. Biomed. Inform."}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/15\/2\/79\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:39:16Z","timestamp":1760121556000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/15\/2\/79"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,17]]},"references-count":79,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,2]]}},"alternative-id":["fi15020079"],"URL":"https:\/\/doi.org\/10.3390\/fi15020079","relation":{},"ISSN":["1999-5903"],"issn-type":[{"value":"1999-5903","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,17]]}}}