{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,2]],"date-time":"2026-07-02T10:48:00Z","timestamp":1782989280408,"version":"3.54.5"},"reference-count":40,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,11,3]],"date-time":"2022-11-03T00:00:00Z","timestamp":1667433600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,11,3]],"date-time":"2022-11-03T00:00:00Z","timestamp":1667433600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"the Ministry of Science and Technology, ROC","award":["109-2221-E-468-014-MY3"],"award-info":[{"award-number":["109-2221-E-468-014-MY3"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Bioinformatics"],"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Background<\/jats:title><jats:p>Biomedical named entity recognition (BioNER) is a basic and important task for biomedical text mining with the purpose of automatically recognizing and classifying biomedical entities. The performance of BioNER systems directly impacts downstream applications. Recently, deep neural networks, especially pre-trained language models, have made great progress for BioNER. However, because of the lack of high-quality and large-scale annotated data and relevant external knowledge, the capability of the BioNER system remains limited.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>In this paper, we propose a novel fully-shared multi-task learning model based on the pre-trained language model in biomedical domain, namely BioBERT, with a new attention module to integrate the auto-processed syntactic information for the BioNER task. We have conducted numerous experiments on seven benchmark BioNER datasets. The proposed best multi-task model obtains F1 score improvements of 1.03% on BC2GM, 0.91% on NCBI-disease, 0.81% on Linnaeus, 1.26% on JNLPBA, 0.82% on BC5CDR-Chemical, 0.87% on BC5CDR-Disease, and 1.10% on Species-800 compared to the single-task BioBERT model.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p>The results demonstrate our model outperforms previous studies on all datasets. Further analysis and case studies are also provided to prove the importance of the proposed attention module and fully-shared multi-task learning method used in our model.<\/jats:p><\/jats:sec>","DOI":"10.1186\/s12859-022-04994-3","type":"journal-article","created":{"date-parts":[[2022,11,3]],"date-time":"2022-11-03T08:03:29Z","timestamp":1667462609000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Biomedical named entity recognition with the combined feature attention and fully-shared multi-task learning"],"prefix":"10.1186","volume":"23","author":[{"given":"Zhiyu","family":"Zhang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Arbee L. P.","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,11,3]]},"reference":[{"key":"4994_CR1","doi-asserted-by":"publisher","first-page":"83","DOI":"10.1016\/j.jbi.2018.03.011","volume":"81","author":"Y Zhang","year":"2018","unstructured":"Zhang Y, Lin H, Yang Z, Wang J, Zhang S, Sun Y, Yang L. A hybrid model based on neural networks for biomedical relation extraction. J Biomed Inform. 2018;81:83\u201392.","journal-title":"J Biomed Inform"},{"issue":"5","key":"4994_CR2","doi-asserted-by":"publisher","first-page":"756","DOI":"10.1002\/asi.20791","volume":"59","author":"J Li","year":"2008","unstructured":"Li J, Zhang Z, Li X, Chen H. Kernel-based learning for biomedical relation extraction. J Am Soc Inform Sci Technol. 2008;59(5):756\u201369.","journal-title":"J Am Soc Inform Sci Technol"},{"key":"4994_CR3","doi-asserted-by":"publisher","first-page":"6918381","DOI":"10.1155\/2016\/6918381","volume":"2016","author":"S Liu","year":"2016","unstructured":"Liu S, Tang B, Chen Q, Wang X. Drug-drug interaction extraction via convolutional neural networks. Comput Math Methods Med. 2016;2016:6918381.","journal-title":"Comput Math Methods Med"},{"issue":"5","key":"4994_CR4","doi-asserted-by":"publisher","first-page":"0122199","DOI":"10.1371\/journal.pone.0122199","volume":"10","author":"A Kolchinsky","year":"2015","unstructured":"Kolchinsky A, Louren\u00e7o A, Wu H-Y, Li L, Rocha LM. Extraction of pharmacokinetic evidence of drug\u2013drug interactions from the literature. PLoS ONE. 2015;10(5):0122199.","journal-title":"PLoS ONE"},{"key":"4994_CR5","doi-asserted-by":"crossref","unstructured":"Hao B, Zhu H, Paschalidis I. Enhancing clinical bert embedding using a biomedical knowledge base. In: Proceedings of the 28th international conference on computational linguistics; 2020. p. 657\u201361.","DOI":"10.18653\/v1\/2020.coling-main.57"},{"key":"4994_CR6","volume-title":"NormCo: deep disease normalization for biomedical knowledge base construction","author":"D Wright","year":"2019","unstructured":"Wright D. NormCo: deep disease normalization for biomedical knowledge base construction. San Diego: University of California; 2019."},{"key":"4994_CR7","doi-asserted-by":"crossref","unstructured":"Settles B. Biomedical named entity recognition using conditional random fields and rich feature sets. In: Proceedings of the international joint workshop on natural language processing in biomedicine and its applications (NLPBA\/BioNLP); 2004. p. 107\u201310","DOI":"10.3115\/1567594.1567618"},{"key":"4994_CR8","unstructured":"Kulick S, Bies A, Liberman M, Mandel M, McDonald R, Palmer M, Schein A, Ungar L, Winters S, White P. Integrated annotation for biomedical information extraction. In: HLT-NAACL 2004 workshop: linking biological literature, ontologies and databases; 2004. p. 61\u20138"},{"issue":"5","key":"4994_CR9","doi-asserted-by":"publisher","first-page":"497","DOI":"10.1197\/jamia.M2085","volume":"13","author":"H Liu","year":"2006","unstructured":"Liu H, Hu Z-Z, Torii M, Wu C, Friedman C. Quantitative assessment of dictionary-based protein named entity tagging. J Am Med Inform Assoc. 2006;13(5):497\u2013507.","journal-title":"J Am Med Inform Assoc"},{"key":"4994_CR10","doi-asserted-by":"crossref","unstructured":"Liao Z, Zhang Z. A generic classifier-ensemble approach for biomedical named entity recognition. In: Pacific-Asia conference on knowledge discovery and data mining. Springer; 2012. p. 86\u201397.","DOI":"10.1007\/978-3-642-30217-6_8"},{"issue":"6","key":"4994_CR11","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1016\/j.jbi.2004.08.012","volume":"37","author":"K-J Lee","year":"2004","unstructured":"Lee K-J, Hwang Y-S, Kim S, Rim H-C. Biomedical named entity recognition using two-phase model based on SVMs. J Biomed Inform. 2004;37(6):436\u201347.","journal-title":"J Biomed Inform"},{"issue":"1","key":"4994_CR12","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/1471-2105-14-54","volume":"14","author":"D Campos","year":"2013","unstructured":"Campos D, Matos S, Oliveira JL. Gimli: open source and high-performance biomedical name recognition. BMC Bioinformatics. 2013;14(1):1\u201314.","journal-title":"BMC Bioinformatics"},{"key":"4994_CR13","doi-asserted-by":"crossref","unstructured":"Liao, Z., Wu, H.: Biomedical named entity recognition based on skip-chain Crfs. In: 2012 International Conference on Industrial Control and Electronics Engineering. IEEE; 2012. p. 1495\u20138.","DOI":"10.1109\/ICICEE.2012.393"},{"issue":"1","key":"4994_CR14","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12859-017-1776-8","volume":"18","author":"G Crichton","year":"2017","unstructured":"Crichton G, Pyysalo S, Chiu B, Korhonen A. A neural network multi-task learning approach to biomedical named entity recognition. BMC Bioinformatics. 2017;18(1):1\u201314.","journal-title":"BMC Bioinformatics"},{"issue":"14","key":"4994_CR15","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1093\/bioinformatics\/btx228","volume":"33","author":"M Habibi","year":"2017","unstructured":"Habibi M, Weber L, Neves M, Wiegandt DL, Leser U. Deep learning with word embeddings improves biomedical named entity recognition. Bioinformatics. 2017;33(14):37\u201348.","journal-title":"Bioinformatics"},{"key":"4994_CR16","unstructured":"Devlin J, Chang M-W, Lee K, Toutanova K. Bert: Pre-training of deep bidirectional transformers for language understanding. 2018. arXiv preprint arXiv:1810.04805"},{"key":"4994_CR17","unstructured":"Yang Z, Dai Z, Yang Y, Carbonell J, Salakhutdinov RR, Le QV. Xlnet: Generalized autoregressive pretraining for language understanding. Advances in Neural Information Processing Systems 2019; 32."},{"key":"4994_CR18","unstructured":"Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M, Zettlemoyer L, Stoyanov V. Roberta: A robustly optimized bert pretraining approach. 2019. arXiv preprint arXiv:1907.11692"},{"issue":"4","key":"4994_CR19","doi-asserted-by":"crossref","first-page":"1234","DOI":"10.1093\/bioinformatics\/btz682","volume":"36","author":"J Lee","year":"2020","unstructured":"Lee J, Yoon W, Kim S, Kim D, Kim S, So CH, Kang J. BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics. 2020;36(4):1234\u201340.","journal-title":"Bioinformatics"},{"issue":"20","key":"4994_CR20","doi-asserted-by":"publisher","first-page":"3539","DOI":"10.1093\/bioinformatics\/bty356","volume":"34","author":"TH Dang","year":"2018","unstructured":"Dang TH, Le H-Q, Nguyen TM, Vu ST. D3NER: biomedical named entity recognition using CRF-bilSTM improved with fine-tuned embeddings of various linguistic information. Bioinformatics. 2018;34(20):3539\u201346.","journal-title":"Bioinformatics"},{"issue":"8","key":"4994_CR21","doi-asserted-by":"publisher","first-page":"1381","DOI":"10.1093\/bioinformatics\/btx761","volume":"34","author":"L Luo","year":"2018","unstructured":"Luo L, Yang Z, Yang P, Zhang Y, Wang L, Lin H, Wang J. An attention-based BilSTM-CRF approach to document-level chemical named entity recognition. Bioinformatics. 2018;34(8):1381\u20138.","journal-title":"Bioinformatics"},{"issue":"8","key":"4994_CR22","first-page":"279","volume":"8","author":"L Yao","year":"2015","unstructured":"Yao L, Liu H, Liu Y, Li X, Anwar MW. Biomedical named entity recognition based on deep neutral network. Int J Hybrid Inf Technol. 2015;8(8):279\u201388.","journal-title":"Int J Hybrid Inf Technol"},{"key":"4994_CR23","doi-asserted-by":"publisher","DOI":"10.1155\/2014\/240403","volume":"2014","author":"B Tang","year":"2014","unstructured":"Tang B, Cao H, Wang X, Chen Q, Xu H. Evaluating word representation features in biomedical named entity recognition tasks. BioMed Res Int. 2014;2014: 240403.","journal-title":"BioMed Res Int"},{"issue":"6","key":"4994_CR24","doi-asserted-by":"publisher","first-page":"411","DOI":"10.1016\/j.jbi.2004.08.005","volume":"37","author":"J Zhang","year":"2004","unstructured":"Zhang J, Shen D, Zhou G, Su J, Tan C-L. Enhancing hmm-based biomedical named entity recognition by studying special phenomena. J Biomed Inform. 2004;37(6):411\u201322.","journal-title":"J Biomed Inform"},{"issue":"1","key":"4994_CR25","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12859-020-03834-6","volume":"21","author":"Y Tian","year":"2020","unstructured":"Tian Y, Shen W, Song Y, Xia F, He M, Li K. Improving biomedical named entity recognition with syntactic information. BMC Bioinformatics. 2020;21(1):1\u201317.","journal-title":"BMC Bioinformatics"},{"key":"4994_CR26","doi-asserted-by":"crossref","unstructured":"Miller A, Fisch A, Dodge J, Karimi A-H, Bordes A, Weston J. Key-value memory networks for directly reading documents. 2016. arXiv preprint arXiv:1606.03126","DOI":"10.18653\/v1\/D16-1147"},{"key":"4994_CR27","doi-asserted-by":"crossref","unstructured":"Tian Y, Song Y, Ao X, Xia F, Quan X, Zhang T, Wang Y. Joint Chinese word segmentation and part-of-speech tagging via two-way attentions of auto-analyzed knowledge. In: Proceedings of the 58th annual meeting of the association for computational linguistics; 2020. p. 8286\u201396.","DOI":"10.18653\/v1\/2020.acl-main.735"},{"issue":"10","key":"4994_CR28","doi-asserted-by":"publisher","first-page":"1745","DOI":"10.1093\/bioinformatics\/bty869","volume":"35","author":"X Wang","year":"2019","unstructured":"Wang X, Zhang Y, Ren X, Zhang Y, Zitnik M, Shang J, Langlotz C, Han J. Cross-type biomedical named entity recognition with deep multi-task learning. Bioinformatics. 2019;35(10):1745\u201352.","journal-title":"Bioinformatics"},{"issue":"1","key":"4994_CR29","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12859-019-3000-5","volume":"20","author":"X Wang","year":"2019","unstructured":"Wang X, Lyu J, Dong L, Xu K. Multitask learning for biomedical named entity recognition with cross-sharing structure. BMC Bioinformatics. 2019;20(1):1\u201313.","journal-title":"BMC Bioinformatics"},{"issue":"10","key":"4994_CR30","first-page":"55","volume":"20","author":"W Yoon","year":"2019","unstructured":"Yoon W, So CH, Lee J, Kang J. Collabonet: collaboration of deep neural networks for biomedical named entity recognition. BMC Bioinformatics. 2019;20(10):55\u201365.","journal-title":"BMC Bioinformatics"},{"issue":"15","key":"4994_CR31","doi-asserted-by":"publisher","first-page":"4331","DOI":"10.1093\/bioinformatics\/btaa515","volume":"36","author":"M Zuo","year":"2020","unstructured":"Zuo M, Zhang Y. Dataset-aware multi-task learning approaches for biomedical named entity recognition. Bioinformatics. 2020;36(15):4331\u20138.","journal-title":"Bioinformatics"},{"key":"4994_CR32","unstructured":"Khan MR, Ziyadi M, AbdelHady M. Mt-bioner: multi-task learning for biomedical named entity recognition using deep bidirectional transformers. 2020. arXiv preprint arXiv:2001.08904"},{"key":"4994_CR33","unstructured":"Akdemir A, Shibuya T. Analyzing the effect of multi-task learning for biomedical named entity recognition. 2020. arXiv preprint arXiv:2011.00425"},{"issue":"1","key":"4994_CR34","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12859-021-04551-4","volume":"23","author":"Z Chai","year":"2022","unstructured":"Chai Z, Jin H, Shi S, Zhan S, Zhuo L, Yang Y. Hierarchical shared transfer learning for biomedical named entity recognition. BMC Bioinformatics. 2022;23(1):1\u201314.","journal-title":"BMC Bioinformatics"},{"key":"4994_CR35","doi-asserted-by":"crossref","unstructured":"Huang K, Huang D, Liu Z, Mo F. A joint multiple criteria model in transfer learning for cross-domain chinese word segmentation. In: Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP); 2020. p. 3873\u201382.","DOI":"10.18653\/v1\/2020.emnlp-main.318"},{"key":"4994_CR36","doi-asserted-by":"crossref","unstructured":"Manning CD, Surdeanu M, Bauer J, Finkel JR, Bethard S, McClosky D. The stanford corenlp natural language processing toolkit. In: Proceedings of 52nd annual meeting of the association for computational linguistics: system demonstrations; 2014. p. 55\u201360","DOI":"10.3115\/v1\/P14-5010"},{"key":"4994_CR37","unstructured":"Sang EF, De\u00a0Meulder F. Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. 2003. arXiv preprint arXiv:cs\/0306050v1."},{"key":"4994_CR38","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser \u0141, Polosukhin I. Attention is all you need. Advances in neural information processing systems. 2017;30."},{"key":"4994_CR39","unstructured":"Kingma DP, Ba J. Adam: A method for stochastic optimization. 2014. arXiv preprint arXiv:1412.6980"},{"key":"4994_CR40","doi-asserted-by":"crossref","unstructured":"Tong Y, Chen Y, Shi X. A multi-task approach for improving biomedical named entity recognition by incorporating multi-granularity information. In: Findings of the association for computational linguistics: ACL-IJCNLP 2021; 2021. p. 4804\u201313","DOI":"10.18653\/v1\/2021.findings-acl.424"}],"container-title":["BMC Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-022-04994-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12859-022-04994-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-022-04994-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,3,11]],"date-time":"2023-03-11T01:28:24Z","timestamp":1678498104000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/s12859-022-04994-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,3]]},"references-count":40,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["4994"],"URL":"https:\/\/doi.org\/10.1186\/s12859-022-04994-3","relation":{},"ISSN":["1471-2105"],"issn-type":[{"value":"1471-2105","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,3]]},"assertion":[{"value":"18 April 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 October 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 November 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"All authors have nothing to disclose.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}},{"value":"All authors have approved the submission of this manuscript.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}}],"article-number":"458"}}