{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,8]],"date-time":"2026-09-08T11:21:05Z","timestamp":1788866465621,"version":"build-2803163510"},"reference-count":43,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2024,12,9]],"date-time":"2024-12-09T00:00:00Z","timestamp":1733702400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Sichan province key research","award":["2023YFG0001"],"award-info":[{"award-number":["2023YFG0001"]}]},{"name":"Sichan province key research","award":["QZQN25F050009"],"award-info":[{"award-number":["QZQN25F050009"]}]},{"name":"Sichan province key research","award":["2023D021"],"award-info":[{"award-number":["2023D021"]}]},{"name":"Quzhou Joint Fund of the Zhejiang Provincial Natural Science Foundation of China","award":["2023YFG0001"],"award-info":[{"award-number":["2023YFG0001"]}]},{"name":"Quzhou Joint Fund of the Zhejiang Provincial Natural Science Foundation of China","award":["QZQN25F050009"],"award-info":[{"award-number":["QZQN25F050009"]}]},{"name":"Quzhou Joint Fund of the Zhejiang Provincial Natural Science Foundation of China","award":["2023D021"],"award-info":[{"award-number":["2023D021"]}]},{"name":"Municipal Government of Quzhou","award":["2023YFG0001"],"award-info":[{"award-number":["2023YFG0001"]}]},{"name":"Municipal Government of Quzhou","award":["QZQN25F050009"],"award-info":[{"award-number":["QZQN25F050009"]}]},{"name":"Municipal Government of Quzhou","award":["2023D021"],"award-info":[{"award-number":["2023D021"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>In the expanding landscape of biomedical literature, numerous latent associations outlined in scholarly papers await discovery and integration into biomedical databases. Biomedical Natural Language Processing (NLP) research focuses on automating knowledge extraction and mining from this literature, particularly emphasizing the essential task of Relation Extraction (RE). However, existing models have limitations, mainly in their applicability to partial datasets for RE tasks. Moreover, conventional models often treat RE as a binary classification challenge, which proves suboptimal given the diverse relationships, including intricate ones like similarity and hierarchy, present in the RE dataset. These limitations are exacerbated by the models\u2019 inability to capture word-level positional nuances and sentence-level language features. In response to these challenges, we present a novel RE model called BicapBert. This model integrates neural networks and capsule networks, enhancing them with hybrid knowledge graph embeddings to extract relevant features. BicapBert utilizes PubMedBERT and capsule networks to extract word-level positional and sentence-level language features. It further captures knowledge features from a biomedical knowledge graph, integrating them with the aforementioned linguistic features. The amalgamated information is then input into a multi-layer perceptron, culminating in results derived through a softmax classifier. Experimental evaluations on three extensive RE task datasets showcase the state-of-the-art performance of our proposed model. Additionally, we validate the model\u2019s efficacy on three randomly selected biomedical datasets for various tasks, further affirming its superiority.<\/jats:p>","DOI":"10.3390\/sym16121629","type":"journal-article","created":{"date-parts":[[2024,12,9]],"date-time":"2024-12-09T06:16:14Z","timestamp":1733724974000},"page":"1629","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Relational Extraction from Biomedical Texts with Capsule Network and Hybrid Knowledge Graph Embeddings"],"prefix":"10.3390","volume":"16","author":[{"given":"Yutong","family":"Chen","sequence":"first","affiliation":[{"name":"Department of Biostatistics, City University of Hong Kong, Hong Kong 999077, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xia","family":"Li","sequence":"additional","affiliation":[{"name":"Yangtze River Delta Research Institute, University of Electronic Science and Technology of China, Quzhou 324000, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yang","family":"Liu","sequence":"additional","affiliation":[{"name":"Yangtze River Delta Research Institute, University of Electronic Science and Technology of China, Quzhou 324000, China"},{"name":"Institute of Integrated Circuit Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611700, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peng","family":"Bi","sequence":"additional","affiliation":[{"name":"Yangtze River Delta Research Institute, University of Electronic Science and Technology of China, Quzhou 324000, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tiangui","family":"Hu","sequence":"additional","affiliation":[{"name":"Yangtze River Delta Research Institute, University of Electronic Science and Technology of China, Quzhou 324000, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,12,9]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Beltagy, I., Lo, K., and Cohan, A. (2019, January 3\u20137). Scibert: A pretrained language model for scientific text. Proceedings of the EMNLP-IJCNLP 2019, Hong Kong, China.","DOI":"10.18653\/v1\/D19-1371"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Peng, Y., Yan, S., and Lu, Z. (2019, January 1). Transfer learning in biomedical natural language processing: An evaluation of BERT and elmo on ten benchmarking datasets. Proceedings of the 18th BioNLP Workshop and Shared Task, BioNLP@ACL 2019, Florence, Italy.","DOI":"10.18653\/v1\/W19-5006"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1234","DOI":"10.1093\/bioinformatics\/btz682","article-title":"Biobert: A pre-trained biomedical language representation model for biomedical text mining","volume":"36","author":"Lee","year":"2020","journal-title":"Bioinformatics"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Peng, Y., Chen, Q., and Lu, Z. (2020, January 9). An empirical study of multi-task learning on BERT for biomedical text mining. Proceedings of the 19th SIGBioMed Workshop on Biomedical Language Processing, BioNLP 2020, Online.","DOI":"10.18653\/v1\/2020.bionlp-1.22"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2:1","DOI":"10.1145\/3458754","article-title":"Domain-specific language model pretraining for biomedical natural language processing","volume":"3","author":"Gu","year":"2022","journal-title":"ACM Trans. Comput. Healthc."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Huang, L., Lin, J., Li, X., Song, L., Zheng, Z., and Wong, K. (2022). EGFI: Drug-drug interaction extraction and generation with fusion of enriched entity and sentence information. Briefings Bioinform., 23.","DOI":"10.1093\/bib\/bbab451"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Jin, X., Sun, X., Chen, J., and Sutcliffe, R.F.E. (2022, January 17\u201321). Extracting drug-drug interactions from biomedical texts using knowledge graph embeddings and multi-focal loss. Proceedings of the 31st ACM International Conference on Information & Knowledge Management, Atlanta, GA, USA.","DOI":"10.1145\/3511808.3557318"},{"key":"ref_8","unstructured":"Veyseh, A.P.B., Dernoncourt, F., Nguyen, T.H., Chang, W., and Celi, L.A. (2021). BERTKG-DDI: Towards incorporating entity-specific knowledge graph information in predicting drug-drug interactions. AAAI, CEUR-WS.org. CEUR Workshop Proceedings."},{"key":"ref_9","first-page":"467","article-title":"Class-based n-gram models of natural language","volume":"18","author":"Brown","year":"1992","journal-title":"Comput. Linguist."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Blitzer, J., and Zhu, X. (2008). Semi-supervised learning for natural language processing. ACL, The Association for Computer Linguistics.","DOI":"10.3115\/1564169.1564172"},{"key":"ref_11","unstructured":"Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013, January 2\u20134). Efficient estimation of word representations in vector space. Proceedings of the International Conference on Learning Representations (ICLR), Scottsdale, AZ, USA."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Pennington, J., Socher, R., and Manning, C.D. (2014, January 25\u201329). Glove: Global vectors for word representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.","DOI":"10.3115\/v1\/D14-1162"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Howard, J., and Ruder, S. (2018). Universal language model fine-tuning for text classification. ACL, Association for Computational Linguistics.","DOI":"10.18653\/v1\/P18-1031"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Peters, M.E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., and Zettlemoyer, L. (2018). Deep contextualized word representations. NAACL-HLT, Association for Computational Linguistics.","DOI":"10.18653\/v1\/N18-1202"},{"key":"ref_15","unstructured":"Devlin, J., Chang, M., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. NAACL-HLT, Association for Computational Linguistics."},{"key":"ref_16","unstructured":"Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). Roberta: A robustly optimized BERT pretraining approach. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Arighi, C.N., and Wu, C.H. (2011). Biocreative III interactive task: An overview. BMC Bioinform., 12.","DOI":"10.1186\/1471-2105-12-S8-S1"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Mao, Y., Auken, K.V., Li, D., Arighi, C.N., Dai, H., and Lu, Z. (2014). Overview of the gene ontology task at biocreative IV. Database J. Biol. Databases Curation, 2014.","DOI":"10.1093\/database\/bau086"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Sennrich, R., Haddow, B., and Birch, A. (2016). Neural machine translation of rare words with subword units. ACL, The Association for Computer Linguistics.","DOI":"10.18653\/v1\/P16-1162"},{"key":"ref_20","unstructured":"Kim, J., Wang, Y., Takagi, T., and Yonezawa, A. (2011). Overview of Genia Event Task in Bionlp Shared Task 2011, Association for Computational Linguistics."},{"key":"ref_21","unstructured":"Diab, M.T., Baldwin, T., and Baroni, M. (2013). NAACL-HLT, The Association for Computer Linguistics."},{"key":"ref_22","first-page":"553","article-title":"Results of the seventh edition of the bioasq challenge","volume":"Volume 1168","author":"Nentidis","year":"2019","journal-title":"ECMLPKDD"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., and Bowman, S.R. (2019). GLUE: A multi-task benchmark and analysis platform for natural language understanding. ICLR, OpenReview.net.","DOI":"10.18653\/v1\/W18-5446"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1739","DOI":"10.1093\/bioinformatics\/btaa907","article-title":"Using drug descriptions and molecular structures for drug-drug interaction extraction from literature","volume":"37","author":"Asada","year":"2021","journal-title":"Bioinformatics"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Farrugia, L., Azzopardi, L.M., Debattista, J., and Abela, C. (2023). Predicting Drug-Drug Interactions Using Knowledge Graphs. arXiv.","DOI":"10.21203\/rs.3.rs-4492557\/v1"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Zhang, S., Liang, X., Niu, S., Zhang, X., Feng, C., and Ma, Y. (2024, January 20\u201327). Biomedical Knowledge Graph Embedding with Householder Projection (Student Abstract). Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada.","DOI":"10.1609\/aaai.v38i21.30535"},{"key":"ref_27","unstructured":"Sabour, S., Frosst, N., and Hinton, G.E. (2017, January 4\u20139). Dynamic routing between capsules. Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, Long Beach, CA, USA."},{"key":"ref_28","unstructured":"Hinton, G.E., Sabour, S., and Frosst, N. (2024, December 02). Matrix Capsules with EM Routing. Available online: https:\/\/openreview.net\/pdf?id=HJWLfGWRb."},{"key":"ref_29","unstructured":"Tsai, Y.H., Srivastava, N., Goh, H., and Salakhutdinov, R. (2024, December 02). Capsules with Inverted Dot-Product Attention Routing. Available online: https:\/\/openreview.net\/pdf?id=HJe6uANtwH."},{"key":"ref_30","unstructured":"Ba, L.J., Kiros, J.R., and Hinton, G.E. (2016). Layer normalization. arXiv."},{"key":"ref_31","unstructured":"Bordes, A., and Usunier, N. Translating embeddings for modeling multi-relational data. Proceedings of the 27th Conference on Neural Information Processing Systems (NIPS)."},{"key":"ref_32","unstructured":"Sun, Z., Deng, Z.H., Nie, J.Y., and Tang, J. (2019, January 6\u20139). Rotate: Knowledge graph embedding by relational rotation in complex space. Proceedings of the International Conference on Learning Representations (ICLR), New Orleans, LA, USA."},{"key":"ref_33","unstructured":"Zhang, S., and Tay, Y. (2019, January 8\u201314). Quaternion knowledge graph embeddings. Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS), Vancouver, BC, Canada."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Cao, Z., Xu, Q., and Yang, Z. (2021). Dual quaternion knowledge graph embeddings. AAAI, AAAI Press.","DOI":"10.1609\/aaai.v35i8.16850"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Guo, J., and Kok, S. (2021). Bique: Biquaternionic embeddings of knowledge graphs. EMNLP, Association for Computational Linguistics.","DOI":"10.18653\/v1\/2021.emnlp-main.657"},{"key":"ref_36","first-page":"5521","article-title":"Geometry interaction knowledge graph embeddings","volume":"36","author":"Cao","year":"2022","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Kudo, T., and Richardson, J. (2018). Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. EMNLP, Association for Computational Linguistics.","DOI":"10.18653\/v1\/D18-2012"},{"key":"ref_38","unstructured":"Wallach, H.M., Larochelle, H., Beygelzimer, A., d\u2019Alch\u00e9-Buc, F., Fox, E.B., and Garnett, R. (2019, January 8\u201314). Multi-relational Poincar\u00e9 graph embeddings. Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS), Vancouver, BC, Canada."},{"key":"ref_39","unstructured":"Kingma, D.P., and Ba, J. (2015, January 7\u20139). Adam: A method for stochastic optimization. Proceedings of the International Conference on Learning Representations (ICLR), San Diego, CA, USA."},{"key":"ref_40","first-page":"2071","article-title":"Complex embeddings for simple link prediction","volume":"Volume 48","author":"Trouillon","year":"2016","journal-title":"Proceedings of the 33rd International Conference on Machine Learning (ICML)"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"914","DOI":"10.1016\/j.jbi.2013.07.011","article-title":"The DDI corpus: An annotated corpus with pharmacological substances and drug-drug interactions","volume":"46","author":"Declerck","year":"2013","journal-title":"J. Biomed. Inform."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"55:1","DOI":"10.1186\/s12859-015-0472-9","article-title":"Extraction of relations between genes and diseases from text and large-scale data analysis: Implications for translational research","volume":"16","author":"Bravo","year":"2015","journal-title":"BMC Bioinform."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Lin, T., Goyal, P., Girshick, R.B., He, K., and Doll\u00e1r, P. (2017, January 22\u201329). Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy.","DOI":"10.1109\/ICCV.2017.324"}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/16\/12\/1629\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:50:20Z","timestamp":1760115020000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/16\/12\/1629"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,9]]},"references-count":43,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["sym16121629"],"URL":"https:\/\/doi.org\/10.3390\/sym16121629","relation":{},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,9]]}}}