{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,13]],"date-time":"2026-01-13T13:53:11Z","timestamp":1768312391244,"version":"3.49.0"},"reference-count":45,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2020,8,18]],"date-time":"2020-08-18T00:00:00Z","timestamp":1597708800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/501100004663","name":"Ministry of Science and Technology of Taiwan","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100004663","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004700","name":"Taipei Medical University","doi-asserted-by":"publisher","award":["MOST 107-2410-H-038-017-MY3"],"award-info":[{"award-number":["MOST 107-2410-H-038-017-MY3"]}],"id":[{"id":"10.13039\/501100004700","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004700","name":"Taipei Medical University","doi-asserted-by":"publisher","award":["MOST 107-2634-F-001-005"],"award-info":[{"award-number":["MOST 107-2634-F-001-005"]}],"id":[{"id":"10.13039\/501100004700","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004700","name":"Taipei Medical University","doi-asserted-by":"publisher","award":["TMU106-AE1-B16"],"award-info":[{"award-number":["TMU106-AE1-B16"]}],"id":[{"id":"10.13039\/501100004700","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,4,20]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Motivation<\/jats:title>\n                  <jats:p>Natural Language Processing techniques are constantly being advanced to accommodate the influx of data as well as to provide exhaustive and structured knowledge dissemination. Within the biomedical domain, relation detection between bio-entities known as the Bio-Entity Relation Extraction (BRE) task has a critical function in knowledge structuring. Although recent advances in deep learning-based biomedical domain embedding have improved BRE predictive analytics, these works are often task selective or use external knowledge-based pre-\/post-processing. In addition, deep learning-based models do not account for local syntactic contexts, which have improved data representation in many kernel classifier-based models. In this study, we propose a universal BRE model, i.e. LBERT, which is a Lexically aware Transformer-based Bidirectional Encoder Representation model, and which explores both local and global contexts representations for sentence-level classification tasks.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>This article presents one of the most exhaustive BRE studies ever conducted over five different bio-entity relation types. Our model outperforms state-of-the-art deep learning models in protein\u2013protein interaction (PPI), drug\u2013drug interaction and protein\u2013bio-entity relation classification tasks by 0.02%, 11.2% and 41.4%, respectively. LBERT representations show a statistically significant improvement over BioBERT in detecting true bio-entity relation for large corpora like PPI. Our ablation studies clearly indicate the contribution of the lexical features and distance-adjusted attention in improving prediction performance by learning additional local semantic context along with bi-directionally learned global context.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Availability and implementation<\/jats:title>\n                  <jats:p>Github. https:\/\/github.com\/warikoone\/LBERT.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Supplementary information<\/jats:title>\n                  <jats:p>Supplementary data are available at Bioinformatics online.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btaa721","type":"journal-article","created":{"date-parts":[[2020,8,13]],"date-time":"2020-08-13T11:56:13Z","timestamp":1597319773000},"page":"404-412","source":"Crossref","is-referenced-by-count":24,"title":["LBERT: Lexically aware Transformer-based Bidirectional Encoder Representation model for learning universal bio-entity relations"],"prefix":"10.1093","volume":"37","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4222-8970","authenticated-orcid":false,"given":"Neha","family":"Warikoo","sequence":"first","affiliation":[{"name":"Institute of Biomedical Informatics, National Yang-Ming University , Taipei 112, Taiwan"},{"name":"Bioinformatics Program, Taiwan International Graduate Program, Institute of Information Science , Academia Sinica, Taipei 115, Taiwan"},{"name":"Institute of Information Science, Academia Sinica , Taipei 11529, Taiwan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9634-8380","authenticated-orcid":false,"given":"Yung-Chun","family":"Chang","sequence":"additional","affiliation":[{"name":"Graduate Institute of Data Science, College of Management, Taipei Medical University , Taipei 106, Taiwan"},{"name":"Clinical Big Data Research Center, Taipei Medical University , Taipei 110, Taiwan"},{"name":"Pervasive AI Research Labs, Ministry of Science and Technology , Hsinchu City 300, Taiwan"}]},{"given":"Wen-Lian","family":"Hsu","sequence":"additional","affiliation":[{"name":"Institute of Information Science, Academia Sinica , Taipei 11529, Taiwan"},{"name":"Pervasive AI Research Labs, Ministry of Science and Technology , Hsinchu City 300, Taiwan"}]}],"member":"286","published-online":{"date-parts":[[2020,8,18]]},"reference":[{"key":"2023051604092635000_btaa721-B1","doi-asserted-by":"crossref","first-page":"S2","DOI":"10.1186\/1471-2105-9-S11-S2","article-title":"All-paths graph kernel for protein\u2013protein interaction extraction with evaluation of cross-corpus learning","volume":"9","author":"Airola","year":"2008","journal-title":"BMC Bioinformatics"},{"key":"2023051604092635000_btaa721-B2","author":"Asadi","year":"2017"},{"key":"2023051604092635000_btaa721-B3","doi-asserted-by":"crossref","first-page":"e0200699","DOI":"10.1371\/journal.pone.0200699","article-title":"Automatic extraction of gene\u2013disease associations from literature using joint ensemble learning","volume":"13","author":"Bhasuran","year":"2018","journal-title":"PLoS One"},{"key":"2023051604092635000_btaa721-B4","doi-asserted-by":"crossref","first-page":"S4","DOI":"10.1186\/1471-2105-13-S11-S4","article-title":"University of Turku in the BioNLP'11 shared task","volume":"13","author":"Bj\u00f6rne","year":"2012","journal-title":"BMC Bioinformatics"},{"key":"2023051604092635000_btaa721-B5","first-page":"2787","author":"Bordes","year":"2013"},{"key":"2023051604092635000_btaa721-B6","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1186\/s12859-015-0472-9","article-title":"Extraction of relations between genes and diseases from text and large-scale data analysis: implications for translational research","volume":"16","author":"Bravo","year":"2015","journal-title":"BMC Bioinformatics"},{"key":"2023051604092635000_btaa721-B7","doi-asserted-by":"crossref","first-page":"baw101","DOI":"10.1093\/database\/baw101","article-title":"PIPE: a protein\u2013protein interaction passage extraction module for BioCreative challenge","volume":"2016","author":"Chang","year":"2016","journal-title":"Database (Oxford)"},{"key":"2023051604092635000_btaa721-B8","author":"Chiu","year":"2016"},{"key":"2023051604092635000_btaa721-B9","doi-asserted-by":"crossref","first-page":"bay066","DOI":"10.1093\/database\/bay066","article-title":"Improving the learning of chemical\u2013protein interactions from literature using transfer learning and specialized word embeddings","volume":"2018","author":"Corbett","year":"2018","journal-title":"Database"},{"key":"2023051604092635000_btaa721-B10","author":"Devlin","year":"2019"},{"key":"2023051604092635000_btaa721-B11","doi-asserted-by":"crossref","DOI":"10.3792\/chmm\/1263317248","volume-title":"Mathematical Monongraphs Algebraic Invariants, No.14","author":"Dickson","year":"1914"},{"key":"2023051604092635000_btaa721-B12","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1186\/s12859-015-0549-5","article-title":"KnowLife: a versatile approach for constructing a large knowledge graph for biomedical sciences","volume":"16","author":"Ernst","year":"2015","journal-title":"BMC Bioinformatics"},{"key":"2023051604092635000_btaa721-B13","doi-asserted-by":"crossref","first-page":"i37","DOI":"10.1093\/bioinformatics\/btx228","article-title":"Deep learning with word embeddings improves biomedical named entity recognition","volume":"33","author":"Habibi","year":"2017","journal-title":"Bioinformatics"},{"key":"2023051604092635000_btaa721-B14","doi-asserted-by":"crossref","first-page":"914","DOI":"10.1016\/j.jbi.2013.07.011","article-title":"The DDI corpus: an annotated corpus with pharmacological substances and drug\u2013drug interactions","volume":"46","author":"Herrero-Zazo","year":"2013","journal-title":"J. Biomed. Inform"},{"key":"2023051604092635000_btaa721-B15","volume-title":"Theory of Algebraic Invariants, (English Translation)","author":"Hilbert","year":"1993"},{"key":"2023051604092635000_btaa721-B16","first-page":"1","article-title":"A shortest dependency path based convolutional neural network for protein\u2013protein relation extraction","volume":"2016","author":"Hua","year":"2016","journal-title":"BioMed. Res. Int"},{"key":"2023051604092635000_btaa721-B17","doi-asserted-by":"crossref","first-page":"1143","DOI":"10.1109\/34.334397","article-title":"Using symbolic computation to find algebraic invariants","volume":"16","author":"Keren","year":"1994","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"2023051604092635000_btaa721-B18","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1016\/j.jbi.2015.03.002","article-title":"Extracting drug-drug interactions from literature using a rich feature-based linear kernel approach","volume":"55","author":"Kim","year":"2015","journal-title":"J. Biomed. Inform"},{"key":"2023051604092635000_btaa721-B19","article-title":"Structured attention networks","author":"Kim","year":"2017"},{"key":"2023051604092635000_btaa721-B20","author":"Krallinger","year":"2017"},{"key":"2023051604092635000_btaa721-B21","first-page":"1234","article-title":"BioBERT: a pre-trained biomedical language representation model for biomedical text mining","volume":"36","author":"Lee","year":"2020"},{"key":"2023051604092635000_btaa721-B22","doi-asserted-by":"crossref","first-page":"e60954","DOI":"10.1371\/journal.pone.0060954","article-title":"Approximate subgraph matching-based literature mining for biomedical events and relations","volume":"8","author":"Liu","year":"2013","journal-title":"PLoS ONE"},{"key":"2023051604092635000_btaa721-B23","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1155\/2016\/4310634","article-title":"Drug\u2013drug interaction extraction via convolutional neural networks","volume":"2016","author":"Liu","year":"2016","journal-title":"Comp. Math. Methods Med"},{"key":"2023051604092635000_btaa721-B24","doi-asserted-by":"crossref","first-page":"bay060","DOI":"10.1093\/database\/bay060","article-title":"Chemical\u2013gene relation extraction using recursive neural network","volume":"2018","author":"Lim","year":"2018","journal-title":"Database"},{"key":"2023051604092635000_btaa721-B25","doi-asserted-by":"crossref","first-page":"bay138","DOI":"10.1093\/database\/bay138","article-title":"Extracting chemical\u2013protein interactions from literature using sentence structure analysis and feature engineering","volume":"2019","author":"Lung","year":"2019","journal-title":"Database"},{"key":"2023051604092635000_btaa721-B26","volume-title":"Foundations of Statistical Natural Language Processing","author":"Manning","year":"1999","edition":"1st edn"},{"key":"2023051604092635000_btaa721-B27","first-page":"3111","volume-title":"Proceedings of the 26th International Conference on Neural Information Processing Systems \u2013 Volume 2 (NIPS'13)","author":"Mikolov","year":"2013"},{"key":"2023051604092635000_btaa721-B28","doi-asserted-by":"crossref","first-page":"e0187379","DOI":"10.1371\/journal.pone.0187379","article-title":"Distributed smoothed tree kernel for protein\u2013protein interaction extraction from the biomedical literature","volume":"12","author":"Murugesan","year":"2017","journal-title":"PLoS ONE"},{"key":"2023051604092635000_btaa721-B29","author":"Peng","year":"2017"},{"key":"2023051604092635000_btaa721-B30","doi-asserted-by":"crossref","first-page":"bay073","DOI":"10.1093\/database\/bay073","article-title":"Extracting chemical-protein relations with ensembles of SVM and deep learning models","volume":"2018","author":"Peng","year":"2018","journal-title":"Database (Oxford)"},{"key":"2023051604092635000_btaa721-B31","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/N18-1202","article-title":"Deep contextualized word representations","author":"Peters","year":"2018"},{"key":"2023051604092635000_btaa721-B32","doi-asserted-by":"crossref","first-page":"S6","DOI":"10.1186\/1471-2105-9-S3-S6","article-title":"Comparative analysis of five protein\u2013protein interaction corpora","volume":"9","author":"Pyysalo","year":"2008","journal-title":"BMC Bioinformatics"},{"key":"2023051604092635000_btaa721-B33","author":"Qu","year":"2018"},{"key":"2023051604092635000_btaa721-B34","first-page":"1","article-title":"Multichannel convolutional neural network for biological relation extraction","volume":"2016","author":"Quan","year":"2016","journal-title":"BioMed. Res. Int"},{"key":"2023051604092635000_btaa721-B35","author":"Radford","year":"2018"},{"key":"2023051604092635000_btaa721-B36","first-page":"83","article-title":"Overview of the entity relations (REL) supporting task of BioNLP Shared Task 2011","volume-title":"Proceedings of the BioNLP Shared Task 2011 Workshop (BioNLP Shared Task '11)","author":"Sampo","year":"2011"},{"key":"2023051604092635000_btaa721-B38","author":"Toutanova","year":"2015"},{"key":"2023051604092635000_btaa721-B39","author":"Trisedya","year":"2019"},{"key":"2023051604092635000_btaa721-B40","author":"Vaswani","year":"2017"},{"key":"2023051604092635000_btaa721-B41","doi-asserted-by":"crossref","first-page":"879","DOI":"10.1016\/j.jbi.2012.04.004","article-title":"The EU-ADR corpus: annotated drugs, diseases, targets, and theirrelationships","volume":"45","author":"Van Mulligen","year":"2012","journal-title":"J. Biomed. Inform"},{"key":"2023051604092635000_btaa721-B42","doi-asserted-by":"crossref","first-page":"bay108","DOI":"10.1093\/database\/bay108","article-title":"LPTK: a linguistic pattern-aware dependency tree kernel approach for the BioCreative VI CHEMPROT task","volume":"2018","author":"Warikoo","year":"2018","journal-title":"Database"},{"key":"2023051604092635000_btaa721-B43","doi-asserted-by":"crossref","first-page":"80","DOI":"10.2307\/3001968","article-title":"Individual comparisons by ranking methods","volume":"1","author":"Wilcoxon","year":"1945","journal-title":"Biometrics"},{"key":"2023051604092635000_btaa721-B44","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1016\/j.knosys.2018.11.020","article-title":"Feature assisted stacked attentive shortest dependency path based Bi-LSTM model for protein\u2013protein interaction","volume":"166","author":"Yadav","year":"2019","journal-title":"Knowl. Based Syst"},{"key":"2023051604092635000_btaa721-B46","doi-asserted-by":"crossref","first-page":"89354","DOI":"10.1109\/ACCESS.2019.2927253","article-title":"Deep residual convolutional neural network for protein\u2013protein interaction extraction","volume":"7","author":"Zhang","year":"2019","journal-title":"IEEE Access"},{"key":"2023051604092635000_btaa721-B47","doi-asserted-by":"crossref","first-page":"3444","DOI":"10.1093\/bioinformatics\/btw486","article-title":"Drug drug interaction extraction from biomedical literature using syntax convolutional neural network","volume":"32","author":"Zhao","year":"2016","journal-title":"Bioinformatics"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btaa721\/34865503\/btaa721.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/37\/3\/404\/50326129\/btaa721.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/37\/3\/404\/50326129\/btaa721.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,16]],"date-time":"2023-05-16T04:10:50Z","timestamp":1684210250000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/37\/3\/404\/5893949"}},"subtitle":[],"editor":[{"given":"Jonathan","family":"Wren","sequence":"additional","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2020,8,18]]},"references-count":45,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2021,4,20]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btaa721","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,2,1]]},"published":{"date-parts":[[2020,8,18]]}}}