{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,24]],"date-time":"2026-02-24T06:31:49Z","timestamp":1771914709136,"version":"3.50.1"},"reference-count":31,"publisher":"Oxford University Press (OUP)","issue":"4","license":[{"start":{"date-parts":[[2023,4,5]],"date-time":"2023-04-05T00:00:00Z","timestamp":1680652800000},"content-version":"vor","delay-in-days":4,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100006129","name":"FCT","doi-asserted-by":"publisher","award":["PTDC\/CCIBIO\/28685\/2017"],"award-info":[{"award-number":["PTDC\/CCIBIO\/28685\/2017"]}],"id":[{"id":"10.13039\/100006129","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,4,3]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Motivation<\/jats:title>\n                  <jats:p>Relation extraction (RE) is a crucial process to deal with the amount of text published daily, e.g. to find missing associations in a database. RE is a text mining task for which the state-of-the-art approaches use bidirectional encoders, namely, BERT. However, state-of-the-art performance may be limited by the lack of efficient external knowledge injection approaches, with a larger impact in the biomedical area given the widespread usage and high quality of biomedical ontologies. This knowledge can propel these systems forward by aiding them in predicting more explainable biomedical associations. With this in mind, we developed K-RET, a novel, knowledgeable biomedical RE system that, for the first time, injects knowledge by handling different types of associations, multiple sources and where to apply it, and multi-token entities.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>We tested K-RET on three independent and open-access corpora (DDI, BC5CDR, and PGR) using four biomedical ontologies handling different entities. K-RET improved state-of-the-art results by 2.68% on average, with the DDI Corpus yielding the most significant boost in performance, from 79.30% to 87.19% in F-measure, representing a P-value of 2.91\u00d710\u221212.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Availability and implementation<\/jats:title>\n                  <jats:p>https:\/\/github.com\/lasigeBioTM\/K-RET.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btad174","type":"journal-article","created":{"date-parts":[[2023,4,5]],"date-time":"2023-04-05T17:03:58Z","timestamp":1680714238000},"source":"Crossref","is-referenced-by-count":12,"title":["K-RET: knowledgeable biomedical relation extraction system"],"prefix":"10.1093","volume":"39","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0597-9273","authenticated-orcid":false,"given":"Diana F","family":"Sousa","sequence":"first","affiliation":[{"name":"Departamento de Inform\u00e1tica, Faculdade de Ci\u00eancias, Universidade de Lisboa , Lisboa 1749-016, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0627-1496","authenticated-orcid":false,"given":"Francisco M","family":"Couto","sequence":"additional","affiliation":[{"name":"Departamento de Inform\u00e1tica, Faculdade de Ci\u00eancias, Universidade de Lisboa , Lisboa 1749-016, Portugal"}]}],"member":"286","published-online":{"date-parts":[[2023,4,5]]},"reference":[{"key":"2023041820544022800_btad174-B1","doi-asserted-by":"crossref","first-page":"e30401","DOI":"10.2196\/30401","article-title":"Machine learning approaches to retrieve high-quality, clinically relevant evidence from the biomedical literature: systematic review","volume":"9","author":"Abdelkader","year":"2021","journal-title":"JMIR Med Inform"},{"key":"2023041820544022800_btad174-B2","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1038\/75556","article-title":"Gene ontology: tool for the unification of biology","volume":"25","author":"Ashburner","year":"2000","journal-title":"Nat Genet"},{"key":"2023041820544022800_btad174-B3","first-page":"3615","author":"Beltagy","year":"2019"},{"key":"2023041820544022800_btad174-B4","doi-asserted-by":"crossref","first-page":"D267","DOI":"10.1093\/nar\/gkh061","article-title":"The unified medical language system (UMLS): integrating biomedical terminology","volume":"32","author":"Bodenreider","year":"2004","journal-title":"Nucleic Acids Res"},{"key":"2023041820544022800_btad174-B5","doi-asserted-by":"crossref","first-page":"D330","DOI":"10.1093\/nar\/gky1055","article-title":"The gene ontology resource: 20 years and still going strong","volume":"47","author":"The Gene Ontology Consortium","year":"2019","journal-title":"Nucleic Acids Res"},{"key":"2023041820544022800_btad174-B6","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-33966-1","volume-title":"Deep Learning Techniques for Biomedical and Health Informatics","author":"Dash","year":"2020"},{"key":"2023041820544022800_btad174-B7","doi-asserted-by":"crossref","first-page":"D344","DOI":"10.1093\/nar\/gkm791","article-title":"ChEBI: a database and ontology for chemical entities of biological interest","volume":"36","author":"Degtyarenko","year":"2008","journal-title":"Nucleic Acids Res"},{"key":"2023041820544022800_btad174-B8","doi-asserted-by":"crossref","first-page":"636","DOI":"10.1007\/s10489-021-02460-w","article-title":"Developing a BERT based triple classification model using knowledge graph embedding for question answering system","volume":"52","author":"Do","year":"2022","journal-title":"Appl Intell"},{"key":"2023041820544022800_btad174-B9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3458754","article-title":"Domain-specific language model pretraining for biomedical natural language processing","volume":"3","author":"Gu","year":"2022","journal-title":"ACM Trans Comput Healthc"},{"key":"2023041820544022800_btad174-B10","author":"Hao","year":"2020"},{"key":"2023041820544022800_btad174-B11","doi-asserted-by":"crossref","first-page":"914","DOI":"10.1016\/j.jbi.2013.07.011","article-title":"The ddi corpus: an annotated corpus with pharmacological substances and drug\u2013drug interactions","volume":"46","author":"Herrero-Zazo","year":"2013","journal-title":"J Biomed Inform"},{"key":"2023041820544022800_btad174-B12","doi-asserted-by":"crossref","first-page":"140628","DOI":"10.1109\/ACCESS.2021.3119621","article-title":"Machine learning techniques for biomedical natural language processing: a comprehensive review","volume":"9","author":"Houssein","year":"2021","journal-title":"IEEE Access"},{"key":"2023041820544022800_btad174-B13","doi-asserted-by":"crossref","first-page":"bbab036","DOI":"10.1093\/bib\/bbab036","article-title":"A survey on computational models for predicting protein\u2013protein interactions","volume":"22","author":"Hu","year":"2021","journal-title":"Brief Bioinform"},{"key":"2023041820544022800_btad174-B14","first-page":"4171","author":"Kenton","year":"2019"},{"key":"2023041820544022800_btad174-B15","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s12859-020-3517-7","article-title":"Broad-coverage biomedical relation extraction with semrep","volume":"21","author":"Kilicoglu","year":"2020","journal-title":"BMC Bioinformatics"},{"key":"2023041820544022800_btad174-B16","doi-asserted-by":"crossref","first-page":"597","DOI":"10.1093\/bioinformatics\/btk016","article-title":"Biocontrasts: extracting and exploiting protein\u2013protein contrastive relations from biomedical literature","volume":"22","author":"Kim","year":"2006","journal-title":"Bioinformatics"},{"key":"2023041820544022800_btad174-B17","doi-asserted-by":"crossref","first-page":"D1207","DOI":"10.1093\/nar\/gkaa1043","article-title":"The human phenotype ontology in 2021","volume":"49","author":"K\u00f6hler","year":"2021","journal-title":"Nucleic Acids Res"},{"key":"2023041820544022800_btad174-B18","doi-asserted-by":"crossref","first-page":"1234","DOI":"10.1093\/bioinformatics\/btz682","article-title":"BioBERT: a pre-trained biomedical language representation model for biomedical text mining","volume":"36","author":"Lee","year":"2020","journal-title":"Bioinformatics"},{"key":"2023041820544022800_btad174-B19","doi-asserted-by":"crossref","first-page":"baw068","DOI":"10.1093\/database\/baw068","article-title":"Biocreative v cdr task corpus: a resource for chemical disease relation extraction","volume":"2016","author":"Li","year":"2016","journal-title":"Database"},{"key":"2023041820544022800_btad174-B20","first-page":"2901","author":"Liu","year":"2020"},{"key":"2023041820544022800_btad174-B21","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3445965","article-title":"Named entity recognition and relation extraction: state-of-the-art","volume":"54","author":"Nasar","year":"2022","journal-title":"ACM Comput Surv"},{"key":"2023041820544022800_btad174-B22","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1016\/j.artmed.2006.08.005","article-title":"Mining of relations between proteins over biomedical scientific literature using a deep-linguistic approach","volume":"39","author":"Rinaldi","year":"2007","journal-title":"Artif Intell Med"},{"key":"2023041820544022800_btad174-B23","doi-asserted-by":"crossref","first-page":"104137","DOI":"10.1016\/j.jbi.2022.104137","article-title":"NILINKER: attention-based approach to NIL entity linking","volume":"132","author":"Ruas","year":"2022","journal-title":"J Biomed Inform"},{"key":"2023041820544022800_btad174-B24","doi-asserted-by":"crossref","first-page":"D1255","DOI":"10.1093\/nar\/gkab1063","article-title":"The human disease ontology 2022 update","volume":"50","author":"Schriml","year":"2022","journal-title":"Nucleic Acids Res"},{"key":"2023041820544022800_btad174-B25","doi-asserted-by":"crossref","first-page":"152","DOI":"10.1016\/j.jbi.2014.05.007","article-title":"Lessons learnt from the ddiextraction-2013 shared task","volume":"51","author":"Segura-Bedmar","year":"2014","journal-title":"J Biomed Inform"},{"key":"2023041820544022800_btad174-B26","first-page":"208","author":"Song","year":"2019"},{"key":"2023041820544022800_btad174-B27","first-page":"367","author":"Sousa","year":"2020"},{"key":"2023041820544022800_btad174-B28","doi-asserted-by":"crossref","first-page":"4207","DOI":"10.1109\/JBHI.2022.3173558","article-title":"Biomedical relation extraction with knowledge graph-based recommendations","volume":"26","author":"Sousa","year":"2022","journal-title":"IEEE J Biomed Health Inform"},{"key":"2023041820544022800_btad174-B29","first-page":"1487","author":"Sousa","year":"2019"},{"key":"2023041820544022800_btad174-B30","doi-asserted-by":"crossref","DOI":"10.1093\/database\/baaa104","article-title":"A hybrid approach toward biomedical relation extraction training corpora: combining distant supervision with crowdsourcing","volume":"2020","author":"Sousa","year":"2020","journal-title":"Database"},{"key":"2023041820544022800_btad174-B31","first-page":"241,","author":"Zhao","year":"2019"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btad174\/49769337\/btad174.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/39\/4\/btad174\/50015871\/btad174.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/39\/4\/btad174\/50015871\/btad174.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,4,18]],"date-time":"2023-04-18T20:55:14Z","timestamp":1681851314000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/doi\/10.1093\/bioinformatics\/btad174\/7108769"}},"subtitle":[],"editor":[{"given":"Jonathan","family":"Wren","sequence":"additional","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2023,4,1]]},"references-count":31,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,4,3]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btad174","relation":{},"ISSN":["1367-4811"],"issn-type":[{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,4,1]]},"published":{"date-parts":[[2023,4,1]]},"article-number":"btad174"}}