{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T03:18:15Z","timestamp":1775013495582,"version":"3.50.1"},"reference-count":27,"publisher":"Springer Science and Business Media LLC","issue":"S16","license":[{"start":{"date-parts":[[2020,12,1]],"date-time":"2020-12-01T00:00:00Z","timestamp":1606780800000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2020,12,16]],"date-time":"2020-12-16T00:00:00Z","timestamp":1608076800000},"content-version":"vor","delay-in-days":15,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61690202"],"award-info":[{"award-number":["61690202"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100011347","name":"State Key Laboratory of Software Development Environment","doi-asserted-by":"crossref","award":["SKLSDE-2017ZX-17"],"award-info":[{"award-number":["SKLSDE-2017ZX-17"]}],"id":[{"id":"10.13039\/501100011347","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Bioinformatics"],"published-print":{"date-parts":[[2020,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Background<\/jats:title><jats:p>Although biomedical publications and literature are growing rapidly, there still lacks structured knowledge that can be easily processed by computer programs. In order to extract such knowledge from plain text and transform them into structural form, the relation extraction problem becomes an important issue. Datasets play a critical role in the development of relation extraction methods. However, existing relation extraction datasets in biomedical domain are mainly human-annotated, whose scales are usually limited due to their labor-intensive and time-consuming nature.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>We construct BioRel, a large-scale dataset for biomedical relation extraction problem, by using Unified Medical Language System as knowledge base and Medline as corpus. We first identify mentions of entities in sentences of Medline and link them to Unified Medical Language System with Metamap. Then, we assign each sentence a relation label by using distant supervision. Finally, we adapt the state-of-the-art deep learning and statistical machine learning methods as baseline models and conduct comprehensive experiments on the BioRel dataset.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>Based on the extensive experimental results, we have shown that BioRel is a suitable large-scale datasets for biomedical relation extraction, which provides both reasonable baseline performance and many remaining challenges for both deep learning and statistical methods.<\/jats:p><\/jats:sec>","DOI":"10.1186\/s12859-020-03889-5","type":"journal-article","created":{"date-parts":[[2020,12,16]],"date-time":"2020-12-16T02:02:40Z","timestamp":1608084160000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":28,"title":["BioRel: towards large-scale biomedical relation extraction"],"prefix":"10.1186","volume":"21","author":[{"given":"Rui","family":"Xing","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4157-9931","authenticated-orcid":false,"given":"Jie","family":"Luo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tengwei","family":"Song","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2020,12,16]]},"reference":[{"key":"3889_CR1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.0040020","author":"KB Cohen","year":"2008","unstructured":"Cohen KB, Hunter L. Getting started in text mining. PLoS Comput Biol. 2008;. https:\/\/doi.org\/10.1371\/journal.pcbi.0040020.","journal-title":"PLoS Comput Biol"},{"key":"3889_CR2","doi-asserted-by":"crossref","unstructured":"Mintz M, Bills S, Snow R, Jurafsky D. Distant supervision for relation extraction without labeled data. In: ACL \u201909. Stroudsburg, PA, USA: Association for Computational Linguistics; 2009. p. 1003\u201311.","DOI":"10.3115\/1690219.1690287"},{"key":"3889_CR3","doi-asserted-by":"crossref","unstructured":"Riedel S, Yao L, McCallum A. Modeling relations and their mentions without labeled text. In: ECML PKDD\u201910. Berlin: Springer; 2010. p. 148\u201363.","DOI":"10.1007\/978-3-642-15939-8_10"},{"key":"3889_CR4","first-page":"455","volume-title":"Multi-instance multi-label learning for relation extraction","author":"M Surdeanu","year":"2012","unstructured":"Surdeanu M, Tibshirani J, Nallapati R, Manning CD. Multi-instance multi-label learning for relation extraction. Jeju Island: Association for Computational Linguistics; 2012. p. 455\u201365."},{"issue":"1","key":"3889_CR5","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1561\/2200000006","volume":"2","author":"Y Bengio","year":"2009","unstructured":"Bengio Y. Learning deep architectures for AI. Found Trends Mach Learn. 2009;2(1):1\u2013127. https:\/\/doi.org\/10.1561\/2200000006.","journal-title":"Found Trends Mach Learn"},{"issue":"7553","key":"3889_CR6","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nature14539","volume":"521","author":"Y LeCun","year":"2015","unstructured":"LeCun Y, Bengio Y, Hinton GE. Deep learning. Nature. 2015;521(7553):436\u201344. https:\/\/doi.org\/10.1038\/nature14539.","journal-title":"Nature"},{"key":"3889_CR7","first-page":"1753","volume-title":"Distant supervision for relation extraction via piecewise convolutional neural networks","author":"D Zeng","year":"2015","unstructured":"Zeng D, Liu K, Chen Y, Zhao J. Distant supervision for relation extraction via piecewise convolutional neural networks. Lisbon: Association for Computational Linguistics; 2015. p. 1753\u201362."},{"key":"3889_CR8","first-page":"2124","volume-title":"Neural relation extraction with selective attention over instances","author":"Y Lin","year":"2016","unstructured":"Lin Y, Shen S, Liu Z, Luan H, Sun M. Neural relation extraction with selective attention over instances. Berlin: Association for Computational Linguistics; 2016. p. 2124\u201333."},{"key":"3889_CR9","doi-asserted-by":"crossref","unstructured":"Ji G, Liu K, He S, Zhao J. Distant supervision for relation extraction with sentence-level attention and entity descriptions; 2017. p. 3060\u20136.","DOI":"10.1609\/aaai.v31i1.10953"},{"key":"3889_CR10","first-page":"1790","volume-title":"A soft-label method for noise-tolerant distantly supervised relation extraction","author":"T Liu","year":"2017","unstructured":"Liu T, Wang K, Chang B, Sui Z. A soft-label method for noise-tolerant distantly supervised relation extraction. Copenhagen: Association for Computational Linguistics; 2017. p. 1790\u20135."},{"key":"3889_CR11","unstructured":"Jat S, Khandelwal S, Talukdar P. Improving distantly supervised relation extraction using word and entity based attention. arXiv e-prints, 1804-06987; 2018. arXiv:1804.06987."},{"key":"3889_CR12","first-page":"2216","volume-title":"Multi-level structured self-attentions for distantly supervised relation extraction","author":"J Du","year":"2018","unstructured":"Du J, Han J, Way A, Wan D. Multi-level structured self-attentions for distantly supervised relation extraction. Brussels: Association for Computational Linguistics; 2018. p. 2216\u201325."},{"key":"3889_CR13","unstructured":"Doddington G, Mitchell A, Przybocki M, Ramshaw L, Strassel S, Weischedel R. The automatic content extraction (ACE) program\u2014tasks, data, and evaluation. Lisbon, Portugal: European Language Resources Association (ELRA); 2004."},{"key":"3889_CR14","unstructured":"Walker C, Strassel S, Medero J, Maeda K. ACE 2005 multilingual training corpus; 2005."},{"key":"3889_CR15","first-page":"33","volume-title":"SemEval-2010 Task 8: multi-way classification of semantic relations between pairs of nominals","author":"I Hendrickx","year":"2010","unstructured":"Hendrickx I, Kim SN, Kozareva Z, Nakov P, \u00d3 S\u00e9aghdha D, Pad\u00f3 S, Pennacchiotti M, Romano L, Szpakowicz S. SemEval-2010 Task 8: multi-way classification of semantic relations between pairs of nominals. Uppsala: Association for Computational Linguistics; 2010. p. 33\u20138."},{"key":"3889_CR16","doi-asserted-by":"publisher","unstructured":"Xing R, Luo J, Song T. Biorel: a large-scale dataset for biomedical relation extraction. In: 2019 IEEE international conference on bioinformatics and biomedicine (BIBM); 2019. p. 1801\u20138. https:\/\/doi.org\/10.1109\/BIBM47256.2019.8983057.","DOI":"10.1109\/BIBM47256.2019.8983057"},{"key":"3889_CR17","doi-asserted-by":"publisher","first-page":"267","DOI":"10.1093\/nar\/gkh061","volume":"32","author":"O Bodenreider","year":"2004","unstructured":"Bodenreider O. The unified medical language system (UMLS): integrating biomedical terminology. Nucleic Acids Res. 2004;32:267\u201370.","journal-title":"Nucleic Acids Res"},{"key":"3889_CR18","doi-asserted-by":"publisher","DOI":"10.1038\/s41597-019-0055-0","author":"Z Yijia","year":"2019","unstructured":"Yijia Z, Chen Q, Yang Z, Lin H, Lu Z. Biowordvec, improving biomedical word embeddings with subword information and mesh. Sci Data. 2019;. https:\/\/doi.org\/10.1038\/s41597-019-0055-0.","journal-title":"Sci Data"},{"key":"3889_CR19","first-page":"541","volume-title":"Knowledge-based weak supervision for information extraction of overlapping relations","author":"R Hoffmann","year":"2011","unstructured":"Hoffmann R, Zhang C, Ling X, Zettlemoyer L, Weld DS. Knowledge-based weak supervision for information extraction of overlapping relations. Portland: Association for Computational Linguistics; 2011. p. 541\u201350."},{"key":"3889_CR20","first-page":"2335","volume-title":"Relation classification via convolutional deep neural network","author":"D Zeng","year":"2014","unstructured":"Zeng D, Liu K, Lai S, Zhou G, Zhao J. Relation classification via convolutional deep neural network. Dublin: Dublin City University and Association for Computational Linguistics; 2014. p. 2335\u201344."},{"key":"3889_CR21","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","volume":"9","author":"S Hochreiter","year":"1997","unstructured":"Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput. 1997;9:1735\u201380. https:\/\/doi.org\/10.1162\/neco.1997.9.8.1735.","journal-title":"Neural Comput"},{"key":"3889_CR22","doi-asserted-by":"publisher","first-page":"1724","DOI":"10.3115\/v1\/D14-1179","volume-title":"Learning phrase representations using RNN encoder\u2013decoder for statistical machine translation","author":"K Cho","year":"2014","unstructured":"Cho K, van Merri\u00ebnboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y. Learning phrase representations using RNN encoder\u2013decoder for statistical machine translation. Doha: Association for Computational Linguistics; 2014. p. 1724\u201334. https:\/\/doi.org\/10.3115\/v1\/D14-1179."},{"key":"3889_CR23","unstructured":"Zhang D, Wang D. Relation classification via recurrent neural network. CoRR; 2015. arXiv:1508.01006."},{"key":"3889_CR24","doi-asserted-by":"publisher","first-page":"207","DOI":"10.18653\/v1\/P16-2034","volume-title":"Attention-based bidirectional long short-term memory networks for relation classification","author":"P Zhou","year":"2016","unstructured":"Zhou P, Shi W, Tian J, Qi Z, Li B, Hao H, Xu B. Attention-based bidirectional long short-term memory networks for relation classification. Berlin: Association for Computational Linguistics; 2016. p. 207\u201312. https:\/\/doi.org\/10.18653\/v1\/P16-2034."},{"key":"3889_CR25","first-page":"1257","volume-title":"RESIDE: improving distantly-supervised neural relation extraction using side information","author":"S Vashishth","year":"2018","unstructured":"Vashishth S, Joshi R, Prayaga SS, Bhattacharyya C, Talukdar P. RESIDE: improving distantly-supervised neural relation extraction using side information. Brussels: Association for Computational Linguistics; 2018. p. 1257\u201366."},{"key":"3889_CR26","first-page":"160","volume-title":"Universal dependency parsing from scratch","author":"P Qi","year":"2018","unstructured":"Qi P, Dozat T, Zhang Y, Manning CD. Universal dependency parsing from scratch. Brussels: Association for Computational Linguistics; 2018. p. 160\u201370."},{"key":"3889_CR27","first-page":"35","volume-title":"Improving distantly supervised extraction of drug\u2013drug and protein\u2013protein interactions","author":"T Bobi\u0107","year":"2012","unstructured":"Bobi\u0107 T, Klinger R, Thomas P, Hofmann-Apitius M. Improving distantly supervised extraction of drug\u2013drug and protein\u2013protein interactions. Avignon: Association for Computational Linguistics; 2012. p. 35\u201343."}],"container-title":["BMC Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-020-03889-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1186\/s12859-020-03889-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-020-03889-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,5]],"date-time":"2022-12-05T05:28:11Z","timestamp":1670218091000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/s12859-020-03889-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12]]},"references-count":27,"journal-issue":{"issue":"S16","published-print":{"date-parts":[[2020,12]]}},"alternative-id":["3889"],"URL":"https:\/\/doi.org\/10.1186\/s12859-020-03889-5","relation":{},"ISSN":["1471-2105"],"issn-type":[{"value":"1471-2105","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,12]]},"assertion":[{"value":"16 November 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 November 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 December 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Not applicable.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"543"}}