{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,23]],"date-time":"2026-01-23T05:42:25Z","timestamp":1769146945224,"version":"3.49.0"},"reference-count":31,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2016,10,12]],"date-time":"2016-10-12T00:00:00Z","timestamp":1476230400000},"content-version":"vor","delay-in-days":405,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2016,1,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Motivation: A complete repository of gene\u2013gene interactions is key for understanding cellular processes, human disease and drug response. These gene\u2013gene interactions include both protein\u2013protein interactions and transcription factor interactions. The majority of known interactions are found in the biomedical literature. Interaction databases, such as BioGRID and ChEA, annotate these gene\u2013gene interactions; however, curation becomes difficult as the literature grows exponentially. DeepDive is a trained system for extracting information from a variety of sources, including text. In this work, we used DeepDive to extract both protein\u2013protein and transcription factor interactions from over 100\u2009000 full-text PLOS articles.<\/jats:p>\n               <jats:p>Methods: We built an extractor for gene\u2013gene interactions that identified candidate gene\u2013gene relations within an input sentence. For each candidate relation, DeepDive computed a probability that the relation was a correct interaction. We evaluated this system against the Database of Interacting Proteins and against randomly curated extractions.<\/jats:p>\n               <jats:p>Results: Our system achieved 76% precision and 49% recall in extracting direct and indirect interactions involving gene symbols co-occurring in a sentence. For randomly curated extractions, the system achieved between 62% and 83% precision based on direct or indirect interactions, as well as sentence-level and document-level precision. Overall, our system extracted 3356 unique gene pairs using 724 features from over 100\u2009000 full-text articles.<\/jats:p>\n               <jats:p>Availability and implementation: Application source code is publicly available at https:\/\/github.com\/edoughty\/deepdive_genegene_app<\/jats:p>\n               <jats:p>Contact: \u00a0russ.altman@stanford.edu<\/jats:p>\n               <jats:p>Supplementary information: \u00a0Supplementary data are available at Bioinformatics online.<\/jats:p>","DOI":"10.1093\/bioinformatics\/btv476","type":"journal-article","created":{"date-parts":[[2015,9,4]],"date-time":"2015-09-04T00:33:36Z","timestamp":1441326816000},"page":"106-113","source":"Crossref","is-referenced-by-count":49,"title":["Large-scale extraction of gene interactions from full-text literature using DeepDive"],"prefix":"10.1093","volume":"32","author":[{"given":"Emily K.","family":"Mallory","sequence":"first","affiliation":[{"name":"1 Biomedical Informatics Training Program, Stanford University, Stanford, CA 94305, USA,"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ce","family":"Zhang","sequence":"additional","affiliation":[{"name":"2 Department of Computer Sciences, University of Wisconsin-Madison, Madison, WI 53706, USA,"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Christopher","family":"R\u00e9","sequence":"additional","affiliation":[{"name":"3 Department of Computer Science,"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Russ B.","family":"Altman","sequence":"additional","affiliation":[{"name":"4 Department of Bioengineering,"},{"name":"5 Department of Genetics and"},{"name":"6 Department of Medicine, Stanford University, Stanford, CA 94305, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2015,9,3]]},"reference":[{"key":"2023020109532864400_btv476-B1","doi-asserted-by":"crossref","first-page":"D396","DOI":"10.1093\/nar\/gkt1079","article-title":"Negatome 2.0: a database of non-interacting proteins derived by literature mining, manual annotation and protein structure analysis","volume":"42","author":"Blohm","year":"2014","journal-title":"Nucleic Acids Res."},{"key":"2023020109532864400_btv476-B2","doi-asserted-by":"crossref","first-page":"D470","DOI":"10.1093\/nar\/gku1204","article-title":"The BioGRID interaction database: 2015 update","volume":"43","author":"Chatr-Aryamontri","year":"2014","journal-title":"Nucleic Acids Res."},{"key":"2023020109532864400_btv476-B3","first-page":"1323","article-title":"An ensemble self-training protein interaction article classifier","volume":"24","author":"Chen","year":"2014","journal-title":"Biomed. Mater. Eng."},{"key":"2023020109532864400_btv476-B4","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1186\/1471-2105-13-172","article-title":"A text-mining system for extracting metabolic reactions from full-text articles","volume":"13","author":"Czarnecki","year":"2012","journal-title":"BMC Bioinformatics"},{"key":"2023020109532864400_btv476-B5","doi-asserted-by":"crossref","first-page":"D808","DOI":"10.1093\/nar\/gks1094","article-title":"STRING v9.1: protein-protein interaction networks, with increased coverage and integration","volume":"41","author":"Franceschini","year":"2013","journal-title":"Nucleic Acids Res."},{"key":"2023020109532864400_btv476-B6","doi-asserted-by":"crossref","first-page":"e4554","DOI":"10.1371\/journal.pone.0004554","article-title":"PPI finder: a mining tool for human protein-protein interactions","volume":"4","author":"He","year":"2009","journal-title":"PLoS One"},{"key":"2023020109532864400_btv476-B7","doi-asserted-by":"crossref","first-page":"664","DOI":"10.1038\/ng0704-664","article-title":"A gene network for navigating the literature","volume":"36","author":"Hoffmann","year":"2004","journal-title":"Nat. Genet."},{"key":"2023020109532864400_btv476-B8","doi-asserted-by":"crossref","first-page":"D767","DOI":"10.1093\/nar\/gkn892","article-title":"Human Protein Reference Database\u20142009 update","volume":"37","author":"Keshava Prasad","year":"2009","journal-title":"Nucleic Acids Res."},{"key":"2023020109532864400_btv476-B9","doi-asserted-by":"crossref","first-page":"597","DOI":"10.1093\/bioinformatics\/btr702","article-title":"PIE the search: searching PubMed literature for protein interaction information","volume":"28","author":"Kim","year":"2012","journal-title":"Bioinformatics"},{"key":"2023020109532864400_btv476-B10","doi-asserted-by":"crossref","first-page":"2438","DOI":"10.1093\/bioinformatics\/btq466","article-title":"ChEA: transcription factor regulation inferred from integrating genome-wide ChIP-X experiments","volume":"26","author":"Lachmann","year":"2010","journal-title":"Bioinformatics"},{"key":"2023020109532864400_btv476-B11","doi-asserted-by":"crossref","first-page":"575","DOI":"10.1007\/s11192-010-0202-z","article-title":"The rate of growth in scientific publication and the decline in coverage provided by Science Citation Index","volume":"84","author":"Larsen","year":"2010","journal-title":"Scientometrics"},{"key":"2023020109532864400_btv476-B12","first-page":"652","article-title":"BANNER: an executable survey of advances in biomedical named entity recognition","author":"Leaman","year":"2008","journal-title":"Pac. Symp. Biocomput."},{"key":"2023020109532864400_btv476-B13","doi-asserted-by":"crossref","first-page":"1237","DOI":"10.1016\/j.cell.2013.02.014","article-title":"Transcriptional regulation and its misregulation in disease","volume":"152","author":"Lee","year":"2013","journal-title":"Cell"},{"key":"2023020109532864400_btv476-B14","first-page":"55","article-title":"The Stanford CoreNLP Natural Language Processing Toolkit","author":"Manning","year":"2014"},{"key":"2023020109532864400_btv476-B15","first-page":"1003","article-title":"Distant supervision for relation extraction using unlabeled data","author":"Mintz","year":"2009"},{"key":"2023020109532864400_btv476-B16","doi-asserted-by":"crossref","first-page":"32","DOI":"10.4018\/jswis.2012070103","article-title":"Elementary: large-scale knowledge-base construction via machine learning and statistical inference","volume":"8","author":"Niu","year":"2012","journal-title":"Int. J. Semantic Web Inf. Syst."},{"key":"2023020109532864400_btv476-B17","first-page":"25","article-title":"DeepDive: web-scale knowledge-base construction using statistical learning and inference","volume":"884","author":"Niu","year":"2012","journal-title":"VLDS"},{"key":"2023020109532864400_btv476-B18","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1016\/j.ymeth.2014.10.026","article-title":"Protein-protein interaction predictions using text mining methods","volume":"74","author":"Papanikolaou","year":"2015","journal-title":"Methods"},{"key":"2023020109532864400_btv476-B19","doi-asserted-by":"crossref","first-page":"e113523","DOI":"10.1371\/journal.pone.0113523","article-title":"A machine reading system for assembling synthetic paleontological databases","volume":"9","author":"Peters","year":"2014","journal-title":"PLoS One"},{"key":"2023020109532864400_btv476-B20","first-page":"120","article-title":"Distant supervision for cancer pathway extraction from text","author":"Poon","year":"2015","journal-title":"Pac. Symp. Biocomput."},{"key":"2023020109532864400_btv476-B21","doi-asserted-by":"crossref","first-page":"e102039","DOI":"10.1371\/journal.pone.0102039","article-title":"An unsupervised text mining method for relation extraction from biomedical literature","volume":"9","author":"Quan","year":"2014","journal-title":"PLoS One"},{"key":"2023020109532864400_btv476-B22","doi-asserted-by":"crossref","first-page":"bas052","DOI":"10.1093\/database\/bas052","article-title":"PPInterFinder\u2014a mining tool for extracting causal relations on human proteins from literature","volume":"2013","author":"Raja","year":"2013","journal-title":"Database"},{"key":"2023020109532864400_btv476-B23","doi-asserted-by":"crossref","first-page":"D449","DOI":"10.1093\/nar\/gkh086","article-title":"The database of interacting proteins: 2004 update","volume":"32","author":"Salwinski","year":"2004","journal-title":"Nucleic Acids Res."},{"key":"2023020109532864400_btv476-B24","first-page":"629","article-title":"An Overview of the Tesseract OCR Engine","author":"Smith","year":"2007"},{"key":"2023020109532864400_btv476-B25","first-page":"318","article-title":"Refining literature curated protein interactions using expert opinions","author":"Tastan","year":"2015","journal-title":"Pac. Symp. Biocomput."},{"key":"2023020109532864400_btv476-B26","doi-asserted-by":"crossref","first-page":"e1000837","DOI":"10.1371\/journal.pcbi.1000837","article-title":"A comprehensive benchmark of kernel methods to extract protein-protein interactions from literature","volume":"6","author":"Tikk","year":"2010","journal-title":"PLoS Comput. Biol."},{"key":"2023020109532864400_btv476-B27","doi-asserted-by":"crossref","first-page":"382","DOI":"10.1007\/11573036_36","article-title":"Developing a robust part-of-speech tagger for biomedical text","volume":"3746","author":"Tsuruoka","year":"2005","journal-title":"Lect. Notes Comput. Sci"},{"key":"2023020109532864400_btv476-B28","doi-asserted-by":"crossref","first-page":"S5","DOI":"10.1186\/1471-2105-12-S8-S5","article-title":"Cross-species gene normalization by species inference","volume":"12","author":"Wei","year":"2011","journal-title":"BMC Bioinformatics"},{"key":"2023020109532864400_btv476-B29","author":"Zhang","year":"2015"},{"key":"2023020109532864400_btv476-B30","first-page":"993","article-title":"GeoDeepDive: statistical inference using familiar data-processing languages","author":"Zhang","year":"2013"},{"key":"2023020109532864400_btv476-B31","doi-asserted-by":"crossref","first-page":"326","DOI":"10.1186\/1471-2105-11-326","article-title":"PPLook: an automated data mining tool for protein-protein interaction","volume":"11","author":"Zhang","year":"2010","journal-title":"BMC Bioinformatics"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/32\/1\/106\/49016510\/bioinformatics_32_1_106.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/32\/1\/106\/49016510\/bioinformatics_32_1_106.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,2,1]],"date-time":"2023-02-01T21:20:07Z","timestamp":1675286407000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/32\/1\/106\/1742428"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,9,3]]},"references-count":31,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2016,1,1]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btv476","relation":{},"ISSN":["1367-4811","1367-4803"],"issn-type":[{"value":"1367-4811","type":"electronic"},{"value":"1367-4803","type":"print"}],"subject":[],"published-other":{"date-parts":[[2016,1,1]]},"published":{"date-parts":[[2015,9,3]]}}}