{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,22]],"date-time":"2025-02-22T00:45:34Z","timestamp":1740185134874,"version":"3.37.3"},"reference-count":57,"publisher":"Oxford University Press (OUP)","issue":"8","license":[{"start":{"date-parts":[[2017,12,8]],"date-time":"2017-12-08T00:00:00Z","timestamp":1512691200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/about_us\/legal\/notices"}],"funder":[{"DOI":"10.13039\/100000185","name":"Defense Advanced Research Projects Agency","doi-asserted-by":"publisher","award":["DARPA-BAA-14-14"],"award-info":[{"award-number":["DARPA-BAA-14-14"]}],"id":[{"id":"10.13039\/100000185","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["EP\/1038099\/1"],"award-info":[{"award-number":["EP\/1038099\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2018,4,15]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Motivation<\/jats:title><jats:p>Pathway models are valuable resources that help us understand the various mechanisms underpinning complex biological processes. Their curation is typically carried out through manual inspection of published scientific literature to find information relevant to a model, which is a laborious and knowledge-intensive task. Furthermore, models curated manually cannot be easily updated and maintained with new evidence extracted from the literature without automated support.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>We have developed LitPathExplorer, a visual text analytics tool that integrates advanced text mining, semi-supervised learning and interactive visualization, to facilitate the exploration and analysis of pathway models using statements (i.e. events) extracted automatically from the literature and organized according to levels of confidence. LitPathExplorer supports pathway modellers and curators alike by: (i) extracting events from the literature that corroborate existing models with evidence; (ii) discovering new events which can update models; and (iii) providing a confidence value for each event that is automatically computed based on linguistic features and article metadata. Our evaluation of event extraction showed a precision of 89% and a recall of 71%. Evaluation of our confidence measure, when used for ranking sampled events, showed an average precision ranging between 61 and 73%, which can be improved to 95% when the user is involved in the semi-supervised learning process. Qualitative evaluation using pair analytics based on the feedback of three domain experts confirmed the utility of our tool within the context of pathway model exploration.<\/jats:p><\/jats:sec><jats:sec><jats:title>Availability and implementation<\/jats:title><jats:p>LitPathExplorer is available at http:\/\/nactem.ac.uk\/LitPathExplorer_BI\/.<\/jats:p><\/jats:sec><jats:sec><jats:title>Supplementary information<\/jats:title><jats:p>Supplementary data are available at Bioinformatics online.<\/jats:p><\/jats:sec>","DOI":"10.1093\/bioinformatics\/btx774","type":"journal-article","created":{"date-parts":[[2017,12,7]],"date-time":"2017-12-07T12:10:59Z","timestamp":1512648659000},"page":"1389-1397","source":"Crossref","is-referenced-by-count":4,"title":["LitPathExplorer: a confidence-based visual text analytics tool for exploring literature-enriched pathway models"],"prefix":"10.1093","volume":"34","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9021-7566","authenticated-orcid":false,"given":"Axel J","family":"Soto","sequence":"first","affiliation":[{"name":"National Centre for Text Mining, School of Computer Science, University of Manchester, Manchester, UK"}]},{"given":"Chrysoula","family":"Zerva","sequence":"additional","affiliation":[{"name":"National Centre for Text Mining, School of Computer Science, University of Manchester, Manchester, UK"}]},{"given":"Riza","family":"Batista-Navarro","sequence":"additional","affiliation":[{"name":"National Centre for Text Mining, School of Computer Science, University of Manchester, Manchester, UK"}]},{"given":"Sophia","family":"Ananiadou","sequence":"additional","affiliation":[{"name":"National Centre for Text Mining, School of Computer Science, University of Manchester, Manchester, UK"}]}],"member":"286","published-online":{"date-parts":[[2017,12,8]]},"reference":[{"key":"2023012713011663400_btx774-B1","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1016\/j.tibtech.2010.04.005","article-title":"Event extraction for systems biology by text mining the literature","volume":"28","author":"Ananiadou","year":"2010","journal-title":"Trends Biotechnol"},{"first-page":"1","year":"2011","author":"Arias-Hernandez","key":"2023012713011663400_btx774-B2"},{"year":"2012","author":"Bakalov","key":"2023012713011663400_btx774-B3"},{"key":"2023012713011663400_btx774-B4","doi-asserted-by":"crossref","first-page":"435","DOI":"10.1186\/1471-2105-12-435","article-title":"PESCADOR, a web-based tool to assist text-mining of biointeractions extracted from PubMed queries","volume":"12","author":"Barbosa-Silva","year":"2011","journal-title":"BMC Bioinformatics"},{"key":"2023012713011663400_btx774-B5","doi-asserted-by":"crossref","first-page":"11.","DOI":"10.1093\/database\/bav043","article-title":"The Confidence Information Ontology: a step towards a standard for asserting confidence in annotations","volume":"2015","author":"Bastian","year":"2015","journal-title":"Database"},{"key":"2023012713011663400_btx774-B6","doi-asserted-by":"crossref","first-page":"R22.","DOI":"10.1186\/gb-2003-4-3-r22","article-title":"Osprey: a network visualization system","volume":"4","author":"Breitkreutz","year":"2003","journal-title":"Genome Biol"},{"key":"2023012713011663400_btx774-B7","doi-asserted-by":"crossref","first-page":"685","DOI":"10.1093\/nar\/gkq1039","article-title":"Pathway Commons, a web resource for biological pathway data","volume":"39","author":"Cerami","year":"2011","journal-title":"Nucleic Acids Res"},{"key":"2023012713011663400_btx774-B8","doi-asserted-by":"crossref","first-page":"1), 147","DOI":"10.1186\/1471-2105-5-147","article-title":"Content-rich biological network constructed by mining PubMed abstracts","volume":"5","author":"Chen","year":"2004","journal-title":"BMC Bioinformatics"},{"key":"2023012713011663400_btx774-B9","doi-asserted-by":"crossref","first-page":"045008.","DOI":"10.1088\/1478-3975\/12\/4\/045008","article-title":"DARPA\u2019s Big Mechanism program","volume":"12","author":"Cohen","year":"2015","journal-title":"Phys. Biol"},{"key":"2023012713011663400_btx774-B10","doi-asserted-by":"crossref","first-page":"2003","DOI":"10.1002\/asi.23309","article-title":"Do \u201caltmetrics\u201d correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective","volume":"66","author":"Costas","year":"2015","journal-title":"J. Assoc. Inf. Sci. Technol"},{"key":"2023012713011663400_btx774-B11","doi-asserted-by":"crossref","first-page":"172.","DOI":"10.1186\/1471-2105-13-172","article-title":"A text-mining system for extracting metabolic reactions from full-text articles","volume":"13","author":"Czarnecki","year":"2012","journal-title":"BMC Bioinformatics"},{"key":"2023012713011663400_btx774-B12","doi-asserted-by":"crossref","first-page":"baw090.","DOI":"10.1093\/database\/baw090","article-title":"Met network in pubmed: a text-mined network visualization and curation system","volume":"2016","author":"Dai","year":"2016","journal-title":"Database"},{"key":"2023012713011663400_btx774-B13","doi-asserted-by":"crossref","first-page":"996","DOI":"10.1093\/bioinformatics\/18.7.996","article-title":"PATIKA: an integrated visual environment for collaborative construction and analysis of cellular pathways","volume":"18","author":"Demir","year":"2002","journal-title":"Bioinformatics"},{"key":"2023012713011663400_btx774-B14","doi-asserted-by":"crossref","first-page":"450","DOI":"10.1093\/nar\/gkr310","article-title":"CoPub update: CoPub 5.0 a text mining system to answer biological questions","volume":"39","author":"Fleuren","year":"2011","journal-title":"Nucleic Acids Res"},{"key":"2023012713011663400_btx774-B15","doi-asserted-by":"crossref","first-page":"1129","DOI":"10.1002\/spe.4380211102","article-title":"Graph drawing by force-directed placement","volume":"21","author":"Fruchterman","year":"1991","journal-title":"Softw. Pract. Exp"},{"key":"2023012713011663400_btx774-B16","doi-asserted-by":"crossref","first-page":"16020","DOI":"10.1038\/npjsba.2016.20","article-title":"Minerva\u2014a platform for visualization and curation of molecular interaction networks","volume":"2","author":"Gawron","year":"2016","journal-title":"Npj Syst. Biol. Appl"},{"key":"2023012713011663400_btx774-B17","first-page":"420","article-title":"Visualization and language processing for supporting analysis across the biomedical literature","author":"G\u00f6rg","year":"2010","journal-title":"Knowledge-Based and Intelligent Information and Engineering Systems. Vol. 6279"},{"key":"2023012713011663400_btx774-B18","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1093\/nar\/gkh462","article-title":"WebInterViewer: visualizing and analyzing molecular interaction networks","volume":"32","author":"Han","year":"2004","journal-title":"Nucleic Acids Res"},{"key":"2023012713011663400_btx774-B19","doi-asserted-by":"crossref","first-page":"S1.","DOI":"10.1186\/1471-2105-6-S1-S1","article-title":"Overview of BioCreAtIvE: critical assessment of information extraction for biology","volume":"6","author":"Hirschman","year":"2005","journal-title":"BMC Bioinformatics"},{"volume-title":"Mastering the Information Age-Solving Problems with Visual Analytics","year":"2010","author":"Keim","key":"2023012713011663400_btx774-B20"},{"key":"2023012713011663400_btx774-B21","doi-asserted-by":"crossref","first-page":"D1301","DOI":"10.1093\/nar\/gkr1074","article-title":"WikiPathways: building research communities on biological pathways","volume":"40","author":"Kelder","year":"2012","journal-title":"Nucleic Acids Res"},{"key":"2023012713011663400_btx774-B22","doi-asserted-by":"crossref","first-page":"374","DOI":"10.1093\/bioinformatics\/btq221","article-title":"PathText: a text mining integrator for biological pathway visualizations","volume":"26","author":"Kemper","year":"2010","journal-title":"Bioinformatics"},{"key":"2023012713011663400_btx774-B23","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1007\/978-3-642-41281-3_7","volume-title":"Approaches in Integrative Bioinformatics","author":"Kerren","year":"2014"},{"key":"2023012713011663400_btx774-B24","doi-asserted-by":"crossref","DOI":"10.3115\/v1\/W15-1303","article-title":"A compositional interpretation of biomedical event factuality","author":"Kilicoglu","year":"2015","journal-title":"Proceedings of the Second Workshop on Extra-Propositional Aspects of Meaning in Computational Semantics (ExProM 2015)"},{"key":"2023012713011663400_btx774-B25","doi-asserted-by":"crossref","first-page":"e1004085","DOI":"10.1371\/journal.pcbi.1004085","article-title":"PathVisio 3: an extendable pathway analysis toolbox","volume":"11","author":"Kutmon","year":"2015","journal-title":"PLoS Comput. Biol"},{"key":"2023012713011663400_btx774-B26","doi-asserted-by":"crossref","first-page":"D488","DOI":"10.1093\/nar\/gkv1024","article-title":"WikiPathways: capturing the full diversity of pathway knowledge","volume":"44","author":"Kutmon","year":"2015","journal-title":"Nucleic Acids Res"},{"key":"2023012713011663400_btx774-B27","doi-asserted-by":"crossref","first-page":"735","DOI":"10.1038\/nbt.1558","article-title":"The systems biology graphical notation","volume":"27","author":"Le Novere","year":"2009","journal-title":"Nat. Biotechnol"},{"key":"2023012713011663400_btx774-B28","doi-asserted-by":"crossref","first-page":"100\u2013117.","DOI":"10.1371\/journal.pcbi.1003117","article-title":"HypothesisFinder: a strategy for the detection of speculative statements in scientific text","volume":"9","author":"Malhotra","year":"2013","journal-title":"PLoS Comput. Biol"},{"key":"2023012713011663400_btx774-B29","doi-asserted-by":"crossref","first-page":"10021.","DOI":"10.1038\/srep10021","article-title":"PALM-IST: pathway assembly from literature mining \u2013 an information search tool","volume":"5","author":"Mandloi","year":"2015","journal-title":"Sci. Rep"},{"key":"2023012713011663400_btx774-B30","doi-asserted-by":"crossref","first-page":"1381.","DOI":"10.1093\/bioinformatics\/btq143","article-title":"Payao: a community platform for SBML pathway model curation","volume":"26","author":"Matsuoka","year":"2010","journal-title":"Bioinformatics"},{"key":"2023012713011663400_btx774-B31","doi-asserted-by":"crossref","first-page":"1759","DOI":"10.1093\/bioinformatics\/bts237","article-title":"Boosting automatic event extraction from the literature using domain adaptation and coreference resolution","volume":"28","author":"Miwa","year":"2012","journal-title":"Bioinformatics"},{"key":"2023012713011663400_btx774-B32","doi-asserted-by":"crossref","first-page":"i44","DOI":"10.1093\/bioinformatics\/btt227","article-title":"A method for integrating and ranking the evidence for biochemical pathways by mining reactions from text","volume":"29","author":"Miwa","year":"2013","journal-title":"Bioinformatics"},{"key":"2023012713011663400_btx774-B33","doi-asserted-by":"crossref","DOI":"10.1186\/1471-2105-14-175","article-title":"Wide coverage biomedical event extraction using multiple partially overlapping corpora","volume":"14","author":"Miwa","year":"2013","journal-title":"BMC Bioinformatics"},{"key":"2023012713011663400_btx774-B34","doi-asserted-by":"crossref","first-page":"541","DOI":"10.1109\/5.24143","article-title":"Petri nets: properties, analysis and applications","volume":"77","author":"Murata","year":"1989","journal-title":"Proc. IEEE"},{"key":"2023012713011663400_btx774-B35","doi-asserted-by":"crossref","first-page":"2155","DOI":"10.1093\/bioinformatics\/btg290","article-title":"Pathway studio\u2014the analysis and navigation of molecular networks","volume":"19","author":"Nikitin","year":"2003","journal-title":"Bioinformatics"},{"key":"2023012713011663400_btx774-B36","doi-asserted-by":"crossref","first-page":"1.","DOI":"10.1186\/1471-2105-9-S3-S5","article-title":"New challenges for text mining: mapping between text and manually curated pathways","volume":"9","author":"Oda","year":"2008","journal-title":"BMC Bioinformatics"},{"year":"2013","author":"Ohta","key":"2023012713011663400_btx774-B37"},{"key":"2023012713011663400_btx774-B38","doi-asserted-by":"crossref","first-page":"D548","DOI":"10.1093\/nar\/gkv1048","article-title":"SIGNOR: a database of causal relationships between biological entities","volume":"44","author":"Perfetto","year":"2015","journal-title":"Nucleic Acids Res"},{"year":"2015","author":"Poon","key":"2023012713011663400_btx774-B39"},{"key":"2023012713011663400_btx774-B40","doi-asserted-by":"crossref","first-page":"S2.","DOI":"10.1186\/1471-2105-16-S10-S2","article-title":"Overview of the cancer genetics and pathway curation tasks of BioNLP shared task 2013","volume":"16","author":"Pyysalo","year":"2015","journal-title":"BMC Bioinformatics"},{"key":"2023012713011663400_btx774-B41","doi-asserted-by":"crossref","first-page":"bas052.","DOI":"10.1093\/database\/bas052","article-title":"PPInterFinder\u2014a mining tool for extracting causal relations on human proteins from literature","volume":"2013","author":"Raja","year":"2013","journal-title":"Database"},{"key":"2023012713011663400_btx774-B42","doi-asserted-by":"crossref","first-page":"10.","DOI":"10.1155\/2017\/6213474","article-title":"A review of recent advancement in integrating omics data with literature mining towards biomedical discoveries","volume":"2017","author":"Raja","year":"2017","journal-title":"Int. J. Genomics"},{"key":"2023012713011663400_btx774-B43","doi-asserted-by":"crossref","first-page":"bas010.","DOI":"10.1093\/database\/bas010","article-title":"Argo: an integrative, interactive, text mining-based workbench supporting curation","volume":"2012","author":"Rak","year":"2012","journal-title":"Database"},{"year":"2014","author":"Ravikumar","key":"2023012713011663400_btx774-B44"},{"year":"2007","author":"Roberts","key":"2023012713011663400_btx774-B45"},{"key":"2023012713011663400_btx774-B46","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1016\/j.jbi.2003.10.001","article-title":"GeneWays: a system for extracting, analyzing, visualizing, and integrating molecular pathway data","volume":"37","author":"Rzhetsky","year":"2004","journal-title":"J. Biomed. Inf"},{"key":"2023012713011663400_btx774-B47","first-page":"42","article-title":"Impact factor: a valid measure of journal quality?","volume":"91","author":"Saha","year":"2003","journal-title":"J. Med. Library Assoc"},{"year":"1999","author":"Salamonsen","key":"2023012713011663400_btx774-B48"},{"year":"2012","author":"Schmidt","key":"2023012713011663400_btx774-B49"},{"year":"1996","author":"Shneiderman","key":"2023012713011663400_btx774-B50"},{"key":"2023012713011663400_btx774-B51","doi-asserted-by":"crossref","first-page":"222","DOI":"10.1016\/j.compbiolchem.2007.03.006","article-title":"CABIN: collective analysis of biological interaction networks","volume":"31","author":"Singhal","year":"2007","journal-title":"Comput. Biol. Chem"},{"key":"2023012713011663400_btx774-B52","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1002\/0471250953.bi0813s47","article-title":"Biological network exploration with Cytoscape 3","volume":"47","author":"Su","year":"2014","journal-title":"Curr. Protoc. Bioinf"},{"key":"2023012713011663400_btx774-B53","doi-asserted-by":"crossref","first-page":"393","DOI":"10.1186\/1471-2105-12-393","article-title":"Enriching a biomedical event corpus with meta-knowledge annotation","volume":"12","author":"Thompson","year":"2011","journal-title":"BMC Bioinformatics"},{"key":"2023012713011663400_btx774-B54","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/1471-2105-12-387","article-title":"Biblio-MetReS: A bibliometric network reconstruction application and server","volume":"12","author":"Usi\u00e9","year":"2011","journal-title":"BMC Bioinformatics"},{"key":"2023012713011663400_btx774-B55","doi-asserted-by":"crossref","first-page":"135.","DOI":"10.1186\/s12859-015-0550-z","article-title":"Visual analysis of biological data-knowledge networks","volume":"16","author":"Vehlow","year":"2015","journal-title":"BMC Bioinformatics"},{"key":"2023012713011663400_btx774-B56","doi-asserted-by":"crossref","first-page":"1221","DOI":"10.1109\/TVCG.2008.172","article-title":"The word tree, an interactive visual concordance","volume":"14","author":"Wattenberg","year":"2008","journal-title":"IEEE Trans. Visual. Comput. Graph"},{"key":"2023012713011663400_btx774-B57","doi-asserted-by":"crossref","first-page":"3784","DOI":"10.1093\/bioinformatics\/btx466","article-title":"Using uncertainty to link and rank evidence from biomedical literature for model curation","volume":"33","author":"Zerva","year":"2017","journal-title":"Bioinformatics"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/34\/8\/1389\/48915989\/bioinformatics_34_8_1389.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/34\/8\/1389\/48915989\/bioinformatics_34_8_1389.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,29]],"date-time":"2024-06-29T14:22:57Z","timestamp":1719670977000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/34\/8\/1389\/4708301"}},"subtitle":[],"editor":[{"given":"Jonathan","family":"Wren","sequence":"additional","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2017,12,8]]},"references-count":57,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2018,4,15]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btx774","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"type":"print","value":"1367-4803"},{"type":"electronic","value":"1367-4811"}],"subject":[],"published-other":{"date-parts":[[2018,4,15]]},"published":{"date-parts":[[2017,12,8]]}}}