{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,13]],"date-time":"2026-05-13T13:53:54Z","timestamp":1778680434270,"version":"3.51.4"},"reference-count":38,"publisher":"Springer Science and Business Media LLC","issue":"1","content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Bioinformatics"],"published-print":{"date-parts":[[2008,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:sec>\n            <jats:title>Background<\/jats:title>\n            <jats:p>The Gene Ontology is a controlled vocabulary for representing knowledge related to genes and proteins in a computable form. The current effort of manually annotating proteins with the Gene Ontology is outpaced by the rate of accumulation of biomedical knowledge in literature, which urges the development of text mining approaches to facilitate the process by automatically extracting the Gene Ontology annotation from literature. The task is usually cast as a text classification problem, and contemporary methods are confronted with unbalanced training data and the difficulties associated with multi-label classification.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Results<\/jats:title>\n            <jats:p>In this research, we investigated the methods of enhancing automatic multi-label classification of biomedical literature by utilizing the structure of the Gene Ontology graph. We have studied three graph-based multi-label classification algorithms, including a novel stochastic algorithm and two top-down hierarchical classification methods for multi-label literature classification. We systematically evaluated and compared these graph-based classification algorithms to a conventional flat multi-label algorithm. The results indicate that, through utilizing the information from the structure of the Gene Ontology graph, the graph-based multi-label classification methods can significantly improve predictions of the Gene Ontology terms implied by the analyzed text. Furthermore, the graph-based multi-label classifiers are capable of suggesting Gene Ontology annotations (to curators) that are closely related to the true annotations even if they fail to predict the true ones directly. A software package implementing the studied algorithms is available for the research community.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Conclusion<\/jats:title>\n            <jats:p>Through utilizing the information from the structure of the Gene Ontology graph, the graph-based multi-label classification methods have better potential than the conventional flat multi-label classification approach to facilitate protein annotation based on the literature.<\/jats:p>\n          <\/jats:sec>","DOI":"10.1186\/1471-2105-9-525","type":"journal-article","created":{"date-parts":[[2008,12,9]],"date-time":"2008-12-09T07:13:59Z","timestamp":1228806839000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":23,"title":["Multi-label literature classification based on the Gene Ontology graph"],"prefix":"10.1186","volume":"9","author":[{"given":"Bo","family":"Jin","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Brian","family":"Muller","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chengxiang","family":"Zhai","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xinghua","family":"Lu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2008,12,8]]},"reference":[{"key":"2510_CR1","first-page":"D262","volume-title":"Nucleic Acids Res","author":"E Camon","year":"2004","unstructured":"Camon E, Magrane M, Barrell D, Lee V, Dimmer E, Maslen J, Binns D, Harte N, Lopez R, Apweiler R: The Gene Ontology Annotation (GOA) Database: sharing knowledge in Uniprot with Gene Ontology. Nucleic Acids Res 2004, (32 Database):D262\u2013266."},{"issue":"13","key":"2510_CR2","doi-asserted-by":"publisher","first-page":"i41","DOI":"10.1093\/bioinformatics\/btm229","volume":"23","author":"WA Baumgartner Jr","year":"2007","unstructured":"Baumgartner WA Jr, Cohen KB, Fox LM, Acquaah-Mensah G, Hunter L: Manual curation is not sufficient for annotation of genomic databases. Bioinformatics 2007, 23(13):i41\u201348.","journal-title":"Bioinformatics"},{"issue":"Suppl 1","key":"2510_CR3","doi-asserted-by":"publisher","first-page":"S17","DOI":"10.1186\/1471-2105-6-S1-S17","volume":"6","author":"EB Camon","year":"2005","unstructured":"Camon EB, Barrell DG, Dimmer EC, Lee V, Magrane M, Maslen J, Binns D, Apweiler R: An evaluation of GO annotation retrieval for BioCreAtIvE and GOA. BMC Bioinformatics 2005, 6(Suppl 1):S17.","journal-title":"BMC Bioinformatics"},{"key":"2510_CR4","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1186\/1747-5333-1-4","volume":"1","author":"AM Cohen","year":"2006","unstructured":"Cohen AM, Hersh WR: The TREC 2004 genomics track categorization task: classifying full text biomedical documents. J Biomed Discov Collab 2006, 1: 4.","journal-title":"J Biomed Discov Collab"},{"key":"2510_CR5","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1186\/1747-5333-1-19","volume":"1","author":"FM Couto","year":"2006","unstructured":"Couto FM, Silva MJ, Lee V, Dimmer E, Camon E, Apweiler R, Kirsch H, Rebholz-Schuhmann D: GOAnnotator: linking protein GO annotations to evidence text. J Biomed Discov Collab 2006, 1: 19.","journal-title":"J Biomed Discov Collab"},{"key":"2510_CR6","doi-asserted-by":"publisher","first-page":"243","DOI":"10.1186\/1471-2105-8-243","volume":"8","author":"N Daraselia","year":"2007","unstructured":"Daraselia N, Yuryev A, Egorov S, Mazo I, Ispolatov I: Automatic extraction of gene ontology annotation and its correlation with clusters in protein networks. BMC Bioinformatics 2007, 8: 243.","journal-title":"BMC Bioinformatics"},{"issue":"5","key":"2510_CR7","doi-asserted-by":"publisher","first-page":"354","DOI":"10.1016\/j.ijmedinf.2007.07.004","volume":"77","author":"R Jelier","year":"2007","unstructured":"Jelier R, Schuemie MJ, Roes PJ, van Mulligen EM, Kors JA: Literature-based concept profiles for gene annotation: The issue of weighting. Int J Med Inform 2007, 77(5):354\u2013362.","journal-title":"Int J Med Inform"},{"issue":"1","key":"2510_CR8","doi-asserted-by":"publisher","first-page":"243","DOI":"10.1186\/1471-2105-8-243","volume":"8","author":"N Daraselia","year":"2007","unstructured":"Daraselia N, Yuryev A, Egorov S, Mazo I, Ispolatov I: Automatic extraction of gene ontology annotation and its correlation with clusters in protein networks. BMC Bioinformatics 2007, 8(1):243.","journal-title":"BMC Bioinformatics"},{"issue":"3","key":"2510_CR9","doi-asserted-by":"publisher","first-page":"1","DOI":"10.4018\/jdwm.2007070101","volume":"3","author":"G Tsoumakas","year":"2007","unstructured":"Tsoumakas G, Katakis I: Multi-Label Classification: An Overview. International Journal of Data Warehousing and Mining 2007, 3(3):1\u201313.","journal-title":"International Journal of Data Warehousing and Mining"},{"issue":"7","key":"2510_CR10","doi-asserted-by":"publisher","first-page":"830","DOI":"10.1093\/bioinformatics\/btk048","volume":"22","author":"Z Barutcuoglu","year":"2006","unstructured":"Barutcuoglu Z, Schapire RE, Troyanskaya OG: Hierarchical multi-label prediction of gene function. Bioinformatics 2006, 22(7):830\u2013836.","journal-title":"Bioinformatics"},{"key":"2510_CR11","volume-title":"ACM 13th Conference on Information Management:2004","author":"L Cai","year":"2004","unstructured":"Cai L, Hofmann T: Hierarchical document categorization with support vector machines. ACM 13th Conference on Information Management:2004 2004."},{"key":"2510_CR12","first-page":"256","volume-title":"ACM Special Interest Group on Information Retrieval (SIGIR): 2000","author":"ST Dumais","year":"2000","unstructured":"Dumais ST, Chen H: Hierarchical classification of web content. ACM Special Interest Group on Information Retrieval (SIGIR): 2000 2000, 256\u2013263."},{"key":"2510_CR13","first-page":"1601","volume":"7","author":"J Rousu","year":"2006","unstructured":"Rousu J, Saunders C, Shawe-Taylor J: Kernel-based learning of hierarchical multilabel classification models. Journal of Machine Learning Research 2006, 7: 1601\u20131626.","journal-title":"Journal of Machine Learning Research"},{"issue":"6","key":"2510_CR14","doi-asserted-by":"publisher","first-page":"1544","DOI":"10.1110\/ps.062184006","volume":"15","author":"K Verspoor","year":"2006","unstructured":"Verspoor K, Cohn J, Mniszewski S, Joslyn C: A categorization approach to automated ontological function annotation. Protein Sci 2006, 15(6):1544\u20131549.","journal-title":"Protein Sci"},{"issue":"14","key":"2510_CR15","doi-asserted-by":"publisher","first-page":"e530","DOI":"10.1093\/bioinformatics\/btl208","volume":"22","author":"K Wolstencroft","year":"2006","unstructured":"Wolstencroft K, Lord P, Tabernero L, Brass A, Stevens R: Protein classification using ontology classification. Bioinformatics 2006, 22(14):e530\u2013538.","journal-title":"Bioinformatics"},{"key":"2510_CR16","volume-title":"BioLINK SIG: Linking Literature, Information and Knowledge for Biology, a Joint Meeting of The ISMB BioLINK Special Interest Group on Text Data Mining and The ACL Workshop on Linking Biological Literature, Ontologies and Databases: Mining Biological Semant","author":"S Kiritchenko","year":"2005","unstructured":"Kiritchenko S, Matwin S, Famili FID: Functional Annotation of Genes Using Hierarchical Text Categorization. BioLINK SIG: Linking Literature, Information and Knowledge for Biology, a Joint Meeting of The ISMB BioLINK Special Interest Group on Text Data Mining and The ACL Workshop on Linking Biological Literature, Ontologies and Databases: Mining Biological Semantics 2005."},{"issue":"Suppl 1","key":"2510_CR17","doi-asserted-by":"publisher","first-page":"S20","DOI":"10.1186\/1471-2105-6-S1-S20","volume":"6","author":"K Verspoor","year":"2005","unstructured":"Verspoor K, Cohn J, Joslyn C, Mniszewski S, Rechtsteiner A, Rocha LM, Simas T: Protein annotation as term categorization in the gene ontology using word proximity networks. BMC bioinformatics 2005, 6(Suppl 1):S20.","journal-title":"BMC bioinformatics"},{"key":"2510_CR18","unstructured":"GOA: Gene Ontology Annotation Project.[http:\/\/www.ebi.ac.uk\/GOA\/]"},{"issue":"5","key":"2510_CR19","first-page":"40","volume":"61","author":"C Lindberg","year":"1990","unstructured":"Lindberg C: The Unified Medical Language System (UMLS) of the National Library of Medicine. Journal (American Medical Record Association) 1990, 61(5):40\u201342.","journal-title":"Journal (American Medical Record Association)"},{"key":"2510_CR20","doi-asserted-by":"crossref","unstructured":"Bodenreider O: The Unified Medical Language System (UMLS): integrating biomedical terminology. Nucleic Acids Res 2004, (32 Database):D267\u2013270.","DOI":"10.1093\/nar\/gkh061"},{"key":"2510_CR21","first-page":"361","volume":"5","author":"DD Lewis","year":"2004","unstructured":"Lewis DD, Yang Y, Rose TG, Li F: RCV1: A new benchmark collection for text categorization research. Journal of Machine Learning Research 2004, 5: 361\u2013397.","journal-title":"Journal of Machine Learning Research"},{"key":"2510_CR22","doi-asserted-by":"publisher","first-page":"58","DOI":"10.1186\/1471-2105-7-58","volume":"7","author":"B Zheng","year":"2006","unstructured":"Zheng B, McLean DC Jr, Lu X: Identifying biological concepts from a protein-related corpus with a probabilistic topic model. BMC Bioinformatics 2006, 7: 58.","journal-title":"BMC Bioinformatics"},{"issue":"1","key":"2510_CR23","first-page":"5","volume":"4","author":"E Camon","year":"2004","unstructured":"Camon E, Barrell D, Lee V, Dimmer E, Apweiler R: The Gene Ontology Annotation (GOA) Database\u2013an integrated resource of GO annotations to the UniProt Knowledgebase. In Silico Biol 2004, 4(1):5\u20136.","journal-title":"In Silico Biol"},{"issue":"3","key":"2510_CR24","doi-asserted-by":"publisher","first-page":"130","DOI":"10.1108\/eb046814","volume":"14","author":"MF Porter","year":"1980","unstructured":"Porter MF: An algorithm for suffix stripping. Program 1980, 14(3):130\u2013137.","journal-title":"Program"},{"key":"2510_CR25","volume-title":"Introduction to algorithms","author":"T Cormen","year":"2001","unstructured":"Cormen T, Leiserson CE, Rivest RL, Stein C: Introduction to algorithms. 2nd edition. Cambridge, MA: MIT Press; 2001.","edition":"2"},{"key":"2510_CR26","doi-asserted-by":"publisher","first-page":"144","DOI":"10.1145\/130385.130401","volume-title":"Proc 5th Annual Workshop on Computational Learning Theory","author":"B Boser","year":"1992","unstructured":"Boser B, Guyon I, Vapnik VN: A training algorithm for optimal margin classifiers. In Proc 5th Annual Workshop on Computational Learning Theory. New York, ACM Press; 1992:144\u2013152."},{"issue":"5","key":"2510_CR27","doi-asserted-by":"publisher","first-page":"526","DOI":"10.1197\/jamia.M2051","volume":"13","author":"X Lu","year":"2006","unstructured":"Lu X, Zheng B, Velivelli A, Zhai C: Enhancing text categorization with semantic-enriched representation and training data augmentation. J Am Med Inform Assoc 2006, 13(5):526\u2013535.","journal-title":"J Am Med Inform Assoc"},{"key":"2510_CR28","volume-title":"Statistical Learning Theory","author":"VN Vapnik","year":"1998","unstructured":"Vapnik VN: Statistical Learning Theory. New York: John Wiley and Sons; 1998."},{"key":"2510_CR29","volume-title":"LIBSVM: a library for support vector machines","author":"C-C Chang","year":"2001","unstructured":"Chang C-C, Lin CJ: LIBSVM: a library for support vector machines. 2001."},{"key":"2510_CR30","volume-title":"the 14th International Conference on Machine Learning (ICML)","author":"D Koller","year":"1997","unstructured":"Koller D, Sahami M: Hierarchically classifying documents using very few words. the 14th International Conference on Machine Learning (ICML) 1997."},{"key":"2510_CR31","first-page":"41","volume-title":"AAAI\/ICML-98 Workshop on Learning for Text Categorization: 1998","author":"A McCallum","year":"1998","unstructured":"McCallum A, Nigam K: A Comparison of Event Models for Naive Bayes Text Classification. In AAAI\/ICML-98 Workshop on Learning for Text Categorization: 1998. AAAI Press; 1998:41\u201348."},{"key":"2510_CR32","doi-asserted-by":"publisher","first-page":"97","DOI":"10.1093\/biomet\/57.1.97","volume":"57","author":"WK Hastings","year":"1970","unstructured":"Hastings WK: Monte Carlo sampling methods using Markov chains and their applications. Biometrika 1970, 57: 97\u2013109.","journal-title":"Biometrika"},{"key":"2510_CR33","volume-title":"Proceedings on International Conference on Research in Computational Linguistics: 1998; Taiwan","author":"J Jiang","year":"1998","unstructured":"Jiang J, Conrath D: Semantic similarity based on corpus statistics and lexical taxonomy. Proceedings on International Conference on Research in Computational Linguistics: 1998; Taiwan 1998."},{"key":"2510_CR34","first-page":"296","volume-title":"Proc 15th International Conf on Machine Learning","author":"D Lin","year":"1998","unstructured":"Lin D: An information-theoretic definition of similarity. In Proc 15th International Conf on Machine Learning. San Francisco, CA: Morgan Kaufmann; 1998:296\u2013304."},{"issue":"10","key":"2510_CR35","doi-asserted-by":"publisher","first-page":"1275","DOI":"10.1093\/bioinformatics\/btg153","volume":"19","author":"P Lord","year":"2003","unstructured":"Lord P, Stevens R, Brass A, Goble C: Investigating semantic similarity measures across the Gene Ontology: the relationship between sequence and annotation. Bioinformatics 2003, 19(10):1275\u20131283.","journal-title":"Bioinformatics"},{"key":"2510_CR36","first-page":"448","volume-title":"Proceedings of the 14th International Joint Conference on Artificial Intelligence","author":"P Resnik","year":"1995","unstructured":"Resnik P: Using information content to evaluate semantic similarity in a taxonomy. Proceedings of the 14th International Joint Conference on Artificial Intelligence 1995, 448\u2013453."},{"key":"2510_CR37","first-page":"1","volume-title":"Evaluation Methods for Machine Learning II: papers from the AAAI-2007 Workshop","author":"EP Costa","year":"2007","unstructured":"Costa EP, Lorena AC, Carvalho AeCPLF, Freitas AA: A Review of Performance Evaluation Measures for Hierarchical Classifiers. In Evaluation Methods for Machine Learning II: papers from the AAAI-2007 Workshop Edited by: Drummond WE C, Japkowicz N, Macskassy SA. 2007, 1\u20136."},{"key":"2510_CR38","doi-asserted-by":"publisher","first-page":"269","DOI":"10.1007\/BF01386390","volume":"1","author":"EW Dijkstra","year":"1959","unstructured":"Dijkstra EW: A note on two problems in connection with graphs. Numerische Mathematic 1959, 1: 269\u2013271.","journal-title":"Numerische Mathematic"}],"container-title":["BMC Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/1471-2105-9-525.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,9,1]],"date-time":"2021-09-01T03:27:30Z","timestamp":1630466850000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/1471-2105-9-525"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2008,12]]},"references-count":38,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2008,12]]}},"alternative-id":["2510"],"URL":"https:\/\/doi.org\/10.1186\/1471-2105-9-525","relation":{},"ISSN":["1471-2105"],"issn-type":[{"value":"1471-2105","type":"electronic"}],"subject":[],"published":{"date-parts":[[2008,12]]},"assertion":[{"value":"28 July 2008","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 December 2008","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 December 2008","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"525"}}