{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,16]],"date-time":"2026-01-16T12:46:01Z","timestamp":1768567561776,"version":"3.49.0"},"reference-count":39,"publisher":"Oxford University Press (OUP)","issue":"14","license":[{"start":{"date-parts":[[2020,5,7]],"date-time":"2020-05-07T00:00:00Z","timestamp":1588809600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61872094"],"award-info":[{"award-number":["61872094"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61572139"],"award-info":[{"award-number":["61572139"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Shanghai Municipal Science and Technology Major Project","award":["2017SHZDZX01"],"award-info":[{"award-number":["2017SHZDZX01"]}]},{"DOI":"10.13039\/501100013314","name":"111 Project","doi-asserted-by":"publisher","award":["B18015"],"award-info":[{"award-number":["B18015"]}],"id":[{"id":"10.13039\/501100013314","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Shanghai Science & Technology","award":["16JC1420402"],"award-info":[{"award-number":["16JC1420402"]}]},{"name":"Shanghai Municipal Science and Technology Major Project","award":["2018SHZDZX01"],"award-info":[{"award-number":["2018SHZDZX01"]}]},{"name":"JST ACCEL","award":["JPMJAC1503"],"award-info":[{"award-number":["JPMJAC1503"]}]},{"name":"MEXT Kakenhi","award":["16H02868"],"award-info":[{"award-number":["16H02868"]}]},{"name":"MEXT Kakenhi","award":["19H04169"],"award-info":[{"award-number":["19H04169"]}]},{"name":"AIPSE program"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2020,7,30]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Motivation<\/jats:title>\n                  <jats:p>Annotating human proteins by abnormal phenotypes has become an important topic. Human Phenotype Ontology (HPO) is a standardized vocabulary of phenotypic abnormalities encountered in human diseases. As of November 2019, only &amp;lt;4000 proteins have been annotated with HPO. Thus, a computational approach for accurately predicting protein\u2013HPO associations would be important, whereas no methods have outperformed a simple Naive approach in the second Critical Assessment of Functional Annotation, 2013\u20132014 (CAFA2).<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>We present HPOLabeler, which is able to use a wide variety of evidence, such as protein\u2013protein interaction (PPI) networks, Gene Ontology, InterPro, trigram frequency and HPO term frequency, in the framework of learning to rank (LTR). LTR has been proved to be powerful for solving large-scale, multi-label ranking problems in bioinformatics. Given an input protein, LTR outputs the ranked list of HPO terms from a series of input scores given to the candidate HPO terms by component learning models (logistic regression, nearest neighbor and a Naive method), which are trained from given multiple evidence. We empirically evaluate HPOLabeler extensively through mainly two experiments of cross validation and temporal validation, for which HPOLabeler significantly outperformed all component models and competing methods including the current state-of-the-art method. We further found that (i) PPI is most informative for prediction among diverse data sources and (ii) low prediction performance of temporal validation might be caused by incomplete annotation of new proteins.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Availability and implementation<\/jats:title>\n                  <jats:p>http:\/\/issubmission.sjtu.edu.cn\/hpolabeler\/.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Contact<\/jats:title>\n                  <jats:p>zhusf@fudan.edu.cn<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Supplementary information<\/jats:title>\n                  <jats:p>Supplementary data are available at Bioinformatics online.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btaa284","type":"journal-article","created":{"date-parts":[[2020,4,30]],"date-time":"2020-04-30T12:53:12Z","timestamp":1588251192000},"page":"4180-4188","source":"Crossref","is-referenced-by-count":29,"title":["HPOLabeler: improving prediction of human protein\u2013phenotype associations by learning to rank"],"prefix":"10.1093","volume":"36","author":[{"given":"Lizhi","family":"Liu","sequence":"first","affiliation":[{"name":"School of Computer Science and Shanghai Key Lab of Intelligent Information Processing"},{"name":"Shanghai Institute of Artificial Intelligence Algorithms and Institute of Science and Technology for Brain-Inspired Intelligence , Fudan University, Shanghai 200433, China"},{"name":"Bio-Med Big Data Center , Key Laboratory of Computational Biology, CAS-MPG Partner Institute for Computational Biology, Shanghai Institute of Nutrition and Health, Shanghai Institutes for Biological Science, Chinese Academy of Sciences, Shanghai 200031, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiaodi","family":"Huang","sequence":"additional","affiliation":[{"name":"School of Computing and Mathematics , Charles Sturt University, Albury, NSW 2640, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hiroshi","family":"Mamitsuka","sequence":"additional","affiliation":[{"name":"Bioinformatics Center , Institute for Chemical Research, Kyoto University, Uji 611-0011, Japan"},{"name":"Department of Computer Science , Aalto University, Espoo, Finland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shanfeng","family":"Zhu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Shanghai Key Lab of Intelligent Information Processing"},{"name":"Shanghai Institute of Artificial Intelligence Algorithms and Institute of Science and Technology for Brain-Inspired Intelligence , Fudan University, Shanghai 200433, China"},{"name":"Bio-Med Big Data Center , Key Laboratory of Computational Biology, CAS-MPG Partner Institute for Computational Biology, Shanghai Institute of Nutrition and Health, Shanghai Institutes for Biological Science, Chinese Academy of Sciences, Shanghai 200031, China"},{"name":"Key Laboratory of Computational Neuroscience and Brain-Inspired Intelligence (Fudan University) , Ministry of Education, Shanghai, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2020,5,7]]},"reference":[{"key":"2023062213545097000_btaa284-B1","doi-asserted-by":"crossref","first-page":"681","DOI":"10.1038\/nrg3555","article-title":"Rare-disease genetics in the era of next-generation sequencing: discovery to translation","volume":"14","author":"Boycott","year":"2013","journal-title":"Nat. Rev. Genet"},{"key":"2023062213545097000_btaa284-B2","author":"Burges","year":"2010"},{"key":"2023062213545097000_btaa284-B3","first-page":"193","article-title":"Learning to rank with nonsmooth cost functions","author":"Burges","year":"2006"},{"key":"2023062213545097000_btaa284-B4","doi-asserted-by":"crossref","first-page":"D369","DOI":"10.1093\/nar\/gkw1102","article-title":"The BioGRID interaction database: 2017 update","volume":"45","author":"Chatr-Aryamontri","year":"2017","journal-title":"Nucleic Acids Res"},{"key":"2023062213545097000_btaa284-B5","first-page":"785","author":"Chen","year":"2016"},{"key":"2023062213545097000_btaa284-B6","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1016\/j.ajhg.2015.06.009","article-title":"The genetic basis of Mendelian phenotypes: discoveries, challenges, and opportunities","volume":"97","author":"Chong","year":"2015","journal-title":"Am. J. Hum. Genet"},{"key":"2023062213545097000_btaa284-B7","doi-asserted-by":"crossref","first-page":"3364","DOI":"10.1093\/bioinformatics\/btm520","article-title":"An efficient strategy for extensive integration of diverse biological data for protein function prediction","volume":"23","author":"Chua","year":"2007","journal-title":"Bioinformatics"},{"key":"2023062213545097000_btaa284-B8","doi-asserted-by":"crossref","first-page":"e5298","DOI":"10.7717\/peerj.5298","article-title":"HPO2GO: prediction of human phenotype ontology term associations for proteins using cross ontology annotation co-occurrences","volume":"6","author":"Do\u011fan","year":"2018","journal-title":"PeerJ"},{"key":"2023062213545097000_btaa284-B9","doi-asserted-by":"crossref","first-page":"D190","DOI":"10.1093\/nar\/gkw1107","article-title":"InterPro in 2017\u2014beyond protein family and domain annotations","volume":"45","author":"Finn","year":"2017","journal-title":"Nucleic Acids Res"},{"key":"2023062213545097000_btaa284-B10","doi-asserted-by":"crossref","first-page":"1189","DOI":"10.1214\/aos\/1013203451","article-title":"Greedy function approximation: a gradient boosting machine","volume":"29","author":"Friedman","year":"2001","journal-title":"Ann. Stat"},{"key":"2023062213545097000_btaa284-B11","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1038\/ng1747","article-title":"Analysis of the human protein interactome and comparison with yeast, worm and fly interaction datasets","volume":"38","author":"Gandhi","year":"2006","journal-title":"Nat. Genet"},{"key":"2023062213545097000_btaa284-B12","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1186\/s12920-019-0625-1","article-title":"HPOAnnotator: improving large-scale prediction of HPO annotations by low-rank approximation with HPO semantic similarities and multiple PPI networks","volume":"12","author":"Gao","year":"2019","journal-title":"BMC Med. Genomics"},{"key":"2023062213545097000_btaa284-B13","doi-asserted-by":"crossref","first-page":"8685","DOI":"10.1073\/pnas.0701361104","article-title":"The human disease network","volume":"104","author":"Goh","year":"2007","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"2023062213545097000_btaa284-B14","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1016\/j.ajhg.2015.05.020","article-title":"The Human Phenotype Ontology: semantic unification of common and rare disease","volume":"97","author":"Groza","year":"2015","journal-title":"Am. J. Hum. Genet"},{"key":"2023062213545097000_btaa284-B15","doi-asserted-by":"crossref","first-page":"D1057","DOI":"10.1093\/nar\/gku1113","article-title":"The GOA database: gene ontology annotation updates for 2015","volume":"43","author":"Huntley","year":"2015","journal-title":"Nucleic Acids Res"},{"key":"2023062213545097000_btaa284-B16","doi-asserted-by":"crossref","first-page":"184","DOI":"10.1186\/s13059-016-1037-6","article-title":"An expanded evaluation of protein function prediction methods shows an improvement in accuracy","volume":"17","author":"Jiang","year":"2016","journal-title":"Genome Biol"},{"key":"2023062213545097000_btaa284-B17","doi-asserted-by":"crossref","first-page":"1236","DOI":"10.1093\/bioinformatics\/btu031","article-title":"InterProScan 5: genome-scale protein function classification","volume":"30","author":"Jones","year":"2014","journal-title":"Bioinformatics"},{"key":"2023062213545097000_btaa284-B18","doi-asserted-by":"crossref","first-page":"259","DOI":"10.12688\/f1000research.6670.1","article-title":"PHENOstruct: prediction of human phenotype ontology terms using heterogeneous data sources [version 1; referees: 2 approved]","volume":"4","author":"Kahanda","year":"2015","journal-title":"F1000Res"},{"key":"2023062213545097000_btaa284-B19","doi-asserted-by":"crossref","first-page":"949","DOI":"10.1016\/j.ajhg.2008.02.013","article-title":"Walking the interactome for prioritization of candidate disease genes","volume":"82","author":"K\u00f6hler","year":"2008","journal-title":"Am. J. Hum. Genet"},{"key":"2023062213545097000_btaa284-B20","doi-asserted-by":"crossref","first-page":"D1018","DOI":"10.1093\/nar\/gky1105","article-title":"Expansion of the Human Phenotype Ontology (HPO) knowledge base and resources","volume":"47","author":"K\u00f6hler","year":"2019","journal-title":"Nucleic Acids Res"},{"key":"2023062213545097000_btaa284-B21","doi-asserted-by":"crossref","first-page":"M111","DOI":"10.1074\/mcp.M111.009993","article-title":"The human proteome project: current state and future direction","volume":"10","author":"Legrain","year":"2011","journal-title":"Mol. Cell. Proteomics"},{"key":"2023062213545097000_btaa284-B22","doi-asserted-by":"crossref","first-page":"i339","DOI":"10.1093\/bioinformatics\/btv237","article-title":"MeSHLabeler: improving the accuracy of large-scale MeSH indexing by integrating diverse evidence","volume":"31","author":"Liu","year":"2015","journal-title":"Bioinformatics"},{"key":"2023062213545097000_btaa284-B23","doi-asserted-by":"crossref","first-page":"1432","DOI":"10.1101\/gr.190603.115","article-title":"Genomic approaches for understanding the genetics of complex disease","volume":"25","author":"Lowe","year":"2015","journal-title":"Genome Res"},{"key":"2023062213545097000_btaa284-B24","doi-asserted-by":"crossref","first-page":"3486","DOI":"10.1093\/emboj\/cdg359","article-title":"Diversity of protein\u2013protein interactions","volume":"22","author":"Nooren","year":"2003","journal-title":"EMBO J"},{"key":"2023062213545097000_btaa284-B25","author":"Notaro","year":"2017"},{"key":"2023062213545097000_btaa284-B26","doi-asserted-by":"crossref","first-page":"449","DOI":"10.1186\/s12859-017-1854-y","article-title":"Prediction of Human Phenotype Ontology terms by means of hierarchical ensemble methods","volume":"18","author":"Notaro","year":"2017","journal-title":"BMC Bioinformatics"},{"key":"2023062213545097000_btaa284-B27","doi-asserted-by":"crossref","first-page":"691","DOI":"10.1136\/jmg.2006.041376","article-title":"Predicting disease genes using protein\u2013protein interactions","volume":"43","author":"Oti","year":"2006","journal-title":"J. Med. Genet"},{"key":"2023062213545097000_btaa284-B28","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1093\/bioinformatics\/btw649","article-title":"Transfer learning across ontologies for phenome\u2013genome association prediction","volume":"33","author":"Petegrosso","year":"2016","journal-title":"Bioinformatics"},{"key":"2023062213545097000_btaa284-B29","author":"Radivojac","year":"2013"},{"key":"2023062213545097000_btaa284-B30","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1142\/S0219720010004744","article-title":"Hierarchical classification of gene ontology terms using the GOstruct method","volume":"8","author":"Sokolov","year":"2010","journal-title":"J. Bioinform. Comput. Biol"},{"key":"2023062213545097000_btaa284-B31","doi-asserted-by":"crossref","first-page":"S10","DOI":"10.1186\/1471-2105-14-S3-S10","article-title":"Combining heterogeneous data sources for accurate functional annotation of proteins","volume":"14","author":"Sokolov","year":"2013","journal-title":"BMC Bioinformatics"},{"key":"2023062213545097000_btaa284-B32","doi-asserted-by":"crossref","first-page":"D447","DOI":"10.1093\/nar\/gku1003","article-title":"STRING v10: protein\u2013protein interaction networks, integrated over the tree of life","volume":"43","author":"Szklarczyk","year":"2015","journal-title":"Nucleic Acids Res"},{"key":"2023062213545097000_btaa284-B33","doi-asserted-by":"crossref","first-page":"D331","DOI":"10.1093\/nar\/gkw1108","article-title":"Expansion of the Gene Ontology knowledgebase and resources","volume":"45","year":"2017","journal-title":"Nucleic Acids Res"},{"key":"2023062213545097000_btaa284-B34","doi-asserted-by":"crossref","first-page":"e77478","DOI":"10.1371\/journal.pone.0077478","article-title":"Inference of gene\u2013phenotype associations via protein\u2013protein interaction and orthology","volume":"8","author":"Wang","year":"2013","journal-title":"PLoS One"},{"key":"2023062213545097000_btaa284-B35","doi-asserted-by":"crossref","first-page":"W214","DOI":"10.1093\/nar\/gkq537","article-title":"The GeneMANIA prediction server: biological network integration for gene prioritization and predicting gene function","volume":"38","author":"Warde-Farley","year":"2010","journal-title":"Nucleic Acids Res"},{"key":"2023062213545097000_btaa284-B36","doi-asserted-by":"crossref","first-page":"2800","DOI":"10.1093\/bioinformatics\/btl467","article-title":"Discovering disease-genes by topological features in human protein\u2013protein interaction network","volume":"22","author":"Xu","year":"2006","journal-title":"Bioinformatics"},{"key":"2023062213545097000_btaa284-B37","doi-asserted-by":"crossref","first-page":"2465","DOI":"10.1093\/bioinformatics\/bty130","article-title":"GOLabeler: improving sequence-based large-scale protein function prediction by learning to rank","volume":"34","author":"You","year":"2018","journal-title":"Bioinformatics"},{"key":"2023062213545097000_btaa284-B38","doi-asserted-by":"crossref","first-page":"i18","DOI":"10.1093\/bioinformatics\/btw244","article-title":"DrugE-Rank: improving drug\u2013target interaction prediction of new candidate drugs or targets by ensemble learning to rank","volume":"32","author":"Yuan","year":"2016","journal-title":"Bioinformatics"},{"key":"2023062213545097000_btaa284-B39","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s13059-019-1835-8","article-title":"The CAFA challenge reports improved protein function prediction and new functional annotations for hundreds of genes through experimental screens","volume":"20","author":"Zhou","year":"2019","journal-title":"Genome Biol"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btaa284\/33440294\/btaa284.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/36\/14\/4180\/50677386\/btaa284.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/36\/14\/4180\/50677386\/btaa284.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,23]],"date-time":"2023-06-23T16:29:16Z","timestamp":1687537756000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/36\/14\/4180\/5831828"}},"subtitle":[],"editor":[{"given":"Jonathan","family":"Wren","sequence":"additional","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2020,5,7]]},"references-count":39,"journal-issue":{"issue":"14","published-print":{"date-parts":[[2020,7,30]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btaa284","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2020,7,15]]},"published":{"date-parts":[[2020,5,7]]}}}