{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,12]],"date-time":"2026-06-12T16:33:06Z","timestamp":1781281986734,"version":"3.54.1"},"reference-count":57,"publisher":"Oxford University Press (OUP)","issue":"2","license":[{"start":{"date-parts":[[2024,2,22]],"date-time":"2024-02-22T00:00:00Z","timestamp":1708560000000},"content-version":"vor","delay-in-days":31,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"name":"High Performance Computing Center of Central South University"},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61832019"],"award-info":[{"award-number":["61832019"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61622213"],"award-info":[{"award-number":["61622213"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61728211"],"award-info":[{"award-number":["61728211"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["G20190018001"],"award-info":[{"award-number":["G20190018001"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Hunan Provincial Science and Technology Program","award":["2018WK4001"],"award-info":[{"award-number":["2018WK4001"]}]},{"name":"Hunan Graduate Research and Innovation Project","award":["CX20190082"],"award-info":[{"award-number":["CX20190082"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,1,22]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Proteins play an important role in life activities and are the basic units for performing functions. Accurately annotating functions to proteins is crucial for understanding the intricate mechanisms of life and developing effective treatments for complex diseases. Traditional biological experiments struggle to keep pace with the growing number of known proteins. With the development of high-throughput sequencing technology, a wide variety of biological data provides the possibility to accurately predict protein functions by computational methods. Consequently, many computational methods have been proposed. Due to the diversity of application scenarios, it is necessary to conduct a comprehensive evaluation of these computational methods to determine the suitability of each algorithm for specific cases. In this study, we present a comprehensive benchmark, BeProf, to process data and evaluate representative computational methods. We first collect the latest datasets and analyze the data characteristics. Then, we investigate and summarize 17 state-of-the-art computational methods. Finally, we propose a novel comprehensive evaluation metric, design eight application scenarios and evaluate the performance of existing methods on these scenarios. Based on the evaluation, we provide practical recommendations for different scenarios, enabling users to select the most suitable method for their specific needs. All of these servers can be obtained from https:\/\/csuligroup.com\/BEPROF and https:\/\/github.com\/CSUBioGroup\/BEPROF.<\/jats:p>","DOI":"10.1093\/bib\/bbae050","type":"journal-article","created":{"date-parts":[[2024,2,23]],"date-time":"2024-02-23T01:31:06Z","timestamp":1708651866000},"source":"Crossref","is-referenced-by-count":11,"title":["A comprehensive computational benchmark for evaluating deep learning-based protein function prediction approaches"],"prefix":"10.1093","volume":"25","author":[{"given":"Wenkang","family":"Wang","sequence":"first","affiliation":[{"name":"School of Computer Science and Engineering, Central South University , 932 South Lushan Road, Yuelu District, Changsha 410083 , China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yunyan","family":"Shuai","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Central South University , 932 South Lushan Road, Yuelu District, Changsha 410083 , China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qiurong","family":"Yang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Central South University , 932 South Lushan Road, Yuelu District, Changsha 410083 , China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fuhao","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Central South University , 932 South Lushan Road, Yuelu District, Changsha 410083 , China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Min","family":"Zeng","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Central South University , 932 South Lushan Road, Yuelu District, Changsha 410083 , China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0188-1394","authenticated-orcid":false,"given":"Min","family":"Li","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Central South University , 932 South Lushan Road, Yuelu District, Changsha 410083 , China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2024,2,22]]},"reference":[{"issue":"4","key":"2024022301305421300_ref1","doi-asserted-by":"crossref","first-page":"1386","DOI":"10.1109\/TCBB.2017.2665482","article-title":"Construction of refined protein interaction network for predicting essential proteins","volume":"16","author":"Li","year":"2017","journal-title":"IEEE\/ACM Trans Comput Biol Bioinform"},{"key":"2024022301305421300_ref2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s12859-019-3076-y","article-title":"DeepEP: a deep learning framework for identifying essential proteins","volume":"20","author":"Zeng","year":"2019","journal-title":"BMC Bioinform"},{"key":"2024022301305421300_ref3","doi-asserted-by":"crossref","first-page":"4569","DOI":"10.1109\/JBHI.2023.3289490","article-title":"CACO: a core-attachment method with cross-species functional ortholog information to detect human protein complexes","volume":"27","author":"Wang","year":"2023","journal-title":"IEEE J Biomed Health Inform"},{"issue":"6220","key":"2024022301305421300_ref4","doi-asserted-by":"crossref","first-page":"1260419","DOI":"10.1126\/science.1260419","article-title":"Tissue-based map of the human proteome","volume":"347","author":"Uhl\u00e9n","year":"2015","journal-title":"Science"},{"issue":"7403","key":"2024022301305421300_ref5","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1038\/nature11159","article-title":"Large-scale prediction and testing of drug activity on side-effect targets","volume":"486","author":"Lounkine","year":"2012","journal-title":"Nature"},{"issue":"D1","key":"2024022301305421300_ref6","doi-asserted-by":"crossref","first-page":"D506","DOI":"10.1093\/nar\/gky1049","article-title":"UniProt: a worldwide hub of protein knowledge","volume":"47","author":"UniProt Consortium","year":"2019","journal-title":"Nucleic Acids Res"},{"key":"2024022301305421300_ref7","doi-asserted-by":"crossref","first-page":"225","DOI":"10.1007\/978-3-319-41279-5_7","article-title":"A survey of computational methods for protein function prediction","author":"Shehu","year":"2016","journal-title":"Big Data Anal Genom"},{"issue":"D1","key":"2024022301305421300_ref8","doi-asserted-by":"crossref","first-page":"D605","DOI":"10.1093\/nar\/gkaa1074","article-title":"The string database in 2021: customizable protein\u2013protein networks, and functional characterization of user-uploaded gene\/measurement sets","volume":"49","author":"Szklarczyk","year":"2021","journal-title":"Nucleic Acids Res"},{"issue":"10","key":"2024022301305421300_ref9","first-page":"517","article-title":"Searching the MEDLINE literature database through PubMed: a short guide","volume":"28","author":"Motschall","year":"2005","journal-title":"Oncologie"},{"key":"2024022301305421300_ref10","doi-asserted-by":"crossref","first-page":"627","DOI":"10.1007\/978-1-4939-7000-1_26","article-title":"Protein Data Bank (PDB): the single global macromolecular structure archive","author":"Burley","year":"2017","journal-title":"Methods Mol Biol"},{"issue":"7873","key":"2024022301305421300_ref11","doi-asserted-by":"crossref","first-page":"583","DOI":"10.1038\/s41586-021-03819-2","article-title":"Highly accurate protein structure prediction with alphafold","volume":"596","author":"Jumper","year":"2021","journal-title":"Nature"},{"issue":"1","key":"2024022301305421300_ref12","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1038\/75556","article-title":"Gene Ontology: tool for the unification of biology","volume":"25","author":"Ashburner","year":"2000","journal-title":"Nat Genet"},{"issue":"4","key":"2024022301305421300_ref13","doi-asserted-by":"crossref","first-page":"660","DOI":"10.1093\/bioinformatics\/btx624","article-title":"DeepGO: predicting protein functions from sequence and interactions using a deep ontology-aware classifier","volume":"34","author":"Kulmanov","year":"2018","journal-title":"Bioinformatics"},{"issue":"2","key":"2024022301305421300_ref14","doi-asserted-by":"crossref","first-page":"422","DOI":"10.1093\/bioinformatics\/btz595","article-title":"DeepGOplus: improved protein function prediction from sequence","volume":"36","author":"Kulmanov","year":"2020","journal-title":"Bioinformatics"},{"issue":"1","key":"2024022301305421300_ref15","doi-asserted-by":"crossref","first-page":"2351","DOI":"10.1038\/s41467-023-37896-w","article-title":"Sequence-structure-function relationships in the microbial protein universe","volume":"14","author":"Koehler Leman","year":"2023","journal-title":"Nat Commun"},{"issue":"22","key":"2024022301305421300_ref16","doi-asserted-by":"crossref","first-page":"2628","DOI":"10.1093\/bioinformatics\/btn486","article-title":"Gosling: a rule-based protein annotator using blast and go","volume":"24","author":"Jones","year":"2008","journal-title":"Bioinformatics"},{"issue":"4","key":"2024022301305421300_ref17","doi-asserted-by":"crossref","first-page":"366","DOI":"10.1038\/s41592-021-01101-x","article-title":"Sensitive protein alignments at tree-of-life scale using diamond","volume":"18","author":"Buchfink","year":"2021","journal-title":"Nat Methods"},{"issue":"12","key":"2024022301305421300_ref18","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pcbi.1010793","article-title":"Integrating unsupervised language model with triplet neural networks for protein Gene Ontology prediction","volume":"18","author":"Zhu","year":"2022","journal-title":"PLoS Comput Biol"},{"issue":"6","key":"2024022301305421300_ref19","doi-asserted-by":"crossref","first-page":"2208","DOI":"10.1109\/TCBB.2020.2968882","article-title":"A deep learning framework for Gene Ontology annotations with sequence-and network-based information","volume":"18","author":"Zhang","year":"2020","journal-title":"IEEE\/ACM Trans Comput Biol Bioinform"},{"issue":"7553","key":"2024022301305421300_ref20","first-page":"436","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Deep Learn Nat"},{"key":"2024022301305421300_ref21","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.neucom.2016.12.038","article-title":"A survey of deep neural network architectures and their applications","volume":"234","author":"Liu","year":"2017","journal-title":"Neurocomputing"},{"key":"2024022301305421300_ref22","article-title":"Attention is all you need","volume":"30","author":"Vaswani","year":"2017","journal-title":"Adv Neural Inform Process Syst"},{"issue":"18","key":"2024022301305421300_ref23","doi-asserted-by":"crossref","first-page":"2825","DOI":"10.1093\/bioinformatics\/btab198","article-title":"Tale: transformer-based protein function annotation with joint sequence\u2013label embedding","volume":"37","author":"Cao","year":"2021","journal-title":"Bioinformatics"},{"key":"2024022301305421300_ref24","doi-asserted-by":"crossref","first-page":"i238","DOI":"10.1093\/bioinformatics\/btac256","article-title":"DeepGOZero: improving protein function prediction from sequence and zero-shot learning based on ontology axioms","volume":"38","author":"Kulmanov","year":"2022","journal-title":"Bioinformatics"},{"key":"2024022301305421300_ref25","doi-asserted-by":"crossref","DOI":"10.24963\/ijcai.2019\/845","article-title":"EL embeddings: geometric construction of models for the description logic EL ++.","author":"Kulmanov","year":"2019"},{"issue":"15","key":"2024022301305421300_ref26","doi-asserted-by":"crossref","DOI":"10.1073\/pnas.2016239118","article-title":"Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences","volume":"118","author":"Rives","year":"2021","journal-title":"Proc Natl Acad Sci"},{"issue":"1","key":"2024022301305421300_ref27","doi-asserted-by":"crossref","first-page":"3168","DOI":"10.1038\/s41467-021-23303-9","article-title":"Structure-based protein function prediction using graph convolutional networks","volume":"12","author":"Gligorijevi\u0107","year":"2021","journal-title":"Nat Commun"},{"key":"2024022301305421300_ref28","article-title":"Semi-supervised classification with graph convolutional networks.","author":"Kipf","year":"2016"},{"key":"2024022301305421300_ref29","article-title":"Generating sequences with recurrent neural networks.","author":"Graves","year":"2013"},{"key":"2024022301305421300_ref30","doi-asserted-by":"crossref","DOI":"10.1093\/bib\/bbab502","article-title":"Accurate protein function prediction via graph attention networks with predicted structure information","volume":"23","author":"Lai","year":"2022","journal-title":"Brief Bioinform"},{"key":"2024022301305421300_ref31","article-title":"Graph attention networks.","author":"Veli\u010dkovi\u0107","year":"2017"},{"issue":"7","key":"2024022301305421300_ref32","doi-asserted-by":"crossref","first-page":"601","DOI":"10.1038\/s42256-021-00348-5","article-title":"Improved protein structure prediction by deep learning irrespective of co-evolution information","volume":"3","author":"Xu","year":"2021","journal-title":"Nat Mach Intell"},{"issue":"4","key":"2024022301305421300_ref33","doi-asserted-by":"crossref","first-page":"386","DOI":"10.1109\/TNB.2012.2210907","article-title":"Identification of hierarchical and overlapping functional modules in PPI networks","volume":"11","author":"Wang","year":"2012","journal-title":"IEEE Trans Nanobiosci"},{"key":"2024022301305421300_ref34","first-page":"1","article-title":"Protein interaction networks: centrality, modularity, dynamics, and applications","volume":"15","author":"Meng","year":"2021","journal-title":"Front Comp Sci"},{"issue":"1","key":"2024022301305421300_ref35","first-page":"1","article-title":"RF-PSSM: a combination of rotation forest algorithm and position-specific scoring matrix for improved prediction of protein-protein interactions between hepatitis C virus and human","volume":"6","author":"Liu","year":"2022","journal-title":"Big Data Mining Anal"},{"issue":"2","key":"2024022301305421300_ref36","doi-asserted-by":"crossref","first-page":"360","DOI":"10.1109\/TCBB.2015.2394314","article-title":"Predicting protein functions by using unbalanced random walk algorithm on three biological networks","volume":"14","author":"Peng","year":"2015","journal-title":"IEEE\/ACM Trans Comput Biol Bioinform"},{"key":"2024022301305421300_ref37","doi-asserted-by":"crossref","first-page":"701","DOI":"10.1145\/2623330.2623732","article-title":"DeepWalk: online learning of social representations","volume-title":"Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","author":"Perozzi","year":"2014"},{"issue":"5\u20136","key":"2024022301305421300_ref38","doi-asserted-by":"crossref","first-page":"602","DOI":"10.1016\/j.neunet.2005.06.042","article-title":"Framewise phoneme classification with bidirectional LSTM and other neural network architectures","volume":"18","author":"Graves","year":"2005","journal-title":"Neural Netw"},{"issue":"9","key":"2024022301305421300_ref39","doi-asserted-by":"crossref","first-page":"1236","DOI":"10.1093\/bioinformatics\/btu031","article-title":"Interproscan 5: genome-scale protein function classification","volume":"30","author":"Jones","year":"2014","journal-title":"Bioinformatics"},{"issue":"1","key":"2024022301305421300_ref40","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1109\/TNNLS.2020.2978386","article-title":"A comprehensive survey on graph neural networks","volume":"32","author":"Wu","year":"2020","journal-title":"IEEE Trans Neural Netw Learn Syst"},{"key":"2024022301305421300_ref41","doi-asserted-by":"crossref","first-page":"i262","DOI":"10.1093\/bioinformatics\/btab270","article-title":"DeepGraphGO: graph neural network for large-scale, multispecies protein function prediction","volume":"37","author":"You","year":"2021","journal-title":"Bioinformatics"},{"issue":"16","key":"2024022301305421300_ref42","doi-asserted-by":"crossref","first-page":"2414","DOI":"10.1093\/bioinformatics\/btab098","article-title":"NetQuilt: deep multispecies network-based protein function prediction using homology-informed network similarity","volume":"37","author":"Barot","year":"2021","journal-title":"Bioinformatics"},{"issue":"12","key":"2024022301305421300_ref43","doi-asserted-by":"crossref","first-page":"i253","DOI":"10.1093\/bioinformatics\/btp203","article-title":"IsoRankN: spectral methods for global alignment of multiple protein networks","volume":"25","author":"Liao","year":"2009","journal-title":"Bioinformatics"},{"issue":"12","key":"2024022301305421300_ref44","doi-asserted-by":"crossref","first-page":"1050","DOI":"10.1038\/s42256-021-00419-7","article-title":"Protein function prediction for newly sequenced organisms","volume":"3","author":"Torres","year":"2021","journal-title":"Nat Mach Intell"},{"key":"2024022301305421300_ref45","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/j.ymeth.2018.05.026","article-title":"DeepText2GO: improving large-scale protein function prediction with deep semantic text representation","volume":"145","author":"You","year":"2018","journal-title":"Methods"},{"key":"2024022301305421300_ref46","first-page":"1188","article-title":"Distributed representations of sentences and documents","volume-title":"International Conference on Machine Learning","author":"Le","year":"2014"},{"issue":"14","key":"2024022301305421300_ref47","doi-asserted-by":"crossref","first-page":"2465","DOI":"10.1093\/bioinformatics\/bty130","article-title":"GOLabeler: improving sequence-based large-scale protein function prediction by learning to rank","volume":"34","author":"You","year":"2018","journal-title":"Bioinformatics"},{"issue":"10","key":"2024022301305421300_ref48","doi-asserted-by":"crossref","first-page":"1854","DOI":"10.1587\/transinf.E94.D.1854","article-title":"A short introduction to learning to rank","volume":"E94-D","author":"Li","year":"2011","journal-title":"IEICE Trans Inform Syst"},{"issue":"21","key":"2024022301305421300_ref49","doi-asserted-by":"crossref","first-page":"3429","DOI":"10.1093\/bioinformatics\/btv345","article-title":"ProFET: feature engineering captures high-level protein functions","volume":"31","author":"Ofer","year":"2015","journal-title":"Bioinformatics"},{"issue":"W1","key":"2024022301305421300_ref50","doi-asserted-by":"crossref","first-page":"W379","DOI":"10.1093\/nar\/gkz388","article-title":"NetGO: improving large-scale protein function prediction with massive network information","volume":"47","author":"You","year":"2019","journal-title":"Nucleic Acids Res"},{"issue":"W1","key":"2024022301305421300_ref51","doi-asserted-by":"crossref","first-page":"W469","DOI":"10.1093\/nar\/gkab398","article-title":"NetGO 2.0: improving large-scale protein function prediction with massive sequence, text, domain, family and network information","volume":"49","author":"Yao","year":"2021","journal-title":"Nucleic Acids Res"},{"key":"2024022301305421300_ref52","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1016\/j.gpb.2023.04.001","article-title":"NetGO 3.0: protein language model improves large-scale functional annotations","volume":"21","author":"Wang","year":"2023","journal-title":"Genom Proteom Bioinform"},{"issue":"1","key":"2024022301305421300_ref53","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s13059-019-1835-8","article-title":"The CAFA challenge reports improved protein function prediction and new functional annotations for hundreds of genes through experimental screens","volume":"20","author":"Zhou","year":"2019","journal-title":"Genome Biol"},{"key":"2024022301305421300_ref54","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1007\/978-1-4939-3167-5_2","article-title":"UniProtKB\/Swiss-Prot, the manually annotated section of the UniProt knowledgebase: how to use the entry view","author":"Boutet","year":"2016","journal-title":"Methods Mol Biol"},{"issue":"D1","key":"2024022301305421300_ref55","doi-asserted-by":"crossref","first-page":"D309","DOI":"10.1093\/nar\/gky1085","article-title":"eggNOG 5.0: a hierarchical, functionally and phylogenetically annotated orthology resource based on 5090 organisms and 2502 viruses","volume":"47","author":"Huerta-Cepas","year":"2019","journal-title":"Nucleic Acids Res"},{"key":"2024022301305421300_ref56","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1007\/s00018-014-1661-9","article-title":"Exceptionally abundant exceptions: comprehensive characterization of intrinsic disorder in all domains of life","volume":"72","author":"Peng","year":"2015","journal-title":"Cell Mol Life Sci"},{"issue":"10","key":"2024022301305421300_ref57","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pcbi.1010668","article-title":"DMFpred: predicting protein disorder molecular functions based on protein cubic language model","volume":"18","author":"Pang","year":"2022","journal-title":"PLoS Comput Biol"}],"container-title":["Briefings in Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/25\/2\/bbae050\/56742374\/bbae050.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/25\/2\/bbae050\/56742374\/bbae050.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,2,23]],"date-time":"2024-02-23T01:32:03Z","timestamp":1708651923000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bib\/article\/doi\/10.1093\/bib\/bbae050\/7611938"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,22]]},"references-count":57,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,1,22]]}},"URL":"https:\/\/doi.org\/10.1093\/bib\/bbae050","relation":{},"ISSN":["1467-5463","1477-4054"],"issn-type":[{"value":"1467-5463","type":"print"},{"value":"1477-4054","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,3,1]]},"published":{"date-parts":[[2024,1,22]]},"article-number":"bbae050"}}