{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,15]],"date-time":"2026-05-15T08:41:36Z","timestamp":1778834496157,"version":"3.51.4"},"reference-count":47,"publisher":"Oxford University Press (OUP)","issue":"20","license":[{"start":{"date-parts":[[2021,5,12]],"date-time":"2021-05-12T00:00:00Z","timestamp":1620777600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"German Ministry for Research and Education"},{"DOI":"10.13039\/501100002347","name":"BMBF","doi-asserted-by":"publisher","award":["031L0168"],"award-info":[{"award-number":["031L0168"]}],"id":[{"id":"10.13039\/501100002347","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002347","name":"BMBF","doi-asserted-by":"publisher","award":["01IS17049"],"award-info":[{"award-number":["01IS17049"]}],"id":[{"id":"10.13039\/501100002347","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["RO1320\/4\u20131"],"award-info":[{"award-number":["RO1320\/4\u20131"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000268","name":"Biotechnology and Biological Sciences Research Council","doi-asserted-by":"publisher","award":["BB\/R009597\/1"],"award-info":[{"award-number":["BB\/R009597\/1"]}],"id":[{"id":"10.13039\/501100000268","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,10,25]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Motivation<\/jats:title>\n                    <jats:p>Classifying proteins into functional families can improve our understanding of protein function and can allow transferring annotations within one family. For this, functional families need to be \u2018pure\u2019, i.e., contain only proteins with identical function. Functional Families (FunFams) cluster proteins within CATH superfamilies into such groups of proteins sharing function. 11% of all FunFams (22 830 of 203 639) contain EC annotations and of those, 7% (1526 of 22 830) have inconsistent functional annotations.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>We propose an approach to further cluster FunFams into functionally more consistent sub-families by encoding their sequences through embeddings. These embeddings originate from language models transferring knowledge gained from predicting missing amino acids in a sequence (ProtBERT) and have been further optimized to distinguish between proteins belonging to the same or a different CATH superfamily (PB-Tucker). Using distances between embeddings and DBSCAN to cluster FunFams and identify outliers, doubled the number of pure clusters per FunFam compared to random clustering. Our approach was not limited to FunFams but also succeeded on families created using sequence similarity alone. Complementing EC annotations, we observed similar results for binding annotations. Thus, we expect an increased purity also for other aspects of function. Our results can help generating FunFams; the resulting clusters with improved functional consistency allow more reliable inference of annotations. We expect this approach to succeed equally for any other grouping of proteins by their phenotypes.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Availability and implementation<\/jats:title>\n                    <jats:p>Code and embeddings are available via GitHub: https:\/\/github.com\/Rostlab\/FunFamsClustering.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Supplementary information<\/jats:title>\n                    <jats:p>Supplementary data are available at Bioinformatics online.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btab371","type":"journal-article","created":{"date-parts":[[2021,5,11]],"date-time":"2021-05-11T10:31:46Z","timestamp":1620729106000},"page":"3449-3455","source":"Crossref","is-referenced-by-count":27,"title":["Clustering FunFams using sequence embeddings improves EC purity"],"prefix":"10.1093","volume":"37","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8533-8163","authenticated-orcid":false,"given":"Maria","family":"Littmann","sequence":"first","affiliation":[{"name":"Department of Informatics, Bioinformatics & Computational Biology\u2014i12, TUM (Technical University of Munich) , 85748 Garching\/Munich, Germany"},{"name":"Center for Doctoral Studies in Informatics and its Applications (CeDoSIA), TUM Graduate School, Center of Doctoral Studies in Informatics and its Applications (CeDoSIA) , 85748 Garching\/Munich, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6568-9035","authenticated-orcid":false,"given":"Nicola","family":"Bordin","sequence":"additional","affiliation":[{"name":"Institute of Structural and Molecular Biology, University College London , London WC1E 6BT, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michael","family":"Heinzinger","sequence":"additional","affiliation":[{"name":"Department of Informatics, Bioinformatics & Computational Biology\u2014i12, TUM (Technical University of Munich) , 85748 Garching\/Munich, Germany"},{"name":"Center for Doctoral Studies in Informatics and its Applications (CeDoSIA), TUM Graduate School, Center of Doctoral Studies in Informatics and its Applications (CeDoSIA) , 85748 Garching\/Munich, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Konstantin","family":"Sch\u00fctze","sequence":"additional","affiliation":[{"name":"Department of Informatics, Bioinformatics & Computational Biology\u2014i12, TUM (Technical University of Munich) , 85748 Garching\/Munich, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4650-6181","authenticated-orcid":false,"given":"Christian","family":"Dallago","sequence":"additional","affiliation":[{"name":"Department of Informatics, Bioinformatics & Computational Biology\u2014i12, TUM (Technical University of Munich) , 85748 Garching\/Munich, Germany"},{"name":"Center for Doctoral Studies in Informatics and its Applications (CeDoSIA), TUM Graduate School, Center of Doctoral Studies in Informatics and its Applications (CeDoSIA) , 85748 Garching\/Munich, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Christine","family":"Orengo","sequence":"additional","affiliation":[{"name":"Institute of Structural and Molecular Biology, University College London , London WC1E 6BT, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Burkhard","family":"Rost","sequence":"additional","affiliation":[{"name":"Department of Informatics, Bioinformatics & Computational Biology\u2014i12, TUM (Technical University of Munich) , 85748 Garching\/Munich, Germany"},{"name":"Institute for Advanced Study (TUM-IAS) , 85748 Garching\/Munich, Germany"},{"name":"TUM School of Life Sciences Weihenstephan (WZW) , 85354 Freising, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2021,5,12]]},"reference":[{"key":"2023051609043309500_btab371-B1","doi-asserted-by":"crossref","first-page":"1315","DOI":"10.1038\/s41592-019-0598-1","article-title":"Unified rational protein engineering with sequence-based deep representation learning","volume":"16","author":"Alley","year":"2019","journal-title":"Nat. Methods"},{"key":"2023051609043309500_btab371-B2","doi-asserted-by":"crossref","first-page":"e0141287","DOI":"10.1371\/journal.pone.0141287","article-title":"Continuous distributed representation of biological sequences for deep proteomics and genomics","volume":"10","author":"Asgari","year":"2015","journal-title":"PLoS One"},{"key":"2023051609043309500_btab371-B3","article-title":"Neural machine translation by jointly learning to align and translate","author":"Bahdanau","year":"2016"},{"key":"2023051609043309500_btab371-B4","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1038\/355161a0","article-title":"Self-organizing neural network that discovers surfaces in random-dot stereograms","volume":"355","author":"Becker","year":"1992","journal-title":"Nature"},{"key":"2023051609043309500_btab371-B5","first-page":"217","author":"Beyer","year":"1999"},{"key":"2023051609043309500_btab371-B6","first-page":"737","author":"Bromley","year":"1993"},{"key":"2023051609043309500_btab371-B7","doi-asserted-by":"crossref","first-page":"1473","DOI":"10.1093\/bioinformatics\/btn214","article-title":"Characterization and prediction of residues determining protein functional specificity","volume":"24","author":"Capra","year":"2008","journal-title":"Bioinformatics"},{"key":"2023051609043309500_btab371-B8","article-title":"Learned embeddings from deep learning to visualize and predict protein sets","volume":"1, e113","author":"Dallago","year":"2021","journal-title":"Curr. Protoc. Bioinf"},{"key":"2023051609043309500_btab371-B9","doi-asserted-by":"crossref","first-page":"W148","DOI":"10.1093\/nar\/gkv488","article-title":"CATH FunFHMMer web server: protein functional annotations using functional family assignments","volume":"43","author":"Das","year":"2015","journal-title":"Nucleic Acids Res"},{"key":"2023051609043309500_btab371-B10","doi-asserted-by":"crossref","first-page":"2889","DOI":"10.1093\/bioinformatics\/btw473","article-title":"Functional classification of CATH superfamilies: a domain-based approach for protein function annotation","volume":"32","author":"Das","year":"2016","journal-title":"Bioinformatics"},{"key":"2023051609043309500_btab371-B11","doi-asserted-by":"crossref","first-page":"1099","DOI":"10.1093\/bioinformatics\/btaa937","article-title":"CATH functional families predict protein functional sites","volume":"37","author":"Das","year":"2020","journal-title":"Bioinformatics"},{"key":"2023051609043309500_btab371-B12","article-title":"BERT: pre-training of Deep Bidirectional Transformers for Language Understanding","author":"Devlin","year":"2019"},{"key":"2023051609043309500_btab371-B13","article-title":"ProtTrans: towards cracking the language of life\u2019s code through self-supervised deep learning and high performance computing","author":"Elnaggar","year":"2020","journal-title":"bioRxiv"},{"key":"2023051609043309500_btab371-B14","first-page":"226","author":"Ester","year":"1996"},{"key":"2023051609043309500_btab371-B15","doi-asserted-by":"crossref","first-page":"2009","DOI":"10.1093\/bioinformatics\/bty937","article-title":"Identifying antimicrobial peptides using word embedding with deep recurrent neural networks","volume":"35","author":"Hamid","year":"2019","journal-title":"Bioinformatics"},{"key":"2023051609043309500_btab371-B16","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1006\/jmbi.2000.4036","article-title":"Analysis and prediction of functional sub-types from protein sequence alignments","volume":"303","author":"Hannenhalli","year":"2000","journal-title":"J. Mol. Biol"},{"key":"2023051609043309500_btab371-B17","doi-asserted-by":"crossref","first-page":"723","DOI":"10.1186\/s12859-019-3220-8","article-title":"Modeling aspects of the language of life through transfer-learning protein sequences","volume":"20","author":"Heinzinger","year":"2019","journal-title":"BMC Bioinformatics"},{"key":"2023051609043309500_btab371-B18","doi-asserted-by":"crossref","first-page":"415","DOI":"10.1016\/S0168-9525(03)00167-7","article-title":"Moonlighting proteins: old proteins learning new tricks","volume":"19","author":"Jeffery","year":"2003","journal-title":"Trends Genet"},{"key":"2023051609043309500_btab371-B19","doi-asserted-by":"crossref","first-page":"471","DOI":"10.1146\/annurev-biochem-030409-143718","article-title":"Enzyme promiscuity: a mechanistic and evolutionary perspective","volume":"79","author":"Khersonsky","year":"2010","journal-title":"Annu. Rev. Biochem"},{"key":"2023051609043309500_btab371-B20","doi-asserted-by":"crossref","first-page":"193907","DOI":"10.1109\/ACCESS.2020.3031549","article-title":"Contrastive representation learning: a framework and review","volume":"8","author":"Le-Khac","year":"2020","journal-title":"IEEE Access"},{"key":"2023051609043309500_btab371-B21","doi-asserted-by":"crossref","first-page":"720","DOI":"10.1093\/nar\/gkp1049","article-title":"GeMMA: functional subfamily classification within superfamilies of predicted protein structural domains","volume":"38","author":"Lee","year":"2010","journal-title":"Nucleic Acids Res"},{"key":"2023051609043309500_btab371-B22","doi-asserted-by":"crossref","first-page":"1160","DOI":"10.1038\/s41598-020-80786-0","article-title":"Embeddings from deep learning transfer GO annotations beyond homology","volume":"11","author":"Littmann","year":"2021","journal-title":"Sci. Rep"},{"key":"2023051609043309500_btab371-B23","doi-asserted-by":"crossref","first-page":"678","DOI":"10.1002\/prot.20095","article-title":"CHOP proteins into structural domain-like fragments","volume":"55","author":"Liu","year":"2004","journal-title":"Proteins Struct. Funct. Bioinf"},{"key":"2023051609043309500_btab371-B24","doi-asserted-by":"crossref","first-page":"4643","DOI":"10.1093\/bioinformatics\/btaa485","article-title":"UniRule: a unified rule resource for automatic annotation in the UniProt Knowledgebase","volume":"36","author":"MacDougall","year":"2020","journal-title":"Bioinformatics"},{"key":"2023051609043309500_btab371-B25","article-title":"ProGen: language modeling for protein generation","author":"Madani","year":"2020","journal-title":"bioRxiv"},{"key":"2023051609043309500_btab371-B26","doi-asserted-by":"crossref","first-page":"1896","DOI":"10.1093\/bioinformatics\/btz817","article-title":"Enzyme annotation in UniProtKB using Rhea","volume":"36","author":"Morgat","year":"2020","journal-title":"Bioinformatics"},{"key":"2023051609043309500_btab371-B27","doi-asserted-by":"crossref","first-page":"1093","DOI":"10.1016\/S0969-2126(97)00260-8","article-title":"CATH\u2014a hierarchic classification of protein domain structures","volume":"5","author":"Orengo","year":"1997","journal-title":"Structure"},{"key":"2023051609043309500_btab371-B28","first-page":"2825","article-title":"Scikit-learn: machine learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J. Mach. Learn. Res"},{"key":"2023051609043309500_btab371-B29","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/N18-1202","article-title":"Deep contextualized word representations","author":"Peters","year":"2018"},{"key":"2023051609043309500_btab371-B30","article-title":"Improving language understanding by generative pre-training","author":"Radford","year":"2018"},{"key":"2023051609043309500_btab371-B31","doi-asserted-by":"crossref","DOI":"10.1101\/676825","article-title":"Evaluating Protein Transfer Learning with TAPE","author":"Rao","year":"2019"},{"key":"2023051609043309500_btab371-B32","article-title":"Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences","author":"Rives","year":"2020","journal-title":"bioRxiv"},{"key":"2023051609043309500_btab371-B33","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1023\/A:1009745219419","article-title":"Density-based clustering in spatial databases: the algorithm GDBSCAN and its applications","volume":"2","author":"Sander","year":"1998","journal-title":"Data Min. Knowledge Discov"},{"key":"2023051609043309500_btab371-B34","doi-asserted-by":"crossref","first-page":"400","DOI":"10.1186\/s12859-019-2988-x","article-title":"FunFam protein families improve residue level molecular function prediction","volume":"20","author":"Scheibenreif","year":"2019","journal-title":"BMC Bioinformatics"},{"key":"2023051609043309500_btab371-B35","article-title":"CATH: increased structural coverage of functional space","author":"Sillitoe","year":"2020","journal-title":"Nucleic Acids Res"},{"key":"2023051609043309500_btab371-B36","doi-asserted-by":"crossref","first-page":"D490","DOI":"10.1093\/nar\/gks1211","article-title":"New functional families (FunFams) in CATH to improve the mapping of conserved functional sites to 3D structures","volume":"41","author":"Sillitoe","year":"2013","journal-title":"Nucleic Acids Res"},{"key":"2023051609043309500_btab371-B37","doi-asserted-by":"crossref","first-page":"D280","DOI":"10.1093\/nar\/gky1097","article-title":"CATH: expanding the horizons of structure-based functional annotations for genome sequences","volume":"47","author":"Sillitoe","year":"2019","journal-title":"Nucleic Acids Res"},{"key":"2023051609043309500_btab371-B38","doi-asserted-by":"crossref","first-page":"473","DOI":"10.1186\/s12859-019-3019-7","article-title":"HH-suite3 for fast remote homology detection and deep protein annotation","volume":"20","author":"Steinegger","year":"2019","journal-title":"BMC Bioinformatics"},{"key":"2023051609043309500_btab371-B39","doi-asserted-by":"crossref","first-page":"D506","DOI":"10.1093\/nar\/gky1049","article-title":"UniProt: a worldwide hub of protein knowledge","volume":"47","year":"2019","journal-title":"Nucleic Acids Res"},{"key":"2023051609043309500_btab371-B40","doi-asserted-by":"crossref","first-page":"565","DOI":"10.1016\/j.str.2018.02.009","article-title":"Ranking enzyme structures in the PDB by bound ligand similarity to biological substrates","volume":"26","author":"Tyzack","year":"2018","journal-title":"Structure"},{"key":"2023051609043309500_btab371-B41","article-title":"BERTology meets biology: interpreting attention in protein language models","author":"Vig","year":"2020","journal-title":"arXiv"},{"key":"2023051609043309500_btab371-B42","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1093\/bioinformatics\/btaa701","article-title":"Unsupervised protein embeddings outperform hand-crafted sequence and structure features at predicting molecular function","volume":"37","author":"Villegas-Morcillo","year":"2020","journal-title":"Bioinformatics"},{"key":"2023051609043309500_btab371-B43","author":"Webb","year":"1992"},{"key":"2023051609043309500_btab371-B44","first-page":"207","article-title":"Distance metric learning for large margin nearest neighbor classification","volume":"10","author":"Weinberger","year":"2009","journal-title":"J. Mach. Learn. Res"},{"key":"2023051609043309500_btab371-B45","doi-asserted-by":"crossref","first-page":"D1096","DOI":"10.1093\/nar\/gks966","article-title":"BioLiP: a semi-manually curated database for biologically relevant ligand\u2013protein interactions","volume":"41","author":"Yang","year":"2013","journal-title":"Nucleic Acids Res"},{"key":"2023051609043309500_btab371-B46","doi-asserted-by":"crossref","first-page":"4138","DOI":"10.1093\/bioinformatics\/bty455","article-title":"Learned protein embeddings for machine learning","volume":"34","author":"Yang","year":"2018","journal-title":"Bioinformatics"},{"key":"2023051609043309500_btab371-B47","doi-asserted-by":"crossref","first-page":"244","DOI":"10.1186\/s13059-019-1835-8","article-title":"The CAFA challenge reports improved protein function prediction and new functional annotations for hundreds of genes through experimental screens","volume":"20","author":"Zhou","year":"2019","journal-title":"Genome Biol"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btab371\/38576414\/btab371.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/37\/20\/3449\/50338653\/btab371.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/37\/20\/3449\/50338653\/btab371.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,3]],"date-time":"2023-11-03T11:47:34Z","timestamp":1699012054000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/37\/20\/3449\/6274663"}},"subtitle":[],"editor":[{"given":"Pier Luigi","family":"Martelli","sequence":"additional","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2021,5,12]]},"references-count":47,"journal-issue":{"issue":"20","published-print":{"date-parts":[[2021,10,25]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btab371","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2021.01.21.427551","asserted-by":"object"}]},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,10,15]]},"published":{"date-parts":[[2021,5,12]]}}}