{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T17:08:56Z","timestamp":1791911336814,"version":"4.3.4"},"reference-count":45,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,2,11]],"date-time":"2022-02-11T00:00:00Z","timestamp":1644537600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,2,11]],"date-time":"2022-02-11T00:00:00Z","timestamp":1644537600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100012659","name":"Foundation for Innovative Research Groups of the National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["No. 61972251"],"award-info":[{"award-number":["No. 61972251"]}],"id":[{"id":"10.13039\/501100012659","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012659","name":"Foundation for Innovative Research Groups of the National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["No. 61972251"],"award-info":[{"award-number":["No. 61972251"]}],"id":[{"id":"10.13039\/501100012659","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Bioinformatics"],"published-print":{"date-parts":[[2022,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Background<\/jats:title><jats:p>For the past decades, benefitting from the rapid growth of protein sequence data in public databases, a lot of machine learning methods have been developed to predict physicochemical properties or functions of proteins using amino acid sequence features. However, the prediction performance often suffers from the lack of labeled data. In recent years, pre-training methods have been widely studied to address the small-sample issue in computer vision and natural language processing fields, while specific pre-training techniques for protein sequences are few.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>In this paper, we propose a pre-training platform for representing protein sequences, called ProtPlat, which uses the Pfam database to train a three-layer neural network, and then uses specific training data from downstream tasks to fine-tune the model. ProtPlat can learn good representations for amino acids, and at the same time achieve efficient classification. We conduct experiments on three protein classification tasks, including the identification of type III secreted effectors, the prediction of subcellular localization, and the recognition of signal peptides. The experimental results show that the pre-training can enhance model performance effectively and ProtPlat is competitive to the state-of-the-art predictors, especially for small datasets. We implement the ProtPlat platform as a web service (<jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/compbio.sjtu.edu.cn\/protplat\">https:\/\/compbio.sjtu.edu.cn\/protplat<\/jats:ext-link>) that is accessible to the public.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>To enhance the feature representation of protein amino acid sequences and improve the performance of sequence-based classification tasks, we develop ProtPlat, a general platform for the pre-training of protein sequences, which is featured by a large-scale supervised training based on Pfam database and an efficient learning model, FastText. The experimental results of three downstream classification tasks demonstrate the efficacy of ProtPlat.<\/jats:p><\/jats:sec>","DOI":"10.1186\/s12859-022-04604-2","type":"journal-article","created":{"date-parts":[[2022,2,11]],"date-time":"2022-02-11T09:03:23Z","timestamp":1644570203000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["ProtPlat: an efficient pre-training platform for protein classification based on FastText"],"prefix":"10.1186","volume":"23","author":[{"given":"Yuan","family":"Jin","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yang","family":"Yang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,2,11]]},"reference":[{"issue":"90001","key":"4604_CR1","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1093\/nar\/gkh131","volume":"32","author":"R Apweiler","year":"2004","unstructured":"Apweiler R, Bairoch AM, Wu CH, Barker WC, Boeckmann B, Ferro S, Gasteiger E, Huang H, Lopez R, Magrane M, et al. Uniprot: the universal protein knowledgebase. Nucleic Acids Res. 2004;32(90001):115\u20139.","journal-title":"Nucleic Acids Res"},{"key":"4604_CR2","doi-asserted-by":"publisher","first-page":"222","DOI":"10.1093\/nar\/gkt1223","volume":"42","author":"RD Finn","year":"2014","unstructured":"Finn RD, Bateman A, Clements J, Coggill P, Eberhardt RY, Eddy SR, Heger A, Hetherington K, Holm L, Mistry J, et al. Pfam: the protein families database. Nucleic Acids Res. 2014;42:222\u201330.","journal-title":"Nucleic Acids Res"},{"key":"4604_CR3","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.ymeth.2019.06.006","volume":"166","author":"W Wang","year":"2019","unstructured":"Wang W, Gao X. Deep learning in bioinformatics. Methods. 2019;166:1\u20133.","journal-title":"Methods"},{"issue":"21","key":"4604_CR4","doi-asserted-by":"publisher","first-page":"3387","DOI":"10.1093\/bioinformatics\/btx431","volume":"33","author":"JJ Almagro Armenteros","year":"2017","unstructured":"Almagro Armenteros JJ, S\u00f8nderby CK, S\u00f8nderby SK, et al. DeepLoc: prediction of protein subcellular localization using deep learning. Bioinformatics. 2017;33(21):3387\u201395.","journal-title":"Bioinformatics"},{"key":"4604_CR5","unstructured":"Mikolov T, Sutskever I, Chen K, Corrado G, Dean J. Distributed representations of words and phrases and their compositionality. 2013; 3111\u20133119"},{"key":"4604_CR6","doi-asserted-by":"crossref","unstructured":"Pennington J, Socher R, Manning CD. Glove: global vectors for word representation. 2014;1532\u20131543","DOI":"10.3115\/v1\/D14-1162"},{"issue":"11","key":"4604_CR7","doi-asserted-by":"publisher","first-page":"0141287","DOI":"10.1371\/journal.pone.0141287","volume":"10","author":"E Asgari","year":"2015","unstructured":"Asgari E, Mofrad MR. Continuous distributed representation of biological sequences for deep proteomics and genomics. PLoS ONE. 2015;10(11):0141287.","journal-title":"PLoS ONE"},{"issue":"1","key":"4604_CR8","doi-asserted-by":"publisher","first-page":"723","DOI":"10.1186\/s12859-019-3220-8","volume":"20","author":"M Heinzinger","year":"2019","unstructured":"Heinzinger M, Elnaggar A, Wang Y, Dallago C, Nechaev D, Matthes F, Rost B. Modeling aspects of the language of life through transfer-learning protein sequences. BMC Bioinform. 2019;20(1):723. https:\/\/doi.org\/10.1186\/s12859-019-3220-8.","journal-title":"BMC Bioinform"},{"issue":"12","key":"4604_CR9","doi-asserted-by":"publisher","first-page":"2009","DOI":"10.1093\/bioinformatics\/bty937","volume":"35","author":"M Hamid","year":"2019","unstructured":"Hamid M, Friedberg I. Identifying antimicrobial peptides using word embedding with deep recurrent neural networks. Bioinformatics. 2019;35(12):2009\u201316.","journal-title":"Bioinformatics"},{"key":"4604_CR10","doi-asserted-by":"crossref","unstructured":"Peters ME, Neumann M, Iyyer M, Gardner M, Clark C, Lee K, Zettlemoyer L. Deep contextualized word representations. arXiv preprint arXiv:1802.05365 (2018)","DOI":"10.18653\/v1\/N18-1202"},{"key":"4604_CR11","unstructured":"Devlin J, Chang M-W, Lee K, Toutanova K. Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)"},{"key":"4604_CR12","doi-asserted-by":"crossref","unstructured":"Joulin A, Grave E, Bojanowski P, et al. Bag of tricks for efficient text classification. arXiv preprint arXiv:1607.01759, 2016.","DOI":"10.18653\/v1\/E17-2068"},{"issue":"4","key":"4604_CR13","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s40484-019-0184-7","volume":"7","author":"XF Fu","year":"2019","unstructured":"Fu XF, Yang Y. WeDeepT3: predicting type iii secreted effectors based on word embedding and deep learning. Quantitative Biology. 2019;7(4):1\u20139. https:\/\/doi.org\/10.1007\/s40484-019-0184-7.","journal-title":"Quantitative Biology"},{"issue":"14","key":"4604_CR14","first-page":"415","volume":"22","author":"A Pierleoni","year":"2006","unstructured":"Pierleoni A, Martelli PL, Fariselli P, Casadio R. Bacello: a balanced subcellular localization predictor. Intell Syst Mol Biol. 2006;22(14):415\u20136.","journal-title":"Intell Syst Mol Biol"},{"issue":"4","key":"4604_CR15","doi-asserted-by":"publisher","first-page":"420","DOI":"10.1038\/s41587-019-0036-z","volume":"37","author":"JJA Armenteros","year":"2019","unstructured":"Armenteros JJA, Tsirigos KD, Sonderby CK, Petersen TN, Winther O, Brunak S, Von Heijne G, Nielsen H. Signalp 50 improves signal peptide predictions using deep neural networks. Nat Biotechnol. 2019;37(4):420\u20133.","journal-title":"Nat Biotechnol"},{"key":"4604_CR16","unstructured":"Mikolov T, Chen K, Corrado G, Dean J. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781;2013"},{"issue":"6","key":"4604_CR17","doi-asserted-by":"publisher","first-page":"777","DOI":"10.1093\/bioinformatics\/btr021","volume":"27","author":"Y Wang","year":"2011","unstructured":"Wang Y, Zhang Q, Sun M-A, Guo D. High-accuracy prediction of bacterial type iii secreted effectors based on position-specific amino acid composition profiles. Bioinformatics. 2011;27(6):777\u201384.","journal-title":"Bioinformatics"},{"issue":"4","key":"4604_CR18","doi-asserted-by":"publisher","first-page":"1000376","DOI":"10.1371\/journal.ppat.1000376","volume":"5","author":"R Arnold","year":"2009","unstructured":"Arnold R, Brandmeier S, Kleine F, Tischler P, Heinz E, Behrens S, Niinikoski A, Mewes H-W, Horn M, Rattei T. Sequence-based prediction of type iii secreted proteins. PLoS Pathog. 2009;5(4):1000376.","journal-title":"PLoS Pathog"},{"issue":"3","key":"4604_CR19","doi-asserted-by":"publisher","first-page":"58173","DOI":"10.1371\/journal.pone.0058173","volume":"8","author":"Y Wang","year":"2013","unstructured":"Wang Y, Bao H, White AP, et al. T3_mm: a markov model effectively classifies bacterial type iii secretion signals. PLoS ONE. 2013;8(3):58173.","journal-title":"PLoS ONE"},{"issue":"12","key":"4604_CR20","doi-asserted-by":"publisher","first-page":"2051","DOI":"10.1093\/bioinformatics\/bty931","volume":"35","author":"L Xue","year":"2019","unstructured":"Xue L, Tang B, Chen W, Luo J. Deept3: deep convolutional neural networks accurately identify gram-negative bacterial type iii secreted effectors using the n-terminal sequence. Bioinformatics. 2019;35(12):2051\u20137.","journal-title":"Bioinformatics"},{"issue":"12","key":"4604_CR21","doi-asserted-by":"publisher","first-page":"2017","DOI":"10.1093\/bioinformatics\/bty914","volume":"35","author":"J Wang","year":"2019","unstructured":"Wang J, Li J, Yang B, Xie R, Marquez-Lago TT, Leier A, Hayashida M, Akutsu T, Zhang Y, Chou K-C, et al. Bastion3: a two-layer ensemble predictor of type iii secreted effectors. Bioinformatics. 2019;35(12):2017\u201328.","journal-title":"Bioinformatics"},{"key":"4604_CR22","doi-asserted-by":"crossref","unstructured":"Dong X, Lu X, Zhang Z. BEAN 2.0: an integrated web resource for the identification and functional analysis of type III secreted effectors. Database. 2015;2015.","DOI":"10.1093\/database\/bav064"},{"issue":"1","key":"4604_CR23","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1016\/j.ygeno.2017.08.005","volume":"110","author":"X Cheng","year":"2018","unstructured":"Cheng X, Xiao X, Chou KC. pLoc-mEuk: predict subcellular localization of multi-label eukaryotic proteins by extracting the key GO information into general PseAAC. Genomics. 2018;110(1):50\u20138.","journal-title":"Genomics"},{"issue":"1","key":"4604_CR24","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1016\/j.jmb.2005.02.025","volume":"348","author":"R Nair","year":"2005","unstructured":"Nair R, Rost B. Mimicking cellular sorting improves prediction of subcellular localization. J Mol Biol. 2005;348(1):85\u2013100.","journal-title":"J Mol Biol"},{"issue":"Web Server issu","key":"4604_CR25","doi-asserted-by":"publisher","first-page":"497","DOI":"10.1093\/nar\/gkq477","volume":"38","author":"S Briesemeister","year":"2010","unstructured":"Briesemeister S, Rahnenf\u00fchrer J, Kohlbacher O. Yloc\u2014an interpretable web server for predicting subcellular localization. Nucleic Acids Res. 2010;38(Web Server issue):497\u2013502.","journal-title":"Nucleic Acids Res"},{"issue":"6","key":"4604_CR26","doi-asserted-by":"publisher","first-page":"2773","DOI":"10.1021\/acs.jcim.0c00073","volume":"60","author":"Y Xu","year":"2020","unstructured":"Xu Y, Verma D, Sheridan RP, et al. Deep dive into machine learning models for protein engineering. J Chem Inf Model. 2020;60(6):2773\u201390.","journal-title":"J Chem Inf Model"},{"key":"4604_CR27","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2021.3095381","author":"A Elnaggar","year":"2021","unstructured":"Elnaggar A, et al. ProtTrans: towards cracking the language of lifes code through self-supervised deep learning and high performance computing. IEEE Trans Pattern Anal Mach Intell. 2021. https:\/\/doi.org\/10.1109\/TPAMI.2021.3095381.","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"issue":"3","key":"4604_CR28","doi-asserted-by":"publisher","first-page":"272","DOI":"10.1016\/j.jtbi.2010.09.007","volume":"267","author":"T Liu","year":"2010","unstructured":"Liu T, Jia C. A high-accuracy protein structural class prediction algorithm using predicted secondary structural information. J Theor Biol. 2010;267(3):272\u20135. https:\/\/doi.org\/10.1016\/j.jtbi.2010.09.007.","journal-title":"J Theor Biol"},{"issue":"4","key":"4604_CR29","doi-asserted-by":"publisher","first-page":"618","DOI":"10.1016\/j.jtbi.2008.12.027","volume":"257","author":"J-Y Yang","year":"2009","unstructured":"Yang J-Y, Peng Z-L, Yu Z-G, Zhang R-J, Anh V, Wang D. Prediction of protein structural classes by recurrence quantification analysis based on chaos game representation. J Theor Biol. 2009;257(4):618\u201326. https:\/\/doi.org\/10.1016\/j.jtbi.2008.12.027.","journal-title":"J Theor Biol"},{"key":"4604_CR30","first-page":"531","volume":"2005","author":"Y Qi","year":"2005","unstructured":"Qi Y, Klein-Seetharaman J, Bar-Joseph Z. Random forest similarity for protein-protein interaction prediction from multiple sources. Biocomputing. 2005;2005:531\u201342.","journal-title":"Biocomputing"},{"issue":"3","key":"4604_CR31","doi-asserted-by":"publisher","first-page":"490","DOI":"10.1002\/prot.20865","volume":"63","author":"Y Qi","year":"2006","unstructured":"Qi Y, Bar-Joseph Z, Klein-Seetharaman J. Evaluation of different biological data and computational classification methods for use in protein interaction prediction. Proteins Struct Funct Bioinform. 2006;63(3):490\u2013500. https:\/\/doi.org\/10.1002\/prot.20865.","journal-title":"Proteins Struct Funct Bioinform"},{"key":"4604_CR32","doi-asserted-by":"crossref","unstructured":"Shen Z, Zhang Q, Han K, et al. A deep learning model for RNA-protein binding preference prediction based on hierarchical LSTM and attention network. IEEE\/ACM Trans Comput Biol Bioinforma. 2020;","DOI":"10.1109\/TCBB.2020.3007544"},{"key":"4604_CR33","doi-asserted-by":"crossref","unstructured":"Ofer D, Brandes N, Linial M. The language of proteins: NLP, machine learning & protein sequences. Comput Struct Biotechnol J. 2021;","DOI":"10.1016\/j.csbj.2021.03.022"},{"issue":"15","key":"4604_CR34","doi-asserted-by":"publisher","first-page":"2642","DOI":"10.1093\/bioinformatics\/bty178","volume":"34","author":"KK Yang","year":"2018","unstructured":"Yang KK, Wu Z, Bedbrook CN, et al. Learned protein embeddings for machine learning. Bioinformatics. 2018;34(15):2642\u20138.","journal-title":"Bioinformatics"},{"key":"4604_CR35","unstructured":"Filipavicius M, Manica M, Cadow J et al. Pre-training protein language models with label-agnostic binding pairs enhances performance in downstream tasks. arXiv preprint arXiv:2012.03084; 2020."},{"key":"4604_CR36","unstructured":"Min S, Park S, Kim S, Choi HS, Yoon S (2019) Pre-training of deep bidirectional protein sequence representations with structural information. arXiv preprint arXiv:1912.05625."},{"issue":"1","key":"4604_CR37","doi-asserted-by":"publisher","first-page":"274","DOI":"10.1186\/1471-2105-10-274","volume":"10","author":"T Blum","year":"2009","unstructured":"Blum T, Briesemeister S, Kohlbacher O. MultiLoc2: integrating phylogeny and gene ontology terms improves subcellular protein localization prediction. BMC Bioinform. 2009;10(1):274.","journal-title":"BMC Bioinform"},{"issue":"1","key":"4604_CR38","first-page":"585","volume":"35","author":"H Paul","year":"2007","unstructured":"Paul H, Keun-Joon P, Takeshi O, et al. WoLF PSORT: protein localization predictor. Nucleic Acids Res. 2007;35(1):585\u20137.","journal-title":"Nucleic Acids Res"},{"issue":"4","key":"4604_CR39","doi-asserted-by":"publisher","first-page":"1280","DOI":"10.1093\/bib\/bbx165","volume":"20","author":"B Liu","year":"2019","unstructured":"Liu B. BioSeq-Analysis: a platform for DNA, RNA and protein sequence analysis based on machine learning approaches. Brief Bioinform. 2019;20(4):1280\u201394. https:\/\/doi.org\/10.1093\/bib\/bbx165.","journal-title":"Brief Bioinform"},{"key":"4604_CR40","unstructured":"Han S, Mao H, Dally WJ. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149; 2015."},{"key":"4604_CR41","first-page":"25","volume":"3345","author":"M Ganapathiraju","year":"2005","unstructured":"Ganapathiraju M, et al. Computational biology and language, ambient intelligence for scientific discovery. LNAI. 2005;3345:25\u201347.","journal-title":"LNAI"},{"key":"4604_CR42","doi-asserted-by":"crossref","first-page":"843","DOI":"10.1093\/bioinformatics\/btw723","volume":"33","author":"H Zhou","year":"2017","unstructured":"Zhou H, Yang Y, Shen HB. Hum-mPLoc 3.0: prediction enhancement of human protein subcellular localization through modeling the hidden correlations of gene ontology and functional domain features. Bioinformatics. 2017;33:843\u201353.","journal-title":"Bioinformatics"},{"issue":"1","key":"4604_CR43","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1142\/S0129065710002206","volume":"20","author":"Y Yang","year":"2010","unstructured":"Yang Y, Lu BL. Prediction of protein subcellular multi-localization by using a min-max modular support vector machine. Int J Neural Syst. 2010;20(1):13\u201328.","journal-title":"Int J Neural Syst"},{"key":"4604_CR44","doi-asserted-by":"publisher","first-page":"i458","DOI":"10.1093\/bioinformatics\/bts390","volume":"28","author":"T Goldberg","year":"2012","unstructured":"Goldberg T, et al. LocTree2 predicts localization for all domains of life. Bioinformatics. 2012;28:i458\u201365.","journal-title":"Bioinformatics"},{"issue":"17","key":"4604_CR45","doi-asserted-by":"publisher","first-page":"3389","DOI":"10.1093\/nar\/25.17.3389","volume":"25","author":"SF Altschul","year":"1997","unstructured":"Altschul SF, Madden TL, Sch\u00e4ffer AA, Zhang J, Zhang Z, Miller W, Lipman DJ. Gapped BLAST and PSI-BLAST: a new generation of protein database search programs. Nucleic Acids Res. 1997;25(17):3389\u2013402.","journal-title":"Nucleic Acids Res"}],"container-title":["BMC Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-022-04604-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12859-022-04604-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-022-04604-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,17]],"date-time":"2023-11-17T09:35:54Z","timestamp":1700213754000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/s12859-022-04604-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,2,11]]},"references-count":45,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,12]]}},"alternative-id":["4604"],"URL":"https:\/\/doi.org\/10.1186\/s12859-022-04604-2","relation":{},"ISSN":["1471-2105"],"issn-type":[{"value":"1471-2105","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,2,11]]},"assertion":[{"value":"13 April 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 February 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 February 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"66"}}