{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,29]],"date-time":"2026-07-29T08:23:48Z","timestamp":1785313428395,"version":"3.55.0"},"reference-count":26,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2020,6,9]],"date-time":"2020-06-09T00:00:00Z","timestamp":1591660800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,6,9]],"date-time":"2020-06-09T00:00:00Z","timestamp":1591660800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"crossref","award":["Research Training Group 1743, \u2018Genes, Environment and Inflammation"],"award-info":[{"award-number":["Research Training Group 1743, \u2018Genes, Environment and Inflammation"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"crossref","award":["Research Training Group 1743, \u2018Genes, Environment and Inflammation\u2019"],"award-info":[{"award-number":["Research Training Group 1743, \u2018Genes, Environment and Inflammation\u2019"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100012627","name":"NASA Astrobiology Institute","doi-asserted-by":"publisher","award":["80NSSC18M0093"],"award-info":[{"award-number":["80NSSC18M0093"]}],"id":[{"id":"10.13039\/100012627","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100012627","name":"NASA Astrobiology Institute","doi-asserted-by":"publisher","award":["NASA Astrobiology Postdoctoral Fellowship"],"award-info":[{"award-number":["NASA Astrobiology Postdoctoral Fellowship"]}],"id":[{"id":"10.13039\/100012627","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Bioinformatics"],"published-print":{"date-parts":[[2020,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n<jats:title>Background<\/jats:title>\n<jats:p>The number of applications of deep learning algorithms in bioinformatics is increasing as they usually achieve superior performance over classical approaches, especially, when bigger training datasets are available. In deep learning applications, discrete data, e.g. words or n-grams in language, or amino acids or nucleotides in bioinformatics, are generally represented as a continuous vector through an embedding matrix. Recently, learning this embedding matrix directly from the data as part of the continuous iteration of the model to optimize the target prediction \u2013 a process called \u2018end-to-end learning\u2019 \u2013 has led to state-of-the-art results in many fields. Although usage of embeddings is well described in the bioinformatics literature, the potential of end-to-end learning for single amino acids, as compared to more classical manually-curated encoding strategies, has not been systematically addressed. To this end, we compared classical encoding matrices, namely one-hot, VHSE8 and BLOSUM62, to end-to-end learning of amino acid embeddings for two different prediction tasks using three widely used architectures, namely recurrent neural networks (RNN), convolutional neural networks (CNN), and the hybrid CNN-RNN.<\/jats:p>\n<\/jats:sec><jats:sec>\n<jats:title>Results<\/jats:title>\n<jats:p>By using different deep learning architectures, we show that end-to-end learning is on par with classical encodings for embeddings of the same dimension even when limited training data is available, and might allow for a reduction in the embedding dimension without performance loss, which is critical when deploying the models to devices with limited computational capacities. We found that the embedding dimension is a major factor in controlling the model performance. Surprisingly, we observed that deep learning models are capable of learning from random vectors of appropriate dimension.<\/jats:p>\n<\/jats:sec><jats:sec>\n<jats:title>Conclusion<\/jats:title>\n<jats:p>Our study shows that end-to-end learning is a flexible and powerful method for amino acid encoding. Further, due to the flexibility of deep learning systems, amino acid encoding schemes should be benchmarked against random vectors of the same dimension to disentangle the information content provided by the encoding scheme from the distinguishability effect provided by the scheme.<\/jats:p>\n<\/jats:sec>","DOI":"10.1186\/s12859-020-03546-x","type":"journal-article","created":{"date-parts":[[2020,6,9]],"date-time":"2020-06-09T12:03:02Z","timestamp":1591704182000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":97,"title":["Amino acid encoding for deep learning applications"],"prefix":"10.1186","volume":"21","author":[{"given":"Hesham","family":"ElAbd","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yana","family":"Bromberg","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Adrienne","family":"Hoarfrost","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tobias","family":"Lenz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1530-5811","authenticated-orcid":false,"given":"Andre","family":"Franke","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mareike","family":"Wendorff","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2020,6,9]]},"reference":[{"key":"3546_CR1","doi-asserted-by":"publisher","unstructured":"Poplin R, Chang P-C, Alexander D, Schwartz S, Colthurst T, Ku A, et al. A universal SNP and small-indel variant caller using deep neural networks. Nat Biotechnol. 2018;36:983. https:\/\/doi.org\/10.1038\/nbt.4235.","DOI":"10.1038\/nbt.4235"},{"issue":"11","key":"3546_CR2","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkx177","volume":"45","author":"H Zeng","year":"2017","unstructured":"Zeng H, Gifford DK. Predicting the impact of non-coding variants on DNA methylation. Nucleic Acids Res. 2017;45(11):e99. https:\/\/doi.org\/10.1093\/nar\/gkx177.","journal-title":"Nucleic Acids Res"},{"issue":"4","key":"3546_CR3","doi-asserted-by":"publisher","first-page":"660","DOI":"10.1093\/bioinformatics\/btx624","volume":"34","author":"M Kulmanov","year":"2018","unstructured":"Kulmanov M, Khan MA, Hoehndorf R. DeepGO: Predicting protein functions from sequence and interactions using a deep ontology-aware classifier. Bioinformatics. 2018;34(4):660\u20138. https:\/\/doi.org\/10.1093\/bioinformatics\/btx624.","journal-title":"Bioinformatics."},{"key":"3546_CR4","doi-asserted-by":"publisher","unstructured":"Zhang B, Li J, L\u00fc Q. Prediction of 8-state protein secondary structures by a novel deep learning architecture. BMC Bioinformatics. 2018;19:293. https:\/\/doi.org\/10.1186\/s12859-018-2280-5.","DOI":"10.1186\/s12859-018-2280-5"},{"key":"3546_CR5","doi-asserted-by":"publisher","unstructured":"Hashemifar S, Neyshabur B, Khan AA, Xu J. Predicting protein-protein interactions through sequence-based deep learning. Bioinformatics. 2018:802\u201310. https:\/\/doi.org\/10.1093\/bioinformatics\/bty573.","DOI":"10.1093\/bioinformatics\/bty573"},{"key":"3546_CR6","doi-asserted-by":"crossref","unstructured":"Zamani M, Kremer SC. Amino acid encoding schemes for machine learning methods. In:  2011 IEEE International Conference on Bioinformatics and Biomedicine Workshops: BIBMW; 2011.","DOI":"10.1109\/BIBMW.2011.6112394"},{"key":"3546_CR7","doi-asserted-by":"publisher","unstructured":"Lin K, May ACW, Taylor WR. Amino acid encoding schemes from protein structure alignments: Multi-dimensional vectors to describe residue types. J Theor Biol. 2002:361\u20135. https:\/\/doi.org\/10.1006\/jtbi.2001.2512.","DOI":"10.1006\/jtbi.2001.2512"},{"key":"3546_CR8","volume-title":"Where did the BLOSUM62 alignment score matrix come from? Nature Biotechnology","author":"SR Eddy","year":"2004","unstructured":"Eddy SR. Where did the BLOSUM62 alignment score matrix come from? Nature Biotechnology; 2004."},{"key":"3546_CR9","doi-asserted-by":"publisher","first-page":"775","DOI":"10.1002\/bip.20296","volume":"80","author":"H Mei","year":"2005","unstructured":"Mei H, Liao ZH, Zhou Y, Li SZ. A new set of amino acid descriptors and its application in peptide QSARs. Biopolymers. 2005;80:775\u201386.","journal-title":"Biopolymers."},{"key":"3546_CR10","doi-asserted-by":"publisher","unstructured":"Torng W, Altman RB. 3D deep convolutional neural networks for amino acid environment similarity analysis. BMC Bioinformatics. 2017;18:302. https:\/\/doi.org\/10.1186\/s12859-017-1702-0.","DOI":"10.1186\/s12859-017-1702-0"},{"key":"3546_CR11","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0141287","volume":"10","author":"E Asgari","year":"2015","unstructured":"Asgari E, Mofrad MRK. Continuous Distributed Representation of Biological Sequences for Deep Proteomics and Genomics. PLoS One. 2015;10:e0141287. https:\/\/doi.org\/10.1371\/journal.pone.0141287.","journal-title":"PLoS One"},{"key":"3546_CR12","unstructured":"Mikolov T, Chen K, Corrado G, Dean J. Efficient Estimation of Word Representations in Vector Space. CoRR. 2013; http:\/\/arxiv.org\/abs\/1301.3781."},{"key":"3546_CR13","doi-asserted-by":"publisher","unstructured":"Rives A, Goyal S, Meier J, Guo D, Ott M, Zitnick CL, et al. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. bioRxiv. 2019:622803. https:\/\/doi.org\/10.1101\/622803.","DOI":"10.1101\/622803"},{"key":"3546_CR14","doi-asserted-by":"publisher","first-page":"1315","DOI":"10.1038\/s41592-019-0598-1","volume":"16","author":"EC Alley","year":"2019","unstructured":"Alley EC, Khimulya G, Biswas S, AlQuraishi M, Church GM. Unified rational protein engineering with sequence-based deep representation learning. Nat Methods. 2019;16:1315\u201322. https:\/\/doi.org\/10.1038\/s41592-019-0598-1.","journal-title":"Nat Methods"},{"key":"3546_CR15","doi-asserted-by":"publisher","first-page":"16932","DOI":"10.1038\/s41598-019-53324-w","volume":"9","author":"D Raimondi","year":"2019","unstructured":"Raimondi D, Orlando G, Vranken WF, Moreau Y. Exploring the limitations of biophysical propensity scales coupled with machine learning for protein sequence analysis. Sci Rep. 2019;9:16932. https:\/\/doi.org\/10.1038\/s41598-019-53324-w.","journal-title":"Sci Rep"},{"key":"3546_CR16","first-page":"5998","volume-title":"Attention is all you need. Advances in Neural Information Processing Systems 30","author":"A Vaswani","year":"2017","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention is all you need. Advances in Neural Information Processing Systems 30; 2017. p. 5998\u20136008."},{"key":"3546_CR17","doi-asserted-by":"publisher","unstructured":"Liu Z, Jin J, Cui Y, Xiong Z, Nasiri A, Zhao Y, et al. DeepSeqPanII: an interpretable recurrent neural network model with attention mechanism for peptide-HLA class II binding prediction. bioRxiv. 2019. https:\/\/doi.org\/10.1101\/817502.","DOI":"10.1101\/817502"},{"key":"3546_CR18","unstructured":"Kocmi T, Bojar O. An Exploration of Word Embedding Initialization in Deep-Learning Tasks. CoRR. 2017; http:\/\/arxiv.org\/abs\/1711.09160."},{"issue":"10","key":"3546_CR19","doi-asserted-by":"publisher","first-page":"793","DOI":"10.1038\/nri1708","volume":"5","author":"W Reith","year":"2005","unstructured":"Reith W, LeibundGut-Landmann S, Waldburger JM. Regulation of MHC class II gene expression by the class II transactivator. Nat Rev Immunol. 2005;5(10):793\u2013806. https:\/\/doi.org\/10.1038\/nri1708.","journal-title":"Nat Rev Immunol"},{"key":"3546_CR20","doi-asserted-by":"publisher","first-page":"394","DOI":"10.1111\/imm.12889","volume":"154","author":"KK Jensen","year":"2018","unstructured":"Jensen KK, Andreatta M, Marcatili P, Buus S, Greenbaum JA, Yan Z, et al. Improved methods for predicting peptide binding affinity to MHC class II molecules. Immunology. 2018;154:394\u2013406.","journal-title":"Immunology."},{"key":"3546_CR21","doi-asserted-by":"crossref","unstructured":"Das J, Yu H. HINT: High-quality protein interactomes and their applications in understanding human disease. BMC Syst Biol. 2012.","DOI":"10.1186\/1752-0509-6-92"},{"key":"3546_CR22","doi-asserted-by":"publisher","unstructured":"Consortium TU. UniProt: a worldwide hub of protein knowledge. Nucleic Acids Res. 2019;47:D506\u201315. https:\/\/doi.org\/10.1093\/nar\/gky1049.","DOI":"10.1093\/nar\/gky1049"},{"key":"3546_CR23","doi-asserted-by":"publisher","first-page":"421","DOI":"10.1186\/1471-2105-10","volume":"10","author":"C Camacho","year":"2009","unstructured":"Camacho C, Coulouris G, Avagyan V, Ma N, Papadopoulos J, Bealer K, et al. BLAST+: architecture and applications. BMC Bioinformatics. 2009;10:421. https:\/\/doi.org\/10.1186\/1471-2105-10.","journal-title":"BMC Bioinformatics"},{"key":"3546_CR24","volume-title":"Keras","author":"F Chollet","year":"2015","unstructured":"Chollet F, et al. Keras. 2015."},{"key":"3546_CR25","volume-title":"3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings","author":"DP Kingma","year":"2015","unstructured":"Kingma DP, Ba J. Adam: A Method for Stochastic Optimization. In:  3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings; 2015. http:\/\/arxiv.org\/abs\/1412.6980."},{"key":"3546_CR26","unstructured":"Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, et al. TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems. CoRR. 2016; http:\/\/arxiv.org\/abs\/1603.04467."}],"container-title":["BMC Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-020-03546-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12859-020-03546-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-020-03546-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,6,8]],"date-time":"2021-06-08T23:14:40Z","timestamp":1623194080000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/s12859-020-03546-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,6,9]]},"references-count":26,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2020,12]]}},"alternative-id":["3546"],"URL":"https:\/\/doi.org\/10.1186\/s12859-020-03546-x","relation":{},"ISSN":["1471-2105"],"issn-type":[{"value":"1471-2105","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,6,9]]},"assertion":[{"value":"20 February 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 May 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 June 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Not applicable.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"235"}}