{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,13]],"date-time":"2026-03-13T08:24:06Z","timestamp":1773390246540,"version":"3.50.1"},"reference-count":36,"publisher":"Oxford University Press (OUP)","issue":"Supplement_2","license":[{"start":{"date-parts":[[2024,9,4]],"date-time":"2024-09-04T00:00:00Z","timestamp":1725408000000},"content-version":"vor","delay-in-days":3,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"ECCB2024"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,9,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Motivation<\/jats:title>\n                  <jats:p>For the alignment of large numbers of protein sequences, tools are predominant that decide to align two residues using only simple prior knowledge, e.g. amino acid substitution matrices, and using only part of the available data. The accuracy of state-of-the-art programs declines with decreasing sequence identity and when increasingly large numbers of sequences are aligned. Recently, transformer-based deep-learning models started to harness the vast amount of protein sequence data, resulting in powerful pretrained language models with the main purpose of generating high-dimensional numerical representations, embeddings, for individual sites that agglomerate evolutionary, structural, and biophysical information.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>We extend the traditional profile hidden Markov model so that it takes as inputs unaligned protein sequences and the corresponding embeddings. We fit the model with gradient descent using our existing differentiable hidden Markov layer. All sequences and their embeddings are jointly aligned to a model of the protein family. We report that our upgraded HMM-based aligner, learnMSA2, combined with the ProtT5-XL protein language model aligns on average almost 6% points more columns correctly than the best amino acid-based competitor and scales well with sequence number. The relative advantage of learnMSA2 over other programs tends to be greater when the sequence identity is lower and when the number of sequences is larger. Our results strengthen the evidence on the rich information contained in protein language models\u2019 embeddings and their potential downstream impact on the field of bioinformatics.<\/jats:p>\n                  <jats:p>Availability and implementation: \u00a0https:\/\/github.com\/Gaius-Augustus\/learnMSA, PyPI and Bioconda, evaluation: https:\/\/github.com\/felbecker\/snakeMSA<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btae381","type":"journal-article","created":{"date-parts":[[2024,9,5]],"date-time":"2024-09-05T07:31:51Z","timestamp":1725521511000},"page":"ii79-ii86","source":"Crossref","is-referenced-by-count":6,"title":["learnMSA2: deep protein multiple alignments with large language and hidden Markov models"],"prefix":"10.1093","volume":"40","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6831-8523","authenticated-orcid":false,"given":"Felix","family":"Becker","sequence":"first","affiliation":[{"name":"Institute of Mathematics and Computer Science, University of Greifswald , 17489 Greifswald, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8696-0384","authenticated-orcid":false,"given":"Mario","family":"Stanke","sequence":"additional","affiliation":[{"name":"Institute of Mathematics and Computer Science, University of Greifswald , 17489 Greifswald, Germany"}]}],"member":"286","published-online":{"date-parts":[[2024,9,4]]},"reference":[{"key":"2024090413581429900_btae381-B1","doi-asserted-by":"crossref","first-page":"1315","DOI":"10.1038\/s41592-019-0598-1","article-title":"Unified rational protein engineering with sequence-based deep representation learning","volume":"16","author":"Aley","year":"2019","journal-title":"Nat Methods"},{"key":"2024090413581429900_btae381-B2","first-page":"747","article-title":"Hidden Markov models in molecular biology: new algorithms and applications","volume":"5","author":"Baldi","year":"1992","journal-title":"Adv Neural Inf Process Syst"},{"key":"2024090413581429900_btae381-B3","doi-asserted-by":"crossref","first-page":"giac104","DOI":"10.1093\/gigascience\/giac104","article-title":"learnMSA: learning and aligning large protein families","volume":"11","author":"Becker","year":"2022","journal-title":"GigaScience"},{"key":"2024090413581429900_btae381-B4","doi-asserted-by":"crossref","first-page":"10556","DOI":"10.1073\/pnas.1405628111","article-title":"Simple chained guide trees give high-quality protein multiple sequence alignments","volume":"111","author":"Becker","year":"2014","journal-title":"Proc Natl Acad Sci USA"},{"key":"2024090413581429900_btae381-B5","doi-asserted-by":"crossref","first-page":"2102","DOI":"10.1093\/bioinformatics\/btac020","article-title":"ProteinBERT: a universal deep-learning model of protein sequence and function","volume":"38","author":"Brandes","year":"2022","journal-title":"Bioinformatics"},{"key":"2024090413581429900_btae381-B6","doi-asserted-by":"crossref","first-page":"33964","DOI":"10.1038\/srep33964","article-title":"FAMSA: fast and accurate multiple sequence alignment of huge protein families","volume":"6","author":"Deorowicz","year":"2016","journal-title":"Sci Rep"},{"key":"2024090413581429900_btae381-B7","doi-asserted-by":"crossref","first-page":"e1000069","DOI":"10.1371\/journal.pcbi.1000069","article-title":"A probabilistic model of local sequence alignment that simplifies statistical significance estimation","volume":"4","author":"Eddy","year":"2008","journal-title":"PLoS Comput Biol"},{"key":"2024090413581429900_btae381-B8","first-page":"114","article-title":"Multiple alignment using hidden Markov models","volume":"3","author":"Eddy","year":"1995","journal-title":"ISMB"},{"key":"2024090413581429900_btae381-B9","doi-asserted-by":"crossref","first-page":"6968","DOI":"10.1038\/s41467-022-34630-w","article-title":"Muscle5: high-accuracy alignment ensembles enable unbiased assessments of sequence homology and phylogeny","volume":"13","author":"Edgar","year":"2022","journal-title":"Nat Commun"},{"key":"2024090413581429900_btae381-B10","doi-asserted-by":"crossref","first-page":"7112","DOI":"10.1109\/TPAMI.2021.3095381","article-title":"Prottrans: toward understanding the language of life through self-supervised learning","volume":"44","author":"Elnaggar","year":"2021","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"2024090413581429900_btae381-B11","doi-asserted-by":"crossref","first-page":"1466","DOI":"10.1038\/s41587-019-0333-6","article-title":"Large multiple sequence alignments with a root-to-leaf regressive method","volume":"37","author":"Garriga","year":"2019","journal-title":"Nat Biotechnol"},{"key":"2024090413581429900_btae381-B12","author":"Hu","year":"2022"},{"key":"2024090413581429900_btae381-B13","doi-asserted-by":"crossref","first-page":"btad579","DOI":"10.1093\/bioinformatics\/btad579","article-title":"pLM-BLAST: distant homology detection based on direct comparison of sequence representations from protein language models","volume":"39","author":"Kaminski","year":"2023","journal-title":"Bioinformatics"},{"key":"2024090413581429900_btae381-B14","doi-asserted-by":"crossref","first-page":"772","DOI":"10.1093\/molbev\/mst010","article-title":"MAFFT multiple sequence alignment software version 7: improvements in performance and usability","volume":"30","author":"Katoh","year":"2013","journal-title":"Mol Biol Evol"},{"key":"2024090413581429900_btae381-B15","doi-asserted-by":"crossref","first-page":"1501","DOI":"10.1006\/jmbi.1994.1104","article-title":"Hidden Markov models in computational biology: applications to protein modeling","volume":"235","author":"Krogh","year":"1994","journal-title":"J Mol Biol"},{"key":"2024090413581429900_btae381-B16","first-page":"1928","author":"Lassmann","year":"2019"},{"key":"2024090413581429900_btae381-B17","author":"Lewin","year":"2022"},{"key":"2024090413581429900_btae381-B18","first-page":"1123","volume-title":"Science","author":"Lin","year":"2023"},{"key":"2024090413581429900_btae381-B19","doi-asserted-by":"crossref","first-page":"104","DOI":"10.1038\/s41592-022-01700-2","article-title":"Deep embedding and alignment of protein sequences","volume":"20","author":"Llinares-L\u00f3pez","year":"2023","journal-title":"Nat Methods"},{"key":"2024090413581429900_btae381-B20","first-page":"277675","article-title":"Leveraging protein language models for accurate multiple sequence alignments","volume":"33","author":"McWhite","year":"2023","journal-title":"Genome Res"},{"key":"2024090413581429900_btae381-B21","doi-asserted-by":"crossref","first-page":"e121","DOI":"10.1093\/nar\/gkt263","article-title":"Challenges in homology search: HMMER3 and convergent evolution of coiled-coil regions","volume":"41","author":"Mistry","year":"2013","journal-title":"Nucleic Acids Res"},{"key":"2024090413581429900_btae381-B22","doi-asserted-by":"crossref","first-page":"D412","DOI":"10.1093\/nar\/gkaa913","article-title":"Pfam: the protein families database in 2021","volume":"49","author":"Mistry","year":"2021","journal-title":"Nucleic Acids Res"},{"key":"2024090413581429900_btae381-B23","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1186\/s13059-015-0688-z","article-title":"Ultra-large alignments using phylogeny-aware profiles","volume":"16","author":"Nguyen","year":"2015","journal-title":"Genome Biol"},{"key":"2024090413581429900_btae381-B24","article-title":"Embedding-based alignment: combining protein language models with dynamic programming alignment to detect structural similarities in the twilight-zone","volume-title":"Bioinformatics","author":"Pantolini","year":"2024"},{"key":"2024090413581429900_btae381-B25","doi-asserted-by":"crossref","first-page":"btad007","DOI":"10.1093\/bioinformatics\/btad007","article-title":"UPP2: fast and accurate alignment of datasets with fragmentary sequences","volume":"39","author":"Park","year":"2023","journal-title":"Bioinformatics"},{"key":"2024090413581429900_btae381-B26","doi-asserted-by":"crossref","first-page":"btac724","DOI":"10.1093\/bioinformatics\/btac724","article-title":"End-to-end learning of multiple sequence alignments with differentiable Smith\u2013Waterman","volume":"39","author":"Petti","year":"2023","journal-title":"Bioinformatics"},{"key":"2024090413581429900_btae381-B27","doi-asserted-by":"crossref","DOI":"10.1101\/2020.12.15.422761","article-title":"Transformer protein language models are unsupervised structure learners","author":"Rao","year":"2020"},{"key":"2024090413581429900_btae381-B28","doi-asserted-by":"crossref","first-page":"e2016239118","DOI":"10.1073\/pnas.2016239118","article-title":"Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences","volume":"118","author":"Rives","year":"2021","journal-title":"Proc Natl Acad Sci USA"},{"key":"2024090413581429900_btae381-B29","doi-asserted-by":"crossref","first-page":"102577","DOI":"10.1016\/j.sbi.2023.102577","article-title":"Towards the accurate alignment of over a million protein sequences: current state of the art","volume":"80","author":"Santus","year":"2023","journal-title":"Curr Opin Struct Biol"},{"key":"2024090413581429900_btae381-B30","doi-asserted-by":"crossref","first-page":"539","DOI":"10.1038\/msb.2011.75","article-title":"Fast, scalable generation of high-quality protein multiple sequence alignments using Clustal Omega","volume":"7","author":"Sievers","year":"2011","journal-title":"Mol Syst Biol"},{"key":"2024090413581429900_btae381-B31","doi-asserted-by":"crossref","first-page":"1666","DOI":"10.1093\/bioinformatics\/btaa992","article-title":"MAGUS: multiple sequence alignment using graph clustering","volume":"37","author":"Smirnov","year":"2021","journal-title":"Bioinformatics"},{"key":"2024090413581429900_btae381-B32","doi-asserted-by":"crossref","first-page":"951","DOI":"10.1093\/bioinformatics\/bti125","article-title":"Protein homology detection by HMM\u2013HMM comparison","volume":"21","author":"S\u00f6ding","year":"2005","journal-title":"Bioinformatics"},{"key":"2024090413581429900_btae381-B33","doi-asserted-by":"crossref","first-page":"1026","DOI":"10.1038\/nbt.3988","article-title":"MMseqs2 enables sensitive protein sequence searching for the analysis of massive data sets","volume":"35","author":"Steinegger","year":"2017","journal-title":"Nat Biotechnol"},{"key":"2024090413581429900_btae381-B34","first-page":"5998","article-title":"Attention is all you need","volume":"30","author":"Vaswani","year":"2017","journal-title":"Adv Neural Inf Process Syst"},{"key":"2024090413581429900_btae381-B35","author":"Vig","year":"2020"},{"key":"2024090413581429900_btae381-B36","doi-asserted-by":"crossref","first-page":"3246","DOI":"10.1093\/bioinformatics\/btw412","article-title":"Application of the MAFFT sequence alignment program to large data\u2013reexamination of the usefulness of chained guide trees","volume":"32","author":"Yamada","year":"2016","journal-title":"Bioinformatics"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/40\/Supplement_2\/ii79\/59016898\/btae381.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/40\/Supplement_2\/ii79\/59016898\/btae381.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,5]],"date-time":"2024-09-05T07:32:05Z","timestamp":1725521525000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/40\/Supplement_2\/ii79\/7749066"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,1]]},"references-count":36,"journal-issue":{"issue":"Supplement_2","published-print":{"date-parts":[[2024,9,1]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btae381","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,9]]},"published":{"date-parts":[[2024,9,1]]}}}