{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,13]],"date-time":"2026-11-13T11:20:30Z","timestamp":1794568830341,"version":"4.9.1"},"reference-count":58,"publisher":"The Royal Society","issue":"225","license":[{"start":{"date-parts":[[2025,4,1]],"date-time":"2025-04-01T00:00:00Z","timestamp":1743465600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2025,4,1]],"date-time":"2025-04-01T00:00:00Z","timestamp":1743465600000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000155","name":"Division of Environmental Biology","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000155","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["royalsocietypublishing.org"],"crossmark-restriction":true},"short-container-title":["J. R. Soc. Interface."],"published-print":{"date-parts":[[2025,4]]},"abstract":"<jats:p>\n                    Predicting the evolutionary patterns of emerging and endemic viruses is key for mitigating their spread. In particular, it is critical to rapidly identify mutations with the potential for immune escape or increased disease burden. Knowing which circulating mutations pose a concern can inform treatment or mitigation strategies such as alternative vaccines or targeted social distancing. In 2021, Hie B, Zhong ED, Berger B, Bryson B. 2021 Learning the language of viral evolution and escape.\n                    <jats:italic>Science<\/jats:italic>\n                    <jats:bold>371<\/jats:bold>\n                    , 284\u2013288. (doi:\n                    <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/www.science.org\/doi\/10.1126\/science.abd7331\" xlink:type=\"simple\">10.1126\/science.abd7331<\/jats:ext-link>\n                    ) proposed that variants of concern can be identified using two quantities extracted from protein language models, grammaticality and semantic change. These quantities are defined by analogy to concepts from natural language processing. Grammaticality is intended to be a measure of whether a variant viral protein is viable, and semantic change is intended to be a measure of potential for immune escape. Here, we systematically test this hypothesis, taking advantage of several high-throughput datasets that have become available, and also comparing this model with several more recently published machine learning models. We find that grammaticality can be a measure of protein viability, though methods that are trained explicitly to predict mutational effects appear to be more effective. By contrast, we do not find compelling evidence that semantic change is a useful tool for identifying immune escape mutations.\n                  <\/jats:p>","DOI":"10.1098\/rsif.2024.0598","type":"journal-article","created":{"date-parts":[[2025,4,29]],"date-time":"2025-04-29T19:13:03Z","timestamp":1745953983000},"update-policy":"https:\/\/doi.org\/10.1098\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["A systematic evaluation of the language-of-viral-escape model using multiple machine learning frameworks"],"prefix":"10.1098","volume":"22","author":[{"given":"Brent E.","family":"Allman","sequence":"first","affiliation":[{"name":"Integrative Biology, The University of Texas at Austin","place":["Austin, USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Luiz","family":"Vieira","sequence":"additional","affiliation":[{"name":"Integrative Biology, The University of Texas at Austin","place":["Austin, USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel J.","family":"Diaz","sequence":"additional","affiliation":[{"name":"Institute for Foundations of Machine Learning, The University of Texas at Austin","place":["Austin, USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7470-9261","authenticated-orcid":true,"given":"Claus O.","family":"Wilke","sequence":"additional","affiliation":[{"name":"Integrative Biology, The University of Texas at Austin","place":["Austin, USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"175","published-online":{"date-parts":[[2025,4,30]]},"reference":[{"key":"e_1_3_8_2_2","doi-asserted-by":"publisher","DOI":"10.3389\/fimmu.2022.801522"},{"key":"e_1_3_8_3_2","doi-asserted-by":"publisher","DOI":"10.3390\/v13071211"},{"key":"e_1_3_8_4_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2308655120"},{"key":"e_1_3_8_5_2","doi-asserted-by":"publisher","DOI":"10.3390\/v8060155"},{"key":"e_1_3_8_6_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-022-34506-z"},{"key":"e_1_3_8_7_2","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.83442"},{"key":"e_1_3_8_8_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.abf9302"},{"key":"e_1_3_8_9_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.abm1208"},{"key":"e_1_3_8_10_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2314518121"},{"key":"e_1_3_8_11_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.abd7331"},{"key":"e_1_3_8_12_2","unstructured":"Lamb KD et al. From a single sequence to evolutionary trajectories: protein language models capture the evolutionary potential of SARS-CoV-2 protein sequences. bioRxiv. (doi:10.1101\/2024.07.05.602129)"},{"key":"e_1_3_8_13_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-023-06617-0"},{"key":"e_1_3_8_14_2","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btac020"},{"key":"e_1_3_8_15_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.ade2574"},{"key":"e_1_3_8_16_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41587-022-01618-2"},{"key":"e_1_3_8_17_2","doi-asserted-by":"publisher","DOI":"10.1111\/cogs.12414"},{"key":"e_1_3_8_18_2","first-page":"141","article-title":"From frequency to meaning: vector space models of semantics","volume":"37","author":"Turney PD","year":"2010","unstructured":"Turney PD, Pantel P. 2010 From frequency to meaning: vector space models of semantics. J. Artif. Intell. 37, 141\u2013188. (doi:10.1613\/jair.2934)","journal-title":"J. Artif. Intell."},{"key":"e_1_3_8_19_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cell.2023.02.001"},{"key":"e_1_3_8_20_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-018-03665-3"},{"key":"e_1_3_8_21_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.immuni.2018.12.017"},{"key":"e_1_3_8_22_2","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.34420"},{"key":"e_1_3_8_23_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.abd0831"},{"key":"e_1_3_8_24_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-024-46356-y"},{"key":"e_1_3_8_25_2","first-page":"15","article-title":"Evolution-inspired loss functions for protein representation learning","volume":"235","author":"Gong C","year":"2024","unstructured":"Gong C, Klivans A, Loy JM, Chen T, Liu Q, Diaz DJ. 2024 Evolution-inspired loss functions for protein representation learning. Proc. Machine Learning Res. 235, 15.","journal-title":"Proc. Machine Learning Res."},{"key":"e_1_3_8_26_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-023-40247-w"},{"key":"e_1_3_8_27_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12859-017-1702-0"},{"key":"e_1_3_8_28_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.sbi.2022.102518"},{"key":"e_1_3_8_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2021.3095381"},{"key":"e_1_3_8_30_2","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btac474"},{"key":"e_1_3_8_31_2","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2105-9-S2-S6"},{"key":"e_1_3_8_32_2","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btu739"},{"key":"e_1_3_8_33_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-024-49780-2"},{"key":"e_1_3_8_34_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-023-06328-6"},{"key":"e_1_3_8_35_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-021-04043-8"},{"key":"e_1_3_8_36_2","unstructured":"Devlin J Chang M Lee K Toutanova K. BERT: pre-training of deep bidirectional transformers for language understanding. arXiv. (doi:10.48550\/arXiv.1810.04805)"},{"key":"e_1_3_8_37_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0406744102"},{"key":"e_1_3_8_38_2","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.00631"},{"key":"e_1_3_8_39_2","doi-asserted-by":"publisher","DOI":"10.1002\/pro.2071"},{"key":"e_1_3_8_40_2","doi-asserted-by":"publisher","DOI":"10.1093\/molbev\/msy131"},{"key":"e_1_3_8_41_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1017572108"},{"key":"e_1_3_8_42_2","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gki387"},{"key":"e_1_3_8_43_2","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2105-12-151"},{"key":"e_1_3_8_44_2","doi-asserted-by":"publisher","DOI":"10.1002\/pro.4527"},{"key":"e_1_3_8_45_2","doi-asserted-by":"publisher","DOI":"10.1101\/2023.12.07.570727"},{"key":"e_1_3_8_46_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2409667121"},{"key":"e_1_3_8_47_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-020-80786-0"},{"key":"e_1_3_8_48_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12859-019-3220-8"},{"key":"e_1_3_8_49_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41551-024-01243-1"},{"key":"e_1_3_8_50_2","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btaa701"},{"key":"e_1_3_8_51_2","first-page":"76229","article-title":"Predicting a protein\u2019s stability under a million mutations","volume":"36","author":"Ouyang-Zhang J","year":"2023","unstructured":"Ouyang-Zhang J, Diaz D, Klivans A, Kraehenbuehl P. 2023 Predicting a protein\u2019s stability under a million mutations. Adv. Neural Inf. Process. Syst. 36, 76229\u201376247.","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_3_8_52_2","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-024-00795-w"},{"key":"e_1_3_8_53_2","unstructured":"Gong C Klivans A Wells J Loy J Liu Q Dimakis A Diaz D. 2023 Binding Oracle: fine-tuning from stability to binding free energy. See https:\/\/openreview.net\/forum?id=ChU7MCLk1J."},{"key":"e_1_3_8_54_2","unstructured":"Evans R et al. 2022 Protein complex prediction with AlphaFold-Multimer. bioRxiv. (doi:10.1101\/2021.10.04.463034)"},{"key":"e_1_3_8_55_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-021-03819-2"},{"key":"e_1_3_8_56_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12859-015-0590-4"},{"key":"e_1_3_8_57_2","doi-asserted-by":"crossref","unstructured":"Allman BE Vieira LC Diaz DJ Wilke CO. 2025 Associated code and data files for: a systematic evaluation of the language-of-viral-escape model using multiple machine learning frameworks. Zenodo (doi:10.5281\/zenodo.14867459)","DOI":"10.1101\/2024.09.04.611278"},{"key":"e_1_3_8_58_2","unstructured":"Allman B. 2025 allmanbrent\/NLP_viral_escape https:\/\/github.com\/allmanbrent\/ NLP_viral_escape"},{"key":"e_1_3_8_59_2","doi-asserted-by":"crossref","unstructured":"Allman BE Vieira L Diaz DJ Wilke CO. 2025 Supplementary material from: A systematic evaluation of the language-of-viral-escape model using multiple machine learning frameworks. Figshare. (doi:10.6084\/m9.figshare.c.7742879)","DOI":"10.1101\/2024.09.04.611278"}],"container-title":["Journal of The Royal Society Interface"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/royalsocietypublishing.org\/doi\/pdf\/10.1098\/rsif.2024.0598","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/royalsocietypublishing.org\/doi\/full-xml\/10.1098\/rsif.2024.0598","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/royalsocietypublishing.org\/doi\/pdf\/10.1098\/rsif.2024.0598","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,4,29]],"date-time":"2025-04-29T19:13:06Z","timestamp":1745953986000},"score":1,"resource":{"primary":{"URL":"https:\/\/royalsocietypublishing.org\/doi\/10.1098\/rsif.2024.0598"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4]]},"references-count":58,"journal-issue":{"issue":"225","published-print":{"date-parts":[[2025,4]]}},"alternative-id":["10.1098\/rsif.2024.0598"],"URL":"https:\/\/doi.org\/10.1098\/rsif.2024.0598","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2024.09.04.611278","asserted-by":"object"}]},"ISSN":["1742-5662"],"issn-type":[{"value":"1742-5662","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,4]]},"assertion":[{"value":"2024-08-29","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-01-02","order":1,"name":"revised","label":"Revised","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-02-18","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-04-30","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"20240598"}}