{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,22]],"date-time":"2026-10-22T20:12:53Z","timestamp":1792699973310,"version":"4.5.0"},"reference-count":159,"publisher":"American Association for the Advancement of Science (AAAS)","issue":"6723","content-domain":{"domain":["www.science.org"],"crossmark-restriction":true},"short-container-title":["Science"],"published-print":{"date-parts":[[2024,11,15]]},"abstract":"<jats:p>The genome is a sequence that encodes the DNA, RNA, and proteins that orchestrate an organism\u2019s function. We present Evo, a long-context genomic foundation model with a frontier architecture trained on millions of prokaryotic and phage genomes, and report scaling laws on DNA to complement observations in language and vision. Evo generalizes across DNA, RNA, and proteins, enabling zero-shot function prediction competitive with domain-specific language models and the generation of functional CRISPR-Cas and transposon systems, representing the first examples of protein-RNA and protein-DNA codesign with a language model. Evo also learns how small mutations affect whole-organism fitness and generates megabase-scale sequences with plausible genomic architecture. These prediction and generation capabilities span molecular to genomic scales of complexity, advancing our understanding and control of biology.<\/jats:p>","DOI":"10.1126\/science.ado9336","type":"journal-article","created":{"date-parts":[[2024,11,14]],"date-time":"2024-11-14T14:00:03Z","timestamp":1731592803000},"update-policy":"https:\/\/doi.org\/10.34133\/aaas_crossmark","source":"Crossref","is-referenced-by-count":563,"title":["Sequence modeling and design from molecular to genome scale with Evo"],"prefix":"10.1126","volume":"386","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-6593-0707","authenticated-orcid":true,"given":"Eric","family":"Nguyen","sequence":"first","affiliation":[{"name":"Arc Institute, Palo Alto, CA, USA."},{"name":"Department of Bioengineering, Stanford University, Stanford, CA, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael","family":"Poli","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Stanford University, Stanford, CA, USA."},{"name":"TogetherAI, San Francisco, CA, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Matthew G.","family":"Durrant","sequence":"additional","affiliation":[{"name":"Arc Institute, Palo Alto, CA, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Brian","family":"Kang","sequence":"additional","affiliation":[{"name":"Arc Institute, Palo Alto, CA, USA."},{"name":"Department of Bioengineering, Stanford University, Stanford, CA, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8028-3244","authenticated-orcid":true,"given":"Dhruva","family":"Katrekar","sequence":"additional","affiliation":[{"name":"Arc Institute, Palo Alto, CA, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8921-5168","authenticated-orcid":true,"given":"David B.","family":"Li","sequence":"additional","affiliation":[{"name":"Arc Institute, Palo Alto, CA, USA."},{"name":"Department of Bioengineering, Stanford University, Stanford, CA, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Liam J.","family":"Bartie","sequence":"additional","affiliation":[{"name":"Arc Institute, Palo Alto, CA, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Armin W.","family":"Thomas","sequence":"additional","affiliation":[{"name":"Stanford Data Science, Stanford University, Stanford, CA, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1260-5045","authenticated-orcid":true,"given":"Samuel H.","family":"King","sequence":"additional","affiliation":[{"name":"Arc Institute, Palo Alto, CA, USA."},{"name":"Department of Bioengineering, Stanford University, Stanford, CA, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1253-0522","authenticated-orcid":true,"given":"Garyk","family":"Brixi","sequence":"additional","affiliation":[{"name":"Arc Institute, Palo Alto, CA, USA."},{"name":"Department of Genetics, Stanford University, Stanford, CA, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jeremy","family":"Sullivan","sequence":"additional","affiliation":[{"name":"Arc Institute, Palo Alto, CA, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3824-9349","authenticated-orcid":true,"given":"Madelena Y.","family":"Ng","sequence":"additional","affiliation":[{"name":"Stanford Center for Biomedical Informatics Research, Stanford, CA, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ashley","family":"Lewis","sequence":"additional","affiliation":[{"name":"Department of Biomedical Data Science, Stanford University, Stanford, CA, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aaron","family":"Lou","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Stanford University, Stanford, CA, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0039-2887","authenticated-orcid":true,"given":"Stefano","family":"Ermon","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Stanford University, Stanford, CA, USA."},{"name":"CZ Biohub, San Francisco, CA, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8692-5685","authenticated-orcid":true,"given":"Stephen A.","family":"Baccus","sequence":"additional","affiliation":[{"name":"Department of Neurobiology, Stanford University, Stanford, CA, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6553-3455","authenticated-orcid":true,"given":"Tina","family":"Hernandez-Boussard","sequence":"additional","affiliation":[{"name":"Department of Biomedical Data Science, Stanford University, Stanford, CA, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christopher","family":"R\u00e9","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Stanford University, Stanford, CA, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9380-2648","authenticated-orcid":true,"given":"Patrick D.","family":"Hsu","sequence":"additional","affiliation":[{"name":"Arc Institute, Palo Alto, CA, USA."},{"name":"Department of Bioengineering and Center for Computational Biology, University of California, Berkeley, Berkeley, CA, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3224-8142","authenticated-orcid":true,"given":"Brian L.","family":"Hie","sequence":"additional","affiliation":[{"name":"Arc Institute, Palo Alto, CA, USA."},{"name":"Stanford Data Science, Stanford University, Stanford, CA, USA."},{"name":"Department of Chemical Engineering, Stanford University, Stanford, CA, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"221","reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.32.812.120"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1038\/171737a0"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.47.10.1588"},{"key":"e_1_3_2_5_2","volume-title":"Genetics and the Origin of Species","author":"Dobzhansky T.","year":"1951","unstructured":"T. Dobzhansky, Genetics and the Origin of Species (Columbia Univ. Press, 1951)."},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-021-03819-2"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2016239118"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-024-00791-0"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1101\/gr.278870.123"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41592-021-01252-x"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-023-06415-8"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41587-022-01618-2"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-023-06728-8"},{"key":"e_1_3_2_14_2","doi-asserted-by":"crossref","unstructured":"L. F. DaSilva S. Senan Z. M. Patel A. J. Reddy S. Gabbita Z. Nussbaum C. M. V. C\u00f3rdova A. Wenteler N. Weber T. M. Tunjic T. A. Khan Z. Li C. Smith M. Bejan L. K. Louis P. Cornejo W. Connell E. S. Wong W. Meuleman L. Pinello DNA-Diffusion: Leveraging Generative Models for Controlling Chromatin Accessibility and Gene Expression via Synthetic Regulatory Elements. bioRxiv 2024.02.01.578352 [Preprint] (2024); https:\/\/doi.org\/10.1101\/2024.02.01.578352.10.1101\/2024.02.01.578352","DOI":"10.1101\/2024.02.01.578352"},{"key":"e_1_3_2_15_2","doi-asserted-by":"crossref","unstructured":"A. Lal D. Garfield T. Biancalani G. Eraslan regLM: Designing realistic regulatory DNA with autoregressive language models. bioRxiv 2024.02.14.580373 [Preprint] (2024); https:\/\/doi.org\/10.1101\/2024.02.14.580373.10.1007\/978-1-0716-3989-4_24","DOI":"10.1101\/2024.02.14.580373"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1177\/10943420231201154"},{"key":"e_1_3_2_17_2","doi-asserted-by":"crossref","unstructured":"H. Dalla-Torre L. Gonzalez J. Mendoza Revilla N. L. Carranza A. H. Grzywaczewski F. Oteri C. Dallago E. Trop H. Sirelkhatim G. Richard M. Skwark K. Beguir M. Lopez T. Pierrot The Nucleotide Transformer: Building and Evaluating Robust Foundation Models for Human Genomics. bioRxiv 2023.01.11.523679 [Preprint] (2023); https:\/\/doi.org\/10.1101\/2023.01.11.523679.10.1101\/2023.01.11.523679","DOI":"10.1101\/2023.01.11.523679"},{"key":"e_1_3_2_18_2","unstructured":"Z. Zhou Y. Ji W. Li P. Dutta R. Davuluri H. Liu DNABERT-2: Efficient foundation model and benchmark for multi-species genome. arXiv:2306.15006 [q-bio.GN] (2023)."},{"key":"e_1_3_2_19_2","unstructured":"Y. Tay V. Q. Tran S. Ruder J. Gupta H. W. Chung D. Bahri Z. Qin S. Baumgartner C. Yu D. Metzler Charformer: Fast Character Transformers via Gradient-based Subword Tokenization. arXiv:2106.12672 [cs.CL] (2022)."},{"key":"e_1_3_2_20_2","unstructured":"S. Chen S. Wong L. Chen Y. Tian Extending context window of large language models via positional interpolation. arXiv:2306.15595 [cs.CL] (2023)."},{"key":"e_1_3_2_21_2","unstructured":"H. Liu M. Zaharia P. Abbeel Ring attention with blockwise transformers for near-infinite context. arXiv:2310.01889 [cs.CL] (2023)."},{"key":"e_1_3_2_22_2","doi-asserted-by":"crossref","unstructured":"V. Fishman Y. Kuratov A. Shmelev M. Petrov D. Penzar D. Shepelin N. Chekanov O. Kardymon M. Burtsev GENA-LM: A family of open-source foundational DNA language models for long sequences. bioRxiv 2023.06.12.544594 [Preprint] (2024); https:\/\/doi.org\/10.1101\/2023.06.12.544594.10.1101\/2023.06.12.544594","DOI":"10.1101\/2023.06.12.544594"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btab083"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-024-46947-9"},{"key":"e_1_3_2_25_2","unstructured":"M. Poli J. Wang S. Massaroli J. Quesnelle R. Carlow E. Nguyen A. Thomas StripedHyena: Moving Beyond Transformers with Hybrid Signal Processing Models GitHub (2023); https:\/\/github.com\/togethercomputer\/stripedhyena."},{"key":"e_1_3_2_26_2","unstructured":"Z. Li N. Kovachki K. Azizzadenesheli B. Liu K. Bhattacharya A. Stuart A. Anandkumar Fourier neural operator for parametric partial differential equations. arXiv:2010.08895 [cs.LG] (2021)."},{"key":"e_1_3_2_27_2","unstructured":"A. Gu K. Goel C. R\u00e9 Efficiently modeling long sequences with structured state spaces. arXiv:2111.00396 [cs.LG] (2022)."},{"key":"e_1_3_2_28_2","unstructured":"A. Orvieto S. L. Smith A. Gu A. Fernando C. Gulcehre R. Pascanu S. De Resurrecting Recurrent Neural Networks for Long Sequences. arXiv:2303.06349 [cs.LG] (2023)."},{"key":"e_1_3_2_29_2","first-page":"17072","volume-title":"Advances in Neural Information Processing Systems","author":"Massaroli S.","year":"2023","unstructured":"S. Massaroli, M. Poli, D. Fu, H. Kumbong, R. Parnichkun, D. Romero, A. Timalsina, Q. McIntyre, B. Chen, A. Rudra, C. Zhang, C. R\u00e9, S. Ermon, Y. Bengio, \u201c Laughing Hyena Distillery: Extracting Compact Recurrences From Convolutions\u201d in Advances in Neural Information Processing Systems, vol. 36, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, S. Levine, Eds. (Curran Associates, Inc., 2023), pp. 17072\u201317116."},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2023.127063"},{"key":"e_1_3_2_31_2","unstructured":"X. Ma C. Zhou X. Kong J. He L. Gui G. Neubig J. May L. Zettlemoyer Mega: Moving average equipped gated attention. arXiv:2209.10655 [cs.LG] (2023)."},{"key":"e_1_3_2_32_2","unstructured":"D. Y. Fu T. Dao K. K. Saab A. W. Thomas A. Rudra C. R\u00e9 Hungry hungry hippos: Towards language modeling with state space models. arXiv:2212.14052 [cs.LG] (2023)."},{"key":"e_1_3_2_33_2","first-page":"7311","volume-title":"Advances in Neural Information Processing Systems","author":"Pilault J.","year":"2023","unstructured":"J. Pilault, M. Fathi, O. Firat, C. Pal, P.-L. Bacon, R. Goroshin, \u201c Block-state transformers\u201d in Advances in Neural Information Processing Systems, vol. 36, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, S. Levine, Eds. (Curran Associates, Inc., 2023), pp. 7311\u20137329."},{"key":"e_1_3_2_34_2","first-page":"43177","volume-title":"Advances in Neural Information Processing Systems","author":"Nguyen E.","year":"2023","unstructured":"E. Nguyen, M. Poli, M. Faizi, A. Thomas, M. Wornow, C. Birch-Sykes, S. Massaroli, A. Patel, C. Rabideau, Y. Bengio, S. Ermon, C. R\u00e9, S. Baccus, \u201c HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution\u201d in Advances in Neural Information Processing Systems, vol. 36, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, S. Levine, Eds. (Curran Associates, Inc., 2023), pp. 43177\u201343201."},{"key":"e_1_3_2_35_2","unstructured":"M. Poli S. Massaroli E. Nguyen D. Y. Fu T. Dao S. Baccus Y. Bengio S. Ermon C. R\u00e9 Hyena Hierarchy: Towards Larger Convolutional Language Models. arXiv:2302.10866 [cs.LG] (2023)."},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkab776"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkac1037"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkad964"},{"key":"e_1_3_2_39_2","unstructured":"J. Hoffmann S. Borgeaud A. Mensch E. Buchatskaya T. Cai E. Rutherford D. de Las Casas L. A. Hendricks J. Welbl A. Clark T. Hennigan E. Noland K. Millican G. van den Driessche B. Damoc A. Guy S. Osindero K. Simonyan E. Elsen J. W. Rae O. Vinyals L. Sifre Training Compute-Optimal Large Language Models. arXiv:2203.15556 [cs.CL] (2022)."},{"key":"e_1_3_2_40_2","unstructured":"J. Kaplan S. McCandlish T. Henighan T. B. Brown B. Chess R. Child S. Gray A. Radford J. Wu D. Amodei Scaling Laws for Neural Language Models. arXiv:2001.08361 [cs.LG] (2020)."},{"key":"e_1_3_2_41_2","unstructured":"A. Gu T. Dao Mamba: Linear-time sequence modeling with selective state spaces. arXiv:2312.00752 [cs.LG] (2024)."},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1101\/2021.07.09.450648"},{"key":"e_1_3_2_43_2","unstructured":"P. Notin M. Dias J. Frazer J. Marchena-Hurtado A. N. Gomez D. Marks Y. Gal \u201cTranception: Protein Fitness Prediction with Autoregressive Transformers and Inference-time Retrieval\u201d in Proceedings of the 39th International Conference on Machine Learning vol. 162 K. Chaudhuri S. Jegelka L. Song C. Szepesvari G. Niu S. Sabato Eds. (PMLR 2022) pp. 16990\u201317017."},{"key":"e_1_3_2_44_2","doi-asserted-by":"crossref","unstructured":"G. Benegas C. Albors A. J. Aw C. Ye Y. S. Song GPN-MSA: an alignment-based DNA language model for genome-wide variant effect prediction. bioRxiv 2023.10.10.561776 [Preprint] (2024); https:\/\/doi.org\/10.1101\/2023.10.10.561776.10.1101\/2023.10.10.561776","DOI":"10.1101\/2023.10.10.561776"},{"key":"e_1_3_2_45_2","doi-asserted-by":"crossref","unstructured":"P. Notin A. W. Kollasch D. Ritter L. van Niekerk S. Paul H. Spinner N. Rollins A. Shaw R. Weitzman J. Frazer M. Dias D. Franceschi R. Orenbuch Y. Gal D. S. Marks ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction. bioRxiv 2023.12.07.570727 [Preprint] (2023); https:\/\/doi.org\/10.1101\/2023.12.07.570727.10.1101\/2023.12.07.570727","DOI":"10.1101\/2023.12.07.570727"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.15252\/msb.202211474"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cels.2024.01.008"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.ade2574"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cels.2023.10.002"},{"key":"e_1_3_2_50_2","unstructured":"F.-Z. Li A. P. Amini Y. Yue K. K. Yang A. X. Lu Feature Reuse and Scaling: Understanding Transfer Learning with Protein Language Models. bioRxiv 2024.02.05.578959 [Preprint] (2024); https:\/\/doi.org\/10.1101\/2024.02.05.578959.10.1101\/2024.02.05.578959"},{"key":"e_1_3_2_51_2","doi-asserted-by":"crossref","unstructured":"J. Chen Z. Hu S. Sun Q. Tan Y. Wang Q. Yu L. Zong L. Hong J. Xiao T. Shen I. King Y. Li Interpretable RNA foundation model from unannotated data for highly accurate RNA structure and function predictions. arXiv:2204.00300 [q-bio.QM] (2022).","DOI":"10.1101\/2022.08.06.503062"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.mimet.2008.10.010"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-022-32829-5"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1021\/acs.biochem.7b01069"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41587-020-0584-2"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-020-20094-3"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1301301110"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1038\/nbt.1568"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1021\/acssynbio.0c00394"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41579-022-00739-4"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cell.2014.05.010"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1098\/rstb.2018.0087"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.1225829"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1038\/nbt.2647"},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-024-07487-w"},{"key":"e_1_3_2_66_2","volume-title":"Mobile DNA III","author":"Craig N. L.","year":"2020","unstructured":"N. L. Craig, M. Chandler, M. Gellert, A. M. Lambowitz, P. A. Rice, S. B. Sandmeyer, Eds., Mobile DNA III (Wiley, ed. 3, 2020).","edition":"3"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-023-06597-1"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-021-04058-1"},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.abj6856"},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cell.2007.12.029"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2406285121"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkj014"},{"key":"e_1_3_2_73_2","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkg859"},{"key":"e_1_3_2_74_2","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkh024"},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pbio.3002416"},{"key":"e_1_3_2_76_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1419677112"},{"key":"e_1_3_2_77_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1749-6632.2011.06418.x"},{"key":"e_1_3_2_78_2","doi-asserted-by":"publisher","DOI":"10.1101\/gr.186072.114"},{"key":"e_1_3_2_79_2","doi-asserted-by":"publisher","DOI":"10.1101\/gr.335003"},{"key":"e_1_3_2_80_2","doi-asserted-by":"publisher","DOI":"10.1186\/2042-5783-2-6"},{"key":"e_1_3_2_81_2","doi-asserted-by":"publisher","DOI":"10.1074\/jbc.M114.606632"},{"key":"e_1_3_2_82_2","unstructured":"T. Seemann barrnap GitHub (2018); https:\/\/github.com\/tseemann\/barrnap."},{"key":"e_1_3_2_83_2","doi-asserted-by":"publisher","DOI":"10.1093\/genetics\/149.1.445"},{"key":"e_1_3_2_84_2","unstructured":"J. Wei M. Bosma V. Y. Zhao K. Guu A. W. Yu B. Lester N. Du A. M. Dai Q. V. Le Finetuned language models are zero-shot learners. arXiv:2109.01652 [cs.CL] (2022)."},{"key":"e_1_3_2_85_2","unstructured":"L. Ouyang J. Wu X. Jiang D. Almeida C. L. Wainwright P. Mishkin C. Zhang S. Agarwal K. Slama A. Ray J. Schulman J. Hilton F. Kelton L. Miller M. Simens A. Askell P. Welinder P. Christiano J. Leike R. Lowe Training language models to follow instructions with human feedback. arXiv:2203.02155 [cs.CL] (2022)."},{"key":"e_1_3_2_86_2","first-page":"53728","volume-title":"Advances in Neural Information Processing Systems","author":"Rafailov R.","year":"2023","unstructured":"R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, C. Finn, \u201c Direct Preference Optimization: Your Language Model is Secretly a Reward Model\u201d in Advances in Neural Information Processing Systems, vol. 36, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, S. Levine, Eds. (Curran Associates, Inc., 2023), pp. 53728\u201353741."},{"key":"e_1_3_2_87_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.xgen.2021.100029"},{"key":"e_1_3_2_88_2","first-page":"22199","volume-title":"Advances in Neural Information Processing Systems","author":"Kojima T.","year":"2022","unstructured":"T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, Y. Iwasawa, \u201c Large Language Models are Zero-Shot Reasoners\u201d in Advances in Neural Information Processing Systems, vol. 35, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, A. Oh, Eds. (Curran Associates, Inc., 2022), pp. 22199\u201322213."},{"key":"e_1_3_2_89_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41587-023-01763-2"},{"key":"e_1_3_2_90_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.adk8946"},{"key":"e_1_3_2_91_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-024-07552-4"},{"key":"e_1_3_2_92_2","first-page":"933","volume-title":"Proceedings of the 34th International Conference on Machine Learning","author":"Dauphin Y. N.","year":"2017","unstructured":"Y. N. Dauphin, A. Fan, M. Auli, D. Grangier, \u201c Language Modeling with Gated Convolutional Networks\u201d in Proceedings of the 34th International Conference on Machine Learning, vol. 70, D. Precup, Y. W. Teh, Eds. (PMLR, 2017), pp. 933\u2013941."},{"key":"e_1_3_2_93_2","unstructured":"N. Shazeer GLU variants improve Transformer. arXiv:2002.05202 [cs.LG] (2020)."},{"key":"e_1_3_2_94_2","volume-title":"Advances in Neural Information Processing Systems","author":"Zhang B.","year":"2019","unstructured":"B. Zhang, R. Sennrich, \u201c Root Mean Square Layer Normalization\u201d in Advances in Neural Information Processing Systems, vol. 32, H. Wallach, H. Larochelle, A. Beygelzimer, F. d\u2019Alch\u00e9-Buc, E. Fox, R. Garnett, Eds. (Curran Associates, Inc., 2019)."},{"key":"e_1_3_2_95_2","first-page":"77546","volume-title":"Advances in Neural Information Processing Systems","author":"Fu D.","year":"2023","unstructured":"D. Fu, S. Arora, J. Grogan, I. Johnson, E. S. Eyuboglu, A. Thomas, B. Spector, M. Poli, A. Rudra, C. R\u00e9, \u201c Monarch Mixer: A simple sub-quadratic GEMM-based architecture\u201d in Advances in Neural Information Processing Systems, vol. 36, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, S. Levine, Eds. (Curran Associates, Inc., 2023), pp. 77546\u201377603."},{"key":"e_1_3_2_96_2","unstructured":"S. Arora S. Eyuboglu A. Timalsina I. Johnson M. Poli J. Zou A. Rudra C. R\u00e9 Zoology: Measuring and improving recall in efficient language models. arXiv:2312.04927 [cs.CL] (2023)."},{"key":"e_1_3_2_97_2","unstructured":"S. Bhattamishra A. Patel P. Blunsom V. Kanade Understanding in-context learning in transformers and LLMs by learning to learn discrete functions. arXiv:2310.03016 [cs.LG] (2023)."},{"key":"e_1_3_2_98_2","unstructured":"D. W. Romero A. Kuzina E. J. Bekkers J. M. Tomczak M. Hoogendoorn CKConv: Continuous kernel convolution for sequential data. arXiv:2102.02611 [cs.LG] (2022)."},{"key":"e_1_3_2_99_2","first-page":"22982","volume-title":"Advances in Neural Information Processing Systems","author":"Gupta A.","year":"2022","unstructured":"A. Gupta, A. Gu, J. Berant, \u201c Diagonal State Spaces are as Effective as Structured State Spaces\u201d in Advances in Neural Information Processing Systems, vol. 35, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, A. Oh, Eds. (Curran Associates, Inc., 2022), pp. 22982\u201322994."},{"key":"e_1_3_2_100_2","first-page":"35971","volume-title":"Advances in Neural Information Processing Systems","author":"Gu A.","year":"2022","unstructured":"A. Gu, K. Goel, A. Gupta, C. R\u00e9, \u201c On the Parameterization and Initialization of Diagonal State Space Models\u201d in Advances in Neural Information Processing Systems, vol. 35, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, A. Oh, Eds. (Curran Associates, Inc., 2022), pp. 35971\u201335983."},{"key":"e_1_3_2_101_2","unstructured":"M. Zhang K. K. Saab M. Poli T. Dao K. Goel C. R\u00e9 Effectively modeling time series with simple discrete state spaces. arXiv:2303.09489 [cs.LG] (2023)."},{"key":"e_1_3_2_102_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cels.2023.11.006"},{"key":"e_1_3_2_103_2","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkv1189"},{"key":"e_1_3_2_104_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41587-020-0603-3"},{"key":"e_1_3_2_105_2","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkaa939"},{"key":"e_1_3_2_106_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cell.2021.01.029"},{"key":"e_1_3_2_107_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41587-018-0009-7"},{"key":"e_1_3_2_108_2","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkz1035"},{"key":"e_1_3_2_109_2","doi-asserted-by":"publisher","DOI":"10.1128\/msystems.01045-20"},{"key":"e_1_3_2_110_2","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2105-9-386"},{"key":"e_1_3_2_111_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.1261359"},{"key":"e_1_3_2_112_2","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkr367"},{"key":"e_1_3_2_113_2","doi-asserted-by":"publisher","DOI":"10.1089\/crispr.2020.0059"},{"key":"e_1_3_2_114_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2308224120"},{"key":"e_1_3_2_115_2","doi-asserted-by":"publisher","DOI":"10.1038\/nbt.3988"},{"key":"e_1_3_2_116_2","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkf436"},{"key":"e_1_3_2_117_2","doi-asserted-by":"crossref","unstructured":"W. Xiong J. Liu I. Molybog H. Zhang P. Bhargava R. Hou L. Martin R. Rungta K. A. Sankararaman B. Oguz M. Khabsa H. Fang Y. Mehdad S. Narang K. Malik A. Fan S. Bhosale S. Edunov M. Lewis S. Wang H. Ma Effective long-context scaling of foundation models. arXiv:2309.16039 [cs.CL] (2023).","DOI":"10.18653\/v1\/2024.naacl-long.260"},{"key":"e_1_3_2_118_2","doi-asserted-by":"crossref","unstructured":"J. Ainslie J. Lee-Thorp M. de Jong Y. Zemlyanskiy F. Lebr\u00f3n S. Sanghai GQA: Training generalized multi-query transformer models from multi-head checkpoints. arXiv:2305.13245 [cs.CL] (2023).","DOI":"10.18653\/v1\/2023.emnlp-main.298"},{"key":"e_1_3_2_119_2","doi-asserted-by":"publisher","DOI":"10.1093\/molbev\/msu081"},{"key":"e_1_3_2_120_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1215206110"},{"key":"e_1_3_2_121_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.str.2011.11.021"},{"key":"e_1_3_2_122_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-023-06328-6"},{"key":"e_1_3_2_123_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cels.2016.11.004"},{"key":"e_1_3_2_124_2","doi-asserted-by":"publisher","DOI":"10.1093\/molbev\/msad047"},{"key":"e_1_3_2_125_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1004421"},{"key":"e_1_3_2_126_2","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.56707"},{"key":"e_1_3_2_127_2","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gku511"},{"key":"e_1_3_2_128_2","doi-asserted-by":"publisher","DOI":"10.1186\/s13073-020-0711-1"},{"key":"e_1_3_2_129_2","doi-asserted-by":"crossref","unstructured":"R. A. Silverstein S. Sun M. Verby J. Weile Y. Wu M. Gebbia I. Fotiadou J. Kitaygorodsky F. P. Roth A systematic genotype-phenotype map for missense variants in the human intellectual disability-associated gene GDI1. bioRxiv 2021.10.06.463360 [Preprint] (2022); https:\/\/doi.org\/10.1101\/2021.10.06.463360.10.1101\/2021.10.06.463360","DOI":"10.1101\/2021.10.06.463360"},{"key":"e_1_3_2_130_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-021-24495-w"},{"key":"e_1_3_2_131_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.molcel.2018.06.012"},{"key":"e_1_3_2_132_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41588-018-0204-y"},{"key":"e_1_3_2_133_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-018-0461-z"},{"key":"e_1_3_2_134_2","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkv265"},{"key":"e_1_3_2_135_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-020-15540-1"},{"key":"e_1_3_2_136_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-018-0170-7"},{"key":"e_1_3_2_137_2","doi-asserted-by":"publisher","DOI":"10.1101\/gad.245936.114"},{"key":"e_1_3_2_138_2","doi-asserted-by":"publisher","DOI":"10.1038\/nature10083"},{"key":"e_1_3_2_139_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.1192001"},{"key":"e_1_3_2_140_2","unstructured":"T. A. Chang B. K. Bergen Language model behavior: A comprehensive survey. arXiv:2303.11504 [cs.CL] (2023)."},{"key":"e_1_3_2_141_2","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2105-11-119"},{"key":"e_1_3_2_142_2","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2105-8-209"},{"key":"e_1_3_2_143_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12859-023-05345-6"},{"key":"e_1_3_2_144_2","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btac466"},{"key":"e_1_3_2_145_2","doi-asserted-by":"publisher","DOI":"10.1186\/1748-7188-6-26"},{"key":"e_1_3_2_146_2","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btt509"},{"key":"e_1_3_2_147_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41592-022-01585-1"},{"key":"e_1_3_2_148_2","unstructured":"Schr\u00f6dinger LLC The PyMOL Molecular Graphics System version 1.8 (2015)."},{"key":"e_1_3_2_149_2","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkn188"},{"key":"e_1_3_2_150_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-77553-1_14"},{"key":"e_1_3_2_151_2","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2105-12-3"},{"key":"e_1_3_2_152_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0022-2836(05)80360-2"},{"key":"e_1_3_2_153_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41592-022-01488-1"},{"key":"e_1_3_2_154_2","unstructured":"B. Hie Code for paper \u201cSequence modeling and design from molecular to genome scale with Evo \u201d Zenodo (2024); https:\/\/doi.org\/10.5281\/zenodo.12693561.10.5281\/zenodo.12693561"},{"key":"e_1_3_2_155_2","doi-asserted-by":"publisher","DOI":"10.1038\/s43856-023-00279-9"},{"key":"e_1_3_2_156_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.ado1671"},{"key":"e_1_3_2_157_2","doi-asserted-by":"publisher","DOI":"10.1016\/S2214-109X(22)00460-0"},{"key":"e_1_3_2_158_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0140-6736(21)02315-1"},{"key":"e_1_3_2_159_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.isci.2022.105423"},{"key":"e_1_3_2_160_2","doi-asserted-by":"publisher","DOI":"10.1038\/s43588-023-00506-2"}],"container-title":["Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.science.org\/doi\/pdf\/10.1126\/science.ado9336","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,14]],"date-time":"2024-11-14T14:00:22Z","timestamp":1731592822000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.science.org\/doi\/10.1126\/science.ado9336"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,15]]},"references-count":159,"journal-issue":{"issue":"6723","published-print":{"date-parts":[[2024,11,15]]}},"alternative-id":["10.1126\/science.ado9336"],"URL":"https:\/\/doi.org\/10.1126\/science.ado9336","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2024.02.27.582234","asserted-by":"object"}]},"ISSN":["0036-8075","1095-9203"],"issn-type":[{"value":"0036-8075","type":"print"},{"value":"1095-9203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,15]]},"assertion":[{"value":"2024-02-27","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-09-09","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-11-15","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"eado9336"}}