{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T16:03:27Z","timestamp":1787328207857,"version":"build-2736575974"},"reference-count":40,"publisher":"Oxford University Press (OUP)","license":[{"start":{"date-parts":[[2026,7,21]],"date-time":"2026-07-21T00:00:00Z","timestamp":1784592000000},"content-version":"vor","delay-in-days":201,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1736030"],"award-info":[{"award-number":["1736030"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["2025567"],"award-info":[{"award-number":["2025567"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100006094","name":"University of Delaware","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100006094","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["P20GM103446"],"award-info":[{"award-number":["P20GM103446"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["S10OD028725"],"award-info":[{"award-number":["S10OD028725"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,1,21]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:label>Aa<\/jats:label>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>The transformer architecture in deep learning has revolutionized protein sequence analysis. Recent advancements in protein language models have paved the way for significant progress across various domains, including protein function and structure prediction, multiple sequence alignments, and mutation effect prediction. A protein language model is commonly trained on individual proteins, ignoring the interdependencies between sequences within a genome. However, biological understanding reveals that protein\u2013protein interactions span entire genomic regions, underscoring the limitations of focusing solely on individual proteins.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:label>Ab<\/jats:label>\n                    <jats:title>Findings<\/jats:title>\n                    <jats:p>To address these limitations, we propose a novel approach that extends the context size of transformer models across the entire viral genome. By training on large genomic fragments, our method captures\u00a0putative long-range dependencies consistent with inter-protein relationships and encodes protein sequences with integrated information from distant proteins within the same genome, offering benefits across downstream tasks. Viruses, with their densely packed genomes, minimal intergenic regions, and protein annotation challenges, are ideal candidates for genome-wide learning. We introduce a long-context protein language model, trained on entire viral genomes, leveraging a biologically informed sparse attention mechanism in which inter-protein links are inferred computationally and used as sparsity priors. Our semi-supervised approach supports long sequences of up to 61,000 amino acids (aa).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:label>Ac<\/jats:label>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Our evaluations show improved prediction of masked aa and improved downstream discrimination relative to single-protein models and long-context baselines, with additional validation that our inferred links correlate with independently curated interaction resources.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/gigascience\/giag081","type":"journal-article","created":{"date-parts":[[2026,7,16]],"date-time":"2026-07-16T11:45:20Z","timestamp":1784202320000},"source":"Crossref","is-referenced-by-count":1,"title":["Extending protein language models to a viral genomic scale using biologically induced sparse attention"],"prefix":"10.1093","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-2394-6283","authenticated-orcid":false,"given":"Thibaut","family":"Dejean","sequence":"first","affiliation":[{"name":"Department of Information and Computer Sciences, University of Hawai`i , Honolulu, HI 96822 ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3057-3478","authenticated-orcid":false,"given":"Barbra D","family":"Ferrell","sequence":"additional","affiliation":[{"name":"Department of Computer and Information Sciences, Delaware Biotechnology Institute, University of Delaware , Newark, DE 19716 ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2935-6863","authenticated-orcid":false,"given":"William","family":"Harrigan","sequence":"additional","affiliation":[{"name":"Hawai`i Institute of Marine Biology, University of Hawai`i , K\u0101ne`ohe, HI 96744 ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6271-2754","authenticated-orcid":false,"given":"Zachary D","family":"Schreiber","sequence":"additional","affiliation":[{"name":"Department of Computer and Information Sciences, Delaware Biotechnology Institute, University of Delaware , Newark, DE 19716 ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-0060-759X","authenticated-orcid":false,"given":"Rajan","family":"Sawhney","sequence":"additional","affiliation":[{"name":"Department of Information and Computer Sciences, University of Hawai`i , Honolulu, HI 96822 ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5797-7128","authenticated-orcid":false,"given":"K Eric","family":"Wommack","sequence":"additional","affiliation":[{"name":"Department of Plant and Soil Sciences, Delaware Biotechnology Institute, University of Delaware , Newark, DE 19716 ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3398-6932","authenticated-orcid":false,"given":"Shawn W","family":"Polson","sequence":"additional","affiliation":[{"name":"Department of Computer and Information Sciences, Delaware Biotechnology Institute, University of Delaware , Newark, DE 19716 ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-5185-0643","authenticated-orcid":false,"given":"Mahdi","family":"Belcaid","sequence":"additional","affiliation":[{"name":"Department of Information and Computer Sciences, University of Hawai`i , Honolulu, HI 96822 ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,7,21]]},"reference":[{"key":"2026082110572106000_bib1","doi-asserted-by":"publisher","first-page":"103994","DOI":"10.48550\/arXiv.1701.08318","article-title":"Deep recurrent neural network for protein function prediction from sequence","author":"Liu","year":"2017"},{"key":"2026082110572106000_bib2","doi-asserted-by":"publisher","first-page":"e82819","DOI":"10.7554\/eLife.82819","article-title":"Transformer-based deep learning for predicting protein properties in the life sciences","volume":"12","author":"Chandra","year":"2023","journal-title":"eLife"},{"key":"2026082110572106000_bib3","doi-asserted-by":"publisher","first-page":"500902","DOI":"10.1101\/2022.07.20.500902","article-title":"Language models of protein sequences at the scale of evolution enable accurate structure prediction","author":"Lin","year":"2022"},{"issue":"10","key":"2026082110572106000_bib4","doi-asserted-by":"publisher","first-page":"7112","DOI":"10.1109\/TPAMI.2021.3095381","article-title":"ProtTrans: toward understanding the language of life through self-supervised learning","volume":"44","author":"Elnaggar","year":"2021","journal-title":"IEEE T Pattern Anal Mach Intell"},{"issue":"11","key":"2026082110572106000_bib5","doi-asserted-by":"publisher","first-page":"968","DOI":"10.1016\/j.cels.2023.10.002","article-title":"ProGen2: exploring the boundaries of protein language models","volume":"14","author":"Nijkamp","year":"2023","journal-title":"Cell Syst"},{"key":"2026082110572106000_bib6","doi-asserted-by":"publisher","DOI":"10.1101\/2022.10.21.513099","article-title":"Vector-clustering Multiple Sequence Alignment: aligning into the twilight zone of protein sequence similarity with protein language models","volume-title":"2022-10","author":"McWhite","year":"2022"},{"key":"2026082110572106000_bib7","doi-asserted-by":"publisher","first-page":"165","DOI":"10.1186\/s12859-024-05779-6","article-title":"Improvements in viral gene annotation using large language models and soft alignments","volume":"25","author":"Harrigan","year":"2024","journal-title":"BMC Bioinformatics"},{"issue":"1","key":"2026082110572106000_bib8","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1186\/s12859-024-05699-5","article-title":"Protein embedding based alignment","volume":"25","author":"Iovino","year":"2024","journal-title":"BMC Bioinformatics"},{"key":"2026082110572106000_bib9","doi-asserted-by":"publisher","first-page":"1123","DOI":"10.1101\/2022.07.20.500902","article-title":"Evolutionary-scale prediction of atomic level protein structure with a language model","author":"Lin","year":"2022"},{"issue":"2","key":"2026082110572106000_bib10","doi-asserted-by":"publisher","first-page":"e0149166","DOI":"10.1371\/journal.pone.0149166","article-title":"Inter-protein sequence co-evolution predicts known physical interactions in bacterial ribosomes and the trp operon","volume":"11","author":"Feinauer","year":"2016","journal-title":"PLoS One"},{"issue":"8","key":"2026082110572106000_bib11","doi-asserted-by":"publisher","first-page":"1269","DOI":"10.1101\/gr.277567.122","article-title":"Ultra-long-range interactions between active regulatory elements","volume":"33","author":"Friman","year":"2023","journal-title":"Genome Res"},{"issue":"1","key":"2026082110572106000_bib12","doi-asserted-by":"publisher","first-page":"140","DOI":"10.1186\/s13578-021-00644-y","article-title":"Genome-wide analysis of protein\u2013protein interactions and involvement of viral proteins in SARS-CoV-2 replication","volume":"11","author":"Jiang","year":"2021","journal-title":"Cell Biosci"},{"key":"2026082110572106000_bib13","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2402.02244","article-title":"Beyond the limits: a survey of techniques to extend the context length in large language models","author":"Wang","year":"2024"},{"key":"2026082110572106000_bib14","article-title":"Attention is all you need","volume-title":"Advances in Neural Information Processing Systems, 30","author":"Vaswani","year":"2017"},{"issue":"D1","key":"2026082110572106000_bib15","doi-asserted-by":"publisher","first-page":"D571","DOI":"10.1093\/nar\/gku1207","article-title":"NCBI viral genomes resource","volume":"43","author":"Brister","year":"2015","journal-title":"Nucleic Acids Res"},{"key":"2026082110572106000_bib16","doi-asserted-by":"publisher","first-page":"127063","DOI":"10.1016\/j.neucom.2023.127063","article-title":"RoFormer: enhanced transformer with rotary position embedding","volume":"568","author":"Su","year":"2024","journal-title":"Neurocomputing"},{"key":"2026082110572106000_bib17","article-title":"Train short, test long: attention with linear biases enables input length extrapolation","volume-title":"International Conference on Learning Representations (ICLR)","author":"Press","year":"2022"},{"key":"2026082110572106000_bib18","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2306.15595","article-title":"Extending context window of large language models via positional interpolation","author":"Chen","year":"2023"},{"key":"2026082110572106000_bib19","doi-asserted-by":"crossref","first-page":"16344","DOI":"10.52202\/068431-1189","article-title":"FlashAttention: fast and memory-efficient exact attention with IO-awareness","volume":"35","author":"Dao","year":"2022","journal-title":"Adv Neur Inf Proc Syst"},{"key":"2026082110572106000_bib20","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1145\/3315508.3329973","article-title":"Triton: an intermediate language and compiler for tiled neural network computations","volume-title":"Proceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages","author":"Tillet","year":"2019"},{"key":"2026082110572106000_bib21","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2004.05150","article-title":"Longformer: the long-document transformer","author":"Beltagy","year":"2020"},{"key":"2026082110572106000_bib22","article-title":"LongLoRA: efficient fine-tuning of long-context large language models","volume-title":"International Conference on Learning Representations (ICLR)","author":"Chen","year":"2024"},{"key":"2026082110572106000_bib23","first-page":"17283","article-title":"Big Bird: transformers for longer sequences","volume":"33","author":"Zaheer","year":"2020","journal-title":"Adv Neur Inf Proc Syst"},{"key":"2026082110572106000_bib24","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1162\/tacl_a_00353","article-title":"Efficient content-based sparse attention with routing transformers","volume":"9","author":"Roy","year":"2021","journal-title":"Trans Assoc Compu Linguistics"},{"issue":"8","key":"2026082110572106000_bib25","doi-asserted-by":"publisher","first-page":"2102","DOI":"10.1093\/bioinformatics\/btac020","article-title":"ProteinBERT: a universal deep-learning model of protein sequence and function","volume":"38","author":"Brandes","year":"2022","journal-title":"Bioinformatics"},{"issue":"6","key":"2026082110572106000_bib26","doi-asserted-by":"publisher","first-page":"926","DOI":"10.1093\/bioinformatics\/btu739","article-title":"UniRef clusters: a comprehensive and scalable alternative for improving sequence similarity searches","volume":"31","author":"Suzek","year":"2015","journal-title":"Bioinformatics"},{"key":"2026082110572106000_bib27","doi-asserted-by":"publisher","first-page":"e19919","DOI":"10.7717\/peerj.19919","article-title":"Fine-tuning protein language models unlocks the potential of underrepresented viral proteomes","volume":"13","author":"Sawhney","year":"2025","journal-title":"PeerJ"},{"key":"2026082110572106000_bib28","doi-asserted-by":"publisher","DOI":"10.1101\/2024.07.26.605391","article-title":"Protein set transformer: a protein-based genome language model to power high diversity viromics","author":"Martin","year":"2024"},{"issue":"1","key":"2026082110572106000_bib29","doi-asserted-by":"publisher","first-page":"2880","DOI":"10.1038\/s41467-024-46947-9","article-title":"Genomic language model predicts protein co-regulation and function","volume":"15","author":"Hwang","year":"2024","journal-title":"Nat Commun"},{"issue":"15","key":"2026082110572106000_bib30","doi-asserted-by":"publisher","first-page":"e2016239118","DOI":"10.1073\/pnas.2016239118","article-title":"Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences","volume":"118","author":"Rives","year":"2021","journal-title":"Proc Natl Acad Sci USA"},{"key":"2026082110572106000_bib31","doi-asserted-by":"publisher","first-page":"858366","DOI":"10.3389\/fmicb.2022.858366","article-title":"Novel viral DNA polymerases from metagenomes suggest genomic sources of strand-displacing biochemical phenotypes","volume":"13","author":"Keown","year":"2022","journal-title":"Front Microbiol"},{"issue":"7","key":"2026082110572106000_bib32","doi-asserted-by":"publisher","first-page":"917","DOI":"10.1016\/j.chom.2022.06.008","article-title":"The logic of virus evolution","volume":"30","author":"Koonin","year":"2022","journal-title":"Cell Host Microbe"},{"key":"2026082110572106000_bib33","doi-asserted-by":"publisher","first-page":"3053","DOI":"10.3389\/fmicb.2018.03053","article-title":"Family A DNA polymerase phylogeny uncovers diversity and replication gene organization in the virioplankton","volume":"9","author":"Nasko","year":"2018","journal-title":"Front Microbiol"},{"key":"2026082110572106000_bib34","article-title":"torch.sparse_bsr_tensor \u2014 PyTorch 2.6 documentation"},{"issue":"D1","key":"2026082110572106000_bib35","doi-asserted-by":"publisher","first-page":"D638","DOI":"10.1093\/nar\/gkac1000","article-title":"The STRING database in 2023: protein\u2013protein association networks and functional enrichment analyses for any sequenced genome of interest","volume":"51","author":"Szklarczyk","year":"2023","journal-title":"Nucleic Acids Res"},{"issue":"16","key":"2026082110572106000_bib36","doi-asserted-by":"publisher","first-page":"2856","DOI":"10.1093\/bioinformatics\/bty1057","article-title":"MMseqs2 desktop and local web server app for fast, interactive sequence searches","volume":"35","author":"Mirdita","year":"2019","journal-title":"Bioinformatics"},{"key":"2026082110572106000_bib37","doi-asserted-by":"publisher","first-page":"eado9336","DOI":"10.1126\/science.ado9336","article-title":"Sequence modeling and design from molecular to genome scale with Evo","volume-title":"Science","author":"Nguyen","year":"2024"},{"key":"2026082110572106000_bib38","doi-asserted-by":"crossref","DOI":"10.1093\/gigascience\/giag081","article-title":"Extending protein language models to a viral genomic scale using biologically induced sparse attention","author":"Dejean","year":"2026"},{"key":"2026082110572106000_bib39","doi-asserted-by":"crossref","unstructured":"Li X, Hua C, Yue Q \u00a0et al. \u00a0GigaDB: a redesigned repository for data publishing and management. GigaScience. 2026; 15: giag047. 10.1093\/gigascience\/giag047.","DOI":"10.1093\/gigascience\/giag047"},{"key":"2026082110572106000_bib40","doi-asserted-by":"crossref","unstructured":"Dejean T, Ferrell B D, Harrigan W \u00a0et al. \u00a0Supporting data for \u201cextending protein language models to a viral genomic scale using biologically induced sparse attention\u201d. GigaScience Database; 2026. 10.5524\/102826.","DOI":"10.1093\/gigascience\/giag081"}],"container-title":["GigaScience"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/gigascience\/advance-article-pdf\/doi\/10.1093\/gigascience\/giag081\/69722681\/giag081.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/gigascience\/article-pdf\/doi\/10.1093\/gigascience\/giag081\/69722681\/giag081.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/gigascience\/article-pdf\/doi\/10.1093\/gigascience\/giag081\/69722681\/giag081.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T14:57:29Z","timestamp":1787324249000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/gigascience\/article\/doi\/10.1093\/gigascience\/giag081\/8739457"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026]]},"references-count":40,"URL":"https:\/\/doi.org\/10.1093\/gigascience\/giag081","relation":{},"ISSN":["2047-217X"],"issn-type":[{"value":"2047-217X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026]]},"published":{"date-parts":[[2026]]},"article-number":"giag081"}}