{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T12:55:55Z","timestamp":1781873755036,"version":"3.54.5"},"reference-count":31,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T00:00:00Z","timestamp":1781827200000},"content-version":"vor","delay-in-days":49,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"name":"Biological Breeding-National Science and Technology Major Project","award":["2023ZD04076"],"award-info":[{"award-number":["2023ZD04076"]}]},{"name":"Guangdong Provincial Project","award":["2024QN11N085"],"award-info":[{"award-number":["2024QN11N085"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["32472855"],"award-info":[{"award-number":["32472855"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["32441082"],"award-info":[{"award-number":["32441082"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Fund of State Key Laboratory of Swine and Poultry Breeding Industry","award":["GDNKY-ZQQZ-K6"],"award-info":[{"award-number":["GDNKY-ZQQZ-K6"]}]},{"name":"Fund of State Key Laboratory of Swine and Poultry Breeding Industry","award":["GDNKY-ZQQZ-K19"],"award-info":[{"award-number":["GDNKY-ZQQZ-K19"]}]},{"name":"Provincial Rural Revitalization Strategy Special Fund Project for Seed Industry Revival Action","award":["2024-XPY-00-015"],"award-info":[{"award-number":["2024-XPY-00-015"]}]},{"name":"Basic Research Center for Livestock and Poultry Sciences","award":["CAAS-BRC-LP-2025-01"],"award-info":[{"award-number":["CAAS-BRC-LP-2025-01"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,5,4]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>The pig (Sus scrofa) is both an economically important livestock species and a valuable biomedical model . Its genome bears regulatory features shaped by domestication and selection that are often poorly captured by genomic language models (gLMs) trained on human or model organism data. To address these challenges, we developed Porcine MutBERT, a suite of lightweight gLMs with 86 million parameters that employs a probabilistic masking strategy targeting evolutionarily informative single-nucleotide polymorphisms. This design captures population-specific variation while reducing computational cost. We further propose PorcineBench, a benchmark that evaluates gLM performance across porcine functional genomics tasks, including chromatin accessibility (ATAC-seq), CTCF binding, and histone modifications (H3K27ac, H3K4me1, and H3K27me3). Results show that Porcine MutBERT family achieves highly competitive performance on PorcineBench relative to substantially larger models, while providing an explicitly porcine-adapted alternative for downstream functional genomics in pigs. These findings underscore the advantages of species-adapted, efficient architectures in agricultural genomics and demonstrate that compact gLMs can expand accessibility and impact in resource-constrained settings. The code and data are available at https:\/\/github.com\/ai4nucleome\/pigmutbert.<\/jats:p>","DOI":"10.1093\/bib\/bbag319","type":"journal-article","created":{"date-parts":[[2026,6,1]],"date-time":"2026-06-01T11:25:18Z","timestamp":1780313118000},"source":"Crossref","is-referenced-by-count":0,"title":["Porcine MutBERT: a family of lightweight genomic foundation models for functional element prediction in pigs"],"prefix":"10.1093","volume":"27","author":[{"given":"Weicai","family":"Long","sequence":"first","affiliation":[{"name":"Data Science and Analytics Thrust, The Hong Kong University of Science and Technology (Guangzhou) , No. 1 Du Xue Road, Nansha District, Guangzhou 511453 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rong","family":"Zhou","sequence":"additional","affiliation":[{"name":"The State Key Laboratory of Animal Biotech Breeding, Institute of Animal Sciences, Chinese Academy of Agricultural Sciences , No. 2 Yuanmingyuan West Road, Haidian District, Beijing 100193 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wenkang","family":"Wei","sequence":"additional","affiliation":[{"name":"Agro-biological Gene Research Center of Guangdong Academy of Agricultural Sciences, State Key Laboratory of Swine and Poultry Breeding Industry , No. 29 Jinying Road, Tianhe District, Guangzhou 510640 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaoai","family":"Zhang","sequence":"additional","affiliation":[{"name":"Agro-biological Gene Research Center of Guangdong Academy of Agricultural Sciences, State Key Laboratory of Swine and Poultry Breeding Industry , No. 29 Jinying Road, Tianhe District, Guangzhou 510640 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shanshan","family":"Wu","sequence":"additional","affiliation":[{"name":"Shenzhen Branch, Guangdong Laboratory for Lingnan Modern Agriculture, Genomics Institute at Shenzhen, Chinese Academy of Agricultural Sciences , No. 1 Funong Road, Dapeng District, Shenzhen 518000 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kui","family":"Li","sequence":"additional","affiliation":[{"name":"Shenzhen Branch, Guangdong Laboratory for Lingnan Modern Agriculture, Genomics Institute at Shenzhen, Chinese Academy of Agricultural Sciences , No. 1 Funong Road, Dapeng District, Shenzhen 518000 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yanlin","family":"Zhang","sequence":"additional","affiliation":[{"name":"Data Science and Analytics Thrust, The Hong Kong University of Science and Technology (Guangzhou) , No. 1 Du Xue Road, Nansha District, Guangzhou 511453 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zishuai","family":"Wang","sequence":"additional","affiliation":[{"name":"Agro-biological Gene Research Center of Guangdong Academy of Agricultural Sciences, State Key Laboratory of Swine and Poultry Breeding Industry , No. 29 Jinying Road, Tianhe District, Guangzhou 510640 ,","place":["China"]},{"name":"Shenzhen Branch, Guangdong Laboratory for Lingnan Modern Agriculture, Genomics Institute at Shenzhen, Chinese Academy of Agricultural Sciences , No. 1 Funong Road, Dapeng District, Shenzhen 518000 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,6,19]]},"reference":[{"key":"2026061908190243500_ref1","doi-asserted-by":"crossref","first-page":"13","DOI":"10.34101\/actaagrar\/73\/1620","article-title":"Global and national economic importance of pig meat production","volume":"73","author":"Balogh","year":"2017","journal-title":"Acta Agrar Debrecen"},{"key":"2026061908190243500_ref2","doi-asserted-by":"crossref","first-page":"eabd5758","DOI":"10.1126\/scitranslmed.abd5758","article-title":"Importance of the pig as a human biomedical model","volume":"13","author":"Lunney","year":"2021","journal-title":"Sci Transl Med"},{"key":"2026061908190243500_ref3","doi-asserted-by":"crossref","first-page":"393","DOI":"10.1038\/nature11622","article-title":"Analyses of pig genomes provide insight into porcine demography and evolution","volume":"491","author":"Groenen","year":"2012","journal-title":"Nature"},{"key":"2026061908190243500_ref4","doi-asserted-by":"crossref","DOI":"10.1016\/j.tig.2024.11.013","article-title":"Genomic language models: opportunities and challenges","volume":"41","author":"Benegas","year":"2025","journal-title":"Trends Genet"},{"key":"2026061908190243500_ref5","doi-asserted-by":"crossref","DOI":"10.1038\/s42256-025-01007-9","article-title":"Transformers and genome language models","volume":"7","author":"Consens","year":"2025","journal-title":"Nat Mach Intell"},{"key":"2026061908190243500_ref6","doi-asserted-by":"crossref","first-page":"2112","DOI":"10.1093\/bioinformatics\/btab083","article-title":"DNABERT: pre-trained bidirectional encoder representations from transformers model for DNA-language in genome","volume":"37","author":"Ji","year":"2021","journal-title":"Bioinformatics"},{"key":"2026061908190243500_ref7","article-title":"DNABERT-2: efficient foundation model and benchmark for multi-species genomes","volume-title":"The Twelfth International Conference on Learning Representations","author":"Zhou","year":"2024"},{"key":"2026061908190243500_ref8","first-page":"43177","article-title":"HyenaDNA: long-range genomic sequence modeling at single nucleotide resolution","volume":"36","author":"Nguyen","year":"2023","journal-title":"Adv Neural Inf Proces Syst"},{"key":"2026061908190243500_ref9","article-title":"DNAGPT: a generalized pretrained tool for multiple DNA sequence analysis tasks","volume":"2023","author":"Zhang","year":"2023","journal-title":"bioRxiv"},{"key":"2026061908190243500_ref10","doi-asserted-by":"crossref","first-page":"gkae1310","DOI":"10.1093\/nar\/gkae1310","article-title":"GENA-LM: a family of open-source foundational DNA language models for long sequences","volume":"53","author":"Fishman","journal-title":"Nucleic Acids Res"},{"key":"2026061908190243500_ref11","doi-asserted-by":"crossref","first-page":"eado9336","DOI":"10.1126\/science.ado9336","article-title":"Sequence modeling and design from molecular to genome scale with evo","volume":"386","author":"Nguyen","year":"2024","journal-title":"Science"},{"key":"2026061908190243500_ref12","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1038\/s41592-024-02523-z","article-title":"Nucleotide transformer: building and evaluating robust foundation models for human genomics","volume":"22","author":"Dalla-Torre","year":"2025","journal-title":"Nat Methods"},{"key":"2026061908190243500_ref13","doi-asserted-by":"crossref","first-page":"bbae195","DOI":"10.1093\/bib\/bbae195","article-title":"BERT-TFBS: a novel BERT-based model for predicting transcription factor binding sites by transfer learning","volume":"25","author":"Wang","year":"2024","journal-title":"Brief Bioinform"},{"key":"2026061908190243500_ref14","doi-asserted-by":"crossref","first-page":"12126","DOI":"10.1073\/pnas.1714980114","article-title":"De novo prediction of human chromosome structures: epigenetic marking patterns encode genome architecture","volume":"114","author":"Di Pierro","year":"2017","journal-title":"Proc Natl Acad Sci USA"},{"key":"2026061908190243500_ref15","doi-asserted-by":"crossref","first-page":"i294","DOI":"10.1093\/bioinformatics\/btaf229","article-title":"MutBERT: probabilistic genome representation improves genomics foundation models","volume":"41","author":"Long","journal-title":"Bioinformatics"},{"key":"2026061908190243500_ref16","article-title":"Attention is all you need","volume-title":"Advances in Neural Information Processing Systems","author":"Vaswani","year":"2017"},{"key":"2026061908190243500_ref17","doi-asserted-by":"crossref","first-page":"127063","DOI":"10.1016\/j.neucom.2023.127063","article-title":"RoFormer: enhanced transformer with rotary position embedding","volume":"568","author":"Jianlin","year":"2024","journal-title":"Neurocomputing"},{"key":"2026061908190243500_ref18","doi-asserted-by":"crossref","first-page":"16344","DOI":"10.52202\/068431-1189","article-title":"FlashAttention: fast and memory-efficient exact attention with IO-awareness","volume":"35","author":"Dao","year":"2022","journal-title":"Adv Neural Inf Proces Syst"},{"key":"2026061908190243500_ref19","doi-asserted-by":"crossref","first-page":"D243","DOI":"10.1093\/nar\/gkae1038","article-title":"NCBI RefSeq: reference sequence standards through 25 years of curation and annotation","volume":"53","author":"Goldfarb","year":"2025","journal-title":"Nucleic Acids Res"},{"key":"2026061908190243500_ref20","first-page":"100954","article-title":"Genomic insights into the demographic history and local adaptation of wild boars across eurasia. Cell","volume":"5","author":"Wang","year":"2025","journal-title":"Genomics"},{"key":"2026061908190243500_ref21","article-title":"Decoupled weight decay regularization","volume-title":"International Conference on Learning Representations","author":"Loshchilov","year":"2019"},{"key":"2026061908190243500_ref22","doi-asserted-by":"crossref","first-page":"1715","DOI":"10.18653\/v1\/P16-1162","article-title":"Neural machine translation of rare words with subword units","volume-title":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Vol. 1: Long Papers)","author":"Sennrich","year":"2016"},{"key":"2026061908190243500_ref23","article-title":"Train short, test long: attention with linear biases enables input length extrapolation","volume-title":"International Conference on Learning Representations","author":"Press","year":"2022"},{"key":"2026061908190243500_ref24","first-page":"1","article-title":"PaLM: scaling language modeling with pathways","volume":"24","author":"Chowdhery","year":"2023","journal-title":"J Mach Learn Res"},{"key":"2026061908190243500_ref25","doi-asserted-by":"crossref","first-page":"3426","DOI":"10.1016\/j.cell.2022.08.004","article-title":"High-coverage whole-genome sequencing of the expanded 1000 genomes project cohort including 602 trios","volume":"185","author":"Byrska-Bishop","year":"2022","journal-title":"Cell"},{"key":"2026061908190243500_ref26","volume":"43","author":"Benegas","journal-title":"Nature Biotechnology"},{"key":"2026061908190243500_ref27","first-page":"4171","article-title":"BERT: pre-training of deep bidirectional transformers for language understanding","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Devlin","year":"2019"},{"key":"2026061908190243500_ref28","first-page":"17283","article-title":"Big bird: Transformers for longer sequences","volume-title":"Advances in Neural Information Processing Systems","author":"Zaheer","year":"2020"},{"key":"2026061908190243500_ref29","doi-asserted-by":"crossref","first-page":"5848","DOI":"10.1038\/s41467-021-26153-7","article-title":"Pig genome functional annotation enhances the biological interpretation of complex traits and human disease","volume":"12","author":"Pan","year":"2021","journal-title":"Nat Commun"},{"key":"2026061908190243500_ref30","doi-asserted-by":"crossref","first-page":"841","DOI":"10.1093\/bioinformatics\/btq033","article-title":"BEDTools: a flexible suite of utilities for comparing genomic features","volume":"26","author":"Quinlan","journal-title":"Bioinformatics"},{"key":"2026061908190243500_ref31","volume-title":"PLoS ONE","author":"Jurman"}],"container-title":["Briefings in Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/27\/3\/bbag319\/68559143\/bbag319.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/27\/3\/bbag319\/68559143\/bbag319.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T12:19:13Z","timestamp":1781871553000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bib\/article\/doi\/10.1093\/bib\/bbag319\/8711664"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5]]},"references-count":31,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2026,5,4]]}},"URL":"https:\/\/doi.org\/10.1093\/bib\/bbag319","relation":{},"ISSN":["1467-5463","1477-4054"],"issn-type":[{"value":"1467-5463","type":"print"},{"value":"1477-4054","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,5]]},"published":{"date-parts":[[2026,5]]},"article-number":"bbag319"}}