{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T03:02:38Z","timestamp":1791428558890,"version":"4.3.3"},"reference-count":34,"publisher":"Oxford University Press (OUP)","issue":"9","license":[{"start":{"date-parts":[[2025,8,18]],"date-time":"2025-08-18T00:00:00Z","timestamp":1755475200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"NIH HPC Biowulf cluster"},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["2138259"],"award-info":[{"award-number":["2138259"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["2138286"],"award-info":[{"award-number":["2138286"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["2138307"],"award-info":[{"award-number":["2138307"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["2137603"],"award-info":[{"award-number":["2137603"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["2138296"],"award-info":[{"award-number":["2138296"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,9,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Motivation<\/jats:title>\n                    <jats:p>Genomic language models have recently emerged as a new method to decode, interpret, and generate genetic sequences. Existing genomic language models have utilized various tokenization methods, including character tokenization, overlapping and nonoverlapping k-mer tokenization, and byte-pair encoding, a method widely used in natural language models. Genomic sequences differ from natural language because of their low character variability, complex and overlapping features, and inconsistent directionality. These features make subword tokenization in genomic language models significantly different from both traditional language models and protein language models.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>This study explores the impact of tokenization in genomic language models by evaluating their downstream performance on 44 classification fine-tuning tasks. We also perform a direct comparison of byte pair encoding and character tokenization in Mamba, a state-space model. Our results indicate that character tokenization outperforms subword tokenization methods on tasks that rely on nucleotide-level resolution, such as splice site prediction and promoter detection. While byte-pair tokenization had stronger performance on the SARS-CoV-2 variant classification task, we observed limited statistically significant differences between tokenization methods on the remaining downstream tasks.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Availability and implementation<\/jats:title>\n                    <jats:p>Detailed results of all benchmarking experiments are available in https:\/\/github.com\/leannmlindsey\/DNAtokenization. Training datasets and pretrained models are available at https:\/\/huggingface.co\/datasets\/leannmlindsey. Datasets and processing scripts are available at doi: 10.5281\/zenodo.16287401 and doi: 10.5281\/zenodo.16287130.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btaf456","type":"journal-article","created":{"date-parts":[[2025,8,14]],"date-time":"2025-08-14T11:59:03Z","timestamp":1755172743000},"source":"Crossref","is-referenced-by-count":12,"title":["The impact of tokenizer selection in genomic language models"],"prefix":"10.1093","volume":"41","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0515-0118","authenticated-orcid":false,"given":"LeAnn M","family":"Lindsey","sequence":"first","affiliation":[{"name":"Kahlert School of Computing, University of Utah , Salt Lake City, Utah 84112,","place":["United States"]},{"name":"National Library of Medicine, National Institutes of Health , Bethesda, Maryland 20892,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nicole L","family":"Pershing","sequence":"additional","affiliation":[{"name":"Department of Pediatrics, School of Medicine, University of Utah , Salt Lake City, Utah 84112,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anisa","family":"Habib","sequence":"additional","affiliation":[{"name":"Kahlert School of Computing, University of Utah , Salt Lake City, Utah 84112,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0991-2255","authenticated-orcid":false,"given":"Keith","family":"Dufault-Thompson","sequence":"additional","affiliation":[{"name":"National Library of Medicine, National Institutes of Health , Bethesda, Maryland 20892,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8072-9023","authenticated-orcid":false,"given":"W Zac","family":"Stephens","sequence":"additional","affiliation":[{"name":"Department of Pathology, School of Medicine, University of Utah , Salt Lake City, Utah 84112,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anne J","family":"Blaschke","sequence":"additional","affiliation":[{"name":"Department of Pediatrics, School of Medicine, University of Utah , Salt Lake City, Utah 84112,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0955-8284","authenticated-orcid":false,"given":"Xiaofang","family":"Jiang","sequence":"additional","affiliation":[{"name":"National Library of Medicine, National Institutes of Health , Bethesda, Maryland 20892,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hari","family":"Sundar","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Tufts University , Medford, MA 02115,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2025,8,18]]},"reference":[{"key":"2025092219523056500_btaf456-B1","author":"Batsuren","year":"2024"},{"key":"2025092219523056500_btaf456-B2","doi-asserted-by":"publisher","first-page":"16","DOI":"10.4204\/eptcs.388.4","author":"Berglund","year":"2023"},{"key":"2025092219523056500_btaf456-B3","doi-asserted-by":"publisher","first-page":"4617","DOI":"10.18653\/v1\/2020.findings-emnlp.414","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2020","author":"Bostrom","year":"2020"},{"key":"2025092219523056500_btaf456-B4","doi-asserted-by":"publisher","first-page":"1582","DOI":"10.18653\/v1\/2024.findings-emnlp.86","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2024","author":"Chai","year":"2024"},{"key":"2025092219523056500_btaf456-B5","author":"Dalla-Torre","year":"2024"},{"key":"2025092219523056500_btaf456-B6","doi-asserted-by":"publisher","first-page":"btae196","DOI":"10.1093\/bioinformatics\/btae196","article-title":"Effect of tokenization on transformers for biological sequences","volume":"40","author":"Dotan","year":"2024","journal-title":"Bioinformatics"},{"key":"2025092219523056500_btaf456-B7","doi-asserted-by":"publisher","first-page":"192","DOI":"10.1214\/aoms\/1177706374","article-title":"Estimation of the medians for dependent variables","volume":"30","author":"Dunn","year":"1959","journal-title":"Ann Math Statist"},{"key":"2025092219523056500_btaf456-B8","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1186\/s12863-023-01123-8","article-title":"Genomic benchmarks: a collection of datasets for genomic sequence classification","volume":"24","author":"Gre\u0161ov\u00e1","year":"2023","journal-title":"BMC Genom Data"},{"key":"2025092219523056500_btaf456-B9","author":"Gu","year":"2024"},{"key":"2025092219523056500_btaf456-B10","doi-asserted-by":"publisher","first-page":"D884","DOI":"10.1093\/nar\/gkaa942","article-title":"Ensembl 2021","volume":"49","author":"Howe","year":"2021","journal-title":"Nucleic Acids Res"},{"key":"2025092219523056500_btaf456-B11","doi-asserted-by":"crossref","first-page":"2112","DOI":"10.1093\/bioinformatics\/btab083","article-title":"DNABERT: pre-trained bidirectional encoder representations from transformers model for DNA-language in genome","volume":"37","author":"Ji","year":"2021","journal-title":"Bioinformatics"},{"key":"2025092219523056500_btaf456-B12","doi-asserted-by":"publisher","first-page":"3165","DOI":"10.1093\/nar\/gki627","article-title":"Structural properties of promoters: similarities and differences between prokaryotes and eukaryotes","volume":"33","author":"Kanhere","year":"2005","journal-title":"Nucleic Acids Res"},{"key":"2025092219523056500_btaf456-B13","doi-asserted-by":"crossref","first-page":"10227","DOI":"10.1073\/pnas.94.19.10227","article-title":"Compositional differences within and between eukaryotic\u2009genomes","volume":"94","author":"Karlin","year":"1997","journal-title":"Proc Natl Acad Sci USA"},{"key":"2025092219523056500_btaf456-B14"},{"key":"2025092219523056500_btaf456-B15","doi-asserted-by":"publisher","first-page":"66","DOI":"10.18653\/v1\/P18-1007","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Kudo","year":"2018"},{"key":"2025092219523056500_btaf456-B16","volume-title":"Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS \u201923, Red Hook, NY, USA","author":"Nguyen","year":"2023"},{"key":"2025092219523056500_btaf456-B17","doi-asserted-by":"publisher","first-page":"959","DOI":"10.1109\/BIBE.2007.4375674","author":"Pham","year":"2007"},{"key":"2025092219523056500_btaf456-B18","author":"Rajaraman","year":"2024"},{"key":"2025092219523056500_btaf456-B19","doi-asserted-by":"publisher","first-page":"D174","DOI":"10.1093\/nar\/gkad1059","article-title":"JASPAR 2024: 20th anniversary of the open-access database of transcription factor binding profiles","volume":"52","author":"Rauluseviciute","year":"2024","journal-title":"Nucleic Acids Res"},{"key":"2025092219523056500_btaf456-B20","doi-asserted-by":"publisher","first-page":"3118","DOI":"10.18653\/v1\/2021.acl-long.243","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Rust","year":"2021"},{"key":"2025092219523056500_btaf456-B21","doi-asserted-by":"publisher","first-page":"43632","DOI":"10.48550\/arXiv.2403.03234","article-title":"Caduceus: bi-directional equivariant long-range DNA sequence modeling","volume":"235","author":"Schiff","year":"2024","journal-title":"Proc Mach Learn Res"},{"key":"2025092219523056500_btaf456-B22","doi-asserted-by":"publisher","first-page":"678","DOI":"10.18653\/v1\/2024.emnlp-main.40","volume-title":"Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing","author":"Schmidt","year":"2024"},{"key":"2025092219523056500_btaf456-B23","doi-asserted-by":"publisher","first-page":"5149","DOI":"10.1109\/ICASSP.2012.6289079","author":"Schuster","year":"2012"},{"key":"2025092219523056500_btaf456-B24","doi-asserted-by":"publisher","first-page":"1715","DOI":"10.18653\/v1\/P16-1162","volume-title":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Sennrich","year":"2016"},{"key":"2025092219523056500_btaf456-B25","doi-asserted-by":"publisher","first-page":"1147","DOI":"10.1145\/3503161.3548208","volume-title":"Proceedings of the 30th ACM International Conference on Multimedia, MM \u201922","author":"Shi","year":"2022"},{"key":"2025092219523056500_btaf456-B26","author":"Singh"},{"key":"2025092219523056500_btaf456-B27","doi-asserted-by":"publisher","first-page":"613","DOI":"10.1038\/nrg3207","article-title":"Transcription factors: from enhancer binding to developmental control","volume":"13","author":"Spitz","year":"2012","journal-title":"Nat Rev Genet"},{"key":"2025092219523056500_btaf456-B28","doi-asserted-by":"publisher","author":"Sun","DOI":"10.18653\/v1\/2023.findings-emnlp.887"},{"key":"2025092219523056500_btaf456-B29","doi-asserted-by":"publisher","first-page":"652","DOI":"10.1186\/s12859-019-3306-3","article-title":"SpliceFinder: ab initio prediction of splice sites using convolutional neural network","volume":"20","author":"Wang","year":"2019","journal-title":"BMC Bioinformatics"},{"key":"2025092219523056500_btaf456-B30","doi-asserted-by":"publisher","author":"Wolf","DOI":"10.18653\/v1\/2020.emnlp-demos.6"},{"key":"2025092219523056500_btaf456-B31","doi-asserted-by":"publisher","first-page":"291","DOI":"10.1162\/tacl_a_00461","author":"Xue","year":"2022"},{"key":"2025092219523056500_btaf456-B32","doi-asserted-by":"publisher","first-page":"52","DOI":"10.1016\/j.gene.2006.09.029","article-title":"Prevalence of the initiator over the TATA box in human and yeast genes and identification of DNA motifs enriched in human TATA-less core promoters","volume":"389","author":"Yang","year":"2007","journal-title":"Gene"},{"key":"2025092219523056500_btaf456-B33","doi-asserted-by":"publisher","first-page":"56","DOI":"10.1186\/s13059-015-0621-5","article-title":"The ensembl regulatory build","volume":"16","author":"Zerbino","year":"2015","journal-title":"Genome Biol"},{"key":"2025092219523056500_btaf456-B34","author":"Zhou","year":"2024"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btaf456\/64076938\/btaf456.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/41\/9\/btaf456\/64076938\/btaf456.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/41\/9\/btaf456\/64076938\/btaf456.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,22]],"date-time":"2025-09-22T23:52:39Z","timestamp":1758585159000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/doi\/10.1093\/bioinformatics\/btaf456\/8237360"}},"subtitle":[],"editor":[{"given":"Macha","family":"Nikolski","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2025,8,18]]},"references-count":34,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2025,9,1]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btaf456","relation":{},"ISSN":["1367-4811"],"issn-type":[{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,9]]},"published":{"date-parts":[[2025,8,18]]},"article-number":"btaf456"}}