{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T17:13:43Z","timestamp":1784135623789,"version":"3.55.0"},"reference-count":62,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,2,20]],"date-time":"2024-02-20T00:00:00Z","timestamp":1708387200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,2,20]],"date-time":"2024-02-20T00:00:00Z","timestamp":1708387200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100000050","name":"National Heart, Lung, and Blood Institute","doi-asserted-by":"publisher","award":["P01HL160476"],"award-info":[{"award-number":["P01HL160476"]}],"id":[{"id":"10.13039\/100000050","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000050","name":"National Heart, Lung, and Blood Institute","doi-asserted-by":"publisher","award":["P01HL160476"],"award-info":[{"award-number":["P01HL160476"]}],"id":[{"id":"10.13039\/100000050","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000050","name":"National Heart, Lung, and Blood Institute","doi-asserted-by":"publisher","award":["P01HL160476"],"award-info":[{"award-number":["P01HL160476"]}],"id":[{"id":"10.13039\/100000050","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000005","name":"U.S. Department of Defense","doi-asserted-by":"publisher","award":["W81XWH2110606"],"award-info":[{"award-number":["W81XWH2110606"]}],"id":[{"id":"10.13039\/100000005","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000005","name":"U.S. Department of Defense","doi-asserted-by":"publisher","award":["W81XWH2110606"],"award-info":[{"award-number":["W81XWH2110606"]}],"id":[{"id":"10.13039\/100000005","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Minnesota Regenerative Medicine"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Bioinformatics"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    The breakthrough high-throughput measurement of the cis-regulatory activity of millions of randomly generated promoters provides an unprecedented opportunity to systematically decode the cis-regulatory logic that determines the expression values. We developed an end-to-end transformer encoder architecture named Proformer to predict the expression values from DNA sequences. Proformer used a Macaron-like Transformer encoder architecture, where two half-step feed forward (FFN) layers were placed at the beginning and the end of each encoder block, and a separable 1D convolution layer was inserted after the first FFN layer and in front of the multi-head attention layer. The sliding\n                    <jats:italic>k<\/jats:italic>\n                    -mers from one-hot encoded sequences were mapped onto a continuous embedding, combined with the learned positional embedding and strand embedding (forward strand vs. reverse complemented strand) as the sequence input. Moreover, Proformer introduced multiple expression heads with mask filling to prevent the transformer models from collapsing when training on relatively small amount of data. We empirically determined that this design had significantly better performance than the conventional design such as using the global pooling layer as the output layer for the regression task. These analyses support the notion that Proformer provides a novel method of learning and enhances our understanding of how cis-regulatory sequences determine the expression values.\n                  <\/jats:p>","DOI":"10.1186\/s12859-024-05645-5","type":"journal-article","created":{"date-parts":[[2024,2,20]],"date-time":"2024-02-20T04:03:20Z","timestamp":1708401800000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Proformer: a hybrid macaron transformer model predicts expression values from promoter sequences"],"prefix":"10.1186","volume":"25","author":[{"given":"Il-Youp","family":"Kwak","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Byeong-Chan","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Juhyun","family":"Lee","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Taein","family":"Kang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel J.","family":"Garry","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jianyi","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wuming","family":"Gong","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,2,20]]},"reference":[{"key":"5645_CR1","doi-asserted-by":"publisher","first-page":"831","DOI":"10.1038\/nbt.3300","volume":"33","author":"B Alipanahi","year":"2015","unstructured":"Alipanahi B, Delong A, Weirauch MT, Frey BJ. Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning. Nat Biotechnol. 2015;33:831\u20138.","journal-title":"Nat Biotechnol"},{"key":"5645_CR2","doi-asserted-by":"publisher","first-page":"329","DOI":"10.1146\/annurev.biophys.36.040306.132725","volume":"36","author":"HJ Bussemaker","year":"2007","unstructured":"Bussemaker HJ, Foat BC, Ward LD. Predictive modeling of genome-wide mRNA expression: from modules to molecules. Annu Rev Bioph Biom. 2007;36:329\u201347.","journal-title":"Annu Rev Bioph Biom"},{"key":"5645_CR3","doi-asserted-by":"publisher","first-page":"1074","DOI":"10.1126\/science.1232542","volume":"339","author":"CD Arnold","year":"2013","unstructured":"Arnold CD, Gerlach D, Stelzer C, Bory\u0144 \u0141M, Rath M, Stark A. Genome-wide quantitative enhancer activity maps identified by STARR-seq. Science. 2013;339:1074\u20137.","journal-title":"Science"},{"key":"5645_CR4","doi-asserted-by":"publisher","first-page":"521","DOI":"10.1038\/nbt.2205","volume":"30","author":"E Sharon","year":"2012","unstructured":"Sharon E, Kalma Y, Sharp A, Raveh-Sadka T, Levo M, Zeevi D, et al. Inferring gene regulatory logic from high-throughput measurements of thousands of systematically designed promoters. Nat Biotechnol. 2012;30:521\u201330.","journal-title":"Nat Biotechnol"},{"key":"5645_CR5","doi-asserted-by":"publisher","first-page":"271","DOI":"10.1038\/nbt.2137","volume":"30","author":"A Melnikov","year":"2012","unstructured":"Melnikov A, Murugan A, Zhang X, Tesileanu T, Wang L, Rogov P, et al. Systematic dissection and optimization of inducible enhancers in human cells using a massively parallel reporter assay. Nat Biotechnol. 2012;30:271\u20137.","journal-title":"Nat Biotechnol"},{"key":"5645_CR6","doi-asserted-by":"publisher","first-page":"265","DOI":"10.1038\/nbt.2136","volume":"30","author":"RP Patwardhan","year":"2012","unstructured":"Patwardhan RP, Hiatt JB, Witten DM, Kim MJ, Smith RP, May D, et al. Massively parallel functional dissection of mammalian enhancers in vivo. Nat Biotechnol. 2012;30:265\u201370.","journal-title":"Nat Biotechnol"},{"key":"5645_CR7","doi-asserted-by":"publisher","first-page":"800","DOI":"10.1101\/gr.144899.112","volume":"23","author":"P Kheradpour","year":"2013","unstructured":"Kheradpour P, Ernst J, Melnikov A, Rogov P, Wang L, Zhang X, et al. Systematic dissection of regulatory motifs in 2000 predicted human enhancers using a massively parallel reporter assay. Genome Res. 2013;23:800\u201311.","journal-title":"Genome Res."},{"key":"5645_CR8","doi-asserted-by":"publisher","first-page":"1008","DOI":"10.1101\/gr.188193.114","volume":"25","author":"S Lubliner","year":"2015","unstructured":"Lubliner S, Regev I, Lotan-Pompan M, Edelheit S, Weinberger A, Segal E. Core promoter sequence in yeast is a major determinant of expression level. Genome Res. 2015;25:1008\u201317.","journal-title":"Genome Res"},{"key":"5645_CR9","doi-asserted-by":"publisher","first-page":"325","DOI":"10.1126\/science.aac6948","volume":"350","author":"EK Farley","year":"2015","unstructured":"Farley EK, Olson KM, Zhang W, Brandt AJ, Rokhsar DS, Levine MS. Suboptimization of developmental enhancers. Science. 2015;350:325\u20138.","journal-title":"Science"},{"key":"5645_CR10","doi-asserted-by":"publisher","first-page":"1023","DOI":"10.1101\/gr.204834.116","volume":"26","author":"TA Nguyen","year":"2016","unstructured":"Nguyen TA, Jones RD, Snavely AR, Pfenning AR, Kirchner R, Hemberg M, et al. High-throughput functional comparison of promoter and enhancer activities. Genome Res. 2016;26:1023\u201333.","journal-title":"Genome Res"},{"key":"5645_CR11","doi-asserted-by":"publisher","first-page":"1021","DOI":"10.1038\/ng.2713","volume":"45","author":"RP Smith","year":"2013","unstructured":"Smith RP, Taher L, Patwardhan RP, Kim MJ, Inoue F, Shendure J, et al. Massively parallel decoding of mammalian regulatory sequences supports a flexible organizational model. Nat Genet. 2013;45:1021\u20138.","journal-title":"Nat Genet"},{"key":"5645_CR12","doi-asserted-by":"publisher","first-page":"1908","DOI":"10.1101\/gr.157891.113","volume":"23","author":"I Mogno","year":"2013","unstructured":"Mogno I, Kwasnieski JC, Cohen BA. Massively parallel synthetic promoter assays reveal the in vivo effects of binding site variants. Genome Res. 2013;23:1908\u201315.","journal-title":"Genome Res"},{"key":"5645_CR13","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1038\/nbt.3754","volume":"35","author":"J van Arensbergen","year":"2017","unstructured":"van Arensbergen J, FitzPatrick VD, de Haas M, Pagie L, Sluimer J, Bussemaker HJ, et al. Genome-wide mapping of autonomous promoter activity in human cells. Nat Biotechnol. 2017;35:145\u201353.","journal-title":"Nat Biotechnol"},{"key":"5645_CR14","doi-asserted-by":"publisher","first-page":"56","DOI":"10.1038\/s41587-019-0315-8","volume":"38","author":"CG de Boer","year":"2020","unstructured":"de Boer CG, Vaishnav ED, Sadeh R, Abeyta EL, Friedman N, Regev A. Deciphering eukaryotic gene-regulatory logic with 100 million random promoters. Nat Biotechnol. 2020;38:56\u201365.","journal-title":"Nat Biotechnol"},{"key":"5645_CR15","doi-asserted-by":"publisher","first-page":"215","DOI":"10.1038\/nature07521","volume":"457","author":"J Gertz","year":"2009","unstructured":"Gertz J, Siggia ED, Cohen BA. Analysis of combinatorial cis-regulation in synthetic and genomic promoters. Nature. 2009;457:215\u20138.","journal-title":"Nature"},{"key":"5645_CR16","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1101\/gr.236075.118","volume":"29","author":"S Weingarten-Gabbay","year":"2019","unstructured":"Weingarten-Gabbay S, Nir R, Lubliner S, Sharon E, Kalma Y, Weinberger A, et al. Systematic interrogation of human promoters. Genome Res. 2019;29:171\u201383.","journal-title":"Genome Res"},{"key":"5645_CR17","doi-asserted-by":"publisher","first-page":"E1291","DOI":"10.1073\/pnas.1621150114","volume":"114","author":"SR Grossman","year":"2017","unstructured":"Grossman SR, Zhang X, Wang L, Engreitz J, Melnikov A, Rogov P, et al. Systematic dissection of genomic features determining transcription factor binding and enhancer function. Proc Nat Acad Sci. 2017;114:E1291\u2013300.","journal-title":"Proc Nat Acad Sci"},{"key":"5645_CR18","doi-asserted-by":"publisher","first-page":"238","DOI":"10.1101\/gr.193789.115","volume":"26","author":"SQ Shen","year":"2016","unstructured":"Shen SQ, Myers CA, Hughes AEO, Byrne LC, Flannery JG, Corbo JC. Massively parallel cis-regulatory analysis in the mammalian central nervous system. Genome Res. 2016;26:238\u201355.","journal-title":"Genome Res"},{"key":"5645_CR19","doi-asserted-by":"publisher","first-page":"122","DOI":"10.1038\/s41586-019-1210-7","volume":"570","author":"V Haberle","year":"2019","unstructured":"Haberle V, Arnold CD, Pagani M, Rath M, Schernhuber K, Stark A. Transcriptional cofactors display specificity for distinct types of core promoters. Nature. 2019;570:122\u20136.","journal-title":"Nature"},{"key":"5645_CR20","doi-asserted-by":"publisher","first-page":"1083","DOI":"10.1038\/s41592-020-0965-y","volume":"17","author":"JC Klein","year":"2020","unstructured":"Klein JC, Agarwal V, Inoue F, Keith A, Martin B, Kircher M, et al. A systematic evaluation of the design and context dependencies of massively parallel reporter assays. Nat Methods. 2020;17:1083\u201391.","journal-title":"Nat Methods"},{"key":"5645_CR21","doi-asserted-by":"publisher","first-page":"1206","DOI":"10.1101\/gr.190090.115","volume":"25","author":"CM Vockley","year":"2015","unstructured":"Vockley CM, Guo C, Majoros WH, Nodzenski M, Scholtens DM, Hayes MG, et al. Massively parallel quantification of the regulatory effects of noncoding genetic variation in a human cohort. Genome Res. 2015;25:1206\u201314.","journal-title":"Genome Res"},{"key":"5645_CR22","doi-asserted-by":"publisher","first-page":"1519","DOI":"10.1016\/j.cell.2016.04.027","volume":"165","author":"R Tewhey","year":"2016","unstructured":"Tewhey R, Kotliar D, Park DS, Liu B, Winnicki S, Reilly SK, et al. Direct identification of hundreds of expression-modulating variants using a multiplexed reporter assay. Cell. 2016;165:1519\u201329.","journal-title":"Cell"},{"key":"5645_CR23","doi-asserted-by":"publisher","first-page":"1530","DOI":"10.1016\/j.cell.2016.04.048","volume":"165","author":"JC Ulirsch","year":"2016","unstructured":"Ulirsch JC, Nandakumar SK, Wang L, Giani FC, Zhang X, Rogov P, et al. Systematic functional dissection of common genetic variation affecting red blood cell traits. Cell. 2016;165:1530\u201345.","journal-title":"Cell"},{"key":"5645_CR24","doi-asserted-by":"publisher","first-page":"194","DOI":"10.1186\/s13059-017-1322-z","volume":"18","author":"S Liu","year":"2017","unstructured":"Liu S, Liu Y, Zhang Q, Wu J, Liang J, Yu S, et al. Systematic identification of regulatory variants associated with cancer risk. Genome Biol. 2017;18:194.","journal-title":"Genome Biol"},{"key":"5645_CR25","doi-asserted-by":"publisher","first-page":"455","DOI":"10.1038\/s41586-022-04506-6","volume":"603","author":"ED Vaishnav","year":"2022","unstructured":"Vaishnav ED, de Boer CG, Molinet J, Yassour M, Fan L, Adiconis X, et al. The evolution, evolvability and engineering of gene regulatory DNA. Nature. 2022;603:455\u201363.","journal-title":"Nature"},{"key":"5645_CR26","doi-asserted-by":"publisher","first-page":"26","DOI":"10.1186\/s13059-023-02856-6","volume":"24","author":"IG Romero","year":"2023","unstructured":"Romero IG, Lea AJ. Leveraging massively parallel reporter assays for evolutionary questions. Genome Biol. 2023;24:26.","journal-title":"Genome Biol"},{"key":"5645_CR27","doi-asserted-by":"publisher","first-page":"1595","DOI":"10.1101\/gr.173518.114","volume":"24","author":"JC Kwasnieski","year":"2014","unstructured":"Kwasnieski JC, Fiore C, Chaudhari HG, Cohen BA. High-throughput functional testing of ENCODE segmentation predictions. Genome Res. 2014;24:1595\u2013602.","journal-title":"Genome Res"},{"key":"5645_CR28","doi-asserted-by":"publisher","first-page":"52","DOI":"10.1164\/rccm.201802-0337OC","volume":"199","author":"PJ Castaldi","year":"2018","unstructured":"Castaldi PJ, Guo F, Qiao D, Du F, Naing ZZC, Li Y, et al. Identification of functional variants in the FAM13A chronic obstructive pulmonary disease genome-wide association study locus by massively parallel reporter assays. Am J Resp Crit Care. 2018;199:52\u201361.","journal-title":"Am J Resp Crit Care"},{"key":"5645_CR29","unstructured":"Shen SQ, Kim-Han JS, Cheng L, Xu D, Gokcumen O, Hughes AEO, et al. A candidate causal variant underlying both enhanced cognitive performance and increased risk of bipolar disorder. Biorxiv. 2021; 580258"},{"key":"5645_CR30","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1186\/s13059-018-1473-6","volume":"19","author":"JC Klein","year":"2018","unstructured":"Klein JC, Keith A, Agarwal V, Durham T, Shendure J. Functional characterization of enhancer evolution in the primate lineage. Genome Biol. 2018;19:99.","journal-title":"Genome Biol"},{"key":"5645_CR31","doi-asserted-by":"publisher","first-page":"685","DOI":"10.1038\/ng.3009","volume":"46","author":"CD Arnold","year":"2014","unstructured":"Arnold CD, Gerlach D, Spies D, Matts JA, Sytnikova YA, Pagani M, et al. Quantitative genome-wide enhancer activity maps for five Drosophila species show functional enhancer conservation and turnover during cis-regulatory evolution. Nat Genet. 2014;46:685\u201392.","journal-title":"Nat Genet"},{"key":"5645_CR32","doi-asserted-by":"publisher","first-page":"931","DOI":"10.1038\/nmeth.3547","volume":"12","author":"J Zhou","year":"2015","unstructured":"Zhou J, Troyanskaya OG. Predicting effects of noncoding variants with deep learning\u2013based sequence model. Nat Methods. 2015;12:931\u20134.","journal-title":"Nat Methods"},{"key":"5645_CR33","doi-asserted-by":"publisher","first-page":"990","DOI":"10.1101\/gr.200535.115","volume":"26","author":"DR Kelley","year":"2016","unstructured":"Kelley DR, Snoek J, Rinn JL. Basset: learning the regulatory code of the accessible genome with deep convolutional neural networks. Genome Res. 2016;26:990\u20139.","journal-title":"Genome Res."},{"key":"5645_CR34","doi-asserted-by":"publisher","first-page":"739","DOI":"10.1101\/gr.227819.117","volume":"28","author":"DR Kelley","year":"2018","unstructured":"Kelley DR, Reshef YA, Bileschi M, Belanger D, McLean CY, Snoek J. Sequential regulatory activity prediction across chromosomes with convolutional neural networks. Genome Res. 2018;28:739\u201350.","journal-title":"Genome Res"},{"key":"5645_CR35","doi-asserted-by":"crossref","unstructured":"Dsouza N, Gong W, Garry DJ. SeATAC: a tool for exploring the chromatin landscape and the role of pioneer factors. Biorxiv. 2022:2022.04.25.489439","DOI":"10.1101\/2022.04.25.489439"},{"key":"5645_CR36","doi-asserted-by":"publisher","first-page":"gkx177","DOI":"10.1093\/nar\/gkx177","volume":"45","author":"H Zeng","year":"2017","unstructured":"Zeng H, Gifford DK. Predicting the impact of non-coding variants on DNA methylation. Nucleic Acids Res. 2017;45:gkx177.","journal-title":"Nucleic Acids Res."},{"key":"5645_CR37","doi-asserted-by":"publisher","first-page":"67","DOI":"10.1186\/s13059-017-1189-z","volume":"18","author":"C Angermueller","year":"2017","unstructured":"Angermueller C, Lee HJ, Reik W, Stegle O. DeepCpG: accurate prediction of single-cell DNA methylation states using deep learning. Genome Biol. 2017;18:67.","journal-title":"Genome Biol"},{"key":"5645_CR38","doi-asserted-by":"publisher","first-page":"511","DOI":"10.1186\/s12864-018-4889-1","volume":"19","author":"X Pan","year":"2018","unstructured":"Pan X, Rijnbeek P, Yan J, Shen H-B. Prediction of RNA-protein sequence and structure binding preferences using deep convolutional and recurrent neural networks. BMC Genom. 2018;19:511.","journal-title":"BMC Genom"},{"key":"5645_CR39","doi-asserted-by":"publisher","first-page":"3035","DOI":"10.1093\/bioinformatics\/bty222","volume":"34","author":"S Budach","year":"2018","unstructured":"Budach S, Marsico A. Pysster: classification of biological sequences by learning sequence and structure motifs with convolutional neural networks. Bioinformatics. 2018;34:3035\u20137.","journal-title":"Bioinformatics"},{"key":"5645_CR40","doi-asserted-by":"publisher","first-page":"1261","DOI":"10.1093\/bioinformatics\/btx727","volume":"34","author":"\u017d Avsec","year":"2018","unstructured":"Avsec \u017d, Barekatain M, Cheng J, Gagneur J. Modeling positional effects of regulatory sequences with spline transformations increases prediction accuracy of deep neural networks. Bioinformatics. 2018;34:1261\u20139.","journal-title":"Bioinformatics"},{"key":"5645_CR41","doi-asserted-by":"publisher","first-page":"1171","DOI":"10.1038\/s41588-018-0160-6","volume":"50","author":"J Zhou","year":"2018","unstructured":"Zhou J, Theesfeld CL, Yao K, Chen KM, Wong AK, Troyanskaya OG. Deep learning sequence-based ab initio prediction of variant effects on expression and disease risk. Nat Genet. 2018;50:1171\u20139.","journal-title":"Nat Genet"},{"key":"5645_CR42","doi-asserted-by":"publisher","first-page":"e107","DOI":"10.1093\/nar\/gkw226","volume":"44","author":"D Quang","year":"2016","unstructured":"Quang D, Xie X. DanQ: a hybrid convolutional and recurrent deep neural network for quantifying the function of DNA sequences. Nucl Acids Res. 2016;44:e107\u2013e107.","journal-title":"Nucl Acids Res"},{"key":"5645_CR43","doi-asserted-by":"publisher","first-page":"bbaa159","DOI":"10.1093\/bib\/bbaa159","volume":"22","author":"J Li","year":"2020","unstructured":"Li J, Pu Y, Tang J, Zou Q, Guo F. DeepATT: a hybrid category attention neural network for identifying functional effects of DNA sequences. Brief Bioinform. 2020;22:bbaa159.","journal-title":"Brief Bioinform."},{"key":"5645_CR44","doi-asserted-by":"publisher","first-page":"1113402","DOI":"10.3389\/fdata.2023.1113402","volume":"6","author":"K Ding","year":"2023","unstructured":"Ding K, Dixit G, Parker BJ, Wen J. CRMnet: A deep learning model for predicting gene expression from large regulatory sequence datasets. Front Big Data. 2023;6:1113402.","journal-title":"Front Big Data"},{"key":"5645_CR45","doi-asserted-by":"crossref","unstructured":"Rafi AM. Evaluation and optimization of sequence-based gene regulatory deep learning models. 2023.","DOI":"10.1101\/2023.04.26.538471"},{"key":"5645_CR46","doi-asserted-by":"publisher","unstructured":"Rafi AM, Penzar D, Nogina D, Lee D, Vaishnav ED, Lee D, et al. Evaluation and optimization of sequence-based gene regulatory deep learning models. 2023. https:\/\/doi.org\/10.1101\/2023.04.26.538471","DOI":"10.1101\/2023.04.26.538471"},{"key":"5645_CR47","doi-asserted-by":"publisher","first-page":"15270","DOI":"10.1038\/s41598-018-33321-1","volume":"8","author":"Z Shen","year":"2018","unstructured":"Shen Z, Bao W, Huang D-S. Recurrent neural network for predicting transcription factor binding sites. Sci Rep. 2018;8:15270.","journal-title":"Sci Rep"},{"key":"5645_CR48","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention is all you need. arXiv.org. 2017;cs.CL."},{"key":"5645_CR49","doi-asserted-by":"crossref","unstructured":"Press O, Smith NA, Levy O. Improving transformer models by reordering their sublayers. Arxiv. 2019.","DOI":"10.18653\/v1\/2020.acl-main.270"},{"key":"5645_CR50","unstructured":"Lu Y, Li Z, He D, Sun Z, Dong B, Qin T, et al. Understanding and improving transformer from a multi-particle dynamic system point of view. Arxiv. 2019."},{"key":"5645_CR51","doi-asserted-by":"publisher","unstructured":"Lu Y, Li Z, He D, Sun Z, Dong B, Qin T, et al. Understanding and improving transformer from a multi-particle dynamic system point of view. Arxiv. 2019. https:\/\/doi.org\/10.48550\/arxiv.1906.02762.","DOI":"10.48550\/arxiv.1906.02762"},{"key":"5645_CR52","doi-asserted-by":"crossref","unstructured":"Gulati A, Qin J, Chiu C-C, Parmar N, Zhang Y, Yu J, et al. Conformer: convolution-augmented transformer for speech recognition. Arxiv. 2020.","DOI":"10.21437\/Interspeech.2020-3015"},{"key":"5645_CR53","doi-asserted-by":"crossref","unstructured":"Pak M, Kim S. A Review of deep learning in image recognition. In: 2017 4th international conference on computer applications and information processing technology Caipt. 2017; 1\u20133.","DOI":"10.1109\/CAIPT.2017.8320684"},{"key":"5645_CR54","doi-asserted-by":"publisher","first-page":"1213","DOI":"10.1038\/nmeth.2688","volume":"10","author":"JD Buenrostro","year":"2013","unstructured":"Buenrostro JD, Giresi PG, Zaba LC, Chang HY, Greenleaf WJ. Transposition of native chromatin for fast and sensitive epigenomic profiling of open chromatin, DNA-binding proteins and nucleosome position. Nat Methods. 2013;10:1213\u20138.","journal-title":"Nat Methods"},{"key":"5645_CR55","unstructured":"Dauphin YN, Fan A, Auli M, Grangier D. Language modeling with gated convolutional networks. Arxiv. 2016"},{"key":"5645_CR56","doi-asserted-by":"publisher","unstructured":"Liu S, Zhu Z, Qu Q, You C. Robust training under label noise by over-parameterization. Arxiv. 2022. https:\/\/doi.org\/10.48550\/arxiv.2202.14026","DOI":"10.48550\/arxiv.2202.14026"},{"key":"5645_CR57","doi-asserted-by":"publisher","first-page":"9158","DOI":"10.1073\/pnas.1004290107","volume":"107","author":"JB Kinney","year":"2010","unstructured":"Kinney JB, Murugan A, Callan CG, Cox EC. Using deep sequencing to characterize the biophysical mechanism of a transcriptional regulatory sequence. Proc National Acad Sci. 2010;107:9158\u201363.","journal-title":"Proc National Acad Sci"},{"key":"5645_CR58","doi-asserted-by":"publisher","first-page":"357","DOI":"10.1038\/nmeth.1923","volume":"9","author":"B Langmead","year":"2012","unstructured":"Langmead B, Salzberg SL. Fast gapped-read alignment with Bowtie 2. Nat Methods. 2012;9:357\u20139.","journal-title":"Nat Methods"},{"key":"5645_CR59","doi-asserted-by":"publisher","first-page":"9354","DOI":"10.1038\/s41598-019-45839-z","volume":"9","author":"HM Amemiya","year":"2019","unstructured":"Amemiya HM, Kundaje A, Boyle AP. The ENCODE blacklist: identification of problematic regions of the genome. Sci Rep-uk. 2019;9:9354.","journal-title":"Sci Rep-uk"},{"key":"5645_CR60","doi-asserted-by":"publisher","first-page":"R137","DOI":"10.1186\/gb-2008-9-9-r137","volume":"9","author":"Y Zhang","year":"2008","unstructured":"Zhang Y, Liu T, Meyer CA, Eeckhoute J, Johnson DS, Bernstein BE, et al. Model-based analysis of ChIP-Seq (MACS). Genome Biol. 2008;9:R137.","journal-title":"Genome Biol"},{"key":"5645_CR61","doi-asserted-by":"publisher","first-page":"1728","DOI":"10.1038\/nprot.2012.101","volume":"7","author":"J Feng","year":"2012","unstructured":"Feng J, Liu T, Qin B, Zhang Y, Liu XS. Identifying ChIP-seq enrichment using MACS. Nat Protoc. 2012;7:1728\u201340.","journal-title":"Nat Protoc"},{"key":"5645_CR62","unstructured":"Gong W. Proformer: a hybrid Macaron transformer model predicts expression values from promoter sequences. In: 14th annual RECOMB\/ISCB conference on regulatory and systems genomics with DREAM challenges, RSGDREAM 2022"}],"container-title":["BMC Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-024-05645-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12859-024-05645-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-024-05645-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,2,20]],"date-time":"2024-02-20T04:04:30Z","timestamp":1708401870000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/s12859-024-05645-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,20]]},"references-count":62,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["5645"],"URL":"https:\/\/doi.org\/10.1186\/s12859-024-05645-5","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2023.03.10.532129","asserted-by":"object"}]},"ISSN":["1471-2105"],"issn-type":[{"value":"1471-2105","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,20]]},"assertion":[{"value":"24 May 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 January 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 February 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"DJG is the founder of NorthStar Genomics. The remaining authors declare no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"81"}}