{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"institution":[{"name":"bioRxiv"}],"indexed":{"date-parts":[[2026,1,15]],"date-time":"2026-01-15T19:49:49Z","timestamp":1768506589541,"version":"3.49.0"},"posted":{"date-parts":[[2023,6,26]]},"group-title":"Bioinformatics","reference-count":56,"publisher":"openRxiv","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"accepted":{"date-parts":[[2023,6,26]]},"abstract":"<jats:title>ABSTRACT<\/jats:title>\n                <jats:p>The tremendous rate with which data is generated and analysis methods emerge makes it increasingly difficult to keep track of their domain of applicability, assumptions, and limitations and consequently, of the efficacy and precision with which they solve specific tasks. Therefore, there is an increasing need for benchmarks, and for the provision of infrastructure for continuous method evaluation. APAeval is an international community effort, organized by the RNA Society in 2021, to benchmark tools for the identification and quantification of the usage of alternative polyadenylation (APA) sites from short-read, bulk RNA-sequencing (RNA-seq) data. Here, we reviewed 17 tools and benchmarked eight on their ability to perform APA identification and quantification, using a comprehensive set of RNA-seq experiments comprising real, synthetic, and matched 3\u2032-end sequencing data. To support continuous benchmarking, we have incorporated the results into the OpenEBench online platform, which allows for seamless extension of the set of methods, metrics, and challenges. We envisage that our analyses will assist researchers in selecting the appropriate tools for their studies. Furthermore, the containers and reproducible workflows generated in the course of this project can be seamlessly deployed and extended in the future to evaluate new methods or datasets.<\/jats:p>","DOI":"10.1101\/2023.06.23.546284","type":"posted-content","created":{"date-parts":[[2023,6,26]],"date-time":"2023-06-26T18:15:14Z","timestamp":1687803314000},"source":"Crossref","is-referenced-by-count":1,"title":["Extensible benchmarking of methods that identify and quantify polyadenylation sites from RNA-seq data"],"prefix":"10.64898","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2087-5050","authenticated-orcid":false,"given":"Sam","family":"Bryce-Smith","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8131-9309","authenticated-orcid":false,"given":"Dominik","family":"Burri","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7710-4551","authenticated-orcid":false,"given":"Matthew R.","family":"Gazzara","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1649-3463","authenticated-orcid":false,"given":"Christina J.","family":"Herrmann","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0957-7477","authenticated-orcid":false,"given":"Weronika","family":"Danecka","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6743-2671","authenticated-orcid":false,"given":"Christina M.","family":"Fitzsimmons","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1774-261X","authenticated-orcid":false,"given":"Yuk Kei","family":"Wan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6391-7225","authenticated-orcid":false,"given":"Farica","family":"Zhuang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4108-4218","authenticated-orcid":false,"given":"Mervin M.","family":"Fansler","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4806-5140","authenticated-orcid":false,"given":"Jos\u00e9 M.","family":"Fern\u00e1ndez","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Meritxell","family":"Ferret","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4159-6096","authenticated-orcid":false,"given":"Asier","family":"Gonzalez-Uriarte","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Samuel","family":"Haynes","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9531-3725","authenticated-orcid":false,"given":"Chelsea","family":"Herdman","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3468-0652","authenticated-orcid":false,"given":"Alexander","family":"Kanitz","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1961-0223","authenticated-orcid":false,"given":"Maria","family":"Katsantoni","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3252-7758","authenticated-orcid":false,"given":"Federico","family":"Marini","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Euan","family":"McDonnel","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0065-7729","authenticated-orcid":false,"given":"Ben","family":"Nicolet","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6298-7099","authenticated-orcid":false,"given":"Chi-Lam","family":"Poon","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gregor","family":"Rot","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0234-7609","authenticated-orcid":false,"given":"Leonard","family":"Sch\u00e4rfen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7158-1429","authenticated-orcid":false,"given":"Pin-Jou","family":"Wu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6062-0979","authenticated-orcid":false,"given":"Yoseop","family":"Yoon","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3005-5048","authenticated-orcid":false,"given":"Yoseph","family":"Barash","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8832-2041","authenticated-orcid":false,"given":"Mihaela","family":"Zavolan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"54368","reference":[{"key":"2023062815551401000_2023.06.23.546284v1.1","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/bty110"},{"key":"2023062815551401000_2023.06.23.546284v1.2","doi-asserted-by":"crossref","unstructured":"Capella-Gutierrez, S. , Iglesia, D. de la , Haas, J. , Lourenco, A. , Fern\u00e1ndez, J. M. , Repchevsky, D. , Dessimoz, C. , Schwede, T. , Notredame, C. , Gelpi, J. L. , & Valencia, A. (2017). Lessons Learned: Recommendations for Establishing Critical Periodic Scientific Benchmarking (p. 181677). bioRxiv. https:\/\/doi.org\/10.1101\/181677","DOI":"10.1101\/181677"},{"issue":"4","key":"2023062815551401000_2023.06.23.546284v1.3","doi-asserted-by":"crossref","first-page":"393","DOI":"10.1016\/j.cels.2019.07.011","article-title":"MountainClimber Identifies Alternative Transcription Start and Polyadenylation Sites in RNA-Seq","volume":"9","year":"2019","journal-title":"Cell Systems"},{"key":"2023062815551401000_2023.06.23.546284v1.4","doi-asserted-by":"publisher","DOI":"10.1093\/bib\/bbz068"},{"key":"2023062815551401000_2023.06.23.546284v1.5","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btx192"},{"key":"2023062815551401000_2023.06.23.546284v1.6","doi-asserted-by":"publisher","DOI":"10.1101\/gr.132563.111"},{"key":"2023062815551401000_2023.06.23.546284v1.7","doi-asserted-by":"publisher","DOI":"10.1038\/nbt.3820"},{"key":"2023062815551401000_2023.06.23.546284v1.8","doi-asserted-by":"publisher","DOI":"10.1038\/nrg3482"},{"issue":"3","key":"2023062815551401000_2023.06.23.546284v1.9","first-page":"396","article-title":"APA-Scan: Detection and visualization of 3\u2032-UTR alternative polyadenylation with RNA-seq and 3\u2032-end-seq data","volume":"23","year":"2022","journal-title":"BMC Bioinformatics"},{"key":"2023062815551401000_2023.06.23.546284v1.10","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkx892"},{"key":"2023062815551401000_2023.06.23.546284v1.11","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2008.11.029"},{"key":"2023062815551401000_2023.06.23.546284v1.12","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkaa1087"},{"issue":"1","key":"2023062815551401000_2023.06.23.546284v1.13","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1186\/s12859-021-04114-7","article-title":"Streamlining differential exon and 3\u2032 UTR usage with diffUTR","volume":"22","year":"2021","journal-title":"BMC Bioinformatics"},{"key":"2023062815551401000_2023.06.23.546284v1.14","doi-asserted-by":"publisher","DOI":"10.1186\/s12864-021-07781-1"},{"key":"2023062815551401000_2023.06.23.546284v1.15","doi-asserted-by":"publisher","DOI":"10.1186\/s12859-016-1254-8"},{"key":"2023062815551401000_2023.06.23.546284v1.16","doi-asserted-by":"publisher","DOI":"10.1186\/s13059-018-1415-3"},{"issue":"10","key":"2023062815551401000_2023.06.23.546284v1.17","doi-asserted-by":"crossref","first-page":"599","DOI":"10.1038\/s41576-019-0145-z","article-title":"Alternative cleavage and polyadenylation in health and disease","volume":"20","year":"2019","journal-title":"Nature Reviews. Genetics"},{"key":"2023062815551401000_2023.06.23.546284v1.18","doi-asserted-by":"publisher","DOI":"10.1186\/s13059-018-1414-4"},{"key":"2023062815551401000_2023.06.23.546284v1.19","doi-asserted-by":"crossref","first-page":"182","DOI":"10.3389\/fgene.2019.00182","article-title":"Detection of Differentially Expressed Cleavage Site Intervals Within 3\u2032 Untranslated Regions Using CSI-UTR Reveals Regulated Interaction Motifs","volume":"10","year":"2019","journal-title":"Frontiers in Genetics"},{"key":"2023062815551401000_2023.06.23.546284v1.20","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkz918"},{"issue":"2","key":"2023062815551401000_2023.06.23.546284v1.21","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1038\/nmeth.2288","article-title":"Analysis of alternative cleavage and polyadenylation by 3\u2032 region extraction and deep sequencing","volume":"10","year":"2013","journal-title":"Nature Methods"},{"issue":"2","key":"2023062815551401000_2023.06.23.546284v1.22","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1016\/j.celrep.2016.03.023","article-title":"PAPERCLIP Identifies MicroRNA Targets and a Role of CstF64\/64tau in Promoting Non-canonical poly(A) Site Usage","volume":"15","year":"2016","journal-title":"Cell Reports"},{"key":"2023062815551401000_2023.06.23.546284v1.23","doi-asserted-by":"publisher","DOI":"10.1038\/nature09616"},{"key":"2023062815551401000_2023.06.23.546284v1.24","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0900028106"},{"key":"2023062815551401000_2023.06.23.546284v1.25","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0900028106"},{"issue":"12","key":"2023062815551401000_2023.06.23.546284v1.26","doi-asserted-by":"crossref","first-page":"1009","DOI":"10.1038\/nmeth.1528","article-title":"Analysis and design of RNA sequencing experiments for identifying isoform regulation","volume":"7","year":"2010","journal-title":"Nature Methods"},{"key":"2023062815551401000_2023.06.23.546284v1.27","doi-asserted-by":"publisher","DOI":"10.1016\/j.ymeth.2015.04.011"},{"key":"2023062815551401000_2023.06.23.546284v1.28","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2105-12-323"},{"key":"2023062815551401000_2023.06.23.546284v1.29","doi-asserted-by":"publisher","DOI":"10.1101\/gad.229328.113"},{"issue":"1","key":"2023062815551401000_2023.06.23.546284v1.30","doi-asserted-by":"crossref","first-page":"1652","DOI":"10.1038\/s41467-021-21894-x","article-title":"Aptardi predicts polyadenylation sites in sample-specific transcriptomes using high-throughput RNA sequencing and DNA sequence","volume":"12","year":"2021","journal-title":"Nature Communications"},{"issue":"6","key":"2023062815551401000_2023.06.23.546284v1.31","doi-asserted-by":"crossref","first-page":"753","DOI":"10.1016\/j.celrep.2012.05.003","article-title":"Genome-wide analysis of pre-mRNA 3\u2032 end processing reveals a decisive role of human cleavage factor I in the regulation of 3\u2032 UTR length","volume":"1","year":"2012","journal-title":"Cell Reports"},{"key":"2023062815551401000_2023.06.23.546284v1.32","first-page":"e56129","article-title":"3\u2032 End Sequencing Library Preparation with A-seq2","volume":"128","year":"2017","journal-title":"JoVE (Journal of Visualized Experiments)"},{"key":"2023062815551401000_2023.06.23.546284v1.33","doi-asserted-by":"crossref","first-page":"33","DOI":"10.12688\/f1000research.29032.2","article-title":"Sustainable data analysis with Snakemake","volume":"10","year":"2021","journal-title":"F1000Research"},{"key":"2023062815551401000_2023.06.23.546284v1.34","doi-asserted-by":"publisher","DOI":"10.1158\/1078-0432.CCR-12-0543"},{"key":"2023062815551401000_2023.06.23.546284v1.35","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1016\/bs.mie.2021.03.022","article-title":"TRENDseq\u2014A highly multiplexed high throughput RNA 3\u2032 end sequencing for mapping alternative polyadenylation","volume":"655","year":"2021","journal-title":"In Methods in Enzymology"},{"key":"2023062815551401000_2023.06.23.546284v1.36","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkv1189"},{"key":"2023062815551401000_2023.06.23.546284v1.37","unstructured":"Patel, H. , Ewels, P. , Peltzer, A. , Hammar\u00e9n, R. , Botvinnik, O. , Sturm, G. , Moreno, D. , Vemuri, P. , silviamorins, Pantano, L. , Binzer-Panchal, M. , BABS-STP1, bot, nf-core, FriederikeHanssen, Garcia, M. U. , Yates, J. A. F. , Cheshire, C. , rfenouil, Espinosa-Carrasco, J., \u2026 Hall, G. (2022). nf-core\/rnaseq: Nf-core\/rnaseq v3.8.1 - Plastered Magnesium Mongoose. Zenodo. https:\/\/doi.org\/10.5281\/zenodo.6587789"},{"key":"2023062815551401000_2023.06.23.546284v1.38","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btq033"},{"key":"2023062815551401000_2023.06.23.546284v1.39","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1016\/j.ymeth.2017.06.003","article-title":"Genome-wide profiling of the 3\u2032 ends of polyadenylated RNAs","volume":"126","year":"2017","journal-title":"Methods (San Diego, Calif.)"},{"key":"2023062815551401000_2023.06.23.546284v1.40","doi-asserted-by":"publisher","DOI":"10.1186\/s13059-021-02298-y"},{"key":"2023062815551401000_2023.06.23.546284v1.41","doi-asserted-by":"publisher","DOI":"10.1186\/s13059-021-02502-z"},{"key":"2023062815551401000_2023.06.23.546284v1.42","doi-asserted-by":"publisher","DOI":"10.1261\/rna.046037.114"},{"key":"2023062815551401000_2023.06.23.546284v1.43","doi-asserted-by":"publisher","DOI":"10.1261\/rna.2581711"},{"key":"2023062815551401000_2023.06.23.546284v1.44","doi-asserted-by":"publisher","DOI":"10.1016\/j.molmed.2021.04.006"},{"key":"2023062815551401000_2023.06.23.546284v1.45","unstructured":"The GTex Consortium. (2020). The GTEx Consortium atlas of genetic regulatory effects across human tissues."},{"key":"2023062815551401000_2023.06.23.546284v1.46","doi-asserted-by":"publisher","DOI":"10.1038\/nrm.2016.116"},{"key":"2023062815551401000_2023.06.23.546284v1.47","doi-asserted-by":"crossref","unstructured":"Vaquero-Garcia, J. , Aicher, J. K. , Jewell, S. , Gazzara, M. R. , Radens, C. M. , Jha, A. , Norton, S. S. , Lahens, N. F. , Grant, G. R. , & Barash, Y. (2023). RNA splicing analysis using heterogeneous and large RNA-seq datasets. Nature Communications, 14(1), Article 1. https:\/\/doi.org\/10.1038\/s41467-023-36585-y","DOI":"10.1038\/s41467-023-36585-y"},{"key":"2023062815551401000_2023.06.23.546284v1.48","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btaa266"},{"issue":"1","key":"2023062815551401000_2023.06.23.546284v1.49","doi-asserted-by":"crossref","first-page":"5274","DOI":"10.1038\/ncomms6274","article-title":"Dynamic analyses of alternative polyadenylation from RNA-seq reveal a 3\u2032-UTR landscape across seven tumour types","volume":"5","year":"2014","journal-title":"Nature Communications"},{"key":"2023062815551401000_2023.06.23.546284v1.50","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/bty029"},{"key":"2023062815551401000_2023.06.23.546284v1.51","doi-asserted-by":"crossref","unstructured":"Ye, W. , Lian, Q. , Ye, C. , & Wu, X . (2022). A Survey on Methods for Predicting Polyadenylation Sites from DNA Sequences, Bulk RNA-seq, and Single-cell RNA-seq. Genomics, Proteomics & Bioinformatics. https:\/\/doi.org\/10.1016\/j.gpb.2022.09.005","DOI":"10.1016\/j.gpb.2022.09.005"},{"key":"2023062815551401000_2023.06.23.546284v1.52","doi-asserted-by":"crossref","first-page":"e46711","DOI":"10.7554\/eLife.46711","article-title":"Embryo polarity in moth flies and mosquitoes relies on distinct old genes with localized transcript isoforms","volume":"8","year":"2019","journal-title":"ELife"},{"key":"2023062815551401000_2023.06.23.546284v1.53","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1016\/bs.mie.2021.03.013","article-title":"PAS-seq 2: A fast and sensitive method for global profiling of polyadenylated RNAs","volume":"655","year":"2021","journal-title":"Methods in Enzymology"},{"key":"2023062815551401000_2023.06.23.546284v1.54","doi-asserted-by":"publisher","DOI":"10.4161\/epi.26931"},{"key":"2023062815551401000_2023.06.23.546284v1.55","doi-asserted-by":"publisher","DOI":"10.1261\/rna.057075.116"},{"key":"2023062815551401000_2023.06.23.546284v1.56","doi-asserted-by":"publisher","DOI":"10.1534\/genetics.116.188508"}],"container-title":[],"original-title":[],"link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.1101\/2023.06.23.546284","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,15]],"date-time":"2026-01-15T04:50:13Z","timestamp":1768452613000},"score":1,"resource":{"primary":{"URL":"http:\/\/biorxiv.org\/lookup\/doi\/10.1101\/2023.06.23.546284"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,26]]},"references-count":56,"URL":"https:\/\/doi.org\/10.1101\/2023.06.23.546284","relation":{"is-preprint-of":[{"id-type":"doi","id":"10.1261\/rna.079849.123","asserted-by":"subject"}]},"subject":[],"published":{"date-parts":[[2023,6,26]]},"subtype":"preprint"}}