{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,26]],"date-time":"2026-03-26T03:44:44Z","timestamp":1774496684667,"version":"3.50.1"},"reference-count":23,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2020,10,15]],"date-time":"2020-10-15T00:00:00Z","timestamp":1602720000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,10,15]],"date-time":"2020-10-15T00:00:00Z","timestamp":1602720000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"NIH \/ NIAID","award":["AI134808"],"award-info":[{"award-number":["AI134808"]}]},{"DOI":"10.13039\/100012173","name":"ASBMT","doi-asserted-by":"crossref","award":["New Investigator Award"],"award-info":[{"award-number":["New Investigator Award"]}],"id":[{"id":"10.13039\/100012173","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Bioinformatics"],"published-print":{"date-parts":[[2020,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>High-throughput sequencing can establish the functional capacity of a microbial community by cataloging the protein-coding sequences (CDS) present in the metagenome of the community. The relative performance of different computational methods for identifying CDS from whole-genome shotgun sequencing is not fully established.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Here we present an automated benchmarking workflow, using synthetic shotgun sequencing reads for which we know the true CDS content of the underlying communities, to determine the relative performance (sensitivity, positive predictive value or PPV, and computational efficiency) of different metagenome analysis tools for extracting the CDS content of a microbial community. Assembly-based methods are limited by coverage depth, with poor sensitivity for CDS at &lt;\u20095X depth of sequencing, but have excellent PPV. Mapping-based techniques are more sensitive at low coverage depths, but can struggle with PPV. We additionally describe an expectation maximization based iterative algorithmic approach which we show to successfully improve the PPV of a mapping based technique while retaining improved sensitivity and computational efficiency.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Our benchmarking approach reveals the trade-offs of assembly versus alignment-based approaches and the relative performance of specific implementations when one wishes to extract the protein coding capacity of microbial communities.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1186\/s12859-020-03802-0","type":"journal-article","created":{"date-parts":[[2020,10,15]],"date-time":"2020-10-15T06:03:36Z","timestamp":1602741816000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["In silico benchmarking of metagenomic tools for coding sequence detection reveals the limits of sensitivity and precision"],"prefix":"10.1186","volume":"21","author":[{"given":"Jonathan Louis","family":"Golob","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1639-3905","authenticated-orcid":false,"given":"Samuel Schwartz","family":"Minot","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2020,10,15]]},"reference":[{"key":"3802_CR1","doi-asserted-by":"publisher","first-page":"2317","DOI":"10.1101\/gr.096651.109","volume":"19","author":"J Peterson","year":"2009","unstructured":"NIH HMP Working Group, Peterson J, Garges S, Giovanni M, McInnes P, Wang L, et al. The NIH human microbiome project. Genome Res. 2009;19:2317\u201323.","journal-title":"Genome Res"},{"key":"3802_CR2","doi-asserted-by":"publisher","first-page":"59","DOI":"10.1038\/nature08821","volume":"464","author":"J Qin","year":"2010","unstructured":"Qin J, Li R, Raes J, Arumugam M, Burgdorf KS, Manichanh C, et al. A human gut microbial gene catalogue established by metagenomic sequencing. Nature. 2010;464:59\u201365.","journal-title":"Nature"},{"key":"3802_CR3","doi-asserted-by":"publisher","first-page":"260","DOI":"10.1038\/nrg3182","volume":"13","author":"I Cho","year":"2012","unstructured":"Cho I, Blaser MJ. The human microbiome: at the interface of health and disease. Nat Rev Genet. 2012;13:260\u201370.","journal-title":"Nat Rev Genet"},{"key":"3802_CR4","doi-asserted-by":"publisher","first-page":"207","DOI":"10.1038\/nature11234","volume":"486","author":"Human Microbiome Project Consortium","year":"2012","unstructured":"Human Microbiome Project Consortium. Structure, function and diversity of the healthy human microbiome. Nature. 2012a;486:207\u201314.","journal-title":"Nature"},{"key":"3802_CR5","doi-asserted-by":"publisher","first-page":"215","DOI":"10.1038\/nature11209","volume":"486","author":"Human Microbiome Project Consortium","year":"2012","unstructured":"Human Microbiome Project Consortium. A framework for human microbiome research. Nature. 2012b;486:215\u201321.","journal-title":"Nature"},{"key":"3802_CR6","doi-asserted-by":"publisher","first-page":"283","DOI":"10.1186\/s12859-017-1690-0","volume":"18","author":"JL Golob","year":"2017","unstructured":"Golob JL, Margolis E, Hoffman NG, Fredricks DN. Evaluating the accuracy of amplicon-based microbiome computational pipelines on simulated human gut microbial communities. BMC Bioinform. 2017;18:283.","journal-title":"BMC Bioinform"},{"key":"3802_CR7","doi-asserted-by":"publisher","first-page":"7537","DOI":"10.1128\/AEM.01541-09","volume":"75","author":"PD Schloss","year":"2009","unstructured":"Schloss PD, Westcott SL, Ryabin T, Hall JR, Hartmann M, Hollister EB, et al. Introducing mothur: open-source, platform-independent, community-supported software for describing and comparing microbial communities. Appl Environ Microbiol. 2009;75:7537\u201341.","journal-title":"Appl Environ Microbiol"},{"key":"3802_CR8","doi-asserted-by":"publisher","first-page":"335","DOI":"10.1038\/nmeth.f.303","volume":"7","author":"JG Caporaso","year":"2010","unstructured":"Caporaso JG, Kuczynski J, Stombaugh J, Bittinger K, Bushman FD, Costello EK, et al. QIIME allows analysis of high-throughput community sequencing data. Nat Methods. 2010;7:335\u20136.","journal-title":"Nat Methods"},{"key":"3802_CR9","doi-asserted-by":"publisher","first-page":"833","DOI":"10.1038\/nbt.3935","volume":"35","author":"C Quince","year":"2017","unstructured":"Quince C, Walker AW, Simpson JT, Loman NJ, Segata N. Shotgun metagenomics, from sampling to analysis. Nat Biotechnol. 2017;35:833\u201344.","journal-title":"Nat Biotechnol"},{"key":"3802_CR10","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1016\/j.copbio.2011.11.013","volume":"23","author":"MB Scholz","year":"2012","unstructured":"Scholz MB, Lo C-C, Chain PSG. Next generation sequencing and bioinformatic bottlenecks: the current state of metagenomic data analysis. Curr Opin Biotechnol. 2012;23:9\u201315.","journal-title":"Curr Opin Biotechnol"},{"key":"3802_CR11","doi-asserted-by":"publisher","first-page":"e1002358","DOI":"10.1371\/journal.pcbi.1002358","volume":"8","author":"S Abubucker","year":"2012","unstructured":"Abubucker S, Segata N, Goll J, Schubert AM, Izard J, Cantarel BL, et al. Metabolic reconstruction for metagenomic data and its application to the human microbiome. PLoS Comput Biol. 2012;8:e1002358.","journal-title":"PLoS Comput Biol"},{"key":"3802_CR12","doi-asserted-by":"publisher","first-page":"1612","DOI":"10.1101\/gr.201863.115","volume":"26","author":"S Nayfach","year":"2016","unstructured":"Nayfach S, Rodriguez-Mueller B, Garud N, Pollard KS. An integrated metagenomics pipeline for strain profiling reveals novel patterns of bacterial transmission and biogeography. Genome Res. 2016;26:1612\u201325.","journal-title":"Genome Res"},{"key":"3802_CR13","doi-asserted-by":"publisher","first-page":"455","DOI":"10.1089\/cmb.2012.0021","volume":"19","author":"A Bankevich","year":"2012","unstructured":"Bankevich A, Nurk S, Antipov D, Gurevich AA, Dvorkin M, Kulikov AS, et al. SPAdes: a new genome assembly algorithm and its applications to single-cell sequencing. J Comput Biol. 2012;19:455\u201377.","journal-title":"J Comput Biol"},{"key":"3802_CR14","doi-asserted-by":"publisher","first-page":"1420","DOI":"10.1093\/bioinformatics\/bts174","volume":"28","author":"Y Peng","year":"2012","unstructured":"Peng Y, Leung HCM, Yiu SM, Chin FYL. IDBA-UD: a de novo assembler for single-cell and metagenomic sequencing data with highly uneven depth. Bioinformatics. 2012;28:1420\u20138.","journal-title":"Bioinformatics"},{"key":"3802_CR15","doi-asserted-by":"publisher","first-page":"227","DOI":"10.1016\/S0168-9525(00)02005-9","volume":"16","author":"WM Fitch","year":"2000","unstructured":"Fitch WM. Homology a personal view on some of the problems. Trends Genet. 2000;16:227\u201331.","journal-title":"Trends Genet"},{"key":"3802_CR16","doi-asserted-by":"publisher","first-page":"182","DOI":"10.1186\/s13059-017-1299-7","volume":"18","author":"ABR McIntyre","year":"2017","unstructured":"McIntyre ABR, Ounit R, Afshinnekoo E, Prill RJ, H\u00e9naff E, Alexander N, et al. Comprehensive benchmarking and ensemble approaches for metagenomic classifiers. Genome Biol. 2017;18:182.","journal-title":"Genome Biol"},{"key":"3802_CR17","doi-asserted-by":"publisher","first-page":"1063","DOI":"10.1038\/nmeth.4458","volume":"14","author":"A Sczyrba","year":"2017","unstructured":"Sczyrba A, Hofmann P, Belmann P, Koslicki D, Janssen S, Dr\u00f6ge J, et al. Critical assessment of metagenome interpretation-a benchmark of metagenomics software. Nat Methods. 2017;14:1063\u201371.","journal-title":"Nat Methods"},{"key":"3802_CR18","doi-asserted-by":"publisher","first-page":"19233","DOI":"10.1038\/srep19233","volume":"6","author":"S Lindgreen","year":"2016","unstructured":"Lindgreen S, Adair KL, Gardner PP. An evaluation of the accuracy and speed of metagenome analysis tools. Sci Rep. 2016;6:19233.","journal-title":"Sci Rep"},{"key":"3802_CR19","doi-asserted-by":"publisher","first-page":"e0176469","DOI":"10.1371\/journal.pone.0176469","volume":"12","author":"TN Petersen","year":"2017","unstructured":"Petersen TN, Lukjancenko O, Thomsen MCF, Maddalena Sperotto M, Lund O, M\u00f8ller Aarestrup F, et al. MGmapper: reference based mapping and taxonomy annotation of metagenomics sequence reads. PLoS ONE. 2017;12:e0176469.","journal-title":"PLoS ONE"},{"key":"3802_CR20","doi-asserted-by":"publisher","first-page":"316","DOI":"10.1038\/nbt.3820","volume":"35","author":"P Di Tommaso","year":"2017","unstructured":"Di Tommaso P, Chatzou M, Floden EW, Barja PP, Palumbo E, Notredame C. Nextflow enables reproducible computational workflows. Nat Biotechnol. 2017;35:316\u20139.","journal-title":"Nat Biotechnol"},{"key":"3802_CR21","doi-asserted-by":"publisher","first-page":"926","DOI":"10.1093\/bioinformatics\/btu739","volume":"31","author":"BE Suzek","year":"2015","unstructured":"Suzek BE, Wang Y, Huang H, McGarvey PB, Wu CH, the UniProt Consortium. UniRef clusters: a comprehensive and scalable alternative for improving sequence similarity searches. Bioinformatics. 2015;31:926\u201332.","journal-title":"Bioinformatics"},{"key":"3802_CR22","doi-asserted-by":"publisher","first-page":"e105776","DOI":"10.1371\/journal.pone.0105776","volume":"9","author":"R Carr","year":"2014","unstructured":"Carr R, Borenstein E. Comparative analysis of functional metagenomic annotation and the mappability of short reads. PLoS ONE. 2014;9:e105776.","journal-title":"PLoS ONE"},{"key":"3802_CR23","doi-asserted-by":"publisher","unstructured":"Minot SS, Willis AD. Clustering co-abundant genes identifies components of the gut microbiome that are reproducibly associated with colorectal cancer and inflammatory bowel disease. bioRxiv. 2019. https:\/\/doi.org\/10.1101\/567818.","DOI":"10.1101\/567818"}],"container-title":["BMC Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-020-03802-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12859-020-03802-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-020-03802-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,10,15]],"date-time":"2021-10-15T10:00:25Z","timestamp":1634292025000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/s12859-020-03802-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,15]]},"references-count":23,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2020,12]]}},"alternative-id":["3802"],"URL":"https:\/\/doi.org\/10.1186\/s12859-020-03802-0","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/295352","asserted-by":"object"}]},"ISSN":["1471-2105"],"issn-type":[{"value":"1471-2105","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,10,15]]},"assertion":[{"value":"14 June 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 October 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 October 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Not applicable.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to publish"}},{"value":"The authors have no conflicts of interest to disclose.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"459"}}