{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,29]],"date-time":"2026-06-29T21:15:01Z","timestamp":1782767701294,"version":"3.54.5"},"reference-count":44,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2019,12,24]],"date-time":"2019-12-24T00:00:00Z","timestamp":1577145600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100004440","name":"Wellcome Trust","doi-asserted-by":"publisher","award":["215840\/Z\/19\/Z"],"award-info":[{"award-number":["215840\/Z\/19\/Z"]}],"id":[{"id":"10.13039\/100004440","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2020,3,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Summary<\/jats:title>\n                  <jats:p>Antibody repertoires reveal insights into the biology of the adaptive immune system and empower diagnostics and therapeutics. There are currently multiple tools available for the annotation of antibody sequences. All downstream analyses such as choosing lead drug candidates depend on the correct annotation of these sequences; however, a thorough comparison of the performance of these tools has not been investigated. Here, we benchmark the performance of commonly used immunoinformatic tools, i.e. IMGT\/HighV-QUEST, IgBLAST and MiXCR, in terms of reproducibility of annotation output, accuracy and speed using simulated and experimental high-throughput sequencing datasets.<\/jats:p>\n                  <jats:p>We analyzed changes in IMGT reference germline database in the last 10\u2009years in order to assess the reproducibility of the annotation output. We found that only 73\/183 (40%) V, D and J human genes were shared between the reference germline sets used by the tools. We found that the annotation results differed between tools. In terms of alignment accuracy, MiXCR had the highest average frequency of gene mishits, 0.02 mishit frequency and IgBLAST the lowest, 0.004 mishit frequency. Reproducibility in the output of complementarity determining three regions (CDR3 amino acids) ranged from 4.3% to 77.6% with preprocessed data. In addition, run time of the tools was assessed: MiXCR was the fastest tool for number of sequences processed per unit of time. These results indicate that immunoinformatic analyses greatly depend on the choice of bioinformatics tool. Our results support informed decision-making to immunoinformaticians based on repertoire composition and sequencing platforms.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Availability and implementation<\/jats:title>\n                  <jats:p>All tools utilized in the paper are free for academic use.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Supplementary information<\/jats:title>\n                  <jats:p>Supplementary data are available at Bioinformatics online.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btz845","type":"journal-article","created":{"date-parts":[[2019,12,20]],"date-time":"2019-12-20T12:11:15Z","timestamp":1576843875000},"page":"1731-1739","source":"Crossref","is-referenced-by-count":54,"title":["Benchmarking immunoinformatic tools for the analysis of antibody repertoire sequences"],"prefix":"10.1093","volume":"36","author":[{"given":"Erand","family":"Smakaj","sequence":"first","affiliation":[{"name":"Institute of Biomedical Engineering and Medical Informatics, School of Life Sciences, FHNW University of Applied Sciences and Arts Northwestern Switzerland , Muttenz 4132, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1211-4825","authenticated-orcid":false,"given":"Lmar","family":"Babrak","sequence":"additional","affiliation":[{"name":"Institute of Biomedical Engineering and Medical Informatics, School of Life Sciences, FHNW University of Applied Sciences and Arts Northwestern Switzerland , Muttenz 4132, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mats","family":"Ohlin","sequence":"additional","affiliation":[{"name":"Department of Immunotechnology, Lund University , Lund 223, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mikhail","family":"Shugay","sequence":"additional","affiliation":[{"name":"Center of Life Sciences, Skolkovo Institute of Science and Technology , Moscow 121205, Russia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bryan","family":"Briney","sequence":"additional","affiliation":[{"name":"Department of Immunology and Microbiology, The Scripps Research Institute , La Jolla, CA 92037, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Deniz","family":"Tosoni","sequence":"additional","affiliation":[{"name":"Institute of Biomedical Engineering and Medical Informatics, School of Life Sciences, FHNW University of Applied Sciences and Arts Northwestern Switzerland , Muttenz 4132, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christopher","family":"Galli","sequence":"additional","affiliation":[{"name":"Institute of Biomedical Engineering and Medical Informatics, School of Life Sciences, FHNW University of Applied Sciences and Arts Northwestern Switzerland , Muttenz 4132, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Vendi","family":"Grobelsek","sequence":"additional","affiliation":[{"name":"Department of Biosystems Science and Engineering , ETH Zurich, Basel 4058, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Igor","family":"D\u2019Angelo","sequence":"additional","affiliation":[{"name":"One Amgen Center Drive , Amgen, Inc., Therapeutic Discovery\/Molecular Engineering, Thousand Oaks, CA 91320, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Branden","family":"Olson","sequence":"additional","affiliation":[{"name":"Computational Biology Program, Fred Hutchinson Cancer Research Center , Seattle, WA 98109, USA"},{"name":"Department of Statistics, University of Washington , Seattle, WA 98195, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sai","family":"Reddy","sequence":"additional","affiliation":[{"name":"Department of Biosystems Science and Engineering , ETH Zurich, Basel 4058, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2622-5032","authenticated-orcid":false,"given":"Victor","family":"Greiff","sequence":"additional","affiliation":[{"name":"Department of Immunology, University of Oslo , Oslo 0372, Norway"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Johannes","family":"Tr\u00fcck","sequence":"additional","affiliation":[{"name":"Paediatric Immunology, Children\u2019s Research Center, University Children's Hospital, University of Zurich , Zurich 8032, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Susanna","family":"Marquez","sequence":"additional","affiliation":[{"name":"Department of Pathology, Yale School of Medicine , New Haven, CT 06511, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9834-6840","authenticated-orcid":false,"given":"William","family":"Lees","sequence":"additional","affiliation":[{"name":"Department of Biological Sciences and Institute of Structural and Molecular Biology, Birkbeck College, University of London , London WC1E 7HX, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6461-0519","authenticated-orcid":false,"given":"Enkelejda","family":"Miho","sequence":"additional","affiliation":[{"name":"Institute of Biomedical Engineering and Medical Informatics, School of Life Sciences, FHNW University of Applied Sciences and Arts Northwestern Switzerland , Muttenz 4132, Switzerland"},{"name":"aiNET GmbH, Switzerland Innovation Park Basel Area AG , Basel 4057, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2019,12,24]]},"reference":[{"key":"2023060911580921000_btz845-B1","doi-asserted-by":"crossref","first-page":"3832","DOI":"10.1016\/j.molimm.2008.05.022","article-title":"Analysis and improvements to Kabat and structurally correct numbering of antibody variable domains","volume":"45","author":"Abhinandan","year":"2008","journal-title":"Mol. Immunol"},{"key":"2023060911580921000_btz845-B2","doi-asserted-by":"crossref","first-page":"569","DOI":"10.1007\/978-1-61779-842-9_32","article-title":"IMGT(\u00ae) tools for the nucleotide analysis of immunoglobulin (IG) and T cell receptor (TR) V-(D)-J repertoires, polymorphisms, and IG mutations: IMGT\/V-QUEST and IMGT\/HighV-QUEST for NGS","volume":"882","author":"Alamyar","year":"2012","journal-title":"Methods Mol. Biol"},{"key":"2023060911580921000_btz845-B3","doi-asserted-by":"crossref","first-page":"927","DOI":"10.1006\/jmbi.1997.1354","article-title":"Standard conformations for the canonical structures of immunoglobulins","volume":"273","author":"Al-Lazikani","year":"1997","journal-title":"J. Mol. Biol"},{"key":"2023060911580921000_btz845-B4","volume-title":"Immunoinformatics: Bioinformatic Strategies for Better Understanding of Immune Function.","author":"Bock","year":"2004"},{"key":"2023060911580921000_btz845-B5","doi-asserted-by":"crossref","first-page":"380","DOI":"10.1038\/nmeth.3364","article-title":"MiXCR: software for comprehensive adaptive immunity profiling","volume":"12","author":"Bolotin","year":"2015","journal-title":"Nat. Methods"},{"key":"2023060911580921000_btz845-B6","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3389\/fimmu.2017.01418","article-title":"Reproducibility and reuse of adaptive immune receptor repertoire data","volume":"8","author":"Breden","year":"2017","journal-title":"Front. Immunol"},{"key":"2023060911580921000_btz845-B7","author":"Briney","year":"2018"},{"key":"2023060911580921000_btz845-B8","doi-asserted-by":"crossref","first-page":"W503","DOI":"10.1093\/nar\/gkn316","article-title":"IMGT\/V-QUEST: the highly customized and integrated system for IG and TR standardized V-J and V-D-J sequence analysis","volume":"36","author":"Brochet","year":"2008","journal-title":"Nucleic Acids Res"},{"key":"2023060911580921000_btz845-B9","first-page":"701","author":"Brown","year":"2019"},{"key":"2023060911580921000_btz845-B10","doi-asserted-by":"crossref","first-page":"669","DOI":"10.1007\/s00251-008-0325-z","article-title":"The reported germline repertoire of human immunoglobulin kappa chain genes is relatively complete and accurate","volume":"60","author":"Collins","year":"2008","journal-title":"Immunogenetics"},{"key":"2023060911580921000_btz845-B11","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3389\/fimmu.2018.00395","article-title":"Many routes to an antibody heavy-chain cdr3: necessary, yet insufficient, for specific binding","volume":"9","author":"D\u2019Angelo","year":"2018","journal-title":"Front. Immunol"},{"key":"2023060911580921000_btz845-B12","doi-asserted-by":"crossref","first-page":"166","DOI":"10.1038\/nbt.2492","article-title":"High-throughput sequencing of the paired human immunoglobulin heavy and light chain repertoire","volume":"31","author":"DeKosky","year":"2013","journal-title":"Nat. Biotechnol"},{"key":"2023060911580921000_btz845-B13","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3389\/fimmu.2018.02278","article-title":"Understanding the significance and implications of antibody numbering and antigen-binding surface\/residue definition","volume":"9","author":"Dondelinger","year":"2018","journal-title":"Front. Immunol"},{"key":"2023060911580921000_btz845-B14","first-page":"1","article-title":"Characterizing immunoglobulin repertoire from whole blood by a personal genome sequencer","volume":"8","author":"Gao","year":"2013","journal-title":"PLoS ONE"},{"key":"2023060911580921000_btz845-B15","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1186\/s13073-015-0169-8","article-title":"A bioinformatic framework for immune repertoire diversity profiling enables detection of immunological status","volume":"7","author":"Greiff","year":"2015","journal-title":"Genome Med"},{"key":"2023060911580921000_btz845-B16","doi-asserted-by":"crossref","first-page":"738","DOI":"10.1016\/j.it.2015.09.006","article-title":"Bioinformatic and statistical analysis of adaptive immune repertoires","volume":"36","author":"Greiff","year":"2015","journal-title":"Trends Immunol"},{"key":"2023060911580921000_btz845-B17","doi-asserted-by":"crossref","first-page":"2985","DOI":"10.4049\/jimmunol.1700594","article-title":"Learning the high-dimensional immunogenomic features that predict public and private antibody repertoires","volume":"199","author":"Greiff","year":"2017","journal-title":"J. Immunol"},{"key":"2023060911580921000_btz845-B18","doi-asserted-by":"crossref","first-page":"1460","DOI":"10.4049\/jimmunol.1601415","article-title":"Dysregulation of B cell repertoire formation in myasthenia gravis patients revealed through deep sequencing","volume":"198","author":"Heiden","year":"2017","journal-title":"J. Immunol"},{"key":"2023060911580921000_btz845-B19","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1098\/rstb.2014.0239","article-title":"The analysis of clonal expansions in normal and autoimmune B cell repertoires","volume":"370","author":"Hershberg","year":"2015","journal-title":"Philos. Trans. R Soc. Lond. B Biol. Sci"},{"key":"2023060911580921000_btz845-B20","doi-asserted-by":"crossref","first-page":"3129","DOI":"10.1093\/bioinformatics\/btq604","article-title":"Benchmarking the performance of human antibody gene alignment utilities using a 454 sequence dataset","volume":"26","author":"Jackson","year":"2010","journal-title":"Bioinformatics"},{"key":"2023060911580921000_btz845-B21","volume-title":"Sequences of Proteins of Immunological Interest","author":"Kabat","year":"1992"},{"key":"2023060911580921000_btz845-B22","doi-asserted-by":"crossref","first-page":"118","DOI":"10.1038\/ni.2787","article-title":"Unifying immunology with informatics and multiscale biology","volume":"15","author":"Kidd","year":"2014","journal-title":"Nat. Immunol"},{"key":"2023060911580921000_btz845-B23","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1016\/j.coi.2016.06.001","article-title":"Antibody-guided vaccine design: identification of protective epitopes","volume":"41","author":"Lanzavecchia","year":"2016","journal-title":"Curr. Opin. Immunol"},{"key":"2023060911580921000_btz845-B24","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1016\/S0167-5699(97)01163-8","article-title":"Unique database numbering system for immunogenetic analysis","volume":"18","author":"Lefranc","year":"1997","journal-title":"Immunol. Today"},{"key":"2023060911580921000_btz845-B25","doi-asserted-by":"crossref","first-page":"22","DOI":"10.3389\/fimmu.2014.00022","article-title":"Immunoglobulin and T cell receptor genes: IIMGT(\u00ae) and the birth and rise of immunoinformatics","volume":"5","author":"Lefranc","year":"2014","journal-title":"Front. Immunol"},{"key":"2023060911580921000_btz845-B26","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1038\/nrrheum.2012.66","article-title":"New tools for classification and monitoring of autoimmune diseases","volume":"8","author":"Maecker","year":"2012","journal-title":"Nat. Rev. Rheumatol"},{"key":"2023060911580921000_btz845-B27","doi-asserted-by":"crossref","first-page":"224","DOI":"10.3389\/fimmu.2018.00224","article-title":"Computational strategies for dissecting the high-dimensional complexity of adaptive immune repertoires","volume":"9","author":"Miho","year":"2018","journal-title":"Front. Immunol"},{"key":"2023060911580921000_btz845-B28","first-page":"1","author":"Nouri","year":"2018"},{"key":"2023060911580921000_btz845-B29","doi-asserted-by":"crossref","first-page":"435","DOI":"10.3389\/fimmu.2019.00435","article-title":"Inferred allelic variants of immunoglobulin receptor genes: a system for their evaluation, documentation, and naming","volume":"10","author":"Ohlin","year":"2019","journal-title":"Front. Immunol"},{"key":"2023060911580921000_btz845-B30","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1111\/imm.12838","article-title":"Integrating high-throughput screening and sequencing for monoclonal antibody discovery and engineering","volume":"153","author":"Parola","year":"2018","journal-title":"Immunology"},{"key":"2023060911580921000_btz845-B31","doi-asserted-by":"crossref","first-page":"965","DOI":"10.1038\/nbt.1673","article-title":"Monoclonal antibodies isolated without screening by analyzing the variable-gene repertoire of plasma cells","volume":"28","author":"Reddy","year":"2010","journal-title":"Nat. Biotechnol"},{"key":"2023060911580921000_btz845-B32","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1038\/nrrheum.2014.220","article-title":"Sequencing the functional antibody repertoire\u2014diagnostic and therapeutic discovery","volume":"11","author":"Robinson","year":"2015","journal-title":"Nat. Rev. Rheumatol"},{"key":"2023060911580921000_btz845-B33","doi-asserted-by":"crossref","first-page":"1274","DOI":"10.1038\/ni.3873","article-title":"Adaptive immune receptor repertoire community recommendations for sharing immune-repertoire sequencing data","volume":"18","author":"Rubelt","year":"2017","journal-title":"Nat. Immunol"},{"key":"2023060911580921000_btz845-B34","doi-asserted-by":"crossref","first-page":"3213","DOI":"10.1093\/bioinformatics\/btv326","article-title":"IgSimulator: a versatile immunosequencing simulator","volume":"31","author":"Safonova","year":"2015","journal-title":"Bioinformatics"},{"key":"2023060911580921000_btz845-B35","doi-asserted-by":"crossref","first-page":"1613","DOI":"10.1093\/bioinformatics\/bty833","article-title":"Misunderstood parameter of NCBI BLAST impacts the correctness of bioinformatics workflows","volume":"35","author":"Shah","year":"2019","journal-title":"Bioinformatics"},{"key":"2023060911580921000_btz845-B36","first-page":"23","volume-title":"Immunoinformatics, Methods in Molecular Biology","author":"Tomar","year":"2014"},{"key":"2023060911580921000_btz845-B37","doi-asserted-by":"crossref","first-page":"575","DOI":"10.1038\/302575a0","article-title":"Somatic generation of antibody diversity","volume":"302","author":"Tonegawa","year":"1983","journal-title":"Nature"},{"key":"2023060911580921000_btz845-B38","doi-asserted-by":"crossref","first-page":"1005","DOI":"10.1038\/s41591-018-0070-2","article-title":"A human anti-IL-2 antibody that potentiates regulatory T cells by a structure-based mechanism","volume":"24","author":"Trotta","year":"2018","journal-title":"Nat. Med"},{"key":"2023060911580921000_btz845-B39","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3389\/fimmu.2018.02206","article-title":"AIRR community standardized representations for annotated immune repertoires","volume":"9","author":"Vander Heiden","year":"2018","journal-title":"Front. Immunol"},{"key":"2023060911580921000_btz845-B40","doi-asserted-by":"crossref","first-page":"123","DOI":"10.3109\/08830189209055568","article-title":"Assembly of IgH CDR3: mechanism, regulation, and influence on antibody diversity","volume":"8","author":"VanDyk","year":"1992","journal-title":"Int. Rev. Immunol"},{"key":"2023060911580921000_btz845-B41","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1016\/S1074-7613(00)00006-6","article-title":"Diversity in the CDR3 region of VH is sufficient for most antibody specificities","volume":"13","author":"Xu","year":"2000","journal-title":"Immunity"},{"key":"2023060911580921000_btz845-B42","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s13073-015-0243-2","article-title":"Practical guidelines for B-cell receptor repertoire sequencing analysis","volume":"7","author":"Yaari","year":"2015","journal-title":"Genome Med"},{"key":"2023060911580921000_btz845-B43","doi-asserted-by":"crossref","first-page":"W34","DOI":"10.1093\/nar\/gkt382","article-title":"IgBLAST: an immunoglobulin variable domain sequence analysis tool","volume":"41","author":"Ye","year":"2013","journal-title":"Nucleic Acids Res"},{"key":"2023060911580921000_btz845-B44","doi-asserted-by":"crossref","first-page":"E4088","DOI":"10.1073\/pnas.1306262110","article-title":"De novo identification of VRC01 class HIV-1-neutralizing antibodies by next-generation sequencing of B-cell transcripts","volume":"110","author":"Zhu","year":"2013","journal-title":"Proc. Natl. Acad. Sci. USA"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btz845\/31677674\/btz845.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/36\/6\/1731\/50554265\/bioinformatics_36_6_1731.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/36\/6\/1731\/50554265\/bioinformatics_36_6_1731.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,9]],"date-time":"2023-06-09T11:59:28Z","timestamp":1686311968000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/36\/6\/1731\/5686386"}},"subtitle":[],"editor":[{"given":"Inanc","family":"Birols","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2019,12,24]]},"references-count":44,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2020,3,1]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btz845","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2020,3,15]]},"published":{"date-parts":[[2019,12,24]]}}}