{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,13]],"date-time":"2026-03-13T08:20:02Z","timestamp":1773390002052,"version":"3.50.1"},"reference-count":63,"publisher":"Oxford University Press (OUP)","issue":"5","license":[{"start":{"date-parts":[[2024,8,12]],"date-time":"2024-08-12T00:00:00Z","timestamp":1723420800000},"content-version":"vor","delay-in-days":18,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001602","name":"Science Foundation Ireland","doi-asserted-by":"publisher","award":["18\/CRT\/6214"],"award-info":[{"award-number":["18\/CRT\/6214"]}],"id":[{"id":"10.13039\/501100001602","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,7,25]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Microsatellite instability (MSI) is a phenomenon seen in several cancer types, which can be used as a biomarker to help guide immune checkpoint inhibitor treatment. To facilitate this, researchers have developed computational tools to categorize samples as having high microsatellite instability, or as being microsatellite stable using next-generation sequencing data. Most of these tools were published with unclear scope and usage, and they have yet to be independently benchmarked. To address these issues, we assessed the performance of eight leading MSI tools across several unique datasets that encompass a wide variety of sequencing methods. While we were able to replicate the original findings of each tool on whole exome sequencing data, most tools had worse receiver operating characteristic and precision-recall area under the curve values on whole genome sequencing data. We also found that they lacked agreement with one another and with commercial MSI software on gene panel data, and that optimal threshold cut-offs vary by sequencing type. Lastly, we tested tools made specifically for RNA sequencing data and found they were outperformed by tools designed for use with DNA sequencing data. Out of all, two tools (MSIsensor2, MANTIS) performed well across nearly all datasets, but when all datasets were combined, their precision decreased. Our results caution that MSI tools can have much lower performance on datasets other than those on which they were originally evaluated, and in the case of RNA sequencing tools, can even perform poorly on the type of data for which they were created.<\/jats:p>","DOI":"10.1093\/bib\/bbae390","type":"journal-article","created":{"date-parts":[[2024,8,12]],"date-time":"2024-08-12T05:39:12Z","timestamp":1723441152000},"source":"Crossref","is-referenced-by-count":6,"title":["Performance assessment of computational tools to detect microsatellite instability"],"prefix":"10.1093","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7028-0170","authenticated-orcid":false,"given":"Harrison","family":"Anthony","sequence":"first","affiliation":[{"name":"School of Mathematical and Statistical Sciences, University of Galway , Galway H91 TK33, Ireland"},{"name":"The SFI Centre for Research Training in Genomics Data Science , Galway D02 FX65, Ireland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Cathal","family":"Seoighe","sequence":"additional","affiliation":[{"name":"School of Mathematical and Statistical Sciences, University of Galway , Galway H91 TK33, Ireland"},{"name":"The SFI Centre for Research Training in Genomics Data Science , Galway D02 FX65, Ireland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2024,8,12]]},"reference":[{"issue":"June","key":"2024081205385771300_ref1","first-page":"558","article-title":"Ubiquitous somatic mutations in simple repeated sequences reveal a new mechanism for colonic carcinogenesis","volume":"363","author":"Ionov","year":"1993","journal-title":"Nat Cell Biol"},{"issue":"5109","key":"2024081205385771300_ref2","doi-asserted-by":"crossref","first-page":"816","DOI":"10.1126\/science.8484122","article-title":"Microsatellite instability in cancer of the proximal colon","volume":"260","author":"Thibodeau","year":"1993","journal-title":"Science"},{"issue":"5109","key":"2024081205385771300_ref3","doi-asserted-by":"crossref","first-page":"812","DOI":"10.1126\/science.8484121","article-title":"Clues to the pathogenesis of familial colorectal cancer","volume":"260","author":"Aaltonen","year":"1993","journal-title":"Science"},{"issue":"10","key":"2024081205385771300_ref4","doi-asserted-by":"crossref","first-page":"1808","DOI":"10.1002\/(SICI)1097-0142(19980515)82:10<1808::AID-CNCR2>3.0.CO;2-J","article-title":"Microsatellite instability in human solid tumors","volume":"82","author":"Arzimanoglou","year":"1998","journal-title":"Cancer"},{"issue":"3","key":"2024081205385771300_ref5","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1053\/ejso.2002.1399","article-title":"The clinical importance and prognostic implications of microsatellite instability in sporadic cancer","volume":"29","author":"Lawes","year":"2003","journal-title":"Eur J Surg Oncol"},{"issue":"June","key":"2024081205385771300_ref6","first-page":"1","article-title":"Mismatch repair pathway, genome stability and cancer","volume":"7","author":"Pe\u0107ina-\u0160laus","year":"2020","journal-title":"Front Mol Biosci"},{"issue":"10","key":"2024081205385771300_ref7","doi-asserted-by":"crossref","first-page":"1200","DOI":"10.1634\/theoncologist.2016-0046","article-title":"Mismatch repair deficiency and response to immune checkpoint blockade","volume":"21","author":"Lee","year":"2016","journal-title":"Oncologist"},{"issue":"3","key":"2024081205385771300_ref8","doi-asserted-by":"crossref","first-page":"305","DOI":"10.2353\/jmoldx.2006.050092","article-title":"Comparison of the microsatellite instability analysis system and the Bethesda panel for the determination of microsatellite instability in colorectal cancers","volume":"8","author":"Murphy","year":"2006","journal-title":"J Mol Diagnostics"},{"issue":"1","key":"2024081205385771300_ref9","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1016\/S1525-1578(10)60611-3","article-title":"Detection of microsatellite instability by fluorescence multiplex polymerase chain reaction","volume":"2","author":"Berg","year":"2000","journal-title":"J Mol Diagnostics"},{"issue":"2055","key":"2024081205385771300_ref10","first-page":"119","article-title":"Detection of microsatellite instability biomarkers via next-generation sequencing","volume":"2020","author":"Bonneville","year":"2017","journal-title":"Methods Mol Biol"},{"issue":"7","key":"2024081205385771300_ref11","doi-asserted-by":"crossref","first-page":"1015","DOI":"10.1093\/bioinformatics\/btt755","article-title":"MSIsensor: microsatellite instability detection using paired tumor-normal sequence data","volume":"30","author":"Niu","year":"2014","journal-title":"Bioinformatics"},{"key":"2024081205385771300_ref12","doi-asserted-by":"crossref","first-page":"668","DOI":"10.1016\/j.csbj.2020.03.007","article-title":"PreMSIm: an R package for predicting microsatellite instability from the expression profiling of a gene panel in cancer","volume":"18","author":"Li","year":"2020","journal-title":"Comput Struct Biotechnol J"},{"issue":"1","key":"2024081205385771300_ref13","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s12859-023-05186-3","article-title":"DeltaMSI: artificial intelligence-based modeling of microsatellite instability scoring on next-generation sequencing data","volume":"24","author":"Swaerts","year":"2023","journal-title":"BMC Bioinformatics"},{"key":"2024081205385771300_ref14","doi-asserted-by":"crossref","DOI":"10.3389\/fonc.2018.00621","article-title":"Molecular and computational methods for the detection of microsatellite instability in cancer","volume":"8","author":"Baudrin","year":"2018","journal-title":"Front Oncol"},{"issue":"3","key":"2024081205385771300_ref15","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1053\/j.seminoncol.2019.08.003","article-title":"An updated review of microsatellite instability in the era of next-generation sequencing and precision medicine","volume":"46","author":"Yamamoto","year":"2019","journal-title":"Semin Oncol"},{"key":"2024081205385771300_ref16","doi-asserted-by":"crossref","first-page":"4931","DOI":"10.1016\/j.csbj.2021.08.037","article-title":"Sensitive detection of microsatellite instability in tissues and liquid biopsies: recent developments and updates","volume":"19","author":"Yu","year":"2021","journal-title":"Comput Struct Biotechnol J"},{"issue":"1","key":"2024081205385771300_ref17","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1016\/j.gpb.2020.02.001","article-title":"MSIsensor-pro: fast, accurate, and matched-normal-sample-free detection of microsatellite instability","volume":"18","author":"Jia","year":"2020","journal-title":"Genom Proteom Bioinform"},{"key":"2024081205385771300_ref18","article-title":"MSI-sensor 2 [Internet]","author":"Niu","year":"2024"},{"issue":"9","key":"2024081205385771300_ref19","doi-asserted-by":"crossref","first-page":"1192","DOI":"10.1373\/clinchem.2014.223677","article-title":"Microsatellite instability detection by next generation sequencing","volume":"60","author":"Salipante","year":"2014","journal-title":"Clin Chem"},{"issue":"5","key":"2024081205385771300_ref20","doi-asserted-by":"crossref","first-page":"7452","DOI":"10.18632\/oncotarget.13918","article-title":"Performance evaluation for rapid detection of pan-cancer microsatellite instability with MANTIS","volume":"8","author":"Kautto","year":"2017","journal-title":"Oncotarget"},{"issue":"1","key":"2024081205385771300_ref21","first-page":"100","article-title":"MSINGB: a novel computational method based on NGBoost for identifying microsatellite instability status from tumor mutation annotation data","volume":"15","author":"Chen","year":"2023","journal-title":"Interdiscip sci \u2013 Comput life sci"},{"key":"2024081205385771300_ref22","doi-asserted-by":"crossref","DOI":"10.1093\/gpbjnl\/qzae004","article-title":"MSIsensor-RNA: microsatellite instability detection for bulk and single-cell gene expression data","author":"Jia","year":"2024","journal-title":"Genom Proteom Bioinform"},{"key":"2024081205385771300_ref23","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/ncomms15180","article-title":"A molecular portrait of microsatellite instability across multiple cancers","volume":"8","author":"Cortes-Ciriano","year":"2017","journal-title":"Nat Commun"},{"issue":"14","key":"2024081205385771300_ref24","doi-asserted-by":"crossref","first-page":"1754","DOI":"10.1093\/bioinformatics\/btp324","article-title":"Fast and accurate short read alignment with Burrows-Wheeler transform","volume":"25","author":"Li","year":"2009","journal-title":"Bioinformatics"},{"issue":"1","key":"2024081205385771300_ref25","doi-asserted-by":"crossref","first-page":"269","DOI":"10.3390\/cancers15010269","volume":"15","author":"Hsieh","year":"2022","journal-title":"Cancers (Basel)"},{"issue":"3","key":"2024081205385771300_ref26","doi-asserted-by":"crossref","first-page":"339","DOI":"10.1007\/s40291-020-00462-x","article-title":"Use of an integrated pan-cancer oncology enrichment next-generation sequencing assay to measure tumour mutational burden and detect clinically actionable variants","volume":"24","author":"Pestinger","year":"2020","journal-title":"Mol Diagnosis Ther"},{"issue":"January","key":"2024081205385771300_ref27","first-page":"1","article-title":"Molecular profiling of male breast cancer by multigene panel testing: implications for precision oncology","volume":"12","author":"Valentini","year":"2023","journal-title":"Front Oncol"},{"issue":"June","key":"2024081205385771300_ref28","first-page":"1","article-title":"A novel algorithm for detecting microsatellite instability based on next-generation sequencing data","volume":"12","author":"Li","year":"2022","journal-title":"Front Oncol"},{"key":"2024081205385771300_ref29","doi-asserted-by":"crossref","DOI":"10.1101\/2020.10.21.349100","article-title":"TruSight Oncology 500: enabling comprehensive genomic profiling and biomarker reporting with targeted sequencing","author":"Zhao","year":"2020"},{"issue":"1","key":"2024081205385771300_ref30","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s12920-021-00952-9","article-title":"Comprehensive tumor molecular profile analysis in clinical practice","volume":"14","author":"\u00d6zdo\u011fan","year":"2021","journal-title":"BMC Med Genomics"},{"issue":"1","key":"2024081205385771300_ref31","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1093\/bioinformatics\/bts635","article-title":"STAR: ultrafast universal RNA-seq aligner","volume":"29","author":"Dobin","year":"2013","journal-title":"Bioinformatics"},{"issue":"7","key":"2024081205385771300_ref32","doi-asserted-by":"crossref","first-page":"923","DOI":"10.1093\/bioinformatics\/btt656","article-title":"FeatureCounts: an efficient general purpose program for assigning sequence reads to genomic features","volume":"30","author":"Liao","year":"2014","journal-title":"Bioinformatics"},{"issue":"20","key":"2024081205385771300_ref33","doi-asserted-by":"crossref","first-page":"6243","DOI":"10.1158\/1078-0432.CCR-18-3440","article-title":"Patient-derived xenografts and matched cell lines identify pharmacogenomic vulnerabilities in colorectal cancer","volume":"25","author":"Lazzari","year":"2019","journal-title":"Clin Cancer Res"},{"issue":"7828","key":"2024081205385771300_ref34","doi-asserted-by":"crossref","first-page":"292","DOI":"10.1038\/s41586-020-2769-8","article-title":"Repeat expansions confer WRN dependence in microsatellite-unstable cancers","volume":"586","author":"Wietmarschen","year":"2020","journal-title":"Nature"},{"issue":"7","key":"2024081205385771300_ref35","doi-asserted-by":"crossref","first-page":"8399","DOI":"10.18632\/oncotarget.6724","article-title":"NTRK1 fusions for the therapeutic intervention of Korean patients with colon cancer","volume":"7","author":"Park","year":"2016","journal-title":"Oncotarget"},{"key":"2024081205385771300_ref36","doi-asserted-by":"crossref","DOI":"10.1002\/0471250953.bi1110s43","article-title":"From fastQ data to high-confidence variant calls: the genome analysis toolkit best practices pipeline","volume":"43","author":"Van der Auwera","year":"2013","journal-title":"Curr Protoc Bioinformatics"},{"key":"2024081205385771300_ref37","unstructured":"Kandoth C, Gao J, Qwangmsk Mattioni M. et\u00a0al. mskcc\/vcf2maf: vcf2maf v1.6.16 [Internet]. Zenodo; 2018. Available \u00a0from: https:\/\/doi.org\/10.5281\/zenodo.1185418"},{"issue":"2","key":"2024081205385771300_ref38","doi-asserted-by":"crossref","first-page":"1982","DOI":"10.3892\/ol.2020.11702","article-title":"A robust method for the rapid detection of microsatellite instability in colorectal cancer","volume":"20","author":"Zhao","year":"2020","journal-title":"Oncol Lett"},{"issue":"December 2019","key":"2024081205385771300_ref39","doi-asserted-by":"crossref","first-page":"e00153","DOI":"10.1016\/j.plabm.2020.e00153","article-title":"Validation and implementation of a modular targeted capture assay for the detection of clinically significant molecular oncology alterations","volume":"19","author":"Kuo","year":"2020","journal-title":"Pract Lab Med"},{"key":"2024081205385771300_ref40","article-title":"Targeted next-generation sequencing-based detection of microsatellite instability in colorectal carcinomas","volume":"16","author":"Lee","year":"2021","journal-title":"PloS One"},{"issue":"18","key":"2024081205385771300_ref41","doi-asserted-by":"crossref","first-page":"2141","DOI":"10.1200\/JCO.2015.65.1067","article-title":"Reliable detection of mismatch repair deficiency in colorectal cancers using mutational load in next-generation sequencing panels","volume":"34","author":"Stadler","year":"2016","journal-title":"J Clin Oncol"},{"issue":"1","key":"2024081205385771300_ref42","doi-asserted-by":"crossref","first-page":"3405","DOI":"10.1038\/s41467-022-30453-x","article-title":"Clinical sequencing of soft tissue and bone sarcomas delineates diverse genomic landscapes and potential therapeutic targets","volume":"13","author":"Nacev","year":"2022","journal-title":"Nat Commun"},{"issue":"8","key":"2024081205385771300_ref43","doi-asserted-by":"crossref","first-page":"861","DOI":"10.1016\/j.patrec.2005.10.010","article-title":"An introduction to ROC analysis","volume":"27","author":"Fawcett","year":"2006","journal-title":"Pattern Recognit Lett"},{"key":"2024081205385771300_ref44","volume-title":"Package \u201cMLeval\u201d Machine Learning Model Evaluation","author":"Christopher","year":"2022"},{"issue":"1","key":"2024081205385771300_ref45","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1186\/1471-2105-12-77","article-title":"pROC: an open-source package for R and S+ to analyze and compare ROC curves","volume":"12","author":"Robin","year":"2011","journal-title":"BMC Bioinformatics"},{"issue":"5","key":"2024081205385771300_ref46","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v028.i05","article-title":"Building predictive models in R using the caret package","volume":"28","author":"Kuhn","year":"2008","journal-title":"J Stat Softw"},{"key":"2024081205385771300_ref47","volume-title":"A Language and Environment for Statistical Computing","author":"R Core Team (2020)Development Core Team","year":"2020"},{"key":"2024081205385771300_ref48","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-24277-4","volume-title":"ggplot2: Elegant Graphics for Data Analysis","author":"Wickham","year":"2016"},{"key":"2024081205385771300_ref49","article-title":"Hyperfine [Internet]","author":"Peter","year":"2023"},{"issue":"6","key":"2024081205385771300_ref50","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1145\/1273442.1250746","article-title":"Valgrind: a framework for heavyweight dynamic binary instrumentation","volume":"42","author":"Nethercote","year":"2007","journal-title":"ACM SIGPLAN Not"},{"issue":"1","key":"2024081205385771300_ref51","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1016\/j.jmoldx.2016.07.010","article-title":"Detection of mismatch repair deficiency and microsatellite instability in colorectal adenocarcinoma by targeted next-generation sequencing","volume":"19","author":"Nowak","year":"2017","journal-title":"J Mol Diagnostics"},{"issue":"6","key":"2024081205385771300_ref52","doi-asserted-by":"crossref","first-page":"1053","DOI":"10.1016\/j.jmoldx.2019.06.011","article-title":"A novel next-generation sequencing approach to detecting microsatellite instability and Pan-tumor characterization of 1000 microsatellite instability\u2013high cases in 67,000 patient samples","volume":"21","author":"Trabucco","year":"2019","journal-title":"J Mol Diagnostics"},{"issue":"1","key":"2024081205385771300_ref53","first-page":"1","article-title":"MSIseq: software for assessing microsatellite instability from catalogs of somatic mutations","volume":"5","author":"Ni Huang","year":"2015","journal-title":"Sci Rep"},{"issue":"3","key":"2024081205385771300_ref54","doi-asserted-by":"crossref","first-page":"334","DOI":"10.1101\/gr.255026.119","article-title":"Comprehensive analysis of indels in whole-genome microsatellite regions and microsatellite instability across 21 cancer types","volume":"30","author":"Fujimoto","year":"2020","journal-title":"Genome Res"},{"issue":"23","key":"2024081205385771300_ref55","doi-asserted-by":"crossref","first-page":"3799","DOI":"10.1093\/bioinformatics\/btx507","article-title":"MIRMMR: binary classification of microsatellite instability using methylation and mutations","volume":"33","author":"Foltz","year":"2017","journal-title":"Bioinformatics"},{"issue":"6","key":"2024081205385771300_ref56","doi-asserted-by":"crossref","DOI":"10.1093\/bib\/bbad362","article-title":"MSI-XGNN: an explainable GNN computational framework integrating transcription- and methylation-level biomarkers for microsatellite instability detection","volume":"24","author":"Cao","year":"2023","journal-title":"Brief Bioinform"},{"issue":"2","key":"2024081205385771300_ref57","doi-asserted-by":"crossref","DOI":"10.1186\/gb-2003-4-2-r13","article-title":"Genome-wide analysis of microsatellite repeats in humans: their abundance and density in specific genomic regions","volume":"4","author":"Subramanian","year":"2003","journal-title":"Genome Biol"},{"issue":"3","key":"2024081205385771300_ref58","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1038\/nrclinonc.2009.237","article-title":"Microsatellite instability in colorectal cancerthe stable evidence","volume":"7","author":"Vilar","year":"2010","journal-title":"Nat Rev Clin Oncol"},{"key":"2024081205385771300_ref59","first-page":"9","volume-title":"Methods in Molecular Biology","author":"Wong","year":"2021"},{"issue":"6","key":"2024081205385771300_ref60","first-page":"97","article-title":"Microsatellite instability in colorectal cancer","volume":"89","author":"De\u2019 Angelis","year":"2018","journal-title":"Acta Biomed"},{"issue":"2","key":"2024081205385771300_ref61","doi-asserted-by":"crossref","first-page":"225","DOI":"10.1016\/j.jmoldx.2017.11.007","article-title":"A novel and reliable method to detect microsatellite instability in colorectal cancer by next-generation sequencing","volume":"20","author":"Zhu","year":"2018","journal-title":"J Mol Diagnostics"},{"issue":"10","key":"2024081205385771300_ref62","doi-asserted-by":"crossref","first-page":"4441","DOI":"10.1172\/JCI121924","article-title":"Immunogenomic analyses associate immunological alterations with mismatch repair defects in prostate cancer","volume":"128","author":"Rodrigues","year":"2018","journal-title":"J Clin Invest"},{"key":"2024081205385771300_ref63","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1200\/PO.17.00084","article-title":"Reliable pan-cancer microsatellite instability assessment by using targeted next-generation sequencing data","volume":"1","author":"Middha","year":"2017","journal-title":"JCO Precis Oncol"}],"container-title":["Briefings in Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/25\/5\/bbae390\/58796739\/bbae390.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/25\/5\/bbae390\/58796739\/bbae390.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,12]],"date-time":"2024-08-12T05:40:04Z","timestamp":1723441204000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bib\/article\/doi\/10.1093\/bib\/bbae390\/7731494"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,25]]},"references-count":63,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2024,7,25]]}},"URL":"https:\/\/doi.org\/10.1093\/bib\/bbae390","relation":{},"ISSN":["1467-5463","1477-4054"],"issn-type":[{"value":"1467-5463","type":"print"},{"value":"1477-4054","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,9]]},"published":{"date-parts":[[2024,7,25]]},"article-number":"bbae390"}}