{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,5]],"date-time":"2026-04-05T08:59:57Z","timestamp":1775379597336,"version":"3.50.1"},"reference-count":47,"publisher":"Oxford University Press (OUP)","issue":"17","license":[{"start":{"date-parts":[[2018,9,9]],"date-time":"2018-09-09T00:00:00Z","timestamp":1536451200000},"content-version":"vor","delay-in-days":8,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"name":"Research Grants Council of the Hong Kong Special Administrative Region","award":["CityU 21200816"],"award-info":[{"award-number":["CityU 21200816"]}]},{"name":"Research Grants Council of the Hong Kong Special Administrative Region","award":["CityU 11203217"],"award-info":[{"award-number":["CityU 11203217"]}]},{"name":"Research Grants Council of the Hong Kong Special Administrative Region","award":["CityU 11200218"],"award-info":[{"award-number":["CityU 11200218"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2018,9,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Motivation<\/jats:title>\n                  <jats:p>The prediction of off-target mutations in CRISPR-Cas9 is a hot topic due to its relevance to gene editing research. Existing prediction methods have been developed; however, most of them just calculated scores based on mismatches to the guide sequence in CRISPR-Cas9. Therefore, the existing prediction methods are unable to scale and improve their performance with the rapid expansion of experimental data in CRISPR-Cas9. Moreover, the existing methods still cannot satisfy enough precision in off-target predictions for gene editing at the clinical level.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>To address it, we design and implement two algorithms using deep neural networks to predict off-target mutations in CRISPR-Cas9 gene editing (i.e. deep convolutional neural network and deep feedforward neural network). The models were trained and tested on the recently released off-target dataset, CRISPOR dataset, for performance benchmark. Another off-target dataset identified by GUIDE-seq was adopted for additional evaluation. We demonstrate that convolutional neural network achieves the best performance on CRISPOR dataset, yielding an average classification area under the ROC curve (AUC) of 97.2% under stratified 5-fold cross-validation. Interestingly, the deep feedforward neural network can also be competitive at the average AUC of 97.0% under the same setting. We compare the two deep neural network models with the state-of-the-art off-target prediction methods (i.e. CFD, MIT, CROP-IT, and CCTop) and three traditional machine learning models (i.e. random forest, gradient boosting trees, and logistic regression) on both datasets in terms of AUC values, demonstrating the competitive edges of the proposed algorithms. Additional analyses are conducted to investigate the underlying reasons from different perspectives.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Availability and implementation<\/jats:title>\n                  <jats:p>The example code are available at https:\/\/github.com\/MichaelLinn\/off_target_prediction. The related datasets are available at https:\/\/github.com\/MichaelLinn\/off_target_prediction\/tree\/master\/data.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/bioinformatics\/bty554","type":"journal-article","created":{"date-parts":[[2018,7,6]],"date-time":"2018-07-06T01:20:56Z","timestamp":1530840056000},"page":"i656-i663","source":"Crossref","is-referenced-by-count":170,"title":["Off-target predictions in CRISPR-Cas9 gene editing using deep learning"],"prefix":"10.1093","volume":"34","author":[{"given":"Jiecong","family":"Lin","sequence":"first","affiliation":[{"name":"Department of Computer Science, City University of Hong Kong, Kowloon Tong, Hong Kong SAR"}]},{"given":"Ka-Chun","family":"Wong","sequence":"additional","affiliation":[{"name":"Department of Computer Science, City University of Hong Kong, Kowloon Tong, Hong Kong SAR"}]}],"member":"286","published-online":{"date-parts":[[2018,9,8]]},"reference":[{"key":"2023061313492216500_bty554-B1","author":"Abadi","year":"2016"},{"key":"2023061313492216500_bty554-B2","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1515\/bc.2011.042","article-title":"Clustered regularly interspaced short palindromic repeats (CRISPRs): the hallmark of an ingenious antiviral defense mechanism in prokaryotes","volume":"392","author":"Al-Attar","year":"2011","journal-title":"Biol. Chem"},{"key":"2023061313492216500_bty554-B3","doi-asserted-by":"crossref","first-page":"3387","DOI":"10.1093\/bioinformatics\/btx431","article-title":"Deeploc: prediction of protein subcellular localization using deep learning","volume":"33","author":"Almagro Armenteros","year":"2017","journal-title":"Bioinformatics"},{"key":"2023061313492216500_bty554-B4","doi-asserted-by":"crossref","first-page":"14958","DOI":"10.1038\/ncomms14958","article-title":"Targeted activation of diverse CRISPR-Cas systems for mammalian genome editing via proximal CRISPR targeting","volume":"8","author":"Chen","year":"2017","journal-title":"Nat. Commun"},{"key":"2023061313492216500_bty554-B5","doi-asserted-by":"crossref","first-page":"407","DOI":"10.1038\/nature24268","article-title":"Enhanced proofreading governs CRISPR\u2013Cas9 targeting accuracy","volume":"550","author":"Chen","year":"2017","journal-title":"Nature"},{"key":"2023061313492216500_bty554-B6","doi-asserted-by":"crossref","first-page":"132","DOI":"10.1101\/gr.162339.113","article-title":"Analysis of off-target effects of CRISPR\/CAS-derived RNA-guided endonucleases and nickases","volume":"24","author":"Cho","year":"2014","journal-title":"Genome Res"},{"key":"2023061313492216500_bty554-B7","doi-asserted-by":"crossref","first-page":"819","DOI":"10.1126\/science.1231143","article-title":"Multiplex genome engineering using CRISPR\/Cas systems","volume":"339","author":"Cong","year":"2013","journal-title":"Science"},{"key":"2023061313492216500_bty554-B8","doi-asserted-by":"crossref","first-page":"796","DOI":"10.1038\/mt.2015.54","article-title":"Genome editing technologies: defining a path to clinic","volume":"23","author":"Corrigan-Curay","year":"2015","journal-title":"Mol. Ther"},{"key":"2023061313492216500_bty554-B9","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1038\/nm.3793","article-title":"Therapeutic genome editing: prospects and challenges","volume":"21","author":"Cox","year":"2015","journal-title":"Nat. Med"},{"key":"2023061313492216500_bty554-B11","doi-asserted-by":"crossref","first-page":"184","DOI":"10.1038\/nbt.3437","article-title":"Optimized sgrna design to maximize activity and minimize off-target effects of CRISPR-Cas9","volume":"34","author":"Doench","year":"2016","journal-title":"Nat. Biotechnol"},{"key":"2023061313492216500_bty554-B12","doi-asserted-by":"crossref","first-page":"1258096.","DOI":"10.1126\/science.1258096","article-title":"The new frontier of genome engineering with CRISPR-Cas9","volume":"346","author":"Doudna","year":"2014","journal-title":"Science"},{"key":"2023061313492216500_bty554-B13","doi-asserted-by":"crossref","first-page":"1116","DOI":"10.1038\/nmeth.2681","article-title":"Orthogonal Cas9 proteins for RNA-guided gene regulation and editing","volume":"10","author":"Esvelt","year":"2013","journal-title":"Nat. Methods"},{"key":"2023061313492216500_bty554-B14","doi-asserted-by":"crossref","first-page":"822","DOI":"10.1038\/nbt.2623","article-title":"High-frequency off-target mutagenesis induced by crispr-cas nucleases in human cells","volume":"31","author":"Fu","year":"2013","journal-title":"Nat. Biotechnol"},{"key":"2023061313492216500_bty554-B15","first-page":"315","author":"Glorot","year":"2011"},{"key":"2023061313492216500_bty554-B16","doi-asserted-by":"crossref","first-page":"148.","DOI":"10.1186\/s13059-016-1012-2","article-title":"Evaluation of off-target and on-target scoring algorithms and integration into the guide RNA selection tool CRISPOR","volume":"17","author":"Haeussler","year":"2016","journal-title":"Genome Biol"},{"key":"2023061313492216500_bty554-B17","doi-asserted-by":"crossref","first-page":"1295","DOI":"10.1093\/bioinformatics\/btx780","article-title":"Deepsf: deep convolutional neural network for mapping protein sequences to folds","volume":"34","author":"Hou","year":"2017","journal-title":"Bioinformatics"},{"key":"2023061313492216500_bty554-B18","doi-asserted-by":"crossref","first-page":"827","DOI":"10.1038\/nbt.2647","article-title":"Dna targeting specificity of RNA-guided Cas9 nucleases","volume":"31","author":"Hsu","year":"2013","journal-title":"Nat. Biotechnol"},{"key":"2023061313492216500_bty554-B19","doi-asserted-by":"crossref","first-page":"1262","DOI":"10.1016\/j.cell.2014.05.010","article-title":"Development and applications of CRISPR-Cas9 for genome engineering","volume":"157","author":"Hsu","year":"2014","journal-title":"Cell"},{"key":"2023061313492216500_bty554-B21","doi-asserted-by":"crossref","first-page":"3685","DOI":"10.1093\/bioinformatics\/btx531","article-title":"An introduction to deep learning on biological sequence data\u2013examples and solutions","volume":"33","author":"Jurtz","year":"2017","journal-title":"Bioinformatics"},{"key":"2023061313492216500_bty554-B22","doi-asserted-by":"crossref","first-page":"791","DOI":"10.1007\/s13238-017-0477-4","article-title":"Addressing challenges in the clinical applications associated with CRISPR\/Cas9 technology and ethical questions to prevent its misuse","volume":"8","author":"Kang","year":"2017","journal-title":"Protein Cell"},{"key":"2023061313492216500_bty554-B23","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1038\/nmeth.3284","article-title":"Digenome-seq: genome-wide profiling of crispr-cas9 off-target effects in human cells","volume":"12","author":"Kim","year":"2015","journal-title":"Nat. Methods"},{"key":"2023061313492216500_bty554-B24","author":"Kingma","year":"2014"},{"key":"2023061313492216500_bty554-B25","doi-asserted-by":"crossref","first-page":"561","DOI":"10.1038\/nbt.3853","article-title":"CRISPR\u2013Cas9 epigenome editing enables high-throughput screening for functional regulatory elements in the human genome","volume":"35","author":"Klann","year":"2017","journal-title":"Nat. Biotechnol"},{"key":"2023061313492216500_bty554-B26","doi-asserted-by":"crossref","first-page":"363","DOI":"10.1007\/s13238-015-0153-5","article-title":"CRISPR\/Cas9-mediated gene editing in human tripronuclear zygotes","volume":"6","author":"Liang","year":"2015","journal-title":"Protein Cell"},{"key":"2023061313492216500_bty554-B10","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1198\/tech.2006.s353","article-title":"The cross-entropy method: a unified approach to combinatorial optimization, Monte-Carlo simulation, and machine learning","volume":"48","author":"Lih-Yuan","year":"2006","journal-title":"Technometrics"},{"key":"2023061313492216500_bty554-B27","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1038\/nature23305","article-title":"Correction of a pathogenic gene mutation in human embryos","volume":"548","author":"Ma","year":"2017","journal-title":"Nature"},{"key":"2023061313492216500_bty554-B28","doi-asserted-by":"crossref","first-page":"833","DOI":"10.1038\/nbt.2675","article-title":"Cas9 transcriptional activators for target specificity screening and paired nickases for cooperative genome engineering","volume":"31","author":"Mali","year":"2013","journal-title":"Nat. Biotechnol"},{"key":"2023061313492216500_bty554-B29","doi-asserted-by":"crossref","first-page":"823","DOI":"10.1126\/science.1232033","article-title":"RNA-guided human genome engineering via Cas9","volume":"339","author":"Mali","year":"2013","journal-title":"Science"},{"key":"2023061313492216500_bty554-B30","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1038\/nature23270","article-title":"In vivo CRISPR screening identifies Ptpn2 as a cancer immunotherapy target","volume":"547","author":"Manguso","year":"2017","journal-title":"Nature"},{"key":"2023061313492216500_bty554-B31","doi-asserted-by":"crossref","first-page":"839","DOI":"10.1038\/nbt.2673","article-title":"High-throughput profiling of off-target DNA cleavage reveals RNA-programmed Cas9 nuclease specificity","volume":"31","author":"Pattanayak","year":"2013","journal-title":"Nat. Biotechnol"},{"key":"2023061313492216500_bty554-B32","first-page":"2825","article-title":"Scikit-learn: machine learning in python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J. Mach. Learn. Res"},{"key":"2023061313492216500_bty554-B33","doi-asserted-by":"crossref","first-page":"1380","DOI":"10.1016\/j.cell.2013.08.021","article-title":"Double nicking by RNA-guided CRISPR Cas9 for enhanced genome editing specificity","volume":"154","author":"Ran","year":"2013","journal-title":"Cell"},{"key":"2023061313492216500_bty554-B20","first-page":"448","author":"Sergey","year":"2015"},{"key":"2023061313492216500_bty554-B34","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1126\/science.1247005","article-title":"Genome-scale crispr-cas9 knockout screening in human cells","volume":"343","author":"Shalem","year":"2014","journal-title":"Science"},{"key":"2023061313492216500_bty554-B35","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1038\/s41564-017-0043-0","article-title":"A CRISPR\u2013Cas9-based gene drive platform for genetic interaction analysis in Candida albicans","volume":"3","author":"Shapiro","year":"2018","journal-title":"Nat. Microbiol"},{"key":"2023061313492216500_bty554-B36","doi-asserted-by":"crossref","first-page":"573","DOI":"10.1038\/nmeth.4225","article-title":"Combinatorial CRISPR\u2013Cas9 screens for de novo mapping of genetic interactions","volume":"14","author":"Shen","year":"2017","journal-title":"Nat. Methods"},{"key":"2023061313492216500_bty554-B37","doi-asserted-by":"crossref","first-page":"1430","DOI":"10.1038\/s41467-017-01466-8","article-title":"Real-space and real-time dynamics of CRISPR-Cas9 visualized by high-speed atomic force microscopy","volume":"8","author":"Shibata","year":"2017","journal-title":"Nat. Commun"},{"key":"2023061313492216500_bty554-B38","doi-asserted-by":"crossref","first-page":"e118","DOI":"10.1093\/nar\/gkv575","article-title":"Cas9-chromatin binding information enables more accurate CRISPR off-target prediction","volume":"43","author":"Singh","year":"2015","journal-title":"Nucleic Acids Res"},{"key":"2023061313492216500_bty554-B39","doi-asserted-by":"crossref","first-page":"570","DOI":"10.1038\/mt.2014.226","article-title":"Efficient and allele-specific genome editing of disease loci in human ipscs","volume":"23","author":"Smith","year":"2015","journal-title":"Mol. Ther"},{"key":"2023061313492216500_bty554-B40","first-page":"1929","article-title":"Dropout: a simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res"},{"key":"2023061313492216500_bty554-B41","doi-asserted-by":"crossref","first-page":"e0124633","DOI":"10.1371\/journal.pone.0124633","article-title":"Cctop: an intuitive, flexible and reliable CRISPR\/Cas9 target prediction tool","volume":"10","author":"Stemmer","year":"2015","journal-title":"PLoS One"},{"key":"2023061313492216500_bty554-B42","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1038\/nbt.3117","article-title":"Guide-seq enables genome-wide profiling of off-target cleavage by crispr-cas nucleases","volume":"33","author":"Tsai","year":"2015","journal-title":"Nat. Biotechnol"},{"key":"2023061313492216500_bty554-B43","doi-asserted-by":"crossref","first-page":"659","DOI":"10.1016\/j.stem.2013.10.016","article-title":"Correction of a genetic disease in mouse via use of CRISPR-Cas9","volume":"13","author":"Wu","year":"2013","journal-title":"Cell Stem Cell"},{"key":"2023061313492216500_bty554-B44","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1038\/s41598-017-00180-1","article-title":"CRISPR-Cas9 cleavage efficiency correlates strongly with target-sgrna folding stability: from physical mechanism to off-target assessment","volume":"7","author":"Xu","year":"2017","journal-title":"Sci. Rep"},{"key":"2023061313492216500_bty554-B45","doi-asserted-by":"crossref","first-page":"i121","DOI":"10.1093\/bioinformatics\/btw255","article-title":"Convolutional neural network architectures for predicting dna\u2013protein binding","volume":"32","author":"Zeng","year":"2016","journal-title":"Bioinformatics"},{"key":"2023061313492216500_bty554-B46","doi-asserted-by":"crossref","first-page":"e264","DOI":"10.1038\/mtna.2015.37","article-title":"Off-target effects in CRISPR\/Cas9-mediated genome engineering","volume":"4","author":"Zhang","year":"2015","journal-title":"Mol. Ther. Nucleic Acids"},{"key":"2023061313492216500_bty554-B47","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1007\/s11515-015-1366-y","article-title":"Overview of guide RNA design tools for CRISPR-Cas9 genome editing technology","volume":"10","author":"Zhu","year":"2015","journal-title":"Front. Biol"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/34\/17\/i656\/50582181\/bioinformatics_34_17_i656.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/34\/17\/i656\/50582181\/bioinformatics_34_17_i656.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,13]],"date-time":"2023-06-13T13:50:38Z","timestamp":1686664238000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/34\/17\/i656\/5093220"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,9,1]]},"references-count":47,"journal-issue":{"issue":"17","published-print":{"date-parts":[[2018,9,1]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/bty554","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2018,9,1]]},"published":{"date-parts":[[2018,9,1]]}}}