{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,20]],"date-time":"2026-04-20T05:21:40Z","timestamp":1776662500468,"version":"3.51.2"},"reference-count":28,"publisher":"Oxford University Press (OUP)","issue":"2","license":[{"start":{"date-parts":[[2024,2,22]],"date-time":"2024-02-22T00:00:00Z","timestamp":1708560000000},"content-version":"vor","delay-in-days":31,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2021YFF1200902"],"award-info":[{"award-number":["2021YFF1200902"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["31871342"],"award-info":[{"award-number":["31871342"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,1,22]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Motivation<\/jats:title>\n                  <jats:p>Cell-type annotation of single-cell RNA-sequencing (scRNA-seq) data is a hallmark of biomedical research and clinical application. Current annotation tools usually assume the simultaneous acquisition of well-annotated data, but without the ability to expand knowledge from new data. Yet, such tools are inconsistent with the continuous emergence of scRNA-seq data, calling for a continuous cell-type annotation model. In addition, by their powerful ability of information integration and model interpretability, transformer-based pre-trained language models have led to breakthroughs in single-cell biology research. Therefore, the systematic combining of continual learning and pre-trained language models for cell-type annotation tasks is inevitable.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>We herein propose a universal cell-type annotation tool, called CANAL, that continuously fine-tunes a pre-trained language model trained on a large amount of unlabeled scRNA-seq data, as new well-labeled data emerges. CANAL essentially alleviates the dilemma of catastrophic forgetting, both in terms of model inputs and outputs. For model inputs, we introduce an experience replay schema that repeatedly reviews previous vital examples in current training stages. This is achieved through a dynamic example bank with a fixed buffer size. The example bank is class-balanced and proficient in retaining cell-type-specific information, particularly facilitating the consolidation of patterns associated with rare cell types. For model outputs, we utilize representation knowledge distillation to regularize the divergence between previous and current models, resulting in the preservation of knowledge learned from past training stages. Moreover, our universal annotation framework considers the inclusion of new cell types throughout the fine-tuning and testing stages. We can continuously expand the cell-type annotation library by absorbing new cell types from newly arrived, well-annotated training datasets, as well as automatically identify novel cells in unlabeled datasets. Comprehensive experiments with data streams under various biological scenarios demonstrate the versatility and high model interpretability of CANAL.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Availability<\/jats:title>\n                  <jats:p>An implementation of CANAL is available from https:\/\/github.com\/aster-ww\/CANAL-torch.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Contact<\/jats:title>\n                  <jats:p>dengmh@pku.edu.cn<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Supplementary information<\/jats:title>\n                  <jats:p>Supplementary data are available at Journal Name online.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/bib\/bbae047","type":"journal-article","created":{"date-parts":[[2024,2,23]],"date-time":"2024-02-23T01:20:55Z","timestamp":1708651255000},"source":"Crossref","is-referenced-by-count":9,"title":["Continually adapting pre-trained language model to universal annotation of single-cell RNA-seq data"],"prefix":"10.1093","volume":"25","author":[{"given":"Hui","family":"Wan","sequence":"first","affiliation":[{"name":"School of Mathematical Sciences, Peking University , Beijing, China, 100871"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Musu","family":"Yuan","sequence":"additional","affiliation":[{"name":"Center for Quantitative Biology, Peking University , Beijing, China, 100871"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yiwei","family":"Fu","sequence":"additional","affiliation":[{"name":"School of Mathematical Sciences, Peking University , Beijing, China, 100871"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Minghua","family":"Deng","sequence":"additional","affiliation":[{"name":"School of Mathematical Sciences, Peking University , Beijing, China, 100871"},{"name":"Center for Quantitative Biology, Peking University , Beijing, China, 100871"},{"name":"Center for Statistical Science, Peking university , Beijing, China, 100871"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2024,2,22]]},"reference":[{"issue":"6","key":"2024022301203232400_ref1","doi-asserted-by":"crossref","first-page":"1575","DOI":"10.1093\/bioinformatics\/btac011","article-title":"Scname: neighborhood contrastive clustering with ancillary mask estimation for scrna-seq data","volume":"38","author":"Wan","year":"2022","journal-title":"Bioinformatics"},{"issue":"6","key":"2024022301203232400_ref2","doi-asserted-by":"crossref","first-page":"775","DOI":"10.1093\/bioinformatics\/btaa908","article-title":"Single-cell rna-seq data semi-supervised clustering and annotation via structural regularized domain adaptation","volume":"37","author":"Chen","year":"2021","journal-title":"Bioinformatics"},{"issue":"1","key":"2024022301203232400_ref3","doi-asserted-by":"crossref","first-page":"e9620","DOI":"10.15252\/msb.20209620","article-title":"Probabilistic harmonization and annotation of single-cell transcriptomics data with deep generative models","volume":"17","author":"Chenling","year":"2021","journal-title":"Mol Syst Biol"},{"issue":"1","key":"2024022301203232400_ref4","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1038\/s41467-023-35923-4","article-title":"Transformer for one stop interpretable cell type annotation","volume":"14","author":"Chen","year":"2023","journal-title":"Nat Commun"},{"issue":"10","key":"2024022301203232400_ref5","doi-asserted-by":"crossref","first-page":"852","DOI":"10.1038\/s42256-022-00534-z","article-title":"Scbert as a large-scale pretrained deep language model for cell type annotation of single-cell rna-seq data","volume":"4","author":"Yang","year":"2022","journal-title":"Nat Mach Intell"},{"key":"2024022301203232400_ref6","article-title":"Bert: pre-training of deep bidirectional transformers for language understanding","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Devlin"},{"key":"2024022301203232400_ref7","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1016\/j.neucom.2021.10.021","article-title":"Online continual learning in image classification: an empirical survey","volume":"469","author":"Mai","year":"2022","journal-title":"Neurocomputing"},{"key":"2024022301203232400_ref8","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1016\/j.neunet.2019.01.012","article-title":"Continual lifelong learning with neural networks: a review","volume":"113","author":"Parisi","year":"2019","journal-title":"Neural Netw"},{"issue":"8","key":"2024022301203232400_ref9","doi-asserted-by":"crossref","first-page":"1000","DOI":"10.1038\/s41587-021-00867-x","article-title":"Iterative single-cell multi-omic integration using online learning","volume":"39","author":"Gao","year":"2021","journal-title":"Nat Biotechnol"},{"issue":"7","key":"2024022301203232400_ref10","doi-asserted-by":"crossref","first-page":"1873","DOI":"10.1016\/j.cell.2019.05.006","article-title":"Single-cell multi-omic integration compares and contrasts features of brain cell identity","volume":"177","author":"Welch","year":"2019","journal-title":"Cell"},{"issue":"1","key":"2024022301203232400_ref11","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1038\/s41587-021-01001-7","article-title":"Mapping single-cell data to reference atlases by transfer learning","volume":"40","author":"Lotfollahi","year":"2022","journal-title":"Nat Biotechnol"},{"issue":"1","key":"2024022301203232400_ref12","doi-asserted-by":"crossref","first-page":"6118","DOI":"10.1038\/s41467-022-33758-z","article-title":"Online single-cell data integration through projecting heterogeneous datasets into a common cell-embedding space","volume":"13","author":"Xiong","year":"2022","journal-title":"Nat Commun"},{"key":"2024022301203232400_ref13","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR.2017.587","article-title":"icarl: Incremental classifier and representation learning","volume-title":"2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Rebuffi"},{"key":"2024022301203232400_ref14","article-title":"Deja vu: continual model generalization for unseen domains","volume-title":"The Eleventh International Conference on Learning Representations","author":"Liu"},{"issue":"5","key":"2024022301203232400_ref15","doi-asserted-by":"crossref","first-page":"939","DOI":"10.1016\/j.gpb.2022.12.008","article-title":"Scemail: universal and source-free annotation method for scrna-seq data with novel cell-type perception","volume":"20","author":"Wan","year":"2022","journal-title":"Genomics Proteomics Bioinformatics"},{"issue":"4","key":"2024022301203232400_ref16","doi-asserted-by":"crossref","first-page":"385","DOI":"10.1016\/j.cels.2016.09.002","article-title":"A single-cell transcriptome atlas of the human pancreas","volume":"3","author":"Muraro","year":"2016","journal-title":"Cell Syst"},{"issue":"2","key":"2024022301203232400_ref17","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1016\/j.cell.2017.09.004","article-title":"Single-cell analysis of human pancreas reveals transcriptional signatures of aging and somatic mutation patterns","volume":"171","author":"Martin Enge","year":"2017","journal-title":"Cell"},{"issue":"4","key":"2024022301203232400_ref18","doi-asserted-by":"crossref","first-page":"346","DOI":"10.1016\/j.cels.2016.08.011","article-title":"A single-cell transcriptomic map of the human and mouse pancreas reveals inter-and intra-cell population structure","volume":"3","author":"Baron","year":"2016","journal-title":"Cell Syst"},{"issue":"4","key":"2024022301203232400_ref19","doi-asserted-by":"crossref","first-page":"593","DOI":"10.1016\/j.cmet.2016.08.020","article-title":"Single-cell transcriptome profiling of human pancreatic islets in health and type 2 diabetes","volume":"24","author":"Segerstolpe","year":"2016","journal-title":"Cell Metab"},{"key":"2024022301203232400_ref20","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/2047-783X-19-S1-S6","article-title":"Stellate cells are mesenchymal stem cells","volume":"19","author":"Kordes","year":"2014","journal-title":"Eur J Med Res"},{"issue":"3-4","key":"2024022301203232400_ref21","doi-asserted-by":"crossref","first-page":"380","DOI":"10.1016\/j.bbrc.2010.11.040","article-title":"Pancreatic stellate cells promote epithelial-mesenchymal transition in pancreatic cancer cells","volume":"403","author":"Kikuta","year":"2010","journal-title":"Biochem Biophys Res Commun"},{"issue":"1","key":"2024022301203232400_ref22","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1038\/s41592-021-01336-8","article-title":"Benchmarking atlas-level data integration in single-cell genomics","volume":"19","author":"Luecken","year":"2022","journal-title":"Nat Methods"},{"issue":"1","key":"2024022301203232400_ref23","doi-asserted-by":"crossref","first-page":"14049","DOI":"10.1038\/ncomms14049","article-title":"Massively parallel digital transcriptional profiling of single cells","volume":"8","author":"Zheng","year":"2017","journal-title":"Nat Commun"},{"issue":"7727","key":"2024022301203232400_ref24","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1038\/s41586-018-0590-4","article-title":"Single-cell transcriptomics of 20 mouse organs creates a tabula muris: the tabula muris consortium","volume":"562","author":"Schaum","year":"2018","journal-title":"Nature"},{"key":"2024022301203232400_ref25","doi-asserted-by":"crossref","first-page":"baz046","DOI":"10.1093\/database\/baz046","article-title":"Panglaodb: a web server for exploration of mouse and human single-cell rna sequencing data","volume":"2019","author":"Franz\u00e9n","year":"2019","journal-title":"Database"},{"issue":"D1","key":"2024022301203232400_ref26","doi-asserted-by":"crossref","first-page":"D870","DOI":"10.1093\/nar\/gkac947","article-title":"Cellmarker 2.0: an updated database of manually curated cell markers in human\/mouse and web tools based on scrna-seq data","volume":"51","author":"Congxue","year":"2023","journal-title":"Nucleic Acids Res"},{"issue":"6","key":"2024022301203232400_ref27","doi-asserted-by":"crossref","first-page":"1505","DOI":"10.7150\/jca.38616","article-title":"Molecular mechanism of pancreatic stellate cells activation in chronic pancreatitis and pancreatic cancer","volume":"11","author":"Jin","year":"2020","journal-title":"J Cancer"},{"issue":"5","key":"2024022301203232400_ref28","doi-asserted-by":"crossref","first-page":"5921","DOI":"10.3934\/mbe.2021296","article-title":"Bioinformatics analysis identified mmp14 and col12a1 as immune-related biomarkers associated with pancreatic adenocarcinoma prognosis","volume":"18","author":"Li","year":"2021","journal-title":"Math Biosci Eng"}],"container-title":["Briefings in Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/25\/2\/bbae047\/56742389\/bbae047.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/25\/2\/bbae047\/56742389\/bbae047.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,2,23]],"date-time":"2024-02-23T01:21:12Z","timestamp":1708651272000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bib\/article\/doi\/10.1093\/bib\/bbae047\/7611935"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,22]]},"references-count":28,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,1,22]]}},"URL":"https:\/\/doi.org\/10.1093\/bib\/bbae047","relation":{},"ISSN":["1467-5463","1477-4054"],"issn-type":[{"value":"1467-5463","type":"print"},{"value":"1477-4054","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,3,1]]},"published":{"date-parts":[[2024,1,22]]},"article-number":"bbae047"}}