{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,4]],"date-time":"2026-06-04T06:01:21Z","timestamp":1780552881985,"version":"3.54.1"},"reference-count":35,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2025,5,29]],"date-time":"2025-05-29T00:00:00Z","timestamp":1748476800000},"content-version":"vor","delay-in-days":28,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"name":"Flemish Government under the Flanders AI Research Program","award":["174K02325"],"award-info":[{"award-number":["174K02325"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,5,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Automatic cell-type annotation methods assign cell-type labels to new, unlabeled datasets by leveraging relationships from a reference RNA-seq atlas. However, new datasets may include labels absent from the reference dataset or exhibit feature distributions that diverge from it. These scenarios can significantly affect the reliability of cell type predictions, a factor often overlooked in current automatic annotation methods. The field of out-of-distribution detection (OOD), primarily focused on computer vision, addresses the identification of instances that differ from the training distribution. Therefore, the implementation of OOD methods in the context of novel cell type annotation and data shift detection for single-cell transcriptomics may enhance annotation accuracy and trustworthiness. We evaluate six OOD detection methods: LogitNorm, MC dropout, Deep Ensembles, Energy-based OOD, Deep NN, and Posterior networks, for their annotation and OOD detection performance in both synthetical and real-life application settings. We show that OOD detection methods can accurately identify novel cell types and demonstrate potential to detect significant data shifts in non-integrated datasets. Moreover, we find that integration of the OOD datasets does not interfere with OOD detection of novel cell types.<\/jats:p>","DOI":"10.1093\/bib\/bbaf239","type":"journal-article","created":{"date-parts":[[2025,5,21]],"date-time":"2025-05-21T08:58:28Z","timestamp":1747817908000},"source":"Crossref","is-referenced-by-count":3,"title":["Evaluation of out-of-distribution detection methods for data shifts in single-cell transcriptomics"],"prefix":"10.1093","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1883-2310","authenticated-orcid":false,"given":"Lauren","family":"Theunissen","sequence":"first","affiliation":[{"name":"Data Mining and Modeling for Biomedicine , VIB Center for Inflammation Research and VIB Center for AI and Computational Biology (VIB.AI), 9000 Ghent,","place":["Belgium"]},{"name":"Department of Data-analysis and Mathematical Modeling , Ghent University Faculty of Bioscience Engineering, 9000 Ghent,","place":["Belgium"]},{"name":"Department of Applied Mathematics , Computer Science and Statistics, Ghent University Faculty of Sciences, 9000 Ghent,","place":["Belgium"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thomas","family":"Mortier","sequence":"additional","affiliation":[{"name":"Department of Data-analysis and Mathematical Modeling , Ghent University Faculty of Bioscience Engineering, 9000 Ghent,","place":["Belgium"]},{"name":"Department of Environment , Ghent University Faculty of Bioscience Engineering, 9000 Ghent,","place":["Belgium"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0415-1506","authenticated-orcid":false,"given":"Yvan","family":"Saeys","sequence":"additional","affiliation":[{"name":"Data Mining and Modeling for Biomedicine , VIB Center for Inflammation Research and VIB Center for AI and Computational Biology (VIB.AI), 9000 Ghent,","place":["Belgium"]},{"name":"Department of Applied Mathematics , Computer Science and Statistics, Ghent University Faculty of Sciences, 9000 Ghent,","place":["Belgium"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5950-3003","authenticated-orcid":false,"given":"Willem","family":"Waegeman","sequence":"additional","affiliation":[{"name":"Department of Data-analysis and Mathematical Modeling , Ghent University Faculty of Bioscience Engineering, 9000 Ghent,","place":["Belgium"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2025,5,29]]},"reference":[{"key":"2025052909333212800_ref1","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.27041","article-title":"The human cell atlas","volume":"6","author":"Regev","year":"2017","journal-title":"eLife"},{"key":"2025052909333212800_ref2","doi-asserted-by":"publisher","first-page":"1563","DOI":"10.1038\/s41591-023-02327-2","article-title":"An integrated cell atlas of the lung in health and disease","volume":"29","author":"Sikkema","year":"2023","journal-title":"Nat Med"},{"key":"2025052909333212800_ref3","doi-asserted-by":"publisher","first-page":"1636","DOI":"10.1164\/rccm.201911-2199OC","article-title":"A single-cell atlas of the human healthy airways","volume":"202","author":"Deprez","year":"2020","journal-title":"Am J Respir Crit Care Med"},{"key":"2025052909333212800_ref4","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1038\/s41592-024-02532-y","article-title":"Considerations for building and using integrated single-cell atlases","volume":"22","author":"Hrovatin","year":"2024","journal-title":"Nat Methods"},{"key":"2025052909333212800_ref5","doi-asserted-by":"publisher","DOI":"10.1093\/bioadv\/vbad029","article-title":"Easycelltype: marker-based cell-type annotation by automatically querying multiple databases","volume":"3","author":"Li","year":"2023","journal-title":"Bioinform Adv"},{"key":"2025052909333212800_ref6","doi-asserted-by":"publisher","first-page":"1462","DOI":"10.1038\/s41592-024-02235-4","article-title":"Assessing gpt-4 for cell type annotation in single-cell RNA-seq analysis","volume":"21","author":"Hou","year":"2024","journal-title":"Nat Methods"},{"key":"2025052909333212800_ref7","doi-asserted-by":"publisher","first-page":"e122","DOI":"10.1093\/nar\/gkab775","article-title":"Scdeepsort: a pre-trained cell-type annotation method for single-cell transcriptomics using deep learning with a weighted graph neural network","volume":"49","author":"Shao","year":"2021","journal-title":"Nucleic Acids Res"},{"key":"2025052909333212800_ref8","doi-asserted-by":"publisher","first-page":"223","DOI":"10.1186\/s12915-023-01728-6","article-title":"Single-cell Mayo Map (scMayoMap): an easy-to-use tool for cell type annotation in single-cell RNA-sequencing data analysis","volume":"21","author":"Yang","year":"2023","journal-title":"BMC Biol"},{"key":"2025052909333212800_ref9","doi-asserted-by":"publisher","first-page":"01","DOI":"10.1093\/bib\/bbab567","article-title":"A comprehensive comparison of supervised and unsupervised methods for cell type identification in single-cell RNA-seq","volume":"23","author":"Sun","year":"2022","journal-title":"Brief Bioinform"},{"key":"2025052909333212800_ref10","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1038\/s41592-021-01336-8","article-title":"Benchmarking atlas-level data integration in single-cell genomics","volume":"19","author":"Luecken","year":"2022","journal-title":"Nat Methods"},{"key":"2025052909333212800_ref11","doi-asserted-by":"publisher","first-page":"5635","DOI":"10.1007\/s11263-024-02117-4","article-title":"Generalized out-of-distribution detection: a survey","volume":"132","author":"Yang","year":"2024","journal-title":"Int J Comput Vis"},{"key":"2025052909333212800_ref12","article-title":"Openood: benchmarking generalized out-of-distribution detection","volume-title":"Advances in Neural Information Processing Systems","author":"Yang","year":"2022"},{"key":"2025052909333212800_ref13","doi-asserted-by":"publisher","article-title":"Generalized Out-of-Distribution Detection: A Survey","author":"Yang","DOI":"10.1007\/s11263-024-02117-4"},{"key":"2025052909333212800_ref14","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2108.13624","article-title":"Towards out-of-distribution generalization: a survey","author":"Liu","year":"2023"},{"key":"2025052909333212800_ref15","doi-asserted-by":"publisher","DOI":"10.3390\/electronics11213500","article-title":"Out-of-distribution (OOD) detection based on deep learning: a review","volume":"11","author":"Cui","year":"2022","journal-title":"Electronics"},{"key":"2025052909333212800_ref16","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2409.11884","article-title":"Recent advances in OOD detection: problems and approaches","author":"Lu","year":"2024"},{"key":"2025052909333212800_ref17","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2110.14051","article-title":"A unified survey on anomaly, novelty, open-set, and out-of-distribution detection: solutions and future challenges","author":"Salehi","year":"2022"},{"key":"2025052909333212800_ref18","doi-asserted-by":"publisher","first-page":"521","DOI":"10.1016\/j.patcog.2011.06.019","article-title":"A unifying view on dataset shift in classification","volume":"45","author":"Moreno-Torres","year":"2012","journal-title":"Pattern Recognition"},{"key":"2025052909333212800_ref19","article-title":"Mitigating neural network overconfidence with logit normalization","volume-title":"Proceedings of Machine Learning Research","author":"Wei","year":"2022"},{"key":"2025052909333212800_ref20","article-title":"Dropout as a bayesian approximation: representing model uncertainty in deep learning","volume-title":"33rd International Conference on Machine Learning, ICML 2016","author":"Gal","year":"2016"},{"key":"2025052909333212800_ref21","article-title":"Simple and scalable predictive uncertainty estimation using deep ensembles","volume-title":"Advances in Neural Information Processing Systems","author":"Lakshminarayanan","year":"2017"},{"key":"2025052909333212800_ref22","article-title":"Energy-based out-of-distribution detection","volume-title":"Advances in Neural Information Processing Systems","author":"Liu","year":"2020"},{"key":"2025052909333212800_ref23","article-title":"Out-of-distribution detection with deep nearest neighbors","volume-title":"Proceedings of Machine Learning Research","author":"Sun","year":"2022"},{"key":"2025052909333212800_ref24","article-title":"Posterior network: uncertainty estimation without OOD samples via density-based pseudo-counts","volume-title":"Advances in Neural Information Processing Systems","author":"Charpentier","year":"2020"},{"key":"2025052909333212800_ref25","article-title":"On calibration of modern neural networks","volume-title":"34th International Conference on Machine Learning, ICML 2017","author":"Guo","year":"2017"},{"key":"2025052909333212800_ref26","article-title":"Dropout: a simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J Mach Learn Res"},{"key":"2025052909333212800_ref27","doi-asserted-by":"publisher","DOI":"10.1126\/sciadv.aba1983","article-title":"Single-cell RNA-seq reveals ectopic and aberrant lung-resident cell populations in idiopathic pulmonary fibrosis","volume":"6","author":"Adams","year":"2020","journal-title":"Sci Adv"},{"key":"2025052909333212800_ref28","article-title":"Pitfalls of epistemic uncertainty quantification through loss minimisation","volume-title":"Advances in Neural Information Processing Systems","author":"Bengs","year":"2022"},{"key":"2025052909333212800_ref29","first-page":"22624","article-title":"Is epistemic uncertainty faithfully represented by evidential deep learning methods?","volume-title":"Proceedings of the 41st International Conference on Machine Learning, Vol. 235 of Proceedings of Machine Learning Research","author":"Juergens","year":"2024"},{"key":"2025052909333212800_ref30","doi-asserted-by":"publisher","first-page":"194","DOI":"10.1186\/s13059-019-1795-z","article-title":"A comparison of automatic cell identification methods for single-cell RNA sequencing data","volume":"20","author":"Abdelaal","year":"2019","journal-title":"Genome Biol"},{"key":"2025052909333212800_ref31","doi-asserted-by":"publisher","DOI":"10.1101\/653907","article-title":"Deep learning does not outperform classical machine learning for cell-type annotation","author":"K\u00f6hler","year":"2021","journal-title":"bioRiv"},{"key":"2025052909333212800_ref32","doi-asserted-by":"publisher","DOI":"10.1093\/bib\/bbab217","article-title":"Evaluation of machine learning approaches for cell-type identification from single-cell transcriptomics data","volume":"22","author":"Huang","year":"2021","journal-title":"Brief Bioinform"},{"key":"2025052909333212800_ref33","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-024-51059-5","article-title":"Sctab: scaling cross-tissue single-cell annotation models","volume":"15","author":"Fischer","year":"2024","journal-title":"Nat Commun"},{"key":"2025052909333212800_ref34","doi-asserted-by":"publisher","first-page":"747","DOI":"10.1038\/s41587-020-0469-4","article-title":"Benchmarking single-cell RNA-sequencing protocols for cell atlas projects","volume":"38","author":"Mereu","year":"2020","journal-title":"Nat Biotechnol"},{"key":"2025052909333212800_ref35","doi-asserted-by":"publisher","first-page":"631","DOI":"10.1016\/j.molcel.2017.01.023","article-title":"Comparative analysis of single-cell RNA sequencing methods","volume":"65","author":"Ziegenhain","year":"2017","journal-title":"Mol Cell"}],"container-title":["Briefings in Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/26\/3\/bbaf239\/63398228\/bbaf239.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/26\/3\/bbaf239\/63398228\/bbaf239.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,29]],"date-time":"2025-05-29T09:33:42Z","timestamp":1748511222000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bib\/article\/doi\/10.1093\/bib\/bbaf239\/8152765"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,1]]},"references-count":35,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2025,5,1]]}},"URL":"https:\/\/doi.org\/10.1093\/bib\/bbaf239","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2025.01.24.634709","asserted-by":"object"}]},"ISSN":["1467-5463","1477-4054"],"issn-type":[{"value":"1467-5463","type":"print"},{"value":"1477-4054","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,5]]},"published":{"date-parts":[[2025,5,1]]},"article-number":"bbaf239"}}