{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"institution":[{"name":"bioRxiv"}],"indexed":{"date-parts":[[2026,6,29]],"date-time":"2026-06-29T08:45:56Z","timestamp":1782722756163,"version":"3.54.5"},"posted":{"date-parts":[[2026,6,25]]},"group-title":"Bioinformatics","reference-count":11,"publisher":"openRxiv","license":[{"start":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T00:00:00Z","timestamp":1782345600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"funder":[{"name":"National Center for Advancing Translational Sciences (NCATS), National Institutes of Health","award":["UM1TR004539"],"award-info":[{"award-number":["UM1TR004539"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"accepted":{"date-parts":[[2026,6,25]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                <jats:sec>\n                  <jats:title>Motivation<\/jats:title>\n                  <jats:p>Cell-type annotation in spatial transcriptomics is challenging due to sparse gene panels, spatial heterogeneity, and limited availability of tissue-matched reference atlases. Recent approaches have explored large language models (LLMs) for integrating biological knowledge during annotation, but unconstrained inference can produce biologically unsupported predictions and hallucinated cell types. In addition, many LLM-based pipelines rely on large cloud-hosted models that limit reproducibility and deployment in privacy-sensitive environments.<\/jats:p>\n                <\/jats:sec>\n                <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>\n                    We introduce\n                    <jats:italic>NanoCellAnnotator<\/jats:italic>\n                    , a biologically constrained and confidence-aware framework for automated cell-type annotation in spatial transcriptomics. The framework de-couples spatial structure discovery, deterministic biological evidence construction, and language-model-based semantic inference. Spatial clusters are identified using hybrid spatially regularized non-negative matrix factorization (hSNMF), after which cluster-level marker genes are abstracted into ontology-derived functional programs using Gene Ontology enrichment and GO-slim projection. A lightweight locally executable language model performs constrained label selection within a curated admissible label space derived from PanglaoDB and CellMarker. Annotation confidence is estimated independently using marker support strength and lineage separation, enabling ambiguous or heterogeneous clusters to be explicitly flagged. We evaluate NanoCellAnnotator on Xenium spatial transcriptomics data from intrahepatic cholangiocarcinoma and an independent breast cancer spatial transcriptomics dataset. The framework recovers canonical cell populations with high confidence while identifying heterogeneous or transitional spatial domains as ambiguous. Agreement with manual annotations was evaluated using accuracy and adjusted Rand index.\n                  <\/jats:p>\n                <\/jats:sec>\n                <jats:sec>\n                  <jats:title>Availability<\/jats:title>\n                  <jats:p>\n                    Code available at\n                    <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/github.com\/ishtyaqmahmud\/NanoCellAnnotator\">https:\/\/github.com\/ishtyaqmahmud\/NanoCellAnnotator<\/jats:ext-link>\n                    .\n                  <\/jats:p>\n                <\/jats:sec>","DOI":"10.64898\/2026.06.21.728965","type":"posted-content","created":{"date-parts":[[2026,6,26]],"date-time":"2026-06-26T06:20:25Z","timestamp":1782454825000},"source":"Crossref","is-referenced-by-count":0,"title":["NanoCellAnnotator: Formalizing Expert Cell Type Annotation with Large Language Models"],"prefix":"10.64898","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6280-7035","authenticated-orcid":false,"given":"Md Ishtyaq","family":"Mahmud","sequence":"first","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Houston, Houston, TX 77004, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2614-3790","authenticated-orcid":false,"given":"Veena","family":"Kochat","sequence":"additional","affiliation":[{"name":"Department of Genomic Medicine and MDACC Epigenomics Therapy Initiative (METI), MD Anderson Cancer Center, Houston, TX 77030, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-4779-8125","authenticated-orcid":false,"given":"Humaira","family":"Anzum","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Houston, Houston, TX 77004, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8988-6113","authenticated-orcid":false,"given":"Suresh","family":"Satpati","sequence":"additional","affiliation":[{"name":"Department of Genomic Medicine and MDACC Epigenomics Therapy Initiative (METI), MD Anderson Cancer Center, Houston, TX 77030, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-2947-4164","authenticated-orcid":false,"given":"Jagan Mohan Reddy","family":"Dwarampudi","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Houston, Houston, TX 77004, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2321-6894","authenticated-orcid":false,"given":"Kunal","family":"Rai","sequence":"additional","affiliation":[{"name":"Department of Genomic Medicine and MDACC Epigenomics Therapy Initiative (METI), MD Anderson Cancer Center, Houston, TX 77030, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4737-0001","authenticated-orcid":false,"given":"Tania","family":"Banerjee","sequence":"additional","affiliation":[{"name":"Department of Information Science Technology, University of Houston, Sugar Land, TX 77479, USA"},{"name":"Department of Electrical and Computer Engineering, University of Houston, Houston, TX 77004, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"54368","reference":[{"issue":"3","key":"2026062901101658000_2026.06.21.728965v1.1","doi-asserted-by":"crossref","first-page":"bbag229","DOI":"10.1093\/bib\/bbag229","article-title":"Artificial intelligence in genomics: a comprehensive survey of methods, resources, challenges, and prospects","volume":"27","year":"2026","journal-title":"Briefings in Bioinformatics"},{"key":"2026062901101658000_2026.06.21.728965v1.2","doi-asserted-by":"crossref","unstructured":"Yiqun Chen and James Zou . Genept: A simple but effective foundation model for genes and cells built from chatgpt. bioRxiv, 2024. Preprint.","DOI":"10.1101\/2023.10.16.562533"},{"key":"2026062901101658000_2026.06.21.728965v1.3","doi-asserted-by":"crossref","unstructured":"Chen Yang , Xianyang Zhang , and Jun Chen . Large language model consensus substantially improves the cell type annotation accuracy for scrna-seq data. bioRxiv, pages 2025\u201304, 2025.","DOI":"10.1101\/2025.04.10.647852"},{"issue":"8","key":"2026062901101658000_2026.06.21.728965v1.4","doi-asserted-by":"crossref","first-page":"1470","DOI":"10.1038\/s41592-024-02201-0","article-title":"scgpt: Toward building a foundation model for single-cell multi-omics using generative ai","volume":"21","year":"2024","journal-title":"Nature Methods"},{"key":"2026062901101658000_2026.06.21.728965v1.5","doi-asserted-by":"crossref","unstructured":"Zhiyin Yu , Chao Zheng , Chong Chen , Xian-Sheng Hua , and Xiao Luo . scrag: Hybrid retrieval-augmented generation for llm-based cross-tissue single-cell annotation. In Findings of the Association for Computational Linguistics: ACL 2025, pages 954\u2013970, 2025.","DOI":"10.18653\/v1\/2025.findings-acl.53"},{"key":"2026062901101658000_2026.06.21.728965v1.6","unstructured":"Sajib Acharjee Dip and Liqing Zhang . Can lightweight llm agents improve spatial transcriptomics annotation? bioRxiv, pages 2025\u201311, 2025."},{"key":"2026062901101658000_2026.06.21.728965v1.7","article-title":"Celltypeagent: Trustworthy cell type annotation with large language models","year":"2025","journal-title":"arXiv preprint"},{"key":"2026062901101658000_2026.06.21.728965v1.8","doi-asserted-by":"crossref","unstructured":"Md Ishtyaq Mahmud , Veena Kochat , Suresh Satpati , Jagan Mohan Reddy Dwarampudi , Humaira Anzum , Kunal Rai , and Tania Banerjee . hsnmf: Hybrid spatially regularized nmf for image-derived spatial transcriptomics. In Proceedings of the IEEE International Symposium on Biomedical Imaging (ISBI). IEEE, 2026.","DOI":"10.1109\/ISBI61048.2026.11515324"},{"key":"2026062901101658000_2026.06.21.728965v1.9","doi-asserted-by":"publisher","DOI":"10.1093\/DATABASE\/BAZ046"},{"key":"2026062901101658000_2026.06.21.728965v1.10","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkac947"},{"key":"2026062901101658000_2026.06.21.728965v1.11","doi-asserted-by":"crossref","unstructured":"Huan Wang . Systematic benchmarking of imaging spatial transcriptomics platforms in ffpe tissues - processed data, September 2025.","DOI":"10.1038\/s41467-025-64990-y"}],"container-title":[],"original-title":[],"link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.64898\/2026.06.21.728965","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,29]],"date-time":"2026-06-29T08:10:52Z","timestamp":1782720652000},"score":1,"resource":{"primary":{"URL":"http:\/\/biorxiv.org\/lookup\/doi\/10.64898\/2026.06.21.728965"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,25]]},"references-count":11,"URL":"https:\/\/doi.org\/10.64898\/2026.06.21.728965","relation":{},"subject":[],"published":{"date-parts":[[2026,6,25]]},"subtype":"preprint"}}