{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,31]],"date-time":"2026-01-31T03:47:22Z","timestamp":1769831242303,"version":"3.49.0"},"reference-count":32,"publisher":"Oxford University Press (OUP)","issue":"Supplement_1","license":[{"start":{"date-parts":[[2025,7,15]],"date-time":"2025-07-15T00:00:00Z","timestamp":1752537600000},"content-version":"vor","delay-in-days":14,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2020YFA0908700"],"award-info":[{"award-number":["2020YFA0908700"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2020YFA0908702"],"award-info":[{"award-number":["2020YFA0908702"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62272246"],"award-info":[{"award-number":["62272246"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100006606","name":"Natural Science Foundation of Tianjin","doi-asserted-by":"publisher","award":["23JCYBJC01740"],"award-info":[{"award-number":["23JCYBJC01740"]}],"id":[{"id":"10.13039\/501100006606","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,7,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Motivation<\/jats:title>\n                  <jats:p>Fine-grained cellular characterization provides critical insights into biological processes, including tissue development, disease progression, and treatment responses. The spatial organization of cells and the interactions among distinct cell types play a pivotal role in shaping the tumor micro-environment, driving heterogeneity, and influencing patient prognosis. While computational pathology can uncover morphological structures from tissue images, conventional methods are often restricted to identifying coarse-grained and limited cell types. In contrast, spatial transcriptomics-based approaches hold promise for pinpointing fine-grained transcriptional cell types using histology data. However, these methods tend to overlook key molecular signatures inherent in gene expression data.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>To this end, we propose a cross-modal unified representation learning framework (CUCA) for identifying fine-grained cell types from histology images. CUCA is trained on paired morphology-molecule spatial transcriptomics data, enabling it to infer fine-grained cell types solely from pathology images. Our model aims to harness the cross-modal embedding alignment paradigm to harmonize the embedding spaces of morphological and molecular modalities, bridging the gap between image patterns and molecular expression signatures. Extensive results across three datasets show that CUCA captures molecule-enhanced cross-modal representations and improves the prediction of fine-grained transcriptional cell abundances. Downstream analyses of cellular spatial architectures and intercellular co-localization reveal that CUCA provides insights into tumor biology, offering potential advancements in cancer research.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Availability and implementation<\/jats:title>\n                  <jats:p>The source code of CUCA is available in Zenodo: 10.5281\/zenodo.15087256.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btaf201","type":"journal-article","created":{"date-parts":[[2025,7,15]],"date-time":"2025-07-15T13:01:58Z","timestamp":1752584518000},"page":"i115-i124","source":"Crossref","is-referenced-by-count":1,"title":["Predicting fine-grained cell types from histology images through cross-modal learning in spatial transcriptomics"],"prefix":"10.1093","volume":"41","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2061-6040","authenticated-orcid":false,"given":"Chaoyang","family":"Yan","sequence":"first","affiliation":[{"name":"College of Computer Science, Nankai University , Tianjin 300350,","place":["China"]},{"name":"Centre for Bioinformatics and Intelligent Medicine, Nankai University , Tianjin 300350,","place":["China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhihan","family":"Ruan","sequence":"additional","affiliation":[{"name":"College of Computer Science, Nankai University , Tianjin 300350,","place":["China"]},{"name":"Centre for Bioinformatics and Intelligent Medicine, Nankai University , Tianjin 300350,","place":["China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Songkang","family":"Chen","sequence":"additional","affiliation":[{"name":"College of Computer Science, Nankai University , Tianjin 300350,","place":["China"]},{"name":"Centre for Bioinformatics and Intelligent Medicine, Nankai University , Tianjin 300350,","place":["China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yichen","family":"Pan","sequence":"additional","affiliation":[{"name":"College of Computer Science, Nankai University , Tianjin 300350,","place":["China"]},{"name":"Centre for Bioinformatics and Intelligent Medicine, Nankai University , Tianjin 300350,","place":["China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xue","family":"Han","sequence":"additional","affiliation":[{"name":"College of Computer Science, Nankai University , Tianjin 300350,","place":["China"]},{"name":"Centre for Bioinformatics and Intelligent Medicine, Nankai University , Tianjin 300350,","place":["China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuanyu","family":"Li","sequence":"additional","affiliation":[{"name":"College of Computer Science, Nankai University , Tianjin 300350,","place":["China"]},{"name":"Centre for Bioinformatics and Intelligent Medicine, Nankai University , Tianjin 300350,","place":["China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5516-0157","authenticated-orcid":false,"given":"Jian","family":"Liu","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Medicinal Chemical Biology, College of Computer Science, Nankai University , Tianjin 300350,","place":["China"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2025,7,15]]},"reference":[{"key":"2025071509014925100_btaf201-B1","doi-asserted-by":"crossref","first-page":"6012","DOI":"10.1038\/s41467-021-26271-2","article-title":"Spatial deconvolution of her2-positive breast cancer delineates tumor-associated cell type interactions","volume":"12","author":"Andersson","year":"2021","journal-title":"Nat Commun"},{"key":"2025071509014925100_btaf201-B2","doi-asserted-by":"crossref","first-page":"476","DOI":"10.1038\/s41587-021-01075-3","article-title":"Super-resolved spatial transcriptomics by deep data fusion","volume":"40","author":"Bergenstr\u00e5hle","year":"2022","journal-title":"Nat Biotechnol"},{"key":"2025071509014925100_btaf201-B3","doi-asserted-by":"crossref","first-page":"1352","DOI":"10.1038\/s41592-021-01264-7","article-title":"Deep learning and alignment of spatially resolved single-cell transcriptomes with tangram","volume":"18","author":"Biancalani","year":"2021","journal-title":"Nat Methods"},{"key":"2025071509014925100_btaf201-B4","doi-asserted-by":"crossref","first-page":"517","DOI":"10.1038\/s41587-021-00830-w","article-title":"Robust decomposition of cell type mixtures in spatial transcriptomics","volume":"40","author":"Cable","year":"2022","journal-title":"Nat Biotechnol"},{"key":"2025071509014925100_btaf201-B5","first-page":"4015","author":"Chen","year":"2021"},{"key":"2025071509014925100_btaf201-B6","doi-asserted-by":"crossref","first-page":"1110","DOI":"10.1038\/s41590-019-0444-8","article-title":"The biology and functional importance of mait cells","volume":"20","author":"Godfrey","year":"2019","journal-title":"Nat Immunol"},{"key":"2025071509014925100_btaf201-B7","doi-asserted-by":"crossref","first-page":"101563","DOI":"10.1016\/j.media.2019.101563","article-title":"Hover-net: simultaneous segmentation and classification of nuclei in multi-tissue histology images","volume":"58","author":"Graham","year":"2019","journal-title":"Med Image Anal"},{"key":"2025071509014925100_btaf201-B8","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1038\/s41580-024-00768-2","article-title":"Profiling cell identity and tissue architecture with single-cell and spatial transcriptomics","volume":"26","author":"Gulati","year":"2025","journal-title":"Nat Rev Mol Cell Biol"},{"key":"2025071509014925100_btaf201-B9","doi-asserted-by":"crossref","first-page":"827","DOI":"10.1038\/s41551-020-0578-x","article-title":"Integrating spatial gene expression and breast tumour morphology via deep learning","volume":"4","author":"He","year":"2020","journal-title":"Nat Biomed Eng"},{"key":"2025071509014925100_btaf201-B10","doi-asserted-by":"crossref","first-page":"103143","DOI":"10.1016\/j.media.2024.103143","article-title":"Cellvit: vision transformers for precise cell segmentation and classification","volume":"94","author":"H\u00f6rst","year":"2024","journal-title":"Medical Image Anal"},{"key":"2025071509014925100_btaf201-B11","first-page":"3","volume-title":"ICLR","author":"Hu","year":"2022"},{"key":"2025071509014925100_btaf201-B12","doi-asserted-by":"crossref","first-page":"1246","DOI":"10.1038\/s41467-022-28803-w","article-title":"Fully-automated and ultra-fast cell-type identification using specific marker combinations from single-cell transcriptomic data","volume":"13","author":"Ianevski","year":"2022","journal-title":"Nat Commun"},{"key":"2025071509014925100_btaf201-B13","first-page":"53798","volume-title":"Adv Neural Inf Process Syst","author":"Jaume","year":"2024"},{"key":"2025071509014925100_btaf201-B14","doi-asserted-by":"crossref","first-page":"bbad464","DOI":"10.1093\/bib\/bbad464","article-title":"Thitogene: a deep learning method for predicting spatial transcriptomics from histological images","volume":"25","author":"Jia","year":"2023","journal-title":"Brief Bioinform"},{"key":"2025071509014925100_btaf201-B15","doi-asserted-by":"crossref","first-page":"661","DOI":"10.1038\/s41587-021-01139-4","article-title":"Cell2location maps fine-grained cell types in spatial transcriptomics","volume":"40","author":"Kleshchevnikov","year":"2022","journal-title":"Nat Biotechnol"},{"key":"2025071509014925100_btaf201-B16","doi-asserted-by":"crossref","first-page":"1548","DOI":"10.1038\/s41467-023-37168-7","article-title":"A comprehensive benchmarking with practical guidelines for cellular deconvolution of spatial transcriptomics","volume":"14","author":"Li","year":"2023","journal-title":"Nat Commun"},{"key":"2025071509014925100_btaf201-B17","doi-asserted-by":"crossref","first-page":"863","DOI":"10.1038\/s41591-024-02856-4","article-title":"A visual-language foundation model for computational pathology","volume":"30","author":"Lu","year":"2024","journal-title":"Nat Med"},{"key":"2025071509014925100_btaf201-B18","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1038\/s41588-022-01243-4","article-title":"A spatially resolved atlas of the human lung characterizes a gland-associated immune niche","volume":"55","author":"Madissoon","year":"2023","journal-title":"Nat Genet"},{"key":"2025071509014925100_btaf201-B19","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1038\/s41423-024-01145-x","article-title":"Development of NK cell-based cancer immunotherapies through receptor engineering","volume":"21","author":"Page","year":"2024","journal-title":"Cell Mol Immunol"},{"key":"2025071509014925100_btaf201-B20","doi-asserted-by":"crossref","DOI":"10.1101\/2021.11.28.470212","article-title":"Leveraging information in spatial transcriptomics to predict super-resolution gene expression from histology images in tumors","author":"Pang","year":"2021"},{"key":"2025071509014925100_btaf201-B21","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1038\/s41586-021-03634-9","article-title":"Exploring tissue architecture using spatial transcriptomics","volume":"596","author":"Rao","year":"2021","journal-title":"Nature"},{"key":"2025071509014925100_btaf201-B22","first-page":"gkae986","article-title":"Pairpot: a database with real-time lasso-based analysis tailored for paired single-cell and spatial transcriptomics","volume":"53","author":"Ruan","year":"2024","journal-title":"Nucleic Acids Res"},{"key":"2025071509014925100_btaf201-B23","doi-asserted-by":"crossref","first-page":"100841","DOI":"10.1016\/j.crmeth.2024.100841","article-title":"Precise detection of cell-type-specific domains in spatial transcriptomics","volume":"4","author":"Ruan","year":"2024","journal-title":"Cell Rep Methods"},{"key":"2025071509014925100_btaf201-B24","doi-asserted-by":"crossref","first-page":"807","DOI":"10.1038\/s41577-023-00884-8","article-title":"Localization, tissue biology and t cell state\u2014implications for cancer immunotherapy","volume":"23","author":"Schenkel","year":"2023","journal-title":"Nat Rev Immunol"},{"key":"2025071509014925100_btaf201-B25","first-page":"20479","author":"St\u00e4rk","year":"2022"},{"key":"2025071509014925100_btaf201-B26","doi-asserted-by":"crossref","first-page":"7848","DOI":"10.1038\/s41467-023-43629-w","article-title":"Integrating spatial and single-cell transcriptomics data using deep generative models with spatialscope","volume":"14","author":"Wan","year":"2023","journal-title":"Nat Commun"},{"key":"2025071509014925100_btaf201-B27","doi-asserted-by":"crossref","first-page":"278","DOI":"10.1186\/s13059-024-03421-5","article-title":"Stascan deciphers fine-resolution cell distribution maps in spatial transcriptomics by deep learning","volume":"25","author":"Wu","year":"2024","journal-title":"Genome Biol"},{"key":"2025071509014925100_btaf201-B28","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1038\/s41586-024-07441-w","article-title":"A whole-slide foundation model for digital pathology from real-world data","volume":"630","author":"Xu","year":"2024","journal-title":"Nature"},{"key":"2025071509014925100_btaf201-B29","doi-asserted-by":"crossref","first-page":"1372","DOI":"10.1038\/s41587-023-02019-9","article-title":"Inferring super-resolution tissue architecture by integrating spatial transcriptomics with histology","volume":"42","author":"Zhang","year":"2024","journal-title":"Nat Biotechnol"},{"key":"2025071509014925100_btaf201-B30","doi-asserted-by":"crossref","first-page":"6796","DOI":"10.1038\/s41467-023-42504-y","article-title":"Single-cell morphological and topological atlas reveals the ecosystem diversity of human breast cancer","volume":"14","author":"Zhao","year":"2023","journal-title":"Nat Commun"},{"key":"2025071509014925100_btaf201-B31","doi-asserted-by":"crossref","DOI":"10.1101\/2024.02.17.580852","article-title":"Hist2cell: deciphering fine-grained cellular architectures from histology images","author":"Zhao","year":"2024"},{"key":"2025071509014925100_btaf201-B32","author":"Zimmermann","year":"2024"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/41\/Supplement_1\/i115\/63745384\/btaf201.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/41\/Supplement_1\/i115\/63745384\/btaf201.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,15]],"date-time":"2025-07-15T13:02:01Z","timestamp":1752584521000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/41\/Supplement_1\/i115\/8199364"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,1]]},"references-count":32,"journal-issue":{"issue":"Supplement_1","published-print":{"date-parts":[[2025,7,1]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btaf201","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,7]]},"published":{"date-parts":[[2025,7,1]]}}}