{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,17]],"date-time":"2026-02-17T19:26:23Z","timestamp":1771356383173,"version":"3.50.1"},"update-to":[{"DOI":"10.1371\/journal.pcbi.1013956","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2026,2,17]],"date-time":"2026-02-17T00:00:00Z","timestamp":1771286400000}}],"reference-count":39,"publisher":"Public Library of Science (PLoS)","issue":"2","license":[{"start":{"date-parts":[[2026,2,11]],"date-time":"2026-02-11T00:00:00Z","timestamp":1770768000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100004917","name":"Cancer Prevention and Research Institute of Texas","doi-asserted-by":"publisher","award":["RP230166"],"award-info":[{"award-number":["RP230166"]}],"id":[{"id":"10.13039\/100004917","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["P50CA217674"],"award-info":[{"award-number":["P50CA217674"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["P01CA296429"],"award-info":[{"award-number":["P01CA296429"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>\n                    Spatial transcriptomics (ST) provides unprecedented insights into gene expression patterns while retaining spatial context, making it a valuable tool for understanding complex tissue architectures, such as those found in cancers. Seurat, by far the most popular tool for analyzing ST data, uses the Wilcoxon rank-sum test by default for differential expression analysis. However, as a nonparametric method that disregards spatial correlations, the Wilcoxon test can lead to inflated false positive rates and misleading findings. This limitation highlights the need for a more robust statistical approach that effectively incorporates spatial correlations. To this end, we propose a Generalized Estimating Equations (GEE) framework as a robust solution for differential gene expression analysis in ST. We conducted a comprehensive comparison of the GEE-based tests with existing methods, including the Wilcoxon rank-sum test and\n                    <jats:italic>z<\/jats:italic>\n                    -test. By appropriately accounting for spatial correlations, extensive simulations showed that the GEE test with robust standard error, referred to as the Independent GEE, demonstrated superior Type I error control and comparable power relative to other methods. Applications to ST datasets from breast and prostate cancer showed poor calibration of the p-values and potential false positive findings from the Wilcoxon rank-sum test. Our comparative study based on simulations and real data applications suggests that the Independent GEE test is well-suited for ST data, offering more accurate identification of biologically relevant gene expression changes and complementing the Wilcoxon rank-sum test. We have implemented the proposed method in R package \u201cSpatialGEE\u201d, available on GitHub.\n                  <\/jats:p>","DOI":"10.1371\/journal.pcbi.1013956","type":"journal-article","created":{"date-parts":[[2026,2,11]],"date-time":"2026-02-11T18:40:46Z","timestamp":1770835246000},"page":"e1013956","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":0,"title":["A comparative study of statistical methods for identifying differentially expressed genes in spatial transcriptomics"],"prefix":"10.1371","volume":"22","author":[{"given":"Yishan","family":"Wang","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chenxuan","family":"Zang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ziyi","family":"Li","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Charles C.","family":"Guo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dejian","family":"Lai","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7758-6116","authenticated-orcid":true,"given":"Peng","family":"Wei","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"340","published-online":{"date-parts":[[2026,2,11]]},"reference":[{"issue":"6294","key":"pcbi.1013956.ref001","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1126\/science.aaf2403","article-title":"Visualization and analysis of gene expression in tissue sections by spatial transcriptomics","volume":"353","author":"PL St\u00e5hl","year":"2016","journal-title":"Science"},{"issue":"5","key":"pcbi.1013956.ref002","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1038\/nmeth.4636","article-title":"SpatialDE: identification of spatially variable genes","volume":"15","author":"V Svensson","year":"2018","journal-title":"Nat Methods"},{"issue":"7922","key":"pcbi.1013956.ref003","doi-asserted-by":"crossref","first-page":"360","DOI":"10.1038\/s41586-022-05023-2","article-title":"Spatially resolved clonal copy number alterations in benign and malignant tissue","volume":"608","author":"A Erickson","year":"2022","journal-title":"Nature"},{"issue":"6","key":"pcbi.1013956.ref004","doi-asserted-by":"crossref","first-page":"773","DOI":"10.1038\/s41587-022-01448-2","article-title":"The expanding vistas of spatial transcriptomics","volume":"41","author":"L Tian","year":"2023","journal-title":"Nat Biotechnol"},{"issue":"1","key":"pcbi.1013956.ref005","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1186\/s13059-024-03289-5","article-title":"iIMPACT: integrating image and molecular profiles for spatial transcriptomics analysis","volume":"25","author":"X Jiang","year":"2024","journal-title":"Genome Biol"},{"issue":"7","key":"pcbi.1013956.ref006","doi-asserted-by":"crossref","first-page":"1231","DOI":"10.1038\/s41592-024-02284-9","article-title":"Accurate and efficient integrative reference-informed spatial domain detection for spatial transcriptomics","volume":"21","author":"Y Ma","year":"2024","journal-title":"Nat Methods"},{"issue":"1","key":"pcbi.1013956.ref007","doi-asserted-by":"crossref","DOI":"10.1093\/biostatistics\/kxaf012","article-title":"Addressing the mean-variance relationship in spatially resolved transcriptomics data with spoon","volume":"26","author":"K Shah","year":"2024","journal-title":"Biostatistics"},{"issue":"2","key":"pcbi.1013956.ref008","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1038\/s41592-019-0701-7","article-title":"Statistical analysis of spatial expression patterns for spatially resolved transcriptomic studies","volume":"17","author":"S Sun","year":"2020","journal-title":"Nat Methods"},{"issue":"9","key":"pcbi.1013956.ref009","doi-asserted-by":"crossref","first-page":"1076","DOI":"10.1038\/s41592-022-01575-3","article-title":"Cell type-specific inference of differential expression in spatial transcriptomics","volume":"19","author":"DM Cable","year":"2022","journal-title":"Nat Methods"},{"issue":"6","key":"pcbi.1013956.ref010","doi-asserted-by":"crossref","first-page":"80","DOI":"10.2307\/3001968","article-title":"Individual comparisons by ranking methods","volume":"1","author":"F Wilcoxon","year":"1945","journal-title":"Biometrics Bull"},{"issue":"5","key":"pcbi.1013956.ref011","doi-asserted-by":"crossref","first-page":"411","DOI":"10.1038\/nbt.4096","article-title":"Integrating single-cell transcriptomic data across different conditions, technologies, and species","volume":"36","author":"A Butler","year":"2018","journal-title":"Nat Biotechnol"},{"issue":"7","key":"pcbi.1013956.ref012","doi-asserted-by":"crossref","DOI":"10.1016\/j.cell.2019.05.031","article-title":"Comprehensive integration of single-cell data","volume":"177","author":"T Stuart","year":"2019","journal-title":"Cell"},{"issue":"2","key":"pcbi.1013956.ref013","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1038\/s41587-023-01767-y","article-title":"Dictionary learning for integrative, multimodal and scalable single-cell analysis","volume":"42","author":"Y Hao","year":"2024","journal-title":"Nat Biotechnol"},{"issue":"9","key":"pcbi.1013956.ref014","doi-asserted-by":"crossref","first-page":"2645","DOI":"10.1093\/bioinformatics\/btac145","article-title":"spatialGE: quantification and visualization of the tumor microenvironment heterogeneity using spatial transcriptomics","volume":"38","author":"OE Ospina","year":"2022","journal-title":"Bioinformatics"},{"issue":"5","key":"pcbi.1013956.ref015","doi-asserted-by":"crossref","first-page":"848","DOI":"10.1158\/0008-5472.CAN-24-2346","article-title":"spatialGE is a user-friendly web application that facilitates spatial transcriptomics data analysis","volume":"85","author":"OE Ospina","year":"2025","journal-title":"Cancer Res"},{"key":"pcbi.1013956.ref016","volume-title":"Testing statistical hypotheses","author":"EL Lehmann","edition":"4"},{"issue":"421","key":"pcbi.1013956.ref017","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1080\/01621459.1993.10594284","article-title":"Approximate inference in generalized linear mixed models","volume":"88","author":"NE Breslow","year":"1993","journal-title":"J Am Stat Assoc"},{"issue":"1","key":"pcbi.1013956.ref018","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1093\/biomet\/73.1.13","article-title":"Longitudinal data analysis using generalized linear models","volume":"73","author":"K-Y Liang","year":"1986","journal-title":"Biometrika"},{"issue":"4","key":"pcbi.1013956.ref019","doi-asserted-by":"crossref","first-page":"1049","DOI":"10.2307\/2531734","article-title":"Models for longitudinal data: a generalized estimating equation approach","volume":"44","author":"SL Zeger","year":"1988","journal-title":"Biometrics"},{"issue":"1","key":"pcbi.1013956.ref020","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1017\/S0305004100023987","article-title":"Large sample tests of statistical hypotheses concerning several parameters with applications to problems of estimation","volume":"44","author":"C Radhakrishna Rao","year":"1948","journal-title":"Math Proc Camb Phil Soc"},{"key":"pcbi.1013956.ref021","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1016\/j.neuroimage.2014.03.061","article-title":"Testing for association with multiple traits in generalized estimation equations, with application to neuroimaging data","volume":"96","author":"Y Zhang","year":"2014","journal-title":"Neuroimage"},{"issue":"4","key":"pcbi.1013956.ref022","doi-asserted-by":"crossref","first-page":"817","DOI":"10.2307\/1912934","article-title":"A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity","volume":"48","author":"H White","year":"1980","journal-title":"Econometrica"},{"key":"pcbi.1013956.ref023","first-page":"221","article-title":"The behavior of maximum likelihood estimates under nonstandard conditions","volume":"1","author":"PJ Huber","year":"1967","journal-title":"Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability"},{"key":"pcbi.1013956.ref024","article-title":"TUSCAN: Tumor segmentation and classification analysis in spatial transcriptomics","author":"C Zang","year":"2024","journal-title":"bioRxiv"},{"issue":"471","key":"pcbi.1013956.ref025","doi-asserted-by":"crossref","first-page":"908","DOI":"10.1198\/016214504000001583","article-title":"Rank-sum tests for clustered data","volume":"100","author":"S Datta","year":"2005","journal-title":"J Am Stat Assoc"},{"issue":"1","key":"pcbi.1013956.ref026","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1111\/j.2517-6161.1995.tb02031.x","article-title":"Controlling the false discovery rate: a practical and powerful approach to multiple testing","volume":"57","author":"Y Benjamini","year":"1995","journal-title":"J R Stat Soc Ser B Stat Methodoly"},{"key":"pcbi.1013956.ref027","volume-title":"Generalized, linear, and mixed models","author":"CE McCulloch","edition":"2"},{"issue":"3","key":"pcbi.1013956.ref028","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1016\/j.tree.2008.10.008","article-title":"Generalized linear mixed models: a practical guide for ecology and evolution","volume":"24","author":"BM Bolker","year":"2009","journal-title":"Trends Ecol Evol"},{"issue":"3","key":"pcbi.1013956.ref029","first-page":"1077","article-title":"Spatial linear mixed models with covariate measurement errors","volume":"19","author":"Y Li","year":"2009","journal-title":"Stat Sin"},{"key":"pcbi.1013956.ref030","first-page":"281","article-title":"Some methods for classification and analysis of multivariate observations","author":"J MacQueen","year":"1967","journal-title":"Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability"},{"key":"pcbi.1013956.ref031","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4614-4818-1","volume-title":"Essential statistical inference: theory and methods","author":"DD Boos","year":"2013"},{"issue":"7","key":"pcbi.1013956.ref032","doi-asserted-by":"crossref","first-page":"104551","DOI":"10.1016\/j.isci.2022.104551","article-title":"The origin of bladder cancer from mucosal field effects","volume":"25","author":"J Bondaruk","year":"2022","journal-title":"iScience"},{"issue":"7","key":"pcbi.1013956.ref033","doi-asserted-by":"crossref","first-page":"391","DOI":"10.1038\/s41585-023-00847-7","article-title":"Molecular profile of bladder cancer progression to clinically aggressive subtypes","volume":"21","author":"CC Guo","year":"2024","journal-title":"Nat Rev Urol"},{"issue":"3","key":"pcbi.1013956.ref034","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1002\/path.6474","article-title":"Dysregulated mitochondrial energy metabolism drives the progression of mucosal field effects to invasive bladder cancer","volume":"267","author":"S Lee","year":"2025","journal-title":"J Pathol"},{"key":"pcbi.1013956.ref035","doi-asserted-by":"crossref","DOI":"10.1093\/nar\/gkad347","article-title":"g:Profiler-interoperable web service for functional enrichment analysis and gene identifier mapping (2023 update)","volume":"51","author":"L Kolberg","year":"2023","journal-title":"Nucleic Acids Res"},{"issue":"7","key":"pcbi.1013956.ref036","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1038\/nrc839","article-title":"The phosphatidylinositol 3-Kinase AKT pathway in human cancer","volume":"2","author":"I Vivanco","year":"2002","journal-title":"Nat Rev Cancer"},{"issue":"3","key":"pcbi.1013956.ref037","doi-asserted-by":"crossref","first-page":"561","DOI":"10.1038\/sj.bjc.6601576","article-title":"Integrins: roles in cancer development and as treatment targets","volume":"90","author":"H Jin","year":"2004","journal-title":"Br J Cancer"},{"issue":"12","key":"pcbi.1013956.ref038","article-title":"Extracellular matrix degradation and remodeling in development and disease","volume":"3","author":"P Lu","year":"2011","journal-title":"Cold Spring Harb Perspect Biol"},{"issue":"5","key":"pcbi.1013956.ref039","doi-asserted-by":"crossref","first-page":"1013","DOI":"10.1111\/j.1582-4934.2010.01236.x","article-title":"Proteoglycans in cancer biology, tumour microenvironment and angiogenesis","volume":"15","author":"RV Iozzo","year":"2011","journal-title":"J Cell Mol Med"}],"updated-by":[{"DOI":"10.1371\/journal.pcbi.1013956","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2026,2,17]],"date-time":"2026-02-17T00:00:00Z","timestamp":1771286400000}}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1013956","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,17]],"date-time":"2026-02-17T18:56:24Z","timestamp":1771354584000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1013956"}},"subtitle":[],"editor":[{"given":"Shaun","family":"Mahony","sequence":"first","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2026,2,11]]},"references-count":39,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2026,2,11]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1013956","relation":{},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,11]]}}}