{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"institution":[{"name":"Research Square"}],"indexed":{"date-parts":[[2026,9,14]],"date-time":"2026-09-14T17:04:07Z","timestamp":1789405447000,"version":"build-2803163510"},"posted":{"date-parts":[[2026,7,10]]},"group-title":"In Review","reference-count":36,"publisher":"Springer Science and Business Media LLC","license":[{"start":{"date-parts":[[2026,7,10]],"date-time":"2026-07-10T00:00:00Z","timestamp":1783641600000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"accepted":{"date-parts":[[2026,7,6]]},"abstract":"<title>Abstract<\/title>\n                <p>Background\n Pathway activity scoring is a foundational step in single-cell RNA-seq (scRNA-seq) analysis, yet method choice is rarely guided by systematic, multi-criterion evidence in the pseudobulk case-control regime that now dominates applied single-cell disease studies. Prior single-cell pathway-scoring benchmarks have focused on perturbation ground truth at cell-level resolution, leaving the donor-level pseudobulk setting \u2014 and the question of why methods disagree \u2014 largely unaddressed.\nResults\n We present PathwayBench, a benchmark comparing five widely used pathway scoring methods (ssGSEA, GSVA, z-score, AUCell, UCell) across eight pseudobulked scRNA-seq case-control datasets spanning five tissues (brain, heart, kidney, lung, blood) and 682 donors. Methods are evaluated against five criteria covering biological relevance (direction accuracy, AUROC, effect size) and four robustness axes (aggregation, outlier, normalization, sample-size stability). We identify rank-window competition, a mechanism by which rank-based methods (AUCell, UCell) lose biological signal \u2014 and in extreme cases invert it \u2014 when non-pathway competitor genes saturate the top-rank scoring window. Controlled simulations demonstrate sign inversion in 40% of replicates under high competitor burden, and real-data confirmation comes from extracellular matrix (ECM) remodeling in chronic kidney disease, where rank-based methods produce wrong-direction effects. Discretizing per-criterion performance as Good\/Intermediate\/Poor reveals that no single method satisfies all five criteria, with GSVA and UCell each meeting four along different axes.\nConclusions\n Rank-window competition provides a mechanistic explanation for systematic divergence between magnitude-aware and rank-based pathway scoring methods on scRNA-seq data and should guide method selection, particularly for fibrotic, inflammatory, or otherwise broadly remodeled transcriptomes. PathwayBench is delivered as a versioned, extensible benchmark with per-dataset scores, an interactive advisor application, and complete reproducibility infrastructure to support evidence-based method selection and community extension.<\/p>","DOI":"10.21203\/rs.3.rs-10271634\/v1","type":"posted-content","created":{"date-parts":[[2026,7,10]],"date-time":"2026-07-10T12:26:30Z","timestamp":1783686390000},"source":"Crossref","is-referenced-by-count":0,"title":["PathwayBench: a multi-criterion benchmark of pseudobulk pathway activity scoring methods reveals rank-window competition as a mechanism of biological signal loss in single-cell RNA-seq"],"prefix":"10.21203","author":[{"given":"Fadhl M.","family":"Alakwaa\u00b9","sequence":"first","affiliation":[{"name":"University of Michigan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ahmed","family":"Elbaz\u00b2","sequence":"additional","affiliation":[{"name":"American University in Cairo"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sara Al","family":"Azzam\u00b3","sequence":"additional","affiliation":[{"name":"Jordan University of Science and Technology"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tasnim","family":"Alshebani\u2074","sequence":"additional","affiliation":[{"name":"Wayne State University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","reference":[{"key":"ref1","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1038\/nature08460","article-title":"Systematic RNA interference reveals that oncogenic KRAS-driven cancers require TBK1","volume":"462","author":"Barbie DA","year":"2009","unstructured":"Barbie DA, et al. Systematic RNA interference reveals that oncogenic KRAS-driven cancers require TBK1. Nature. 2009;462:108\u201312.","journal-title":"Nature"},{"key":"ref2","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1186\/1471-2105-14-7","article-title":"GSVA: gene set variation analysis for microarray and RNA-seq data","volume":"14","author":"H\u00e4nzelmann S","year":"2013","unstructured":"H\u00e4nzelmann S, Castelo R, Guinney J. GSVA: gene set variation analysis for microarray and RNA-seq data. BMC Bioinformatics. 2013;14:7.","journal-title":"BMC Bioinformatics"},{"key":"ref3","doi-asserted-by":"crossref","first-page":"e1000217","DOI":"10.1371\/journal.pcbi.1000217","article-title":"Inferring pathway activity toward precise disease classification","volume":"4","author":"Lee E","year":"2008","unstructured":"Lee E, Chuang H-Y, Kim J-W, Ideker T, Lee D. Inferring pathway activity toward precise disease classification. PLoS Comput Biol. 2008;4:e1000217.","journal-title":"PLoS Comput Biol"},{"key":"ref4","doi-asserted-by":"crossref","first-page":"1083","DOI":"10.1038\/nmeth.4463","article-title":"SCENIC: single-cell regulatory network inference and clustering","volume":"14","author":"Aibar S","year":"2017","unstructured":"Aibar S, et al. SCENIC: single-cell regulatory network inference and clustering. Nat Methods. 2017;14:1083\u20136.","journal-title":"Nat Methods"},{"key":"ref5","doi-asserted-by":"crossref","first-page":"3796","DOI":"10.1016\/j.csbj.2021.06.043","article-title":"UCell: robust and scalable single-cell gene signature scoring","volume":"19","author":"Andreatta M","year":"2021","unstructured":"Andreatta M, Carmona SJ. UCell: robust and scalable single-cell gene signature scoring. Comput Struct Biotechnol J. 2021;19:3796\u20138.","journal-title":"Comput Struct Biotechnol J"},{"key":"ref6","doi-asserted-by":"crossref","first-page":"1141","DOI":"10.12688\/f1000research.15666.2","article-title":"A systematic performance evaluation of clustering methods for single-cell RNA-seq data","volume":"7","author":"Du\u00f2 A","year":"2018","unstructured":"Du\u00f2 A, Robinson MD, Soneson C. A systematic performance evaluation of clustering methods for single-cell RNA-seq data. F1000Res. 2018;7:1141.","journal-title":"F1000Res"},{"key":"ref7","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1038\/s41592-021-01336-8","article-title":"Benchmarking atlas-level data integration in single-cell genomics","volume":"19","author":"Luecken MD","year":"2022","unstructured":"Luecken MD, et al. Benchmarking atlas-level data integration in single-cell genomics. Nat Methods. 2022;19:41\u201350.","journal-title":"Nat Methods"},{"key":"ref8","doi-asserted-by":"crossref","first-page":"547","DOI":"10.1038\/s41587-019-0071-9","article-title":"A comparison of single-cell trajectory inference methods","volume":"37","author":"Saelens W","year":"2019","unstructured":"Saelens W, Cannoodt R, Todorov H, Saeys Y. A comparison of single-cell trajectory inference methods. Nat Biotechnol. 2019;37:547\u201354.","journal-title":"Nat Biotechnol"},{"key":"ref9","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1038\/nmeth.4612","article-title":"Bias, robustness and scalability in single-cell differential expression analysis","volume":"15","author":"Soneson C","year":"2018","unstructured":"Soneson C, Robinson MD. Bias, robustness and scalability in single-cell differential expression analysis. Nat Methods. 2018;15:255\u201361.","journal-title":"Nat Methods"},{"key":"ref10","doi-asserted-by":"crossref","first-page":"5692","DOI":"10.1038\/s41467-021-25960-2","article-title":"Confronting false discoveries in single-cell differential expression","volume":"12","author":"Squair JW","year":"2021","unstructured":"Squair JW, et al. Confronting false discoveries in single-cell differential expression. Nat Commun. 2021;12:5692.","journal-title":"Nat Commun"},{"key":"ref11","doi-asserted-by":"crossref","first-page":"6077","DOI":"10.1038\/s41467-020-19894-4","article-title":"muscat detects subpopulation-specific state transitions from multi-sample multi-condition single-cell transcriptomics data","volume":"11","author":"Crowell HL","year":"2020","unstructured":"Crowell HL, et al. muscat detects subpopulation-specific state transitions from multi-sample multi-condition single-cell transcriptomics data. Nat Commun. 2020;11:6077.","journal-title":"Nat Commun"},{"key":"ref12","doi-asserted-by":"crossref","first-page":"7851","DOI":"10.1038\/s41467-022-35519-4","article-title":"A balanced measure shows superior performance of pseudobulk methods in single-cell RNA-sequencing analysis","volume":"13","author":"Murphy AE","year":"2022","unstructured":"Murphy AE, Skene NG. A balanced measure shows superior performance of pseudobulk methods in single-cell RNA-sequencing analysis. Nat Commun. 2022;13:7851.","journal-title":"Nat Commun"},{"key":"ref13","doi-asserted-by":"crossref","first-page":"vbac016","DOI":"10.1093\/bioadv\/vbac016","article-title":"decoupleR: ensemble of computational methods to infer biological activities from omics data","volume":"2","author":"Badia-i-Mompel P","year":"2022","unstructured":"Badia-i-Mompel P, et al. decoupleR: ensemble of computational methods to infer biological activities from omics data. Bioinform Adv. 2022;2:vbac016.","journal-title":"Bioinform Adv"},{"key":"ref14","doi-asserted-by":"crossref","first-page":"446","DOI":"10.1016\/j.cels.2021.04.005","article-title":"Hotspot identifies informative gene modules across modalities of single-cell genomics","volume":"12","author":"DeTomaso D","year":"2021","unstructured":"DeTomaso D, Yosef N. Hotspot identifies informative gene modules across modalities of single-cell genomics. Cell Syst. 2021;12:446\u2013e4569.","journal-title":"Cell Syst"},{"key":"ref15","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1038\/nmeth.3734","article-title":"Characterizing transcriptional heterogeneity through pathway and gene set overdispersion analysis","volume":"13","author":"Fan J","year":"2016","unstructured":"Fan J, et al. Characterizing transcriptional heterogeneity through pathway and gene set overdispersion analysis. Nat Methods. 2016;13:241\u20134.","journal-title":"Nat Methods"},{"issue":"D1","key":"ref16","doi-asserted-by":"crossref","first-page":"D672","DOI":"10.1093\/nar\/gkad1025","article-title":"The Reactome Pathway Knowledgebase 2024","volume":"52","author":"Milacic M","year":"2024","unstructured":"Milacic M, Beavers D, Conley P, et al. The Reactome Pathway Knowledgebase 2024. Nucleic Acids Res. 2024;52(D1):D672\u20138.","journal-title":"Nucleic Acids Res"},{"key":"ref17","doi-asserted-by":"crossref","first-page":"332","DOI":"10.1038\/s41586-019-1195-2","article-title":"Single-cell transcriptomic analysis of Alzheimer\u2019s disease","volume":"570","author":"Mathys H","year":"2019","unstructured":"Mathys H, et al. Single-cell transcriptomic analysis of Alzheimer\u2019s disease. Nature. 2019;570:332\u20137.","journal-title":"Nature"},{"key":"ref18","doi-asserted-by":"crossref","first-page":"964","DOI":"10.1093\/brain\/awab446","article-title":"Single-cell sequencing of human midbrain reveals glial activation and a Parkinson-specific neuronal state","volume":"145","author":"Smaji\u0107 S","year":"2022","unstructured":"Smaji\u0107 S, et al. Single-cell sequencing of human midbrain reveals glial activation and a Parkinson-specific neuronal state. Brain. 2022;145:964\u201378.","journal-title":"Brain"},{"key":"ref19","doi-asserted-by":"crossref","first-page":"eaba1983","DOI":"10.1126\/sciadv.aba1983","article-title":"Single-cell RNA-seq reveals ectopic and aberrant lung-resident cell populations in idiopathic pulmonary fibrosis","volume":"6","author":"Adams TS","year":"2020","unstructured":"Adams TS, et al. Single-cell RNA-seq reveals ectopic and aberrant lung-resident cell populations in idiopathic pulmonary fibrosis. Sci Adv. 2020;6:eaba1983.","journal-title":"Sci Adv"},{"key":"ref20","doi-asserted-by":"crossref","first-page":"842","DOI":"10.1038\/s41591-020-0901-9","article-title":"Single-cell landscape of bronchoalveolar immune cells in patients with COVID-19","volume":"26","author":"Liao M","year":"2020","unstructured":"Liao M, et al. Single-cell landscape of bronchoalveolar immune cells in patients with COVID-19. Nat Med. 2020;26:842\u20134.","journal-title":"Nat Med"},{"key":"ref21","doi-asserted-by":"crossref","first-page":"1070","DOI":"10.1038\/s41591-020-0944-y","article-title":"A single-cell atlas of the peripheral immune response in patients with severe COVID-19","volume":"26","author":"Wilk AJ","year":"2020","unstructured":"Wilk AJ, et al. A single-cell atlas of the peripheral immune response in patients with severe COVID-19. Nat Med. 2020;26:1070\u20136.","journal-title":"Nat Med"},{"key":"ref22","doi-asserted-by":"crossref","first-page":"eabf1970","DOI":"10.1126\/science.abf1970","article-title":"Single-cell RNA-seq reveals cell type-specific molecular and genetic associations to lupus","volume":"376","author":"Perez RK","year":"2022","unstructured":"Perez RK, et al. Single-cell RNA-seq reveals cell type-specific molecular and genetic associations to lupus. Science. 2022;376:eabf1970.","journal-title":"Science"},{"key":"ref23","doi-asserted-by":"crossref","first-page":"174","DOI":"10.1038\/s41586-022-04817-8","article-title":"Single-nucleus profiling of human dilated and hypertrophic cardiomyopathy","volume":"608","author":"Chaffin M","year":"2022","unstructured":"Chaffin M, et al. Single-nucleus profiling of human dilated and hypertrophic cardiomyopathy. Nature. 2022;608:174\u201380.","journal-title":"Nature"},{"key":"ref24","doi-asserted-by":"crossref","first-page":"585","DOI":"10.1038\/s41586-023-05769-3","article-title":"An atlas of healthy and injured cell states and niches in the human kidney","volume":"619","author":"Lake BB","year":"2023","unstructured":"Lake BB, et al. An atlas of healthy and injured cell states and niches in the human kidney. Nature. 2023;619:585\u201394.","journal-title":"Nature"},{"key":"ref25","doi-asserted-by":"crossref","first-page":"498","DOI":"10.1016\/j.kint.2020.08.039","article-title":"Rationale and design of the Kidney Precision Medicine Project","volume":"99","author":"Boer IH","year":"2021","unstructured":"de Boer IH, et al. Rationale and design of the Kidney Precision Medicine Project. Kidney Int. 2021;99:498\u2013510.","journal-title":"Kidney Int"},{"key":"ref26","doi-asserted-by":"crossref","first-page":"M111014647","DOI":"10.1074\/mcp.M111.014647","article-title":"The matrisome: in silico definition and in vivo characterization by proteomics of normal and tumor extracellular matrices","volume":"11","author":"Naba A","year":"2012","unstructured":"Naba A, et al. The matrisome: in silico definition and in vivo characterization by proteomics of normal and tumor extracellular matrices. Mol Cell Proteom. 2012;11:M111014647.","journal-title":"Mol Cell Proteom"},{"key":"ref27","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1146\/annurev-physiol-022516-034227","article-title":"Mechanisms of renal fibrosis","volume":"80","author":"Humphreys BD","year":"2018","unstructured":"Humphreys BD. Mechanisms of renal fibrosis. Annu Rev Physiol. 2018;80:309\u201326.","journal-title":"Annu Rev Physiol"},{"key":"ref28","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1038\/s41587-023-01767-y","article-title":"Dictionary learning for integrative, multimodal and scalable single-cell analysis","volume":"42","author":"Hao Y","year":"2024","unstructured":"Hao Y, et al. Dictionary learning for integrative, multimodal and scalable single-cell analysis. Nat Biotechnol. 2024;42:293\u2013304.","journal-title":"Nat Biotechnol"},{"key":"ref29","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1186\/s13059-017-1382-0","article-title":"SCANPY: large-scale single-cell gene expression data analysis","volume":"19","author":"Wolf FA","year":"2018","unstructured":"Wolf FA, Angerer P, Theis FJ. SCANPY: large-scale single-cell gene expression data analysis. Genome Biol. 2018;19:15.","journal-title":"Genome Biol"},{"key":"ref30","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1093\/bioinformatics\/btp616","article-title":"edgeR: a Bioconductor package for differential expression analysis of digital gene expression data","volume":"26","author":"Robinson MD","year":"2010","unstructured":"Robinson MD, McCarthy DJ, Smyth GK. edgeR: a Bioconductor package for differential expression analysis of digital gene expression data. Bioinformatics. 2010;26:139\u201340.","journal-title":"Bioinformatics"},{"key":"ref31","doi-asserted-by":"crossref","first-page":"550","DOI":"10.1186\/s13059-014-0550-8","article-title":"Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2","volume":"15","author":"Love MI","year":"2014","unstructured":"Love MI, Huber W, Anders S. Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2. Genome Biol. 2014;15:550.","journal-title":"Genome Biol"},{"key":"ref32","doi-asserted-by":"crossref","first-page":"e47","DOI":"10.1093\/nar\/gkv007","article-title":"limma powers differential expression analyses for RNA-sequencing and microarray studies","volume":"43","author":"Ritchie ME","year":"2015","unstructured":"Ritchie ME, et al. limma powers differential expression analyses for RNA-sequencing and microarray studies. Nucleic Acids Res. 2015;43:e47.","journal-title":"Nucleic Acids Res"},{"key":"ref33","doi-asserted-by":"crossref","first-page":"15545","DOI":"10.1073\/pnas.0506580102","article-title":"Gene set enrichment analysis: a knowledge-based approach for interpreting genome-wide expression profiles","volume":"102","author":"Subramanian A","year":"2005","unstructured":"Subramanian A, et al. Gene set enrichment analysis: a knowledge-based approach for interpreting genome-wide expression profiles. Proc Natl Acad Sci USA. 2005;102:15545\u201350.","journal-title":"Proc Natl Acad Sci USA"},{"key":"ref34","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1186\/s13059-020-1949-z","article-title":"Robustness and applicability of transcription factor and pathway analysis tools on single-cell RNA-seq data","volume":"21","author":"Holland CH","year":"2020","unstructured":"Holland CH, et al. Robustness and applicability of transcription factor and pathway analysis tools on single-cell RNA-seq data. Genome Biol. 2020;21:36.","journal-title":"Genome Biol"},{"key":"ref35","doi-asserted-by":"crossref","first-page":"D886","DOI":"10.1093\/nar\/gkae1142","article-title":"CZ CELLxGENE Discover: a single-cell data platform for scalable exploration, analysis and modeling of aggregated data","volume":"53","author":"CZI Single-Cell Biology Program","year":"2025","unstructured":"CZI Single-Cell Biology Program. CZ CELLxGENE Discover: a single-cell data platform for scalable exploration, analysis and modeling of aggregated data. Nucleic Acids Res. 2025;53:D886\u2013900.","journal-title":"Nucleic Acids Res"},{"key":"ref36","doi-asserted-by":"crossref","first-page":"2953","DOI":"10.1016\/j.csbj.2020.10.007","article-title":"Benchmarking algorithms for pathway activity transformation of single-cell RNA-seq data","volume":"18","author":"Zhang Y","year":"2020","unstructured":"Zhang Y, et al. Benchmarking algorithms for pathway activity transformation of single-cell RNA-seq data. Comput Struct Biotechnol J. 2020;18:2953\u201361.","journal-title":"Comput Struct Biotechnol J"}],"container-title":[],"original-title":[],"link":[{"URL":"https:\/\/www.researchsquare.com\/article\/rs-10271634\/v1","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.researchsquare.com\/article\/rs-10271634\/v1.html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,14]],"date-time":"2026-09-14T16:16:07Z","timestamp":1789402567000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.researchsquare.com\/article\/rs-10271634\/v1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,10]]},"references-count":36,"URL":"https:\/\/doi.org\/10.21203\/rs.3.rs-10271634\/v1","relation":{"is-preprint-of":[{"id-type":"doi","id":"10.1186\/s12859-026-06632-8","asserted-by":"subject"}]},"subject":[],"published":{"date-parts":[[2026,7,10]]},"subtype":"preprint"}}