{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,20]],"date-time":"2026-10-20T14:29:30Z","timestamp":1792506570579,"version":"4.5.0"},"reference-count":32,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2026,7,22]],"date-time":"2026-07-22T00:00:00Z","timestamp":1784678400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,7,22]],"date-time":"2026-07-22T00:00:00Z","timestamp":1784678400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100002345","name":"Eberhard Karls Universit\u00e4t T\u00fcbingen","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100002345","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Genomics"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Single-cell foundation models such as scGPT and Geneformer are increasingly used for gene regulatory network (GRN) inference, with attention-derived edge scores routinely interpreted as regulatory proxies. Prior benchmarks have evaluated curated-reference recovery but have not systematically tested whether attention adds information beyond expression statistics for predicting the outcomes of genetic perturbations, nor whether attention-identified \u201cregulatory\u201d components are causally required for such predictions. This gap matters because the NLP interpretability literature has established that attention weights do not reliably indicate feature importance, and biological foundation models are being deployed without analogous scrutiny.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We present an evaluation framework comprising thirty-seven analyses and 153 statistical tests under Benjamini-Hochberg FDR correction, spanning two architectures (scGPT, Geneformer V2-316M), four cell types (K562, RPE1, primary T cells, iPSC neurons), and two perturbation modalities (CRISPRi, CRISPRa). The framework separates two objectives: (A)\u00a0mechanistic interpretability \/ GRN recovery against curated references, and (B)\u00a0perturbation-target prediction, i.e. classifying which genes show differential expression after a CRISPR perturbation. Five test families\u2014trivial-baseline comparison, conditional incremental-value testing, residualisation and propensity matching, causal ablation with intervention-fidelity diagnostics, and cross-context replication\u2014address Objective B, supplemented by a synthetic positive control establishing pipeline sensitivity.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      Attention patterns encode layer-specific biological structure\u2014protein\u2013protein interactions in early layers, transcriptional regulation in late layers\u2014and Cell-State Stratified Interpretability (CSSI) exploits this structure to improve curated GRN recovery up to\n                      <jats:inline-formula>\n                        <jats:tex-math>$$1.85\\times$$<\/jats:tex-math>\n                      <\/jats:inline-formula>\n                      on the Objective A task. On Objective B, however, attention-derived edge scores add no incremental value beyond trivial gene-level features (variance, mean expression, dropout rate): gene-level baselines outperform both attention and correlation edges (AUROC 0.81\u20130.88 versus 0.70), augmenting gene-level predictors with pairwise edges produces\n                      <jats:inline-formula>\n                        <jats:tex-math>$$\\Delta$$<\/jats:tex-math>\n                      <\/jats:inline-formula>\n                      AUROC of\n                      <jats:inline-formula>\n                        <jats:tex-math>$$-0.0004$$<\/jats:tex-math>\n                      <\/jats:inline-formula>\n                      to\n                      <jats:inline-formula>\n                        <jats:tex-math>$$-0.002$$<\/jats:tex-math>\n                      <\/jats:inline-formula>\n                      across 559,720 perturbation\u2013gene observations, and causal ablation of TRRUST-ranked attention heads produces no degradation across three independent intervention channels. The attention\u2013correlation relationship is context-dependent (equal in K562 CRISPRi, worse in CRISPRa, better in RPE1), but gene-level dominance on Objective B is universal across both cell types where adequate power is available.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Attention patterns in single-cell foundation models encode biologically structured information, including layer-specific regulatory signals recoverable via CSSI, but provide no unique predictive information beyond simple gene-level statistics for the perturbation-target prediction task. The paper thus offers both a cautionary finding for Objective B and a constructive method (CSSI) for Objective A. Practitioners should apply trivial-baseline and incremental-value tests before claiming pairwise regulatory signal, and should stratify by cell state when extracting attention-derived GRNs.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1186\/s12864-026-12965-8","type":"journal-article","created":{"date-parts":[[2026,7,22]],"date-time":"2026-07-22T02:05:12Z","timestamp":1784685912000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Systematic evaluation of single-cell foundation model interpretability: attention-derived edge scores add no incremental value over gene-level features for perturbation-target prediction"],"prefix":"10.1186","volume":"27","author":[{"given":"Ihor","family":"Kendiukhov","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,7,22]]},"reference":[{"issue":"8","key":"12965_CR1","doi-asserted-by":"publisher","first-page":"1470","DOI":"10.1038\/s41592-024-02201-0","volume":"21","author":"H Cui","year":"2024","unstructured":"Cui H, Wang C, Maan H, Pang K, Luo F, Wang B. scGPT: toward building a foundation model for single-cell multi-omics using generative AI. Nat Methods. 2024;21(8):1470\u20139.","journal-title":"Nat Methods"},{"issue":"7965","key":"12965_CR2","doi-asserted-by":"publisher","first-page":"616","DOI":"10.1038\/s41586-023-06139-9","volume":"618","author":"CV Theodoris","year":"2023","unstructured":"Theodoris CV, Xiao L, Chopra A, Chaffin MD, Al Sayed ZR, Hill MC, et al. Transfer learning enables predictions in network biology. Nature. 2023;618(7965):616\u201324.","journal-title":"Nature"},{"issue":"10","key":"12965_CR3","doi-asserted-by":"publisher","first-page":"852","DOI":"10.1038\/s42256-022-00534-z","volume":"4","author":"F Yang","year":"2022","unstructured":"Yang F, Wang W, Wang F, Fang Y, Tang D, Huang J, et al. scBERT as a large-scale pretrained deep language model for cell type annotation of single-cell RNA-seq data. Nat Mach Intel. 2022;4(10):852\u201366.","journal-title":"Nat Mach Intel"},{"issue":"8","key":"12965_CR4","doi-asserted-by":"publisher","first-page":"1481","DOI":"10.1038\/s41592-024-02305-7","volume":"21","author":"M Hao","year":"2024","unstructured":"Hao M, Gong J, Zeng X, Liu C, Guo Y, Cheng X, et al. Large-scale foundation model on single-cell transcriptomics. Nat Methods. 2024;21(8):1481\u201391.","journal-title":"Nat Methods"},{"key":"12965_CR5","doi-asserted-by":"publisher","unstructured":"Chen Y, Bhatt K, et\u00a0al. GenePT: a simple but effective foundation model for genes and cells built from ChatGPT. bioRxiv. 2024. https:\/\/doi.org\/10.1101\/2023.10.16.562533.","DOI":"10.1101\/2023.10.16.562533"},{"key":"12965_CR6","doi-asserted-by":"publisher","unstructured":"Rosen Y, Roohani Y, Agrawal A, Samotorcan L, Consortium TS, Quake SR, et\u00a0al. Universal cell embeddings: a foundation model for cell biology. bioRxiv. 2024. https:\/\/doi.org\/10.1101\/2023.11.28.568918.","DOI":"10.1101\/2023.11.28.568918"},{"key":"12965_CR7","doi-asserted-by":"publisher","unstructured":"Zheng R, Gao S, et al. Benchmarking computational methods for single-cell gene regulatory network reconstruction from multimodal data. Nat Methods. 2024. https:\/\/doi.org\/10.1038\/s41592-024-02410-x.","DOI":"10.1038\/s41592-024-02410-x"},{"key":"12965_CR8","doi-asserted-by":"publisher","first-page":"17359","DOI":"10.52202\/068431-1262","volume":"35","author":"K Meng","year":"2022","unstructured":"Meng K, Bau D, Andonian A, Belinkov Y. Locating and editing factual associations in GPT. Adv Neural Inf Process Syst. 2022;35:17359\u201372.","journal-title":"Adv Neural Inf Process Syst"},{"key":"12965_CR9","unstructured":"Goldowsky-Dill N, MacLeod C, Sato L, Arora A. Localizing model behavior with path patching. 2023. arXiv preprint arXiv:2304.05969."},{"key":"12965_CR10","doi-asserted-by":"crossref","unstructured":"Conmy A, Mavor-Parker AN, Lynch A, Heimersheim S, Garriga-Alonso A. Towards automated circuit discovery for mechanistic interpretability. Adv Neural Inf Process Syst. 2023;36:16318\u201352.","DOI":"10.52202\/075280-0719"},{"key":"12965_CR11","unstructured":"Elhage N, Nanda N, Olsson C, Henighan T, Joseph N, Mann B, et\u00a0al. A mathematical framework for transformer circuits. Transformer Circ Thread. 2021:12."},{"key":"12965_CR12","unstructured":"Olsson C, Elhage N, Nanda N, Joseph N, DasSarma N, Henighan T, et\u00a0al. In-context learning and induction heads. 2022. arXiv preprint arXiv:2209.11895."},{"key":"12965_CR13","unstructured":"Wang K, Variengien A, Conmy A, Shlegeris B, Steinhardt J. Interpretability in the wild: a circuit for indirect object identification in GPT-2 small. 2022. arXiv preprint arXiv:2211.00593."},{"issue":"D1","key":"12965_CR14","doi-asserted-by":"publisher","first-page":"D380","DOI":"10.1093\/nar\/gkx1013","volume":"46","author":"H Han","year":"2018","unstructured":"Han H, Cho JW, Lee S, Yun A, Kim H, Bae D, et al. TRRUST v2: an expanded reference database of human and mouse transcriptional regulatory interactions. Nucleic Acids Res. 2018;46(D1):D380\u20136.","journal-title":"Nucleic Acids Res"},{"issue":"8","key":"12965_CR15","doi-asserted-by":"publisher","first-page":"1363","DOI":"10.1101\/gr.240663.118","volume":"29","author":"L Garcia-Alonso","year":"2019","unstructured":"Garcia-Alonso L, Holland CH, Ibrahim MM, Turei D, Saez-Rodriguez J. Benchmark and integration of resources for the estimation of human transcription factor activities. Genome Res. 2019;29(8):1363\u201375.","journal-title":"Genome Res"},{"issue":"2","key":"12965_CR16","doi-asserted-by":"publisher","first-page":"147","DOI":"10.1038\/s41592-019-0690-6","volume":"17","author":"A Pratapa","year":"2020","unstructured":"Pratapa A, Jalihal AP, Law JN, Bharadwaj A, Murali TM. Benchmarking algorithms for gene regulatory network inference from single-cell transcriptomic data. Nat Methods. 2020;17(2):147\u201354.","journal-title":"Nat Methods"},{"key":"12965_CR17","unstructured":"Jain S, Wallace BC. Attention is not explanation. 2019. arXiv preprint arXiv:1902.10186."},{"key":"12965_CR18","unstructured":"Serrano S, Smith NA. Is attention interpretable? 2019. arXiv preprint arXiv:1906.03731."},{"key":"12965_CR19","doi-asserted-by":"crossref","unstructured":"Bibal A, Cardon R, Demeester T, et\u00a0al. Is attention explanation? An introduction to the debate. 2022. arXiv preprint arXiv:2204.12710.","DOI":"10.18653\/v1\/2022.acl-long.269"},{"issue":"7","key":"12965_CR20","doi-asserted-by":"publisher","first-page":"1853","DOI":"10.1016\/j.cell.2016.11.038","volume":"167","author":"A Dixit","year":"2016","unstructured":"Dixit A, Parnas O, Li B, Chen J, Fulco CP, Jerber L, et al. Perturb-seq: dissecting molecular circuits with scalable single-cell RNA profiling of pooled genetic screens. Cell. 2016;167(7):1853\u201366.","journal-title":"Cell"},{"issue":"D1","key":"12965_CR21","doi-asserted-by":"publisher","first-page":"D483","DOI":"10.1093\/nar\/gkaa1074","volume":"49","author":"D Szklarczyk","year":"2021","unstructured":"Szklarczyk D, Gable AL, Nastou KC, Lyon D, Kirsch R, Pyysalo S, et al. The STRING database in 2021: customizable protein-protein networks, and functional characterization of user-uploaded gene\/measurement sets. Nucleic Acids Res. 2021;49(D1):D483\u20139.","journal-title":"Nucleic Acids Res"},{"issue":"14","key":"12965_CR22","doi-asserted-by":"publisher","first-page":"2559","DOI":"10.1016\/j.cell.2022.05.013","volume":"185","author":"JM Replogle","year":"2022","unstructured":"Replogle JM, Saunders RA, Pogson AN, Hussmann JA, Lenail A, Guna A, et al. Mapping information-rich genotype-phenotype landscapes with genome-scale Perturb-seq. Cell. 2022;185(14):2559\u201375.","journal-title":"Cell"},{"key":"12965_CR23","doi-asserted-by":"crossref","unstructured":"Voita E, Talbot D, Moiseev F, Sennrich R, Titov I. Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics; 2019. p. 5797\u2013808.","DOI":"10.18653\/v1\/P19-1580"},{"key":"12965_CR24","doi-asserted-by":"crossref","unstructured":"Clark K, Khandelwal U, Levy O, Manning CD. What does BERT look at? An analysis of BERT\u2019s attention. In: Proceedings of the 2019 ACL Workshop BlackboxNLP. 2019. pp. 276\u2013286.","DOI":"10.18653\/v1\/W19-4828"},{"issue":"7","key":"12965_CR25","doi-asserted-by":"publisher","first-page":"1867","DOI":"10.1016\/j.cell.2016.11.048","volume":"167","author":"B Adamson","year":"2016","unstructured":"Adamson B, Norman TM, Jost M, Cho MY, Nu\u00f1ez JK, Chen Y, et al. A multiplexed single-cell CRISPR screening platform enables systematic dissection of the unfolded protein response. Cell. 2016;167(7):1867\u201382.","journal-title":"Cell"},{"issue":"7","key":"12965_CR26","doi-asserted-by":"publisher","first-page":"1958","DOI":"10.1016\/j.cell.2018.10.024","volume":"175","author":"E Shifrut","year":"2018","unstructured":"Shifrut E, Carnevale J, Tober V, Greenstein RA, Deitch M, Haliburton J, et al. Genome-wide CRISPR screens in primary human T cells reveal key regulators of immune function. Cell. 2018;175(7):1958\u201371.","journal-title":"Cell"},{"key":"12965_CR27","doi-asserted-by":"crossref","unstructured":"Wiegreffe S, Pinter Y. Attention is not not explanation. 2019. arXiv preprint arXiv:1908.04626.","DOI":"10.18653\/v1\/D19-1002"},{"key":"12965_CR28","unstructured":"Vovk V, Gammerman A, Shafer G. Algorithmic Learning in a Random World. Springer; 2005."},{"key":"12965_CR29","unstructured":"The Tabula Sapiens Consortium. The Tabula Sapiens: a multiple-organ, single-cell transcriptomic atlas of humans. Science. 2022;376(6594):eabl4896."},{"issue":"3","key":"12965_CR30","doi-asserted-by":"publisher","first-page":"425","DOI":"10.1038\/s41593-020-00787-0","volume":"24","author":"KR Maynard","year":"2021","unstructured":"Maynard KR, Collado-Torres L, Weber LM, Uytingco C, Barry BK, Williams SR, et al. Transcriptome-scale spatial gene expression in the human dorsolateral prefrontal cortex. Nat Neurosci. 2021;24(3):425\u201336.","journal-title":"Nat Neurosci"},{"key":"12965_CR31","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1186\/s13059-017-1382-0","volume":"19","author":"FA Wolf","year":"2018","unstructured":"Wolf FA, Angerer P, Theis FJ. SCANPY: large-scale single-cell gene expression data analysis. Genome Biol. 2018;19:15.","journal-title":"Genome Biol"},{"key":"12965_CR32","doi-asserted-by":"crossref","unstructured":"Benjamini Y, Hochberg Y.the false discovery rate: a practical and Controlling powerful approach to multiple testing. J R Stat Soc Ser B. 1995;57(1):289\u2013300.","DOI":"10.1111\/j.2517-6161.1995.tb02031.x"}],"container-title":["BMC Genomics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12864-026-12965-8","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12864-026-12965-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12864-026-12965-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,22]],"date-time":"2026-07-22T06:04:36Z","timestamp":1784700276000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1186\/s12864-026-12965-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,22]]},"references-count":32,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,12]]}},"alternative-id":["12965"],"URL":"https:\/\/doi.org\/10.1186\/s12864-026-12965-8","relation":{},"ISSN":["1471-2164"],"issn-type":[{"value":"1471-2164","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,22]]},"assertion":[{"value":"10 March 2026","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 May 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 July 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Not applicable. This study used only publicly available, de-identified single-cell transcriptomic datasets. No new human or animal data were generated.","order":1,"name":"Ethics","label":"Ethical approval and consent to participate","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","label":"Consent for publication","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":3,"name":"Ethics","label":"Competing interests","group":{"name":"EthicsHeading","label":"Declarations"}}],"article-number":"634"}}