{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,25]],"date-time":"2026-07-25T05:48:54Z","timestamp":1784958534136,"version":"3.55.0"},"reference-count":39,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T00:00:00Z","timestamp":1774569600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T00:00:00Z","timestamp":1774569600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100000002","name":"U.S. Department of Health & Human Services | National Institutes of Health","doi-asserted-by":"publisher","award":["DP5OD036170"],"award-info":[{"award-number":["DP5OD036170"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000861","name":"Burroughs Wellcome Fund","doi-asserted-by":"publisher","award":["1022136.01"],"award-info":[{"award-number":["1022136.01"]}],"id":[{"id":"10.13039\/100000861","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Biswas Foundation, Milken Institute"},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["2034836"],"award-info":[{"award-number":["2034836"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000015","name":"U.S. Department of Energy","doi-asserted-by":"publisher","award":["DE-AC02-06CH11357"],"award-info":[{"award-number":["DE-AC02-06CH11357"]}],"id":[{"id":"10.13039\/100000015","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000015","name":"U.S. Department of Energy","doi-asserted-by":"publisher","award":["DE-AC02-06CH11357"],"award-info":[{"award-number":["DE-AC02-06CH11357"]}],"id":[{"id":"10.13039\/100000015","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000054","name":"U.S. Department of Health & Human Services | NIH | National Cancer Institute","doi-asserted-by":"publisher","award":["R00CA259511"],"award-info":[{"award-number":["R00CA259511"]}],"id":[{"id":"10.13039\/100000054","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000054","name":"U.S. Department of Health & Human Services | NIH | National Cancer Institute","doi-asserted-by":"publisher","award":["Cancer Cell Map Initiative"],"award-info":[{"award-number":["Cancer Cell Map Initiative"]}],"id":[{"id":"10.13039\/100000054","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Parker Institute for Cancer Immunotherapy, funds from the CRISPR Cure for Cancer Initiative, the UCSF Program for Breakthrough Biomedical Research, Biswas Foundation, and Milken Institute"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Nat Comput Sci"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Foundation models for network biology are pretrained on large-scale biological data to enable context-aware predictions in a diverse array of downstream tasks through transfer learning. However, increasing model sizes with the expansion of available pretraining data also increases the computational resources required for fine-tuning and inference in downstream applications. Here we first assemble a corpus comprising ~104\u2009million human single-cell transcriptomes from a broad range of tissues and diseases and pretrain successively larger models, defining the scaling laws for transcriptional masked learning. We then demonstrate that model quantization preserves the contextual gene and cell embedding space of the full-precision model, matching performance in zero-shot and fine-tuning applications while requiring only 15% of the time and 34% of the memory as the full model for fine-tuning with the same batch size. Overall, model quantization represents an effective method for resource-efficient fine-tuning and inference while preserving biological knowledge.<\/jats:p>","DOI":"10.1038\/s43588-026-00972-4","type":"journal-article","created":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T10:02:18Z","timestamp":1774605738000},"page":"450-463","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Scaling and quantization of large-scale foundation model enables resource-efficient predictions in network biology"],"prefix":"10.1038","volume":"6","author":[{"given":"Han","family":"Chen","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Madhavan S.","family":"Venkatesh","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Javier","family":"G\u03ccmez Ortega","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Siddharth V.","family":"Mahesh","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tarak N.","family":"Nandi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2130-2887","authenticated-orcid":false,"given":"Ravi K.","family":"Madduri","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2856-8602","authenticated-orcid":false,"given":"Karin","family":"Pelka","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1658-1447","authenticated-orcid":false,"given":"Christina V.","family":"Theodoris","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,3,27]]},"reference":[{"key":"972_CR1","doi-asserted-by":"publisher","first-page":"1072","DOI":"10.1016\/j.cell.2015.02.035","volume":"160","author":"CV Theodoris","year":"2015","unstructured":"Theodoris, C. V. et al. Human disease modeling reveals integrated transcriptional and epigenetic mechanisms of NOTCH1 haploinsufficiency. Cell 160, 1072\u20131086 (2015).","journal-title":"Cell"},{"key":"972_CR2","doi-asserted-by":"publisher","first-page":"eabd0724","DOI":"10.1126\/science.abd0724","volume":"371","author":"CV Theodoris","year":"2021","unstructured":"Theodoris, C. V. et al. Network-based screen in iPSC-derived cells reveals therapeutic candidate for heart valve disease. Science 371, eabd0724 (2021).","journal-title":"Science"},{"key":"972_CR3","unstructured":"Vaswani, A. et al. Attention is all you need. Adv. Neural Inf. Process. Syst. 30, (2017)."},{"key":"972_CR4","unstructured":"Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. BERT: pre-training of deep bidirectional transformers for language understanding. Preprint at https:\/\/arxiv.org\/abs\/1810.04805 (2018)."},{"key":"972_CR5","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 770\u2013778 (IEEE, 2015).","DOI":"10.1109\/CVPR.2016.90"},{"key":"972_CR6","doi-asserted-by":"publisher","first-page":"616","DOI":"10.1038\/s41586-023-06139-9","volume":"618","author":"CV Theodoris","year":"2023","unstructured":"Theodoris, C. V. et al. Transfer learning enables predictions in network biology. Nature 618, 616\u2013624 (2023).","journal-title":"Nature"},{"key":"972_CR7","doi-asserted-by":"publisher","first-page":"852","DOI":"10.1038\/s42256-022-00534-z","volume":"4","author":"F Yang","year":"2022","unstructured":"Yang, F. et al. scBERT as a large-scale pretrained deep language model for cell type annotation of single-cell RNA-seq data. Nat. Mach. Intell. 4, 852\u2013866 (2022).","journal-title":"Nat. Mach. Intell."},{"key":"972_CR8","doi-asserted-by":"publisher","DOI":"10.1016\/j.isci.2023.106536","volume":"26","author":"H Shen","year":"2023","unstructured":"Shen, H. et al. Generative pretraining from large-scale transcriptomes for single-cell deciphering. iScience 26, 106536 (2023).","journal-title":"iScience"},{"key":"972_CR9","doi-asserted-by":"publisher","first-page":"1470","DOI":"10.1038\/s41592-024-02201-0","volume":"21","author":"H Cui","year":"2024","unstructured":"Cui, H. et al. scGPT: toward building a foundation model for single-cell multi-omics using generative AI. Nat. Methods 21, 1470\u20131480 (2024).","journal-title":"Nat. Methods"},{"key":"972_CR10","doi-asserted-by":"publisher","first-page":"1481","DOI":"10.1038\/s41592-024-02305-7","volume":"21","author":"M Hao","year":"2024","unstructured":"Hao, M. et al. Large-scale foundation model on single-cell transcriptomics. Nat. Methods 21, 1481\u20131491 (2024).","journal-title":"Nat. Methods"},{"key":"972_CR11","doi-asserted-by":"publisher","first-page":"830","DOI":"10.1038\/s41422-024-01034-y","volume":"34","author":"X Yang","year":"2024","unstructured":"Yang, X. et al. GeneCompass: deciphering universal gene regulatory mechanisms with a knowledge-informed cross-species foundation model. Cell Res 34, 830\u2013845 (2024).","journal-title":"Cell Res"},{"key":"972_CR12","doi-asserted-by":"publisher","unstructured":"Rosen, Y. et al. Universal cell embeddings: a foundation model for cell biology. Preprint at bioRxiv https:\/\/doi.org\/10.1101\/2023.11.28.568918 (2024).","DOI":"10.1101\/2023.11.28.568918"},{"key":"972_CR13","first-page":"2525","volume":"22","author":"AC Schaar","year":"2024","unstructured":"Schaar, A. C. et al. Nicheformer: a foundation model for single-cell and spatial omics. Bioinformatics 22, 2525\u20132538 (2024).","journal-title":"Bioinformatics"},{"key":"972_CR14","unstructured":"Kaplan, J. et al. Scaling laws for neural language models. Preprint at https:\/\/arxiv.org\/abs\/2001.08361 (2020)."},{"key":"972_CR15","doi-asserted-by":"publisher","first-page":"101","DOI":"10.1186\/s13059-025-03574-x","volume":"26","author":"KZ Kedzierska","year":"2025","unstructured":"Kedzierska, K. Z., Crawford, L., Amini, A. P. & Lu, A. X. Zero-shot evaluation reveals limitations of single-cell foundation models. Genome Biol. 26, 101 (2025).","journal-title":"Genome Biol."},{"key":"972_CR16","doi-asserted-by":"crossref","unstructured":"Dettmers, T., Pagnoni, A., Holtzman, A. & Zettlemoyer, L. QLoRA: efficient finetuning of quantized LLMs. Preprint at https:\/\/arxiv.org\/abs\/2305.14314 (2023).","DOI":"10.52202\/075280-0441"},{"key":"972_CR17","doi-asserted-by":"publisher","first-page":"lqaa078","DOI":"10.1093\/nargab\/lqaa078","volume":"2","author":"Y Zhang","year":"2020","unstructured":"Zhang, Y., Parmigiani, G. & Johnson, W. E. ComBat-seq: batch effect adjustment for RNA-seq count data. NAR Genom. Bioinform. 2, lqaa078 (2020).","journal-title":"NAR Genom. Bioinform."},{"key":"972_CR18","doi-asserted-by":"publisher","first-page":"1289","DOI":"10.1038\/s41592-019-0619-0","volume":"16","author":"I Korsunsky","year":"2019","unstructured":"Korsunsky, I. et al. Fast, sensitive and accurate integration of single-cell data with Harmony. Nat. Methods 16, 1289\u20131296 (2019).","journal-title":"Nat. Methods"},{"key":"972_CR19","doi-asserted-by":"publisher","first-page":"292","DOI":"10.1016\/j.devcel.2020.11.020","volume":"56","author":"IS Kathiriya","year":"2021","unstructured":"Kathiriya, I. S. et al. Modeling human TBX5 haploinsufficiency predicts regulatory networks for congenital heart disease. Dev. Cell 56, 292\u2013309.e9 (2021).","journal-title":"Dev. Cell"},{"key":"972_CR20","doi-asserted-by":"publisher","first-page":"1734","DOI":"10.1016\/j.cell.2016.11.033","volume":"167","author":"Y-S Ang","year":"2016","unstructured":"Ang, Y.-S. et al. Disease model of GATA4 mutation reveals transcription factor cooperativity in human cardiogenesis. Cell 167, 1734\u20131749.e22 (2016).","journal-title":"Cell"},{"key":"972_CR21","unstructured":"Warner, B. et al. Smarter, better, faster, longer: a modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference. Preprint at https:\/\/arxiv.org\/abs\/2412.13663 (2024)."},{"key":"972_CR22","doi-asserted-by":"crossref","unstructured":"Svensson, V., da Veiga Beltrame, E. & Pachter, L. A curated database reveals trends in single-cell transcriptomics. Database 2020, (2020).","DOI":"10.1093\/database\/baaa073"},{"key":"972_CR23","doi-asserted-by":"publisher","DOI":"10.1186\/s13059-016-0953-9","volume":"17","author":"J Xin","year":"2016","unstructured":"Xin, J. et al. High-performance web services for querying gene and variant annotation. Genome Biol. 17, 91 (2016).","journal-title":"Genome Biol."},{"key":"972_CR24","doi-asserted-by":"crossref","unstructured":"Rajbhandari, S., Rasley, J., Ruwase, O. & He, Y. ZeRO: memory optimizations toward training trillion parameter models. Preprint at https:\/\/arxiv.org\/abs\/1910.02054 (2019).","DOI":"10.1109\/SC41405.2020.00024"},{"key":"972_CR25","unstructured":"Ren, J. et al. ZeRO-Offload: democratizing billion-scale model training. Preprint at https:\/\/arxiv.org\/abs\/2101.06840 (2021)."},{"key":"972_CR26","unstructured":"Hu, E. J. et al. LoRA: low-rank adaptation of large language models. Preprint at https:\/\/arxiv.org\/abs\/2106.09685 (2021)."},{"key":"972_CR27","unstructured":"Aghajanyan, A., Zettlemoyer, L. & Gupta, S. Intrinsic dimensionality explains the effectiveness of language model fine-tuning. Preprint at https:\/\/arxiv.org\/abs\/2012.13255 (2020)."},{"key":"972_CR28","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-020-58327-6","volume":"10","author":"A Selewa","year":"2020","unstructured":"Selewa, A. et al. Systematic comparison of high-throughput single-cell and single-nucleus transcriptomes during cardiomyocyte differentiation. Sci. Rep. 10, 1535 (2020).","journal-title":"Sci. Rep."},{"key":"972_CR29","unstructured":"4k PBMCs from a healthy donor. 10x Genomics (accessed 2021); https:\/\/www.10xgenomics.com\/datasets\/4-k-pbm-cs-from-a-healthy-donor-2-standard-2-1-0"},{"key":"972_CR30","doi-asserted-by":"publisher","unstructured":"Abugessaisa, I. et al. Impacts of Cell Ranger versions on Chromium gene expression data. Preprint at bioRxiv https:\/\/doi.org\/10.1101\/2024.08.10.607413 (2024).","DOI":"10.1101\/2024.08.10.607413"},{"key":"972_CR31","doi-asserted-by":"publisher","first-page":"285","DOI":"10.1038\/nature19057","volume":"536","author":"M Lek","year":"2016","unstructured":"Lek, M. et al. Analysis of protein-coding genetic variation in 60,706 humans. Nature 536, 285\u2013291 (2016).","journal-title":"Nature"},{"key":"972_CR32","doi-asserted-by":"publisher","first-page":"1751","DOI":"10.1093\/bioinformatics\/btx028","volume":"33","author":"HA Shihab","year":"2017","unstructured":"Shihab, H. A., Rogers, M. F., Campbell, C. & Gaunt, T. R. HIPred: an integrative approach to predicting haploinsufficient genes. Bioinformatics 33, 1751\u20131757 (2017).","journal-title":"Bioinformatics"},{"key":"972_CR33","doi-asserted-by":"publisher","first-page":"1208","DOI":"10.3389\/fgene.2019.01208","volume":"10","author":"Z Ni","year":"2019","unstructured":"Ni, Z., Zhou, X.-Y., Aslam, S. & Niu, D.-K. Characterization of human dosage-sensitive transcription factor genes. Front. Genet. 10, 1208 (2019).","journal-title":"Front. Genet."},{"key":"972_CR34","doi-asserted-by":"publisher","first-page":"315","DOI":"10.1016\/j.cell.2006.02.041","volume":"125","author":"BE Bernstein","year":"2006","unstructured":"Bernstein, B. E. et al. A bivalent chromatin structure marks key developmental genes in embryonic stem cells. Cell 125, 315\u2013326 (2006).","journal-title":"Cell"},{"key":"972_CR35","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-020-16106-x","volume":"11","author":"C-H Chen","year":"2020","unstructured":"Chen, C.-H. et al. Determinants of transcription factor regulatory range. Nat. Commun. 11, 2472 (2020).","journal-title":"Nat. Commun."},{"key":"972_CR36","doi-asserted-by":"publisher","first-page":"299","DOI":"10.1016\/j.stem.2007.08.003","volume":"1","author":"G Pan","year":"2007","unstructured":"Pan, G. et al. Whole-genome analysis of histone H3 lysine 4 and lysine 27 methylation in human embryonic stem cells. Cell Stem Cell 1, 299\u2013312 (2007).","journal-title":"Cell Stem Cell"},{"key":"972_CR37","doi-asserted-by":"publisher","first-page":"569","DOI":"10.1016\/j.tig.2013.05.010","volume":"29","author":"E Eisenberg","year":"2013","unstructured":"Eisenberg, E. & Levanon, E. Y. Human housekeeping genes, revisited. Trends Genet. 29, 569\u2013574 (2013).","journal-title":"Trends Genet."},{"key":"972_CR38","doi-asserted-by":"publisher","unstructured":"Genecorpus-104M. Hugging Face (2026); https:\/\/doi.org\/10.57967\/hf\/7859.","DOI":"10.57967\/hf\/7859"},{"key":"972_CR39","doi-asserted-by":"publisher","unstructured":"Geneformer. Zenodo (2026); https:\/\/doi.org\/10.5281\/zenodo.18708994.","DOI":"10.5281\/zenodo.18708994"}],"container-title":["Nature Computational Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s43588-026-00972-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s43588-026-00972-4","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s43588-026-00972-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,27]],"date-time":"2026-05-27T22:14:09Z","timestamp":1779920049000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s43588-026-00972-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,27]]},"references-count":39,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2026,5]]}},"alternative-id":["972"],"URL":"https:\/\/doi.org\/10.1038\/s43588-026-00972-4","relation":{},"ISSN":["2662-8457"],"issn-type":[{"value":"2662-8457","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,27]]},"assertion":[{"value":"9 June 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 February 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 March 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"K.P. is a consultant to Santa Ana Bio, Inc. The other authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}]}}