{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"institution":[{"name":"bioRxiv"}],"indexed":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T06:34:26Z","timestamp":1782369266256,"version":"3.54.5"},"posted":{"date-parts":[[2026,4,2]]},"group-title":"Bioinformatics","reference-count":30,"publisher":"openRxiv","license":[{"start":{"date-parts":[[2026,4,2]],"date-time":"2026-04-02T00:00:00Z","timestamp":1775088000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"accepted":{"date-parts":[[2026,4,2]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                <jats:p>Foundation models have transformed scientific data modelling across domains, yet flow cytometry has lacked one. Despite the abundance of high-dimensional cellular data, automated analysis remains bottlenecked by marker variability: prior studies are typically confined to fixed marker panels and homogeneous data, limiting scalability and generalisation due to architectural constraints. We present the Generalised Pretrained Cytometry Transformer (GPCT), an interpretable framework designed to learn from heterogeneous marker panels for sample-level predictive modelling. Through a novel cytometry-specific pretraining regime, GPCT learns transferable cellular representations that achieve high classification accuracy across diverse datasets. Notably, pretraining significantly boosts performance on data-scarce downstream tasks, marking a pivotal step towards a cytometry foundation model. Furthermore, GPCT maintains interpretability and identifies the specific cell subsets most influential to its predictions. This enables direct biological validation of learned patterns and provides a data-driven basis for refining traditional gating strategies.<\/jats:p>","DOI":"10.64898\/2026.03.31.712806","type":"posted-content","created":{"date-parts":[[2026,4,3]],"date-time":"2026-04-03T03:25:11Z","timestamp":1775186711000},"source":"Crossref","is-referenced-by-count":1,"title":["Towards a Cytometry Foundation Model: Interpretable Sample-level Predictive Modelling via Pretrained Transformers"],"prefix":"10.64898","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-2864-2638","authenticated-orcid":false,"given":"Zixin","family":"Zhuang","sequence":"first","affiliation":[{"name":"School of Computing, The Australian National University, Canberra, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Benjamin S.","family":"Mashford","sequence":"additional","affiliation":[{"name":"John Curtin School of Medical Research, The Australian National University, Canberra, Australia"},{"name":"School of Computing, The Australian National University, Canberra, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Liang","family":"Zheng","sequence":"additional","affiliation":[{"name":"School of Computing, The Australian National University, Canberra, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"T. Daniel","family":"Andrews","sequence":"additional","affiliation":[{"name":"John Curtin School of Medical Research, The Australian National University, Canberra, Australia"},{"name":"School of Computing, The Australian National University, Canberra, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"54368","reference":[{"issue":"3","key":"2026040609000540000_2026.03.31.712806v1.1","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1038\/nmeth.2365","article-title":"Critical assessment of automated flow cytometry data analysis techniques","volume":"10","year":"2013","journal-title":"Nature methods"},{"key":"2026040609000540000_2026.03.31.712806v1.2","doi-asserted-by":"crossref","first-page":"787574","DOI":"10.3389\/fimmu.2021.787574","article-title":"Application of machine learning for cytometry data","volume":"12","year":"2022","journal-title":"Frontiers in immunology"},{"issue":"1","key":"2026040609000540000_2026.03.31.712806v1.3","doi-asserted-by":"crossref","first-page":"14825","DOI":"10.1038\/ncomms14825","article-title":"Sensitive detection of rare disease-associated cell subsets via representation learning","volume":"8","year":"2017","journal-title":"Nature communications"},{"key":"2026040609000540000_2026.03.31.712806v1.4","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2003026117"},{"key":"2026040609000540000_2026.03.31.712806v1.5","doi-asserted-by":"crossref","unstructured":"Weijler, L. , Kowarsch, F. , Reiter, M. , Hermosilla, P. , Maurer-Granofszky, M. , Dworzak, M. : Fate: Feature-agnostic transformer-based encoder for learning generalized embedding spaces in flow cytometry data. In: Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision, pp. 7956\u20137964 (2024)","DOI":"10.1109\/WACV57701.2024.00777"},{"key":"2026040609000540000_2026.03.31.712806v1.6","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2022.105314"},{"key":"2026040609000540000_2026.03.31.712806v1.7","doi-asserted-by":"publisher","DOI":"10.1016\/j.modpat.2023.100373"},{"issue":"10","key":"2026040609000540000_2026.03.31.712806v1.8","doi-asserted-by":"crossref","first-page":"585","DOI":"10.1093\/bioinformatics\/btad585","article-title":"A cell-level discriminative neural network model for diagnosis of blood cancers","volume":"39","year":"2023","journal-title":"Bioinformatics"},{"key":"2026040609000540000_2026.03.31.712806v1.9","doi-asserted-by":"crossref","unstructured":"Yi, H. , Stanley, N. : Cytoset: Predicting clinical outcomes via set-modeling of cytometry data. In: Proceedings of the 12th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics, pp. 1\u20138 (2021)","DOI":"10.1145\/3459930.3469529"},{"issue":"3","key":"2026040609000540000_2026.03.31.712806v1.10","doi-asserted-by":"crossref","first-page":"264","DOI":"10.1093\/ajcp\/aqad145","article-title":"Auto-mated prediction of acute promyelocytic leukemia from flow cytometry data using a graph neural network pipeline","volume":"161","year":"2024","journal-title":"American journal of clinical pathology"},{"key":"2026040609000540000_2026.03.31.712806v1.11","unstructured":"Vaswani, A. , Shazeer, N. , Parmar, N. , Uszkoreit, J. , Jones, L. , Gomez, A.N. , Kaiser, \u0141 ., Polosukhin, I. : Attention is all you need. Advances in neural information processing systems 30 (2017)"},{"key":"2026040609000540000_2026.03.31.712806v1.12","article-title":"Large language models: A survey","year":"2024","journal-title":"arXiv preprint"},{"issue":"8","key":"2026040609000540000_2026.03.31.712806v1.13","doi-asserted-by":"crossref","first-page":"1430","DOI":"10.1038\/s41592-024-02353-z","article-title":"Transformers in single-cell omics: a review and new perspectives","volume":"21","year":"2024","journal-title":"Nature methods"},{"key":"2026040609000540000_2026.03.31.712806v1.14","doi-asserted-by":"publisher","DOI":"10.1038\/S41586-024-07487-W"},{"key":"2026040609000540000_2026.03.31.712806v1.15","article-title":"A survey of large language models","year":"2023","journal-title":"arXiv preprint"},{"key":"2026040609000540000_2026.03.31.712806v1.16","unstructured":"Lee, J. , Lee, Y. , Kim, J. , Kosiorek, A. , Choi, S. , Teh, Y.W. : Set transformer: A framework for attention-based permutation-invariant neural networks. In: International Conference on Machine Learning, pp. 3744\u20133753 (2019). PMLR"},{"key":"2026040609000540000_2026.03.31.712806v1.17","doi-asserted-by":"crossref","unstructured":"Kowarsch, F. , Weijler, L. , W\u00f6dlinger, M. , Reiter, M. , Maurer-Granofszky, M. , Schumich, A. , Sajaroff, E.O. , Groeneveld-Krentz, S. , Rossi, J.G. , Karawajew, L. , et al.: Towards self-explainable transformers for cell classification in flow cytometry data. In: International Workshop on Interpretability of Machine Intelligence in Medical Image Computing, pp. 22\u201332 (2022). Springer","DOI":"10.1007\/978-3-031-17976-1_3"},{"key":"2026040609000540000_2026.03.31.712806v1.18","doi-asserted-by":"publisher","DOI":"10.1016\/S1074-7613(01)00199-6"},{"key":"2026040609000540000_2026.03.31.712806v1.19","doi-asserted-by":"publisher","DOI":"10.1109\/TCBBIO.2025.3562597"},{"key":"2026040609000540000_2026.03.31.712806v1.20","doi-asserted-by":"publisher","DOI":"10.1007\/s00335-012-9422-2"},{"key":"2026040609000540000_2026.03.31.712806v1.21","doi-asserted-by":"publisher","DOI":"10.1038\/s41590-019-0549-0"},{"key":"2026040609000540000_2026.03.31.712806v1.22","article-title":"Umap: Uniform manifold approximation and projection for dimension reduction","year":"2018","journal-title":"arXiv preprint"},{"key":"2026040609000540000_2026.03.31.712806v1.23","doi-asserted-by":"crossref","unstructured":"Mashford, B.S. , Hewitt, T. , May, M. , Zhuang, Z. , Jain, A. , Diamand, K.E. , Li, F.-J. , Kwong, K. , Read, S.H. , Davies, A.R. , et al.: Superior batch alignment and hyper-dimensional cytometry representations allow ultra-sensitive classification of disease phenotypes. bioRxiv, 2025\u201307 (2025)","DOI":"10.1101\/2025.07.28.666458"},{"key":"2026040609000540000_2026.03.31.712806v1.24","doi-asserted-by":"crossref","unstructured":"Chefer, H. , Gur, S. , Wolf, L. : Transformer interpretability beyond attention visualization. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, pp. 782\u2013791 (2021)","DOI":"10.1109\/CVPR46437.2021.00084"},{"key":"2026040609000540000_2026.03.31.712806v1.25","doi-asserted-by":"crossref","unstructured":"Chefer, H. , Gur, S. , Wolf, L. : Generic attention-model explainability for interpreting bi-modal and encoder-decoder transformers. In: Proceedings of the IEEE\/CVF International Conference on Computer Vision, pp. 397\u2013406 (2021)","DOI":"10.1109\/ICCV48922.2021.00045"},{"key":"2026040609000540000_2026.03.31.712806v1.26","unstructured":"Xiong, R. , Yang, Y. , He, D. , Zheng, K. , Zheng, S. , Xing, C. , Zhang, H. , Lan, Y. , Wang, L. , Liu, T. : On layer normalization in the transformer architecture. In: International Conference on Machine Learning, pp. 10524\u201310533 (2020). PMLR"},{"key":"2026040609000540000_2026.03.31.712806v1.27","doi-asserted-by":"crossref","unstructured":"Vincent, P. , Larochelle, H. , Bengio, Y. , Manzagol, P.-A. : Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th International Conference on Machine Learning, pp. 1096\u20131103 (2008)","DOI":"10.1145\/1390156.1390294"},{"key":"2026040609000540000_2026.03.31.712806v1.28","first-page":"4171","article-title":"Bert: Pre-training of deep bidirectional transformers for language understanding","volume":"1","year":"2019","journal-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies"},{"key":"2026040609000540000_2026.03.31.712806v1.29","article-title":"Beit: Bert pre-training of image transformers","year":"2021","journal-title":"arXiv preprint"},{"key":"2026040609000540000_2026.03.31.712806v1.30","article-title":"A survey on masked autoencoder for self-supervised learning in vision and beyond","year":"2022","journal-title":"arXiv preprint"}],"container-title":[],"original-title":[],"link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.64898\/2026.03.31.712806","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,6]],"date-time":"2026-04-06T16:00:22Z","timestamp":1775491222000},"score":1,"resource":{"primary":{"URL":"http:\/\/biorxiv.org\/lookup\/doi\/10.64898\/2026.03.31.712806"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,2]]},"references-count":30,"URL":"https:\/\/doi.org\/10.64898\/2026.03.31.712806","relation":{},"subject":[],"published":{"date-parts":[[2026,4,2]]},"subtype":"preprint"}}