{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,2]],"date-time":"2026-04-02T16:27:35Z","timestamp":1775147255965,"version":"3.50.1"},"reference-count":30,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2023,4,25]],"date-time":"2023-04-25T00:00:00Z","timestamp":1682380800000},"content-version":"vor","delay-in-days":1,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000925","name":"National Health and Medical Research Council","doi-asserted-by":"publisher","award":["1173469"],"award-info":[{"award-number":["1173469"]}],"id":[{"id":"10.13039\/501100000925","id-type":"DOI","asserted-by":"publisher"}]},{"name":"AIR@innoHK programme of the Innovation and Technology Commission of Hong Kong"},{"name":"Chan Zuckerberg Initiative Single Cell Biology Data Insights","award":["DI-0000000027"],"award-info":[{"award-number":["DI-0000000027"]}]},{"name":"Australian Research Council Discovery Early Career Researcher Awards","award":["DE220100964"],"award-info":[{"award-number":["DE220100964"]}]},{"name":"University of Sydney Postgraduate Award Stipend Scholarship"},{"name":"Research Training Program Tuition Fee Offset"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,5,19]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>The advances of single-cell transcriptomic technologies have led to increasing use of single-cell RNA sequencing (scRNA-seq) data in large-scale patient cohort studies. The resulting high-dimensional data can be summarized and incorporated into patient outcome prediction models in several ways; however, there is a pressing need to understand the impact of analytical decisions on such model quality. In this study, we evaluate the impact of analytical choices on model choices, ensemble learning strategies and integrate approaches on patient outcome prediction using five scRNA-seq COVID-19 datasets. First, we examine the difference in performance between using single-view feature space versus multi-view feature space. Next, we survey multiple learning platforms from classical machine learning to modern deep learning methods. Lastly, we compare different integration approaches when combining datasets is necessary. Through benchmarking such analytical combinations, our study highlights the power of ensemble learning, consistency among different learning methods and robustness to dataset normalization when using multiple datasets as the model input.<\/jats:p>","DOI":"10.1093\/bib\/bbad159","type":"journal-article","created":{"date-parts":[[2023,4,25]],"date-time":"2023-04-25T09:05:13Z","timestamp":1682413513000},"source":"Crossref","is-referenced-by-count":4,"title":["Benchmarking of analytical combinations for COVID-19 outcome prediction using single-cell RNA sequencing data"],"prefix":"10.1093","volume":"24","author":[{"given":"Yue","family":"Cao","sequence":"first","affiliation":[{"name":"The University of Sydney School of Mathematics and Statistics, , NSW 2006 , Australia"},{"name":"The University of Sydney Charles Perkins Centre, , NSW 2006 , Australia"},{"name":"University of Sydney Sydney Precision Data Science Centre, , NSW 2006 , Australia"},{"name":"Laboratory of Data Discovery for Health Limited (D24H) , Science Park, Hong Kong SAR , China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shila","family":"Ghazanfar","sequence":"additional","affiliation":[{"name":"The University of Sydney School of Mathematics and Statistics, , NSW 2006 , Australia"},{"name":"The University of Sydney Charles Perkins Centre, , NSW 2006 , Australia"},{"name":"University of Sydney Sydney Precision Data Science Centre, , NSW 2006 , Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pengyi","family":"Yang","sequence":"additional","affiliation":[{"name":"The University of Sydney School of Mathematics and Statistics, , NSW 2006 , Australia"},{"name":"The University of Sydney Charles Perkins Centre, , NSW 2006 , Australia"},{"name":"The University of Sydney Children\u2019s Medical Research Institute, Faculty of Medicine and Health, , NSW 2145 , Australia"},{"name":"University of Sydney Sydney Precision Data Science Centre, , NSW 2006 , Australia"},{"name":"Laboratory of Data Discovery for Health Limited (D24H), Science Park , Hong Kong SAR , China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jean","family":"Yang","sequence":"additional","affiliation":[{"name":"The University of Sydney School of Mathematics and Statistics, , NSW 2006 , Australia"},{"name":"The University of Sydney Charles Perkins Centre, , NSW 2006 , Australia"},{"name":"University of Sydney Sydney Precision Data Science Centre, , NSW 2006 , Australia"},{"name":"Laboratory of Data Discovery for Health Limited (D24H), Science Park , Hong Kong SAR , China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2023,4,24]]},"reference":[{"key":"2023052022330986500_ref1","doi-asserted-by":"crossref","DOI":"10.1093\/database\/baaa073","article-title":"A curated database reveals trends in single-cell transcriptomics","volume":"2020","author":"Svensson","year":"2020","journal-title":"Database"},{"key":"2023052022330986500_ref2","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1186\/s13059-021-02519-4","article-title":"Over 1000 tools reveal trends in the single-cell RNA-seq analysis landscape","volume":"22","author":"Zappia","year":"2021","journal-title":"Genome Biol"},{"key":"2023052022330986500_ref3","doi-asserted-by":"crossref","first-page":"2765","DOI":"10.1093\/bioinformatics\/btac171","article-title":"BSDE: barycenter single-cell differential expression for case\u2013control studies","volume":"38","author":"Zhang","year":"2022","journal-title":"Bioinformatics"},{"key":"2023052022330986500_ref4","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1186\/s13059-021-02544-3","article-title":"Feature selection revisited in the single-cell era","volume":"22","author":"Yang","year":"2021","journal-title":"Genome Biol"},{"key":"2023052022330986500_ref5","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1016\/j.omto.2021.04.001","article-title":"Single-cell RNA sequencing in cancer: applications, advances, and emerging challenges","volume":"21","author":"Sun","year":"2021","journal-title":"Mol Ther Oncolytics"},{"key":"2023052022330986500_ref6","doi-asserted-by":"crossref","first-page":"53763","DOI":"10.18632\/oncotarget.17893","article-title":"Advances in single-cell RNA sequencing and its applications in cancer research","volume":"8","author":"Zhu","year":"2017","journal-title":"Oncotarget"},{"key":"2023052022330986500_ref7","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1038\/s41576-020-00292-x","article-title":"Deciphering cell\u2013cell interactions and communication from gene expression","volume":"22","author":"Armingol","year":"2020","journal-title":"Nat Rev Genet"},{"key":"2023052022330986500_ref8","doi-asserted-by":"crossref","DOI":"10.1093\/bioinformatics\/btac590","article-title":"scFeatures: multi-view representations of single-cell and spatial data for disease outcome prediction","volume":"38","author":"Cao","year":"2022","journal-title":"Bioinformatics"},{"key":"2023052022330986500_ref9","doi-asserted-by":"crossref","first-page":"3243","DOI":"10.1093\/bioinformatics\/btab337","article-title":"Differential gene expression analysis for multi-subject single-cell RNA-sequencing studies with aggregateBioVar","volume":"37","author":"Thurman","year":"2021","journal-title":"Bioinformatics"},{"key":"2023052022330986500_ref10","article-title":"Maximizing statistical power to detect differentially abundant cell states with scPOST","volume":"1","author":"Millard","year":"2021","journal-title":"Cell Rep Methods"},{"key":"2023052022330986500_ref11","doi-asserted-by":"crossref","DOI":"10.1073\/pnas.2100293118","article-title":"Detection of differentially abundant cell subpopulations in scRNA-seq data","volume":"118","author":"Zhao","year":"2021","journal-title":"Proc Natl Acad Sci U S A"},{"key":"2023052022330986500_ref12","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"2023052022330986500_ref13","doi-asserted-by":"crossref","first-page":"30033","DOI":"10.1073\/pnas.1907373117","article-title":"The unreasonable effectiveness of deep learning in artificial intelligence","volume":"117","author":"Sejnowski","year":"2020","journal-title":"Proc Natl Acad Sci U S A"},{"key":"2023052022330986500_ref14","doi-asserted-by":"crossref","first-page":"296","DOI":"10.2174\/157489310794072508","article-title":"A review of ensemble methods in bioinformatics","volume":"5","author":"Yang","year":"2010","journal-title":"Curr Bioinform"},{"key":"2023052022330986500_ref15","doi-asserted-by":"crossref","first-page":"e0245230","DOI":"10.1371\/journal.pone.0245230","article-title":"Multi-view classification with convolutional neural networks","volume":"16","author":"Seeland","year":"2021","journal-title":"PLoS One"},{"key":"2023052022330986500_ref16","doi-asserted-by":"crossref","first-page":"500","DOI":"10.1038\/s42256-020-0217-y","article-title":"Ensemble deep learning in bioinformatics","volume":"2","author":"Cao","year":"2020","journal-title":"Nat Mach Intell"},{"key":"2023052022330986500_ref17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v028.i05","article-title":"Building predictive models in R using the caret package","volume":"28","author":"Kuhn","year":"2008","journal-title":"J Stat Softw"},{"key":"2023052022330986500_ref18","doi-asserted-by":"crossref","first-page":"896","DOI":"10.1038\/nbt.2931","article-title":"Normalization of RNA-seq data using factor analysis of control genes or samples","volume":"32","author":"Risso","year":"2014","journal-title":"Nat Biotechnol"},{"key":"2023052022330986500_ref19","doi-asserted-by":"crossref","first-page":"e47","DOI":"10.1093\/nar\/gkv007","article-title":"Limma powers differential expression analyses for RNA-sequencing and microarray studies","volume":"43","author":"Ritchie","year":"2015","journal-title":"Nucleic Acids Res"},{"key":"2023052022330986500_ref20","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1038\/s41592-021-01336-8","article-title":"Benchmarking atlas-level data integration in single-cell genomics","volume":"19","author":"Luecken","year":"2021","journal-title":"Nat Methods"},{"key":"2023052022330986500_ref21","doi-asserted-by":"crossref","first-page":"430","DOI":"10.1038\/s41586-020-2521-4","article-title":"Factors associated with COVID-19-related death using OpenSAFELY","volume":"584","author":"Williamson","year":"2020","journal-title":"Nature"},{"key":"2023052022330986500_ref22","first-page":"101","article-title":"In defense of one-vs-all classification","volume":"5","author":"Rifkin","year":"2004","journal-title":"J Mach Learn Res"},{"key":"2023052022330986500_ref23","doi-asserted-by":"crossref","first-page":"916","DOI":"10.1016\/j.cell.2022.01.012","article-title":"A blood atlas of COVID-19 defines hallmarks of disease severity and specificity","volume":"185","author":"Covid-19 Multi-omics Blood ATlas (COMBAT) Consortium","year":"2022","journal-title":"Cell"},{"key":"2023052022330986500_ref24","doi-asserted-by":"crossref","first-page":"5838","DOI":"10.1016\/j.cell.2021.10.023","article-title":"COVID-19 immune features revealed by a large-scale single-cell transcriptome atlas","volume":"184","author":"Ren","year":"2021","journal-title":"Cell"},{"key":"2023052022330986500_ref25","doi-asserted-by":"crossref","first-page":"1419","DOI":"10.1016\/j.cell.2020.08.001","article-title":"Severe COVID-19 is marked by a dysregulated myeloid cell compartment","volume":"182","author":"Schulte-Schrepping","year":"2020","journal-title":"Cell"},{"key":"2023052022330986500_ref26","doi-asserted-by":"crossref","first-page":"904","DOI":"10.1038\/s41591-021-01329-2","article-title":"Single-cell multi-omics analysis of the immune response in COVID-19","volume":"27","author":"Stephenson","year":"2021","journal-title":"Nat Med"},{"key":"2023052022330986500_ref27","doi-asserted-by":"crossref","DOI":"10.1084\/jem.20210582","article-title":"Multi-omic profiling reveals widespread dysregulation of innate immunity and hematopoiesis in COVID-19","volume":"218","author":"Wilk","year":"2021","journal-title":"J Exp Med"},{"key":"2023052022330986500_ref28","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1016\/j.cels.2015.12.004","article-title":"The molecular signatures database (MSigDB) hallmark gene set collection","volume":"1","author":"Liberzon","year":"2015","journal-title":"Cell Syst"},{"key":"2023052022330986500_ref29","doi-asserted-by":"crossref","first-page":"1083","DOI":"10.1038\/nmeth.4463","article-title":"SCENIC: single-cell regulatory network inference and clustering","volume":"14","author":"Aibar","year":"2017","journal-title":"Nat Methods"},{"key":"2023052022330986500_ref30","doi-asserted-by":"crossref","first-page":"1088","DOI":"10.1038\/s41467-021-21246-9","article-title":"Inference and analysis of cell-cell communication using CellChat","volume":"12","author":"Jin","year":"2021","journal-title":"Nat Commun"}],"container-title":["Briefings in Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/24\/3\/bbad159\/50410612\/bbad159.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/24\/3\/bbad159\/50410612\/bbad159.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,20]],"date-time":"2023-05-20T22:35:12Z","timestamp":1684622112000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bib\/article\/doi\/10.1093\/bib\/bbad159\/7140296"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,24]]},"references-count":30,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2023,5,19]]}},"URL":"https:\/\/doi.org\/10.1093\/bib\/bbad159","relation":{},"ISSN":["1467-5463","1477-4054"],"issn-type":[{"value":"1467-5463","type":"print"},{"value":"1477-4054","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,5]]},"published":{"date-parts":[[2023,4,24]]},"article-number":"bbad159"}}