{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,15]],"date-time":"2025-11-15T17:21:48Z","timestamp":1763227308891,"version":"3.41.2"},"reference-count":51,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2023,9,26]],"date-time":"2023-09-26T00:00:00Z","timestamp":1695686400000},"content-version":"vor","delay-in-days":4,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004462","name":"National Research Council of Italy","doi-asserted-by":"publisher","award":["DBA.AD005.225"],"award-info":[{"award-number":["DBA.AD005.225"]}],"id":[{"id":"10.13039\/501100004462","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,9,22]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Single-cell RNA-sequencing (scRNA-seq) allows for obtaining genomic and transcriptomic profiles of individual cells. That data make it possible to characterize tissues at the cell level. In this context, one of the main analyses exploiting scRNA-seq data is identifying the cell types within tissue to estimate the quantitative composition of cell populations. Due to the massive amount of available scRNA-seq data, automatic classification approaches for cell typing, based on the most recent deep learning technology, are needed. Here, we present the gene ontology-driven wide and deep learning (GOWDL) model for classifying cell types in several tissues. GOWDL implements a hybrid architecture that considers the functional annotations found in Gene Ontology and the marker genes typical of specific cell types. We performed cross-validation and independent external testing, comparing our algorithm with 12 other state-of-the-art predictors. Classification scores demonstrated that GOWDL reached the best results over five different tissues, except for recall, where we got about 92% versus 97% of the best tool. Finally, we presented a case study on classifying immune cell populations in breast cancer using a hierarchical approach based on GOWDL.<\/jats:p>","DOI":"10.1093\/bib\/bbad332","type":"journal-article","created":{"date-parts":[[2023,9,27]],"date-time":"2023-09-27T18:48:06Z","timestamp":1695840486000},"source":"Crossref","is-referenced-by-count":4,"title":["GOWDL: gene ontology-driven wide and deep learning model for cell typing of scRNA-seq data"],"prefix":"10.1093","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0114-4984","authenticated-orcid":false,"given":"Antonino","family":"Fiannaca","sequence":"first","affiliation":[{"name":"ICAR-CNR, National Research Council of Italy , Via Ugo La Malfa 153, 90146, Palermo , Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6718-7269","authenticated-orcid":false,"given":"Massimo","family":"La Rosa","sequence":"additional","affiliation":[{"name":"ICAR-CNR, National Research Council of Italy , Via Ugo La Malfa 153, 90146, Palermo , Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1077-5657","authenticated-orcid":false,"given":"Laura","family":"La Paglia","sequence":"additional","affiliation":[{"name":"ICAR-CNR, National Research Council of Italy , Via Ugo La Malfa 153, 90146, Palermo , Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5480-2100","authenticated-orcid":false,"given":"Salvatore","family":"Gaglio","sequence":"additional","affiliation":[{"name":"ICAR-CNR, National Research Council of Italy , Via Ugo La Malfa 153, 90146, Palermo , Italy"},{"name":"Dipartimento di Ingegneria, Universit\u00e0 degli studi di Palermo , Viale Delle Scienze, ed. 6, 90128, Palermo , Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3018-1458","authenticated-orcid":false,"given":"Alfonso","family":"Urso","sequence":"additional","affiliation":[{"name":"ICAR-CNR, National Research Council of Italy , Via Ugo La Malfa 153, 90146, Palermo , Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2023,9,26]]},"reference":[{"issue":"5","key":"2023092718475380400_ref1","doi-asserted-by":"crossref","first-page":"377","DOI":"10.1038\/nmeth.1315","article-title":"Mrna-seq whole-transcriptome analysis of a single cell","volume":"6","author":"Tang","year":"2009","journal-title":"Nat Methods"},{"key":"2023092718475380400_ref2","doi-asserted-by":"crossref","first-page":"1553","DOI":"10.3389\/fimmu.2018.01553","article-title":"Single cell rna sequencing of rare immune cell populations","volume":"9","author":"Nguyen","year":"2018","journal-title":"Front Immunol"},{"key":"2023092718475380400_ref3","doi-asserted-by":"crossref","first-page":"602539","DOI":"10.3389\/fimmu.2021.602539","article-title":"Single-cell transcriptomic analyses define distinct peripheral b cell subsets and discrete development pathways","volume":"12","author":"Stewart","year":"2021","journal-title":"Front Immunol"},{"issue":"1","key":"2023092718475380400_ref4","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41598-020-76972-9","article-title":"Characterisation of cd4+ t-cell subtypes using single cell rna sequencing and the impact of cell number and sequencing depth","volume":"10","author":"Ding","year":"2020","journal-title":"Sci Rep"},{"issue":"1","key":"2023092718475380400_ref5","doi-asserted-by":"crossref","first-page":"194","DOI":"10.1186\/s13059-019-1795-z","article-title":"A comparison of automatic cell identification methods for single-cell RNA sequencing data","volume":"20","author":"Abdelaal","year":"2019","journal-title":"Genome Biol"},{"key":"2023092718475380400_ref6","doi-asserted-by":"crossref","first-page":"296","DOI":"10.12688\/f1000research.18490.1","article-title":"Evaluation of methods to assign cell type labels to cell clusters from single-cell RNA-sequencing data","volume":"8","author":"Javier Diaz-Mejia","year":"2019","journal-title":"F1000Research"},{"issue":"5","key":"2023092718475380400_ref7","doi-asserted-by":"crossref","first-page":"1581","DOI":"10.1093\/bib\/bbz096","article-title":"Evaluation of single-cell classifiers for single-cell RNA sequencing data sets","volume":"21","author":"Zhao","year":"2020","journal-title":"Brief Bioinform"},{"issue":"2","key":"2023092718475380400_ref8","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1038\/s41590-018-0276-y","article-title":"Reference-based analysis of lung single-cell sequencing reveals a transitional profibrotic macrophage","volume":"20","author":"Aran","year":"2019","journal-title":"Nat Immunol"},{"issue":"7","key":"2023092718475380400_ref9","doi-asserted-by":"crossref","first-page":"531","DOI":"10.3390\/genes10070531","article-title":"SCINA: semi-supervised analysis of single cells in Silico","volume":"10","author":"Zhang","year":"2019","journal-title":"Genes"},{"issue":"1","key":"2023092718475380400_ref10","doi-asserted-by":"crossref","first-page":"1818","DOI":"10.1038\/s41467-020-15523-2","article-title":"SciBet as a portable and fast single cell type identifier","volume":"11","author":"Li","year":"2020","journal-title":"Nat Commun"},{"issue":"1","key":"2023092718475380400_ref11","doi-asserted-by":"crossref","first-page":"264","DOI":"10.1186\/s13059-019-1862-5","article-title":"scPred: accurate supervised method for cell-type classification from single-cell RNA-seq data","volume":"20","author":"Alquicira-Hernandez","year":"2019","journal-title":"Genome Biol"},{"issue":"5","key":"2023092718475380400_ref12","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1038\/nmeth.4644","article-title":"Scmap: projection of single-cell RNA-seq data across data sets","volume":"15","author":"Kiselev","year":"2018","journal-title":"Nat Methods"},{"issue":"16","key":"2023092718475380400_ref13","doi-asserted-by":"crossref","first-page":"e95","DOI":"10.1093\/nar\/gkz543","article-title":"CHETAH: a selective, hierarchical cell type identification method for single-cell RNA sequencing","volume":"47","author":"Kanter de","year":"2019","journal-title":"Nucleic Acids Res"},{"issue":"10","key":"2023092718475380400_ref14","doi-asserted-by":"crossref","first-page":"983","DOI":"10.1038\/s41592-019-0535-3","article-title":"Supervised classification enables rapid annotation of cell atlases","volume":"16","author":"Pliner","year":"2019","journal-title":"Nat Methods"},{"issue":"3","key":"2023092718475380400_ref15","doi-asserted-by":"crossref","first-page":"100914","DOI":"10.1016\/j.isci.2020.100914","article-title":"Scid uses discriminant analysis to identify transcriptionally equivalent cell types across single-cell rna-seq data with batch effect","volume":"23","author":"Boufea","year":"2020","journal-title":"iScience"},{"key":"2023092718475380400_ref16","doi-asserted-by":"crossref","first-page":"490","DOI":"10.3389\/fgene.2020.00490","article-title":"Scsa: a cell type annotation tool for single-cell rna-seq data","volume":"11","author":"Cao","year":"2020","journal-title":"Front Genet"},{"issue":"21","key":"2023092718475380400_ref17","doi-asserted-by":"crossref","first-page":"e122","DOI":"10.1093\/nar\/gkab775","article-title":"Scdeepsort: a pre-trained cell-type annotation method for single-cell transcriptomics using deep learning with a weighted graph neural network","volume":"49","author":"Shao","year":"2021","journal-title":"Nucleic Acids Res"},{"issue":"1","key":"2023092718475380400_ref18","doi-asserted-by":"crossref","first-page":"364","DOI":"10.1186\/s12859-021-04278-2","article-title":"Single-cell classification using graph convolutional networks","volume":"22","author":"Wang","year":"2021","journal-title":"BMC Bioinformatics"},{"issue":"3","key":"2023092718475380400_ref19","doi-asserted-by":"crossref","first-page":"100882","DOI":"10.1016\/j.isci.2020.100882","article-title":"Sccatch: automatic annotation on cell types of clusters from single-cell rna sequencing data","volume":"23","author":"Shao","year":"2020","journal-title":"Iscience"},{"issue":"6","key":"2023092718475380400_ref20","doi-asserted-by":"crossref","DOI":"10.15252\/msb.20199389","article-title":"scClassify: sample size estimation and multiscale classification of cells using single and multiple reference","volume":"16","author":"Lin","year":"2020","journal-title":"Mol Syst Biol"},{"issue":"1","key":"2023092718475380400_ref21","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1186\/s12859-022-04574-5","article-title":"scAnnotatR: framework to accurately classify cell types in single-cell RNA-sequencing data","volume":"23","author":"Nguyen","year":"2022","journal-title":"BMC Bioinformatics"},{"issue":"2","key":"2023092718475380400_ref22","doi-asserted-by":"crossref","first-page":"533","DOI":"10.1093\/bioinformatics\/btz592","article-title":"ACTINN: automated identification of cell types in single cell RNA sequencing","volume":"36","author":"Ma","year":"2020","journal-title":"Bioinformatics"},{"issue":"D1","key":"2023092718475380400_ref23","doi-asserted-by":"crossref","first-page":"D325","DOI":"10.1093\/nar\/gkaa1113","article-title":"The gene ontology resource: enriching a GOld mine","volume":"49","author":"Carbon","year":"2021","journal-title":"Nucleic Acids Res"},{"key":"2023092718475380400_ref24","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1145\/2988450.2988454","article-title":"Wide & Deep Learning for Recommender Systems","volume-title":"Proceedings of the 1st Workshop on Deep Learning for Recommender Systems","author":"Cheng","year":"2016"},{"key":"2023092718475380400_ref25","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/ICEngTechnol.2017.8308186","article-title":"Understanding of a convolutional neural network","volume-title":"2017 International Conference on Engineering and Technology (ICET)","author":"Albawi","year":"2017"},{"key":"2023092718475380400_ref26","doi-asserted-by":"crossref","first-page":"1052","DOI":"10.1016\/j.csbj.2021.01.027","article-title":"Wide and deep learning for automatic cell type identification","volume":"19","author":"Wilson","year":"2021","journal-title":"Comput Struct Biotechnol J"},{"key":"2023092718475380400_ref27","doi-asserted-by":"crossref","first-page":"323","DOI":"10.1007\/978-3-031-08223-8_27","article-title":"A Gene Ontology-Driven Wide and Deep Learning Architecture for Cell-Type Classification from Single-Cell RNA-seq Data","volume-title":"Engineering Applications of Neural Networks","author":"Coppola","year":"2022"},{"issue":"13","key":"2023092718475380400_ref28","doi-asserted-by":"crossref","first-page":"3573","DOI":"10.1016\/j.cell.2021.04.048","article-title":"Integrated analysis of multimodal single-cell data","volume":"184","author":"Hao","year":"2021","journal-title":"Cell"},{"volume-title":"Single Cell Portal. Study: ICA: Blood Mononuclear Cells","year":"2022","key":"2023092718475380400_ref29"},{"issue":"19","key":"2023092718475380400_ref30","doi-asserted-by":"crossref","first-page":"e104063","DOI":"10.15252\/embj.2019104063","article-title":"Stromal cell diversity associated with immune evasion in human triple-negative breast cancer","volume":"39","author":"Wu","year":"2020","journal-title":"EMBO J"},{"issue":"6460","key":"2023092718475380400_ref31","doi-asserted-by":"crossref","first-page":"1461","DOI":"10.1126\/science.aat5031","article-title":"Spatiotemporal immune zonation of the human kidney","volume":"365","author":"Stewart","year":"2019","journal-title":"Science"},{"issue":"4","key":"2023092718475380400_ref32","doi-asserted-by":"crossref","first-page":"346","DOI":"10.1016\/j.cels.2016.08.011","article-title":"A single-cell transcriptomic map of the human and mouse pancreas reveals inter- and intra-cell population structure","volume":"3","author":"Baron","year":"2016","journal-title":"Cell Syst"},{"volume-title":"3k PBMCs from a Healthy Donor, Single Cell Gene Expression Dataset by Cell Ranger 1.1.0","year":"2016","author":"10x Genomics","key":"2023092718475380400_ref33"},{"issue":"4","key":"2023092718475380400_ref34","doi-asserted-by":"crossref","first-page":"779","DOI":"10.1016\/j.kint.2021.11.031","article-title":"Glomerular endothelial cell-podocyte stresses and crosstalk in structurally normal kidney transplants","volume":"101","author":"Menon","year":"2022","journal-title":"Kidney Int"},{"issue":"1","key":"2023092718475380400_ref35","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1038\/s41588-022-01243-4","article-title":"A spatially resolved atlas of the human lung characterizes a gland-associated immune niche","volume":"55","author":"Madissoon","year":"2023","journal-title":"Nat Genet"},{"issue":"4","key":"2023092718475380400_ref36","doi-asserted-by":"crossref","first-page":"984","DOI":"10.1016\/j.cell.2018.09.006","article-title":"A cancer cell program promotes T cell exclusion and resistance to checkpoint blockade","volume":"175","author":"Jerby-Arnon","year":"2018","journal-title":"Cell"},{"issue":"2","key":"2023092718475380400_ref37","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1016\/j.cell.2017.09.004","article-title":"Single-cell analysis of human pancreas reveals transcriptional signatures of aging and somatic mutation patterns","volume":"171","author":"Martin Enge","year":"2017","journal-title":"Cell"},{"key":"2023092718475380400_ref38","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1007\/978-3-030-34585-3_22","article-title":"Designing and Evaluating Deep Learning Models for Cancer Detection on Gene Expression Data","volume-title":"Computational Intelligence Methods for Bioinformatics and Biostatistics","author":"Canakoglu","year":"2020"},{"issue":"4","key":"2023092718475380400_ref39","doi-asserted-by":"crossref","first-page":"611","DOI":"10.1007\/s13244-018-0639-9","article-title":"Convolutional neural networks: an overview and application in radiology","volume":"9","author":"Yamashita","year":"2018","journal-title":"Insights Imaging"},{"key":"2023092718475380400_ref40","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1016\/B978-0-32-385787-1.00008-7","article-title":"Convolutional neural networks","volume-title":"Deep Learning for Robot Perception and Cognition","author":"Raitoharju","year":"2022"},{"issue":"1","key":"2023092718475380400_ref41","doi-asserted-by":"crossref","first-page":"15107","DOI":"10.1038\/s41598-018-33219-y","article-title":"GOGO: an improved algorithm to measure the semantic similarity between gene ontology terms","volume":"8","author":"Zhao","year":"2018","journal-title":"Sci Rep"},{"volume-title":"Keras","year":"2015","author":"Chollet","key":"2023092718475380400_ref42"},{"volume-title":"TensorFlow: Large-scale machine learning on heterogeneous systems","year":"2015","author":"Abadi","key":"2023092718475380400_ref43"},{"issue":"1","key":"2023092718475380400_ref44","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1186\/s13040-017-0155-3","article-title":"Ten quick tips for machine learning in computational biology","volume":"10","author":"Chicco","year":"2017","journal-title":"BioData Mining"},{"key":"2023092718475380400_ref45","doi-asserted-by":"crossref","first-page":"785","DOI":"10.1145\/2939672.2939785","article-title":"XGBoost: A Scalable Tree Boosting System","volume-title":"Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","author":"Chen","year":"2016"},{"key":"2023092718475380400_ref46","first-page":"6639","article-title":"Catboost: Unbiased boosting with categorical features","volume-title":"Proceedings of the 32nd International Conference on Neural Information Processing Systems","author":"Prokhorenkova","year":"2018"},{"key":"2023092718475380400_ref47","doi-asserted-by":"crossref","first-page":"539","DOI":"10.1007\/s10549-014-2896-8","article-title":"Immune cell quantitation in normal breast tissue lobules with and without lobulitis","volume":"144","author":"Degnim","year":"2014","journal-title":"Breast Cancer Res Treat"},{"issue":"1","key":"2023092718475380400_ref48","doi-asserted-by":"crossref","first-page":"e63","DOI":"10.1016\/j.clbc.2020.06.011","article-title":"The role of immune cells in breast tissue and immunotherapy for the treatment of breast cancer","volume":"21","author":"Goff","year":"2021","journal-title":"Clin Breast Cancer"},{"issue":"6797","key":"2023092718475380400_ref49","doi-asserted-by":"crossref","first-page":"747","DOI":"10.1038\/35021093","article-title":"Molecular portraits of human breast tumours","volume":"406","author":"Perou","year":"2000","journal-title":"Nature"},{"issue":"2","key":"2023092718475380400_ref50","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1159\/000507055","article-title":"The multifaceted nature of tumor microenvironment in breast carcinomas","volume":"87","author":"Annaratone","year":"2020","journal-title":"Pathobiology"},{"issue":"1","key":"2023092718475380400_ref51","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1111\/coin.12011","article-title":"An extensive evaluation of decision tree\u2013based hierarchical multilabel classification methods and performance measures","volume":"31","author":"Cerri","year":"2015","journal-title":"Comput Intell"}],"container-title":["Briefings in Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/24\/6\/bbad332\/51766347\/bbad332.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/24\/6\/bbad332\/51766347\/bbad332.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,27]],"date-time":"2023-09-27T18:48:37Z","timestamp":1695840517000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bib\/article\/doi\/10.1093\/bib\/bbad332\/7281740"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,22]]},"references-count":51,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2023,9,22]]}},"URL":"https:\/\/doi.org\/10.1093\/bib\/bbad332","relation":{},"ISSN":["1467-5463","1477-4054"],"issn-type":[{"type":"print","value":"1467-5463"},{"type":"electronic","value":"1477-4054"}],"subject":[],"published-other":{"date-parts":[[2023,11,1]]},"published":{"date-parts":[[2023,9,22]]},"article-number":"bbad332"}}