{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,2]],"date-time":"2026-11-02T21:56:47Z","timestamp":1793656607479,"version":"4.5.1"},"reference-count":149,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,7,26]],"date-time":"2023-07-26T00:00:00Z","timestamp":1690329600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Bioinform."],"abstract":"<jats:p>Since its introduction into the field of oncology, deep learning (DL) has impacted clinical discoveries and biomarker predictions. DL-driven discoveries and predictions in oncology are based on a variety of biological data such as genomics, proteomics, and imaging data. DL-based computational frameworks can predict genetic variant effects on gene expression, as well as protein structures based on amino acid sequences. Furthermore, DL algorithms can capture valuable mechanistic biological information from several spatial \u201comics\u201d technologies, such as spatial transcriptomics and spatial proteomics. Here, we review the impact that the combination of artificial intelligence (AI) with spatial omics technologies has had on oncology, focusing on DL and its applications in biomedical image analysis, encompassing cell segmentation, cell phenotype identification, cancer prognostication, and therapy prediction. We highlight the advantages of using highly multiplexed images (spatial proteomics data) compared to single-stained, conventional histopathological (\u201csimple\u201d) images, as the former can provide deep mechanistic insights that cannot be obtained by the latter, even with the aid of explainable AI. Furthermore, we provide the reader with the advantages\/disadvantages of DL-based pipelines used in preprocessing highly multiplexed images (cell segmentation, cell type annotation). Therefore, this review also guides the reader to choose the DL-based pipeline that best fits their data. In conclusion, DL continues to be established as an essential tool in discovering novel biological mechanisms when combined with technologies such as highly multiplexed tissue imaging data. In balance with conventional medical data, its role in clinical routine will become more important, supporting diagnosis and prognosis in oncology, enhancing clinical decision-making, and improving the quality of care for patients.<\/jats:p>","DOI":"10.3389\/fbinf.2023.1159381","type":"journal-article","created":{"date-parts":[[2023,7,26]],"date-time":"2023-07-26T07:23:30Z","timestamp":1690356210000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":32,"title":["A review on deep learning applications in highly multiplexed tissue imaging data analysis"],"prefix":"10.3389","volume":"3","author":[{"given":"Mohammed","family":"Zidane","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ahmad","family":"Makky","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Matthias","family":"Bruhns","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alexander","family":"Rochwarger","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sepideh","family":"Babaei","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Manfred","family":"Claassen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christian M.","family":"Sch\u00fcrch","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2023,7,26]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"769","DOI":"10.1002\/cyto.a.23738","article-title":"Predicting cell populations in single cell mass cytometry data","volume":"95","author":"Abdelaal","year":"2019","journal-title":"Cytom. Part A"},{"key":"B2","unstructured":"American cancer society2023"},{"key":"B3","doi-asserted-by":"publisher","first-page":"8","DOI":"10.14569\/IJACSA.2017.080853","article-title":"Lung cancer detection and classification with 3D convolutional neural network (3D-CNN)","volume":"8","author":"Alakwaa","year":"2017","journal-title":"Int. J. Adv. Comput. Sci. Appl. (ijacsa)"},{"key":"B4","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1038\/s41698-022-00305-4","article-title":"Spatially variant immune infiltration scoring in human cancer tissues","volume":"6","author":"Allam","year":"2022","journal-title":"npj Precis. Onc"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.1101\/2022.02.28.482296","article-title":"MAPLE: a hybrid framework for multi-sample spatial transcriptomics data","author":"Allen","year":"2022"},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.1101\/2022.11.07.515441","article-title":"CellSighter\u2014a neural network to classify cells in highly multiplexed images","author":"Amitay","year":"2022"},{"key":"B7","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nm.3488","article-title":"Multiplexed ion beam imaging (MIBI) of human breast tumors","volume":"20","author":"Angelo","year":"2014","journal-title":"Nat. Med."},{"key":"B8","first-page":"533167","article-title":"S3-CIMA: Supervised spatial single-cell image analysis for the identification of disease-associated cell type compositions in tissue","author":"Babaei","year":"2023"},{"key":"B9","doi-asserted-by":"publisher","first-page":"1317","DOI":"10.1001\/jama.2017.18391","article-title":"Big data and machine learning in health care","volume":"319","author":"Beam","year":"2018","journal-title":"JAMA"},{"key":"B10","doi-asserted-by":"publisher","first-page":"1226","DOI":"10.1038\/s41592-019-0582-9","article-title":"ilastik: interactive machine learning for (bio)image analysis","volume":"16","author":"Berg","year":"2019","journal-title":"Nat. Methods"},{"key":"B11","doi-asserted-by":"publisher","first-page":"1352","DOI":"10.1038\/s41592-021-01264-7","article-title":"Deep learning and alignment of spatially resolved single-cell transcriptomes with Tangram","volume":"18","author":"Biancalani","year":"2021","journal-title":"Nat. Methods"},{"key":"B12","doi-asserted-by":"publisher","first-page":"3802","DOI":"10.1038\/s41596-021-00556-8","article-title":"CODEX multiplexed tissue imaging with DNA-conjugated antibodies","volume":"16","author":"Black","year":"2021","journal-title":"Nat. Protoc."},{"key":"B13","doi-asserted-by":"publisher","first-page":"225","DOI":"10.1016\/j.cels.2016.03.008","article-title":"Multiplexed epitope-based tissue imaging for discovery and healthcare applications","volume":"2","author":"Bodenmiller","year":"2016","journal-title":"Cell Syst."},{"key":"B14","doi-asserted-by":"publisher","first-page":"1411","DOI":"10.1038\/s41592-022-01651-8","article-title":"Annotation of spatially resolved single-cell data with STELLAR","volume":"19","author":"Brbi\u0107","year":"2022","journal-title":"Nat. Methods"},{"key":"B15","doi-asserted-by":"publisher","first-page":"4037","DOI":"10.1038\/s41598-021-83102-6","article-title":"Deep learning identifies morphological features in breast cancer predictive of cancer ERBB2 status and trastuzumab treatment efficacy","volume":"11","author":"Bychkov","year":"2021","journal-title":"Sci. Rep."},{"key":"B16","first-page":"406","article-title":"CAN-IT: domain segmentation of spatial transcriptomics images by graph neural network","volume":"32","author":"Cang","year":"2023","journal-title":"BMVC"},{"key":"B17","doi-asserted-by":"publisher","first-page":"612","DOI":"10.1016\/j.cels.2018.02.010","article-title":"Compensation of signal spillover in suspension and imaging mass cytometry","volume":"6","author":"Chevrier","year":"2018","journal-title":"Cell Syst."},{"key":"B18","doi-asserted-by":"publisher","first-page":"1901","DOI":"10.1038\/s41379-021-00838-2","article-title":"Identification of nodal micrometastasis in colorectal cancer using deep learning on annotation-free whole-slide images","volume":"34","author":"Chuang","year":"2021","journal-title":"Mod. Pathol."},{"key":"B19","first-page":"411","article-title":"Mitosis detection in breast cancer histology images with deep neural networks","volume-title":"Medical image computing and computer-assisted intervention \u2013 MICCAI 2013. Lecture notes in computer science","author":"Cire\u015fan","year":"2013"},{"key":"B20","doi-asserted-by":"publisher","first-page":"e2030072","DOI":"10.1001\/jamanetworkopen.2020.30072","article-title":"Assessment of time-to-treatment initiation and survival in a cohort of patients with common cancers","volume":"3","author":"Cone","year":"2020","journal-title":"JAMA Netw. Open"},{"key":"B21","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1038\/s41523-018-0079-1","article-title":"Image analysis with deep learning to predict breast cancer grade, ER status, histologic subtype, and intrinsic subtype","volume":"4","author":"Couture","year":"2018","journal-title":"npj Breast Cancer"},{"key":"B22","doi-asserted-by":"publisher","first-page":"4814","DOI":"10.1038\/s41467-022-32430-w","article-title":"Single cell spatial analysis reveals the topology of immunomodulatory purinergic signaling in glioblastoma","volume":"13","author":"Coy","year":"2022","journal-title":"Nat. Commun."},{"key":"B23","doi-asserted-by":"publisher","first-page":"1384","DOI":"10.1038\/s41467-020-15114-1","article-title":"Predicting gene expression using morphological cell responses to nanotopography","volume":"11","author":"Cutiongco","year":"2020","journal-title":"Nat. Commun."},{"key":"B24","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1038\/s41598-021-03984-4","article-title":"Quantifying the cell morphology and predicting biological behavior of signet ring cell carcinoma using deep learning","volume":"12","author":"Da","year":"2022","journal-title":"Sci. Rep."},{"key":"B25","doi-asserted-by":"publisher","first-page":"1342","DOI":"10.1038\/s41591-018-0107-6","article-title":"Clinically applicable deep learning for diagnosis and referral in retinal disease","volume":"24","author":"De Fauw","year":"2018","journal-title":"Nat. Med."},{"key":"B26","first-page":"248","article-title":"ImageNet: A large-scale hierarchical image database","author":"Deng","year":"2009"},{"key":"B27","first-page":"78","article-title":"Giotto: a toolbox for integrative analysis and visualization of spatial expression data","volume-title":"Genome. Biol.","author":"Dries","year":"2021"},{"key":"B28","doi-asserted-by":"publisher","first-page":"686","DOI":"10.1038\/s41416-020-01122-x","article-title":"Deep learning in cancer pathology: A new generation of clinical biomarkers","volume":"124","author":"Echle","year":"2021","journal-title":"Br. J. Cancer"},{"key":"B29","doi-asserted-by":"publisher","first-page":"2199","DOI":"10.1001\/jama.2017.14585","article-title":"Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer","volume":"318","author":"Ehteshami Bejnordi","year":"2017","journal-title":"JAMA"},{"key":"B30","doi-asserted-by":"publisher","first-page":"801","DOI":"10.1007\/s00428-023-03509-6","article-title":"High-multiplex tissue imaging in routine pathology\u2014Are we there yet?","volume":"482","author":"Einhaus","year":"2023","journal-title":"Virchows Arch."},{"key":"B31","doi-asserted-by":"publisher","first-page":"348","DOI":"10.1038\/s41586-021-03922-4","article-title":"Biologically informed deep neural network for prostate cancer discovery","volume":"598","author":"Elmarakeby","year":"2021","journal-title":"Nature"},{"key":"B32","doi-asserted-by":"crossref","DOI":"10.1101\/2021.09.03.458852","article-title":"Bringing TrackMate into the era of machine-learning and deep-learning","author":"Ershov","year":"2021"},{"key":"B33","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1038\/nature21056","article-title":"Dermatologist\u2013level classification of skin cancer with deep neural networks","volume":"542","author":"Esteva","year":"2017","journal-title":"Nature"},{"key":"B34","doi-asserted-by":"publisher","first-page":"585","DOI":"10.1126\/science.280.5363.585","article-title":"Visualization of single RNA transcripts in situ","volume":"280","author":"Femino","year":"1998","journal-title":"Science"},{"key":"B35","doi-asserted-by":"publisher","first-page":"11982","DOI":"10.1073\/pnas.1300136110","article-title":"Highly multiplexed single-cell analysis of formalin-fixed, paraffin-embedded cancer tissue","volume":"110","author":"Gerdes","year":"2013","journal-title":"Proc. Natl. Acad. Sci."},{"key":"B36","doi-asserted-by":"publisher","first-page":"1173","DOI":"10.1016\/j.cels.2021.08.012","article-title":"Automated assignment of cell identity from single-cell multiplexed imaging and proteomic data","volume":"12","author":"Geuenich","year":"2021","journal-title":"Cell Syst."},{"key":"B37","doi-asserted-by":"publisher","first-page":"417","DOI":"10.1038\/nmeth.2869","article-title":"Highly multiplexed imaging of tumor tissues with subcellular resolution by mass cytometry","volume":"11","author":"Giesen","year":"2014","journal-title":"Nat. Methods"},{"key":"B38","doi-asserted-by":"publisher","first-page":"1797","DOI":"10.1200\/JCO.2004.09.059","article-title":"Pooled analysis of fluorouracil-based adjuvant therapy for stage II and III colon cancer: Who benefits and by how much?","volume":"22","author":"Gill","year":"2004","journal-title":"JCO"},{"key":"B39","doi-asserted-by":"publisher","first-page":"968","DOI":"10.1016\/j.cell.2018.07.010","article-title":"Deep profiling of mouse splenic architecture with CODEX multiplexed imaging","volume":"174","author":"Goltsev","year":"2017","journal-title":"Cell"},{"key":"B40","volume-title":"Deep learning book","author":"Goodfellow","year":"2016"},{"key":"B41","doi-asserted-by":"publisher","first-page":"555","DOI":"10.1038\/s41587-021-01094-0","article-title":"Whole-cell segmentation of tissue images with human-level performance using large-scale data annotation and deep learning","volume":"40","author":"Greenwald","year":"2022","journal-title":"Nat. Biotechnol."},{"key":"B42","doi-asserted-by":"publisher","first-page":"845","DOI":"10.1038\/nmeth.3971","article-title":"Diffusion pseudotime robustly reconstructs lineage branching","volume":"13","author":"Haghverdi","year":"2016","journal-title":"Nat. Methods"},{"key":"B43","doi-asserted-by":"publisher","first-page":"9804","DOI":"10.1038\/s41598-021-89352-8","article-title":"A combined microfluidic deep learning approach for lung cancer cell high throughput screening toward automatic cancer screening applications","volume":"11","author":"Hashemzadeh","year":"2021","journal-title":"Sci. Rep."},{"key":"B44","unstructured":"Mask R-CNN\n            HeK.\n            GkioxariG.\n            Doll\u00e1rP.\n            GirshickR.\n          2018"},{"key":"B45","volume-title":"Deep residual learning for image recognition","author":"He","year":"2015"},{"key":"B46","doi-asserted-by":"publisher","first-page":"91","DOI":"10.1016\/j.ejca.2019.06.012","article-title":"Deep learning outperformed 11 pathologists in the classification of histopathological melanoma images","volume":"118","author":"Hekler","year":"2019","journal-title":"Eur. J. Cancer"},{"key":"B47","first-page":"011306","article-title":"Deep learning in spatial transcriptomics: learning from the next next-generation sequencing","volume-title":"Biophys. Rev.","author":"Heydari","year":"2023"},{"key":"B48","doi-asserted-by":"publisher","first-page":"453","DOI":"10.1016\/j.cels.2020.04.003","article-title":"nucleAIzer: A parameter-free deep learning framework for nucleus segmentation using image style transfer","volume":"10","author":"Hollandi","year":"2020","journal-title":"Cell Syst."},{"key":"B49","first-page":"1","article-title":"Brain tumor detection using convolutional neural network","author":"Hossain","year":"2019"},{"key":"B50","doi-asserted-by":"publisher","first-page":"1342","DOI":"10.1038\/s41592-021-01255-8","article-title":"SpaGCN: Integrating gene expression, spatial location and histology to identify spatial domains and spatially variable genes by graph convolutional network","volume":"18","author":"Hu","year":"2021","journal-title":"Nat. Methods"},{"key":"B51","unstructured":"Squeeze-and-Excitation networks\n            HuJ.\n            ShenL.\n            AlbanieS.\n            SunG.\n            WuE.\n          2019"},{"key":"B52","doi-asserted-by":"publisher","first-page":"2015","DOI":"10.2147\/OTT.S80733","article-title":"Computer-aided classification of lung nodules on computed tomography images via deep learning technique","volume":"8","author":"Hua","year":"2015","journal-title":"Onco Targets Ther."},{"key":"B53","volume-title":"SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size","author":"Iandola","year":"2016"},{"key":"B54","doi-asserted-by":"publisher","first-page":"100104","DOI":"10.1016\/j.crmeth.2021.100104","article-title":"Multiplexed imaging and automated signal quantification in formalin-fixed paraffin-embedded tissues by ChipCytometry","volume":"1","author":"Jarosch","year":"2021","journal-title":"Cell Rep. Methods"},{"key":"B55","doi-asserted-by":"publisher","first-page":"102384","DOI":"10.1016\/j.media.2022.102384","article-title":"NaroNet: Discovery of tumor microenvironment elements from highly multiplexed images","volume":"78","author":"Jim\u00e9nez-S\u00e1nchez","year":"2022","journal-title":"Med. Image Anal."},{"key":"B56","doi-asserted-by":"publisher","first-page":"1851","DOI":"10.1038\/s41467-021-22188-y","article-title":"Predicting treatment response from longitudinal images using multi-task deep learning","volume":"12","author":"Jin","year":"2021","journal-title":"Nat. Commun."},{"key":"B57","doi-asserted-by":"publisher","first-page":"1262","DOI":"10.1002\/eji.202048891","article-title":"Highly multiplexed tissue imaging using repeated oligonucleotide exchange reaction","volume":"51","author":"Kennedy-Darling","year":"2020","journal-title":"Eur. J. Immunol."},{"key":"B58","doi-asserted-by":"publisher","first-page":"1373","DOI":"10.1016\/j.cell.2018.08.039","article-title":"A structured tumor-immune microenvironment in triple negative breast cancer revealed by multiplexed ion beam imaging","volume":"174","author":"Keren","year":"2018","journal-title":"Cell"},{"key":"B59","doi-asserted-by":"publisher","first-page":"1653","DOI":"10.1038\/s41592-022-01657-2","article-title":"Unsupervised discovery of tissue architecture in multiplexed imaging","volume":"19","author":"Kim","year":"2022","journal-title":"Nat. Methods"},{"key":"B60","doi-asserted-by":"publisher","first-page":"34","DOI":"10.1016\/j.csbj.2018.01.001","article-title":"Machine learning methods for histopathological image analysis","volume":"16","author":"Komura","year":"2018","journal-title":"Comput. Struct. Biotechnol. J."},{"key":"B61","doi-asserted-by":"publisher","first-page":"e43803","DOI":"10.7554\/eLife.43803","article-title":"Identifying gene expression programs of cell-type identity and cellular activity with single-cell RNA-Seq","volume":"8","author":"Kotliar","year":"2019","journal-title":"eLife"},{"key":"B62","doi-asserted-by":"publisher","first-page":"e145488","DOI":"10.1172\/JCI145488","article-title":"Computerized tumor multinucleation index (MuNI) is prognostic in p16+ oropharyngeal carcinoma","volume":"131","author":"Koyuncu","year":"2021","journal-title":"J. Clin. Investigation"},{"key":"B63","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1145\/3065386","article-title":"ImageNet classification with deep convolutional neural networks","volume":"60","author":"Krizhevsky","year":"2017","journal-title":"Commun. ACM"},{"key":"B64","doi-asserted-by":"publisher","first-page":"1689","DOI":"10.1093\/bioinformatics\/btx054","article-title":"Automated cell type discovery and classification through knowledge transfer","volume":"33","author":"Lee","year":"2017","journal-title":"Bioinformatics"},{"key":"B65","doi-asserted-by":"publisher","first-page":"46","DOI":"10.1186\/s12859-022-04570-9","article-title":"CellSeg: A robust, pre-trained nucleus segmentation and pixel quantification software for highly multiplexed fluorescence images","volume":"23","author":"Lee","year":"2022","journal-title":"BMC Bioinform."},{"key":"B66","doi-asserted-by":"publisher","first-page":"184","DOI":"10.1016\/j.cell.2015.05.047","article-title":"Data-driven phenotypic dissection of AML reveals progenitor-like cells that correlate with prognosis","volume":"162","author":"Levine","year":"2015","journal-title":"Cell"},{"key":"B67","doi-asserted-by":"publisher","first-page":"e31657","DOI":"10.7554\/eLife.31657","article-title":"Highly multiplexed immunofluorescence imaging of human tissues and tumors using t-CyCIF and conventional optical microscopes","volume":"7","author":"Lin","year":"2018","journal-title":"eLife"},{"key":"B68","doi-asserted-by":"crossref","DOI":"10.1101\/2021.03.31.437984","volume-title":"Multiplexed 3D atlas of state transitions and immune interactions in colorectal cancer","author":"Lin","year":"2021"},{"key":"B69","doi-asserted-by":"publisher","DOI":"10.1101\/2022.08.16.504171","article-title":"Robust phenotyping of highly multiplexed tissue imaging data using pixel-level clustering","author":"Liu","year":"2022"},{"key":"B70","doi-asserted-by":"publisher","first-page":"1360","DOI":"10.1038\/s41587-022-01272-8","article-title":"DestVI identifies continuums of cell types in spatial transcriptomics data","volume":"40","author":"Lopez","year":"2022","journal-title":"Nat. Biotechnol."},{"key":"B71","doi-asserted-by":"publisher","first-page":"6654","DOI":"10.1038\/s41467-021-26990-6","article-title":"Deep learning for the prediction of early on-treatment response in metastatic colorectal cancer from serial medical imaging","volume":"12","author":"Lu","year":"2021","journal-title":"Nat. Commun."},{"key":"B72","doi-asserted-by":"publisher","first-page":"e8746","DOI":"10.15252\/msb.20188746","article-title":"Current best practices in single\u2010cell RNA\u2010seq analysis: A tutorial","volume":"15","author":"Luecken","year":"2019","journal-title":"Mol. Syst. Biol."},{"key":"B73","doi-asserted-by":"publisher","first-page":"767836","DOI":"10.3389\/fmedt.2021.767836","article-title":"Deep learning for automatic image segmentation in stomatology and its clinical application","volume":"3","author":"Luo","year":"2021","journal-title":"Front. Med. Technol."},{"key":"B74","doi-asserted-by":"publisher","first-page":"2040","DOI":"10.1002\/jcc.23718","article-title":"Predicting backbone C\u03b1 angles and dihedrals from protein sequences by stacked sparse auto-encoder deep neural network","volume":"35","author":"Lyons","year":"2014","journal-title":"J. Comput. Chem."},{"key":"B75","doi-asserted-by":"publisher","first-page":"1550","DOI":"10.1038\/s41467-021-21735-x","article-title":"Whole-brain tissue mapping toolkit using large-scale highly multiplexed immunofluorescence imaging and deep neural networks","volume":"12","author":"Maric","year":"2021","journal-title":"Nat. Commun."},{"key":"B76","first-page":"3063","article-title":"A graph based neural network approach to immune profiling of multiplexed tissue samples","author":"Martin","year":"2022"},{"key":"B77","doi-asserted-by":"publisher","first-page":"636743","DOI":"10.3389\/fgene.2021.636743","article-title":"DEEPsc: A deep learning-based map connecting single-cell transcriptomics and spatial imaging data","volume":"12","author":"Maseda","year":"2021","journal-title":"Front. Genet."},{"key":"B78","doi-asserted-by":"publisher","first-page":"851","DOI":"10.1093\/bib\/bbw068","article-title":"Deep learning in bioinformatics","volume":"18","author":"Min","year":"2017","journal-title":"Briefings in Bioinformatic"},{"key":"B79","unstructured":"Image segmentation using deep learning: A survey\n            MinaeeS.\n            BoykovY.\n            PorikliF.\n            PlazaA.\n            KehtarnavazN.\n            TerzopoulosD.\n          2020"},{"key":"B80","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1016\/j.eururo.2016.02.029","article-title":"The 2016 WHO classification of tumours of the urinary system and male genital organs\u2014Part A: Renal, penile, and testicular tumours","volume":"70","author":"Moch","year":"2016","journal-title":"Eur. Urol."},{"key":"B81","doi-asserted-by":"publisher","DOI":"10.1101\/2022.11.03.514908","article-title":"A cellular and spatial map of salivary glands at single cell resolution reveals the functional basis of tertiary lymphoid structure formation in Sjogren\u2019s syndrome","author":"Nayar","year":"2022"},{"key":"B82","doi-asserted-by":"publisher","first-page":"1518","DOI":"10.1158\/2159-8290.CD-21-1357","article-title":"The spatial landscape of progression and immunoediting in primary melanoma at single-cell resolution","volume":"12","author":"Nirmal","year":"2022","journal-title":"Cancer Discov."},{"key":"B83","doi-asserted-by":"publisher","first-page":"852","DOI":"10.1038\/s42003-021-02361-1","article-title":"Multiplexed imaging analysis of the tumor-immune microenvironment reveals predictors of outcome in triple-negative breast cancer","volume":"4","author":"Patwa","year":"2021","journal-title":"Commun. Biol."},{"key":"B84","doi-asserted-by":"publisher","first-page":"1240","DOI":"10.1109\/TMI.2016.2538465","article-title":"Brain tumor segmentation using convolutional neural networks in MRI images","volume":"35","author":"Pereira","year":"2016","journal-title":"IEEE Trans. Med. Imaging"},{"key":"B85","first-page":"2020","volume-title":"stLearn: integrating spatial location, tissue morphology and gene expression to find cell types, cell-cell interactions and spatial trajectories within undissociated tissues","author":"Pham","year":"2020"},{"key":"B86","doi-asserted-by":"publisher","first-page":"6726","DOI":"10.1038\/s41467-021-26974-6","article-title":"Immune cell topography predicts response to PD-1 blockade in cutaneous T cell lymphoma","volume":"12","author":"Phillips","year":"2021","journal-title":"Nat. Commun."},{"key":"B87","doi-asserted-by":"publisher","first-page":"522","DOI":"10.1038\/s41551-021-00711-2","article-title":"Prospective assessment of breast cancer risk from multimodal multiview ultrasound images via clinically applicable deep learning","volume":"5","author":"Qian","year":"2021","journal-title":"Nat. Biomed. Eng."},{"key":"B88","doi-asserted-by":"publisher","first-page":"e6201","DOI":"10.7717\/peerj.6201","article-title":"Breast cancer detection using deep convolutional neural networks and support vector machines","volume":"7","author":"Ragab","year":"2019","journal-title":"PeerJ"},{"key":"B107","first-page":"1127","article-title":"Dynamic CD8+ T cell responses to cancer immunotherapy in human regional lymph nodes are disrupted in metastatic lymph nodes","volume-title":"Cell","author":"Rahim","year":"2023"},{"key":"B89","unstructured":"Towards real-time object detection with region proposal networks\n            RenS.\n            HeK.\n            GirshickR.\n            SunJ.\n            FasterR-C. N. N.\n          2016"},{"key":"B90","doi-asserted-by":"publisher","first-page":"299","DOI":"10.1016\/j.cell.2021.12.023","article-title":"Transition to invasive breast cancer is associated with progressive changes in the structure and composition of tumor stroma","volume":"185","author":"Risom","year":"2022","journal-title":"Cell"},{"key":"B91","doi-asserted-by":"crossref","DOI":"10.1101\/2022.11.04.514366","article-title":"Multimodal and spatially resolved profiling identifies distinct patterns of T-cell infiltration in nodal B-cell lymphoma entities","author":"Roider","year":"2022"},{"key":"B92","unstructured":"U-Net: Convolutional networks for biomedical image segmentation\n            RonnebergerO.\n            FischerP.\n            BroxT.\n          2015"},{"key":"B93","doi-asserted-by":"publisher","first-page":"1090","DOI":"10.1038\/s41587-019-0207-y","article-title":"Immuno-SABER enables highly multiplexed and amplified protein imaging in tissues","volume":"37","author":"Saka","year":"2019","journal-title":"Nat. Biotechnol."},{"key":"B94","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1016\/j.celrep.2018.03.086","article-title":"Spatial organization and molecular correlation of tumor-infiltrating lymphocytes using deep learning on pathology images","volume":"23","author":"Saltz","year":"2018","journal-title":"Cell Rep."},{"key":"B95","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1109\/TNN.2008.2005605","article-title":"The graph neural network model","volume":"20","author":"Scarselli","year":"2009","journal-title":"IEEE Trans. Neural Netw."},{"key":"B96","first-page":"265","volume-title":"Cell detection with star-convex polygons","author":"Schmidt","year":"2018"},{"key":"B97","doi-asserted-by":"publisher","first-page":"1270","DOI":"10.1038\/nbt1250","article-title":"Analyzing proteome topology and function by automated multidimensional fluorescence microscopy","volume":"24","author":"Schubert","year":"2006","journal-title":"Nat. Biotechnol."},{"key":"B98","doi-asserted-by":"publisher","first-page":"1341","DOI":"10.1016\/j.cell.2020.07.005","article-title":"Coordinated cellular neighborhoods orchestrate antitumoral immunity at the colorectal cancer invasive front","volume":"182","author":"Sch\u00fcrch","year":"2020","journal-title":"Cell"},{"key":"B99","doi-asserted-by":"publisher","first-page":"336","DOI":"10.1007\/s11263-019-01228-7","article-title":"Grad-CAM: Visual explanations from deep networks via gradient-based localization","volume":"128","author":"Selvaraju","year":"2020","journal-title":"Int. J. Comput. Vis."},{"key":"B100","doi-asserted-by":"publisher","first-page":"e197700","DOI":"10.1001\/jamanetworkopen.2019.7700","article-title":"Artificial intelligence algorithms to assess hormonal status from tissue microarrays in patients with breast cancer","volume":"2","author":"Shamai","year":"2019","journal-title":"JAMA Netw. Open"},{"key":"B101","doi-asserted-by":"publisher","first-page":"221","DOI":"10.1146\/annurev-bioeng-071516-044442","article-title":"Deep learning in medical image analysis","volume":"19","author":"Shen","year":"2017","journal-title":"Annu. Rev. Biomed. Eng."},{"key":"B102","volume-title":"Learning to read chest X-rays: Recurrent neural cascade model for automated image annotation","author":"Shin","year":"2016"},{"key":"B103","doi-asserted-by":"publisher","first-page":"17","DOI":"10.3322\/caac.21763","article-title":"Cancer statistics, 2023","volume":"73","author":"Siegel","year":"2023","journal-title":"CA A Cancer J. Clin."},{"key":"B104","unstructured":"Very deep convolutional networks for large-scale image recognition\n            SimonyanK.\n            ZissermanA.\n          2015"},{"key":"B105","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1038\/s41586-019-1629-x","article-title":"The human body at cellular resolution: The NIH human biomolecular atlas program","volume":"574","author":"Snyder","year":"2019","journal-title":"Nature"},{"key":"B106","doi-asserted-by":"publisher","first-page":"bbaa414","DOI":"10.1093\/bib\/bbaa414","article-title":"Dstg: Deconvoluting spatial transcriptomics data through graph-based artificial intelligence","volume":"22","author":"Song","year":"2021","journal-title":"Briefings Bioinforma."},{"key":"B108","doi-asserted-by":"publisher","first-page":"313","DOI":"10.1038\/s41587-020-0739-1","article-title":"Highly sensitive spatial transcriptomics at near-cellular resolution with Slide-seqV2","volume":"39","author":"Stickels","year":"2021","journal-title":"Nat. Biotechnol."},{"key":"B109","doi-asserted-by":"publisher","first-page":"100","DOI":"10.1038\/s41592-020-01018-x","article-title":"Cellpose: A generalist algorithm for cellular segmentation","volume":"18","author":"Stringer","year":"2021","journal-title":"Nat. Methods"},{"key":"B110","doi-asserted-by":"publisher","first-page":"569","DOI":"10.1016\/j.neuroimage.2014.06.077","article-title":"Hierarchical feature representation and multimodal fusion with deep learning for AD\/MCI diagnosis","volume":"101","author":"Suk","year":"2014","journal-title":"Neuroimage"},{"key":"B111","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1007\/978-94-017-7239-6_14","article-title":"Deep learning in diagnosis of brain disorders","volume-title":"Recent Progress in Brain and cognitive engineering. Trends in augmentation of human performance","author":"Suk","year":"2015"},{"key":"B112","doi-asserted-by":"publisher","first-page":"226","DOI":"10.1002\/cyto.a.23973","article-title":"Deep learning-based single-cell optical image studies","volume":"97","author":"Sun","year":"2020","journal-title":"Cytom. Part A"},{"key":"B113","doi-asserted-by":"publisher","first-page":"257","DOI":"10.1007\/s12194-017-0406-5","article-title":"Overview of deep learning in medical imaging","volume":"10","author":"Suzuki","year":"2017","journal-title":"Radiol. Phys. Technol."},{"key":"B114","doi-asserted-by":"publisher","first-page":"381","DOI":"10.1038\/nmeth.4220","article-title":"Power analysis of single-cell RNA-sequencing experiments","volume":"14","author":"Svensson","year":"2017","journal-title":"Nat. Methods"},{"key":"B115","doi-asserted-by":"publisher","first-page":"343","DOI":"10.1038\/nmeth.4636","article-title":"SpatialDE: Identification of spatially variable genes","volume":"15","author":"Svensson","year":"2018","journal-title":"Nat. Methods"},{"key":"B116","first-page":"1","article-title":"Going deeper with convolutions","author":"Szegedy","year":"2015"},{"key":"B117","first-page":"2818","article-title":"Rethinking the inception architecture for computer vision","author":"Szegedy","year":"2016"},{"key":"B118","doi-asserted-by":"publisher","first-page":"2244","DOI":"10.1038\/s41598-022-06100-2","article-title":"Deep learning of quantitative ultrasound multi-parametric images at pre-treatment to predict breast cancer response to chemotherapy","volume":"12","author":"Taleghamar","year":"2022","journal-title":"Sci. Rep."},{"key":"B119","doi-asserted-by":"publisher","first-page":"135","DOI":"10.1002\/cac2.12023","article-title":"Overview of multiplex immunohistochemistry\/immunofluorescence techniques in the era of cancer immunotherapy","volume":"40","author":"Tan","year":"","journal-title":"Cancer Commun."},{"key":"B120","doi-asserted-by":"publisher","first-page":"2293","DOI":"10.1093\/bioinformatics\/btz914","article-title":"SpaCell: Integrating tissue morphology and spatial gene expression to predict disease cells","volume":"36","author":"Tan","year":"","journal-title":"Bioinformatics"},{"key":"B121","volume-title":"Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results","author":"Tarvainen","year":"2018"},{"key":"B122","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1038\/s41591-018-0300-7","article-title":"High-performance medicine: The convergence of human and artificial intelligence","volume":"25","author":"Topol","year":"2019","journal-title":"Nat. Med."},{"key":"B123","doi-asserted-by":"publisher","first-page":"54","DOI":"10.1038\/s41698-021-00195-y","article-title":"Deep learning for end-to-end kidney cancer diagnosis on multi-phase abdominal computed tomography","volume":"5","author":"Uhm","year":"2021","journal-title":"npj Precis. Onc."},{"key":"B124","doi-asserted-by":"publisher","first-page":"636","DOI":"10.1002\/cyto.a.22625","article-title":"FlowSOM: Using self-organizing maps for visualization and interpretation of cytometry data: FlowSOM","volume":"87","author":"Van Gassen","year":"2015","journal-title":"Cytometry"},{"key":"B125","doi-asserted-by":"publisher","first-page":"5906","DOI":"10.1038\/s41467-021-26214-x","article-title":"Characterisation of tumour microenvironment remodelling following oncogene inhibition in preclinical studies with imaging mass cytometry","volume":"12","author":"van Maldegem","year":"2021","journal-title":"Nat. Commun."},{"key":"B126","doi-asserted-by":"publisher","first-page":"e1005177","DOI":"10.1371\/journal.pcbi.1005177","article-title":"Deep learning automates the quantitative analysis of individual cells in live-cell imaging experiments","volume":"12","author":"Van Valen","year":"2016","journal-title":"PLoS Comput. Biol."},{"key":"B127","first-page":"778","article-title":"Ovarian cancer mutational processes drive site-specific immune evasion","volume-title":"Nature","author":"V\u00e1zquez-Garc\u00eda","year":"2022"},{"key":"B128","doi-asserted-by":"publisher","first-page":"139","DOI":"10.1016\/j.ymeth.2017.07.015","article-title":"Single cell-based automated quantification of therapy responses of invasive cancer spheroids in organotypic 3D culture","volume":"128","author":"Veelken","year":"2017","journal-title":"Methods"},{"key":"B129","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1038\/s41592-021-01343-9","article-title":"Identifying temporal and spatial patterns of variation from multimodal data using MEFISTO","volume":"19","author":"Velten","year":"2022","journal-title":"Nat. Methods"},{"key":"B130","doi-asserted-by":"publisher","first-page":"8428","DOI":"10.1038\/s41598-022-12604-8","article-title":"CT-based deep learning enables early postoperative recurrence prediction for intrahepatic cholangiocarcinoma","volume":"12","author":"Wakiya","year":"2022","journal-title":"Sci. Rep."},{"key":"B149","doi-asserted-by":"publisher","DOI":"10.1126\/science.aat5691","article-title":"Three-dimensional intact-tissue sequencing of single-cell transcriptional states","volume":"361","author":"Wang","year":"2018","journal-title":"Science"},{"key":"B131","doi-asserted-by":"publisher","first-page":"39670","DOI":"10.1109\/ACCESS.2019.2906890","article-title":"RP-Net: a 3D convolutional neural network for brain segmentation from magnetic resonance imaging","volume":"7","author":"Wang","year":"","journal-title":"IEEE Access"},{"key":"B132","doi-asserted-by":"publisher","first-page":"1686","DOI":"10.1016\/j.ajpath.2019.05.007","article-title":"Pathology image analysis using segmentation deep learning algorithms","volume":"189","author":"Wang","year":"","journal-title":"Am. J. Pathology"},{"key":"B133","doi-asserted-by":"crossref","DOI":"10.1101\/2022.06.10.494732","article-title":"A single cell spatial temporal atlas of skeletal muscle reveals cellular neighborhoods that orchestrate regeneration and become disrupted in aging","author":"Wang","year":"2022"},{"key":"B134","unstructured":"The Challenge of crafting intelligible intelligence\n            WeldD. S.\n            BansalG.\n          2018"},{"key":"B135","doi-asserted-by":"publisher","first-page":"e131","DOI":"10.1093\/nar\/gkac901","article-title":"DeepST: Identifying spatial domains in spatial transcriptomics by deep learning","volume":"50","author":"Xu","year":"2022","journal-title":"Nucleic Acids Res."},{"key":"B136","doi-asserted-by":"publisher","first-page":"397","DOI":"10.1186\/s12859-021-04314-1","article-title":"CoSTA: Unsupervised convolutional neural network learning for spatial transcriptomics analysis","volume":"22","author":"Xu","year":"2021","journal-title":"BMC Bioinform."},{"key":"B137","doi-asserted-by":"publisher","first-page":"1263","DOI":"10.1038\/s42003-022-04076-3","article-title":"UnMICST: Deep learning with real augmentation for robust segmentation of highly multiplexed images of human tissues","volume":"5","author":"Yapp","year":"2022","journal-title":"Commun. Biol."},{"key":"B138","volume-title":"GNNExplainer: Generating explanations for graph neural networks","author":"Ying","year":"2019"},{"key":"B139","doi-asserted-by":"publisher","first-page":"19518","DOI":"10.1038\/s41598-019-55972-4","article-title":"Prostate cancer detection using deep convolutional neural networks","volume":"9","author":"Yoo","year":"2019","journal-title":"Sci. Rep."},{"key":"B140","doi-asserted-by":"publisher","first-page":"127","DOI":"10.5946\/ce.2020.046","article-title":"Lesion-based convolutional neural network in diagnosis of early gastric cancer","volume":"53","author":"Yoon","year":"2020","journal-title":"Clin. Endosc."},{"key":"B141","volume-title":"Deep layer aggregation","author":"Yu","year":"2017"},{"key":"B142","doi-asserted-by":"publisher","first-page":"6311","DOI":"10.1038\/s41467-021-26643-8","article-title":"Accurate recognition of colorectal cancer with semi-supervised deep learning on pathological images","volume":"12","author":"Yu","year":"2021","journal-title":"Nat. Commun."},{"key":"B143","doi-asserted-by":"publisher","first-page":"759","DOI":"10.1038\/s41592-022-01498-z","article-title":"Identification of cell types in multiplexed in situ images by combining protein expression and spatial information using CELESTA","volume":"19","author":"Zhang","year":"2022","journal-title":"Nat. Methods"},{"key":"B144","doi-asserted-by":"publisher","first-page":"214","DOI":"10.1016\/j.neuroimage.2014.12.061","article-title":"Deep convolutional neural networks for multi-modality isointense infant brain image segmentation","volume":"108","author":"Zhang","year":"2015","journal-title":"Neuroimage"},{"key":"B145","unstructured":"Pyramid Scene parsing network\n            ZhaoH.\n            ShiJ.\n            QiX.\n            WangX.\n            JiaJ.\n          2017"},{"key":"B146","doi-asserted-by":"publisher","first-page":"1236","DOI":"10.1038\/s41467-020-15027-z","article-title":"Deep learning radiomics can predict axillary lymph node status in early-stage breast cancer","volume":"11","author":"Zheng","year":"2020","journal-title":"Nat. Commun."},{"key":"B147","first-page":"2921","article-title":"Learning deep features for discriminative localization","author":"Zhou","year":"2016"},{"key":"B148","doi-asserted-by":"publisher","DOI":"10.1101\/2022.01.14.476408","article-title":"conST: an interpretable multi-modal contrastive learning framework for spatial transcriptomics","author":"Zong","year":"2022"}],"updated-by":[{"DOI":"10.3389\/fbinf.2023.1287407","type":"corrigendum","label":"Corrigendum","source":"publisher","updated":{"date-parts":[[2023,9,13]],"date-time":"2023-09-13T00:00:00Z","timestamp":1694563200000}}],"container-title":["Frontiers in Bioinformatics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fbinf.2023.1159381\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,13]],"date-time":"2023-09-13T19:35:18Z","timestamp":1694633718000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fbinf.2023.1159381\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,26]]},"references-count":149,"alternative-id":["10.3389\/fbinf.2023.1159381"],"URL":"https:\/\/doi.org\/10.3389\/fbinf.2023.1159381","relation":{},"ISSN":["2673-7647"],"issn-type":[{"value":"2673-7647","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,26]]},"article-number":"1159381"}}