{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,21]],"date-time":"2026-05-21T14:11:25Z","timestamp":1779372685200,"version":"3.53.1"},"reference-count":49,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2026,5,21]],"date-time":"2026-05-21T00:00:00Z","timestamp":1779321600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"funder":[{"name":"National Key Research and Development Program of China","award":["2022YFF0902500"],"award-info":[{"award-number":["2022YFF0902500"]}]},{"name":"King Saud University, Riyadh, Saudi Arabia","award":["ORF-2026-493"],"award-info":[{"award-number":["ORF-2026-493"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2026,6,30]]},"abstract":"<jats:p>The classification of glioma subtypes is critical in clinical diagnosis and treatment planning. Multimodal deep learning enables more informative representations through the integration of diverse and complementary feature sources. In medical imaging, multimodal fusion of radiological and pathological data offers a practical approach for leveraging heterogeneous information for improved glioma subtype classification. Nevertheless, effectively extracting and integrating discriminative features from different modalities remains challenging. In this article, glioma multi-omics ResNet (GMRNet), which is a deep learning model for glioma subtype classification, is proposed. The model integrates radiomic and pathomic features through label-supervised semantic feature fusion to improve the classification performance. It uses a ResNet50 backbone to extract multiscale features from magnetic resonance imaging (MRI) data and whole-slide imaging (WSI) data separately and then fuses these features through feature concatenation. Multi-omics fusion increases the classification accuracy across glioma subtypes, including astrocytoma, oligodendroglioma, anaplastic astrocytoma, anaplastic oligodendroglioma, and glioblastoma. First, MRI and WSI data are passed through a systematic preprocessing pipeline to improve cross-sample consistency and preserve discriminative imaging structures. To overcome the limitations of scarce clinical data, diverse augmentation strategies, such as cropping, flipping, contrast adjustment, and affine transformations, are applied. Second, modality-specific features are extracted using two independent ResNet50-based branches. An MRI branch captures tumor morphology and structural patterns across multiple modalities, while a WSI branch focuses on cellular and tissue-level characteristics. This dual-branch design preserves the unique information of each modality and provides complementary perspectives for downstream fusion. Third, the extracted features are concatenated and passed through a fully connected bottleneck layer to integrate complementary cross-modal information and produce a compact fused representation. The fused representation is then fed into a multilayer classifier to predict glioma subtypes. Finally, the experimental results demonstrate that the proposed model achieves accuracies of 93.65% on the Huashan multimodal glioma (HMG) dataset and 86.47% on the open-access glioma (OAG) dataset. Compared with single-modality configurations, the multimodal fusion setting yields consistent empirical performance gains across both datasets.<\/jats:p>","DOI":"10.1145\/3812544","type":"journal-article","created":{"date-parts":[[2026,4,27]],"date-time":"2026-04-27T12:27:17Z","timestamp":1777292837000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["GMRNet: Deep Residual Network-Based Radiopathomic Glioma Classification with the Fusion of Multi-omics Data"],"prefix":"10.1145","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4215-3446","authenticated-orcid":false,"given":"Hongwei","family":"Zeng","sequence":"first","affiliation":[{"name":"School of Computer Engineering and Science, Shanghai University, Shanghai, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-6012-1124","authenticated-orcid":false,"given":"Fang","family":"Zhao","sequence":"additional","affiliation":[{"name":"School of Computer Engineering and Science, Shanghai University, Shanghai, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-9982-9544","authenticated-orcid":false,"given":"Lingchao","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Neurosurgery, Fudan University Huashan Hospital, Shanghai, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5384-4627","authenticated-orcid":false,"given":"Hongxia","family":"Xu","sequence":"additional","affiliation":[{"name":"Innovation Institute for Artificial Intelligence in Medicine, College of Pharmaceutical Sciences, Zhejiang University, Hangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6861-9684","authenticated-orcid":false,"given":"Honghao","family":"Gao","sequence":"additional","affiliation":[{"name":"School of Computer Engineering and Science, Shanghai University, Shanghai, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7479-7102","authenticated-orcid":false,"given":"Bader Fahad","family":"Alkhamees","sequence":"additional","affiliation":[{"name":"Department of Information Systems, College of Computer and Information Sciences, King Saud University, Riyadh, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,5,21]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2023.121453"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2010.09.025"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11063-020-10398-2"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMI.2015.2476509"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1001\/jamaoncol.2022.2844"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.3389\/fonc.2022.1005805"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1186\/s13059-021-02577-8"},{"key":"e_1_3_2_9_2","unstructured":"Maria Correia de Verdier Rachit Saluja Louis Gagnon Dominic LaBella Ujjwall Baid Nourel Hoda Tahon Martha Foltyn-Dumitru Jikai Zhang Maram Alafif Saif Baig et al. 2024. The 2024 Brain Tumor Segmentation (BraTS) challenge: Glioma segmentation on post-treatment MRI. arXiv:2405.18368. Retrieved from https:\/\/arxiv.org\/abs\/2405.18368"},{"key":"e_1_3_2_10_2","first-page":"1467","volume-title":"Proceedings of the 32nd ACM International Conference on Multimedia (MM \u201924)","author":"Trung Duc Tran Dang","year":"2024","unstructured":"Tran Dang Trung Duc, Byeongkeun Kang, and Yeejin Lee. 2024. Multi-scale twin-attention for 3D instance segmentation. In Proceedings of the 32nd ACM International Conference on Multimedia (MM \u201924), 1467\u20131475."},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/3713075"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1016\/S2589-7500(21)00215-6"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.bspc.2024.106774"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2023.122672"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNSE.2025.3541138"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/JBHI.2023.3266614"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-022-09985-1"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-020-00341-z"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41592-020-01008-z"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-72087-2_11"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1002\/hbm.24750"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1186\/s13059-021-02565-y"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1093\/neuonc\/noaa163"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-021-10121-0"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jneumeth.2016.03.001"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2024.108635"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-022-35031-9"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ccell.2022.09.012"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-025-58675-9"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.compeleceng.2024.109400"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.3390\/life13020349"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0200730"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1148\/rg.230086"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3767748"},{"key":"e_1_3_2_35_2","first-page":"85","article-title":"Towards generation, management, and exploration of combined radiomics and pathomics datasets for cancer research","volume":"2017","author":"Saltz Joel","year":"2017","unstructured":"Joel Saltz, Jonas Almeida, Yi Gao, Ashish Sharma, Erich Bremer, Tammy DiPrima, Mary Saltz, Jayashree Kalpathy-Cramer, and Tahsin Kurc. 2017. Towards generation, management, and exploration of combined radiomics and pathomics datasets for cancer research. AMIA Joint Summits on Translational Science Proceedings 2017 (2017), 85\u201394.","journal-title":"AMIA Joint Summits on Translational Science Proceedings"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.13005\/bpj\/1511"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMI.2022.3202037"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-023-00633-5"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2024.125443"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2019.101544"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMI.2016.2529665"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3664647.3681274"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1186\/s13045-022-01225-3"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-023-41195-9"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.3389\/fbioe.2022.841958"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-024-07894-z"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/3664647.3680588"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00330-023-10378-9"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.compmedimag.2015.04.002"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/3737461"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3812544","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,21]],"date-time":"2026-05-21T13:33:00Z","timestamp":1779370380000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3812544"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,21]]},"references-count":49,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2026,6,30]]}},"alternative-id":["10.1145\/3812544"],"URL":"https:\/\/doi.org\/10.1145\/3812544","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,21]]},"assertion":[{"value":"2025-09-28","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-04-15","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-05-21","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}