{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,18]],"date-time":"2026-05-18T06:07:53Z","timestamp":1779084473915,"version":"3.51.4"},"reference-count":38,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,5,18]],"date-time":"2026-05-18T00:00:00Z","timestamp":1779062400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Oncol."],"abstract":"<jats:sec>\n                    <jats:title>Objectives<\/jats:title>\n                    <jats:p>Aimed to develop and evaluate a hierarchical multimodal framework to improve HER2 status prediction in Gastric Cancer (GC) patients.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>This retrospective study included 402 endoscopically identified GC patients (2014\u20132023), among whom 92 had pathologically confirmed HER2 status. Endoscopic images were analyzed using ResNet-50 to extract deep learning (DL) features and to develop a separate model for predicting invasion depth to aid in HER2 prediction. Radiomic features were extracted from contrast-enhanced CT images using Pyradiomics, with optimal features selected based on reproducibility (ICC &amp;gt; 0.75) and feature importance. Six machine learning algorithms were used to build multimodal models, and performance metrics such as area under the curve (AUC) and sensitivity were evaluated. Additionally, SHapley Additive exPlanations (SHAP) values were used to assess the contributions of individual features within the best-performing model.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Using ResNet-50 analysis of endoscopic images, an invasion depth prediction model was developed (validation AUC: 0.79; 95% CI: 0.77-0.81). The probability output from this model was integrated as a variable and combined with three optimal DL endoscopic features (1, 2, 8) for HER2 prediction. Additionally, seven discriminative features were extracted via radiomic analysis. When integrated with endoscopic, radiomics and clinical data, the multimodal model consistently outperformed unimodal approaches, with Logistic Regression yielding the highest validation AUC (0.83, 95% CI: 0.74-0.91). SHAP analysis highlighted Wavelength-HHL_GLSZM_SAHGE and DLfeature_8 as key HER2 predictors.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>By integrating clinical, endoscopic, and radiomic data, the hierarchical multimodal approach enhances HER2 expression prediction in GC, thereby optimizing clinical decision-making, and enabling tailored HER2-targeted therapies.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fonc.2026.1745228","type":"journal-article","created":{"date-parts":[[2026,5,18]],"date-time":"2026-05-18T05:43:37Z","timestamp":1779083017000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Interpretable deep learning-based hierarchical multi-modal fusion model for predicting HER2 expression in gastric cancer"],"prefix":"10.3389","volume":"16","author":[{"given":"Chenxi","family":"Hu","sequence":"first","affiliation":[{"name":"The Fourth School of Clinical Medicine, Zhejiang Chinese Medical University, Hangzhou First People's Hospital","place":["Hangzhou, Zhejiang, China"]},{"name":"Department of Radiology, Affiliated Hangzhou First People's Hospital, School of Medicine, Westlake University","place":["Hangzhou, Zhejiang, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Changfeng","family":"Feng","sequence":"additional","affiliation":[{"name":"Department of Radiology, Hangzhou Children\u2019s Hospital","place":["Hangzhou, Zhejiang, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ziyi","family":"Ye","sequence":"additional","affiliation":[{"name":"The Fourth School of Clinical Medicine, Zhejiang Chinese Medical University, Hangzhou First People's Hospital","place":["Hangzhou, Zhejiang, China"]},{"name":"Department of Radiology, Affiliated Hangzhou First People's Hospital, School of Medicine, Westlake University","place":["Hangzhou, Zhejiang, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shuaihang","family":"Kong","sequence":"additional","affiliation":[{"name":"The Fourth School of Clinical Medicine, Zhejiang Chinese Medical University, Hangzhou First People's Hospital","place":["Hangzhou, Zhejiang, China"]},{"name":"Department of Radiology, Affiliated Hangzhou First People's Hospital, School of Medicine, Westlake University","place":["Hangzhou, Zhejiang, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jing","family":"Han","sequence":"additional","affiliation":[{"name":"Department of Radiology, Zhejiang Kangjing Hospital","place":["Hangzhou, Zhejiang, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tingting","family":"Yang","sequence":"additional","affiliation":[{"name":"The Fourth School of Clinical Medicine, Zhejiang Chinese Medical University, Hangzhou First People's Hospital","place":["Hangzhou, Zhejiang, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kai","family":"Zhong","sequence":"additional","affiliation":[{"name":"The Fourth School of Clinical Medicine, Zhejiang Chinese Medical University, Hangzhou First People's Hospital","place":["Hangzhou, Zhejiang, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xingguang","family":"Zhong","sequence":"additional","affiliation":[{"name":"Department of Radiology, Zhejiang Cancer Hospital","place":["Hangzhou, Zhejiang, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qijun","family":"Shen","sequence":"additional","affiliation":[{"name":"Department of Radiology, Affiliated Hangzhou First People's Hospital, School of Medicine, Westlake University","place":["Hangzhou, Zhejiang, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2026,5,18]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.3322\/caac.21834","article-title":"Global cancer statistics 2022: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries","volume":"74","author":"Bray","year":"2024","journal-title":"CA Cancer J Clin"},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.3390\/ijms21114012","article-title":"Gastric cancer: Epidemiology, risk factors, classification, genomic characteristics and treatment strategies","volume":"21","author":"Machlowska","year":"2020","journal-title":"Int J Mol Sci"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.3390\/ijms19061658","article-title":"The pattern of signatures in gastric cancer prognosis","volume":"19","author":"Machlowska","year":"2018","journal-title":"Int J Mol Sci"},{"key":"B4","doi-asserted-by":"publisher","DOI":"10.1200\/JCO.20.02761","article-title":"HER2 expression, test deviations, and their impact on survival in metastatic gastric cancer: Results from the prospective multicenter VARIANZ study","volume":"39","author":"Haffner","year":"2021","journal-title":"J Clin Oncol"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-021-04161-3","article-title":"The KEYNOTE-811 trial of dual PD-1 and HER2 blockade in HER2-positive gastric cancer","volume":"600","author":"Janjigian","year":"2021","journal-title":"Nature"},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.1200\/JCO.2016.69.4836","article-title":"HER2 testing and clinical decision making in gastroesophageal adenocarcinoma: Guideline from the College of American Pathologists, American Society for Clinical Pathology, and the American Society of Clinical Oncology","volume":"35","author":"Bartley","year":"2017","journal-title":"J Clin Oncol"},{"key":"B7","doi-asserted-by":"publisher","first-page":"866","DOI":"10.1016\/j.gie.2020.06.047","article-title":"Highly accurate artificial intelligence systems to predict the invasion depth of gastric cancer: Efficacy of conventional white-light imaging, nonmagnifying narrow-band imaging, and indigo-carmine dye contrast imaging","volume":"92","author":"Nagao","year":"2020","journal-title":"Gastrointest Endosc"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.1016\/j.gie.2017.11.029","article-title":"Computer-aided diagnosis for identifying and delineating early gastric cancers in magnifying narrow-band imaging","volume":"87","author":"Kanesaka","year":"2018","journal-title":"Gastrointest Endosc"},{"key":"B9","doi-asserted-by":"publisher","first-page":"806","DOI":"10.1016\/j.gie.2018.11.011","article-title":"Application of convolutional neural network in the diagnosis of the invasion depth of gastric cancer based on conventional endoscopy","volume":"89","author":"Zhu","year":"2019","journal-title":"Gastrointest Endosc"},{"key":"B10","first-page":"1","volume-title":"AJCC cancer staging manual","author":"Amin","year":"2017"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1155\/2015\/946359","article-title":"Correlations of human epithelial growth factor receptor 2 overexpression with MUC2, MUC5AC, MUC6, p53, and clinicopathological characteristics in gastric cancer patients with curative resection","volume":"2015","author":"Park","year":"2015","journal-title":"Gastroenterol Res Pract"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.1007\/s10120-015-0518-8","article-title":"Clinicopathological factors associated with HER2 status in gastric cancer: Results from a prospective multicenter observational cohort study in a Japanese population (JFMC44-1101)","volume":"19","author":"Matsusaka","year":"2016","journal-title":"Gastric Cancer"},{"key":"B13","doi-asserted-by":"publisher","DOI":"10.1055\/a-0981-6133","article-title":"Automated classification of gastric neoplasms in endoscopic images using a convolutional neural network","volume":"51","author":"Cho","year":"2019","journal-title":"Endoscopy"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1016\/S0140-6736(20)31288-5","article-title":"Gastric cancer","volume":"396","author":"Smyth","year":"2020","journal-title":"Lancet"},{"key":"B15","doi-asserted-by":"publisher","first-page":"26","DOI":"10.1007\/s00261-008-9380-z","article-title":"Gastric cancer staging at isotropic MDCT including coronal and sagittal MPR images: Endoscopically diagnosed early vs. advanced gastric cancer","volume":"34","author":"Kim","year":"2009","journal-title":"Abdom Imaging"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.1102\/1470-7330.2013.0023","article-title":"Gastric carcinoma: Imaging diagnosis, staging and assessment of treatment response","volume":"13","author":"Hallinan","year":"2013","journal-title":"Cancer Imaging"},{"key":"B17","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1186\/s13244-022-01361-x","article-title":"Development and external validation of a radiomics model for assessment of HER2 positivity in men and women presenting with gastric cancer","volume":"14","author":"Zhao","year":"2023","journal-title":"Insights Imaging"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.2147\/CMAR.S230138","article-title":"Contrast-enhanced CT parameters of gastric adenocarcinoma: Can radiomic features be surrogate biomarkers for HER2 over-expression status","volume":"12","author":"Wang","year":"2020","journal-title":"Cancer Manag Res"},{"key":"B19","doi-asserted-by":"publisher","first-page":"62","DOI":"10.21147\/j.issn.1000-9604.2020.01.08","article-title":"A CT-based radiomics nomogram for prediction of human epidermal growth factor receptor 2 status in patients with gastric cancer","volume":"32","author":"Li","year":"2020","journal-title":"Chin J Cancer Res"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.1016\/j.ajpath.2025.06.006","article-title":"From single-cancer to pan-cancer prognosis: A multimodal deep learning framework for survival analysis with robust generalization capability","volume":"195","author":"Zhang","year":"2025","journal-title":"Am J Pathol"},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.1038\/s43018-022-00388-9","article-title":"Multimodal data integration using machine learning improves risk stratification of high-grade serous ovarian cancer","volume":"3","author":"Boehm","year":"2022","journal-title":"Nat Cancer"},{"key":"B22","doi-asserted-by":"publisher","first-page":"3510","DOI":"10.1038\/s41467-025-58675-9","article-title":"Multimodal fusion of radio-pathology and proteogenomics identify integrated glioma subtypes with prognostic and therapeutic opportunities","volume":"16","author":"Liu","year":"2025","journal-title":"Nat Commun"},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.6004\/jnccn.2022.0008","article-title":"Gastric cancer, version 2.2022, NCCN clinical practice guidelines in oncology","volume":"20","author":"Ajani","year":"2022","journal-title":"J Natl Compr Canc Netw"},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.1007\/s10120-011-0041-5","article-title":"Japanese classification of gastric carcinoma: 3rd English edition","volume":"14","year":"2011","journal-title":"Gastric Cancer"},{"key":"B25","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1186\/s40644-024-00741-4","article-title":"Development and validation of a machine learning-based 18F-fluorodeoxyglucose PET\/CT radiomics signature for predicting gastric cancer survival","volume":"24","author":"Zhi","year":"2024","journal-title":"Cancer Imaging"},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.3390\/cancers16030614","article-title":"Preoperative prediction of perineural invasion and prognosis in gastric cancer based on machine learning through a radiomics-clinicopathological nomogram","volume":"16","author":"Jia","year":"2024","journal-title":"Cancers (Basel)"},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.1007\/s00261-023-04165-9","article-title":"Intratumoral and peritumoral CT-based radiomics for predicting the microsatellite instability in gastric cancer","volume":"49","author":"Chen","year":"2024","journal-title":"Abdom Radiol (NY)"},{"key":"B28","doi-asserted-by":"publisher","DOI":"10.7717\/peerj.17111","article-title":"Predictive nomogram for lymph node metastasis and survival in gastric cancer using contrast-enhanced computed tomography-based radiomics: A retrospective study","volume":"12","author":"Zhang","year":"2024","journal-title":"PeerJ"},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.1245\/s10434-024-15631-z","article-title":"18F-FDG PET\/CT radiomics-based multimodality fusion model for preoperative individualized noninvasive prediction of peritoneal metastasis in advanced gastric cancer","volume":"31","author":"Chen","year":"2024","journal-title":"Ann Surg Oncol"},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.3390\/diagnostics14151689","article-title":"Prediction of bone marrow metastases using computed tomography (CT) radiomics in patients with gastric cancer: Uncovering invisible metastases","volume":"14","author":"Park","year":"2024","journal-title":"Diagnostics (Basel)"},{"key":"B31","doi-asserted-by":"publisher","DOI":"10.1158\/0008-5472.CAN-17-0339","article-title":"Computational radiomics system to decode the radiographic phenotype","volume":"77","author":"van Griethuysen","year":"2017","journal-title":"Cancer Res"},{"key":"B32","doi-asserted-by":"publisher","DOI":"10.1038\/emm.2013.111","article-title":"Combined blockade of HER2 and VEGF exerts greater growth inhibition of HER2-overexpressing gastric cancer xenografts than individual blockade","volume":"45","author":"Singh","year":"2013","journal-title":"Exp Mol Med"},{"key":"B33","doi-asserted-by":"publisher","first-page":"396","DOI":"10.4132\/jptm.2017.04.24","article-title":"HER2 status and its heterogeneity in gastric carcinoma of Vietnamese patient","volume":"51","author":"Phan","year":"2017","journal-title":"J Pathol Transl Med"},{"key":"B34","doi-asserted-by":"publisher","first-page":"238","DOI":"10.1186\/s12876-022-02309-7","article-title":"Relationship between HER2 overexpression and long-term outcomes of early gastric cancer: A prospective observational study with a 6-year follow-up","volume":"22","author":"Li","year":"2022","journal-title":"BMC Gastroenterol"},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.1016\/j.acra.2020.02.018","article-title":"CT radiomics for distinction of human epidermal growth factor receptor 2 negative gastric cancer","volume":"28","author":"Wang","year":"2021","journal-title":"Acad Radiol"},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-019-0048-x","article-title":"Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead","volume":"1","author":"Rudin","year":"2019","journal-title":"Nat Mach Intell"},{"key":"B37","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1016\/j.jclinepi.2019.02.004","article-title":"A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models","volume":"110","author":"Christodoulou","year":"2019","journal-title":"J Clin Epidemiol"},{"key":"B38","doi-asserted-by":"publisher","DOI":"10.1161\/CIRCULATIONAHA.106.682658","article-title":"Logistic regression","volume":"117","author":"LaValley","year":"2008","journal-title":"Circulation"}],"container-title":["Frontiers in Oncology"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fonc.2026.1745228\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,18]],"date-time":"2026-05-18T05:43:38Z","timestamp":1779083018000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fonc.2026.1745228\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,18]]},"references-count":38,"alternative-id":["10.3389\/fonc.2026.1745228"],"URL":"https:\/\/doi.org\/10.3389\/fonc.2026.1745228","relation":{},"ISSN":["2234-943X"],"issn-type":[{"value":"2234-943X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,18]]},"article-number":"1745228"}}