{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,3]],"date-time":"2026-06-03T02:22:27Z","timestamp":1780453347710,"version":"3.54.1"},"reference-count":50,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2025,11,11]],"date-time":"2025-11-11T00:00:00Z","timestamp":1762819200000},"content-version":"vor","delay-in-days":10,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National High Level Hospital Clinical Research Funding","award":["2025-NHLHCRF-JBGS-B-WZ-08"],"award-info":[{"award-number":["2025-NHLHCRF-JBGS-B-WZ-08"]}]},{"name":"Elite Medical Professionals Project of China-Japan Friendship Hospital","award":["ZRJY2024-GG01"],"award-info":[{"award-number":["ZRJY2024-GG01"]}]},{"name":"Beijing Chaoyang Digital Health Proof of Concept Project","award":["2025SLZD020"],"award-info":[{"award-number":["2025SLZD020"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,11,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Microsatellite instability (MSI) and tumor mutational burden (TMB) are crucial biomarkers in gastric (GC) and colorectal cancer (CRC), yet their conventional sequencing-based detection is costly and time-consuming. Since only ~20% of patients are MSI-high or TMB-high and likely to benefit from immunotherapy, expensive genomic testing is often unjustified. This study developed a deep learning framework to predict MSI and TMB status directly from routinely available Hematoxylin and Eosin (H&amp;E)-stained whole-slide images, leveraging fused nuclear segmentation features to improve accuracy. Using samples from TCGA (350 GC and 376 CRC for MSI; 400 GC and 387 CRC for TMB), image features were extracted with CLAM and nuclear features with Hover-Net. These features were combined via Multimodal Compact Bilinear Pooling and utilized in six distinct deep learning models. By fusing the nucleus segmentation features, the model increased area under the receiver operating characteristic curve (AUC) by 1%\u20133% and recall by 5%\u201311% in five-fold cross-validation, significantly outperforming models that relied solely on image features. External validation on a CRC dataset from the China-Japan Friendship hospital further validated the model's robustness, achieving an AUC of 0.81 and a recall of 0.80 for MSI prediction. Additionally, notable differences in cellular composition were observed across cancer types and clinical groups, emphasizing the pivotal role of cellular features in cancer development. These findings highlight the advantages of integrating H&amp;E-stained image features with nuclear segmentation data and advanced deep learning techniques to improve predictive accuracy and reduce the cost of MSI\/TMB testing, potentially advancing personalized cancer treatment strategies.<\/jats:p>","DOI":"10.1093\/bib\/bbaf580","type":"journal-article","created":{"date-parts":[[2025,11,11]],"date-time":"2025-11-11T04:27:37Z","timestamp":1762835257000},"source":"Crossref","is-referenced-by-count":9,"title":["Deep learning-based fusion of nuclear segmentation features for microsatellite instability and tumor mutational burden prediction in digestive tract cancers: a multicenter validation study"],"prefix":"10.1093","volume":"26","author":[{"given":"Yanping","family":"Zhang","sequence":"first","affiliation":[{"name":"School of Mathematics and Physics, Hebei University of Engineering , 19 Taiji Road, Handan 056038 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-3860-0502","authenticated-orcid":false,"given":"Jiaying","family":"Han","sequence":"additional","affiliation":[{"name":"School of Mathematics and Physics, Hebei University of Engineering , 19 Taiji Road, Handan 056038 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Huang","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Pathology, China-Japan Friendship Hospital , 2 East Yinghuayuan Street, Beijing 100029 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fengyuan","family":"Hu","sequence":"additional","affiliation":[{"name":"School of Mathematics and Physics, Hebei University of Engineering , 19 Taiji Road, Handan 056038 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yaping","family":"Huang","sequence":"additional","affiliation":[{"name":"School of Mathematics and Physics, Hebei University of Engineering , 19 Taiji Road, Handan 056038 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Geng","family":"Tian","sequence":"additional","affiliation":[{"name":"Geneis Beijing Co., Ltd. , 31 Xinbei Road, Beijing 100102 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dingrong","family":"Zhong","sequence":"additional","affiliation":[{"name":"Department of Pathology, China-Japan Friendship Hospital , 2 East Yinghuayuan Street, Beijing 100029 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4689-8672","authenticated-orcid":false,"given":"Jialiang","family":"Yang","sequence":"additional","affiliation":[{"name":"Geneis Beijing Co., Ltd. , 31 Xinbei Road, Beijing 100102 ,","place":["China"]},{"name":"Academician Workstation, Changsha Medical University , 1501 Leifeng Road, Changsha 410219 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2025,11,11]]},"reference":[{"key":"2025111023273188300_ref1","doi-asserted-by":"publisher","first-page":"229","DOI":"10.3322\/caac.21834","article-title":"Global cancer statistics 2022: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries","volume":"74","author":"Bray","year":"2024","journal-title":"CA Cancer J Clin"},{"key":"2025111023273188300_ref2","doi-asserted-by":"publisher","first-page":"1353446","DOI":"10.3389\/fonc.2024.1353446","article-title":"Predicting rectal cancer prognosis from histopathological images and clinical information using multi-modal deep learning","volume":"14","author":"Xu","year":"2024","journal-title":"Front Oncol"},{"key":"2025111023273188300_ref3","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2025.112839","article-title":"Predicting cell\u2013cell communication by combining heterogeneous ensemble deep learning and weighted geometric mean","volume":"172","author":"Peng","year":"2025","journal-title":"Appl Soft Comput"},{"key":"2025111023273188300_ref4","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1038\/s41698-022-00285-5","article-title":"Cell graph neural networks enable the precise prediction of patient survival in gastric cancer","volume":"6","author":"Wang","year":"2022","journal-title":"NPJ Precis Oncol"},{"key":"2025111023273188300_ref5","doi-asserted-by":"publisher","first-page":"1003419","DOI":"10.3389\/fimmu.2022.1003419","article-title":"Single-cell transcriptome analysis reveals heterogeneity and convergence of the tumor microenvironment in colorectal cancer","volume":"13","author":"Xie","year":"2022","journal-title":"Front Immunol"},{"key":"2025111023273188300_ref6","doi-asserted-by":"publisher","DOI":"10.1002\/cam4.6736","article-title":"Heterogeneity-induced NGF-NGFR communication inefficiency promotes mitotic spindle disorganization in exhausted T cells through PREX1 suppression to impair the anti-tumor immunotherapy with PD-1 mAb in hepatocellular carcinoma","volume":"13","author":"Wang","year":"2024","journal-title":"Cancer Med"},{"key":"2025111023273188300_ref7","doi-asserted-by":"publisher","first-page":"104342","DOI":"10.1016\/j.critrevonc.2024.104342","article-title":"Tumor mutational burden in colorectal cancer: implications for treatment","volume":"197","author":"Marques","year":"2024","journal-title":"Crit Rev Oncol Hematol"},{"key":"2025111023273188300_ref8","doi-asserted-by":"publisher","first-page":"298","DOI":"10.1002\/ijc.34251","article-title":"Deep learning captures selective features for discrimination of microsatellite instability from pathologic tissue slides of gastric cancer","volume":"152","author":"Lee","year":"2023","journal-title":"Int J Cancer"},{"key":"2025111023273188300_ref9","doi-asserted-by":"publisher","first-page":"907","DOI":"10.1007\/s10120-024-01523-4","article-title":"Potent therapeutic strategy in gastric cancer with microsatellite instability-high and\/or deficient mismatch repair","volume":"27","author":"Ooki","year":"2024","journal-title":"Gastric Cancer"},{"key":"2025111023273188300_ref10","doi-asserted-by":"publisher","first-page":"1370031","DOI":"10.3389\/fonc.2024.1370031","article-title":"Nomogram based on dual-energy CT-derived extracellular volume fraction for the prediction of microsatellite instability status in gastric cancer","volume":"14","author":"Hu","year":"2024","journal-title":"Front Oncol"},{"key":"2025111023273188300_ref11","doi-asserted-by":"publisher","first-page":"1406","DOI":"10.1053\/j.gastro.2020.06.021","article-title":"Clinical-grade detection of microsatellite instability in colorectal Tumors by deep learning","volume":"159","author":"Echle","year":"2020","journal-title":"Gastroenterology"},{"key":"2025111023273188300_ref12","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1016\/j.ejca.2020.02.038","article-title":"Tumour mutational burden as a biomarker for immunotherapy: current data and emerging concepts","volume":"131","author":"Fumet","year":"2020","journal-title":"Eur J Cancer"},{"key":"2025111023273188300_ref13","doi-asserted-by":"publisher","first-page":"282","DOI":"10.1186\/s12885-021-07942-1","article-title":"A next-generation sequencing-based strategy combining microsatellite instability and tumor mutation burden for comprehensive molecular diagnosis of advanced colorectal cancer","volume":"21","author":"Xiao","year":"2021","journal-title":"BMC Cancer"},{"key":"2025111023273188300_ref14","doi-asserted-by":"publisher","first-page":"103372","DOI":"10.1016\/j.media.2024.103372","article-title":"Ensemble transformer-based multiple instance learning to predict pathological subtypes and tumor mutational burden from histopathological whole slide images of endometrial and colorectal cancer","volume":"99","author":"Wang","year":"2025","journal-title":"Med Image Anal"},{"key":"2025111023273188300_ref15","doi-asserted-by":"publisher","first-page":"17164","DOI":"10.1109\/ACCESS.2024.3359989","article-title":"Prediction of tuberculosis from lung tissue images of diversity outbred mice using jump knowledge based cell graph neural network","volume":"12","author":"Acharya","year":"2024","journal-title":"IEEE Access"},{"key":"2025111023273188300_ref16","doi-asserted-by":"publisher","first-page":"e48320","DOI":"10.2196\/48320","article-title":"The use of deep learning and machine learning on longitudinal electronic health Records for the Early Detection and Prevention of diseases: scoping review","volume":"26","author":"Swinckels","year":"2024","journal-title":"J Med Internet Res"},{"key":"2025111023273188300_ref17","doi-asserted-by":"publisher","first-page":"5420","DOI":"10.21037\/qims-23-1743","article-title":"Deep learning-based detection of primary bone tumors around the knee joint on radiographs: a multicenter study","volume":"14","author":"Xu","year":"2024","journal-title":"Quant Imaging Med Surg"},{"key":"2025111023273188300_ref18","doi-asserted-by":"publisher","DOI":"10.1038\/s41698-025-00855-3","article-title":"Deep learning models in classifying primary bone tumors and bone infections based on radiographs. npj precision","volume":"9","author":"Wang","year":"2025","journal-title":"Oncology"},{"key":"2025111023273188300_ref19","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btaf024","article-title":"CAMIL: channel attention-based multiple instance learning for whole slide image classification","volume":"41","author":"Mao","year":"2025","journal-title":"Bioinformatics"},{"key":"2025111023273188300_ref20","doi-asserted-by":"publisher","first-page":"19436","DOI":"10.1038\/s41598-023-46607-w","article-title":"Histological classification of canine and feline lymphoma using a modular approach based on deep learning and advanced image processing","volume":"13","author":"Haghofer","year":"2023","journal-title":"Sci Rep"},{"key":"2025111023273188300_ref21","doi-asserted-by":"publisher","first-page":"6367","DOI":"10.1038\/s41467-020-20030-5","article-title":"Deep learning-based cross-classifications reveal conserved spatial behaviors within tumor histological images","volume":"11","author":"Noorbakhsh","year":"2020","journal-title":"Nat Commun"},{"key":"2025111023273188300_ref22","doi-asserted-by":"publisher","first-page":"757","DOI":"10.1093\/jamia\/ocz230","article-title":"Classifying non-small cell lung cancer types and transcriptomic subtypes using convolutional neural networks","volume":"27","author":"Yu","year":"2020","journal-title":"J Am Med Inform Assoc"},{"key":"2025111023273188300_ref23","doi-asserted-by":"crossref","first-page":"282","DOI":"10.1186\/s12916-024-03482-0","article-title":"Mining the interpretable prognostic features from pathological image of intrahepatic cholangiocarcinoma using multi-modal deep learning","volume":"22","author":"Ding","year":"2024","journal-title":"BMC Med"},{"key":"2025111023273188300_ref24","doi-asserted-by":"publisher","first-page":"107095","DOI":"10.1016\/j.cmpb.2022.107095","article-title":"PPsNet: an improved deep learning model for microsatellite instability high prediction in colorectal cancer from whole slide images","volume":"225","author":"Lou","year":"2022","journal-title":"Comput Methods Prog Biomed"},{"key":"2025111023273188300_ref25","doi-asserted-by":"publisher","DOI":"10.1093\/bib\/bbaf209","article-title":"MMsurv: a multimodal multi-instance multi-cancer survival prediction model integrating pathological images, clinical information, and sequencing data","volume":"26","author":"Yang","year":"2025","journal-title":"Brief Bioinform"},{"key":"2025111023273188300_ref26","doi-asserted-by":"publisher","first-page":"333","DOI":"10.1016\/j.csbj.2021.12.028","article-title":"Prediction of HER2-positive breast cancer recurrence and metastasis risk from histopathological images and clinical information via multimodal deep learning","volume":"20","author":"Yang","year":"2022","journal-title":"Comput Struct Biotechnol J"},{"key":"2025111023273188300_ref27","doi-asserted-by":"publisher","first-page":"107608","DOI":"10.1016\/j.bspc.2025.107608","article-title":"Prediction of colorectal cancer microsatellite instability and tumor mutational burden from histopathological images using multiple instance learning","volume":"104","author":"Wang","year":"2025","journal-title":"Biomedical Signal Processing and Control"},{"key":"2025111023273188300_ref28","doi-asserted-by":"publisher","first-page":"102464","DOI":"10.1016\/j.media.2022.102464","article-title":"DeepSMILE: contrastive self-supervised pre-training benefits MSI and HRD classification directly from H&E whole-slide images in colorectal and breast cancer","volume":"79","author":"Schirris","year":"2022","journal-title":"Med Image Anal"},{"key":"2025111023273188300_ref29","doi-asserted-by":"publisher","first-page":"16605","DOI":"10.1038\/s41598-021-95747-4","article-title":"Comparative analysis of machine learning approaches to classify tumor mutation burden in lung adenocarcinoma using histopathology images","volume":"11","author":"Sadhwani","year":"2021","journal-title":"Sci Rep"},{"key":"2025111023273188300_ref30","doi-asserted-by":"publisher","first-page":"635","DOI":"10.1016\/S0140-6736(20)31288-5","article-title":"Gastric cancer","volume":"396","author":"Smyth","year":"2020","journal-title":"Lancet (London, England)"},{"key":"2025111023273188300_ref31","doi-asserted-by":"publisher","first-page":"1843","DOI":"10.1016\/j.cmet.2022.08.016","article-title":"Tumor-associated macrophages are shaped by intratumoral high potassium via Kir2.1","volume":"34","author":"Chen","year":"2022","journal-title":"Cell Metab"},{"key":"2025111023273188300_ref32","doi-asserted-by":"publisher","DOI":"10.1093\/bib\/bbac448","article-title":"ICSDA: a multi-modal deep learning model to predict breast cancer recurrence and metastasis risk by integrating pathological, clinical and gene expression data","volume":"23","author":"Yao","year":"2022","journal-title":"Brief Bioinform"},{"key":"2025111023273188300_ref33","doi-asserted-by":"publisher","first-page":"925079","DOI":"10.3389\/fonc.2022.925079","article-title":"Evaluating the microsatellite instability of colorectal cancer based on multimodal deep learning integrating histopathological and molecular data","volume":"12","author":"Qiu","year":"2022","journal-title":"Front Oncol"},{"key":"2025111023273188300_ref34","doi-asserted-by":"publisher","first-page":"5108","DOI":"10.1093\/bioinformatics\/btac641","article-title":"Predicting colorectal cancer tumor mutational burden from histopathological images and clinical information using multi-modal deep learning","volume":"38","author":"Huang","year":"2022","journal-title":"Bioinformatics"},{"key":"2025111023273188300_ref35","doi-asserted-by":"crossref","first-page":"1613","DOI":"10.1038\/s41467-021-21896-9","article-title":"Human-interpretable image features derived from densely mapped cancer pathology slides predict diverse molecular phenotypes","volume":"12","author":"Diao","year":"2021","journal-title":"Nat Commun"},{"key":"2025111023273188300_ref36","doi-asserted-by":"publisher","first-page":"2251","DOI":"10.21037\/tcr-23-1890","article-title":"Expression of copper metabolism-related genes is associated with the tumor immune microenvironment and predicts the prognosis of hepatocellular carcinoma","volume":"13","author":"Kong","year":"2024","journal-title":"Transl Cancer Res"},{"key":"2025111023273188300_ref37","doi-asserted-by":"publisher","first-page":"6796","DOI":"10.1038\/s41467-023-42504-y","article-title":"Single-cell morphological and topological atlas reveals the ecosystem diversity of human breast cancer","volume":"14","author":"Zhao","year":"2023","journal-title":"Nat Commun"},{"key":"2025111023273188300_ref38","doi-asserted-by":"publisher","first-page":"555","DOI":"10.1038\/s41551-020-00682-w","article-title":"Data-efficient and weakly supervised computational pathology on whole-slide images","volume":"5","author":"Lu","year":"2021","journal-title":"Nat Biomed Eng"},{"key":"2025111023273188300_ref39","doi-asserted-by":"publisher","first-page":"101563","DOI":"10.1016\/j.media.2019.101563","article-title":"Hover-net: simultaneous segmentation and classification of nuclei in multi-tissue histology images","volume":"58","author":"Graham","year":"2019","journal-title":"Med Image Anal"},{"key":"2025111023273188300_ref40","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/D16-1044","article-title":"Multimodal compact bilinear pooling for visual question answering and visual grounding","author":"Fukui","year":"2016;457\u2013468","journal-title":"Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing"},{"key":"2025111023273188300_ref41","article-title":"Attention-based deep multiple instance learning","author":"Ilse","year":"2018","journal-title":"PMLR"},{"key":"2025111023273188300_ref42","article-title":"TransMIL: transformer based correlated multiple instance learning for whole slide image classification","author":"Shao","year":"2021","journal-title":"NeurIPS"},{"key":"2025111023273188300_ref43","article-title":"DTFD-MIL: double-tier feature distillation multiple instance learning for histopathology whole slide image classification","author":"Zhang","year":"2022","journal-title":"CVPR"},{"key":"2025111023273188300_ref44","article-title":"Multiple instance learning framework with masked hard instance Mining for Whole Slide Image Classification","author":"Tang","year":"2023","journal-title":"ICCV"},{"key":"2025111023273188300_ref45","doi-asserted-by":"publisher","first-page":"212106","DOI":"10.1007\/s11432-024-4171-9","article-title":"SBSM-pro: support bio-sequence machine for proteins","volume":"67","author":"Wang","year":"2024","journal-title":"SCIENCE CHINA Inf Sci"},{"key":"2025111023273188300_ref46","doi-asserted-by":"publisher","first-page":"205","DOI":"10.1261\/rna.069112.118","article-title":"Gene2vec: gene subsequence embedding for prediction of mammalian N(6)-methyladenosine sites from mRNA","volume":"25","author":"Zou","year":"2019","journal-title":"RNA"},{"key":"2025111023273188300_ref47","doi-asserted-by":"publisher","first-page":"676","DOI":"10.1016\/j.omtn.2020.07.003","article-title":"An improved anticancer drug-response prediction based on an ensemble method integrating matrix completion and ridge regression","volume":"21","author":"Liu","year":"2020","journal-title":"Mol Ther Nucleic Acids"},{"key":"2025111023273188300_ref48","doi-asserted-by":"publisher","DOI":"10.1093\/bib\/bbab581","article-title":"A weighted bilinear neural collaborative filtering approach for drug repositioning","volume":"23","author":"Meng","year":"2022","journal-title":"Brief Bioinform"},{"key":"2025111023273188300_ref49","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1016\/j.semcancer.2021.07.010","article-title":"Breast cancer heterogeneity through the lens of single-cell analysis and spatial pathologies","volume":"82","author":"Zhao","year":"2022","journal-title":"Semin Cancer Biol"},{"key":"2025111023273188300_ref50","doi-asserted-by":"publisher","first-page":"480","DOI":"10.1200\/CCI.19.00126","article-title":"Deep-learning-based characterization of tumor-infiltrating lymphocytes in breast cancers from histopathology images and multiomics data","volume":"4","author":"Lu","year":"2020","journal-title":"JCO Clin Cancer Inform"}],"container-title":["Briefings in Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/26\/6\/bbaf580\/65266515\/bbaf580.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/26\/6\/bbaf580\/65266515\/bbaf580.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,11]],"date-time":"2025-11-11T04:27:39Z","timestamp":1762835259000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bib\/article\/doi\/10.1093\/bib\/bbaf580\/8320155"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,1]]},"references-count":50,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2025,11,1]]}},"URL":"https:\/\/doi.org\/10.1093\/bib\/bbaf580","relation":{},"ISSN":["1467-5463","1477-4054"],"issn-type":[{"value":"1467-5463","type":"print"},{"value":"1477-4054","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,11]]},"published":{"date-parts":[[2025,11,1]]},"article-number":"bbaf580"}}