{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T11:40:30Z","timestamp":1792755630999,"version":"4.5.0"},"reference-count":62,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,7,18]],"date-time":"2025-07-18T00:00:00Z","timestamp":1752796800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Oncol."],"abstract":"<jats:sec><jats:title>Background<\/jats:title><jats:p>Accurate and automated segmentation of pancreatic tumors from CT images via deep learning is essential for the clinical diagnosis of pancreatic cancer. However, two key challenges persist: (a) complex phenotypic variations in pancreatic morphology cause segmentation models to focus predominantly on healthy tissue over tumors, compromising tumor feature extraction and segmentation accuracy; (b) existing methods often struggle to retain fine-grained local features, leading to performance degradation in pancreas-tumor segmentation.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>To overcome these limitations, we propose SMF-Net (Semantic-Guided Multimodal Fusion Network), a novel multimodal medical image segmentation framework integrating a CNN-Transformer hybrid encoder. The framework incorporates AMBERT, a progressive feature extraction module, and the Multimodal Token Transformer (MTT) to fuse visual and semantic features for enhanced tumor localization. Additionally, The Multimodal Enhanced Attention Module (MEAM) further improves the retention of local discriminative features. To address multimodal data scarcity, we adopt a semi-supervised learning paradigm based on a Dual-Adversarial-Student Network (DAS-Net). Furthermore, in collaboration with Zhuzhou Central Hospital, we constructed the Multimodal Pancreatic Tumor Dataset (MPTD).<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>The experimental results on the MPTD indicate that our model achieved Dice scores of 79.25% and 64.21% for pancreas and tumor segmentation, respectively, showing improvements of 2.24% and 4.18% over the original model. Furthermore, the model outperformed existing state-of-the-art methods on the QaTa-COVID-19 and MosMedData lung infection segmentation datasets in terms of average Dice scores, demonstrating its strong generalization ability.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p>The experimental results demonstrate that SMF-Net delivers accurate segmentation of both pancreatic, tumor and pulmonary regions, highlighting its strong potential for real-world clinical applications.<\/jats:p><\/jats:sec>","DOI":"10.3389\/fonc.2025.1622426","type":"journal-article","created":{"date-parts":[[2025,7,18]],"date-time":"2025-07-18T04:11:12Z","timestamp":1752811872000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["SMF-net: semantic-guided multimodal fusion network for precise pancreatic tumor segmentation in medical CT image"],"prefix":"10.3389","volume":"15","author":[{"given":"Wenyi","family":"Zhou","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ziyang","family":"Shi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bin","family":"Xie","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fang","family":"Li","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiehao","family":"Yin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yongzhong","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Linan","family":"Hu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lin","family":"Li","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yongming","family":"Yan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiajun","family":"Wei","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhen","family":"Hu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhengmao","family":"Luo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wanxiang","family":"Peng","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaochun","family":"Xie","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaoli","family":"Long","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2025,7,18]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2021.4708","article-title":"Estimated projection of US cancer incidence and death to 2040","volume":"4","author":"Rahib","year":"2021","journal-title":"JAMA Network Open"},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.1002\/cncr.34479","article-title":"Annual report to the nation on the status of cancer, part 1: National cancer statistics","volume":"128","author":"Cronin","year":"2022","journal-title":"Cancer."},{"key":"B3","doi-asserted-by":"publisher","first-page":"17","DOI":"10.3322\/caac.21763","article-title":"Cancer statistics, 2023","volume":"73","author":"Siegel","year":"2023","journal-title":"CA Cancer J Clin"},{"key":"B4","doi-asserted-by":"crossref","DOI":"10.1109\/WACV57701.2024.00047","article-title":"MIST: Medical image segmentation transformer with convolutional attention mixing (CAM) decoder","volume-title":"2024 IEEE\/CVF winter conference on applications of computer vision (WACV)","author":"Rahman","year":"2024"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2023.11.006","article-title":"A survey on cancer detection via convolutional neural networks: current challenges and future directions","volume":"169","author":"Sharma","year":"2023","journal-title":"Neural Networks."},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.1145\/3065386","article-title":"ImageNet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky","year":"2012","journal-title":"Neural Inf Process Systems."},{"key":"B7","first-page":"1556","article-title":"Very deep convolutional networks for large-scale image recognition","volume":"1409","author":"Simonyan","year":"2014","journal-title":"arXiv preprint arXiv"},{"key":"B8","first-page":"1","article-title":"Going deeper with convolutions","volume-title":"Proceedings of the IEEE conference on computer vision and pattern recognition","author":"Szegedy","year":"2015"},{"key":"B9","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-46493-0_38","article-title":"Identity mappings in deep residual networks","volume-title":"Computer vision\u2013ECCV 2016: 14th european conference, amsterdam, the Netherlands, october 11\u201314, 2016, proceedings, part IV","author":"He","year":"2016"},{"key":"B10","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR.2017.243","article-title":"Densely connected convolutional networks","volume-title":"Proceedings of the IEEE conference on computer vision and pattern recognition","author":"Huang","year":"2017"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2016.2572683","article-title":"Fully convolutional networks for semantic segmentation","volume":"39","author":"Shelhamer","year":"2016","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"B12","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-24574-4_28","article-title":". U-net: convolutional networks for biomedical image segmentation","volume-title":"Lecture notes in computer science. (LNCS 9351)","author":"Ronneberger","year":"2015"},{"key":"B13","article-title":"Learning transferable visual models from natural language supervision","volume-title":"International conference on machine learning (ICML)","author":"Radford","year":"2021"},{"key":"B14","article-title":"ViLT: vision-and-language transformer without convolution or region supervision","volume-title":"Proceedings of the 38th international conference on machine learning (ICML)","author":"Kim","year":"2021"},{"key":"B15","first-page":"1","article-title":"Coarse-to-fine COVID-19 segmentation via vision-language alignment","volume-title":"2022 IEEE international conference on acoustics, speech and signal processing (ICASSP)","author":"Shan","year":"2022"},{"key":"B16","doi-asserted-by":"publisher","first-page":"96","DOI":"10.1109\/TMI.2023.3291719","article-title":"LVIT: language meets vision transformer in medical image segmentation","volume":"43","author":"Li","year":"2023","journal-title":"IEEE Trans Med Imaging."},{"key":"B17","article-title":"Multi-granularity cross-modal alignment for generalized medical visual representation learning","volume":"35","author":"Wang","year":"2022","journal-title":"Adv Neural Inf Process Systems."},{"key":"B18","doi-asserted-by":"crossref","DOI":"10.1109\/WACV61041.2025.00314","article-title":"Personalized mixture of experts for multi-site medical image segmentation","volume-title":"2025 IEEE\/CVF winter conference on applications of computer vision (WACV).","author":"Rahman","year":"2025"},{"key":"B19","doi-asserted-by":"crossref","DOI":"10.1109\/JBHI.2025.3569491","article-title":"Text-assisted vision model for medical image segmentation","volume-title":"IEEE J biomed health inform","author":"Rahman","year":"2025"},{"key":"B20","article-title":"CDDFUSE: correlation-driven dual-branch feature decomposition for multi-modality image fusion","volume-title":"2022 IEEE\/CVF conference on computer vision and pattern recognition (CVPR)","author":"Zhao","year":"2022"},{"key":"B21","first-page":"2","article-title":"Contrastive learning of medical visual representations from paired images and text","volume-title":"Machine learning for healthcare conference (MLHC)","author":"Zhang","year":"2022"},{"key":"B22","first-page":"1","article-title":"Survey of research on multimodal fusion technology for deep learning","volume":"46","author":"He","year":"2020","journal-title":"Comput Engineering."},{"key":"B23","doi-asserted-by":"crossref","DOI":"10.1609\/aaai.v36i2.20098","article-title":"Label-efficient hybrid-supervised learning for medical image segmentation","volume-title":"Proceedings of the AAAI conference on artificial intelligence","author":"Pan","year":"2022"},{"key":"B24","doi-asserted-by":"publisher","first-page":"387","DOI":"10.1016\/j.neunet.2022.10.026","article-title":"Temperature guided network for 3D joint segmentation of the pancreas and tumors","volume":"157","author":"Li","year":"2023","journal-title":"Neural Networks."},{"key":"B25","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2021.findings-acl.37","article-title":"Ambert: a pre-trained language model with multi-grained tokenization","volume-title":"Findings of the association for computational linguistics: ACL-IJCNLP 2021.","author":"Zhang","year":"2021"},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.1007\/s43657-023-00152-8","article-title":"A multimodal approach for detection and assessment of depression using text, audio and video","volume":"4","author":"Zhang","year":"2024","journal-title":"Phenomics."},{"key":"B27","doi-asserted-by":"crossref","DOI":"10.1145\/2512530.2512532","article-title":"Depression recognition based on dynamic facial and vocal expression features using partial least square regression","volume-title":"Proceedings of the 3rd ACM international workshop on audio\/visual emotion challenge.","author":"Meng","year":"2013"},{"key":"B28","doi-asserted-by":"crossref","first-page":"13","DOI":"10.18653\/v1\/W18-0602","article-title":"A linguistically-informed fusion approach for multimodal depression detection","volume-title":"Proceedings of the fifth workshop on computational linguistics and clinical psychology: from keyboard to clinic","author":"Morales","year":"2018"},{"key":"B29","article-title":"DeeP-HOSEQ: Deep higher order sequence fusion for multimodal sentiment analysis","volume-title":"2021 IEEE international conference on data mining (ICDM)","author":"Verma","year":"2021"},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.3969\/j.issn.1005-202X.2023.06.004","article-title":"Multimodal fusion of PET-CT for semantic image segmentation: a review","volume":"40","author":"Zhou","year":"2023","journal-title":"Chin J Med Physics."},{"key":"B31","first-page":"0473","article-title":"Neural machine translation by jointly learning to align and translate","volume":"1409","author":"Bahdanau","year":"2014","journal-title":"arXiv preprint arXiv"},{"key":"B32","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR.2017.683","article-title":"Residual attention network for image classification","volume-title":"Proceedings of the IEEE conference on computer vision and pattern recognition","author":"Wang","year":"2017"},{"key":"B33","doi-asserted-by":"publisher","DOI":"10.4855\/arXiv.1706.03762","article-title":"Attention is all you need","volume":"30","author":"Vaswani","year":"2017","journal-title":"Adv Neural Inf Process systems."},{"key":"B34","article-title":"Segment anything","volume-title":"2021 IEEE\/CVF international conference on computer vision (ICCV)","author":"Kirillov","year":"2021"},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2022.118625","article-title":"Multi-organ segmentation network for abdominal CT images based on spatial attention and deformable convolution","volume":"211","author":"Shen","year":"2022","journal-title":"Expert Syst Appl"},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2024.123586","article-title":"Few-shot learning based on prototype rectification with a self-attention mechanism","volume":"249","author":"Zhao","year":"2024","journal-title":"Expert Syst Appl"},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.1109\/JBHI.2021.3129245","article-title":"AGMB-Transformer: Anatomy-guided multi-branch transformer network for automated evaluation of root canal therapy","volume":"26","author":"Li","year":"2021","journal-title":"IEEE J BioMed Health Inform."},{"key":"B38","doi-asserted-by":"publisher","DOI":"10.1109\/TCBB.2020.2973971","article-title":"A deep segmentation network of multi-scale feature fusion based on attention mechanism for IVOCT lumen contour","volume":"18","author":"Huang","year":"2020","journal-title":"IEEE\/ACM Trans Comput Biol Bioinform"},{"key":"B39","doi-asserted-by":"publisher","DOI":"10.1007\/s11548-022-02738-5","article-title":"Multi-scale feature pyramid fusion network for medical image segmentation","volume":"18","author":"Zhang","year":"2022","journal-title":"Int J Comput Assist Radiol Surg"},{"key":"B40","doi-asserted-by":"publisher","DOI":"10.1007\/s10278-024-01051-8","article-title":"ConTEXTual Net: A multimodal Vision-Language model for segmentation of pneumothorax","volume":"37","author":"Huemann","year":"2024","journal-title":"Deleted J"},{"key":"B41","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2023.107596","article-title":"SUnet: A multi-organ segmentation network based on multiple attention","volume":"167","author":"Li","year":"2023","journal-title":"Comput Biol Med"},{"key":"B42","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2023.102017","article-title":"IIFDD: Intra and inter-modal fusion for depression detection with multi-modal information from Internet of Medical Things","volume":"102","author":"Chen","year":"2023","journal-title":"Inf Fusion."},{"key":"B43","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2023.123052","article-title":"Deep semi-supervised learning for medical image segmentation: A review","volume":"245","author":"Han","year":"2024","journal-title":"Expert Syst Appl"},{"key":"B44","doi-asserted-by":"publisher","first-page":"90","DOI":"10.1016\/j.neunet.2021.10.008","article-title":"Interpolation consistency training for semi-supervised learning","volume":"145","author":"Verma","year":"2022","journal-title":"Neural Networks."},{"key":"B45","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2024.103111","article-title":"Mutual learning with reliable pseudo label for semi-supervised medical image segmentation","volume":"94","author":"Su","year":"2024","journal-title":"Med Image Anal"},{"key":"B46","doi-asserted-by":"crossref","DOI":"10.24963\/ijcai.2023\/467","article-title":"Co-training with high-confidence pseudo labels for semi-supervised medical image segmentation","volume-title":"International joint conference on artificial intelligence","author":"Shen","year":"2023"},{"key":"B47","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2023.102792","article-title":"Local contrastive loss with pseudo-label based self-training for semi-supervised medical image segmentation","volume":"87","author":"Chaitanya","year":"2023","journal-title":"Med Image Anal"},{"key":"B48","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2023.122567","article-title":"Combining contrastive learning and shape awareness for semi-supervised medical image segmentation","volume":"242","author":"Chen","year":"2023","journal-title":"Expert Syst Appl"},{"key":"B49","doi-asserted-by":"crossref","DOI":"10.1109\/ICCV.2019.00683","article-title":"Dual student: Breaking the limits of the teacher in semi-supervised learning","volume-title":"Proceedings of the IEEE\/CVF international conference on computer vision","author":"Ke","year":"2019"},{"key":"B50","doi-asserted-by":"publisher","first-page":"4128","DOI":"10.1038\/s41467-022-31953-8","article-title":"The medical segmentation decathlon","volume":"13","author":"Antonelli","year":"2022","journal-title":"Nat Commun"},{"key":"B51","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2020.101829","article-title":"Statistical deformation reconstruction using multi-organ shape features for pancreatic cancer localization","volume":"67","author":"Nakao","year":"2021","journal-title":"Med Image Anal"},{"key":"B52","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2024.124270","article-title":"Transformer based multiple superpixel-instance learning for weakly supervised segmenting lesions of interstitial lung disease","volume":"253","author":"Lai","year":"2024","journal-title":"Expert Syst Appl"},{"key":"B53","first-page":"11929","article-title":"An image is worth 16x16 words: Transformers for image recognition at scale","volume":"2010","author":"Dosovitskiy","year":"2020","journal-title":"arXiv preprint arXiv"},{"key":"B54","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1007\/978-3-030-01234-2_1","article-title":"CBAM: Convolutional block attention module","author":"Woo","year":"2018","journal-title":"In: Lecture Notes Comput Science. (LNCS"},{"key":"B55","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-031-15937-4_65","article-title":"TFCNS: A CNN-Transformer hybrid network for medical image segmentation","volume-title":"Lecture notes in computer science. (LNCS 14349)","author":"Li","year":"2022"},{"key":"B56","doi-asserted-by":"publisher","DOI":"10.12290\/xhyxzz.2023-0409","article-title":"Research progress on generative adversarial network in cross-modal medical image reconstruction","volume":"14","author":"Li","year":"2023","journal-title":"Med J Peking Union Med Coll Hosp."},{"key":"B57","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2023.107234","article-title":"Joint learning-based feature reconstruction and enhanced network for incomplete multi-modal brain tumor segmentation","volume":"163","author":"Diao","year":"2023","journal-title":"Comput Biol Med"},{"key":"B58","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2022.3217852","article-title":"VLT: Vision-Language Transformer and query generation for referring segmentation","volume":"45","author":"Ding","year":"2022","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"B59","article-title":"Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results","volume":"30","author":"Tarvainen","year":"2017","journal-title":"Adv Neural Inf Process Syst (NeurIPS)"},{"key":"B60","article-title":"BERT: Pre-training of deep bidirectional transformers for language understanding","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Devlin","year":"2019"},{"key":"B61","doi-asserted-by":"publisher","DOI":"10.1109\/icip46576.2022.9897412","article-title":"OSEGNET: operational segmentation network for COVID-19 detection using chest X-ray images","author":"Degerli","year":"2022","journal-title":"IEEE Int Conf Image Process (ICIP)."},{"key":"B62","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2005.06465","article-title":"Mosmeddata: Chest CT scans with COVID-19 related findings dataset","volume":"2005","author":"Morozov","year":"2020","journal-title":"arXiv preprint arXiv"}],"container-title":["Frontiers in Oncology"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fonc.2025.1622426\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,18]],"date-time":"2025-07-18T04:11:21Z","timestamp":1752811881000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fonc.2025.1622426\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,18]]},"references-count":62,"alternative-id":["10.3389\/fonc.2025.1622426"],"URL":"https:\/\/doi.org\/10.3389\/fonc.2025.1622426","relation":{},"ISSN":["2234-943X"],"issn-type":[{"value":"2234-943X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,18]]},"article-number":"1622426"}}