{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,3]],"date-time":"2026-09-03T17:06:57Z","timestamp":1788455217920,"version":"build-2803163510"},"reference-count":32,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Med Inform"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec sec-type=\"background\">\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Early diagnosis and intervention in glottic carcinoma (GC) can significantly improve long-term prognosis. However, the accurate diagnosis of early GC is challenging due to its morphological similarity to vocal cord dysplasia, with the difficulty further exacerbated in medically underserved areas.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"objective\">\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>This study aims to address the limitations of existing technologies by designing a vision-language multimodal model, providing a more efficient and accurate early diagnostic method for GC.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"methods\">\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>\n                      The data used in this study were sourced from the information system of the First Affiliated Hospital of Sun Yat-sen University, comprising laryngoscopy reports and 5796 laryngoscopic images from 404 patients with glottic lesions. We propose a vision-language\u2013guided multimodal fusion network (VLMF-Net) based on a large vision-language model for the early automated diagnosis of GC. The text processing module of this model uses the pretrained Large Language Model Meta AI (LLaMa) to generate text vector representations, while the image processing module uses a pretrained vision transformer to extract features from laryngoscopic images, achieving cross-modal alignment through the Q-Former module. By leveraging a feature fusion module, deep integration of text and image features is achieved, ultimately enabling classification diagnosis. To validate the model\u2019s performance, the study selected contrastive language-image pretraining (CLIP), bootstrapping language-image pretraining with frozen image encoders and large language models (BLIP-2), a large-scale image and noisy-text embedding (ALIGN), and vision-and-language transformer (VILT) as baseline methods for experimental evaluation on the same dataset, with comprehensive performance assessment conducted using accuracy, recall, precision,\n                      <jats:italic>F<\/jats:italic>\n                      <jats:sub>1<\/jats:sub>\n                      -score, and area under the curve.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"results\">\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>We found that on the internal test set, the VLMF-Net model significantly outperformed existing methods with an accuracy of 77.6% (CLIP: 70.5%; BLIP-2: 71.5%; ALIGN: 67.3%; and VILT: 64.3%), achieving a 6.1-percentage point improvement over the best baseline model (BLIP-2). On the external test set, our method also demonstrated robust performance, achieving an accuracy of 73.9%, which is 4.6 percentage points higher than the second-best model (BLIP-2: 69.3%). This indicates that our model surpasses these methods in the early diagnosis of GC and exhibits strong generalization ability and robustness.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"conclusions\">\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>The proposed VLMF-Net model can be effectively used for the early diagnosis of GC, helping to address the challenges in its early detection.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.2196\/74902","type":"journal-article","created":{"date-parts":[[2025,10,8]],"date-time":"2025-10-08T19:11:39Z","timestamp":1759950699000},"page":"e74902-e74902","source":"Crossref","is-referenced-by-count":2,"title":["A Vision-Language\u2013Guided Multimodal Fusion Network for Glottic Carcinoma Early Diagnosis: Model Development and Validation Study"],"prefix":"10.2196","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-2291-3228","authenticated-orcid":false,"given":"Zhaohui","family":"Jin","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-0496-396X","authenticated-orcid":false,"given":"Yi","family":"Shuai","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0935-9274","authenticated-orcid":false,"given":"Yun","family":"Li","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-0328-6696","authenticated-orcid":false,"given":"Mianmian","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-0888-6575","authenticated-orcid":false,"given":"Yumeng","family":"Liu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9720-1997","authenticated-orcid":false,"given":"Wenbin","family":"Lei","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8160-1294","authenticated-orcid":false,"given":"Xiaomao","family":"Fan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2025,10,8]]},"reference":[{"issue":"1","key":"R1","doi-asserted-by":"publisher","first-page":"31","DOI":"10.3322\/caac.21386","article-title":"An update on larynx cancer","volume":"67","author":"Steuer","journal-title":"CA Cancer J Clin"},{"issue":"3","key":"R2","doi-asserted-by":"publisher","first-page":"229","DOI":"10.3322\/caac.21834","article-title":"Global cancer statistics 2022: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries","volume":"74","author":"Bray","journal-title":"CA Cancer J Clin"},{"issue":"22","key":"R3","doi-asserted-by":"publisher","first-page":"5047","DOI":"10.1002\/cncr.26130","article-title":"Patterns of extralaryngeal spread of laryngeal cancer: thyroid cartilage penetration occurs in a minority of patients with extralaryngeal spread of laryngeal squamous cell cancers","volume":"117","author":"Chen","journal-title":"Cancer"},{"issue":"2","key":"R4","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1001\/jamaoto.2014.2998","article-title":"Improvements in survival and disparities for advanced-stage laryngeal cancer","volume":"141","author":"LeBlanc","journal-title":"JAMA Otolaryngol Head Neck Surg"},{"key":"R5","doi-asserted-by":"publisher","DOI":"10.1002\/hed.24300","article-title":"Narrow band imaging for the diagnosis of head and neck squamous cell carcinoma: a systematic review","volume":"38 Suppl 1","author":"Cosway","journal-title":"Head Neck"},{"issue":"12","key":"R6","doi-asserted-by":"publisher","first-page":"3129","DOI":"10.1002\/hed.27543","article-title":"Vocal cord leukoplakia classification using deep learning models in white light and narrow band imaging endoscopy images","volume":"45","author":"You","journal-title":"Head Neck"},{"issue":"1","key":"R7","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1002\/hed.23838","article-title":"Value of narrow band imaging in the early diagnosis of laryngeal cancer","volume":"38","author":"Kraft","journal-title":"Head Neck"},{"issue":"1","key":"R8","doi-asserted-by":"publisher","first-page":"432","DOI":"10.1002\/mp.15371","article-title":"Vocal cord lesions classification based on deep convolutional neural network and transfer learning","volume":"49","author":"Zhao","journal-title":"Med Phys"},{"issue":"S1","key":"R9","doi-asserted-by":"publisher","first-page":"E2284","DOI":"10.1002\/hed.24185","article-title":"Grade of dysplasia and malignant transformation in adults with premalignant laryngeal lesions","volume":"38 Suppl 1","author":"van Hulst","journal-title":"Head Neck"},{"issue":"2","key":"R10","doi-asserted-by":"publisher","first-page":"429","DOI":"10.1002\/lary.27346","article-title":"Diagnosis of vocal cord leukoplakia: the role of a novel narrow band imaging endoscopic classification","volume":"129","author":"Ni","journal-title":"Laryngoscope"},{"key":"R11","doi-asserted-by":"publisher","DOI":"10.1016\/j.critrevonc.2016.07.004","article-title":"Laryngeal preneoplastic lesions and cancer: challenging diagnosis. Qualitative literature review and meta-analysis","volume":"106","author":"Mannelli","journal-title":"Crit Rev Oncol Hematol"},{"key":"R12","doi-asserted-by":"publisher","DOI":"10.2196\/66365","article-title":"Identifying kidney stone risk factors through patient experiences with a large language model: text analysis and empirical study","volume":"27","author":"Mao","journal-title":"J Med Internet Res"},{"issue":"22","key":"R13","doi-asserted-by":"publisher","DOI":"10.3390\/s22228834","article-title":"An improvised deep-learning-based mask R-CNN model for laryngeal cancer detection using CT images","volume":"22","author":"Sahoo","journal-title":"Sensors (Basel)"},{"issue":"3","key":"R14","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1177\/01945998221110839","article-title":"Artificial intelligence and laryngeal cancer: from screening to prognosis: a state of the art review","volume":"168","author":"Bensoussan","journal-title":"Otolaryngol Head Neck Surg"},{"key":"R15","doi-asserted-by":"publisher","DOI":"10.1016\/j.bspc.2021.103443","article-title":"Diagnosis of ulcerative colitis from endoscopic images based on deep learning","volume":"73","author":"Luo","journal-title":"Biomed Signal Process Control"},{"issue":"1","key":"R16","doi-asserted-by":"publisher","first-page":"48","DOI":"10.1109\/TIV.2022.3164899","article-title":"MTANet: multitask-aware network with hierarchical multimodal fusion for RGB-T urban scene understanding","volume":"8","author":"Zhou","journal-title":"IEEE Trans Intell Veh"},{"key":"R17","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2023.102832","article-title":"DLGNet: a dual-branch lesion-aware network with the supervised gaussian mixture model for colon lesions classification in colonoscopy images","volume":"87","author":"Wang","journal-title":"Med Image Anal"},{"issue":"1","key":"R18","doi-asserted-by":"publisher","first-page":"1358","DOI":"10.1109\/TIV.2023.3329708","article-title":"REDFormer: radar enlightens the darkness of camera perception with transformers","volume":"9","author":"Cui","journal-title":"IEEE Trans Intell Veh"},{"key":"R19","doi-asserted-by":"crossref","unstructured":"Wei J Li Y Qiu M Chen H Fan X Lei W . SAM-FNet: SAM-guided fusion network for laryngo-pharyngeal tumor detection. Presented at: 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM); Dec 3-6, 2024; Lisbon, Portugal. [doi: 10.1109\/BIBM62325.2024.10822832]","DOI":"10.1109\/BIBM62325.2024.10822832"},{"issue":"5","key":"R20","doi-asserted-by":"publisher","DOI":"10.2196\/28868","article-title":"A multimodal imaging-based deep learning model for detecting treatment-requiring retinal vascular diseases: model development and validation study","volume":"9","author":"Kang","journal-title":"JMIR Med Inform"},{"issue":"1","key":"R21","doi-asserted-by":"publisher","DOI":"10.2196\/58342","article-title":"Feasibility of multimodal artificial intelligence using GPT-4 vision for the classification of middle ear disease: qualitative study and validation","volume":"3","author":"Noda","journal-title":"JMIR AI"},{"issue":"10s","key":"R22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3505244","article-title":"Transformers in vision: a survey","volume":"54","author":"Khan","journal-title":"ACM Comput Surv"},{"key":"R23","unstructured":"Grattafiori A Dubey A Jauhri A Pandey A Kadian A Al-Dahle A et al. The Llama 3 herd of models. arXiv. Preprint posted online on Jul 31, 2024. [doi: 10.48550\/arXiv.2407.21783]"},{"key":"R24","unstructured":"Cui Y Yang Z Yao X . Efficient and effective text encoding for Chinese LLaMA and Alpaca. arXiv. Preprint posted online on Apr 17, 2023. [doi: 10.48550\/arXiv.2304.08177]"},{"key":"R25","unstructured":"Li J Li D Savarese S Hoi S . BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models. arXiv. Preprint posted online on Jan 23, 2023. [doi: 10.48550\/arXiv.2301.12597]"},{"key":"R26","unstructured":"Lee C Jang J Lee J . Personalizing text-to-image generation with visual prompts using BLIP-2. Presented at: Eleventh International Conference on Learning Representations (ICLR 2023); May 1-5, 2023; Kigali, Rwanda. URL: https:\/\/openreview.net\/pdf?id=7wxCMgdj5F [Accessed 17-09-2025]"},{"key":"R27","unstructured":"Nguyen T Gadre SY Ilharco G Oh S Schmidt L . Improving multimodal datasets with image captioning. arXiv. Preprint posted online on Jul 19, 2023. [doi: 10.48550\/arXiv.2307.10350]"},{"key":"R28","unstructured":"Zhu D Chen J Haydarov K Shen X Zhang W Elhoseiny M . ChatGPT asks, BLIP-2 answers: automatic questioning towards enriched visual descriptions. arXiv. Preprint posted online on Mar 12, 2023. [doi: 10.48550\/arXiv.2303.06594]"},{"key":"R29","unstructured":"Kim W Son B Kim I . ViLT: vision-and-language transformer without convolution or region supervision. arXiv. Preprint posted online on Feb 5, 2021. [doi: 10.48550\/arXiv.2102.03334]"},{"key":"R30","unstructured":"Radford A Kim JW Hallacy C Ramesh A Goh G Agarwal S et al. Learning transferable visual models from natural language supervision. Presented at: 38th International Conference on Machine Learning (ICML 2021); Jul 18-24, 2021. URL: https:\/\/proceedings.mlr.press\/v139\/radford21a\/radford21a.pdf [Accessed 17-09-2025]"},{"key":"R31","unstructured":"Jia C Yang Y Xia Y et al. Scaling up visual and vision-language representation learning with noisy text supervision. arXiv. Preprint posted online on Feb 11, 2021. [doi: 10.48550\/arXiv.2102.05918]"},{"key":"R32","doi-asserted-by":"crossref","unstructured":"Selvaraju RR Cogswell M Das A Vedantam R Parikh D Batra D . Grad-CAM: visual explanations from deep networks via gradient-based localization. Presented at: International Conference on Computer Vision (ICCV); Oct 22-29, 2017; Venice, Italy. [doi: 10.1007\/s11263-019-01228-7]","DOI":"10.1109\/ICCV.2017.74"}],"container-title":["JMIR Medical Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/medinform.jmir.org\/2025\/1\/e74902","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T18:05:40Z","timestamp":1782842740000},"score":1,"resource":{"primary":{"URL":"https:\/\/medinform.jmir.org\/2025\/1\/e74902"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,8]]},"references-count":32,"URL":"https:\/\/doi.org\/10.2196\/74902","relation":{},"ISSN":["2291-9694"],"issn-type":[{"value":"2291-9694","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,8]]},"article-number":"v13i1e74902"}}