{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T19:04:35Z","timestamp":1782846275691,"version":"3.54.5"},"reference-count":28,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR AI"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Accurate tumor node metastasis (TNM) staging is fundamental for treatment planning and prognosis in non-small cell lung cancer (NSCLC). However, its complexity poses significant challenges. Traditional rule-based natural language processing methods are constrained by their reliance on manually crafted rules and are susceptible to inconsistencies in clinical reporting.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study aimed to develop and validate a robust, accurate, and operationally efficient artificial intelligence framework for the TNM staging of NSCLC by strategically enhancing a large language model, GLM-4-Air (general language model), through advanced prompt engineering and supervised fine-tuning (SFT).<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n We constructed a curated dataset of 492 deidentified real-world medical imaging reports, with TNM staging annotations rigorously validated by senior physicians according to the AJCC (American Joint Committee on Cancer) 8th edition guidelines. The GLM-4-Air model was systematically optimized via a multi-phase process: iterative prompt engineering incorporating chain-of-thought reasoning and domain knowledge injection for all staging tasks, followed by parameter-efficient SFT using low-rank adaptation for the reasoning-intensive primary tumor characteristics (T) and regional lymph node involvement (N) staging tasks. The final hybrid model was evaluated on a completely held-out test set (black-box) and benchmarked against GPT-4o using standard metrics, statistical tests, and a clinical impact analysis of staging errors.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n The optimized hybrid GLM-4-Air model demonstrated reliable performance. It achieved higher staging accuracies on the black-box test set: 92% (95% CI 0.850\u20100.959) for T, 86% (95% CI 0.779\u20100.915) for N, 92% (95% CI 0.850\u20100.959) for distant metastasis status (M), and 90% for overall clinical staging; by comparison, GPT-4o attained 87% (95% CI 0.790\u20100.922), 70% (95% CI 0.604\u20100.781), 78% (95% CI 0.689\u20100.850), and 80%, respectively. The model\u2019s robustness was further evidenced by its macro-average\n F<\/jats:italic>\n 1<\/jats:sub>\n -scores of 0.914 (T), 0.815 (N), and 0.831 (M), consistently surpassing those of GPT-4o (0.836, 0.620, and 0.698). Analysis of confusion matrices confirmed the model\u2019s proficiency in identifying critical staging features while effectively minimizing false negatives. Crucially, the clinical impact assessment showed a substantial reduction in severe category I errors, which are defined as misclassifications that could significantly influence subsequent clinical decisions. Our model committed 0 category I errors in M staging and fewer category I errors in T and N staging. Furthermore, the framework demonstrated practical deployability, achieving efficient inference on consumer-grade hardware (eg, 4 RTX 4090 GPUs) with latencies suitable and acceptable for clinical workflows.\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n The proposed hybrid framework, integrating structured prompt engineering and applying SFT to reasoning-heavy tasks (T\/N), enables the GLM-4-Air model to serve as a highly accurate, clinically reliable, and cost-efficient solution for automated NSCLC TNM staging. This work demonstrates the efficacy and potential of a domain-optimized smaller model compared with an off-the-shelf generalist model, holding promise for enhancing diagnostic standardization in resource-aware health care environments.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/77988","type":"journal-article","created":{"date-parts":[[2026,1,29]],"date-time":"2026-01-29T13:50:08Z","timestamp":1769694608000},"page":"e77988-e77988","source":"Crossref","is-referenced-by-count":0,"title":["Augmenting Large Language Model With Prompt Engineering and Supervised Fine-Tuning in Non-Small Cell Lung Cancer Tumor-Node-Metastasis Staging: Framework Development and Validation"],"prefix":"10.2196","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-5567-5398","authenticated-orcid":false,"given":"Ruonan","family":"Jin","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-3578-8862","authenticated-orcid":false,"given":"Chao","family":"Ling","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-3488-4121","authenticated-orcid":false,"given":"Yixuan","family":"Hou","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-0044-5532","authenticated-orcid":false,"given":"Yuhan","family":"Sun","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-6640-6585","authenticated-orcid":false,"given":"Ning","family":"Li","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4686-2530","authenticated-orcid":false,"given":"Jiefei","family":"Han","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1090-6426","authenticated-orcid":false,"given":"Jin","family":"Sheng","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-3297-434X","authenticated-orcid":false,"given":"Qizhao","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-5932-0826","authenticated-orcid":false,"given":"Yuepeng","family":"Liu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-0822-2976","authenticated-orcid":false,"given":"Shen","family":"Zheng","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-3526-3035","authenticated-orcid":false,"given":"Xingyu","family":"Ren","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-3767-7158","authenticated-orcid":false,"given":"Chiyu","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-3342-5736","authenticated-orcid":false,"given":"Jue","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-9681-7652","authenticated-orcid":false,"given":"Cheng","family":"Li","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,4,15]]},"reference":[{"issue":"1","key":"R1","doi-asserted-by":"publisher","first-page":"47","DOI":"10.1016\/j.jncc.2024.01.006","article-title":"Cancer incidence and mortality in China, 2022","volume":"4","author":"Han","journal-title":"J Natl Cancer Cent"},{"issue":"3","key":"R2","doi-asserted-by":"publisher","first-page":"277","DOI":"10.1007\/s13258-016-0492-5","article-title":"Transcriptome analysis of non-small cell lung cancer and genetically matched adjacent normal tissues identifies novel prognostic marker genes","volume":"39","author":"Bang","journal-title":"Genes Genom"},{"issue":"1","key":"R3","doi-asserted-by":"publisher","first-page":"50","DOI":"10.3390\/jor4010005","article-title":"Lung cancer staging\u2014a clinical practice review","volume":"4","author":"Rueschhoff","journal-title":"JoR"},{"issue":"2","key":"R4","doi-asserted-by":"publisher","first-page":"93","DOI":"10.3322\/caac.21388","article-title":"The eighth edition AJCC cancer staging manual: continuing to build a bridge from a population-based to a more \u201cpersonalized\u201d approach to cancer staging","volume":"67","author":"Amin","journal-title":"CA Cancer J Clin"},{"issue":"2","key":"R5","doi-asserted-by":"publisher","first-page":"399","DOI":"10.12290\/xhyxzz.2024-0265","article-title":"Quality control and analysis of treatment for hospitalized cancer patients: an interview and medical records study from nine hospitals in Beijing","volume":"16","author":"Lu","journal-title":"Med J Peking Union Med Coll Hosp"},{"key":"R6","doi-asserted-by":"publisher","DOI":"10.2196\/38125","article-title":"How natural language processing can aid with pulmonary oncology tumor node metastasis staging from free-text radiology reports: algorithm development and validation","volume":"7","author":"Puts","journal-title":"JMIR Form Res"},{"issue":"1","key":"R7","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1007\/s10278-023-00913-x","article-title":"Natural language processing algorithm used for staging pulmonary oncology from free-text radiological reports: \u201cIncluding PET-CT and Validation Towards Clinical Use\u201d","volume":"37","author":"Nobel","journal-title":"J Imaging Inform Med"},{"issue":"1","key":"R8","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-022-01975-7","article-title":"Automated extraction of information of lung cancer staging from unstructured reports of PET-CT interpretation: natural language processing with deep-learning","volume":"22","author":"Park","journal-title":"BMC Med Inform Decis Mak"},{"issue":"1","key":"R9","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1186\/s13244-021-01018-1","article-title":"T-staging pulmonary oncology from radiological reports using natural language processing: translating into a multi-language setting","volume":"12","author":"Nobel","journal-title":"Insights Imaging"},{"issue":"Suppl 2","key":"R10","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-021-01575-x","article-title":"A novel deep learning approach to extract Chinese clinical entities for lung cancer screening and staging","volume":"21","author":"Zhang","journal-title":"BMC Med Inform Decis Mak"},{"key":"R11","unstructured":"Goel MA . LLMs accelerate annotation for medical information extraction. arXiv. Preprint posted online on Dec 4, 2023. [doi: 10.48550\/arXiv.2312.02296]"},{"issue":"3","key":"R12","doi-asserted-by":"publisher","first-page":"310","DOI":"10.1148\/radiol.231362","article-title":"Potential of ChatGPT and GPT-4 for data mining of free-text CT reports on lung cancer","volume":"308","author":"Fink","journal-title":"Radiology"},{"issue":"3","key":"R13","doi-asserted-by":"publisher","first-page":"310","DOI":"10.1002\/path.6232","article-title":"Extracting structured information from unstructured histopathology reports using generative pre-trained transformer 4 (GPT-4)","volume":"262","author":"Truhn","journal-title":"J Pathol"},{"key":"R14","doi-asserted-by":"publisher","DOI":"10.2196\/78432","article-title":"Medical feature extraction from clinical examination notes: development and evaluation of a two-phase large language model framework","volume":"13","author":"Abumelha","journal-title":"JMIR Med Inform"},{"key":"R15","doi-asserted-by":"publisher","DOI":"10.2196\/70096","article-title":"Extracting multifaceted characteristics of patients with chronic disease comorbidity: framework development using large language models","volume":"13","author":"Zhang","journal-title":"JMIR Med Inform"},{"key":"R16","doi-asserted-by":"crossref","unstructured":"Nakamura Y et al. ChatGPT for automating lung cancer staging: feasibility study on open radiology report dataset. medRxiv. Preprint posted online on Dec 3, 2023. [doi: 10.1101\/2023.12.11.23299107] [Medline: 37662351]","DOI":"10.1101\/2023.12.11.23299107"},{"issue":"21","key":"R17","doi-asserted-by":"publisher","first-page":"51","DOI":"10.3390\/cancers16213621","article-title":"Exploring multilingual large language models for enhanced TNM classification of radiology report in lung cancer staging","volume":"16","author":"Matsuo","journal-title":"Cancers (Basel)"},{"key":"R18","unstructured":"Fujimoto K Nishio M Tanaka C et al. Classification of cancer TNM stage from Japanese radiology report using on-premise LLM at NTCIR-17 mednlp-SC RR- TNM subtask. NII institutional repository. In: NTCIR Conference on Evaluation of Information Access Technologies. 2023. [doi: 10.20736\/0002001299]"},{"issue":"1","key":"R19","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1007\/s11604-024-01643-y","article-title":"Preliminary assessment of TNM classification performance for pancreatic cancer in Japanese radiology reports using GPT-4","volume":"43","author":"Suzuki","journal-title":"Jpn J Radiol"},{"key":"R20","doi-asserted-by":"crossref","unstructured":"Chang CH Lucas MM Lu-Yao G Yang CC . Classifying cancer stage with open-source clinical large language models. Presented at: Orlando, FL, USA; Jun 3-6, 2024. [doi: 10.1109\/ICHI61247.2024.00018]","DOI":"10.1109\/ICHI61247.2024.00018"},{"key":"R21","doi-asserted-by":"publisher","DOI":"10.20736\/0002001283","article-title":"Zero-shot classification of TNM staging for Japanese radiology report using ChatGPT at RR-TNM subtask of NTCIR-17 mednlp-SC","author":"Nishio","journal-title":"Proceedings of the 17th NTCIR Conference on Evaluation of Information Access Technologies"},{"key":"R22","unstructured":"General-purpose character recognition (high-precision version). In: Tencent Cloud. URL: https:\/\/cloud.tencent.com\/document\/product\/866\/34937 [Accessed 06-04-2026]"},{"issue":"1","key":"R23","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1016\/j.jtho.2015.09.009","article-title":"The IASLC lung cancer staging project: proposals for revision of the TNM stage groupings in the forthcoming (Eighth) edition of the TNM classification for lung cancer","volume":"11","author":"Goldstraw","journal-title":"J Thorac Oncol"},{"key":"R24","unstructured":"Attribution 4.0 international (CC BY 4.0). Creative Commons. URL: https:\/\/creativecommons.org\/licenses\/by\/4.0\/ [Accessed 20-03-2026]"},{"key":"R25","unstructured":"Team GLM: Zeng A, Xu B, Wang B, et al. ChatGLM: a family of large language models from GLM-130B to GLM-4 all tools. arXiv. Preprint posted online on Jun 18, 2024. [doi: 10.48550\/arXiv.2406.12793]"},{"key":"R26","unstructured":"Hu E . LoRA: low-rank adaptation of large language models. arXiv. Preprint posted online on Oct 10, 2021. [doi: 10.48550\/arXiv.2106.09685]"},{"key":"R27","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2013.00863","article-title":"Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs","volume":"4","author":"Lakens","journal-title":"Front Psychol"},{"key":"R28","unstructured":"Cohen J . Statistical Power Analysis for the Behavioral Sciences. 1988. ISBN: 9780203771587"}],"container-title":["JMIR AI"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/ai.jmir.org\/2026\/1\/e77988","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T18:10:24Z","timestamp":1782843024000},"score":1,"resource":{"primary":{"URL":"https:\/\/ai.jmir.org\/2026\/1\/e77988"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,15]]},"references-count":28,"URL":"https:\/\/doi.org\/10.2196\/77988","relation":{},"ISSN":["2817-1705"],"issn-type":[{"value":"2817-1705","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,15]]},"article-number":"v5i1e77988"}}