{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,10]],"date-time":"2026-09-10T09:07:38Z","timestamp":1789031258386,"version":"build-2803163510"},"reference-count":46,"publisher":"SAGE Publications","license":[{"start":{"date-parts":[[2026,2,1]],"date-time":"2026-02-01T00:00:00Z","timestamp":1769904000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"},{"start":{"date-parts":[[2026,2,1]],"date-time":"2026-02-01T00:00:00Z","timestamp":1769904000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"name":"The Science and Technology Plan Project of Beijing","award":["Z241100007724010"],"award-info":[{"award-number":["Z241100007724010"]}]},{"DOI":"10.13039\/501100012166","name":"National Key R&D Program of China","doi-asserted-by":"crossref","award":["2024YFC3505400"],"award-info":[{"award-number":["2024YFC3505400"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"crossref"}]},{"name":"National High Level Chinese Medicine Hospital Clinical Research Funding","award":["DFRCZY-2024GJRC009"],"award-info":[{"award-number":["DFRCZY-2024GJRC009"]}]},{"name":"National High Level Chinese Medicine Hospital Clinical Research Funding","award":["DFRCZY-2024GJRC017"],"award-info":[{"award-number":["DFRCZY-2024GJRC017"]}]},{"name":"2025 Postdoctoral Program for Innovative Talents funded by the National Postdoctoral Management Committee of China","award":["GZC20252622"],"award-info":[{"award-number":["GZC20252622"]}]},{"name":"the China Scholarship Council (CSC), Ministry of Education of the People\u2019s Republic of China","award":["202405350013"],"award-info":[{"award-number":["202405350013"]}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["DIGITAL HEALTH"],"published-print":{"date-parts":[[2026,2]]},"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Lung cancer remains the leading cause of cancer-related mortality worldwide. Large language models (LLMs), including ChatGPT, DeepSeek, and Grok, have shown promise in clinical decision support, but differences in training and alignment may lead to variable performance. Current evaluations often rely on aggregate metrics or isolated tasks, which may not capture real-world clinical complexity.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We conducted a structured evaluation of three LLMs using nine simulated lung cancer cases across five clinical domains. LLMs\u2019 outputs were anonymized, randomized, and independently scored by five senior lung cancer specialists under a double-blind design using a five-point Likert scale evaluating accuracy, comprehensiveness, relevance, and clinical applicability. Qualitative error analysis was also performed.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Inter-rater agreement was moderate (Fleiss\u2019 \u03ba = 0.463; ICC (2, k) = 0.675). All LLMs achieved high scores across evaluation dimensions without statistically significant differences (P &gt; 0.05). Given the limited number of simulated cases, these findings should be interpreted cautiously. Descriptive analyses suggested context-dependent performance patterns across clinical domains: Grok tended to show more consistent performance in diagnosis and treatment decision-making, DeepSeek showed comparatively lower descriptive performance in therapeutic decisions but higher applicability in prognosis and rehabilitation, and GPT exhibited relatively stable intermediate performance. No single LLM consistently outperformed others across all clinical scenarios.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>LLMs demonstrate substantial potential in supporting lung cancer clinical workflows, but their performance appears to be context-dependent. The present findings are exploratory and suggest that task-specific evaluation may provide a more clinically informative framework than overall model ranking. Continued validation using larger and more diverse clinical datasets, together with appropriate governance and specialist oversight, remains essential for the safe integration of LLMs into clinical practice.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1177\/20552076261487079","type":"journal-article","created":{"date-parts":[[2026,9,7]],"date-time":"2026-09-07T04:21:27Z","timestamp":1788754887000},"update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":0,"title":["Exploratory task-specific evaluation of large language models in lung cancer clinical scenarios: A comparative study"],"prefix":"10.1177","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-8425-8317","authenticated-orcid":false,"given":"Wenzheng","family":"Zhang","sequence":"first","affiliation":[{"name":"Graduate School, Beijing University of Chinese Medicine"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xue","family":"Li","sequence":"additional","affiliation":[{"name":"China Academy of Chinese Medical Sciences"},{"name":"Uppsala University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Run","family":"Yuan","sequence":"additional","affiliation":[{"name":"China Academy of Chinese Medical Sciences"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yinuo","family":"Zhang","sequence":"additional","affiliation":[{"name":"Chongqing Medical School"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yufei","family":"Yang","sequence":"additional","affiliation":[{"name":"China Academy of Chinese Medical Sciences"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yun","family":"Xu","sequence":"additional","affiliation":[{"name":"China Academy of Chinese Medical Sciences"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ruikang","family":"Zhong","sequence":"additional","affiliation":[{"name":"Graduate School, Beijing University of Chinese Medicine"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Siyi","family":"Chen","sequence":"additional","affiliation":[{"name":"Graduate School, Beijing University of Chinese Medicine"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dianna","family":"Liu","sequence":"additional","affiliation":[{"name":"Beijing University of Chinese Medicine"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lei","family":"Gao","sequence":"additional","affiliation":[{"name":"Beijing University of Chinese Medicine"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kaiwen","family":"Hu","sequence":"additional","affiliation":[{"name":"Beijing University of Chinese Medicine"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2026,9,6]]},"reference":[{"key":"e_1_3_6_2_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41571-023-00798-3"},{"key":"e_1_3_6_3_2","doi-asserted-by":"publisher","DOI":"10.1136\/bmj-2021-068044"},{"key":"e_1_3_6_4_2","article-title":"NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines\u00ae) for Small Cell Lung Cancer","author":"National Comprehensive Cancer Network","year":"2025","unstructured":"National Comprehensive Cancer Network. NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines\u00ae) for Small Cell Lung Cancer. Fort Washington (PA): National Comprehensive Cancer Network 2025.","journal-title":"Fort Washington (PA): National Comprehensive Cancer Network"},{"key":"e_1_3_6_5_2","article-title":"NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines\u00ae) for Non-Small Cell Lung Cancer","author":"National Comprehensive Cancer Network","year":"2025","unstructured":"National Comprehensive Cancer Network. NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines\u00ae) for Non-Small Cell Lung Cancer. Fort Washington (PA): National Comprehensive Cancer Network 2025.","journal-title":"Fort Washington (PA): National Comprehensive Cancer Network"},{"key":"e_1_3_6_6_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.annonc.2021.03.207"},{"key":"e_1_3_6_7_2","doi-asserted-by":"publisher","DOI":"10.1002\/tox.70121"},{"key":"e_1_3_6_8_2","doi-asserted-by":"publisher","DOI":"10.1186\/s13012-024-01357-9"},{"key":"e_1_3_6_9_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.taml.2025.100583"},{"key":"e_1_3_6_10_2","unstructured":"Lila. Grok 3 has the ability to handle complex tasks with accuracy [Internet]. [cited 2026 Apr 1]. Available from. https:\/\/elonmuskpower.com\/grok-3\/ (2025) [Webpage no longer available online; no public archived version could be located.]."},{"key":"e_1_3_6_11_2","first-page":"1","article-title":"Comparative benchmarking of the DeepSeek large language model on medical tasks and clinical reasoning","volume":"23","author":"Tordjman M","year":"2025","unstructured":"Tordjman M, Liu Z, Yuce M, et al. Comparative benchmarking of the DeepSeek large language model on medical tasks and clinical reasoning. Nature medicine 2025; 23: 1\u20132555. https:\/\/doi.org\/10.1038\/s41591-025-03726-3","journal-title":"Nature medicine"},{"key":"e_1_3_6_12_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-023-06291-2"},{"key":"e_1_3_6_13_2","doi-asserted-by":"publisher","DOI":"10.56305\/001c.122787"},{"key":"e_1_3_6_14_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-025-03983-2"},{"key":"e_1_3_6_15_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-023-02448-8"},{"key":"e_1_3_6_16_2","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2023.43689"},{"key":"e_1_3_6_17_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ebiom.2025.105695"},{"issue":"1","key":"e_1_3_6_18_2","first-page":"14","article-title":"Exploring the role of large language models in the scientific method: from hypothesis to discovery","volume":"1","author":"Zhang Y","year":"2025","unstructured":"Zhang Y, Khan SA, Mahmud A, et al. Exploring the role of large language models in the scientific method: from hypothesis to discovery. Artificial Intelligence 2025; 1(1): 14. https:\/\/doi.org\/10.1038\/s44387-025-00019-5","journal-title":"Artificial Intelligence"},{"key":"e_1_3_6_19_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ebiom.2024.105441"},{"key":"e_1_3_6_20_2","first-page":"1","article-title":"Towards accurate differential diagnosis with large language models","volume":"9","author":"McDuff D","year":"2025","unstructured":"McDuff D, Schaekermann M, Tu T, et al. Towards accurate differential diagnosis with large language models. Nature 2025; 9: 1\u20137. https:\/\/doi.org\/10.1038\/s41586-025-08869-4","journal-title":"Nature"},{"key":"e_1_3_6_21_2","doi-asserted-by":"publisher","DOI":"10.2196\/59069"},{"key":"e_1_3_6_22_2","article-title":"Unlocking the potential of ChatGPT: a comprehensive exploration of its applications, advantages, limitations, and future directions in natural language processing","author":"Hariri W","year":"2025","unstructured":"Hariri W. Unlocking the potential of ChatGPT: a comprehensive exploration of its applications, advantages, limitations, and future directions in natural language processing. arXiv preprint arXiv:2304.02017v14 [cs.CL] 2025, [cited 2026 Apr 1]. Available from. https:\/\/arxiv.org\/abs\/2304.02017","journal-title":"arXiv preprint arXiv:2304.02017v14 [cs.CL]"},{"key":"e_1_3_6_23_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01390-4"},{"key":"e_1_3_6_24_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-026-04229-5"},{"key":"e_1_3_6_25_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-025-02954-4"},{"key":"e_1_3_6_26_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-025-02174-0"},{"key":"e_1_3_6_27_2","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2013.7835"},{"key":"e_1_3_6_28_2","doi-asserted-by":"publisher","DOI":"10.1097\/JS9.0000000000001066"},{"key":"e_1_3_6_29_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijnurstu.2024.104771"},{"key":"e_1_3_6_30_2","doi-asserted-by":"publisher","DOI":"10.1055\/a-2289-5732"},{"key":"e_1_3_6_31_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01239-w"},{"key":"e_1_3_6_32_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-025-01476-7"},{"key":"e_1_3_6_33_2","doi-asserted-by":"publisher","DOI":"10.1016\/S2665-9913(23)00216-3"},{"key":"e_1_3_6_34_2","doi-asserted-by":"publisher","DOI":"10.2196\/90673"},{"key":"e_1_3_6_35_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00296-025-06053-5"},{"key":"e_1_3_6_36_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jdent.2026.106675"},{"key":"e_1_3_6_37_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10067-026-08073-3"},{"key":"e_1_3_6_38_2","doi-asserted-by":"publisher","DOI":"10.5435\/JAAOS-D-25-01242"},{"issue":"2","key":"e_1_3_6_39_2","first-page":"264","article-title":"Evolving Paradigms in Lung Cancer: Latest Trends in Diagnosis, Management, and Radiopharmaceuticals","volume":"55","author":"Cangut B","year":"2025","unstructured":"Cangut B, Akinlusi R, Mohseny A, et al. Evolving Paradigms in Lung Cancer: Latest Trends in Diagnosis, Management, and Radiopharmaceuticals. InSeminars in Nuclear Medicine 2025; 55(2): 264\u2013276, WB Saunders.","journal-title":"InSeminars in Nuclear Medicine"},{"key":"e_1_3_6_40_2","article-title":"Deepseek-v3 technical report","author":"Liu A","year":"2024","unstructured":"Liu A, Feng B, Xue B, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437 2024.","journal-title":"arXiv preprint arXiv:2412.19437"},{"issue":"16","key":"e_1_3_6_41_2","first-page":"8001","article-title":"Neoadjuvant (neoadj) osimertinib (osi) \u00b1 chemotherapy (CT) vs CT alone in resectable (R) epidermal growth factor receptor-mutated","volume":"43","author":"Chaft JE","year":"2025","unstructured":"Chaft JE, Weder W, He J, et al. Neoadjuvant (neoadj) osimertinib (osi) \u00b1 chemotherapy (CT) vs CT alone in resectable (R) epidermal growth factor receptor-mutated. EGFRm) NSCLC: NeoADAURA 2025; 43(16_SUPPL): 8001.","journal-title":"EGFRm) NSCLC: NeoADAURA"},{"key":"e_1_3_6_42_2","volume-title":"CSCO guidelines for diagnosis and treatment of non-small cell lung cancer (2025 edition)","author":"Chinese Society of Clinical Oncology","year":"2025","unstructured":"Chinese Society of Clinical Oncology. CSCO guidelines for diagnosis and treatment of non-small cell lung cancer (2025 edition). Chinese Society of Clinical Oncology, 2025."},{"key":"e_1_3_6_43_2","doi-asserted-by":"publisher","DOI":"10.1080\/1750743X.2025.2580272"},{"key":"e_1_3_6_44_2","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2021.15319"},{"key":"e_1_3_6_45_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-023-02540-z"},{"key":"e_1_3_6_46_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jksuci.2023.01.014"},{"key":"e_1_3_6_47_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01283-6"}],"container-title":["DIGITAL HEALTH"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/20552076261487079","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/20552076261487079","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/20552076261487079","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,10]],"date-time":"2026-09-10T08:20:54Z","timestamp":1789028454000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/20552076261487079"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2]]},"references-count":46,"alternative-id":["10.1177\/20552076261487079"],"URL":"https:\/\/doi.org\/10.1177\/20552076261487079","relation":{},"ISSN":["2055-2076","2055-2076"],"issn-type":[{"value":"2055-2076","type":"print"},{"value":"2055-2076","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2]]},"article-number":"20552076261487079"}}