{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,15]],"date-time":"2026-10-15T16:09:13Z","timestamp":1792080553172,"version":"4.4.0"},"reference-count":36,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,9,29]],"date-time":"2025-09-29T00:00:00Z","timestamp":1759104000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Endocrinol."],"abstract":"<jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>To evaluate the diagnostic performance and clinical utility of advanced large language models (LLMs) -GPT-4o, GPT-o3-mini, and DeepSeek-R1- in stratifying thyroid nodule malignancy risk and generating guideline-aligned management recommendations based on structured narrative ultrasound descriptions.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>This diagnostic modeling study evaluated three LLMs\u2014GPT-4o, GPT-o3-mini, and DeepSeek-R1\u2014using standardized narrative ultrasound descriptors. These descriptors were annotated by consensus among three senior board-certified sonologists and processed independently in a stateless manner to ensure unbiased outputs. LLM outputs were assessed under both ACR-TIRADS and C-TIRADS frameworks. Two experienced clinicians (a thyroid surgeon and an endocrinologist) independently rated the outputs across five clinical dimensions using 5-point Likert scales. Primary outcomes included the area under the receiver operating characteristic curve (AUC) for malignancy prediction, and clinician ratings of guideline adherence, patient safety, operational feasibility, clinical applicability, and overall performance.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>GPT-4o achieved the highest predictive AUC (0.898) under C-TIRADS, approaching expert-level accuracy. DeepSeek-R1, particularly with C-TIRADS, received the highest clinician ratings (mean Likert: surgeon 4.65, endocrinologist 4.63), reflecting greater trust in its practical recommendations. Clinicians consistently favored the C-TIRADS framework across all models. GPT-4o and GPT-o3-mini received lower ratings in trustworthiness and recommendation quality, especially from the endocrinologist.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>While GPT-4o demonstrated superior diagnostic accuracy, clinicians most trusted DeepSeek-R1 combined with the C-TIRADS framework for generating practical, guideline-consistent recommendations. The findings highlight the critical need for alignment between AI-generated outputs and clinician expectations, and the importance of incorporating region-specific clinical guidelines (like C-TIRADS) for the effective real-world implementation of LLMs in thyroid nodule management decision support.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fendo.2025.1667809","type":"journal-article","created":{"date-parts":[[2025,9,29]],"date-time":"2025-09-29T04:12:44Z","timestamp":1759119164000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["LLM evaluation for thyroid nodule assessment: comparing ACR-TIRADS, C-TIRADS, and clinician-AI trust gap"],"prefix":"10.3389","volume":"16","author":[{"given":"Xi","family":"Dai","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yu","family":"Xi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yong","family":"Hu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qingyan","family":"Ding","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yu","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hui","family":"Liu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Piaofei","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xi","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wenjun","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chaoxue","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2025,9,29]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.1016\/S2213-8587(22)00101-2","article-title":"Diagnosis of thyroid nodules","volume":"10","author":"Alexander","year":"2022","journal-title":"Lancet Diabetes Endocrinol"},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.230006","article-title":"Us predictors of papillary thyroid microcarcinoma progression at active surveillance","volume":"309","author":"Lee","year":"2023","journal-title":"Radiology"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.1038\/s41574-024-01025-4","article-title":"Thyroid nodules: diagnosis and management","volume":"20","author":"Grani","year":"2024","journal-title":"Nat Rev Endocrinol"},{"key":"B4","doi-asserted-by":"publisher","DOI":"10.1016\/j.jacr.2017.01.046","article-title":"Acr thyroid imaging, reporting and data system (Ti-rads): white paper of the acr ti-rads committee","volume":"14","author":"Tessler","year":"2017","journal-title":"J Am Coll Radiol"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.1007\/s12020-020-02441-y","article-title":"2020 Chinese guidelines for ultrasound Malignancy risk stratification of thyroid nodules: the C-Tirads","volume":"70","author":"Zhou","year":"2020","journal-title":"Endocrine"},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.3389\/fendo.2023.1227339","article-title":"Diagnostic efficiency among eu-\/C-\/acr-tirads and S-detect for thyroid nodules: A systematic review and network meta-analysis","volume":"14","author":"Yang","year":"2023","journal-title":"Front Endocrinol"},{"key":"B7","doi-asserted-by":"publisher","first-page":"245","DOI":"10.1038\/s41746-025-01652-9","article-title":"Multimodal gpt model for assisting thyroid nodule diagnosis and management","volume":"8","author":"Yao","year":"2025","journal-title":"NPJ Digital Med"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.1007\/s00405-025-09236-9","article-title":"Human-ai collaboration for ultrasound diagnosis of thyroid nodules: A clinical trial","volume":"282","author":"Edstr\u00f6m","year":"2025","journal-title":"Eur Arch Oto-Rhino-Laryngology"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.231481","article-title":"International expert consensus on us lexicon for thyroid nodules","volume":"309","author":"Durante","year":"2023","journal-title":"Radiology"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.21037\/jtd-2025b-04","article-title":"Deepseek\u2019s impact on thoracic surgeons\u2019 Work patterns-past, present and future","volume":"17","author":"Chen","year":"2025","journal-title":"J Thorac Dis"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.3389\/fmed.2024.1402457","article-title":"Large language model answers medical questions about standard pathology reports","volume":"11","author":"Wang","year":"2024","journal-title":"Front Med (Lausanne)"},{"key":"B12","unstructured":"Openai"},{"key":"B13","unstructured":"Openai"},{"key":"B14","unstructured":"Deepseek-R1"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.1016\/j.clinimag.2024.110366","article-title":"A real-world comparison of the diagnostic performances of six different ti-rads guidelines, including acr-\/kwak-\/K-\/eu-\/ata-\/C-tirads","volume":"117","author":"Topcuoglu","year":"2025","journal-title":"Clin Imaging"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.1016\/j.acra.2023.04.013","article-title":"Comparative study of C-tirads, acr-tirads, and eu-tirads for diagnosis and management of thyroid nodules","volume":"30","author":"Jin","year":"2023","journal-title":"Acad Radiol"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.1038\/d41586-025-00275-0","article-title":"Scientists flock to deepseek_ how they\u2019re using the blockbuster ai model","volume":"29","author":"Gibney","year":"2025","journal-title":"nature"},{"key":"B18","doi-asserted-by":"publisher","first-page":"791","DOI":"10.2214\/ajr.22.28556","article-title":"Diagnostic performance of six ultrasound risk stratification systems for thyroid nodules: A systematic review and network meta-analysis","volume":"220","author":"Kim","year":"2023","journal-title":"Am J roentgenol"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.21037\/qims-24-282","article-title":"Comparison of diagnostic performance of the current score-based ultrasound risk stratification systems according to thyroid nodule size","volume":"14","author":"Si","year":"2024","journal-title":"Quant Imaging Med Surg"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.3389\/fendo.2024.1349272","article-title":"Trends of the prevalence rate of central lymph node metastasis and multifocality in patients with low-risk papillary thyroid carcinoma after delayed thyroid surgery","volume":"15","author":"Wang","year":"2024","journal-title":"Front Endocrinol (Lausanne)"},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.231593","article-title":"Quantitative evaluation of large language models to streamline radiology report impressions: A multimodal retrospective analysis","volume":"310","author":"Doshi","year":"2024","journal-title":"Radiology"},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.232255","article-title":"Collaborative enhancement of consistency and accuracy in us diagnosis of thyroid nodules using large language models","volume":"310","author":"Wu","year":"2024","journal-title":"Radiology"},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.1007\/s12020-024-04066-x","article-title":"Assessing the feasibility of chatgpt-4o and claude 3-opus in thyroid nodule classification based on ultrasound images","volume":"87","author":"Chen","year":"2025","journal-title":"Endocrine"},{"key":"B24","doi-asserted-by":"publisher","first-page":"635","DOI":"10.3390\/diagnostics15050635","article-title":"Comparing chatgpt 4.0\u2019s performance in interpreting thyroid nodule ultrasound reports using acr-ti-rads 2017: analysis across different levels of ultrasound user experience","volume":"15","author":"Wakonig","year":"2025","journal-title":"Diagnostics (Basel)"},{"key":"B25","doi-asserted-by":"publisher","first-page":"499","DOI":"10.3348\/kjr.2024.0229","article-title":"Accuracy of large language models in thyroid nodule-related questions based on the korean thyroid imaging reporting and data system (K-tirads)","volume":"25","author":"Kaba","year":"2024","journal-title":"Korean J Radiol"},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.1007\/s12020-024-03981-3","article-title":"Clinical application potential of large language model: A study based on thyroid nodules","volume":"87","author":"Xia","year":"2025","journal-title":"Endocrine"},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.1007\/s00405-024-08525-z","article-title":"Exploring the landscape of ai-assisted decision-making in head and neck cancer treatment: A comparative analysis of nccn guidelines and chatgpt responses","volume":"281","author":"Marchi","year":"2024","journal-title":"Eur Arch Otorhinolaryngol"},{"key":"B28","doi-asserted-by":"publisher","DOI":"10.1001\/jamasurg.2024.1621","article-title":"Llm capabilities in perioperative risk prediction and prognostication","volume":"159","author":"Chung P","year":"2024","journal-title":"JAMA Surg"},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2025.104791","article-title":"From gpt to deepseek: significant gaps remain in realizing ai in healthcare","volume":"163","author":"Peng","year":"2025","journal-title":"J BioMed Inform"},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.21037\/jtd-2025b-03","article-title":"Deepseek: the \u201cWatson\u201d to doctors-from assistance to collaboration","volume":"17","author":"Liang","year":"2025","journal-title":"J Thorac Dis"},{"key":"B31","doi-asserted-by":"publisher","DOI":"10.1530\/etj-23-0242","article-title":"Head-to-head comparison of american, european, and asian tiradss in thyroid nodule assessment: systematic review and meta-analysis","volume":"13","author":"Piticchio","year":"2024","journal-title":"Eur Thyroid J"},{"key":"B32","doi-asserted-by":"publisher","DOI":"10.3389\/fonc.2022.1013410","article-title":"Diagnostic performance of six ultrasound-based risk stratification systems in thyroid follicular neoplasm: A retrospective multi-center study","volume":"12","author":"Yang","year":"2022","journal-title":"Front Oncol"},{"key":"B33","doi-asserted-by":"publisher","DOI":"10.1111\/liv.15974","article-title":"Optimizing large language models in digestive disease: strategies and challenges to improve clinical outcomes","volume":"44","author":"Giuffre","year":"2024","journal-title":"Liver Int"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01258-7","article-title":"A framework for human evaluation of large language models in healthcare derived from literature review","volume":"7","author":"Tam","year":"2024","journal-title":"NPJ Digit Med"},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.3389\/fendo.2022.967380","article-title":"The prevalence of thyroid nodules and its factors among chinese adult women: A cross-sectional study","volume":"13","author":"Dong","year":"2022","journal-title":"Front Endocrinol (Lausanne)"},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.2147\/IJGM.S517851","article-title":"Epidemiological and pathogenic characteristics of benign and Malignant thyroid nodules undergoing ultrasonography for health checkup population","volume":"18","author":"Yao","year":"2025","journal-title":"Int J Gen Med"}],"container-title":["Frontiers in Endocrinology"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fendo.2025.1667809\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,29]],"date-time":"2025-09-29T04:12:45Z","timestamp":1759119165000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fendo.2025.1667809\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,29]]},"references-count":36,"alternative-id":["10.3389\/fendo.2025.1667809"],"URL":"https:\/\/doi.org\/10.3389\/fendo.2025.1667809","relation":{},"ISSN":["1664-2392"],"issn-type":[{"value":"1664-2392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,9,29]]},"article-number":"1667809"}}