{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,28]],"date-time":"2026-08-28T23:49:51Z","timestamp":1787960991488,"version":"build-2784847793"},"reference-count":28,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Med Inform"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Most artificial intelligence (AI) models for thyroid nodules are designed to screen for malignancy to guide further interventions; however, these models have not yet been fully implemented in clinical practice.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study aimed to evaluate AI in real clinical settings for identifying potentially benign thyroid nodules initially deemed to be at risk for malignancy by radiologists, reducing unnecessary fine needle aspiration (FNA) and optimizing management.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n We retrospectively collected a validation cohort of thyroid nodules that had undergone FNA. These nodules were initially assessed as \u201csuspicious for malignancy\u201d by radiologists based on ultrasound features, following standard clinical practice, which prompted further FNA procedures. Ultrasound images of these nodules were re-evaluated using a deep learning\u2013based AI system, and its diagnostic performance was assessed in terms of correct identification of benign nodules and error identification of malignant nodules. Performance metrics such as sensitivity, specificity, and the area under the receiver operating characteristic curve were calculated. In addition, a separate comparison cohort was retrospectively assembled to compare the AI system\u2019s ability to correctly identify benign thyroid nodules with that of radiologists.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n The validation cohort comprised 4572 thyroid nodules (benign: n=3134, 68.5%; malignant: n=1438, 31.5%). AI correctly identified 2719 (86.8% among benign nodules) and reduced unnecessary FNAs from 68.5% (3134\/4572) to 9.1% (415\/4572). However, 123 malignant nodules (8.6% of malignant cases) were mistakenly identified as benign, with the majority of these being of low or intermediate suspicion. In the comparison cohort, AI successfully identified 81.4% (96\/118) of benign nodules. It outperformed junior and senior radiologists, who identified only 40% and 55%, respectively. The area under the curve (AUC) for the AI model was 0.88 (95% CI 0.85\u20100.91), demonstrating a superior AUC compared with that of the junior radiologists (AUC=0.43, 95% CI 0.36\u20100.50;\n P<\/jats:italic>\n =.002) and senior radiologists (AUC=0.63, 95% CI 0.55\u20100.70;\n P<\/jats:italic>\n =.003).\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n Compared with radiologists, AI can better serve as a \u201cgoalkeeper\u201d in reducing unnecessary FNAs by identifying benign nodules that are initially assessed as malignant by radiologists. However, active surveillance is still necessary for all these nodules since a very small number of low-aggressiveness malignant nodules may be mistakenly identified.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/71740","type":"journal-article","created":{"date-parts":[[2025,7,30]],"date-time":"2025-07-30T19:21:30Z","timestamp":1753903290000},"page":"e71740-e71740","source":"Crossref","is-referenced-by-count":6,"title":["Optimizing Thyroid Nodule Management With Artificial Intelligence: Multicenter Retrospective Study on Reducing Unnecessary Fine Needle Aspirations"],"prefix":"10.2196","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-3633-9278","authenticated-orcid":false,"given":"Jia-Hui","family":"Ni","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5865-4993","authenticated-orcid":false,"given":"Yun-Yun","family":"Liu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0865-6971","authenticated-orcid":false,"given":"Chao","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1907-8214","authenticated-orcid":false,"given":"Yi-Lei","family":"Shi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-3943-3705","authenticated-orcid":false,"given":"Xing","family":"Zhao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7886-5122","authenticated-orcid":false,"given":"Xiao-Long","family":"Li","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-8607-1220","authenticated-orcid":false,"given":"Bei-Bei","family":"Ye","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0690-5260","authenticated-orcid":false,"given":"Jing-Liang","family":"Hu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8407-6413","authenticated-orcid":false,"given":"Li-Chao","family":"Mou","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4301-4520","authenticated-orcid":false,"given":"Li-Ping","family":"Sun","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0640-5186","authenticated-orcid":false,"given":"Hui-Jun","family":"Fu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5530-3613","authenticated-orcid":false,"given":"Xiao-Xiang","family":"Zhu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0789-322X","authenticated-orcid":false,"given":"Yi-Feng","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5685-1280","authenticated-orcid":false,"given":"Lehang","family":"Guo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8699-854X","authenticated-orcid":false,"given":"Hui-Xiong","family":"Xu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2025,7,30]]},"reference":[{"issue":"1","key":"R1","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1089\/thy.2015.0020","article-title":"2015 American Thyroid Association management guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: The American Thyroid Association Guidelines Task Force on thyroid nodules and differentiated thyroid cancer","volume":"26","author":"Haugen","journal-title":"Thyroid"},{"issue":"13","key":"R2","doi-asserted-by":"publisher","first-page":"1338","DOI":"10.1001\/jama.2017.2719","article-title":"Trends in thyroid cancer incidence and mortality in the United States, 1974-2013","volume":"317","author":"Lim","journal-title":"JAMA"},{"issue":"3","key":"R3","doi-asserted-by":"publisher","first-page":"784","DOI":"10.1158\/1055-9965.EPI-08-0960","article-title":"Rising thyroid cancer incidence in the United States by demographic and tumor characteristics, 1980-2005","volume":"18","author":"Enewold","journal-title":"Cancer Epidemiol Biomarkers Prev"},{"issue":"4","key":"R4","doi-asserted-by":"publisher","first-page":"310","DOI":"10.1056\/NEJMp1804426","article-title":"Saving thyroids - overtreatment of small papillary cancers","volume":"379","author":"Welch","journal-title":"N Engl J Med"},{"issue":"4","key":"R5","doi-asserted-by":"publisher","first-page":"e130","DOI":"10.1016\/j.suronc.2010.06.003","article-title":"Thyroid nodules; interpretation and importance of fine-needle aspiration (FNA) for the clinician - practical considerations","volume":"19","author":"Sakorafas","journal-title":"Surg Oncol"},{"issue":"5","key":"R6","doi-asserted-by":"publisher","first-page":"31690937","DOI":"10.1210\/clinem\/dgz170","article-title":"Performance of five ultrasound risk stratification systems in selecting thyroid nodules for FNA","volume":"105","author":"Castellana","journal-title":"J Clin Endocrinol Metab"},{"issue":"8","key":"R7","doi-asserted-by":"publisher","first-page":"1916","DOI":"10.1016\/j.ultrasmedbio.2020.03.031","article-title":"Diagnostic performance evaluation of practice guidelines, elastography and their combined results for thyroid nodules: a multicenter study","volume":"46","author":"Wang","journal-title":"Ultrasound Med Biol"},{"issue":"1","key":"R8","doi-asserted-by":"publisher","first-page":"185","DOI":"10.1148\/radiol.2018172572","article-title":"Reduction in thyroid nodule biopsies and improved accuracy with American College of Radiology thyroid imaging reporting and data system","volume":"287","author":"Hoang","journal-title":"Radiology"},{"issue":"4","key":"R9","doi-asserted-by":"publisher","first-page":"900","DOI":"10.2214\/AJR.19.21904","article-title":"Retrospective cohort study of 1947 thyroid nodules: a comparison of the 2017 American College of Radiology TI-RADS and the 2015 American Thyroid Association Classifications","volume":"214","author":"Pandya","journal-title":"AJR Am J Roentgenol"},{"issue":"5","key":"R10","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.221408","article-title":"Multimodality US versus thyroid imaging reporting and data system criteria in recommending fine-needle aspiration of thyroid nodules","volume":"307","author":"Xiao","journal-title":"Radiology"},{"issue":"6","key":"R11","doi-asserted-by":"publisher","first-page":"1296","DOI":"10.1200\/JCO.22.00232","article-title":"High diagnostic accuracy of epigenetic imprinting biomarkers in thyroid nodules","volume":"41","author":"Xu","journal-title":"J Clin Oncol"},{"issue":"4","key":"R12","doi-asserted-by":"publisher","first-page":"e250","DOI":"10.1016\/S2589-7500(21)00041-8","article-title":"Deep learning-based artificial intelligence model to assist thyroid nodule diagnosis and management: a multicentre diagnostic study","volume":"3","author":"Peng","journal-title":"Lancet Digit Health"},{"issue":"2","key":"R13","doi-asserted-by":"publisher","first-page":"150","DOI":"10.1089\/thy.2022.0560","article-title":"Artificial intelligence for evaluation of thyroid nodules: a primer","volume":"33","author":"Tessler","journal-title":"Thyroid"},{"issue":"4","key":"R14","doi-asserted-by":"publisher","first-page":"546","DOI":"10.1089\/thy.2016.0372","article-title":"A computer-aided diagnosis system using artificial intelligence for the diagnosis and characterization of thyroid nodules on ultrasound: initial clinical assessment","volume":"27","author":"Choi","journal-title":"Thyroid"},{"issue":"3","key":"R15","doi-asserted-by":"publisher","first-page":"2120","DOI":"10.1007\/s00330-021-08298-7","article-title":"An integrated AI model to improve diagnostic accuracy of ultrasound and output known risk features in suspicious thyroid nodules","volume":"32","author":"Wang","journal-title":"Eur Radiol"},{"key":"R16","doi-asserted-by":"publisher","DOI":"10.1016\/j.ejrad.2020.108992","article-title":"Differential diagnosis of benign and malignant thyroid nodules using deep learning radiomics of thyroid ultrasound images","volume":"127","author":"Zhou","journal-title":"Eur J Radiol"},{"issue":"8","key":"R17","doi-asserted-by":"publisher","first-page":"903","DOI":"10.1089\/thy.2023.0132","article-title":"Artificial intelligence in thyroidology: a narrative review of the current applications, associated challenges, and future directions","volume":"33","author":"Toro-Tobon","journal-title":"Thyroid"},{"issue":"3","key":"R18","doi-asserted-by":"publisher","first-page":"613","DOI":"10.1148\/radiol.211455","article-title":"An artificial intelligence model based on ACR TI-RADS characteristics for US diagnosis of thyroid nodules","volume":"303","author":"Chen","journal-title":"Radiology"},{"issue":"3","key":"R19","doi-asserted-by":"publisher","first-page":"408","DOI":"10.2214\/AJR.22.28231","article-title":"Thyroid nodules on ultrasound in children and young adults: comparison of diagnostic performance of radiologists\u2019 impressions, ACR TI-RADS, and a deep learning algorithm","volume":"220","author":"Yang","journal-title":"AJR Am J Roentgenol"},{"issue":"11","key":"R20","doi-asserted-by":"publisher","first-page":"1341","DOI":"10.1089\/thy.2017.0500","article-title":"The 2017 Bethesda system for reporting thyroid cytopathology","volume":"27","author":"Cibas","journal-title":"Thyroid"},{"key":"R21","unstructured":"Zbontar J Jing L Misra I LeCun Y Deny S . Barlow twins: self-supervised learning via redundancy reduction. arXiv. Preprint posted online on 2021. [doi: 10.48550\/arXiv.2103.03230]"},{"key":"R22","doi-asserted-by":"publisher","DOI":"10.4018\/978-1-60566-766-9.ch011","article-title":"Transfer learning","author":"Torrey","journal-title":"Handbook of Research on Machine Learning Applications"},{"issue":"2","key":"R23","doi-asserted-by":"publisher","first-page":"523","DOI":"10.1109\/TNNLS.2020.2995319","article-title":"Transformation-consistent self-ensembling model for semisupervised medical image segmentation","volume":"32","author":"Li","journal-title":"IEEE Trans Neural Netw Learning Syst"},{"issue":"1018321","key":"R24","doi-asserted-by":"publisher","first-page":"1018321","DOI":"10.3389\/fendo.2022.1018321","article-title":"Identification of benign and malignant thyroid nodules based on dynamic AI ultrasound intelligent auxiliary diagnosis system","volume":"13","author":"Wang","journal-title":"Front Endocrinol (Lausanne)"},{"issue":"194","key":"R25","doi-asserted-by":"publisher","DOI":"10.3791\/64480","article-title":"A swin transformer-based model for thyroid nodule detection in ultrasound images","volume":"37154577","author":"Tian","journal-title":"J Vis Exp"},{"issue":"3","key":"R26","doi-asserted-by":"publisher","first-page":"695","DOI":"10.1148\/radiol.2019181343","article-title":"Management of thyroid nodules seen on US images: deep learning may match performance of radiologists","volume":"292","author":"Buda","journal-title":"Radiology"},{"issue":"2","key":"R27","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1016\/S1470-2045(18)30762-9","article-title":"Diagnosis of thyroid cancer using deep convolutional neural network models applied to sonographic images: a retrospective, multicohort, diagnostic study","volume":"20","author":"Li","journal-title":"Lancet Oncol"},{"issue":"1","key":"R28","doi-asserted-by":"publisher","first-page":"92","DOI":"10.1148\/radiol.2019181723","article-title":"Diagnostic performance of practice guidelines for thyroid nodules: thyroid nodule size versus biopsy rates","volume":"291","author":"Ha","journal-title":"Radiology"}],"container-title":["JMIR Medical Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/medinform.jmir.org\/2025\/1\/e71740","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T18:01:31Z","timestamp":1782842491000},"score":1,"resource":{"primary":{"URL":"https:\/\/medinform.jmir.org\/2025\/1\/e71740"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,30]]},"references-count":28,"URL":"https:\/\/doi.org\/10.2196\/71740","relation":{},"ISSN":["2291-9694"],"issn-type":[{"value":"2291-9694","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,30]]},"article-number":"v13i1e71740"}}