{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,24]],"date-time":"2026-09-24T16:19:32Z","timestamp":1790266772484,"version":"4.1.0"},"reference-count":59,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,8,29]],"date-time":"2025-08-29T00:00:00Z","timestamp":1756425600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Endocrinol."],"abstract":"<jats:sec><jats:title>Objective<\/jats:title><jats:p>To develop and validate a machine learning (ML)-based prediction model of Bethesda III nodules and create a nomogram based on the best model.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>We collected data on patients with Bethesda III nodules who were admitted between January 2020 and June 2024, including 276 Bethesda III nodules from 7371 patients who underwent ultrasound-guided fine needle aspiration (US-FNA). Clinical, ultrasonographic, cytological, laboratory, and molecular data were collected and randomly split into training and validation cohorts at a ratio of 7: 3. Six feature selection methods and ML algorithms\u2014Logistic Regression (LR), Decision Tree (DT), Random Forest (RF), Extreme Gradient Boosting (XGB), Support Vector Machine (SVM), and K-Nearest Neighbors (KNN)\u2014were evaluated. A nomogram was then created based on the best-performing model.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>The study cohort included 276 Bethesda III nodules with a final malignancy rate of 65.2% (180\/276). LR exhibited the highest area under the receiver operating characteristic (ROC) curve (AUC: 0.823) in cross-validation of the validation set. Additionally, the calibration curves and Decision Curve Analysis (DCA) results were also favorable. The model included BRAF, composition, shape, orientation, and the thyroid imaging reporting and data system (TI-RADS). The nomogram exhibited robust discrimination (AUC: 0.846 in the validation set), calibration, and clinical applicability across the two datasets after 500 bootstraps.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p>Among the six ML algorithms, the LR algorithm demonstrated the best performance. A nomogram was developed to predict the malignancy risk in Bethesda III nodules. This nomogram may serve as a valuable tool to reduce diagnostic uncertainty and provide personalized risk stratification for patients.<\/jats:p><\/jats:sec>","DOI":"10.3389\/fendo.2025.1655828","type":"journal-article","created":{"date-parts":[[2025,8,29]],"date-time":"2025-08-29T04:12:05Z","timestamp":1756440725000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Prediction of malignancy risk in Bethesda III nodules: development and validation of multiple machine learning models"],"prefix":"10.3389","volume":"16","author":[{"given":"Wentian","family":"Li","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiayu","family":"Zhu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ying","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jingxiu","family":"Li","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhonghui","family":"Li","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cuicui","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jingli","family":"Xue","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peng","family":"Zhou","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qingqing","family":"He","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2025,8,29]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2018.0898","article-title":"The diagnosis and management of thyroid nodules: A review","volume":"319","author":"Durante","year":"2018","journal-title":"JAMA"},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.1038\/s41574-024-01025-4","article-title":"Thyroid nodules: Diagnosis and management","volume":"20","author":"Grani","year":"2024","journal-title":"Nat Rev Endocrinol"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.1089\/thy.2023.0141","article-title":"The 2023 Bethesda system for reporting thyroid cytopathology","volume":"33","author":"Ali","year":"2023","journal-title":"Thyroid"},{"key":"B4","doi-asserted-by":"publisher","first-page":"1026","DOI":"10.3390\/diagnostics14101026","article-title":"The incidence of thyroid cancer in Bethesda III thyroid nodules: A retrospective analysis at a single endocrine surgery center","volume":"14","author":"Hassan","year":"2024","journal-title":"Diagnostics (Basel)"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.1055\/s-0039-1698784","article-title":"Malignancy rate in thyroid nodules with atypia or follicular lesion of undetermined significance","volume":"24","author":"Pasha","year":"2020","journal-title":"Int Arch Otorhinolaryngol"},{"key":"B6","doi-asserted-by":"publisher","first-page":"16","DOI":"10.1016\/j.jasc.2023.09.006","article-title":"The risk of Malignancy in the atypia of undetermined significance\/follicular lesion of undetermined significance (AUS\/FLUS) category subgroups: A Thai institute experience","volume":"13","author":"Jiragawasan","year":"2024","journal-title":"J Am Soc Cytopathol"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.1097\/md.0000000000018780","article-title":"Malignancy rate of Bethesda category III thyroid nodules according to ultrasound risk stratification system and cytological subtype","volume":"99","author":"Yoo","year":"2020","journal-title":"Med (Baltimore)"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.1016\/j.surg.2023.05.046","article-title":"Diagnostic performance of molecular testing in indeterminate (Bethesda III and IV) thyroid nodules with H\u00fcrthle cell cytology","volume":"175","author":"Raghunathan","year":"2024","journal-title":"Surgery"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.1007\/s11427-017-9083-8","article-title":"Correlation between BRAF (V600E) mutation and clinicopathological features in pediatric papillary thyroid carcinoma","volume":"60","author":"Geng","year":"2017","journal-title":"Sci China Life Sci"},{"key":"B10","doi-asserted-by":"publisher","first-page":"4666","DOI":"10.1038\/s41598-017-04948-3","article-title":"BRAF and NRAS mutations in papillary thyroid carcinoma and concordance in BRAF mutations between primary and corresponding lymph node metastases","volume":"7","author":"Fakhruddin","year":"2017","journal-title":"Sci Rep"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1530\/eje.1.02072","article-title":"Detection of BRAF mutation in thyroid papillary carcinomas by mutant allele-specific PCR amplification (MASA)","volume":"154","author":"Sapio","year":"2006","journal-title":"Eur J Endocrinol"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.1210\/jc.2011-1469","article-title":"Impact of mutational testing on the diagnosis and management of patients with cytologically indeterminate thyroid nodules: A prospective analysis of 1056 FNA samples","volume":"96","author":"Nikiforov","year":"2011","journal-title":"J Clin Endocrinol Metab"},{"key":"B13","doi-asserted-by":"publisher","DOI":"10.1001\/jamasurg.2018.1153","article-title":"Performance of a genomic sequencing classifier for the preoperative diagnosis of cytologically indeterminate thyroid nodules","volume":"153","author":"Patel","year":"2018","journal-title":"JAMA Surg"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.3389\/fendo.2024.1366687","article-title":"Optimizing thyroid AUS nodules Malignancy prediction: A comprehensive study of logistic regression and machine learning models","volume":"15","author":"Cao","year":"2024","journal-title":"Front Endocrinol (Lausanne)"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.1016\/j.surg.2013.12.035","article-title":"A nomogram for predicting Malignancy in thyroid nodules diagnosed as atypia of undetermined significance\/follicular lesions of undetermined significance on fine needle aspiration","volume":"155","author":"Yoon","year":"2014","journal-title":"Surgery"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.21037\/gs-24-310","article-title":"Combined model integrating clinical, radiomics, BRAF(V600E) and ultrasound for differentiating between benign and Malignant indeterminate cytology (Bethesda III) thyroid nodules: A bi-center retrospective study","volume":"13","author":"Zhong","year":"2024","journal-title":"Gland Surg"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.3389\/fendo.2020.577537","article-title":"Machine learning algorithms for the prediction of central lymph node metastasis in patients with papillary thyroid cancer","volume":"11","author":"Wu","year":"2020","journal-title":"Front Endocrinol (Lausanne)"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2402.02401","article-title":"AI-generated content enhanced computer-aided diagnosis model for thyroid nodules: A chatGPT-style assistant","volume":"2402","author":"Yao","year":"2024","journal-title":"arXiv preprint arXiv"},{"key":"B19","doi-asserted-by":"publisher","first-page":"245","DOI":"10.1038\/s41746-025-01652-9","article-title":"Multimodal GPT model for assisting thyroid nodule diagnosis and management","volume":"8","author":"Yao","year":"2025","journal-title":"NPJ Digit Med"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.1016\/j.isci.2023.108114","article-title":"AI diagnosis of Bethesda category IV thyroid nodules","volume":"26","author":"Yao","year":"2023","journal-title":"iScience"},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2025.109918","article-title":"Potential of AI and ML in oncology research including diagnosis, treatment and future directions: A comprehensive prospective","volume":"189","author":"Gupta","year":"2025","journal-title":"Comput Biol Med"},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.1136\/bmj-2023-078378","article-title":"TRIPOD+AI statement: Updated guidance for reporting clinical prediction models that use regression or machine learning methods","volume":"385","author":"Collins","year":"2024","journal-title":"BMJ"},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.3348\/kjr.2024.0292","article-title":"Risk stratification of thyroid nodules diagnosed as Bethesda category III by ultrasound, size, and cytology","volume":"25","author":"Ahn","year":"2024","journal-title":"Korean J Radiol"},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.1007\/s12020-020-02441-y","article-title":"2020 Chinese guidelines for ultrasound Malignancy risk stratification of thyroid nodules: The C-TIRADS","volume":"70","author":"Zhou","year":"2020","journal-title":"Endocrine"},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.1093\/aje\/kwk052","article-title":"Relaxing the rule of ten events per variable in logistic and Cox regression","volume":"165","author":"Vittinghoff","year":"2007","journal-title":"Am J Epidemiol"},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.1177\/0962280218784726","article-title":"Sample size for binary logistic prediction models: Beyond events per variable criteria","volume":"28","author":"van Smeden","year":"2019","journal-title":"Stat Methods Med Res"},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.1016\/j.eclinm.2024.103023","article-title":"Development and validation of a model to predict cognitive impairment in traumatic brain injury patients: A prospective observational study","volume":"80","author":"Yuan","year":"2025","journal-title":"EClinicalMedicine"},{"key":"B28","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.m441","article-title":"Calculating the sample size required for developing a clinical prediction model","volume":"368","author":"Riley","year":"2020","journal-title":"BMJ"},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.2196\/68509","article-title":"Machine learning models with prognostic implications for predicting gastrointestinal bleeding after coronary artery bypass grafting and guiding personalized medicine: Multicenter cohort study","volume":"27","author":"Dong","year":"2025","journal-title":"J Med Internet Res"},{"key":"B30","doi-asserted-by":"publisher","first-page":"403","DOI":"10.1186\/s12891-025-08619-7","article-title":"Screening risk factors for the occurrence of wedge effects in intramedullary nail fixation for intertrochanteric fractures in older people via machine learning and constructing a prediction model: a retrospective study","volume":"26","author":"Xu","year":"2025","journal-title":"BMC Musculoskelet Disord"},{"key":"B31","doi-asserted-by":"publisher","first-page":"291","DOI":"10.1186\/s12967-025-06289-6","article-title":"Prediction of postpartum depression in women: development and validation of multiple machine learning models","volume":"23","author":"Qi","year":"2025","journal-title":"J Transl Med"},{"key":"B32","doi-asserted-by":"publisher","DOI":"10.1109\/tpami.2017.2765321","article-title":"Ensembles of lasso screening rules","volume":"40","author":"Lee","year":"2018","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"B33","doi-asserted-by":"publisher","DOI":"10.1111\/cyt.13426","article-title":"The Bethesda system for reporting thyroid cytopathology in the African American population: A tertiary centre experience","volume":"35","author":"Saoud","year":"2024","journal-title":"Cytopathology"},{"key":"B34","doi-asserted-by":"publisher","first-page":"16","DOI":"10.1186\/s40463-019-0338-7","article-title":"The role of repeat fine needle aspiration in managing indeterminate thyroid nodules","volume":"48","author":"Allen","year":"2019","journal-title":"J Otolaryngol Head Neck Surg"},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.1177\/01945998221093527","article-title":"Repeat fine-needle aspiration with molecular analysis in management of indeterminate thyroid nodules","volume":"168","author":"Papazian","year":"2023","journal-title":"Otolaryngol Head Neck Surg"},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.1210\/clinem\/dgae112","article-title":"Outcomes of cytologically indeterminate thyroid nodules managed with genomic sequencing classifier","volume":"109","author":"Ahmadi","year":"2024","journal-title":"J Clin Endocrinol Metab"},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.1001\/jamaoncol.2018.4616","article-title":"Performance of a multigene genomic classifier in thyroid nodules with indeterminate cytology: A prospective blinded multicenter study","volume":"5","author":"Steward","year":"2019","journal-title":"JAMA Oncol"},{"key":"B38","doi-asserted-by":"publisher","DOI":"10.1089\/thy.2016.0048","article-title":"Correlation between histological diagnosis and mutational panel testing of thyroid nodules: A two-year institutional experience","volume":"26","author":"Shrestha","year":"2016","journal-title":"Thyroid"},{"key":"B39","doi-asserted-by":"publisher","DOI":"10.1177\/00031348241265143","article-title":"The combined use of fine-needle aspiration (FNA) and BRAF V600E gene testing: Can it increase the definitive diagnosis rate of nodules categorized as Bethesda III for papillary thyroid carcinoma","volume":"90","author":"Lei","year":"2024","journal-title":"Am Surg"},{"key":"B40","doi-asserted-by":"publisher","DOI":"10.1007\/s42000-024-00597-0","article-title":"The value of preoperative molecular testing in the management of Bethesda V and Bethesda VI thyroid tumors","volume":"24","author":"Paspala","year":"2025","journal-title":"(Hormones (Athens)"},{"key":"B41","doi-asserted-by":"publisher","DOI":"10.3803\/EnM.2024.2034","article-title":"Small multi-gene DNA panel can aid in reducing the surgical resection rate and predicting the Malignancy risk of thyroid nodules","volume":"39","author":"Oh","year":"2024","journal-title":"Endocrinol Metab (Seoul)"},{"key":"B42","doi-asserted-by":"publisher","DOI":"10.3389\/fendo.2024.1326134","article-title":"Ultrasonographic predictors of thyroid cancer in Bethesda III and IV thyroid nodules","volume":"15","author":"Alyusuf","year":"2024","journal-title":"Front Endocrinol (Lausanne)"},{"key":"B43","doi-asserted-by":"publisher","first-page":"5934","DOI":"10.1038\/s41598-022-09822-5","article-title":"Diagnostic value of thyroid imaging reporting and data system combined with BRAF(V600E) mutation analysis in Bethesda categories III-V thyroid nodules","volume":"12","author":"Wu","year":"2022","journal-title":"Sci Rep"},{"key":"B44","doi-asserted-by":"publisher","DOI":"10.1016\/j.ultrasmedbio.2016.05.011","article-title":"An algorithm of image heterogeneity with contrast-enhanced ultrasound in differential diagnosis of solid thyroid nodules","volume":"43","author":"Jin","year":"2017","journal-title":"Ultrasound Med Biol"},{"key":"B45","doi-asserted-by":"publisher","first-page":"67","DOI":"10.1507\/endocrj.EJ21-0300","article-title":"Approach to Bethesda System category III thyroid nodules according to US-risk stratification","volume":"69","author":"Kim","year":"2022","journal-title":"Endocr J"},{"key":"B46","doi-asserted-by":"publisher","DOI":"10.1089\/thy.2015.0637","article-title":"Subcategorization of Bethesda System category III by ultrasonography","volume":"26","author":"Lee","year":"2016","journal-title":"Thyroid"},{"key":"B47","doi-asserted-by":"publisher","DOI":"10.21873\/invivo.12014","article-title":"Taller-than-wide thyroid nodules with microcalcifications are at high risk of Malignancy","volume":"34","author":"Papapostolou","year":"2020","journal-title":"In Vivo"},{"key":"B48","doi-asserted-by":"publisher","DOI":"10.1097\/ms9.0000000000002676","article-title":"The accuracy of fine-needle aspiration cytology and ultrasonography in assessing thyroid nodules in correlation with histopathology: A retrospective study","volume":"86","author":"Alhajlan","year":"2024","journal-title":"Ann Med Surg (Lond)"},{"key":"B49","doi-asserted-by":"publisher","DOI":"10.1002\/dc.25261","article-title":"Two-tier subclassification of the Bethesda category III (Atypia of undetermined significance\/follicular lesion of undetermined significance) in thyroid cytology","volume":"52","author":"Alden","year":"2024","journal-title":"Diagn Cytopathol"},{"key":"B50","doi-asserted-by":"publisher","DOI":"10.1002\/cncy.22831","article-title":"Malignant risk of pediatric Bethesda category III thyroid nodules subcategorized by nuclear atypia and other: A single institution experience","volume":"132","author":"Jin","year":"2024","journal-title":"Cancer Cytopathol"},{"key":"B51","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1007\/s12022-024-09797-1","article-title":"A comprehensive approach to the thyroid Bethesda category III (AUS) in the transition zone between 2nd edition and 3rd edition of the Bethesda System for Reporting Thyroid Cytopathology: Subcategorization, nuclear scoring, and more","volume":"35","author":"Bagis","year":"2024","journal-title":"Endocr Pathol"},{"key":"B52","doi-asserted-by":"publisher","DOI":"10.3389\/fendo.2020.00452","article-title":"Clinical analysis of preoperative anti-thyroglobulin antibody in papillary thyroid cancer between 2011 and 2015 in Beijing, China: A retrospective study","volume":"11","author":"Jia","year":"2020","journal-title":"Front Endocrinol (Lausanne)"},{"key":"B53","doi-asserted-by":"publisher","first-page":"92","DOI":"10.1016\/j.cell.2020.03.022","article-title":"How machine learning will transform biomedicine","volume":"181","author":"Goecks","year":"2020","journal-title":"Cell"},{"key":"B54","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btab659","article-title":"Accelerating big data analysis through lasso-random forest algorithm in QSAR studies","volume":"38","author":"Motamedi","year":"2022","journal-title":"Bioinformatics"},{"key":"B55","doi-asserted-by":"publisher","DOI":"10.1109\/tpami.1984.4767533","article-title":"Classification bias of the k-nearest neighbor algorithm","volume":"6","author":"Goin","year":"1984","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"B56","doi-asserted-by":"publisher","DOI":"10.1109\/tpami.2023.3310908","article-title":"Handling multi-class problem by intuitionistic fuzzy twin support vector machines based on relative density information","volume":"45","author":"Rezvani","year":"2023","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"B57","doi-asserted-by":"publisher","DOI":"10.1159\/000541294","article-title":"Comparison of machine learning algorithms and nomogram construction for diabetic retinopathy prediction in type 2 diabetes mellitus patients","volume":"67","author":"Jiang","year":"2024","journal-title":"Ophthalmic Res"},{"key":"B58","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1016\/j.jclinepi.2019.02.004","article-title":"A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models","volume":"110","author":"Christodoulou","year":"2019","journal-title":"J Clin Epidemiol"},{"key":"B59","doi-asserted-by":"publisher","DOI":"10.1159\/000494720","article-title":"The Malignancy risk assessment of cytologically indeterminate thyroid nodules improves markedly by using a predictive model","volume":"8","author":"\u00d6cal","year":"2019","journal-title":"Eur Thyroid J"}],"container-title":["Frontiers in Endocrinology"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fendo.2025.1655828\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,29]],"date-time":"2025-08-29T04:12:06Z","timestamp":1756440726000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fendo.2025.1655828\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,29]]},"references-count":59,"alternative-id":["10.3389\/fendo.2025.1655828"],"URL":"https:\/\/doi.org\/10.3389\/fendo.2025.1655828","relation":{},"ISSN":["1664-2392"],"issn-type":[{"value":"1664-2392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,29]]},"article-number":"1655828"}}