{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,1]],"date-time":"2026-06-01T23:19:27Z","timestamp":1780355967708,"version":"3.54.1"},"reference-count":19,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,7,24]],"date-time":"2025-07-24T00:00:00Z","timestamp":1753315200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001469","name":"Singapore General Hospital","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100001469","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:sec><jats:title>Introduction<\/jats:title><jats:p>Thyroid nodule segmentation in ultrasound (US) images is a valuable yet challenging task, playing a critical role in diagnosing thyroid cancer. The difficulty arises from factors such as the absence of prior knowledge about the thyroid region, low contrast between anatomical structures, and speckle noise, all of which obscure boundary detection and introduce variability in nodule appearance across different images.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>To address these challenges, we propose a transformer-based model for thyroid nodule segmentation. Unlike traditional convolutional neural networks (CNNs), transformers capture global context from the first layer, enabling more comprehensive image representation, which is crucial for identifying subtle nodule boundaries. In this study, We first pre-train a Masked Autoencoder (MAE) to reconstruct masked patches, then fine-tune on thyroid US data, and further explore a cross-attention mechanism to enhance information flow between encoder and decoder.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>Our experiments on the public AIMI, TN3K, and DDTI datasets show that MAE pre-training accelerates convergence. However, overall improvements are modest: the model achieves Dice Similarity Coefficient (DSC) scores of 0.63, 0.64, and 0.65 on AIMI, TN3K, and DDTI, respectively, highlighting limitations under small-sample conditions. Furthermore, adding cross-attention did not yield consistent gains, suggesting that data volume and diversity may be more critical than additional architectural complexity.<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussion<\/jats:title><jats:p>MAE pre-training notably reduces training time and helps themodel learn transferable features, yet overall accuracy remains constrained by limited data and nodule variability. Future work will focus on scaling up data, pre-training cross-attention layers, and exploring hybrid architectures to further boost segmentation performance.<\/jats:p><\/jats:sec>","DOI":"10.3389\/frai.2025.1618426","type":"journal-article","created":{"date-parts":[[2025,7,24]],"date-time":"2025-07-24T05:33:13Z","timestamp":1753335193000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["Thyroid nodule segmentation in ultrasound images using transformer models with masked autoencoder pre-training"],"prefix":"10.3389","volume":"8","author":[{"given":"Yi","family":"Xiang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rajendra","family":"Acharya","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Quan","family":"Le","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jen Hong","family":"Tan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chiaw-Ling","family":"Chng","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2025,7,24]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"107614","DOI":"10.1016\/j.cmpb.2023.107614","article-title":"Bpat-unet: boundary preserving assembled transformer unet for ultrasound thyroid nodule segmentation","volume":"238","author":"Bi","year":"2023","journal-title":"Comput. Methods Programs Biomed"},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2102.04306","article-title":"Transunet: transformers make strong encoders for medical image segmentation","author":"Chen","year":"2021","journal-title":"arXiv"},{"key":"B3","doi-asserted-by":"publisher","first-page":"901","DOI":"10.1016\/j.beem.2008.09.019","article-title":"Epidemiology of thyroid nodules","volume":"22","author":"Dean","year":"2008","journal-title":"Best Pract. Res. Clin. Endocrinol. Metab"},{"key":"B4","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2010.11929","article-title":"An image is worth 16x16 words: transformers for image recognition at scale","author":"Dosovitskiy","year":"2020","journal-title":"arXiv"},{"key":"B5","doi-asserted-by":"publisher","first-page":"167","DOI":"10.1023\/B:VISI.0000022288.19776.77","article-title":"Efficient graph-based image segmentation","volume":"59","author":"Felzenszwalb","year":"2004","journal-title":"Int. J. Comput. Vis"},{"key":"B6","doi-asserted-by":"publisher","first-page":"257","DOI":"10.1109\/ISBI48211.2021.9434087","article-title":"\u201cMulti-task learning for thyroid nodule segmentation with thyroid region prior,\u201d","author":"Gong","year":"2021"},{"key":"B7","volume-title":"Pattern Classification","author":"Hart","year":"2000"},{"key":"B8","doi-asserted-by":"publisher","first-page":"16000","DOI":"10.1109\/CVPR52688.2022.01553","author":"He","year":"2022"},{"key":"B9","doi-asserted-by":"publisher","first-page":"266","DOI":"10.1016\/j.ultras.2011.08.011","article-title":"A robust graph-based segmentation method for breast tumors in ultrasound images","volume":"52","author":"Huang","year":"2012","journal-title":"Ultrasonics"},{"key":"B10","first-page":"3431","article-title":"\u201cFully convolutional networks for semantic segmentation,\u201d","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Long","year":"2015"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1804.03999","article-title":"Attention u-net: learning where to look for the pancreas","author":"Oktay","year":"2018","journal-title":"arXiv"},{"key":"B12","doi-asserted-by":"publisher","first-page":"188","DOI":"10.1117\/12.2073532","author":"Pedraza","year":"2015"},{"key":"B13","doi-asserted-by":"publisher","first-page":"267","DOI":"10.1007\/978-3-030-87589-3_28","author":"Petit","year":"2021"},{"key":"B14","doi-asserted-by":"publisher","first-page":"107974","DOI":"10.1016\/j.compbiomed.2024.107974","article-title":"A multitask approach for automated detection and segmentation of thyroid nodules in ultrasound images","volume":"170","author":"Radhachandran","year":"2024","journal-title":"Comput. Biol. Med"},{"key":"B15","first-page":"1","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel","year":"2020","journal-title":"J. Mach. Learn. Res"},{"key":"B16","first-page":"234","author":"Ronneberger","year":"2015"},{"key":"B17","doi-asserted-by":"publisher","first-page":"150","DOI":"10.1089\/thy.2022.0560","article-title":"Artificial intelligence for evaluation of thyroid nodules: a primer","volume":"33","author":"Tessler","year":"2023","journal-title":"Thyroid"},{"key":"B18","author":"Vaswani","year":"2017"},{"key":"B19","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.ultras.2018.07.006","article-title":"Medical breast ultrasound image segmentation by machine learning","volume":"91","author":"Xu","year":"2019","journal-title":"Ultrasonics"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2025.1618426\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,24]],"date-time":"2025-07-24T05:33:15Z","timestamp":1753335195000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2025.1618426\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,24]]},"references-count":19,"alternative-id":["10.3389\/frai.2025.1618426"],"URL":"https:\/\/doi.org\/10.3389\/frai.2025.1618426","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,24]]},"article-number":"1618426"}}