{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,14]],"date-time":"2026-10-14T16:13:15Z","timestamp":1791994395387,"version":"4.4.0"},"reference-count":24,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,12,21]],"date-time":"2022-12-21T00:00:00Z","timestamp":1671580800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,12,21]],"date-time":"2022-12-21T00:00:00Z","timestamp":1671580800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"the Korea Medical Device Development Fund grant funded by the Korea government","award":["1711174345"],"award-info":[{"award-number":["1711174345"]}]},{"name":"the Korea Medical Device Development Fund grant funded by the Korea government","award":["KMDF_PR_20200901_0114"],"award-info":[{"award-number":["KMDF_PR_20200901_0114"]}]},{"name":"the Promotion of Innovative Businesses for Regulation-Free Special Zones funded by the Ministry of SMEs and Startups","award":["1425156792"],"award-info":[{"award-number":["1425156792"]}]},{"name":"the Promotion of Innovative Businesses for Regulation-Free Special Zones funded by the Ministry of SMEs and Startups","award":["P0016195"],"award-info":[{"award-number":["P0016195"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Although the clinical activity score (CAS) is a validated scoring system for identifying disease activity of thyroid-associated orbitopathy (TAO), it may produce differing results depending on the evaluator, and an experienced ophthalmologist is required for accurate evaluation. In this study, we developed a machine learning (ML)-assisted system to mimic an expert\u2019s CAS assessment using digital facial images and evaluated its accuracy for predicting the CAS and diagnosing active TAO (CAS\u2009\u2265\u20093). An ML-assisted system was designed to assess five CAS components related to inflammatory signs (redness of the eyelids, redness of the conjunctiva, swelling of the eyelids, inflammation of the caruncle and\/or plica, and conjunctival edema) in patients\u2019 facial images and to predict the CAS by considering two components of subjective symptoms (spontaneous retrobulbar pain and pain on gaze). To train and test the system, 3,060 cropped images from 1020 digital facial images of TAO patients were used. The reference CAS for each image was scored by three ophthalmologists, each with\u2009&gt;\u200915\u00a0years of clinical experience. We repeated the experiments for 30 randomly split training and test sets at a ratio of 8:2. The sensitivity and specificity of the ML-assisted system for diagnosing active TAO were 72.7% and 83.2% in the test set constructed from the entire dataset. For the test set constructed from the dataset with consistent results for the three ophthalmologists, the sensitivity and specificity for diagnosing active TAO were 88.1% and 86.9%. In the test sets from the entire dataset and from the dataset with consistent results, 40.0% and 49.9% of the predicted CAS values were the same as the reference CAS, respectively. The system predicted the CAS within 1 point of the reference CAS in 84.6% and 89.0% of cases when tested using the entire dataset and in the dataset with consistent results, respectively. An ML-assisted system estimated the clinical activity of TAO and detect inflammatory active TAO with reasonable accuracy. The accuracy could be improved further by obtaining more data. This ML-assisted system can help evaluate the disease activity consistently as well as accurately and enable the early diagnosis and timely treatment of active TAO.<\/jats:p>","DOI":"10.1038\/s41598-022-25887-8","type":"journal-article","created":{"date-parts":[[2022,12,21]],"date-time":"2022-12-21T09:05:35Z","timestamp":1671613535000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":39,"title":["Machine learning-assisted system using digital facial images to predict the clinical activity score in thyroid-associated orbitopathy"],"prefix":"10.1038","volume":"12","author":[{"given":"Jae Hoon","family":"Moon","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kyubo","family":"Shin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gyeong Min","family":"Lee","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jaemin","family":"Park","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Min Joung","family":"Lee","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hokyung","family":"Choung","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Namju","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,12,21]]},"reference":[{"key":"25887_CR1","doi-asserted-by":"publisher","first-page":"16","DOI":"10.3390\/jcm10010016","volume":"10","author":"K Gontarz-Nowak","year":"2020","unstructured":"Gontarz-Nowak, K., Szychlinska, M., Matuszewski, W., Stefanowicz-Rutkowska, M. & Bandurska-Stankiewicz, E. Current knowledge on graves\u2019 orbitopathy. J. Clin. Med. 10, 16. https:\/\/doi.org\/10.3390\/jcm10010016 (2020).","journal-title":"J. Clin. Med."},{"key":"25887_CR2","doi-asserted-by":"publisher","first-page":"353","DOI":"10.2169\/internalmedicine.53.1518","volume":"53","author":"Y Hiromatsu","year":"2014","unstructured":"Hiromatsu, Y., Eguchi, H., Tani, J., Kasaoka, M. & Teshima, Y. Graves\u2019 ophthalmopathy: Epidemiology and natural history. Intern. Med. 53, 353\u2013360. https:\/\/doi.org\/10.2169\/internalmedicine.53.1518 (2014).","journal-title":"Intern. Med."},{"key":"25887_CR3","doi-asserted-by":"publisher","first-page":"1443","DOI":"10.1210\/jc.2012-3873","volume":"98","author":"ML Tanda","year":"2013","unstructured":"Tanda, M. L. et al. Prevalence and natural history of Graves\u2019 orbitopathy in a large series of patients with newly diagnosed graves\u2019 hyperthyroidism seen at a single center. J. Clin. Endocrinol. Metab. 98, 1443\u20131449. https:\/\/doi.org\/10.1210\/jc.2012-3873 (2013).","journal-title":"J. Clin. Endocrinol. Metab."},{"key":"25887_CR4","doi-asserted-by":"publisher","first-page":"2830","DOI":"10.1210\/jcem.80.9.7673432","volume":"80","author":"MC Villadolid","year":"1995","unstructured":"Villadolid, M. C. et al. Untreated Graves\u2019 disease patients without clinical ophthalmopathy demonstrate a high frequency of extraocular muscle (EOM) enlargement by magnetic resonance. J. Clin. Endocrinol. Metab. 80, 2830\u20132833. https:\/\/doi.org\/10.1210\/jcem.80.9.7673432 (1995).","journal-title":"J. Clin. Endocrinol. Metab."},{"key":"25887_CR5","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1007\/s40123-019-00226-9","volume":"9","author":"NM Hodgson","year":"2020","unstructured":"Hodgson, N. M. & Rajaii, F. Current understanding of the progression and management of thyroid associated orbitopathy: A systematic review. Ophthalmol. Ther. 9, 21\u201333. https:\/\/doi.org\/10.1007\/s40123-019-00226-9 (2020).","journal-title":"Ophthalmol. Ther."},{"key":"25887_CR6","doi-asserted-by":"publisher","first-page":"396","DOI":"10.1038\/ncpendmet0497","volume":"3","author":"WM Wiersinga","year":"2007","unstructured":"Wiersinga, W. M. Management of Graves\u2019 ophthalmopathy. Nat. Clin. Pract. Endocrinol. Metab. 3, 396\u2013404. https:\/\/doi.org\/10.1038\/ncpendmet0497 (2007).","journal-title":"Nat. Clin. Pract. Endocrinol. Metab."},{"key":"25887_CR7","doi-asserted-by":"publisher","first-page":"G43","DOI":"10.1530\/EJE-21-0479","volume":"185","author":"L Bartalena","year":"2021","unstructured":"Bartalena, L. et al. The 2021 European Group on Graves\u2019 orbitopathy (EUGOGO) clinical practice guidelines for the medical management of Graves\u2019 orbitopathy. Eur. J. Endocrinol. 185, G43\u2013G67. https:\/\/doi.org\/10.1530\/EJE-21-0479 (2021).","journal-title":"Eur. J. Endocrinol."},{"key":"25887_CR8","doi-asserted-by":"publisher","first-page":"229","DOI":"10.1016\/j.beem.2011.11.007","volume":"26","author":"PJ Dolman","year":"2012","unstructured":"Dolman, P. J. Evaluating Graves\u2019 orbitopathy. Best Pract. Res. Clin. Endocrinol. Metab. 26, 229\u2013248. https:\/\/doi.org\/10.1016\/j.beem.2011.11.007 (2012).","journal-title":"Best Pract. Res. Clin. Endocrinol. Metab."},{"key":"25887_CR9","doi-asserted-by":"publisher","first-page":"639","DOI":"10.1136\/bjo.73.8.639","volume":"73","author":"MP Mourits","year":"1989","unstructured":"Mourits, M. P. et al. Clinical criteria for the assessment of disease activity in Graves\u2019 ophthalmopathy: A novel approach. Br. J. Ophthalmol. 73, 639\u2013644. https:\/\/doi.org\/10.1136\/bjo.73.8.639 (1989).","journal-title":"Br. J. Ophthalmol."},{"key":"25887_CR10","doi-asserted-by":"publisher","first-page":"333","DOI":"10.1089\/thy.2007.0315","volume":"18","author":"L Bartalena","year":"2008","unstructured":"Bartalena, L. et al. Consensus statement of the European group on Graves\u2019 orbitopathy (EUGOGO) on management of Graves\u2019 orbitopathy. Thyroid 18, 333\u2013346. https:\/\/doi.org\/10.1089\/thy.2007.0315 (2008).","journal-title":"Thyroid"},{"key":"25887_CR11","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1038\/s41746-020-00376-2","volume":"4","author":"A Esteva","year":"2021","unstructured":"Esteva, A. et al. Deep learning-enabled medical computer vision. NPJ Digit. Med. 4, 5. https:\/\/doi.org\/10.1038\/s41746-020-00376-2 (2021).","journal-title":"NPJ Digit. Med."},{"key":"25887_CR12","doi-asserted-by":"publisher","first-page":"1531","DOI":"10.1136\/bjophthalmol-2015-306733","volume":"99","author":"P Perros","year":"2015","unstructured":"Perros, P. et al. PREGO (presentation of Graves\u2019 orbitopathy) study: Changes in referral patterns to European Group On Graves\u2019 Orbitopathy (EUGOGO) centres over the period from 2000 to 2012. Br. J. Ophthalmol. 99, 1531\u20131535. https:\/\/doi.org\/10.1136\/bjophthalmol-2015-306733 (2015).","journal-title":"Br. J. Ophthalmol."},{"key":"25887_CR13","doi-asserted-by":"publisher","unstructured":"Bartalena, L., Piantanida, E., Gallo, D., Lai, A. & Tanda, M. L. Epidemiology, natural history, risk factors, and prevention of graves' orbitopathy. Front. Endocrinol. (Lausanne) 11, 615993. https:\/\/doi.org\/10.3389\/fendo.2020.615993 (2020).","DOI":"10.3389\/fendo.2020.615993"},{"key":"25887_CR14","doi-asserted-by":"publisher","first-page":"1707","DOI":"10.1007\/s40618-020-01479-z","volume":"44","author":"S Ippolito","year":"2021","unstructured":"Ippolito, S. et al. Change in newly diagnosed Graves\u2019 disease phenotype between the twentieth and the twenty-first centuries: Meta-analysis and meta-regression. J. Endocrinol. Invest. 44, 1707\u20131718. https:\/\/doi.org\/10.1007\/s40618-020-01479-z (2021).","journal-title":"J. Endocrinol. Invest."},{"key":"25887_CR15","doi-asserted-by":"publisher","first-page":"635","DOI":"10.1530\/EJE-18-0039","volume":"178","author":"W Wiersinga","year":"2018","unstructured":"Wiersinga, W. et al. Predictive score for the development or progression of Graves\u2019 orbitopathy in patients with newly diagnosed Graves\u2019 hyperthyroidism. Eur. J. Endocrinol. 178, 635\u2013643. https:\/\/doi.org\/10.1530\/EJE-18-0039 (2018).","journal-title":"Eur. J. Endocrinol."},{"key":"25887_CR16","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1159\/000443828","volume":"5","author":"L Bartalena","year":"2016","unstructured":"Bartalena, L. et al. The 2016 European Thyroid Association\/European Group on Graves\u2019 Orbitopathy Guidelines for the Management of Graves\u2019 Orbitopathy. Eur Thyroid J 5, 9\u201326. https:\/\/doi.org\/10.1159\/000443828 (2016).","journal-title":"Eur Thyroid J"},{"key":"25887_CR17","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1111\/j.1365-2265.2005.02186.x","volume":"62","author":"CB Terwee","year":"2005","unstructured":"Terwee, C. B. et al. Measuring disease activity to predict therapeutic outcome in Graves\u2019 ophthalmopathy. Clin. Endocrinol. (Oxf) 62, 145\u2013155. https:\/\/doi.org\/10.1111\/j.1365-2265.2005.02186.x (2005).","journal-title":"Clin. Endocrinol. (Oxf)"},{"key":"25887_CR18","first-page":"177","volume":"5","author":"FF Rundle","year":"1945","unstructured":"Rundle, F. F. & Wilson, C. W. Development and course of exophthalmos and ophthalmoplegia in Graves\u2019 disease with special reference to the effect of thyroidectomy. Clin Sci 5, 177\u2013194 (1945).","journal-title":"Clin Sci"},{"key":"25887_CR19","first-page":"36","volume":"6","author":"FF Rundle","year":"1957","unstructured":"Rundle, F. F. Management of exophthalmos and related ocular changes in Graves\u2019 disease. Metabolis 6, 36\u201348 (1957).","journal-title":"Metabolis"},{"key":"25887_CR20","doi-asserted-by":"publisher","DOI":"10.1136\/bjo-2022-321833","author":"J Karlin","year":"2022","unstructured":"Karlin, J. et al. Ensemble neural network model for detecting thyroid eye disease using external photographs. Br. J. Ophthalmol. https:\/\/doi.org\/10.1136\/bjo-2022-321833 (2022).","journal-title":"Br. J. Ophthalmol."},{"key":"25887_CR21","doi-asserted-by":"publisher","unstructured":"Huang, X. et al. An Intelligent diagnostic system for thyroid-associated ophthalmopathy based on facial images. Front. Med. (Lausanne) 9, 920716. https:\/\/doi.org\/10.3389\/fmed.2022.920716 (2022).","DOI":"10.3389\/fmed.2022.920716"},{"key":"25887_CR22","volume-title":"Pattern recognition and machine learning","author":"CM Bishop","year":"2016","unstructured":"Bishop, C. M. Pattern recognition and machine learning (Springer, 2016)."},{"key":"25887_CR23","doi-asserted-by":"publisher","first-page":"387","DOI":"10.1530\/eje.1.02230","volume":"155","author":"WM Wiersinga","year":"2006","unstructured":"Wiersinga, W. M. et al. Clinical assessment of patients with Graves\u2019 orbitopathy: The European Group on Graves\u2019 Orbitopathy recommendations to generalists, specialists and clinical researchers. Eur. J. Endocrinol. 155, 387\u2013389. https:\/\/doi.org\/10.1530\/eje.1.02230 (2006).","journal-title":"Eur. J. Endocrinol."},{"key":"25887_CR24","doi-asserted-by":"publisher","first-page":"283","DOI":"10.1046\/j.1365-2265.2001.01349.x","volume":"55","author":"AJ Dickinson","year":"2001","unstructured":"Dickinson, A. J. & Perros, P. Controversies in the clinical evaluation of active thyroid-associated orbitopathy: Use of a detailed protocol with comparative photographs for objective assessment. Clin. Endocrinol. (Oxf) 55, 283\u2013303. https:\/\/doi.org\/10.1046\/j.1365-2265.2001.01349.x (2001).","journal-title":"Clin. Endocrinol. (Oxf)"}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-022-25887-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-022-25887-8","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-022-25887-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,21]],"date-time":"2022-12-21T09:10:31Z","timestamp":1671613831000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-022-25887-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,21]]},"references-count":24,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["25887"],"URL":"https:\/\/doi.org\/10.1038\/s41598-022-25887-8","relation":{},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,21]]},"assertion":[{"value":"29 July 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 December 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 December 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"THYROSCOPE INC. has a patent pending for the ML-assisted system to predict the clinical activity of TAO. JHM, JP are the stock owners of THYROSCOPE INC. KS is the board member of THYROSCOPE INC. MJL, HC, and NK are members of the medical advisory board of THYROSCOPE INC. GML declares no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"22085"}}