{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T22:23:07Z","timestamp":1784326987398,"version":"3.55.0"},"reference-count":27,"publisher":"The Endocrine Society","issue":"12","license":[{"start":{"date-parts":[[2021,6,23]],"date-time":"2021-06-23T00:00:00Z","timestamp":1624406400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100017559","name":"Garron Family Cancer Centre","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100017559","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000024","name":"Canadian Institutes of Health Research","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100000024","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100007631","name":"Canadian Institute for Advanced Research","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100007631","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100008762","name":"Genome Canada","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100008762","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,11,19]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>To develop a machine learning tool to integrate clinical data for the prediction of non-benign thyroid cytology and histology.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Context<\/jats:title>\n                  <jats:p>Papillary thyroid carcinoma is the most common endocrine malignancy. Since most nodules are benign, the challenge for the clinician is to identify those most likely to harbor malignancy while limiting exposure to surgical risks among those with benign nodules.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Methods<\/jats:title>\n                  <jats:p>Random forests (augmented to select features based on our clinical measure of interest), in conjunction with interpretable rule sets, were used on demographic, ultrasound, and biopsy data of thyroid nodules from children younger than 18 years at a tertiary pediatric hospital. Accuracy, false-positive rate (FPR), false-negative rate (FNR), and area under the receiver operator curve (AUROC) are reported.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>Our models predict nonbenign cytology and malignant histology better than historical outcomes. Specifically, we expect a 68.04% improvement in the FPR, 11.90% increase in accuracy, and 24.85% increase in AUROC for biopsy predictions in 67 patients (28 with benign and 39 with nonbenign histology). We expect a 23.22% decrease in FPR, 32.19% increase in accuracy, and 3.84% decrease in AUROC for surgery prediction in 53 patients (42 with benign and 11 with nonbenign histology). This improvement comes at the expense of the FNR, for which we expect 10.27% with malignancy would be discouraged from performing biopsy, and 11.67% from surgery. Given the small number of patients, these improvements are estimates and are not tested on an independent test set.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>This work presents a first attempt at developing an interpretable machine learning based clinical tool to aid clinicians. Future work will involve sourcing more data and developing probabilistic estimates for predictions.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1210\/clinem\/dgab435","type":"journal-article","created":{"date-parts":[[2021,6,23]],"date-time":"2021-06-23T13:32:58Z","timestamp":1624455178000},"page":"e5236-e5246","source":"Crossref","is-referenced-by-count":13,"title":["Predicting Malignancy in Pediatric Thyroid Nodules: Early Experience With Machine Learning for Clinical Decision Support"],"prefix":"10.1210","volume":"106","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3184-9165","authenticated-orcid":false,"given":"Lebohang","family":"Radebe","sequence":"first","affiliation":[{"name":"Genetics and Genome Biology Program, The Hospital for Sick Children, Toronto, Ontario, Canada"},{"name":"Department of Computer Science, University of Toronto, Toronto, Ontario, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dani\u00eblle C M","family":"van der Kaay","sequence":"additional","affiliation":[{"name":"Erasmus Medical Center - Sophia Children\u2019s Hospital, Rotterdam CB, the Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7088-8146","authenticated-orcid":false,"given":"Jonathan D","family":"Wasserman","sequence":"additional","affiliation":[{"name":"Division of Endocrinology, The Hospital for Sick Children, Toronto, Ontario, Canada"},{"name":"Department of Paediatrics, Faculty of Medicine, The University of Toronto, Ontario, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2416-833X","authenticated-orcid":false,"given":"Anna","family":"Goldenberg","sequence":"additional","affiliation":[{"name":"Genetics and Genome Biology Program, The Hospital for Sick Children, Toronto, Ontario, Canada"},{"name":"Department of Computer Science, University of Toronto, Toronto, Ontario, Canada"},{"name":"Vector Institute, Toronto, Ontario, Canada"},{"name":"CIFAR, Toronto, Ontario, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"80","published-online":{"date-parts":[[2021,6,23]]},"reference":[{"issue":"6","key":"2022020822521858000_CIT0001","doi-asserted-by":"crossref","first-page":"1481","DOI":"10.1016\/j.jpeds.2014.01.059","article-title":"Increase in the incidence of differentiated thyroid carcinoma in children, adolescents, and young adults: a population-based study","volume":"164","author":"Vergamini","year":"2014","journal-title":"J Pediatr."},{"issue":"8","key":"2022020822521858000_CIT0002","doi-asserted-by":"crossref","first-page":"3238","DOI":"10.1210\/jc.2013-1796","article-title":"A standardized assessment of thyroid nodules in children confirms higher cancer prevalence than in adults","volume":"98","author":"Gupta","year":"2013","journal-title":"J Clin Endocrinol Metab."},{"issue":"3","key":"2022020822521858000_CIT0003","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1111\/cen.12786","article-title":"Accuracy of thyroid nodule sonography for the detection of thyroid cancer in children: systematic review and meta-analysis","volume":"84","author":"Al Nofal","year":"2016","journal-title":"Clin Endocrinol (Oxf)."},{"issue":"1","key":"2022020822521858000_CIT0004","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1007\/s00247-017-3974-y","article-title":"Utility of adult-based ultrasound malignancy risk stratifications in pediatric thyroid nodules","volume":"48","author":"Martinez-Rios","year":"2018","journal-title":"Pediatr Radiol."},{"issue":"5","key":"2022020822521858000_CIT0005","doi-asserted-by":"crossref","first-page":"1958","DOI":"10.1210\/jc.2017-02464","article-title":"Diagnostic accuracy of ultrasound with color flow Doppler in children with thyroid nodules","volume":"103","author":"Gannon","year":"2018","journal-title":"J Clin Endocrinol Metab."},{"issue":"8","key":"2022020822521858000_CIT0006","doi-asserted-by":"crossref","first-page":"1097","DOI":"10.1089\/thy.2018.0728","article-title":"Differences in thyroid nodule cytology and malignancy risk between children and adults","volume":"29","author":"Cherella","year":"2019","journal-title":"Thyroid."},{"issue":"11","key":"2022020822521858000_CIT0007","doi-asserted-by":"crossref","first-page":"801","DOI":"10.1002\/cncy.21750","article-title":"Pediatric thyroid FNA biopsy: outcomes and impact on management over 24 years at a tertiary care center","volume":"124","author":"Amirazodi","year":"2016","journal-title":"Cancer Cytopathol."},{"issue":"5","key":"2022020822521858000_CIT0008","doi-asserted-by":"crossref","first-page":"657","DOI":"10.1530\/EJE-20-1480","article-title":"Clinical utility of sonographic features in indeterminate pediatric thyroid nodules","volume":"184","author":"Richman","year":"2021","journal-title":"Eur J Endocrinol."},{"issue":"5","key":"2022020822521858000_CIT0009","doi-asserted-by":"crossref","first-page":"826","DOI":"10.1016\/j.jpedsurg.2007.12.019","article-title":"The Canadian Pediatric Thyroid Nodule Study: an evaluation of current management practices","volume":"43","author":"Canadian Pediatric Thyroid Nodule (CaPTN) Study Group","year":"2008","journal-title":"J Pediatr Surg."},{"key":"2022020822521858000_CIT0010","first-page":"S1","article-title":"McGill Thyroid Nodule Score (MTNS):\u201crating the risk,\u201d a novel predictive scheme for cancer risk determination","volume":"40","author":"Sands","year":"2011","journal-title":"J Otolaryngol Head Neck Surg."},{"issue":"5","key":"2022020822521858000_CIT0011","doi-asserted-by":"crossref","first-page":"852","DOI":"10.1089\/thy.2013.0191","article-title":"Serum thyroglobulin improves the sensitivity of the McGill Thyroid Nodule Score for well-differentiated thyroid cancer","volume":"24","author":"Scheffler","year":"2014","journal-title":"Thyroid."},{"issue":"4","key":"2022020822521858000_CIT0012","doi-asserted-by":"crossref","first-page":"589","DOI":"10.1177\/0194599817715629","article-title":"McGill Thyroid Nodule Score in differentiating benign and malignant pediatric thyroid nodules: a pilot study","volume":"157","author":"Canfarotta","year":"2017","journal-title":"Otolaryngol Head Neck Surg."},{"issue":"1","key":"2022020822521858000_CIT0013","doi-asserted-by":"crossref","first-page":"200","DOI":"10.1111\/cen.13878","article-title":"Diagnostic accuracy of the McGill Thyroid Nodule Score in paediatric patients","volume":"90","author":"Creo","year":"2019","journal-title":"Clin Endocrinol (Oxf)."},{"issue":"3 Pt B","key":"2022020822521858000_CIT0014","doi-asserted-by":"crossref","first-page":"512","DOI":"10.1016\/j.jacr.2017.12.028","article-title":"Machine learning in medical imaging","volume":"15","author":"Giger","year":"2018","journal-title":"J Am Coll Radiol."},{"issue":"6","key":"2022020822521858000_CIT0015","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1038\/nrg3920","article-title":"Machine learning applications in genetics and genomics","volume":"16","author":"Libbrecht","year":"2015","journal-title":"Nat Rev Genet."},{"issue":"2","key":"2022020822521858000_CIT0016","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1016\/j.hlpt.2018.04.006","article-title":"Why policymakers should care about \u201cbig data\u201d in healthcare","volume":"7","author":"Bates","year":"2018","journal-title":"Health Policy Technol."},{"issue":"1","key":"2022020822521858000_CIT0017","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach Learn."},{"key":"2022020822521858000_CIT0018","doi-asserted-by":"crossref","first-page":"456","DOI":"10.1016\/j.bspc.2017.01.012","article-title":"Decision tree and random forest models for outcome prediction in antibody incompatible kidney transplantation","volume":"52","author":"Shaikhina","year":"2019","journal-title":"Biomed Signal Process Control."},{"issue":"10","key":"2022020822521858000_CIT0019","doi-asserted-by":"crossref","first-page":"2811","DOI":"10.1245\/s10434-008-0034-8","article-title":"Molecular phenotyping of thyroid tumors identifies a marker panel for differentiated thyroid cancer diagnosis","volume":"15","author":"Wiseman","year":"2008","journal-title":"Ann Surg Oncol."},{"issue":"6","key":"2022020822521858000_CIT0020","doi-asserted-by":"crossref","first-page":"858","DOI":"10.1089\/thy.2018.0380","article-title":"Machine learning-assisted system for thyroid nodule diagnosis","volume":"29","author":"Zhang","year":"2019","journal-title":"Thyroid."},{"issue":"4","key":"2022020822521858000_CIT0021","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1007\/s41060-018-0144-8","article-title":"Interpreting tree ensembles with inTrees","volume":"7","author":"Deng","year":"2019","journal-title":"Int J Data Sci Anal."},{"issue":"Mar","key":"2022020822521858000_CIT0022","first-page":"1157","article-title":"An introduction to variable and feature selection","volume":"3","author":"Guyon","year":"2003","journal-title":"J Mach Learn Res."},{"issue":"7","key":"2022020822521858000_CIT0023","doi-asserted-by":"crossref","first-page":"716","DOI":"10.1089\/thy.2014.0460","article-title":"Management guidelines for children with thyroid nodules and differentiated thyroid cancer","volume":"25","author":"Francis","year":"2015","journal-title":"Thyroid."},{"key":"2022020822521858000_CIT0024","author":"Radebe"},{"key":"2022020822521858000_CIT0025"},{"issue":"3","key":"2022020822521858000_CIT0026","first-page":"18","article-title":"Classification and regression by randomForest","volume":"2","author":"Liaw","year":"2002","journal-title":"R News"},{"key":"2022020822521858000_CIT0027","article-title":"Interpreting Tree Ensembles with inTrees. Technical Report, 2014","author":"Houtao"}],"container-title":["The Journal of Clinical Endocrinology &amp; Metabolism"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jcem\/advance-article-pdf\/doi\/10.1210\/clinem\/dgab435\/40532992\/dgab435.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jcem\/article-pdf\/106\/12\/e5236\/42380673\/dgab435.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jcem\/article-pdf\/106\/12\/e5236\/42380673\/dgab435.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,2,8]],"date-time":"2022-02-08T22:53:40Z","timestamp":1644360820000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jcem\/article\/106\/12\/e5236\/6308313"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,23]]},"references-count":27,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2021,6,23]]},"published-print":{"date-parts":[[2021,11,19]]}},"URL":"https:\/\/doi.org\/10.1210\/clinem\/dgab435","relation":{},"ISSN":["0021-972X","1945-7197"],"issn-type":[{"value":"0021-972X","type":"print"},{"value":"1945-7197","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,12,1]]},"published":{"date-parts":[[2021,6,23]]}}}