{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,26]],"date-time":"2026-06-26T12:44:13Z","timestamp":1782477853139,"version":"3.54.5"},"reference-count":54,"publisher":"Oxford University Press (OUP)","issue":"4","license":[{"start":{"date-parts":[[2020,12,19]],"date-time":"2020-12-19T00:00:00Z","timestamp":1608336000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Korea government","award":["NRF-2019R1F1A1062023"],"award-info":[{"award-number":["NRF-2019R1F1A1062023"]}]},{"name":"Korea government","award":["NRF-2020M3A9D8038014"],"award-info":[{"award-number":["NRF-2020M3A9D8038014"]}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000054","name":"National Cancer Institute","doi-asserted-by":"publisher","award":["P30 CA008748"],"award-info":[{"award-number":["P30 CA008748"]}],"id":[{"id":"10.13039\/100000054","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000054","name":"National Cancer Institute","doi-asserted-by":"publisher","award":["R21 CA234752"],"award-info":[{"award-number":["R21 CA234752"]}],"id":[{"id":"10.13039\/100000054","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,7,20]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Thyroid nodules are neoplasms commonly found among adults, with papillary thyroid carcinoma (PTC) being the most prevalent malignancy. However, current diagnostic methods often subject patients to unnecessary surgical burden. In this study, we developed and validated an automated, highly accurate multi-study-derived diagnostic model for PTCs using personalized biological pathways coupled with a sophisticated machine learning algorithm. Surprisingly, the algorithm achieved near-perfect performance in discriminating PTCs from non-tumoral thyroid samples with an overall cross-study-validated area under the receiver operating characteristic curve (AUROC) of 0.999 (95% confidence interval [CI]: 0.995\u20131) and a Brier score of 0.013 on three independent development cohorts. In addition, the algorithm showed excellent generalizability and transferability on two large-scale external blind PTC cohorts consisting of The Cancer Genome Atlas (TCGA), which is the largest genomic PTC cohort studied to date, and the post-Chernobyl cohort, which includes PTCs reported after exposure to radiation from the Chernobyl accident. When applied to the TCGA cohort, the model yielded an AUROC of 0.969 (95% CI: 0.950\u20130.987) and a Brier score of 0.109. On the post-Chernobyl cohort, it yielded an AUROC of 0.962 (95% CI: 0.918\u20131) and a Brier score of 0.073. This algorithm also is robust against other various types of clinical scenarios, discriminating malignant from benign lesions as well as clinically aggressive thyroid cancer with poor prognosis from indolent ones. Furthermore, we discovered novel pathway alterations and prognostic signatures for PTC, which can provide directions for follow-up studies.<\/jats:p>","DOI":"10.1093\/bib\/bbaa336","type":"journal-article","created":{"date-parts":[[2020,10,28]],"date-time":"2020-10-28T12:09:43Z","timestamp":1603886983000},"source":"Crossref","is-referenced-by-count":12,"title":["Highly accurate diagnosis of papillary thyroid carcinomas based on personalized pathways coupled with machine learning"],"prefix":"10.1093","volume":"22","author":[{"given":"Kyoung Sik","family":"Park","sequence":"first","affiliation":[{"name":"Konkuk University School of Medicine, South Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Seong Hoon","family":"Kim","sequence":"additional","affiliation":[{"name":"Konkuk University School of Medicine, South Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jung Hun","family":"Oh","sequence":"additional","affiliation":[{"name":"Department of Medical Physics at Memorial Sloan Kettering Cancer Center, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sung Young","family":"Kim","sequence":"additional","affiliation":[{"name":"Konkuk University School of Medicine, South Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2020,12,19]]},"reference":[{"key":"2021111804322890400_ref1","doi-asserted-by":"crossref","DOI":"10.1155\/2013\/965212","article-title":"Worldwide increasing incidence of thyroid cancer: update on epidemiology and risk factors","volume":"2013","author":"Pellegriti","year":"2013","journal-title":"J Cancer Epidemiol"},{"key":"2021111804322890400_ref2","doi-asserted-by":"crossref","DOI":"10.1186\/s12885-017-3104-0","article-title":"Thyroid fine-needle aspiration biopsy positively correlates with increased diagnosis of thyroid cancer in South Korean patients","volume":"17","author":"Cho","year":"2017","journal-title":"BMC Cancer"},{"key":"2021111804322890400_ref3","doi-asserted-by":"crossref","first-page":"1121","DOI":"10.5858\/arpa.2015-0154-SA","article-title":"Application of the Bethesda classification for thyroid fine-needle aspiration: institutional experience and meta-analysis","volume":"140","author":"Krauss","year":"2016","journal-title":"Arch Pathol Lab Med"},{"key":"2021111804322890400_ref4","doi-asserted-by":"crossref","first-page":"490","DOI":"10.1016\/j.otohns.2004.09.028","article-title":"Surgical complications after thyroid surgery performed in a cancer hospital","volume":"132","author":"Gon\u00e7alves Filho","year":"2005","journal-title":"Otolaryngol Head Neck Surg"},{"key":"2021111804322890400_ref5","doi-asserted-by":"crossref","first-page":"274","DOI":"10.1097\/MD.0b013e31826a9c71","article-title":"BRAF mutation in papillary thyroid cancer and its value in tailoring initial treatment: a systematic review and meta-analysis","volume":"91","author":"Tufano","year":"2012","journal-title":"Medicine (Baltimore)"},{"key":"2021111804322890400_ref6","doi-asserted-by":"crossref","first-page":"122","DOI":"10.1111\/cyt.12248","article-title":"DNA methylation analysis for the diagnosis of thyroid nodules - a pilot study with reference to BRAF(V) (600E) mutation and cytopathology results","volume":"27","author":"Chang","year":"2016","journal-title":"Cytopathology"},{"key":"2021111804322890400_ref7","doi-asserted-by":"crossref","first-page":"926","DOI":"10.1634\/theoncologist.2013-0072","article-title":"RAS mutations in thyroid cancer","volume":"18","author":"Howell","year":"2013","journal-title":"Oncologist"},{"key":"2021111804322890400_ref8","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1038\/nrc2294","article-title":"The properties of high-dimensional data spaces: implications for exploring gene and protein expression data","volume":"8","author":"Clarke","year":"2008","journal-title":"Nat Rev Cancer"},{"key":"2021111804322890400_ref9","doi-asserted-by":"crossref","first-page":"803","DOI":"10.1038\/nmeth.4014","article-title":"Points of significance: regularization","volume":"13","author":"Lever","year":"2016","journal-title":"Nat Methods"},{"key":"2021111804322890400_ref10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v033.i01","article-title":"Regularization paths for generalized linear models via coordinate descent","volume":"33","author":"Friedman","year":"2010","journal-title":"J Stat Softw"},{"key":"2021111804322890400_ref11","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1111\/j.1467-9868.2005.00503.x","article-title":"Regularization and variable selection via the elastic net","volume":"67","author":"Zou","year":"2005","journal-title":"J Royal Statistical Soc B"},{"key":"2021111804322890400_ref12","article-title":"Survey and comparative assessments of computational multi-omics integrative methods with multiple regulatory networks identifying distinct tumor compositions across pan-cancer data sets","author":"Wei","year":"2020","journal-title":"Brief Bioinform"},{"key":"2021111804322890400_ref13","doi-asserted-by":"crossref","first-page":"649","DOI":"10.1186\/s12859-019-3224-4","article-title":"Vertical and horizontal integration of multi-omics data with miodin","volume":"20","author":"Ulfenborg","year":"2019","journal-title":"BMC Bioinformatics"},{"key":"2021111804322890400_ref14","doi-asserted-by":"crossref","first-page":"440","DOI":"10.1093\/bib\/bbv044","article-title":"Using prior knowledge from cellular pathways and molecular networks for diagnostic specimen classification","volume":"17","author":"Glaab","year":"2016","journal-title":"Brief Bioinform"},{"key":"2021111804322890400_ref15","doi-asserted-by":"crossref","first-page":"6388","DOI":"10.1073\/pnas.1219651110","article-title":"Pathway-based personalized analysis of cancer","volume":"110","author":"Drier","year":"2013","journal-title":"Proc Natl Acad Sci U S A"},{"key":"2021111804322890400_ref16","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1093\/bib\/bbv030","article-title":"Individualized identification of disease-associated pathways with disrupted coordination of gene expression","volume":"17","author":"Wang","year":"2016","journal-title":"Brief Bioinform"},{"key":"2021111804322890400_ref17","doi-asserted-by":"crossref","first-page":"i422","DOI":"10.1093\/bioinformatics\/btu449","article-title":"Personalized identification of altered pathways in cancer using accumulated normal tissue data","volume":"30","author":"Ahn","year":"2014","journal-title":"Bioinformatics"},{"key":"2021111804322890400_ref18","article-title":"A novel unsupervised algorithm for biological process-based analysis on cancer","volume":"7","author":"Song","year":"2017","journal-title":"Sci Rep"},{"key":"2021111804322890400_ref19","doi-asserted-by":"crossref","first-page":"i237","DOI":"10.1093\/bioinformatics\/btq182","article-title":"Inference of patient-specific pathway activities from multi-dimensional cancer genomics data using PARADIGM","volume":"26","author":"Vaske","year":"2010","journal-title":"Bioinformatics"},{"key":"2021111804322890400_ref20","doi-asserted-by":"crossref","first-page":"250","DOI":"10.1016\/j.ebiom.2019.05.010","article-title":"Pathway-based biomarker identification with crosstalk analysis for robust prognosis prediction in hepatocellular carcinoma","volume":"44","author":"Fa","year":"2019","journal-title":"EBioMedicine"},{"key":"2021111804322890400_ref21","doi-asserted-by":"crossref","DOI":"10.1186\/s13073-016-0289-9","article-title":"Novel personalized pathway-based metabolomics models reveal key metabolic pathways for breast cancer diagnosis","volume":"8","author":"Huang","year":"2016","journal-title":"Genome Med"},{"key":"2021111804322890400_ref22","doi-asserted-by":"crossref","first-page":"1471","DOI":"10.1016\/j.molonc.2015.04.006","article-title":"Pathway-based personalized analysis of breast cancer expression data","volume":"9","author":"Livshits","year":"2015","journal-title":"Mol Oncol"},{"key":"2021111804322890400_ref23","doi-asserted-by":"crossref","first-page":"789","DOI":"10.1093\/bib\/bbx149","article-title":"Developing a \u201cpersonalome\u201d for precision medicine: emerging methods that compute interpretable effect sizes from single-subject transcriptomes","volume":"20","author":"Vitali","year":"2019","journal-title":"Brief Bioinform"},{"key":"2021111804322890400_ref24","doi-asserted-by":"crossref","first-page":"118","DOI":"10.1093\/biostatistics\/kxj037","article-title":"Adjusting batch effects in microarray expression data using empirical Bayes methods","volume":"8","author":"Johnson","year":"2007","journal-title":"Biostatistics"},{"key":"2021111804322890400_ref25","doi-asserted-by":"crossref","first-page":"502","DOI":"10.1080\/01621459.1989.10478797","article-title":"Principal curves","volume":"84","author":"Hastie","year":"1989","journal-title":"J Am Stat Assoc"},{"key":"2021111804322890400_ref26","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1093\/nar\/28.1.27","article-title":"Kyoto encyclopedia of genes and genomes","volume":"28","author":"Kanehisa","year":"2000","journal-title":"Nucleic Acids Res"},{"key":"2021111804322890400_ref27","doi-asserted-by":"crossref","first-page":"D674","DOI":"10.1093\/nar\/gkn653","article-title":"PID: the pathway interaction database","volume":"37","author":"Schaefer","year":"2009","journal-title":"Nucleic Acids Res"},{"key":"2021111804322890400_ref28","first-page":"117","volume-title":"BioCarta. Biotech Software & Internet Report 2","author":"Nishimura","year":"2001"},{"key":"2021111804322890400_ref29","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v062.i05","article-title":"c060: extended inference with lasso and elastic-net regularized cox and generalized linear models","volume":"62","author":"Sill","year":"2014","journal-title":"J Stat Softw"},{"key":"2021111804322890400_ref30","doi-asserted-by":"crossref","first-page":"957","DOI":"10.1093\/bib\/bbz046","article-title":"A comprehensive overview of oncogenic pathways in human cancer","volume":"21","author":"Li","year":"2020","journal-title":"Brief Bioinform"},{"key":"2021111804322890400_ref31","doi-asserted-by":"crossref","first-page":"40792","DOI":"10.18632\/oncotarget.8215","article-title":"Tall cell variant of papillary thyroid carcinoma: current evidence on clinicopathologic features and molecular biology","volume":"7","author":"Wang","year":"2016","journal-title":"Oncotarget"},{"key":"2021111804322890400_ref32","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1089\/thy.2009.0352","article-title":"Tall-cell variant of papillary thyroid carcinoma: a matched-pair analysis of survival","volume":"20","author":"Morris","year":"2010","journal-title":"Thyroid"},{"key":"2021111804322890400_ref33","doi-asserted-by":"crossref","first-page":"264","DOI":"10.1210\/jc.2015-2917","article-title":"Differential clinicopathological risk and prognosis of major papillary thyroid cancer variants","volume":"101","author":"Shi","year":"2016","journal-title":"J Clin Endocrinol Metab"},{"key":"2021111804322890400_ref34","doi-asserted-by":"crossref","first-page":"1241","DOI":"10.5858\/2008-132-1241-TTHOCA","article-title":"The thyroid H\u00fcrthle (oncocytic) cell and its associated pathologic conditions: a surgical pathology and cytopathology review","volume":"132","author":"Montone","year":"2008","journal-title":"Arch Pathol Lab Med"},{"key":"2021111804322890400_ref35","doi-asserted-by":"crossref","first-page":"665","DOI":"10.1111\/j.1365-2559.2009.03441.x","article-title":"Oncocytic versus mitochondrion-rich follicular thyroid tumours: should we make a difference?","volume":"55","author":"Tsybrovskyy","year":"2009","journal-title":"Histopathology"},{"key":"2021111804322890400_ref36","article-title":"Late bone metastasis from an apparently benign oncocytic follicular thyroid tumor","volume":"2013","author":"Boronat","year":"2013","journal-title":"Endocrinol Diabetes Metab Case Rep"},{"key":"2021111804322890400_ref37","doi-asserted-by":"crossref","DOI":"10.1186\/gb-2011-12-4-r41","article-title":"GISTIC2.0 facilitates sensitive and confident localization of the targets of focal somatic copy-number alteration in human cancers","volume":"12","author":"Mermel","year":"2011","journal-title":"Genome Biol"},{"key":"2021111804322890400_ref38","doi-asserted-by":"crossref","first-page":"410","DOI":"10.1245\/s10434-015-4762-2","article-title":"Defining a valid age cutoff in staging of well-differentiated thyroid cancer","volume":"23","author":"Nixon","year":"2016","journal-title":"Ann Surg Oncol"},{"key":"2021111804322890400_ref39","doi-asserted-by":"crossref","first-page":"1409","DOI":"10.1089\/thy.2019.0060","article-title":"Mortality risk of nonoperative papillary thyroid carcinoma: a corollary for active surveillance","volume":"29","author":"Ho","year":"2019","journal-title":"Thyroid"},{"key":"2021111804322890400_ref40","article-title":"Thyroid carcinoma. National Comprehensive Cancer network. Version","author":"NCCN Clinical Practice Guidelines in Oncology","year":"2017"},{"key":"2021111804322890400_ref41","doi-asserted-by":"crossref","first-page":"93","DOI":"10.3322\/caac.21388","article-title":"The eighth edition AJCC cancer staging manual: continuing to build a bridge from a population-based to a more \u201cpersonalized\u201d approach to cancer staging","volume":"67","author":"Amin","year":"2017","journal-title":"CA Cancer J Clin"},{"key":"2021111804322890400_ref42","doi-asserted-by":"crossref","first-page":"4413","DOI":"10.3390\/ijms20184413","article-title":"Immune and inflammatory cells in thyroid cancer microenvironment","volume":"20","author":"Ferrari","year":"2019","journal-title":"Int J Mol Sci"},{"key":"2021111804322890400_ref43","article-title":"The immune network in thyroid cancer","volume":"5","author":"Galdiero","year":"2016","journal-title":"Onco Targets Ther"},{"key":"2021111804322890400_ref44","doi-asserted-by":"crossref","first-page":"220","DOI":"10.1186\/s13059-017-1349-1","article-title":"xCell: digitally portraying the tissue cellular heterogeneity landscape","volume":"18","author":"Aran","year":"2017","journal-title":"Genome Biol"},{"key":"2021111804322890400_ref45","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pone.0118432","article-title":"The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets","volume":"10","author":"Saito","year":"2015","journal-title":"PLoS One"},{"key":"2021111804322890400_ref46","first-page":"459","article-title":"Sporadic and radiation-associated papillary thyroid cancers can be distinguished using routine immunohistochemistry","volume":"22","author":"Boltze","year":"2009","journal-title":"Oncol Rep"},{"key":"2021111804322890400_ref47","doi-asserted-by":"crossref","first-page":"1267","DOI":"10.1007\/s00259-015-3303-3","article-title":"Gene signature of the post-Chernobyl papillary thyroid cancer","volume":"43","author":"Handkiewicz-Junak","year":"2016","journal-title":"Eur J Nucl Med Mol Imaging"},{"key":"2021111804322890400_ref48","doi-asserted-by":"crossref","first-page":"994","DOI":"10.1038\/bjc.2012.302","article-title":"A gene expression signature distinguishes normal tissues of sporadic and radiation-induced papillary thyroid carcinomas","volume":"107","author":"Dom","year":"2012","journal-title":"Br J Cancer"},{"key":"2021111804322890400_ref49","doi-asserted-by":"crossref","first-page":"676","DOI":"10.1016\/j.cell.2014.09.050","article-title":"Integrated genomic characterization of papillary thyroid carcinoma","volume":"159","author":"Cancer Genome Atlas Research Network","year":"2014","journal-title":"Cell"},{"key":"2021111804322890400_ref50","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1175\/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2","article-title":"Verification of forecasts expressed in terms of probability","volume":"78","author":"Brier","year":"1950","journal-title":"Mon Weather Rev"},{"key":"2021111804322890400_ref51","doi-asserted-by":"crossref","first-page":"457","DOI":"10.1080\/01621459.1958.10501452","article-title":"Nonparametric estimation from incomplete observations","volume":"53","author":"Kaplan","year":"1958","journal-title":"J Am Stat Assoc"},{"key":"2021111804322890400_ref52","doi-asserted-by":"crossref","first-page":"185","DOI":"10.2307\/2344317","article-title":"Asymptotically efficient rank invariant test procedures -nuffield department of population health","volume":"135","author":"Peto","year":"1972","journal-title":"J Roy Stat Soc Ser A"},{"key":"2021111804322890400_ref53","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1111\/j.2517-6161.1972.tb00899.x","article-title":"Regression models and life-tables","volume":"34","author":"Cox","year":"1972","journal-title":"J R Stat Soc B Methodol"},{"key":"2021111804322890400_ref54","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1093\/biomet\/69.1.239","article-title":"Partial residuals for the proportional hazards regression model","volume":"69","author":"Schoenfeld","year":"1982","journal-title":"Biometrika"}],"container-title":["Briefings in Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/22\/4\/bbaa336\/41158001\/bbaa336.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/22\/4\/bbaa336\/41158001\/bbaa336.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,16]],"date-time":"2024-08-16T14:56:24Z","timestamp":1723820184000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bib\/article\/doi\/10.1093\/bib\/bbaa336\/6042204"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12,19]]},"references-count":54,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2021,7,20]]}},"URL":"https:\/\/doi.org\/10.1093\/bib\/bbaa336","relation":{},"ISSN":["1467-5463","1477-4054"],"issn-type":[{"value":"1467-5463","type":"print"},{"value":"1477-4054","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,7]]},"published":{"date-parts":[[2020,12,19]]},"article-number":"bbaa336"}}