{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,14]],"date-time":"2026-10-14T18:44:30Z","timestamp":1792003470765,"version":"4.4.0"},"reference-count":29,"publisher":"Public Library of Science (PLoS)","issue":"9","license":[{"start":{"date-parts":[[2021,9,22]],"date-time":"2021-09-22T00:00:00Z","timestamp":1632268800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001656","name":"Helmholtz-Gemeinschaft","doi-asserted-by":"publisher","award":["Natural, Artificial, and Cognitive Information Processing"],"award-info":[{"award-number":["Natural, Artificial, and Cognitive Information Processing"]}],"id":[{"id":"10.13039\/501100001656","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001656","name":"Helmholtz-Gemeinschaft","doi-asserted-by":"publisher","award":["Natural, Artificial, and Cognitive Information Processing"],"award-info":[{"award-number":["Natural, Artificial, and Cognitive Information Processing"]}],"id":[{"id":"10.13039\/501100001656","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001656","name":"Helmholtz-Gemeinschaft","doi-asserted-by":"publisher","award":["Natural, Artificial, and Cognitive Information Processing"],"award-info":[{"award-number":["Natural, Artificial, and Cognitive Information Processing"]}],"id":[{"id":"10.13039\/501100001656","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS ONE"],"abstract":"<jats:p>When approaching thyroid gland tumor classification, the differentiation between samples with and without \u201cpapillary thyroid carcinoma-like\u201d nuclei is a daunting task with high inter-observer variability among pathologists. Thus, there is increasing interest in the use of machine learning approaches to provide pathologists real-time decision support. In this paper, we optimize and quantitatively compare two automated machine learning methods for thyroid gland tumor classification on two datasets to assist pathologists in decision-making regarding these methods and their parameters. The first method is a feature-based classification originating from common image processing and consists of cell nucleus segmentation, feature extraction, and subsequent thyroid gland tumor classification utilizing different classifiers. The second method is a deep learning-based classification which directly classifies the input images with a convolutional neural network without the need for cell nucleus segmentation. On the Tharun and Thompson dataset, the feature-based classification achieves an accuracy of 89.7% (Cohen\u2019s Kappa 0.79), compared to the deep learning-based classification of 89.1% (Cohen\u2019s Kappa 0.78). On the Nikiforov dataset, the feature-based classification achieves an accuracy of 83.5% (Cohen\u2019s Kappa 0.46) compared to the deep learning-based classification 77.4% (Cohen\u2019s Kappa 0.35). Thus, both automated thyroid tumor classification methods can reach the classification level of an expert pathologist. To our knowledge, this is the first study comparing feature-based and deep learning-based classification regarding their ability to classify samples with and without papillary thyroid carcinoma-like nuclei on two large-scale datasets.<\/jats:p>","DOI":"10.1371\/journal.pone.0257635","type":"journal-article","created":{"date-parts":[[2021,9,22]],"date-time":"2021-09-22T13:49:26Z","timestamp":1632318566000},"page":"e0257635","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":37,"title":["Machine learning methods for automated classification of tumors with papillary thyroid carcinoma-like nuclei: A quantitative analysis"],"prefix":"10.1371","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9321-8169","authenticated-orcid":true,"given":"Moritz","family":"B\u00f6hland","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lars","family":"Tharun","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8755-2825","authenticated-orcid":true,"given":"Tim","family":"Scherr","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ralf","family":"Mikut","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3572-9083","authenticated-orcid":true,"given":"Veit","family":"Hagenmeyer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lester D. R.","family":"Thompson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sven","family":"Perner","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Markus","family":"Reischl","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2021,9,22]]},"reference":[{"key":"pone.0257635.ref001","volume-title":"WHO classification of tumours of endocrine organs","author":"RV Lloyd","year":"2017"},{"key":"pone.0257635.ref002","doi-asserted-by":"crossref","first-page":"736","DOI":"10.1309\/AJCPKP2QUVN4RCCP","article-title":"Interobserver and intraobserver variation among experts in the diagnosis of thyroid follicular lesions with borderline nuclear features of papillary carcinoma","volume":"130","author":"TM Elsheikh","year":"2008","journal-title":"Am J Clin Pathol"},{"key":"pone.0257635.ref003","doi-asserted-by":"crossref","first-page":"242","DOI":"10.1007\/s12022-018-9520-0","article-title":"An international interobserver variability reporting of the nuclear scoring criteria to diagnose noninvasive follicular thyroid neoplasm with papillary-like nuclear features: a validation study","volume":"29","author":"LDR Thompson","year":"2018","journal-title":"Endocr Pathol"},{"key":"pone.0257635.ref004","doi-asserted-by":"crossref","first-page":"468","DOI":"10.21037\/atm.2019.08.54","article-title":"Using deep convolutional neural networks for multi-classification of thyroid tumor by histopathology: a large-scale pilot study","volume":"7","author":"Y Wang","year":"2019","journal-title":"Ann Transl Med"},{"key":"pone.0257635.ref005","doi-asserted-by":"crossref","first-page":"485","DOI":"10.1002\/cyto.a.20853","article-title":"Detection and classification of thyroid follicular lesions based on nuclear structure from histopathology images","volume":"77","author":"W Wang","year":"2010","journal-title":"Cytometry A"},{"key":"pone.0257635.ref006","doi-asserted-by":"crossref","first-page":"652","DOI":"10.1016\/j.asoc.2016.02.030","article-title":"Effective segmentation and classification of thyroid histopathology images","volume":"46","author":"J Angel Arul Jothi","year":"2016","journal-title":"Appl. Soft Comput"},{"key":"pone.0257635.ref007","doi-asserted-by":"crossref","first-page":"862","DOI":"10.1038\/s41379-020-00724-3","article-title":"Deep learning prediction of BRAF-RAS gene expression signature identifies noninvasive follicular thyroid neoplasms with papillary-like nuclear features","volume":"34","author":"JM Dolezal","year":"2021","journal-title":"Mod Pathol"},{"key":"pone.0257635.ref008","doi-asserted-by":"crossref","first-page":"101814","DOI":"10.1016\/j.media.2020.101814","article-title":"Weakly supervised instance learning for thyroid malignancy prediction from whole slide cytopathology images","volume":"67","author":"D Dov","year":"2021","journal-title":"Med Image Anal"},{"key":"pone.0257635.ref009","doi-asserted-by":"crossref","first-page":"1023","DOI":"10.1001\/jamaoncol.2016.0386","article-title":"Nomenclature revision for encapsulated follicular variant of papillary thyroid carcinoma a paradigm shift to reduce overtreatment of indolent tumors","volume":"2","author":"YE Nikiforov","year":"2016","journal-title":"JAMA Oncol"},{"key":"pone.0257635.ref010","doi-asserted-by":"crossref","first-page":"1380","DOI":"10.1109\/TMI.2019.2947628","article-title":"A multi-organ nucleus segmentation challenge","volume":"39","author":"N Kumar","year":"2020","journal-title":"IEEE Trans Med Imaging"},{"key":"pone.0257635.ref011","doi-asserted-by":"crossref","first-page":"1550","DOI":"10.1109\/TMI.2017.2677499","article-title":"A dataset and a technique for generalized nuclear segmentation for computational pathology","volume":"36","author":"N Kumar","year":"2017","journal-title":"IEEE Trans Med Imaging"},{"key":"pone.0257635.ref012","doi-asserted-by":"crossref","first-page":"772","DOI":"10.1016\/j.media.2014.04.004","article-title":"Accurate diagnosis of thyroid follicular lesions from nuclear morphology using supervised learning","volume":"18","author":"JA Ozolek","year":"2014","journal-title":"Med Image Anal"},{"key":"pone.0257635.ref013","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1038\/s41597-020-0528-1","article-title":"Dataset of segmented nuclei in hematoxylin and eosin stained histopathology images of ten cancer types","volume":"7","author":"L Hou","year":"2020","journal-title":"Sci Data"},{"key":"pone.0257635.ref014","first-page":"234","volume-title":"MICCAI 2015","author":"O Ronneberger","year":"2015"},{"key":"pone.0257635.ref015","doi-asserted-by":"crossref","first-page":"1247","DOI":"10.1038\/s41592-019-0612-7","article-title":"Nucleus segmentation across imaging experiments: the 2018 Data Science Bowl","volume":"16","author":"JC Caicedo","year":"2019","journal-title":"Nat Methods"},{"key":"pone.0257635.ref016","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1371\/journal.pone.0243219","article-title":"Cell segmentation and tracking using CNN-based distance predictions and a graph-based matching strategy","volume":"15","author":"T Scherr","year":"2020","journal-title":"PLOS ONE"},{"key":"pone.0257635.ref017","doi-asserted-by":"crossref","first-page":"638","DOI":"10.21105\/joss.00638","article-title":"MLxtend: providing machine learning and data science utilities and extensions to Python\u2019s scientific computing stack","volume":"3","author":"S Raschka","year":"2018","journal-title":"J Open Source Softw"},{"key":"pone.0257635.ref018","first-page":"2079","article-title":"On over-fitting in model selection and subsequent selection bias in performance evaluation","volume":"11","author":"GC Cawley","year":"2010","journal-title":"J Mach Learn Res"},{"key":"pone.0257635.ref019","doi-asserted-by":"crossref","unstructured":"Deng J, Dong W, Socher R, Li L, Kai Li, Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In: 2009 IEEE Conf on Comput Vis Pattern Recognit; 2009. p. 248\u2013255. https:\/\/doi.org\/10.1109\/CVPR.2009.5206848","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"pone.0257635.ref020","unstructured":"Kingma DP, Ba JL. Adam: a method for stochastic optimization; 2014. Available from: https:\/\/arxiv.org\/abs\/1412.6980v9."},{"key":"pone.0257635.ref021","first-page":"2825","article-title":"Scikit-learn: machine learning in Python","volume":"12","author":"F Pedregosa","year":"2011","journal-title":"J Mach Learn Res"},{"key":"pone.0257635.ref022","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: 2016 IEEE Conf on Comput Vis Pattern Recognit; 2016. p. 770\u2013778.","DOI":"10.1109\/CVPR.2016.90"},{"key":"pone.0257635.ref023","doi-asserted-by":"crossref","first-page":"125","DOI":"10.3390\/info11020125","article-title":"Albumentations: fast and flexible image augmentations","volume":"11","author":"A Buslaev","year":"2020","journal-title":"Information"},{"key":"pone.0257635.ref024","doi-asserted-by":"crossref","unstructured":"Yang Y, Soatto S. FDA: Fourier domain adaptation for semantic segmentation. In: Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit; 2020. p. 4084\u20134094.","DOI":"10.1109\/CVPR42600.2020.00414"},{"key":"pone.0257635.ref025","first-page":"1","volume-title":"Comput Vis\u2014ECCV 2020","author":"R Hataya","year":"2020"},{"key":"pone.0257635.ref026","doi-asserted-by":"crossref","first-page":"276","DOI":"10.11613\/BM.2012.031","article-title":"Interrater reliability: The Kappa statistic","volume":"22","author":"ML McHugh","year":"2012","journal-title":"Biochem Med"},{"key":"pone.0257635.ref027","doi-asserted-by":"crossref","unstructured":"Ribeiro MT, Singh S, Guestrin C. \u201cWhy should I trust you?\u201d Explaining the predictions of any classifier. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Association for Computing Machinery; 2016. p. 1135\u20131144. https:\/\/doi.org\/10.1145\/2939672.2939778","DOI":"10.1145\/2939672.2939778"},{"key":"pone.0257635.ref028","first-page":"4765","volume-title":"Adv Neural Inf Process Syst","author":"S Lundberg","year":"2017"},{"key":"pone.0257635.ref029","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1002\/dc.23191","article-title":"Cytological characteristics of papillary thyroid carcinoma on LBC specimens, compared with conventional specimens","volume":"43","author":"A Suzuki","year":"2015","journal-title":"Diagn Cytopathol"}],"container-title":["PLOS ONE"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0257635","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,9]],"date-time":"2023-11-09T01:04:56Z","timestamp":1699491896000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0257635"}},"subtitle":[],"editor":[{"given":"Andrey","family":"Bychkov","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2021,9,22]]},"references-count":29,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2021,9,22]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0257635","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,22]]}}}