{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,28]],"date-time":"2026-03-28T04:36:50Z","timestamp":1774672610215,"version":"3.50.1"},"reference-count":59,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2020,3,25]],"date-time":"2020-03-25T00:00:00Z","timestamp":1585094400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["NRF-2018R1D1A1B07041921"],"award-info":[{"award-number":["NRF-2018R1D1A1B07041921"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["NRF-2019R1A2C1083813"],"award-info":[{"award-number":["NRF-2019R1A2C1083813"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["NRF-2019R1F1A1041123"],"award-info":[{"award-number":["NRF-2019R1F1A1041123"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Computer-aided diagnosis systems have been developed to assist doctors in diagnosing thyroid nodules to reduce errors made by traditional diagnosis methods, which are mainly based on the experiences of doctors. Therefore, the performance of such systems plays an important role in enhancing the quality of a diagnosing task. Although there have been the state-of-the art studies regarding this problem, which are based on handcrafted features, deep features, or the combination of the two, their performances are still limited. To overcome these problems, we propose an ultrasound image-based diagnosis of the malignant thyroid nodule method using artificial intelligence based on the analysis in both spatial and frequency domains. Additionally, we propose the use of weighted binary cross-entropy loss function for the training of deep convolutional neural networks to reduce the effects of unbalanced training samples of the target classes in the training data. Through our experiments with a popular open dataset, namely the thyroid digital image database (TDID), we confirm the superiority of our method compared to the state-of-the-art methods.<\/jats:p>","DOI":"10.3390\/s20071822","type":"journal-article","created":{"date-parts":[[2020,3,25]],"date-time":"2020-03-25T13:10:47Z","timestamp":1585141847000},"page":"1822","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":98,"title":["Ultrasound Image-Based Diagnosis of Malignant Thyroid Nodule Using Artificial Intelligence"],"prefix":"10.3390","volume":"20","author":[{"given":"Dat Tien","family":"Nguyen","sequence":"first","affiliation":[{"name":"Division of Electronics and Electrical Engineering, Dongguk University, 30 Pildong-ro 1-gil, Jung-gu, Seoul 04620, Korea"}]},{"given":"Jin Kyu","family":"Kang","sequence":"additional","affiliation":[{"name":"Division of Electronics and Electrical Engineering, Dongguk University, 30 Pildong-ro 1-gil, Jung-gu, Seoul 04620, Korea"}]},{"given":"Tuyen Danh","family":"Pham","sequence":"additional","affiliation":[{"name":"Division of Electronics and Electrical Engineering, Dongguk University, 30 Pildong-ro 1-gil, Jung-gu, Seoul 04620, Korea"}]},{"given":"Ganbayar","family":"Batchuluun","sequence":"additional","affiliation":[{"name":"Division of Electronics and Electrical Engineering, Dongguk University, 30 Pildong-ro 1-gil, Jung-gu, Seoul 04620, Korea"}]},{"given":"Kang Ryoung","family":"Park","sequence":"additional","affiliation":[{"name":"Division of Electronics and Electrical Engineering, Dongguk University, 30 Pildong-ro 1-gil, Jung-gu, Seoul 04620, Korea"}]}],"member":"1968","published-online":{"date-parts":[[2020,3,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Vuong, Q.H., Ho, M.T., Vuong, T.T., La, V.P., Ho, M.T., Nghiem, K.C.P., Tran, B.X., Giang, H.H., Giang, T.V., and Latkin, C. (2019). Artificial intelligence vs. natural stupidity: Evaluating AI readiness for the Vietnamese medical information system. J. Clin. Med., 8.","DOI":"10.3390\/jcm8020168"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1016\/j.media.2016.05.004","article-title":"Brain tumor segmentation with deep neural networks","volume":"35","author":"Havaei","year":"2017","journal-title":"Med. Image Anal."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Cheng, C.-H., and Liu, W.-X. (2018). Identifying degenerative brain disease using rough set classifier based on wavelet packet method. J. Clin. Med., 7.","DOI":"10.3390\/jcm7060124"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"340","DOI":"10.1016\/j.patcog.2018.02.012","article-title":"Automatic breast ultrasound image segmentation: A survey","volume":"79","author":"Xian","year":"2018","journal-title":"Pattern Recognit."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1016\/j.cviu.2017.04.002","article-title":"Hough-CNN: Deep learning for segmentation of deep brain regions in MRI and ultrasound","volume":"16","author":"Milletar","year":"2017","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Owais, M., Arsalan, M., Choi, J., and Park, K.R. (2019). Effective diagnosis and treatment through content-based medical image retrieval (CBMIR) by using artificial intelligence. J. Clin. Med., 8.","DOI":"10.3390\/jcm8040462"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"214","DOI":"10.1016\/j.neuroimage.2014.12.061","article-title":"Deep convolutional neural networks for multi-modality isointense infant brain image segmentation","volume":"108","author":"Zhang","year":"2015","journal-title":"Neuroimage"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"700","DOI":"10.1016\/j.ultrasmedbio.2011.02.003","article-title":"Breast tumor classification using fuzzy clustering for breast elastography","volume":"37","author":"Moon","year":"2011","journal-title":"Ultrasound Med. Biol."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Hrzic, F., Stajduhar, I., Tschauner, S., Sorantin, E., and Lerga, J. (2019). Local-entropy based approach for x-ray image segmentation and fracture detection. Entropy, 21.","DOI":"10.3390\/e21040338"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"271","DOI":"10.1016\/j.patrec.2019.11.013","article-title":"Deep-learning framework to detect lung abnormality\u2014A study with chest x-ray and lung CT scan images","volume":"129","author":"Bhandary","year":"2020","journal-title":"Pattern Recogn. Lett."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"168","DOI":"10.1186\/1477-7819-12-168","article-title":"Who could benefit the most from using a computer-aided detection system in full-field digital mammography?","volume":"12","author":"Jung","year":"2014","journal-title":"World, J. Surg. Oncol."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1016\/j.media.2016.10.004","article-title":"Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation","volume":"36","author":"Kamnitsas","year":"2017","journal-title":"Med. Image Anal."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1016\/j.ultras.2017.03.010","article-title":"Computer-aided tumor diagnosis using shear wave breast elastography","volume":"78","author":"Moon","year":"2017","journal-title":"Ultrasonics"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"400","DOI":"10.1016\/j.bspc.2016.11.004","article-title":"Data mining framework for breast lesion classification in shear wave ultrasound: A hybrid feature paradigm","volume":"33","author":"Acharya","year":"2017","journal-title":"Biomed. Signal Process. Control"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1016\/j.ultras.2016.12.017","article-title":"The adaptive computer-aided diagnosis system based on tumor sizes for the classification of breast tumors detected at screening ultrasound","volume":"76","author":"Moon","year":"2017","journal-title":"Ultrasonics"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.ultras.2018.07.006","article-title":"Medical breast ultrasound image segmentation by machine learning","volume":"91","author":"Xu","year":"2019","journal-title":"Ultrasonics"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1016\/j.bspc.2017.08.025","article-title":"Computer aided thyroid nodule detection system using medical ultrasound images","volume":"40","author":"Koundal","year":"2018","journal-title":"Biomed. Signal Process. Control"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"587","DOI":"10.1016\/j.jacr.2017.01.046","article-title":"ACR thyroid imaging, reporting and data system (TI-RADS): White paper of the ACR TI-RADS committee","volume":"14","author":"Tessler","year":"2017","journal-title":"J. Am. Coll. Radiol."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1016\/j.ultras.2016.09.011","article-title":"A pretrained convolutional neural network based method for thyroid nodule diagnosis","volume":"73","author":"Ma","year":"2017","journal-title":"Ultrasonics"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Pedraza, L., Vargas, C., Narvaez, F., Duran, O., Munoz, E., and Romero, E. (2015, January 28). An open access thyroid ultrasound-image database. Proceedings of the 10th International Symposium on Medical Information Processing and Analysis, Cartagena de Indias, Colombia.","DOI":"10.1117\/12.2073532"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zhu, Y., Fu, Z., and Fei, J. (2017, January 13\u201316). An image augmentation method using convolutional network for thyroid nodule classification by transfer learning. Proceedings of the 3rd IEEE International Conference on Computer and Communication, Chengdu, China.","DOI":"10.1109\/CompComm.2017.8322853"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1016\/j.compbiomed.2015.12.006","article-title":"Application of wavelet techniques for cancer diagnosis using ultrasound images: A review","volume":"69","author":"Sudarshan","year":"2016","journal-title":"Comput. Biol. Med."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"477","DOI":"10.1007\/s10278-017-9997-y","article-title":"Thyroid nodule classification in ultrasound images by fine-tuning deep convolutional neural network","volume":"30","author":"Chi","year":"2017","journal-title":"J. Digit. Imaging"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"3494","DOI":"10.1016\/j.patcog.2010.04.023","article-title":"Application of support-vector-machine-based method for feature selection and classification of thyroid nodules in ultrasound images","volume":"43","author":"Chang","year":"2010","journal-title":"Pattern Recognit."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"425","DOI":"10.1016\/j.ultras.2010.11.008","article-title":"Thyroid nodule classification using ultrasound elastography via linear discriminant analysis","volume":"51","author":"Luo","year":"2011","journal-title":"Ultrasonics"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1016\/j.ejrad.2019.02.029","article-title":"Comparison between linear and nonlinear machine-learning algorithms for the classification of thyroid nodule","volume":"113","author":"Ouyang","year":"2019","journal-title":"Eur. J. Radiol."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1215","DOI":"10.1109\/JBHI.2018.2852718","article-title":"Multitask cascade convolution neural networks for automatic thyroid nodule detection and recognition","volume":"23","author":"Song","year":"2019","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_28","unstructured":"Sundar, K.V.S., Rajamani, K.T., and Sai, S.-S.S. (2018, January 16\u201317). Exploring image classification of thyroid ultrasound images using deep learning. Proceedings of the International Conference on ISMAC in Computational Vision and Bio-Engineering, Palladam, India."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"e15133","DOI":"10.1097\/MD.0000000000015133","article-title":"Ultrasound image analysis using deep learning algorithm for the diagnosis of thyroid nodules","volume":"98","author":"Song","year":"2019","journal-title":"Medicine"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1186\/s12957-019-1558-z","article-title":"Automatic thyroid nodule recognition and diagnosis in ultrasound imaging with the Yolov2 neural network","volume":"17","author":"Wang","year":"2019","journal-title":"World, J. Surg. Oncol."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1016\/j.ultras.2017.02.003","article-title":"Fusion of spatial gray level dependency and fractal texture features for the characterization of thyroid lessons","volume":"77","author":"Raghavendra","year":"2017","journal-title":"Ultrasonics"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1016\/j.cmpb.2017.06.005","article-title":"Ultrasound-based differentiation of malignant and benign thyroid nodules: An extreme learning machine approach","volume":"147","author":"Xia","year":"2017","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1949","DOI":"10.1016\/j.ejrad.2015.06.021","article-title":"Computerized analysis of calcification of thyroid nodules as visualized by ultrasonography","volume":"84","author":"Choi","year":"2015","journal-title":"Eur. J. Radiol."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1016\/j.compmedimag.2018.10.001","article-title":"Path-based classification of thyroid nodules in ultrasound images using direction independent features extracted by two-threshold binary decomposition","volume":"71","author":"Prochazka","year":"2019","journal-title":"Comput. Med. Imaging Graph."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1016\/j.knosys.2016.06.010","article-title":"Thyroid lesion classification in 242 patient population using Gabor transform features from high resolution ultrasound images","volume":"107","author":"Acharya","year":"2016","journal-title":"Knowl. Based Syst."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Nguyen, D.T., Pham, D.T., Batchuluun, G., Yoon, H.S., and Park, K.R. (2019). Artificial intelligence-based thyroid nodule classification using information from spatial and frequency domains. J. Clin. Med., 8.","DOI":"10.3390\/jcm8111976"},{"key":"ref_37","unstructured":"(2019, December 28). Enhanced Ultrasound Thyroid Nodule Classification (US-TNC-V2) Algorithm. Available online: http:\/\/dm.dongguk.edu\/link.html."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1109\/TSMC.1979.4310076","article-title":"A threshold selection method from gray-level histogram","volume":"9","author":"Otsu","year":"1979","journal-title":"IEEE Trans. Syst. Man Cybern."},{"key":"ref_39","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3\u20138). ImageNet classification with deep convolutional neural networks. Proceedings of the Neural Information Processing Systems, Lake Tahoe, NV, USA."},{"key":"ref_40","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional neural networks for large-scale image recognition. arXiv, Available online: https:\/\/arxiv.org\/abs\/1409.1556v6."},{"key":"ref_41","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2015). Deep residual learning for image recognition. arXiv, Available online: https:\/\/arxiv.org\/abs\/1512.03385v1."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2014). Going deeper with convolutions. arXiv, Available online: https:\/\/arxiv.org\/abs\/1409.4842v1.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_43","first-page":"1929","article-title":"Dropout: A simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_44","unstructured":"Ren, S., He, K., Girshick, R., and Sun, J. (2015). Faster R-CNN: Towards real-time object detection with region proposal networks. arXiv, Available online: https:\/\/arxiv.org\/abs\/1506.01497."},{"key":"ref_45","unstructured":"Redmon, J., and Farhadi, A. (2018). YOLOv3: An incremental improvement. arXiv, Available online: https:\/\/arxiv.org\/abs\/1804.02767."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1021","DOI":"10.1109\/ACCESS.2018.2886213","article-title":"Generative adversarial network-based method for transforming single RGB image into 3D point cloud","volume":"7","author":"Chu","year":"2018","journal-title":"IEEE Access"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Nguyen, D.T., Yoon, H.S., Pham, D.T., and Park, K.R. (2017). Spoof detection for finger-vein recognition system using NIR camera. Sensors, 17.","DOI":"10.3390\/s17102261"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Nguyen, D.T., Pham, D.T., Lee, M.B., and Park, K.R. (2019). Visible-light camera sensor-based presentation attack detection for face recognition by combining spatial and temporal information. Sensors, 19.","DOI":"10.3390\/s19020410"},{"key":"ref_49","unstructured":"Young, T., Hazarika, D., Poria, S., and Cambria, E. (2017). Recent trends in deep learning based natural language processing. arXiv, Available online: https:\/\/arxiv.org\/abs\/1708.02709."},{"key":"ref_50","unstructured":"Otter, D.W., Medina, J.R., and Kalita, J.K. (2018). A survey of the usages of deep learning in natural language processing. arXiv, Available online: https:\/\/arxiv.org\/abs\/1807.10854."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Islam, M.R. (2014). Feature and score fusion based multiple classifier selection for iris recognition. Comput. Intell. Neurosci., 380585.","DOI":"10.1155\/2014\/380585"},{"key":"ref_52","unstructured":"Vishi, K., and Mavroeidis, V. (2018). An evaluation of score level fusion approaches for fingerprint and finger-vein biometrics. arXiv, Available online: https:\/\/arxiv.org\/abs\/1805.10666."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Nguyen, D.T., Park, Y.H., Lee, H.C., Shin, K.Y., Kang, B.J., and Park, K.R. (2012). Combining touched fingerprint and finger-vein of a finger, and its usability evaluation. Adv. Sci. Lett., 85\u201395.","DOI":"10.1166\/asl.2012.2177"},{"key":"ref_54","unstructured":"Baloch, B.K., Kumar, S., Haresh, S., Rehman, A., and Syed, T. (2019, January 17\u201319). Focused anchors loss: Cost-sensitive learning of discriminative features for imbalanced classification. Proceedings of the machine learning research, Nagoya, Japan."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Lin, T.-Y., Goyal, P., Girshick, R., He, K., and Dollar, P. (2017). Focal loss for dense object detection. arXiv, Available online: https:\/\/arxiv.org\/abs\/1708.02002.","DOI":"10.1109\/ICCV.2017.324"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Cui, Y., Jia, M., Lin, T.-Y., Song, Y., and Belongie, S. (2019). Class-balanced loss based on effective number of samples. arXiv, Available online: https:\/\/arxiv.org\/abs\/1901.05555.","DOI":"10.1109\/CVPR.2019.00949"},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"511","DOI":"10.1542\/pir.31.12.511","article-title":"Research and statistics: Sensitivity, specificity, predictive values, and likelihood ratios","volume":"31","author":"Carvajal","year":"2010","journal-title":"Pediatr. Rev."},{"key":"ref_58","unstructured":"(2019, September 20). NVIDIA TitanX GPU. Available online: https:\/\/www.nvidia.com\/en-us\/geforce\/products\/10series\/titan-x-pascal\/."},{"key":"ref_59","unstructured":"(2019, September 20). Tensorflow Deep-Learning Library. Available online: https:\/\/www.tensorflow.org\/."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/7\/1822\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:11:26Z","timestamp":1760173886000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/7\/1822"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,3,25]]},"references-count":59,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2020,4]]}},"alternative-id":["s20071822"],"URL":"https:\/\/doi.org\/10.3390\/s20071822","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,3,25]]}}}