{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,8]],"date-time":"2025-11-08T12:15:41Z","timestamp":1762604141702,"version":"build-2065373602"},"reference-count":31,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2019,2,15]],"date-time":"2019-02-15T00:00:00Z","timestamp":1550188800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>The tooth-marked tongue is an important indicator in traditional Chinese medicinal diagnosis. However, the clinical competence of tongue diagnosis is determined by the experience and knowledge of the practitioners. Due to the characteristics of different tongues, having many variations such as different colors and shapes, tooth-marked tongue recognition is challenging. Most existing methods focus on partial concave features and use specific threshold values to classify the tooth-marked tongue. They lose the overall tongue information and lack the ability to be generalized and interpretable. In this paper, we try to solve these problems by proposing a visual explanation method which takes the entire tongue image as an input and uses a convolutional neural network to extract features (instead of setting a fixed threshold artificially) then classifies the tongue and produces a coarse localization map highlighting tooth-marked regions using Gradient-weighted Class Activation Mapping. Experimental results demonstrate the effectiveness of the proposed method.<\/jats:p>","DOI":"10.3390\/fi11020045","type":"journal-article","created":{"date-parts":[[2019,2,17]],"date-time":"2019-02-17T22:11:50Z","timestamp":1550441510000},"page":"45","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":25,"title":["Tooth-Marked Tongue Recognition Using Gradient-Weighted Class Activation Maps"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1675-7115","authenticated-orcid":false,"given":"Yue","family":"Sun","sequence":"first","affiliation":[{"name":"School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China"}]},{"given":"Songmin","family":"Dai","sequence":"additional","affiliation":[{"name":"School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China"}]},{"given":"Jide","family":"Li","sequence":"additional","affiliation":[{"name":"School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China"}]},{"given":"Yin","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China"}]},{"given":"Xiaoqiang","family":"Li","sequence":"additional","affiliation":[{"name":"School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China"},{"name":"Shanghai Institute for Advanced Communication and Data Science, Shanghai University, Shanghai 200444, China"}]}],"member":"1968","published-online":{"date-parts":[[2019,2,15]]},"reference":[{"key":"ref_1","first-page":"643","article-title":"Basic theory of traditional Chinese medicine","volume":"17","author":"Shen","year":"1997","journal-title":"Chin. J. Integr. Tradit. West. Med."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1016\/j.ins.2005.01.010","article-title":"Tongue image analysis for appendicitis diagnosis","volume":"175","author":"Pang","year":"2005","journal-title":"Inf. Sci."},{"key":"ref_3","first-page":"1299","article-title":"Color atlas of oral diseases","volume":"100","author":"McLean","year":"2004","journal-title":"Br. J. Plast. Surg."},{"key":"ref_4","unstructured":"Li, W., Luo, J., Hu, S., Xu, J., and Zhang, Z. (2008, January 12\u201314). Towards the Objectification of Tongue Diagnosis: the Degree of Tooth-marked. Proceedings of the IEEE International Symposium on It in Medicine and Education, Xiamen, China."},{"key":"ref_5","first-page":"2283","article-title":"Study on the correlation between dental scar tongue and constitution of traditional Chinese medicine in physical examination population","volume":"14","author":"Ren","year":"2012","journal-title":"World Sci. Technol.-Mod. Tradit. Chin. Med."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"380","DOI":"10.1109\/TCYB.2017.2772289","article-title":"Tooth-Marked Tongue Recognition Using Multiple Instance Learning and CNN Features","volume":"49","author":"Li","year":"2019","journal-title":"IEEE Trans. Cybern."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"859","DOI":"10.1142\/S0192415X05003466","article-title":"Computerized diagnosis from tongue appearance using quantitative feature classification","volume":"33","author":"Zhang","year":"2005","journal-title":"Am. J. Chin. Med."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.patrec.2006.06.004","article-title":"Region partition and feature matching based color recognition of tongue image","volume":"28","author":"Wang","year":"2007","journal-title":"Pattern Recognit. Lett."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017, January 22\u201329). Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. Proceedings of the International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.74"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1016\/S0169-2607(99)00031-0","article-title":"A novel approach based on computerized image analysis for traditional Chinese medical diagnosis of the tongue","volume":"61","author":"Chiu","year":"2000","journal-title":"Comput. Methods Progr. Biomed."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1803","DOI":"10.1109\/TBME.2004.831534","article-title":"Computerized tongue diagnosis based on Bayesian networks","volume":"51","author":"Pang","year":"2004","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_12","unstructured":"Zuo, W., Wang, K., Zhang, D., and Zhang, H. (2004, January 18\u201320). Combination of polar edge detection and active contour model for automated tongue segmentation. Proceedings of the International Conference on Image and Graphics, Hong Kong, China."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Yu, S., Yang, J., Wang, Y., and Zhang, Y. (2007, January 6\u20138). Color Active Contour Models Based Tongue Segmentation in Traditional Chinese Medicine. Proceedings of the International Conference on Bioinformatics and Biomedical Engineering, Wuhan, China.","DOI":"10.1109\/ICBBE.2007.275"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"5336","DOI":"10.1109\/TIP.2013.2284070","article-title":"Statistical analysis of tongue images for feature extraction and diagnostics","volume":"22","author":"Wang","year":"2013","journal-title":"IEEE Trans. Image Process."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"312","DOI":"10.1016\/j.ins.2009.09.016","article-title":"Tongue shape classification by geometric features","volume":"180","author":"Huang","year":"2010","journal-title":"Inf. Sci."},{"key":"ref_16","unstructured":"Zhang, Y. (2005). Research on Analysis Method of Tongue and Teeth-Marked Tongue. [Ph.D. Thesis, Beijing University of Chinese Medicine]."},{"key":"ref_17","unstructured":"Li, J.F., Li, N.M., Wang, K.Q., and Zhang, H.Z. (2009, January 1). Extracting feature of teeth-marked tongue image. Proceedings of the Diagnosis Section of China Society of Integrated Traditional Chinese and Western Medicine, Fuzhou, China."},{"key":"ref_18","unstructured":"Wang, H., Zhang, X., and Cai, Y. (June, January 30). Research on Teeth Marks Recognition in Tongue Image. Proceedings of the Academic Conference on National Diagnosis of Chinese Society of Integrated Traditional Chinese and Western Medicine, Shenzhen, China."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Shao, Q., Li, X., and Fu, Z. (2015, January 7\u20139). Recognition of teeth-marked tongue based on gradient of concave region. Proceedings of the International Conference on Audio, Language and Image Processing, Shanghai, China.","DOI":"10.1109\/ICALIP.2014.7009938"},{"key":"ref_20","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3\u20136). ImageNet classification with deep convolutional neural networks. Proceedings of the International Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Long, J., Shelhamer, E., and Darrell, T. (2015, January 7\u201312). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Zeiler, M.D., and Fergus, R. (2014). Visualizing and understanding convolutional networks. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-10590-1_53"},{"key":"ref_23","unstructured":"Springenberg, J.T., Dosovitskiy, A., Brox, T., and Riedmiller, M. (arXiv, 2014). Striving for Simplicity: The All Convolutional Net, arXiv."},{"key":"ref_24","unstructured":"Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., and Torralba, A. (June, January 27). Learning Deep Features for Discriminative Localization. Proceedings of the Computer Vision and Pattern Recognition, Las Vegas, NV, USA."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Mahendran, A., and Vedaldi, A. (2015, January 7\u201312). Understanding deep image representations by inverting them. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7299155"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1038\/nature21056","article-title":"Dermatologist-level classification of skin cancer with deep neural networks","volume":"542","author":"Esteva","year":"2017","journal-title":"Nature"},{"key":"ref_27","unstructured":"Rajpurkar, P., Irvin, J., Zhu, K., Yang, B., Mehta, H., Duan, T., Ding, D., Bagul, A., Langlotz, C.P., and Shpanskaya, K. (arXiv, 2017). CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning, arXiv."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1007\/s11263-016-0911-8","article-title":"Visualizing Deep Convolutional Neural Networks Using Natural Pre-images","volume":"120","author":"Mahendran","year":"2016","journal-title":"Int. J. Comput. Vis."},{"key":"ref_29","unstructured":"Kingma, D., and Ba, J. (arXiv, 2014). Adam: A Method for Stochastic Optimization, arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Girshick, R.B., Donahue, J., Darrell, T., and Malik, J. (2014, January 23\u201328). Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. Proceedings of the Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.81"},{"key":"ref_31","unstructured":"Simonyan, K., and Zisserman, A. (arXiv, 2015). Very Deep Convolutional Networks for Large-Scale Image Recognition, arXiv."}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/11\/2\/45\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:32:26Z","timestamp":1760185946000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/11\/2\/45"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,2,15]]},"references-count":31,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2019,2]]}},"alternative-id":["fi11020045"],"URL":"https:\/\/doi.org\/10.3390\/fi11020045","relation":{},"ISSN":["1999-5903"],"issn-type":[{"type":"electronic","value":"1999-5903"}],"subject":[],"published":{"date-parts":[[2019,2,15]]}}}