{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T14:04:10Z","timestamp":1776089050900,"version":"3.50.1"},"reference-count":36,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,10,19]],"date-time":"2023-10-19T00:00:00Z","timestamp":1697673600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:sec><jats:title>Background<\/jats:title><jats:p>Due to the lower reliability of laboratory tests, skin diseases are more suitable for diagnosis with AI models. There are limited AI dermatology diagnostic models combining images and text; few of these are for Asian populations, and few cover the most common types of diseases.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>Leveraging a dataset sourced from Asia comprising over 200,000 images and 220,000 medical records, we explored a deep learning-based system for Dual-channel images and extracted text for the diagnosis of skin diseases model DIET-AI to diagnose 31 skin diseases, which covers the majority of common skin diseases. From 1 September to 1 December 2021, we prospectively collected images from 6,043 cases and medical records from 15 hospitals in seven provinces in China. Then the performance of DIET-AI was compared with that of six doctors of different seniorities in the clinical dataset.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>The average performance of DIET-AI in 31 diseases was not less than that of all the doctors of different seniorities. By comparing the area under the curve, sensitivity, and specificity, we demonstrate that the DIET-AI model is effective in clinical scenarios. In addition, medical records affect the performance of DIET-AI and physicians to varying degrees.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p>This is the largest dermatological dataset for the Chinese demographic. For the first time, we built a Dual-channel image classification model on a non-cancer dermatitis dataset with both images and medical records and achieved comparable diagnostic performance to senior doctors about common skin diseases. It provides references for exploring the feasibility and performance evaluation of DIET-AI in clinical use afterward.<\/jats:p><\/jats:sec>","DOI":"10.3389\/frai.2023.1213620","type":"journal-article","created":{"date-parts":[[2023,10,19]],"date-time":"2023-10-19T07:45:38Z","timestamp":1697701538000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["Deep skin diseases diagnostic system with Dual-channel Image and Extracted Text"],"prefix":"10.3389","volume":"6","author":[{"given":"Huanyu","family":"Li","sequence":"first","affiliation":[]},{"given":"Peng","family":"Zhang","sequence":"additional","affiliation":[]},{"given":"Zikun","family":"Wei","sequence":"additional","affiliation":[]},{"given":"Tian","family":"Qian","sequence":"additional","affiliation":[]},{"given":"Yiqi","family":"Tang","sequence":"additional","affiliation":[]},{"given":"Kun","family":"Hu","sequence":"additional","affiliation":[]},{"given":"Xianqiong","family":"Huang","sequence":"additional","affiliation":[]},{"given":"Xinxin","family":"Xia","sequence":"additional","affiliation":[]},{"given":"Yishuang","family":"Zhang","sequence":"additional","affiliation":[]},{"given":"Haixing","family":"Cheng","sequence":"additional","affiliation":[]},{"given":"Fubing","family":"Yu","sequence":"additional","affiliation":[]},{"given":"Wenjia","family":"Zhang","sequence":"additional","affiliation":[]},{"given":"Kena","family":"Dan","sequence":"additional","affiliation":[]},{"given":"Xuan","family":"Liu","sequence":"additional","affiliation":[]},{"given":"Shujun","family":"Ye","sequence":"additional","affiliation":[]},{"given":"Guangqiao","family":"He","sequence":"additional","affiliation":[]},{"given":"Xia","family":"Jiang","sequence":"additional","affiliation":[]},{"given":"Liwei","family":"Liu","sequence":"additional","affiliation":[]},{"given":"Yukun","family":"Fan","sequence":"additional","affiliation":[]},{"given":"Tingting","family":"Song","sequence":"additional","affiliation":[]},{"given":"Guomin","family":"Zhou","sequence":"additional","affiliation":[]},{"given":"Ziyi","family":"Wang","sequence":"additional","affiliation":[]},{"given":"Daojun","family":"Zhang","sequence":"additional","affiliation":[]},{"given":"Junwei","family":"Lv","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2023,10,19]]},"reference":[{"key":"B1","volume-title":"Histologic Diagnosis of Inflammatory Skin Diseases: An Algorithmic Method Based on Pattern Analysis","author":"Ackerman","year":"2005"},{"key":"B2","doi-asserted-by":"publisher","first-page":"38306","DOI":"10.1109\/ACCESS.2019.2906241","article-title":"Skin lesion classification using convolutional neural network with novel regularizer","volume":"7","author":"Albahar","year":"2019","journal-title":"IEEE Access"},{"key":"B3","doi-asserted-by":"publisher","first-page":"1146","DOI":"10.1109\/ICIP.2014.7025716","article-title":"Improving dermoscopy image classification using color constancy","volume":"19","author":"Barata","year":"2014","journal-title":"IEEE J. Biomed. Health Inform"},{"key":"B4","doi-asserted-by":"publisher","first-page":"361","DOI":"10.1016\/S0190-9622(87)70050-4","article-title":"A possible new tool for clinical diagnosis of melanoma: the computer","volume":"16","author":"Cascinelli","year":"1987","journal-title":"J. Am. Acad. Dermatol"},{"key":"B5","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1038\/nature21056","article-title":"Dermatologist-level classification of skin cancer with deep neural networks","volume":"542","author":"Esteva","year":"2017","journal-title":"Nature"},{"key":"B6","first-page":"1487","article-title":"The abilities of primary care physicians in dermatology: implications for quality of care","volume":"3","author":"Federman","year":"1997","journal-title":"Am. J. Manag. Care"},{"key":"B7","doi-asserted-by":"publisher","first-page":"170","DOI":"10.1001\/archfami.8.2.170","article-title":"Comparison of dermatologic diagnoses by primary care practitioners and dermatologists: a review of the literature","volume":"8","author":"Federman","year":"1999","journal-title":"Arch. Family Med"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2020.104065","article-title":"Artificial intelligence-based image classification methods for diagnosis of skin cancer: Challenges and opportunities","volume":"127","author":"Goyal","year":"2020","journal-title":"Comput. Biol. Med"},{"key":"B9","first-page":"1321","article-title":"On calibration of modern neural networks","volume-title":"International Conference on Machine Learning","author":"Guo","year":"2017"},{"key":"B10","doi-asserted-by":"publisher","first-page":"1836","DOI":"10.1093\/annonc\/mdy166","article-title":"Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists","volume":"29","author":"Haenssle","year":"2018","journal-title":"Ann. Oncol"},{"key":"B11","doi-asserted-by":"publisher","first-page":"202","DOI":"10.1016\/j.ejca.2021.06.049","article-title":"Skin cancer classification via convolutional neural networks: systematic review of studies involving human experts","volume":"156","author":"Haggenm\u00fcller","year":"2021","journal-title":"Eur. J. Cancer"},{"key":"B12","doi-asserted-by":"publisher","first-page":"770","DOI":"10.1109\/CVPR.2016.90","article-title":"Deep residual learning for image recognition","author":"He","year":"2016","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B13","article-title":"Meddialog: Two large-scale medical dialogue datasets","author":"He","year":"2020","journal-title":"arXiv preprint arXiv:2004.03329"},{"key":"B14","doi-asserted-by":"publisher","first-page":"e49","DOI":"10.1016\/S2589-7500(20)30004-2","article-title":"Leaving cancer diagnosis to the computers","volume":"2","year":"2020","journal-title":"Lancet Dig. Health"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.2196\/26025","article-title":"The classification of six common skin diseases based on xiangya-derm: development of a chinese database for artificial intelligence","volume":"23","author":"Huang","year":"2021","journal-title":"J. Med. Internet Res"},{"key":"B16","doi-asserted-by":"publisher","first-page":"e217249","DOI":"10.1001\/jamanetworkopen.2021.7249","article-title":"Development and assessment of an artificial intelligence-based tool for skin condition diagnosis by primary care physicians and nurse practitioners in teledermatology practices","volume":"4","author":"Jain","year":"2021","journal-title":"JAMA Netw. Open"},{"key":"B17","doi-asserted-by":"publisher","first-page":"114822","DOI":"10.1109\/ACCESS.2020.3003890","article-title":"Skin lesions classification into eight classes for isic 2019 using deep convolutional neural network and transfer learning","volume":"8","author":"Kassem","year":"2020","journal-title":"IEEE Access"},{"key":"B18","doi-asserted-by":"publisher","first-page":"42","DOI":"10.46439\/biomedres.3.025","article-title":"Racial underrepresentation in dermatological datasets leads to biased machine learning models and inequitable healthcare","volume":"3","author":"Kleinberg","year":"2022","journal-title":"J. Biomed. Res"},{"key":"B19","doi-asserted-by":"publisher","first-page":"56","DOI":"10.1016\/j.imed.2021.04.003","article-title":"Construction of an artificial intelligence system in dermatology: Effectiveness and consideration of chinese skin image database (csid)","volume":"1","author":"Li","year":"2021","journal-title":"Intell. Med"},{"key":"B20","article-title":"Visualizing the loss landscape of neural nets","author":"Li","year":"2017","journal-title":"arXiv preprint arXiv:1712.09913"},{"key":"B21","article-title":"Flat: Chinese ner using flat-lattice transformer","author":"Li","year":"2020","journal-title":"arXiv preprint arXiv:2004.11795"},{"key":"B22","doi-asserted-by":"publisher","first-page":"719","DOI":"10.1109\/TMI.2012.2236349","article-title":"Automatic segmentation of scaling in 2-d psoriasis skin images","volume":"32","author":"Lu","year":"2012","journal-title":"IEEE Trans. Med. Imag"},{"key":"B23","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1111\/j.1440-0960.2007.00340.x","article-title":"Prospective study to assess general practitioners' dermatological diagnostic skills in a referral setting","volume":"48","author":"Moreno","year":"2007","journal-title":"Austral. J. Dermatol"},{"key":"B24","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10916-016-0460-2","article-title":"Novel approaches for diagnosing melanoma skin lesions through supervised and deep learning algorithms","volume":"40","author":"Premaladha","year":"2016","journal-title":"J. Med. Syst"},{"key":"B25","doi-asserted-by":"publisher","first-page":"e549","DOI":"10.1136\/bmj.m3210","article-title":"Guidelines for clinical trial protocols for interventions involving artificial intelligence: the spirit-ai extension","volume":"2","author":"Rivera","year":"2020","journal-title":"Lancet Digital Health"},{"key":"B26","first-page":"234","article-title":"U-net: Convolutional networks for biomedical image segmentation","volume-title":"Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015: 18th International Conference, Proceedings, Part III 18","author":"Ronneberger","year":"2015"},{"key":"B27","doi-asserted-by":"publisher","first-page":"1736","DOI":"10.1016\/S0140-6736(18)32203-7","article-title":"Global, regional, and national age-sex-specific mortality for 282 causes of death in 195 countries and territories, 1980\u20132017: a systematic analysis for the global burden of disease study 2017","volume":"392","author":"Roth","year":"2018","journal-title":"Lancet"},{"key":"B28","doi-asserted-by":"publisher","first-page":"1252","DOI":"10.1111\/exd.14022","article-title":"Development and accuracy of an artificial intelligence algorithm for acne grading from smartphone photographs","volume":"28","author":"Seit\u00e9","year":"2019","journal-title":"Exper. Dermatol"},{"key":"B29","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1111\/j.1600-0846.2007.00181.x","article-title":"Combination of features from skin pattern and abcd analysis for lesion classification","volume":"13","author":"She","year":"2007","journal-title":"Skin Res. Technol"},{"key":"B30","doi-asserted-by":"publisher","first-page":"1107","DOI":"10.1001\/jama.2018.11029","article-title":"Clinical implications and challenges of artificial intelligence and deep learning","volume":"320","author":"Stead","year":"2018","journal-title":"JAMA"},{"key":"B31","doi-asserted-by":"publisher","first-page":"230","DOI":"10.1111\/j.1440-0960.2005.00189.x","article-title":"Assessing diagnostic skill in dermatology: a comparison between general practitioners and dermatologists","volume":"46","author":"Tran","year":"2005","journal-title":"Austral. J. Dermatol"},{"key":"B32","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1016\/j.jdin.2020.10.006","article-title":"Burden of skin disease and associated socioeconomic status in asia: A cross-sectional analysis from the global burden of disease study 1990\u20132017","volume":"2","author":"Urban","year":"2021","journal-title":"JAAD Int"},{"key":"B33","doi-asserted-by":"publisher","first-page":"451","DOI":"10.3390\/diagnostics11030451","article-title":"The role in teledermoscopy of an inexpensive and easy-to-use smartphone device for the classification of three types of skin lesions using convolutional neural networks","volume":"11","author":"Veronese","year":"2021","journal-title":"Diagnostics"},{"key":"B34","doi-asserted-by":"publisher","first-page":"1204","DOI":"10.1016\/S0140-6736(20)30925-9","article-title":"Global burden of 369 diseases and injuries in 204 countries and territories, 1990\u20132019: a systematic analysis for the global burden of disease study 2019","volume":"396","author":"Vos","year":"2020","journal-title":"Lancet"},{"key":"B35","article-title":"Objects as points","author":"Zhou","year":"2019","journal-title":"arXiv preprint arXiv:1904.07850"},{"key":"B36","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1038\/jidsymp.2015.9","article-title":"Dermatology in china","volume":"17","author":"Zhou","year":"2015","journal-title":"J. Investig. Dermatol. Symp. Proc"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2023.1213620\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,19]],"date-time":"2023-10-19T07:45:44Z","timestamp":1697701544000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2023.1213620\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,19]]},"references-count":36,"alternative-id":["10.3389\/frai.2023.1213620"],"URL":"https:\/\/doi.org\/10.3389\/frai.2023.1213620","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,10,19]]},"article-number":"1213620"}}