{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,26]],"date-time":"2026-10-26T22:58:13Z","timestamp":1793055493592,"version":"4.5.0"},"reference-count":52,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2020,7,29]],"date-time":"2020-07-29T00:00:00Z","timestamp":1595980800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Biomolecules"],"abstract":"<jats:p>Recent studies have demonstrated the usefulness of convolutional neural networks (CNNs) to classify images of melanoma, with accuracies comparable to those achieved by dermatologists. However, the performance of a CNN trained with only clinical images of a pigmented skin lesion in a clinical image classification task, in competition with dermatologists, has not been reported to date. In this study, we extracted 5846 clinical images of pigmented skin lesions from 3551 patients. Pigmented skin lesions included malignant tumors (malignant melanoma and basal cell carcinoma) and benign tumors (nevus, seborrhoeic keratosis, senile lentigo, and hematoma\/hemangioma). We created the test dataset by randomly selecting 666 patients out of them and picking one image per patient, and created the training dataset by giving bounding-box annotations to the rest of the images (4732 images, 2885 patients). Subsequently, we trained a faster, region-based CNN (FRCNN) with the training dataset and checked the performance of the model on the test dataset. In addition, ten board-certified dermatologists (BCDs) and ten dermatologic trainees (TRNs) took the same tests, and we compared their diagnostic accuracy with FRCNN. For six-class classification, the accuracy of FRCNN was 86.2%, and that of the BCDs and TRNs was 79.5% (p = 0.0081) and 75.1% (p &lt; 0.00001), respectively. For two-class classification (benign or malignant), the accuracy, sensitivity, and specificity were 91.5%, 83.3%, and 94.5% by FRCNN; 86.6%, 86.3%, and 86.6% by BCD; and 85.3%, 83.5%, and 85.9% by TRN, respectively. False positive rates and positive predictive values were 5.5% and 84.7% by FRCNN, 13.4% and 70.5% by BCD, and 14.1% and 68.5% by TRN, respectively. We compared the classification performance of FRCNN with 20 dermatologists. As a result, the classification accuracy of FRCNN was better than that of the dermatologists. In the future, we plan to implement this system in society and have it used by the general public, in order to improve the prognosis of skin cancer.<\/jats:p>","DOI":"10.3390\/biom10081123","type":"journal-article","created":{"date-parts":[[2020,7,30]],"date-time":"2020-07-30T03:36:38Z","timestamp":1596080198000},"page":"1123","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":246,"title":["The Development of a Skin Cancer Classification System for Pigmented Skin Lesions Using Deep Learning"],"prefix":"10.3390","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3984-8493","authenticated-orcid":false,"given":"Shunichi","family":"Jinnai","sequence":"first","affiliation":[{"name":"Department of Dermatologic Oncology, National Cancer Center Hospital, 5-1-1 Tsukiji, Chuo-ku, Tokyo 104-0045, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Naoya","family":"Yamazaki","sequence":"additional","affiliation":[{"name":"Department of Dermatologic Oncology, National Cancer Center Hospital, 5-1-1 Tsukiji, Chuo-ku, Tokyo 104-0045, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuichiro","family":"Hirano","sequence":"additional","affiliation":[{"name":"Preferred Networks, 1-6-1 Otemachi, Chiyoda-ku, Tokyo 100-0004, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yohei","family":"Sugawara","sequence":"additional","affiliation":[{"name":"Preferred Networks, 1-6-1 Otemachi, Chiyoda-ku, Tokyo 100-0004, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6137-073X","authenticated-orcid":false,"given":"Yuichiro","family":"Ohe","sequence":"additional","affiliation":[{"name":"Department of Thoracic Oncology, National Cancer Center Hospital, 5-1-1 Tsukiji, Chuo-ku, Tokyo 104-0045, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ryuji","family":"Hamamoto","sequence":"additional","affiliation":[{"name":"Division of Molecular Modification and Cancer Biology, National Cancer Center Research Institute, 5-1-1 Tsukiji, Chuo-ku, Tokyo 104-0045, Japan"},{"name":"Cancer Translational Research Team, RIKEN Center for Advanced Intelligence Project, 1-4-1 Nihonbashi, Chuo-ku, Tokyo 103-0027, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,7,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"971","DOI":"10.1016\/S0140-6736(18)31559-9","article-title":"Melanoma","volume":"392","author":"Schadendorf","year":"2018","journal-title":"Lancet"},{"key":"ref_2","first-page":"1373","article-title":"Melanoma detection by analysis of clinical images using convolutional neural network","volume":"2016","author":"Samavi","year":"2016","journal-title":"Conf. Proc. IEEE Eng. Med. Biol. Soc."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1016\/j.ejca.2019.02.005","article-title":"A convolutional neural network trained with dermoscopic images performed on par with 145 dermatologists in a clinical melanoma image classification task","volume":"111","author":"Brinker","year":"2019","journal-title":"Eur. J. Cancer"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1038\/nature21056","article-title":"Dermatologist-level classification of skin cancer with deep neural networks","volume":"542","author":"Esteva","year":"2017","journal-title":"Nature"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Fujisawa, Y., Otomo, Y., Ogata, Y., Nakamura, Y., Fujita, R., Ishitsuka, Y., Watanabe, R., Okiyama, N., Ohara, K., and Fujimoto, M. (2018). Deep-learning-based, computer-aided classifier developed with a small dataset of clinical images surpasses board-certified dermatologists in skin tumour diagnosis. Br. J. Dermatol.","DOI":"10.1111\/bjd.16924"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"843","DOI":"10.1111\/ijd.14925","article-title":"Dermoscopy in differential diagnosis of inflammatory dermatoses and mycosis fungoides","volume":"59","author":"Bilgic","year":"2020","journal-title":"Int. J. Dermatol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"305","DOI":"10.1097\/DSS.0000000000002240","article-title":"Extramammary Paget\u2019s Disease: A Review of the Literature Part II: Treatment and Prognosis","volume":"46","author":"Morris","year":"2020","journal-title":"Derm. Surg."},{"key":"ref_8","unstructured":"Marques, E., and Chen, T.M. (2020). Actinic Keratosis, StatPearls Publishing."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Asada, K., Kobayashi, K., Joutard, S., Tubaki, M., Takahashi, S., Takasawa, K., Komatsu, M., Kaneko, S., Sese, J., and Hamamoto, R. (2020). Uncovering Prognosis-Related Genes and Pathways by Multi-Omics Analysis in Lung Cancer. Biomolecules, 10.","DOI":"10.3390\/biom10040524"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Hamamoto, R., Komatsu, M., Takasawa, K., Asada, K., and Kaneko, S. (2020). Epigenetics Analysis and Integrated Analysis of Multiomics Data, Including Epigenetic Data, Using Artificial Intelligence in the Era of Precision Medicine. Biomolecules, 10.","DOI":"10.3390\/biom10010062"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"14465","DOI":"10.1038\/s41598-019-50567-5","article-title":"Development of a real-time endoscopic image diagnosis support system using deep learning technology in colonoscopy","volume":"9","author":"Yamada","year":"2019","journal-title":"Sci. Rep."},{"key":"ref_12","unstructured":"Yasutomi, S., Arakaki, T., and Hamamoto, R. (2019). Shadow Detection for Ultrasound Images Using Unlabeled Data and Synthetic Shadows. arXiv."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.1109\/TPAMI.2016.2577031","article-title":"Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks","volume":"39","author":"Ren","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"64","DOI":"10.3389\/fnbot.2018.00064","article-title":"Faster R-CNN for Robust Pedestrian Detection Using Semantic Segmentation Network","volume":"12","author":"Liu","year":"2018","journal-title":"Front. Neurorobot."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 24\u201327). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.81"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Shao, F., Wang, X., Meng, F., Zhu, J., Wang, D., and Dai, J. (2019). Improved Faster R-CNN Traffic Sign Detection Based on a Second Region of Interest and Highly Possible Regions Proposal Network. Sensors, 19.","DOI":"10.3390\/s19102288"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Huang, Y.Q., Zheng, J.C., Sun, S.D., Yang, C.F., and Liu, J. (2020). Optimized YOLOv3 Algorithm and Its Application inTraffic Flow Detections. Appl. Sci., 10.","DOI":"10.3390\/app10093079"},{"key":"ref_18","first-page":"122","article-title":"Optimized Convolutional Neural Network-Based Object Recognition for Humanoid Robot","volume":"4","author":"Xiao","year":"2020","journal-title":"J. Robot. Autom."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"709","DOI":"10.1002\/jemt.23217","article-title":"An effective and accurate identification system of Mycobacterium tuberculosis using convolution neural networks","volume":"82","author":"Kuok","year":"2019","journal-title":"Microsc. Res. Tech."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"103912","DOI":"10.1016\/j.compbiomed.2020.103912","article-title":"Faster R-CNN approach for detection and quantification of DNA damage in comet assay images","volume":"123","author":"Rosati","year":"2020","journal-title":"Comput. Biol. Med."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1730","DOI":"10.1109\/JBHI.2018.2868656","article-title":"Robust Methods for Real-Time Diabetic Foot Ulcer Detection and Localization on Mobile Devices","volume":"23","author":"Goyal","year":"2019","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"72601W","DOI":"10.1117\/12.811675","article-title":"Interactive segmentation in multimodal med-ical imagery using a bayesian transductive learning approach","volume":"7260","author":"Lee","year":"2009","journal-title":"Med. Imaging 2009 Comput.-Aided Diagn."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"212","DOI":"10.1109\/TCBB.2016.2527657","article-title":"Transductive Learning for Multi-Label Protein Subchloroplast Localization Prediction","volume":"14","author":"Wan","year":"2017","journal-title":"IEEE\/Acm Trans. Comput. Biol. Bioinform."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"e018280","DOI":"10.1136\/bmjopen-2017-018280","article-title":"Evidence assessing the diagnostic performance of medical smartphone apps: A systematic review and exploratory meta-analysis","volume":"7","author":"Buechi","year":"2017","journal-title":"Bmj Open"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"e175","DOI":"10.1111\/ajd.12599","article-title":"Efficacy of smartphone applications in high-risk pigmented lesions","volume":"59","author":"Ngoo","year":"2018","journal-title":"Australas J. Dermatol."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Singh, N., and Gupta, S.K. (2018). Recent advancement in the early detection of melanoma using computerized tools: An image analysis perspective. Ski. Res. Technol.","DOI":"10.1111\/srt.12622"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"4017","DOI":"10.1109\/TCYB.2018.2859482","article-title":"Face Detection With Different Scales Based on Faster R-CNN","volume":"49","author":"Wu","year":"2019","journal-title":"IEEE Trans. Cybern"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"6600","DOI":"10.1038\/s41598-018-25005-7","article-title":"An improved deep learning approach for detection of thyroid papillary cancer in ultrasound images","volume":"8","author":"Li","year":"2018","journal-title":"Sci. Rep."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Ji, Y., Zhang, S., and Xiao, W. (2019). Electrocardiogram Classification Based on Faster Regions with Convolutional Neural Network. Sensors, 19.","DOI":"10.3390\/s19112558"},{"key":"ref_30","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1016\/S0893-6080(98)00116-6","article-title":"On the momentum term in gradient descent learning algorithms","volume":"12","author":"Qian","year":"1999","journal-title":"Neural Netw."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., and Berg, A. (2016). SSD: Single Shot MultiBox Detector. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-46448-0_2"},{"key":"ref_33","unstructured":"Tokui, S., Oono, K., Hido, S., and Clayton, J. (2015, January 7). Chainer: A next-generation open source framework for deep learning. Proceedings of the Workshop on Machine Learning Systems (LearningSys) in the Twenty-Ninth Annual Conference on Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Niitani, Y., Ogawa, T., Saito, S., and Saito, M. (2017, January 23\u201327). ChainerCV: A library for deep learning in computer vision. Proceedings of the 25th ACM International Conference on Multimedia, Mountain View, CA, USA.","DOI":"10.1145\/3123266.3129395"},{"key":"ref_35","unstructured":"Nishino, R., and Loomis, S.H.C. (2017, January 4). CuPy: A numpy-compatible library for nvidia gpu calculations. Proceedings of the Workshop on Machine Learning Systems (LearningSys) in the Thirty-first Annual Conference on Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1836","DOI":"10.1093\/annonc\/mdy166","article-title":"Man against machine: Diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists","volume":"29","author":"Haenssle","year":"2018","journal-title":"Ann. Oncol."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1176","DOI":"10.1016\/j.jaad.2019.06.042","article-title":"Assessing the effectiveness of artificial intelligence methods for melanoma: A retrospective review","volume":"81","author":"Cui","year":"2019","journal-title":"J. Am. Acad. Dermatol."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1507","DOI":"10.1111\/bjd.13665","article-title":"Smartphone applications for melanoma detection by community, patient and generalist clinician users: A review","volume":"172","author":"Kassianos","year":"2015","journal-title":"Br. J. Dermatol."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"422","DOI":"10.1001\/jamadermatol.2013.2382","article-title":"Diagnostic inaccuracy of smartphone applications for melanoma detection","volume":"149","author":"Wolf","year":"2013","journal-title":"Jama Dermatol."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Udrea, A., Mitra, G.D., Costea, D., Noels, E.C., Wakkee, M., Siegel, D.M., de Carvalho, T.M., and Nijsten, T.E.C. (2019). Accuracy of a smartphone application for triage of skin lesions based on machine learning algorithms. J. Eur. Acad. Dermatol. Venereol.","DOI":"10.1111\/jdv.15935"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"284","DOI":"10.1016\/j.jdermsci.2019.04.003","article-title":"Classification of 3097 patients from the Japanese melanoma study database using the American joint committee on cancer eighth edition cancer staging system","volume":"94","author":"Fujisawa","year":"2019","journal-title":"J. Dermatol. Sci."},{"key":"ref_42","unstructured":"American Cancer Society (2020). Cancer Facts & Figures 2020. Am. Cancer Soc. J., Available online: https:\/\/www.cancer.org\/content\/dam\/cancer-org\/research\/cancer-facts-and-statistics\/annual-cancer-facts-and-figures\/2020\/cancer-facts-and-figures-2020.pdf."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"2146","DOI":"10.1002\/cam4.2110","article-title":"Clinical and histopathological characteristics and survival analysis of 4594 Japanese patients with melanoma","volume":"8","author":"Fujisawa","year":"2019","journal-title":"Cancer Med."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1016\/j.jval.2019.07.003","article-title":"Economic Evaluation of Systemic Treatments for Advanced Melanoma: A Systematic Review","volume":"23","author":"Gorry","year":"2020","journal-title":"Value Health"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"2064","DOI":"10.1002\/1097-0142(19911101)68:9<2064::AID-CNCR2820680937>3.0.CO;2-3","article-title":"Delay in the diagnosis of cutaneous malignant melanoma. A prospective study in 250 patients","volume":"68","author":"Krige","year":"1991","journal-title":"Cancer"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"271","DOI":"10.1002\/1097-0215(20000520)89:3<271::AID-IJC10>3.0.CO;2-7","article-title":"Delays in diagnosis and melanoma prognosis (I): The role of patients","volume":"89","author":"Richard","year":"2000","journal-title":"Int. J. Cancer"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1016\/j.jaad.2004.04.031","article-title":"Melanoma detection in British Columbia, Canada","volume":"52","author":"Tyler","year":"2005","journal-title":"J. Am. Acad. Dermatol."},{"key":"ref_48","unstructured":"Fujisawa, Y. (2017). Japanese Melanoma Study: Annual Report 2017. Jpn. Ski. Cancer Soc., Available online: http:\/\/www.skincancer.jp\/report-skincancer_melanoma_2017.pdf."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"581","DOI":"10.1038\/bjc.2014.304","article-title":"Risk factors for delay in symptomatic presentation: A survey of cancer patients","volume":"111","author":"Forbes","year":"2014","journal-title":"Br. J. Cancer"},{"key":"ref_50","unstructured":"(2019). Melanoma of the Skin 2019. Cancer Stat Facts, National Cancer Institute."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.neucom.2016.12.038","article-title":"A survey of deep neural network architectures and their applications","volume":"234","author":"Liu","year":"2017","journal-title":"Neurocomputing"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"106838","DOI":"10.1109\/ACCESS.2019.2932731","article-title":"An improved faster R-CNN for small object detection","volume":"7","author":"Cao","year":"2019","journal-title":"IEEE Access"}],"container-title":["Biomolecules"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2218-273X\/10\/8\/1123\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:52:45Z","timestamp":1760176365000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2218-273X\/10\/8\/1123"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,7,29]]},"references-count":52,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2020,8]]}},"alternative-id":["biom10081123"],"URL":"https:\/\/doi.org\/10.3390\/biom10081123","relation":{},"ISSN":["2218-273X"],"issn-type":[{"value":"2218-273X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,7,29]]}}}