{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,31]],"date-time":"2026-03-31T13:59:18Z","timestamp":1774965558875,"version":"3.50.1"},"reference-count":70,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,3,25]],"date-time":"2024-03-25T00:00:00Z","timestamp":1711324800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Big Data"],"abstract":"<jats:sec><jats:title>Background<\/jats:title><jats:p>Melanoma is one of the deadliest skin cancers that originate from melanocytes due to sun exposure, causing mutations. Early detection boosts the cure rate to 90%, but misclassification drops survival to 15\u201320%. Clinical variations challenge dermatologists in distinguishing benign nevi and melanomas. Current diagnostic methods, including visual analysis and dermoscopy, have limitations, emphasizing the need for Artificial Intelligence understanding in dermatology.<\/jats:p><\/jats:sec><jats:sec><jats:title>Objectives<\/jats:title><jats:p>In this paper, we aim to explore dermoscopic structures for the classification of melanoma lesions. The training of AI models faces a challenge known as brittleness, where small changes in input images impact the classification. A study explored AI vulnerability in discerning melanoma from benign lesions using features of size, color, and shape. Tests with artificial and natural variations revealed a notable decline in accuracy, emphasizing the necessity for additional information, such as dermoscopic structures.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methodology<\/jats:title><jats:p>The study utilizes datasets with clinically marked dermoscopic images examined by expert clinicians. Transformers and CNN-based models are employed to classify these images based on dermoscopic structures. Classification results are validated using feature visualization. To assess model susceptibility to image variations, classifiers are evaluated on test sets with original, duplicated, and digitally modified images. Additionally, testing is done on ISIC 2016 images. The study focuses on three dermoscopic structures crucial for melanoma detection: Blue-white veil, dots\/globules, and streaks.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>In evaluating model performance, adding convolutions to Vision Transformers proves highly effective for achieving up to 98% accuracy. CNN architectures like VGG-16 and DenseNet-121 reach 50\u201360% accuracy, performing best with features other than dermoscopic structures. Vision Transformers without convolutions exhibit reduced accuracy on diverse test sets, revealing their brittleness. OpenAI Clip, a pre-trained model, consistently performs well across various test sets. To address brittleness, a mitigation method involving extensive data augmentation during training and 23 transformed duplicates during test time, sustains accuracy.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>This paper proposes a melanoma classification scheme utilizing three dermoscopic structures across Ph2 and Derm7pt datasets. The study addresses AI susceptibility to image variations. Despite a small dataset, future work suggests collecting more annotated datasets and automatic computation of dermoscopic structural features.<\/jats:p><\/jats:sec>","DOI":"10.3389\/fdata.2024.1366312","type":"journal-article","created":{"date-parts":[[2024,3,25]],"date-time":"2024-03-25T04:59:17Z","timestamp":1711342757000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["Exploring dermoscopic structures for melanoma lesions' classification"],"prefix":"10.3389","volume":"7","author":[{"given":"Fiza Saeed","family":"Malik","sequence":"first","affiliation":[]},{"given":"Muhammad Haroon","family":"Yousaf","sequence":"additional","affiliation":[]},{"given":"Hassan Ahmed","family":"Sial","sequence":"additional","affiliation":[]},{"given":"Serestina","family":"Viriri","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2024,3,25]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"811","DOI":"10.1007\/s10462-020-09865-y","article-title":"Deep learning techniques for skin lesion analysis and melanoma cancer detection: a survey of state-of-the-art","volume":"54","author":"Adegun","year":"2021","journal-title":"Artif. Intell. Rev"},{"key":"B2","doi-asserted-by":"publisher","first-page":"11873","DOI":"10.1007\/s11042-022-13618-0","article-title":"A new method proposed to melanoma-skin cancer lesion detection and segmentation based on hybrid convolutional neural network","volume":"82","author":"Ahmed","year":"2023","journal-title":"Multim. Tools Applic"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.1109\/SIU49456.2020.9302125","article-title":"\u201cSkin lesion classification with deep CNN ensembles,\u201d","author":"Ahmed","year":"2020","journal-title":"2020 28th Signal Processing and Communications Applications Conference (SIU)"},{"key":"B4","doi-asserted-by":"publisher","first-page":"421","DOI":"10.1117\/12.912389","article-title":"A systematic review of automated melanoma detection in dermatoscopic images and its ground truth data","volume":"8318","author":"Ali","year":"2012","journal-title":"Med. Imaging"},{"key":"B5","article-title":"\u201cSkin lesion classification and segmentation for imbalanced classes using deep learning,\u201d","author":"Amro","year":"2018","journal-title":"ISIC Challenge 2018"},{"key":"B6","doi-asserted-by":"publisher","first-page":"1563","DOI":"10.1001\/archderm.134.12.1563","article-title":"Epiluminescence microscopy for the diagnosis of doubtful melanocytic skin lesions: comparison of the abcd rule of dermatoscopy and a new 7-point checklist based on pattern analysis","volume":"134","author":"Argenziano","year":"1998","journal-title":"Arch. Dermatol"},{"key":"B7","doi-asserted-by":"publisher","first-page":"679","DOI":"10.1067\/mjd.2003.281","article-title":"Dermoscopy of pigmented skin lesions: results of a consensus meeting via the internet","volume":"48","author":"Argenziano","year":"2003","journal-title":"J. Am. Acad. Dermatol"},{"key":"B8","article-title":"Why do deep convolutional networks generalize so poorly to small image transformations?","author":"Azulay","year":"2018"},{"key":"B9","doi-asserted-by":"publisher","first-page":"1096","DOI":"10.1109\/JBHI.2018.2845939","article-title":"A survey of feature extraction in dermoscopy image analysis of skin cancer","volume":"23","author":"Barata","year":"2018","journal-title":"IEEE J. Biomed. Health Inf"},{"key":"B10","doi-asserted-by":"crossref","first-page":"3527","DOI":"10.1109\/ICIP.2014.7025716","article-title":"\u201cImproving dermoscopy image analysis using color constancy,\u201d","volume-title":"2014 IEEE International Conference on Image Processing (ICIP)","author":"Barata","year":"2014"},{"key":"B11","article-title":"Deep-learning ensembles for skin-lesion segmentation, analysis, classification: RECOD titans at ISIC challenge 2018","author":"Bissoto","year":"2018","journal-title":"arXiv preprint arXiv:1808.08480"},{"key":"B12","doi-asserted-by":"publisher","first-page":"1556","DOI":"10.1001\/archderm.138.12.1556","article-title":"Dermoscopy of pigmented seborrheic keratosis: a morphological study","volume":"138","author":"Braun","year":"2002","journal-title":"Arch. Dermatol"},{"key":"B13","doi-asserted-by":"publisher","first-page":"109","DOI":"10.1016\/j.jaad.2001.11.001","article-title":"Dermoscopy of pigmented skin lesions","volume":"52","author":"Braun","year":"2005","journal-title":"J. Am. Acad. Dermatol"},{"key":"B14","doi-asserted-by":"publisher","first-page":"200109","DOI":"10.1016\/j.iswa.2022.200109","article-title":"Argumentation approaches for explainable AI in medical informatics","volume":"16","author":"Caroprese","year":"2022","journal-title":"Intell. Syst. Applic"},{"key":"B15","doi-asserted-by":"publisher","first-page":"798","DOI":"10.1001\/jamadermatol.2016.0624","article-title":"Validity and reliability of dermoscopic criteria used to differentiate nevi from melanoma: a web-based international dermoscopy society study","volume":"152","author":"Carrera","year":"2016","journal-title":"JAMA Dermatol"},{"key":"B16","doi-asserted-by":"publisher","first-page":"362","DOI":"10.1016\/j.compmedimag.2007.01.003","article-title":"A methodological approach to the classification of dermoscopy images","volume":"31","author":"Celebi","year":"2007","journal-title":"Computer. Med. Imag. Graph"},{"key":"B17","article-title":"An image is worth 16x16 words: Transformers for image recognition at scale","author":"Dosovitskiy","year":"2020","journal-title":"arXiv preprint arXiv:2010.11929"},{"key":"B18","doi-asserted-by":"publisher","first-page":"1753","DOI":"10.3390\/s20061753","article-title":"Deep learning-based methods for automatic diagnosis of skin lesions","volume":"20","author":"El-Khatib","year":"2020","journal-title":"Sensors"},{"key":"B19","first-page":"1802","article-title":"\u201cExploring the landscape of spatial robustness,\u201d","volume-title":"International Conference on Machine Learning","author":"Engstrom","year":"2019"},{"key":"B20","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1038\/nature21056","article-title":"Dermatologist-level classification of skin cancer with deep neural networks","volume":"542","author":"Esteva","year":"2017","journal-title":"Nature"},{"key":"B21","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1155\/2011\/125762","article-title":"Teledermatology: from prevention to diagnosis of nonmelanoma and melanoma skin cancer","volume":"2011","author":"Fabbrocini","year":"2011","journal-title":"Int. J. Telemed. Applic"},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.5244\/C.29.106","article-title":"Manitest: are classifiers really invariant?","author":"Fawzi","year":"2015"},{"key":"B23","doi-asserted-by":"publisher","first-page":"1287","DOI":"10.1126\/science.aaw4399","article-title":"Adversarial attacks on medical machine learning","volume":"363","author":"Finlayson","year":"2019","journal-title":"Science"},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.1016\/j.mex.2020.100864","article-title":"\u201cSkin lesion classification using loss balancing and ensembles of multi-resolution efficientnets,\u201d","author":"Gessert","year":"2019","journal-title":"ISIC Chellange"},{"key":"B25","article-title":"Skin lesion diagnosis using ensembles, unscaled multi-crop evaluation and loss weighting","author":"Gessert","year":"2018","journal-title":"arXiv preprint arXiv:1808.01694"},{"key":"B26","doi-asserted-by":"publisher","first-page":"202","DOI":"10.1016\/j.ejca.2021.06.049","article-title":"Skin cancer classification via convolutional neural networks: systematic review of studies involving human experts","volume":"156","author":"Haggenm\u00fcller","year":"2021","journal-title":"Eur. J. Cancer"},{"key":"B27","doi-asserted-by":"publisher","first-page":"770","DOI":"10.1109\/CVPR.2016.90","article-title":"\u201cDeep residual learning for image recognition,\u201d","author":"He","year":"2016","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B28","doi-asserted-by":"publisher","first-page":"163","DOI":"10.1038\/d41586-019-03013-5","article-title":"Why deep-learning ais are so easy to fool","volume":"574","author":"Heaven","year":"2019","journal-title":"Nature"},{"key":"B29","doi-asserted-by":"publisher","first-page":"e20708","DOI":"10.2196\/20708","article-title":"Integrating patient data into skin cancer classification using convolutional neural networks: systematic review","volume":"23","author":"H\u00f6hn","year":"2021","journal-title":"J. Med. Internet Res"},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.243","article-title":"\u201cDensely connected convolutional networks,\u201d","author":"Huang","year":"2017","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B31","doi-asserted-by":"publisher","first-page":"101843","DOI":"10.1016\/j.compmedimag.2020.101843","article-title":"Automated multi-class classification of skin lesions through deep convolutional neural network with dermoscopic images","volume":"88","author":"Iqbal","year":"2021","journal-title":"Computer. Med. Imaging Graph"},{"key":"B32","doi-asserted-by":"publisher","first-page":"1123","DOI":"10.3390\/biom10081123","article-title":"The development of a skin cancer classification system for pigmented skin lesions using deep learning","volume":"10","author":"Jinnai","year":"2020","journal-title":"Biomolecules"},{"key":"B33","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12880-020-00534-8","article-title":"Melanoma diagnosis using deep learning techniques on dermatoscopic images","volume":"21","author":"Jojoa Acosta","year":"2021","journal-title":"BMC Med. Imag"},{"key":"B34","doi-asserted-by":"publisher","first-page":"76","DOI":"10.1016\/j.tice.2019.04.009","article-title":"A comparative study of deep learning architectures on melanoma detection","volume":"58","author":"Kassani","year":"2019","journal-title":"Tissue Cell"},{"key":"B35","doi-asserted-by":"publisher","first-page":"538","DOI":"10.1109\/JBHI.2018.2824327","article-title":"Seven-point checklist and skin lesion classification using multitask multimodal neural nets","volume":"23","author":"Kawahara","year":"2018","journal-title":"IEEE J. Biomed. Health Inform"},{"key":"B36","doi-asserted-by":"publisher","first-page":"1096","DOI":"10.1038\/s41467-019-08987-4","article-title":"Unmasking clever hans predictors and assessing what machines really learn","volume":"10","author":"Lapuschkin","year":"2019","journal-title":"Nature Commun"},{"key":"B37","article-title":"Wonderm: skin lesion classification with fine-tuned neural networks","author":"Lee","year":"2018","journal-title":"arXiv preprint arXiv:1808.03426"},{"key":"B38","doi-asserted-by":"publisher","first-page":"556","DOI":"10.3390\/s18020556","article-title":"Skin lesion analysis towards melanoma detection using deep learning network","volume":"18","author":"Li","year":"2018","journal-title":"Sensors"},{"key":"B39","doi-asserted-by":"crossref","first-page":"5437","DOI":"10.1109\/EMBC.2013.6610779","article-title":"\u201cPh 2-a dermoscopic image database for research and benchmarking,\u201d","volume-title":"2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)","author":"Mendon\u00e7a","year":"2013"},{"key":"B40","doi-asserted-by":"publisher","first-page":"1167","DOI":"10.1001\/jamadermatol.2019.1514","article-title":"Evaluation of the number-needed-to-biopsy metric for the diagnosis of cutaneous melanoma: a systematic review and meta-analysis","volume":"155","author":"Nelson","year":"2019","journal-title":"JAMA Dermatol"},{"key":"B41","article-title":"\u201cEnsembling convolutional neural networks for skin cancer classification,\u201d","author":"Nozdryn-Plotnicki","year":"2018","journal-title":"International Skin Imaging Collaboration (ISIC) Challenge on Skin Image Analysis for Melanoma Detection. MICCAI"},{"key":"B42","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1016\/j.engappai.2018.04.028","article-title":"A survey on automated melanoma detection","volume":"73","author":"Okur","year":"2018","journal-title":"Eng. Applic. Artif. Intell"},{"key":"B43","article-title":"Skin cancer detection based on deep learning and entropy to detect outlier samples","author":"Pacheco","year":"2019","journal-title":"arXiv preprint arXiv:1909.04525"},{"key":"B44","article-title":"Residual network based aggregation model for skin lesion classification","author":"Pan","year":"2018","journal-title":"arXiv preprint arXiv:1807.09150"},{"key":"B45","first-page":"8748","article-title":"\u201cLearning transferable visual models from natural language supervision,\u201d","volume-title":"International Conference on Machine Learning","author":"Radford","year":"2021"},{"key":"B46","doi-asserted-by":"publisher","first-page":"107040","DOI":"10.1016\/j.cmpb.2022.107040","article-title":"Dermocc-gan: a new approach for standardizing dermatological images using generative adversarial networks","volume":"225","author":"Salvi","year":"2022","journal-title":"Comput. Methods Progr. Biomed"},{"key":"B47","doi-asserted-by":"publisher","first-page":"e23436","DOI":"10.2196\/23436","article-title":"Hidden variables in deep learning digital pathology and their potential to cause batch effects: Prediction model study","volume":"23","author":"Schmitt","year":"2021","journal-title":"J. Med. Internet Res"},{"key":"B48","doi-asserted-by":"publisher","first-page":"1398","DOI":"10.1111\/j.1524-4725.2006.32312.x","article-title":"Nonmelanocytic lesions defying the two-step dermoscopy algorithm","volume":"32","author":"Scope","year":"2006","journal-title":"Dermatol. Surg"},{"key":"B49","article-title":"Very deep convolutional networks for large-scale image recognition","author":"Simonyan","year":"2014","journal-title":"arXiv preprint arXiv:1409.1556"},{"key":"B50","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1159\/000075042","article-title":"Three-point checklist of dermoscopya new screening method for early detection of melanoma","volume":"208","author":"Soyer","year":"2004","journal-title":"Dermatology"},{"key":"B51","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1109\/ICICICT46008.2019.8993219","article-title":"\u201cSkin lesion analysis towards melanoma detection,\u201d","volume-title":"2019 2nd International Conference on Intelligent Computing, Instrumentation and Control Technologies (ICICICT)","author":"Sreena","year":"2019"},{"key":"B52","doi-asserted-by":"publisher","first-page":"2852","DOI":"10.3390\/s21082852","article-title":"Classification of skin disease using deep learning neural networks with mobilenet v2 and LSTM","volume":"21","author":"Srinivasu","year":"2021","journal-title":"Sensors"},{"key":"B53","doi-asserted-by":"publisher","first-page":"203","DOI":"10.1111\/pcmr.13027","article-title":"Artificial intelligence and melanoma: A comprehensive review of clinical, dermoscopic, and histologic applications","volume":"35","author":"Stiff","year":"2022","journal-title":"Pigment Cell Melan. Res"},{"key":"B54","doi-asserted-by":"publisher","first-page":"1222","DOI":"10.1111\/j.1468-3083.2010.03920.x","article-title":"Cloudy and starry milia-like cysts: how well do they distinguish seborrheic keratoses from malignant melanomas?","volume":"25","author":"Stricklin","year":"2011","journal-title":"J. Eur. Acad. Dermatol. Venereol"},{"key":"B55","unstructured":"Melanoma Skin Cancer Causes, Risk Factors, and Prevention2023"},{"key":"B56","author":"T\u00f4","year":"2019","journal-title":"Ensembled skin cancer classification (ISIC 2019 challenge submission)"},{"key":"B57","doi-asserted-by":"publisher","first-page":"669","DOI":"10.1111\/j.1365-2133.2008.08713.x","article-title":"Dermoscopy compared with naked eye examination for the diagnosis of primary melanoma: a meta-analysis of studies performed in a clinical setting","volume":"159","author":"Vestergaard","year":"2008","journal-title":"Br. J. Dermatol"},{"key":"B58","doi-asserted-by":"publisher","first-page":"2310","DOI":"10.1109\/BIBM47256.2019.8983165","article-title":"\u201cMachine learning techniques for automated melanoma detection,\u201d","author":"Vocaturo","year":"2019","journal-title":"2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)"},{"key":"B59","doi-asserted-by":"publisher","DOI":"10.1145\/3410566.3410611","article-title":"\u201cDc-smil: a multiple instance learning solution via spherical separation for automated detection of displastyc nevi,\u201d","author":"Vocaturo","year":"2020","journal-title":"Proceedings of the 24th Symposium on International Database Engineering"},{"key":"B60","doi-asserted-by":"publisher","first-page":"102428","DOI":"10.1016\/j.bspc.2021.102428","article-title":"Unlabeled skin lesion classification by self-supervised topology clustering network","volume":"66","author":"Wang","year":"2021","journal-title":"Biomed. Signal Proc. Control"},{"key":"B61","doi-asserted-by":"publisher","first-page":"1135","DOI":"10.1001\/jamadermatol.2019.1735","article-title":"Association between surgical skin markings in dermoscopic images and diagnostic performance of a deep learning convolutional neural network for melanoma recognition","volume":"155","author":"Winkler","year":"2019","journal-title":"JAMA Dermatol"},{"key":"B62","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1109\/ICCV48922.2021.00009","article-title":"\u201cCvt: introducing convolutions to vision transformers,\u201d","author":"Wu","year":"2021","journal-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision"},{"key":"B63","doi-asserted-by":"publisher","first-page":"849","DOI":"10.1109\/TMI.2016.2633551","article-title":"Melanoma classification on dermoscopy images using a neural network ensemble model","volume":"36","author":"Xie","year":"2016","journal-title":"IEEE Trans. Med. Imag"},{"key":"B64","doi-asserted-by":"publisher","first-page":"1261","DOI":"10.1111\/exd.13777","article-title":"Multimodal skin lesion classification using deep learning","volume":"27","author":"Yap","year":"2018","journal-title":"Exper. Dermatol"},{"key":"B65","doi-asserted-by":"publisher","first-page":"e1002683","DOI":"10.1371\/journal.pmed.1002683","article-title":"Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: a cross-sectional study","volume":"15","author":"Zech","year":"2018","journal-title":"PLoS Med"},{"key":"B66","doi-asserted-by":"publisher","first-page":"2092","DOI":"10.1109\/TMI.2019.2893944","article-title":"Attention residual learning for skin lesion classification","volume":"38","author":"Zhang","year":"2019","journal-title":"IEEE Trans. Med. Imag"},{"key":"B67","article-title":"\u201cMelanet: a deep dense attention network for melanoma detection in dermoscopy images,\u201d","author":"Zhang","year":"2019","journal-title":"ISIC"},{"key":"B68","first-page":"7324","article-title":"\u201cMaking convolutional networks shift-invariant again,\u201d","volume-title":"International Conference on Machine Learning","author":"Zhang","year":"2019"},{"key":"B69","doi-asserted-by":"publisher","first-page":"2921","DOI":"10.1109\/CVPR.2016.319","article-title":"\u201cLearning deep features for discriminative localization,\u201d","author":"Zhou","year":"2016","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B70","article-title":"\u201cMulti-category skin lesion diagnosis using dermoscopy images and deep CNN ensembles,\u201d","author":"Zhou","year":"2019","journal-title":"ISIC Chellange"}],"container-title":["Frontiers in Big Data"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdata.2024.1366312\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,25]],"date-time":"2024-03-25T04:59:33Z","timestamp":1711342773000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdata.2024.1366312\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,25]]},"references-count":70,"alternative-id":["10.3389\/fdata.2024.1366312"],"URL":"https:\/\/doi.org\/10.3389\/fdata.2024.1366312","relation":{},"ISSN":["2624-909X"],"issn-type":[{"value":"2624-909X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,25]]},"article-number":"1366312"}}