{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T21:05:24Z","timestamp":1787087124926,"version":"build-2736575974"},"reference-count":54,"publisher":"Wiley","issue":"4","license":[{"start":{"date-parts":[[2023,1,2]],"date-time":"2023-01-02T00:00:00Z","timestamp":1672617600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Acad Dermatol Venereol"],"published-print":{"date-parts":[[2023,4]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Machine learning (ML) models for skin cancer recognition may have variable performance across different skin phototypes and skin cancer types. Overall performance metrics alone are insufficient to detect poor subgroup performance. We aimed (1) to assess whether studies of ML models reported results separately for different skin phototypes and rarer skin cancers, and (2) to graphically represent the skin cancer training datasets used by current ML models. In this systematic review, we searched PubMed, Embase and CENTRAL. We included all studies in medical journals assessing an ML technique for skin cancer diagnosis that used clinical or dermoscopic images from 1 January 2012 to 22 September 2021. No language restrictions were applied. We considered rarer skin cancers to be skin cancers other than pigmented melanoma, basal cell carcinoma and squamous cell carcinoma. We identified 114 studies for inclusion. Rarer skin cancers were included by 8\/114 studies (7.0%), and results for a rarer skin cancer were reported separately in 1\/114 studies (0.9%). Performance was reported across all skin phototypes in 1\/114 studies (0.9%), but performance was uncertain in skin phototypes I and VI from minimal representation of the skin phototypes in the test dataset (9\/3756 and 1\/3756, respectively). For training datasets, although public datasets were most frequently used, with the most widely used being the International Skin Imaging Collaboration (ISIC) archive (65\/114 studies, 57.0%), the largest datasets were private. Our review identified that most ML models did not report performance separately for rarer skin cancers and different skin phototypes. A degree of variability in ML model performance across subgroups is expected, but the current lack of transparency is not justifiable and risks models being used inappropriately in populations in whom accuracy is low.<\/jats:p>","DOI":"10.1111\/jdv.18814","type":"journal-article","created":{"date-parts":[[2022,12,14]],"date-time":"2022-12-14T09:21:02Z","timestamp":1671009662000},"page":"657-665","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Determining the clinical applicability of machine learning models through assessment of reporting across skin phototypes and rarer skin cancer types: A systematic review"],"prefix":"10.1111","volume":"37","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4745-1338","authenticated-orcid":false,"given":"Lloyd","family":"Steele","sequence":"first","affiliation":[{"name":"Department of Dermatology The Royal London Hospital  London UK"},{"name":"Centre for Cell Biology and Cutaneous Research, Blizard Institute Queen Mary University of London  London UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiang Li","family":"Tan","sequence":"additional","affiliation":[{"name":"St George's University Hospitals NHS Foundation Trust  London UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bayanne","family":"Olabi","sequence":"additional","affiliation":[{"name":"Biosciences Institute Newcastle University  Newcastle UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jing Mia","family":"Gao","sequence":"additional","affiliation":[{"name":"Department of Dermatology The Royal London Hospital  London UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0769-9382","authenticated-orcid":false,"given":"Reiko J.","family":"Tanaka","sequence":"additional","affiliation":[{"name":"Department of Bioengineering Imperial College London  London UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5646-3093","authenticated-orcid":false,"given":"Hywel C.","family":"Williams","sequence":"additional","affiliation":[{"name":"Centre of Evidence\u2010Based Dermatology, School of Medicine University of Nottingham  Nottingham UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2023,1,2]]},"reference":[{"key":"e_1_2_11_2_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature21056"},{"key":"e_1_2_11_3_1","doi-asserted-by":"publisher","DOI":"10.1001\/jamadermatol.2018.4378"},{"key":"e_1_2_11_4_1","doi-asserted-by":"publisher","DOI":"10.1126\/scitranslmed.abb3652"},{"key":"e_1_2_11_5_1","doi-asserted-by":"publisher","DOI":"10.1093\/annonc\/mdy166"},{"key":"e_1_2_11_6_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-020-0842-3"},{"key":"e_1_2_11_7_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-021-96707-8"},{"key":"e_1_2_11_8_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmjinnov-2019-000359"},{"key":"e_1_2_11_9_1","unstructured":"American Academy of Dermatology.Position Statement on Augmented Intelligence (AuI).2018.https:\/\/server.aad.org\/Forms\/Policies\/Uploads\/PS\/PS\u2010Augmented%20Intelligence.pdfAccessed 21 June 2021."},{"key":"e_1_2_11_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300830"},{"key":"e_1_2_11_11_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1912.10389"},{"key":"e_1_2_11_12_1","doi-asserted-by":"publisher","DOI":"10.1001\/jamadermatol.2021.3129"},{"key":"e_1_2_11_13_1","doi-asserted-by":"publisher","DOI":"10.1016\/S2589-7500(21)00252-1"},{"key":"e_1_2_11_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jaad.2021.10.010"},{"key":"e_1_2_11_15_1","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-021-00370-7"},{"key":"e_1_2_11_16_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.m127"},{"key":"e_1_2_11_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3368555.3384468"},{"key":"e_1_2_11_18_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1810.03993"},{"key":"e_1_2_11_19_1","doi-asserted-by":"publisher","DOI":"10.1016\/S2589-7500(22)00003-6"},{"key":"e_1_2_11_20_1","unstructured":"ClarkJ.GPT\u20102 Model Card.2019.https:\/\/github.com\/openai\/gpt\u20102\/blob\/master\/model_card.md. Accessed 9 Jan 2022."},{"key":"e_1_2_11_21_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1803.09010"},{"key":"e_1_2_11_22_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.b2700"},{"key":"e_1_2_11_23_1","first-page":"1090","article-title":"ImageNet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky A","year":"2012","journal-title":"Proc Adv Neural Inform Process Syst"},{"key":"e_1_2_11_24_1","doi-asserted-by":"publisher","DOI":"10.1111\/jdv.15935"},{"key":"e_1_2_11_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10916-019-1414-2"},{"key":"e_1_2_11_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMI.2016.2633551"},{"key":"e_1_2_11_27_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jid.2018.01.028"},{"key":"e_1_2_11_28_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jid.2020.01.019"},{"key":"e_1_2_11_29_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jid.2020.06.040"},{"key":"e_1_2_11_30_1","doi-asserted-by":"publisher","DOI":"10.1111\/bjd.16924"},{"key":"e_1_2_11_31_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ejca.2019.11.020"},{"issue":"9","key":"e_1_2_11_32_1","first-page":"1063","article-title":"Clinical image identification of basal cell carcinoma and pigmented nevi based on convolutional neural network","volume":"44","author":"Xie B","year":"2019","journal-title":"Zhong Nan Da Xue Xue Bao Yi Xue Ban"},{"key":"e_1_2_11_33_1","doi-asserted-by":"publisher","DOI":"10.1097\/CM9.0000000000001002"},{"key":"e_1_2_11_34_1","doi-asserted-by":"publisher","DOI":"10.1001\/jamadermatol.2019.3807"},{"key":"e_1_2_11_35_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11548-021-02440-y"},{"key":"e_1_2_11_36_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.annonc.2019.10.013"},{"key":"e_1_2_11_37_1","first-page":"1","article-title":"Gender shades: intersectional accuracy disparities in commercial gender classification","volume":"81","author":"Buolamwini J","year":"2018","journal-title":"Proc Mach Learn Res"},{"key":"e_1_2_11_38_1","doi-asserted-by":"publisher","DOI":"10.1001\/jamadermatol.2018.2348"},{"key":"e_1_2_11_39_1","doi-asserted-by":"publisher","DOI":"10.1038\/d41586-018-05707-8"},{"key":"e_1_2_11_40_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2104.09957"},{"key":"e_1_2_11_41_1","doi-asserted-by":"publisher","DOI":"10.1101\/826057v1"},{"key":"e_1_2_11_42_1","doi-asserted-by":"publisher","DOI":"10.1126\/sciadv.abq6147"},{"key":"e_1_2_11_43_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2021.102274"},{"key":"e_1_2_11_44_1","doi-asserted-by":"publisher","DOI":"10.1016\/S2589-7500(22)00021-8"},{"key":"e_1_2_11_45_1","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-020-00257-z"},{"key":"e_1_2_11_46_1","doi-asserted-by":"publisher","DOI":"10.1111\/jdv.17517"},{"key":"e_1_2_11_47_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-019-0548-6"},{"key":"e_1_2_11_48_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1910.13268"},{"key":"e_1_2_11_49_1","doi-asserted-by":"publisher","DOI":"10.1001\/jamadermatol.2021.3128"},{"key":"e_1_2_11_50_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1901.10436"},{"key":"e_1_2_11_51_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41592-021-01362-6"},{"key":"e_1_2_11_52_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41592-021-01380-4"},{"key":"e_1_2_11_53_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2022.105966"},{"key":"e_1_2_11_54_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2003.09365"},{"key":"e_1_2_11_55_1","doi-asserted-by":"crossref","unstructured":"CombaliaM HuetoF PuigS MalvehyJ VilaplanaV.Uncertainty estimation in deep neural networks for Dermoscopic image classification. IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops2020:744\u20135.","DOI":"10.1109\/CVPRW50498.2020.00380"}],"container-title":["Journal of the European Academy of Dermatology and Venereology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/jdv.18814","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1111\/jdv.18814","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/jdv.18814","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,20]],"date-time":"2023-09-20T12:11:33Z","timestamp":1695211893000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/jdv.18814"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,2]]},"references-count":54,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,4]]}},"alternative-id":["10.1111\/jdv.18814"],"URL":"https:\/\/doi.org\/10.1111\/jdv.18814","archive":["Portico"],"relation":{},"ISSN":["0926-9959","1468-3083"],"issn-type":[{"value":"0926-9959","type":"print"},{"value":"1468-3083","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,2]]},"assertion":[{"value":"2022-05-13","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-11-09","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-01-02","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}