{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,4]],"date-time":"2026-06-04T19:10:05Z","timestamp":1780600205283,"version":"3.54.1"},"reference-count":78,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,5,14]],"date-time":"2024-05-14T00:00:00Z","timestamp":1715644800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,5,14]],"date-time":"2024-05-14T00:00:00Z","timestamp":1715644800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Scientific research of artificial intelligence (AI) in dermatology has increased exponentially. The objective of this study was to perform a systematic review and meta-analysis to evaluate the performance of AI algorithms for skin cancer classification in comparison to clinicians with different levels of expertise. Based on PRISMA guidelines, 3 electronic databases (PubMed, Embase, and Cochrane Library) were screened for relevant articles up to August 2022. The quality of the studies was assessed using QUADAS-2. A meta-analysis of sensitivity and specificity was performed for the accuracy of AI and clinicians. Fifty-three studies were included in the systematic review, and 19 met the inclusion criteria for the meta-analysis. Considering all studies and all subgroups of clinicians, we found a sensitivity (Sn) and specificity (Sp) of 87.0% and 77.1% for AI algorithms, respectively, and a Sn of 79.78% and Sp of 73.6% for all clinicians (overall); differences were statistically significant for both Sn and Sp. The difference between AI performance (Sn 92.5%, Sp 66.5%) vs. generalists (Sn 64.6%, Sp 72.8%), was greater, when compared with expert clinicians. Performance between AI algorithms (Sn 86.3%, Sp 78.4%) vs expert dermatologists (Sn 84.2%, Sp 74.4%) was clinically comparable. Limitations of AI algorithms in clinical practice should be considered, and future studies should focus on real-world settings, and towards AI-assistance.<\/jats:p>","DOI":"10.1038\/s41746-024-01103-x","type":"journal-article","created":{"date-parts":[[2024,5,14]],"date-time":"2024-05-14T19:02:04Z","timestamp":1715713324000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":113,"title":["A systematic review and meta-analysis of artificial intelligence versus clinicians for skin cancer diagnosis"],"prefix":"10.1038","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5610-988X","authenticated-orcid":false,"given":"Maria Paz","family":"Salinas","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Javiera","family":"Sep\u00falveda","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Leonel","family":"Hidalgo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dominga","family":"Peirano","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Macarena","family":"Morel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pablo","family":"Uribe","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0639-2677","authenticated-orcid":false,"given":"Veronica","family":"Rotemberg","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4373-0014","authenticated-orcid":false,"given":"Juan","family":"Briones","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Domingo","family":"Mery","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4040-3640","authenticated-orcid":false,"given":"Cristian","family":"Navarrete-Dechent","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,5,14]]},"reference":[{"key":"1103_CR1","doi-asserted-by":"publisher","first-page":"75","DOI":"10.1016\/j.ypmed.2014.01.003","volume":"61","author":"NA Lakhani","year":"2014","unstructured":"Lakhani, N. A. et al. Total body skin examination for skin cancer screening among U.S. adults from 2000 to 2010. Prev. Med. 61, 75\u201380 (2014).","journal-title":"Prev. Med."},{"key":"1103_CR2","doi-asserted-by":"publisher","first-page":"893972","DOI":"10.3389\/fonc.2022.893972","volume":"12","author":"Y Wu","year":"2022","unstructured":"Wu, Y. et al. Skin cancer classification with deep learning: A systematic review. Front Oncol. 12, 893972 (2022).","journal-title":"Front Oncol."},{"key":"1103_CR3","doi-asserted-by":"publisher","first-page":"e466","DOI":"10.1016\/S2589-7500(22)00023-1","volume":"4","author":"OT Jones","year":"2022","unstructured":"Jones, O. T. et al. Artificial intelligence and machine learning algorithms for early detection of skin cancer in community and primary care settings: a systematic review. Lancet Digit Health 4, e466\u2013e476 (2022).","journal-title":"Lancet Digit Health"},{"key":"1103_CR4","doi-asserted-by":"publisher","unstructured":"Sangers, T. E. et al. Position statement of the EADV Artificial Intelligence (AI) Task Force on AI-assisted smartphone apps and web-based services for skin disease. J. Eur. Acad. Dermatol Venereol https:\/\/doi.org\/10.1111\/jdv.19521 (2023).","DOI":"10.1111\/jdv.19521"},{"key":"1103_CR5","doi-asserted-by":"publisher","first-page":"529","DOI":"10.7326\/0003-4819-155-8-201110180-00009","volume":"155","author":"PF Whiting","year":"2011","unstructured":"Whiting, P. F. QUADAS-2: A revised tool for the quality assessment of diagnostic accuracy studies. Ann. Intern Med. 155, 529 (2011).","journal-title":"Ann. Intern Med."},{"key":"1103_CR6","doi-asserted-by":"publisher","first-page":"106288","DOI":"10.1016\/j.cmpb.2021.106288","volume":"208","author":"F Cabitza","year":"2021","unstructured":"Cabitza, F. et al. The importance of being external. methodological insights for the external validation of machine learning models in medicine. Comput. Methods Prog. Biomed. 208, 106288 (2021).","journal-title":"Comput. Methods Prog. Biomed."},{"key":"1103_CR7","doi-asserted-by":"publisher","first-page":"2078","DOI":"10.1007\/s10620-019-05645-z","volume":"64","author":"D Shung","year":"2019","unstructured":"Shung, D., Simonov, M., Gentry, M., Au, B. & Laine, L. Machine learning to predict outcomes in patients with acute gastrointestinal bleeding: A systematic review. Dig. Dis. Sci. 64, 2078\u20132087 (2019).","journal-title":"Dig. Dis. Sci."},{"key":"1103_CR8","doi-asserted-by":"publisher","first-page":"245","DOI":"10.1016\/j.jclinepi.2015.04.005","volume":"69","author":"EW Steyerberg","year":"2016","unstructured":"Steyerberg, E. W. & Harrell, F. E. Prediction models need appropriate internal, internal\u2013external, and external validation. J. Clin. Epidemiol. 69, 245\u2013247 (2016).","journal-title":"J. Clin. Epidemiol."},{"key":"1103_CR9","doi-asserted-by":"publisher","first-page":"481","DOI":"10.1046\/j.1365-2133.2002.04978.x","volume":"147","author":"D Piccolo","year":"2002","unstructured":"Piccolo, D. et al. Dermoscopic diagnosis by a trained clinician vs. a clinician with minimal dermoscopy training vs. computer-aided diagnosis of 341 pigmented skin lesions: a comparative study. Br. J. Dermatol 147, 481\u2013486 (2002).","journal-title":"Br. J. Dermatol"},{"key":"1103_CR10","unstructured":"The International Skin Imaging Collaboration. https:\/\/www.isic-archive.com\/"},{"key":"1103_CR11","doi-asserted-by":"publisher","first-page":"180161","DOI":"10.1038\/sdata.2018.161","volume":"14","author":"P Tschandl","year":"2018","unstructured":"Tschandl, P. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci. Data 14, 180161 (2018).","journal-title":"Sci. Data"},{"key":"1103_CR12","doi-asserted-by":"publisher","first-page":"476","DOI":"10.1001\/archderm.144.4.476","volume":"144","author":"RJ Friedman","year":"2008","unstructured":"Friedman, R. J. et al. The diagnostic performance of expert dermoscopists vs a computer-vision system on small-diameter melanomas. Arch. Dermatol 144, 476\u2013482 (2008).","journal-title":"Arch. Dermatol"},{"key":"1103_CR13","doi-asserted-by":"publisher","first-page":"270","DOI":"10.1016\/j.jaad.2017.08.016","volume":"78","author":"MA Marchetti","year":"2018","unstructured":"Marchetti, M. A. et al. Results of the 2016 International Skin Imaging Collaboration International Symposium on Biomedical Imaging challenge: Comparison of the accuracy of computer algorithms to dermatologists for the diagnosis of melanoma from dermoscopic images. J. Am. Acad. Dermatol 78, 270 (2018).","journal-title":"J. Am. Acad. Dermatol"},{"key":"1103_CR14","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1016\/j.ejca.2018.12.016","volume":"111","author":"TJ Brinker","year":"2019","unstructured":"Brinker, T. J. et al. Comparing artificial intelligence algorithms to 157 German dermatologists: the melanoma classification benchmark. Eur. J. Cancer 111, 30\u201337 (2019).","journal-title":"Eur. J. Cancer"},{"key":"1103_CR15","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1016\/j.ejca.2019.05.023","volume":"119","author":"TJ Brinker","year":"2019","unstructured":"Brinker, T. J. et al. Deep neural networks are superior to dermatologists in melanoma image classification. Eur. J. Cancer 119, 11\u201317 (2019).","journal-title":"Eur. J. Cancer"},{"key":"1103_CR16","doi-asserted-by":"publisher","first-page":"e18091","DOI":"10.2196\/18091","volume":"22","author":"RC Maron","year":"2020","unstructured":"Maron, R. C. et al. Artificial intelligence and its effect on dermatologists\u2019 accuracy in dermoscopic melanoma image classification: Web-based survey study. J. Med. Internet Res 22, e18091 (2020).","journal-title":"J. Med. Internet Res"},{"key":"1103_CR17","doi-asserted-by":"publisher","first-page":"1842","DOI":"10.1111\/jdv.16185","volume":"34","author":"S Lee","year":"2020","unstructured":"Lee, S. et al. Augmented decision-making for acral lentiginous melanoma detection using deep convolutional neural networks. J. Eur. Acad. Dermatol Venereol. 34, 1842\u20131850 (2020).","journal-title":"J. Eur. Acad. Dermatol Venereol."},{"key":"1103_CR18","doi-asserted-by":"publisher","first-page":"622","DOI":"10.1016\/j.jaad.2019.07.016","volume":"82","author":"MA Marchetti","year":"2020","unstructured":"Marchetti, M. A. et al. Computer algorithms show potential for improving dermatologists\u2019 accuracy to diagnose cutaneous melanoma: Results of the International Skin Imaging Collaboration 2017. J. Am. Acad. Dermatol 82, 622\u2013627 (2020).","journal-title":"J. Am. Acad. Dermatol"},{"key":"1103_CR19","doi-asserted-by":"publisher","first-page":"1355","DOI":"10.1111\/jdv.16165","volume":"34","author":"C Fink","year":"2020","unstructured":"Fink, C. et al. Diagnostic performance of a deep learning convolutional neural network in the differentiation of combined naevi and melanomas. J. Eur. Acad. Dermatol Venereol. 34, 1355\u20131361 (2020).","journal-title":"J. Eur. Acad. Dermatol Venereol."},{"key":"1103_CR20","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1016\/j.jdermsci.2020.11.009","volume":"101","author":"L Tognetti","year":"2021","unstructured":"Tognetti, L. et al. A new deep learning approach integrated with clinical data for the dermoscopic differentiation of early melanomas from atypical nevi. J. Dermatol Sci. 101, 115\u2013122 (2021).","journal-title":"J. Dermatol Sci."},{"key":"1103_CR21","doi-asserted-by":"publisher","first-page":"137","DOI":"10.1016\/j.annonc.2019.10.013","volume":"31","author":"HA Haenssle","year":"2020","unstructured":"Haenssle, H. A. et al. Man against machine reloaded: Performance of a market-approved convolutional neural network in classifying a broad spectrum of skin lesions in comparison with 96 dermatologists working under less artificial conditions. Ann. Oncol. 31, 137\u2013143 (2020).","journal-title":"Ann. Oncol."},{"key":"1103_CR22","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-021-96707-8","volume":"11","author":"TC Pham","year":"2021","unstructured":"Pham, T. C., Luong, C. M., Hoang, V. D. & Doucet, A. AI outperformed every dermatologist in dermoscopic melanoma diagnosis, using an optimized deep-CNN architecture with custom mini-batch logic and loss function. Sci. Rep. 11, 17485 (2021).","journal-title":"Sci. Rep."},{"key":"1103_CR23","doi-asserted-by":"publisher","first-page":"633","DOI":"10.1109\/TMI.2021.3120091","volume":"41","author":"Z Yu","year":"2022","unstructured":"Yu, Z. et al. Early melanoma diagnosis with sequential dermoscopic images. IEEE Trans. Med. Imaging 41, 633\u2013646 (2022).","journal-title":"IEEE Trans. Med. Imaging"},{"key":"1103_CR24","doi-asserted-by":"publisher","first-page":"769","DOI":"10.1016\/j.jaad.2015.07.028","volume":"73","author":"LK Ferris","year":"2015","unstructured":"Ferris, L. K. et al. Computer-aided classification of melanocytic lesions using dermoscopic images. J. Am. Acad. Dermatol 73, 769\u2013776 (2015).","journal-title":"J. Am. Acad. Dermatol"},{"key":"1103_CR25","doi-asserted-by":"publisher","first-page":"867","DOI":"10.1111\/bjd.15695","volume":"177","author":"P Tschandl","year":"2017","unstructured":"Tschandl, P., Kittler, H. & Argenziano, G. A pretrained neural network shows similar diagnostic accuracy to medical students in categorizing dermatoscopic images after comparable training conditions. Br. J. Dermatol. 177, 867\u2013869 (2017).","journal-title":"Br. J. Dermatol."},{"key":"1103_CR26","doi-asserted-by":"publisher","first-page":"e0193321","DOI":"10.1371\/journal.pone.0193321","volume":"13","author":"C Yu","year":"2018","unstructured":"Yu, C. et al. Acral melanoma detection using a convolutional neural network for dermoscopy images. PLoS One 13, e0193321 (2018).","journal-title":"PLoS One"},{"key":"1103_CR27","doi-asserted-by":"publisher","first-page":"e1913436","DOI":"10.1001\/jamanetworkopen.2019.13436","volume":"2","author":"M Phillips","year":"2019","unstructured":"Phillips, M. et al. Assessment of accuracy of an artificial intelligence algorithm to detect melanoma in images of skin lesions. JAMA Netw. Open 2, e1913436 (2019).","journal-title":"JAMA Netw. Open"},{"key":"1103_CR28","doi-asserted-by":"publisher","first-page":"180","DOI":"10.1097\/CMR.0b013e32832a1e41","volume":"19","author":"S Dreiseitl","year":"2009","unstructured":"Dreiseitl, S., Binder, M., Hable, K. & Kittler, H. Computer versus human diagnosis of melanoma: evaluation of the feasibility of an automated diagnostic system in a prospective clinical trial. Melanoma Res. 19, 180\u2013184 (2009).","journal-title":"Melanoma Res."},{"key":"1103_CR29","doi-asserted-by":"publisher","first-page":"180","DOI":"10.1016\/j.ejca.2021.10.030","volume":"160","author":"JK Winkler","year":"2022","unstructured":"Winkler, J. K. et al. Monitoring patients at risk for melanoma: May convolutional neural networks replace the strategy of sequential digital dermoscopy? Eur. J. Cancer 160, 180\u2013188 (2022).","journal-title":"Eur. J. Cancer"},{"key":"1103_CR30","doi-asserted-by":"publisher","first-page":"938","DOI":"10.1016\/S1470-2045(19)30333-X","volume":"20","author":"P Tschandl","year":"2019","unstructured":"Tschandl, P. et al. Comparison of the accuracy of human readers versus machine-learning algorithms for pigmented skin lesion classification: an open, web-based, international, diagnostic study. Lancet Oncol. 20, 938\u2013947 (2019).","journal-title":"Lancet Oncol."},{"key":"1103_CR31","doi-asserted-by":"publisher","first-page":"232","DOI":"10.1111\/1346-8138.15640","volume":"48","author":"A Minagawa","year":"2021","unstructured":"Minagawa, A. et al. Dermoscopic diagnostic performance of Japanese dermatologists for skin tumors differs by patient origin: A deep learning convolutional neural network closes the gap. J. Dermatol 48, 232\u2013236 (2021).","journal-title":"J. Dermatol"},{"key":"1103_CR32","doi-asserted-by":"publisher","first-page":"1229","DOI":"10.1038\/s41591-020-0942-0","volume":"26","author":"P Tschandl","year":"2020","unstructured":"Tschandl, P. et al. Human-computer collaboration for skin cancer recognition. Nat. Med. 26, 1229\u20131234 (2020).","journal-title":"Nat. Med."},{"key":"1103_CR33","doi-asserted-by":"publisher","first-page":"e330","DOI":"10.1016\/S2589-7500(22)00021-8","volume":"4","author":"M Combalia","year":"2022","unstructured":"Combalia, M. et al. Validation of artificial intelligence prediction models for skin cancer diagnosis using dermoscopy images: the 2019 International Skin Imaging Collaboration Grand Challenge. Lancet Digit Health 4, e330\u2013e339 (2022).","journal-title":"Lancet Digit Health"},{"key":"1103_CR34","doi-asserted-by":"publisher","first-page":"2027","DOI":"10.1097\/CM9.0000000000001023","volume":"133","author":"SQ Wang","year":"2020","unstructured":"Wang, S. Q. et al. Deep learning-based, computer-aided classifier developed with dermoscopic images shows comparable performance to 164 dermatologists in cutaneous disease diagnosis in the Chinese population. Chin. Med. J. (Engl.) 133, 2027\u20132036 (2020).","journal-title":"Chin. Med. J. (Engl.)"},{"key":"1103_CR35","doi-asserted-by":"publisher","first-page":"969","DOI":"10.3390\/diagnostics10110969","volume":"10","author":"M Lucius","year":"2020","unstructured":"Lucius, M. et al. Deep neural frameworks improve the accuracy of general practitioners in the classification of pigmented skin lesions. Diagnostics (Basel) 10, 969 (2020).","journal-title":"Diagnostics (Basel)"},{"key":"1103_CR36","doi-asserted-by":"publisher","first-page":"626369","DOI":"10.3389\/fmed.2021.626369","volume":"8","author":"CY Zhu","year":"2021","unstructured":"Zhu, C. Y. et al. A deep learning based framework for diagnosing multiple skin diseases in a clinical environment. Front Med. (Lausanne) 8, 626369 (2021).","journal-title":"Front Med. (Lausanne)"},{"key":"1103_CR37","doi-asserted-by":"publisher","first-page":"1191","DOI":"10.1016\/j.jaad.2022.03.006","volume":"87","author":"P Van Molle","year":"2022","unstructured":"Van Molle, P. et al. The value of measuring uncertainty in neural networks in dermoscopy. J. Am. Acad. Dermatol 87, 1191\u20131193 (2022).","journal-title":"J. Am. Acad. Dermatol"},{"key":"1103_CR38","doi-asserted-by":"publisher","first-page":"114","DOI":"10.1016\/j.ejca.2019.07.019","volume":"120","author":"A Hekler","year":"2019","unstructured":"Hekler, A. et al. Superior skin cancer classification by the combination of human and artificial intelligence. Eur. J. Cancer 120, 114\u2013121 (2019).","journal-title":"Eur. J. Cancer"},{"key":"1103_CR39","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1016\/j.ejca.2019.06.013","volume":"119","author":"RC Maron","year":"2019","unstructured":"Maron, R. C. et al. Systematic outperformance of 112 dermatologists in multiclass skin cancer image classification by convolutional neural networks. Eur. J. Cancer 119, 57\u201365 (2019).","journal-title":"Eur. J. Cancer"},{"key":"1103_CR40","first-page":"1178","volume":"19","author":"JK Winkler","year":"2021","unstructured":"Winkler, J. K. et al. Collective human intelligence outperforms artificial intelligence in a skin lesion classification task. J. Dtsch Dermatol Ges. 19, 1178\u20131184 (2021).","journal-title":"J. Dtsch Dermatol Ges."},{"key":"1103_CR41","doi-asserted-by":"publisher","first-page":"156","DOI":"10.1016\/j.ejca.2022.04.015","volume":"169","author":"W Ba","year":"2022","unstructured":"Ba, W. et al. Convolutional neural network assistance significantly improves dermatologists\u2019 diagnosis of cutaneous tumours using clinical images. Eur. J. Cancer 169, 156\u2013165 (2022).","journal-title":"Eur. J. Cancer"},{"key":"1103_CR42","doi-asserted-by":"publisher","first-page":"1529","DOI":"10.1016\/j.jid.2018.01.028","volume":"138","author":"SS Han","year":"2018","unstructured":"Han, S. S. et al. Classification of the clinical images for benign and malignant cutaneous tumors using a deep learning algorithm. J. Investig. Dermatol. 138, 1529\u20131538 (2018).","journal-title":"J. Investig. Dermatol."},{"key":"1103_CR43","doi-asserted-by":"publisher","first-page":"373","DOI":"10.1111\/bjd.16924","volume":"180","author":"Y Fujisawa","year":"2019","unstructured":"Fujisawa, Y. et al. Deep-learning-based, computer-aided classifier developed with a small dataset of clinical images surpasses board-certified dermatologists in skin tumour diagnosis. Br. J. Dermatol. 180, 373\u2013381 (2019).","journal-title":"Br. J. Dermatol."},{"key":"1103_CR44","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1001\/jamadermatol.2019.3807","volume":"156","author":"SS Han","year":"2020","unstructured":"Han, S. S. et al. Keratinocytic skin cancer detection on the face using region-based convolutional neural network. JAMA Dermatol 156, 29\u201337 (2020).","journal-title":"JAMA Dermatol"},{"key":"1103_CR45","doi-asserted-by":"publisher","first-page":"1713904","DOI":"10.1155\/2020\/1713904","volume":"2020","author":"K Huang","year":"2020","unstructured":"Huang, K. et al. Assistant diagnosis of basal cell carcinoma and seborrheic keratosis in Chinese population using convolutional neural network. J. Health. Eng. 2020, 1713904 (2020).","journal-title":"J. Health. Eng."},{"key":"1103_CR46","doi-asserted-by":"publisher","first-page":"723914","DOI":"10.3389\/fmed.2021.723914","volume":"8","author":"S Polesie","year":"2021","unstructured":"Polesie, S. et al. Discrimination between invasive and in situ melanomas using clinical close-up images and a de novo convolutional neural network. Front Med (Lausanne) 8, 723914 (2021).","journal-title":"Front Med (Lausanne)"},{"key":"1103_CR47","doi-asserted-by":"publisher","first-page":"e76212","DOI":"10.1371\/journal.pone.0076212","volume":"8","author":"WY Chang","year":"2013","unstructured":"Chang, W. Y. et al. Computer-aided diagnosis of skin lesions using conventional digital photography: A reliability and feasibility study. PLoS One 8, e76212 (2013).","journal-title":"PLoS One"},{"key":"1103_CR48","doi-asserted-by":"publisher","DOI":"10.1007\/s10916-019-1414-2","volume":"43","author":"XY Zhao","year":"2019","unstructured":"Zhao, X. Y. et al. The application of deep learning in the risk grading of skin tumors for patients using clinical images. J. Med. Syst. 43, 283 (2019).","journal-title":"J. Med. Syst."},{"key":"1103_CR49","doi-asserted-by":"publisher","first-page":"e51","DOI":"10.1111\/ijd.15242","volume":"60","author":"R Pangti","year":"2021","unstructured":"Pangti, R. et al. Performance of a deep learning-based application for the diagnosis of basal cell carcinoma in Indian patients as compared to dermatologists and nondermatologists. Int J. Dermatol 60, e51\u2013e52 (2021).","journal-title":"Int J. Dermatol"},{"key":"1103_CR50","doi-asserted-by":"publisher","first-page":"1310","DOI":"10.1111\/ced.14688","volume":"46","author":"S Agarwala","year":"2021","unstructured":"Agarwala, S., Mata, D. A. & Hafeez, F. Accuracy of a convolutional neural network for dermatological diagnosis of tumours and skin lesions in a clinical setting. Clin. Exp. Dermatol 46, 1310\u20131311 (2021).","journal-title":"Clin. Exp. Dermatol"},{"key":"1103_CR51","doi-asserted-by":"publisher","first-page":"e0260895","DOI":"10.1371\/journal.pone.0260895","volume":"17","author":"YJ Kim","year":"2022","unstructured":"Kim, Y. J. et al. Augmenting the accuracy of trainee doctors in diagnosing skin lesions suspected of skin neoplasms in a real-world setting: A prospective controlled before-and-after study. PLoS One 17, e0260895 (2022).","journal-title":"PLoS One"},{"key":"1103_CR52","doi-asserted-by":"publisher","first-page":"e1003381","DOI":"10.1371\/journal.pmed.1003381","volume":"17","author":"SS Han","year":"2020","unstructured":"Han, S. S. et al. Assessment of deep neural networks for the diagnosis of benign and malignant skin neoplasms in comparison with dermatologists: A retrospective validation study. PLoS Med. 17, e1003381 (2020).","journal-title":"PLoS Med"},{"key":"1103_CR53","doi-asserted-by":"publisher","first-page":"1753","DOI":"10.1016\/j.jid.2020.01.019","volume":"140","author":"SS Han","year":"2020","unstructured":"Han, S. S. et al. Augmented intelligence dermatology: Deep neural networks empower medical professionals in diagnosing skin cancer and predicting treatment options for 134 skin disorders. J. Investig. Dermatol. 140, 1753\u20131761 (2020).","journal-title":"J. Investig. Dermatol."},{"key":"1103_CR54","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3390\/biom10081123","volume":"10","author":"S Jinnai","year":"2020","unstructured":"Jinnai, S. et al. The development of a skin cancer classification system for pigmented skin lesions using deep learning. Biomolecules 10, 1\u201313 (2020).","journal-title":"Biomolecules"},{"key":"1103_CR55","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1038\/nature21056","volume":"542","author":"A Esteva","year":"2017","unstructured":"Esteva, A. et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115\u2013118 (2017).","journal-title":"Nature"},{"key":"1103_CR56","doi-asserted-by":"publisher","first-page":"58","DOI":"10.1001\/jamadermatol.2018.4378","volume":"155","author":"P Tschandl","year":"2019","unstructured":"Tschandl, P. et al. Expert-level diagnosis of nonpigmented skin cancer by combined convolutional neural networks. JAMA Dermatol 155, 58\u201365 (2019).","journal-title":"JAMA Dermatol"},{"key":"1103_CR57","doi-asserted-by":"publisher","first-page":"1836","DOI":"10.1093\/annonc\/mdy166","volume":"29","author":"HA Haenssle","year":"2018","unstructured":"Haenssle, H. A. et al. Man against Machine: Diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists. Ann. Oncol. 29, 1836\u20131842 (2018).","journal-title":"Ann. Oncol."},{"key":"1103_CR58","doi-asserted-by":"publisher","first-page":"148","DOI":"10.1016\/j.ejca.2019.02.005","volume":"111","author":"TJ Brinker","year":"2019","unstructured":"Brinker, T. J. et al. A convolutional neural network trained with dermoscopic images performed on par with 145 dermatologists in a clinical melanoma image classification task. Eur. J. Cancer 111, 148\u2013154 (2019).","journal-title":"Eur. J. Cancer"},{"key":"1103_CR59","doi-asserted-by":"publisher","first-page":"2020","DOI":"10.1097\/CM9.0000000000001002","volume":"133","author":"CX Li","year":"2020","unstructured":"Li, C. X. et al. Diagnostic capacity of skin tumor artificial intelligence-assisted decision-making software in real-world clinical settings. Chin. Med. J. (Engl.) 133, 2020\u20132026 (2020).","journal-title":"Chin. Med. J. (Engl.)"},{"key":"1103_CR60","doi-asserted-by":"publisher","first-page":"504","DOI":"10.1097\/CMR.0000000000000779","volume":"31","author":"ML Willingham","year":"2021","unstructured":"Willingham, M. L. et al. The potential of using artificial intelligence to improve skin cancer diagnoses in Hawai\u2019i\u2019s multiethnic population. Melanoma Res. 31, 504\u2013514 (2021).","journal-title":"Melanoma Res."},{"key":"1103_CR61","doi-asserted-by":"publisher","first-page":"e26025","DOI":"10.2196\/26025","volume":"23","author":"K Huang","year":"2021","unstructured":"Huang, K. et al. The classification of six common skin diseases based on xiangya-derm: Development of a chinese database for artificial intelligence. J. Med. Internet Res 23, e26025 (2021).","journal-title":"J. Med. Internet Res"},{"key":"1103_CR62","doi-asserted-by":"publisher","first-page":"2353","DOI":"10.1016\/j.jid.2022.02.003","volume":"142","author":"SS Han","year":"2022","unstructured":"Han, S. S. et al. Evaluation of artificial intelligence-assisted diagnosis of skin neoplasms: a single-center, paralleled, unmasked, randomized controlled trial. J. Invest Dermatol 142, 2353\u20132362.e2 (2022).","journal-title":"J. Invest Dermatol"},{"key":"1103_CR63","doi-asserted-by":"publisher","first-page":"546","DOI":"10.1111\/jdv.16979","volume":"35","author":"C Mu\u00f1oz\u2010L\u00f3pez","year":"2021","unstructured":"Mu\u00f1oz\u2010L\u00f3pez, C. et al. Performance of a deep neural network in teledermatology: A single\u2010centre prospective diagnostic study. J. Eur. Acad. Dermatol. Venereol. 35, 546\u2013553 (2021).","journal-title":"J. Eur. Acad. Dermatol. Venereol."},{"key":"1103_CR64","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-022-20632-7","volume":"12","author":"SS Han","year":"2022","unstructured":"Han, S. S. et al. The degradation of performance of a state-of-the-art skin image classifier when applied to patient-driven internet search. Sci. Rep. 12, 16260 (2022).","journal-title":"Sci. Rep."},{"key":"1103_CR65","doi-asserted-by":"publisher","first-page":"e271","DOI":"10.1016\/S2589-7500(19)30123-2","volume":"1","author":"X Liu","year":"2019","unstructured":"Liu, X. et al. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. Lancet Digit Health 1, e271\u2013e297 (2019).","journal-title":"Lancet Digit Health"},{"key":"1103_CR66","doi-asserted-by":"publisher","first-page":"202","DOI":"10.1016\/j.ejca.2021.06.049","volume":"156","author":"S Haggenmuller","year":"2021","unstructured":"Haggenmuller, S. et al. Skin cancer classification via convolutional neural networks: Systematic review of studies involving human experts. Eur. J. Cancer 156, 202\u2013216 (2021).","journal-title":"Eur. J. Cancer"},{"key":"1103_CR67","doi-asserted-by":"publisher","first-page":"e22934","DOI":"10.2196\/22934","volume":"23","author":"A Takiddin","year":"2021","unstructured":"Takiddin, A., Schneider, J., Yang, Y., Abd-Alrazaq, A. & Househ, M. Artificial intelligence for skin cancer detection: Scoping review. J. Med. Internet Res. 23, e22934 (2021).","journal-title":"J. Med. Internet Res."},{"key":"1103_CR68","doi-asserted-by":"publisher","first-page":"e11936","DOI":"10.2196\/11936","volume":"20","author":"TJ Brinker","year":"2018","unstructured":"Brinker, T. J. et al. Skin cancer classification using convolutional neural networks: Systematic review. J. Med. Internet Res. 20, e11936 (2018).","journal-title":"J. Med. Internet Res."},{"key":"1103_CR69","doi-asserted-by":"publisher","DOI":"10.1186\/s12880-019-0307-7","volume":"19","author":"A Marka","year":"2019","unstructured":"Marka, A., Carter, J. B., Toto, E. & Hassanpour, S. Automated detection of nonmelanoma skin cancer using digital images: A systematic review. BMC Med. Imaging 19, 21 (2019).","journal-title":"BMC Med. Imaging"},{"key":"1103_CR70","doi-asserted-by":"publisher","unstructured":"Wu, K. et al. Characterizing the clinical adoption of medical AI devices through U.S. insurance claims. NEJM AI https:\/\/doi.org\/10.1056\/AIoa2300030 (2023).","DOI":"10.1056\/AIoa2300030"},{"key":"1103_CR71","doi-asserted-by":"publisher","first-page":"2277","DOI":"10.1016\/j.jid.2018.04.040","volume":"138","author":"C Navarrete-Dechent","year":"2018","unstructured":"Navarrete-Dechent, C. et al. Automated dermatological diagnosis: Hype or reality? J. Investig. Dermatol. 138, 2277\u20132279 (2018).","journal-title":"J. Investig. Dermatol."},{"key":"1103_CR72","doi-asserted-by":"publisher","first-page":"1325","DOI":"10.1016\/j.jid.2020.06.040","volume":"141","author":"C Navarrete-Dechent","year":"2021","unstructured":"Navarrete-Dechent, C., Liopyris, K. & Marchetti, M. A. Multiclass artificial intelligence in dermatology: Progress but still room for improvement. J. Investig. Dermatol. 141, 1325\u20131328 (2021).","journal-title":"J. Investig. Dermatol."},{"key":"1103_CR73","doi-asserted-by":"publisher","first-page":"1247","DOI":"10.1001\/jamadermatol.2018.2348","volume":"154","author":"AS Adamson","year":"2018","unstructured":"Adamson, A. S. & Smith, A. Machine learning and health care disparities in dermatology. JAMA Dermatol 154, 1247\u20131248 (2018).","journal-title":"JAMA Dermatol"},{"key":"1103_CR74","doi-asserted-by":"publisher","first-page":"1362","DOI":"10.1001\/jamadermatol.2021.3129","volume":"157","author":"R Daneshjou","year":"2021","unstructured":"Daneshjou, R., Smith, M. P., Sun, M. D., Rotemberg, V. & Zou, J. Lack of transparency and potential bias in artificial intelligence data sets and algorithms: A scoping review. JAMA Dermatol 157, 1362\u20131369 (2021).","journal-title":"JAMA Dermatol"},{"key":"1103_CR75","doi-asserted-by":"publisher","first-page":"e1","DOI":"10.1111\/ajd.13690","volume":"63","author":"D Jobson","year":"2022","unstructured":"Jobson, D., Mar, V. & Freckelton, I. Legal and ethical considerations of artificial intelligence in skin cancer diagnosis. Australas. J. Dermatol 63, e1\u2013e5 (2022).","journal-title":"Australas. J. Dermatol"},{"key":"1103_CR76","doi-asserted-by":"publisher","first-page":"90","DOI":"10.1001\/jamadermatol.2021.4915","volume":"158","author":"R Daneshjou","year":"2022","unstructured":"Daneshjou, R. et al. Checklist for evaluation of image-based artificial intelligence reports in dermatology: CLEAR derm consensus guidelines from the international skin imaging collaboration artificial intelligence working group. JAMA Dermatol 158, 90\u201396 (2022).","journal-title":"JAMA Dermatol"},{"key":"1103_CR77","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1111\/j.1600-0846.2009.00385.x","volume":"16","author":"A Tenenhaus","year":"2010","unstructured":"Tenenhaus, A. et al. Detection of melanoma from dermoscopic images of naevi acquired under uncontrolled conditions. Skin Res. Technol. 16, 85\u201397 (2010).","journal-title":"Skin Res. Technol."},{"key":"1103_CR78","doi-asserted-by":"publisher","first-page":"47","DOI":"10.1016\/j.ejca.2019.04.001","volume":"113","author":"T Brinker","year":"2019","unstructured":"Brinker, T. et al. Deep learning outperformed 136 of 157 dermatologists in a head-to-head dermoscopic melanoma image classification task. Eur. J. Cancer 113, 47\u201354 (2019).","journal-title":"Eur. J. Cancer"}],"updated-by":[{"DOI":"10.1038\/s41746-024-01138-0","type":"correction","label":"Correction","source":"publisher","updated":{"date-parts":[[2024,5,24]],"date-time":"2024-05-24T00:00:00Z","timestamp":1716508800000}}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-024-01103-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-024-01103-x","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-024-01103-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,5,24]],"date-time":"2024-05-24T20:25:03Z","timestamp":1716582303000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-024-01103-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,14]]},"references-count":78,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["1103"],"URL":"https:\/\/doi.org\/10.1038\/s41746-024-01103-x","relation":{"correction":[{"id-type":"doi","id":"10.1038\/s41746-024-01138-0","asserted-by":"object"}]},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,14]]},"assertion":[{"value":"22 September 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 April 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 May 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 May 2024","order":4,"name":"change_date","label":"Change Date","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Correction","order":5,"name":"change_type","label":"Change Type","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"A Correction to this paper has been published:","order":6,"name":"change_details","label":"Change Details","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"https:\/\/doi.org\/10.1038\/s41746-024-01138-0","URL":"https:\/\/doi.org\/10.1038\/s41746-024-01138-0","order":7,"name":"change_details","label":"Change Details","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"125"}}