{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,2]],"date-time":"2026-10-02T03:48:09Z","timestamp":1790912889437,"version":"4.1.0"},"reference-count":35,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,3,23]],"date-time":"2023-03-23T00:00:00Z","timestamp":1679529600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,3,23]],"date-time":"2023-03-23T00:00:00Z","timestamp":1679529600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000265","name":"Medical Research Council","doi-asserted-by":"publisher","award":["MR\/S001530\/1"],"award-info":[{"award-number":["MR\/S001530\/1"]}],"id":[{"id":"10.13039\/501100000265","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000272","name":"National Institute for Health and Care Research","doi-asserted-by":"publisher","award":["DRF-2018-11-ST2-028"],"award-info":[{"award-number":["DRF-2018-11-ST2-028"]}],"id":[{"id":"10.13039\/501100000272","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100014013","name":"UK Research and Innovation","doi-asserted-by":"publisher","award":["104687"],"award-info":[{"award-number":["104687"]}],"id":[{"id":"10.13039\/100014013","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The current subjective histopathological assessment of cutaneous melanoma is challenging. The application of image analysis algorithms to histological images may facilitate improvements in workflow and prognostication. To date, several individual algorithms applied to melanoma histological images have been reported with variations in approach and reported accuracies. Histological digital images can be created using a camera mounted on a light microscope, or through whole slide image (WSI) generation using a whole slide scanner. Before any such tool could be integrated into clinical workflow, the accuracy of the technology should be carefully evaluated and summarised. Therefore, the objective of this review was to evaluate the accuracy of existing image analysis algorithms applied to digital\u00a0histological images of cutaneous melanoma.\nDatabase searching of PubMed and Embase from inception to 11th March 2022 was conducted alongside citation checking and examining reports from organisations. All studies reporting accuracy of any image analysis applied to histological images of cutaneous melanoma, were included. The reference standard was any histological assessment of haematoxylin and eosin-stained slides and\/or immunohistochemical staining. Citations were independently deduplicated and screened by two review authors and disagreements were resolved through discussion. The data was extracted concerning study demographics; type of image analysis; type of reference standard; conditions included and test statistics to construct 2\u2009\u00d7\u20092 tables. Data was extracted in accordance with our protocol and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses-Diagnostic Test Accuracy (PRISMA-DTA) Statement. A bivariate random-effects meta-analysis was used to estimate summary sensitivities and specificities with 95% confidence intervals (CI). Assessment of methodological quality was conducted using a tailored version of the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool. The primary outcome was the pooled sensitivity and specificity of image analysis applied to cutaneous melanoma histological images. Sixteen studies were included in the systematic review, representing 4,888 specimens. Six studies were included in the meta-analysis. The mean sensitivity and specificity of automated image analysis algorithms applied to melanoma histological images was 90% (CI 82%, 95%) and 92% (CI 79%, 97%), respectively. Based on limited and heterogeneous data, image analysis appears to offer high accuracy when applied to histological images of cutaneous melanoma. However, given the early exploratory nature of these studies, further development work is necessary to improve their performance.<\/jats:p>","DOI":"10.1038\/s41598-023-31526-7","type":"journal-article","created":{"date-parts":[[2023,3,23]],"date-time":"2023-03-23T12:04:15Z","timestamp":1679573055000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":19,"title":["Image analysis of cutaneous melanoma histology: a systematic review and meta-analysis"],"prefix":"10.1038","volume":"13","author":[{"given":"Emily L.","family":"Clarke","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ryckie G.","family":"Wade","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Derek","family":"Magee","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Julia","family":"Newton-Bishop","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Darren","family":"Treanor","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,3,23]]},"reference":[{"key":"31526_CR1","unstructured":"Cancer Research UK. Melanoma Skin Cancer Survival. https:\/\/www.cancerresearchuk.org\/about-cancer\/melanoma\/survival"},{"key":"31526_CR2","unstructured":"Cancer Research UK. Melanoma Skin Cancer Statistics. https:\/\/www.cancerresearchuk.org\/health-professional\/cancer-statistics\/statistics-by-cancer-type\/melanoma-skin-cancer#heading-Zero"},{"key":"31526_CR3","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.j2813","author":"JG Elmore","year":"2017","unstructured":"Elmore, J. G. et al. Pathologists\u2019 diagnosis of invasive melanoma and melanocytic proliferations: Observer accuracy and reproducibility study. BMJ Online https:\/\/doi.org\/10.1136\/bmj.j2813 (2017).","journal-title":"BMJ Online"},{"key":"31526_CR4","doi-asserted-by":"publisher","unstructured":"Keung EZ, Gershenwald JE. (2018) The eighth edition American joint committee on cnacer (AJCC) melanoma staging system: Implications for melanoma treatment and care.Exp. Rev. Anticancer Ther. 18(8):775\u2013784. https:\/\/doi.org\/10.4049\/jimmunol.1801473.","DOI":"10.4049\/jimmunol.1801473"},{"issue":"5","key":"31526_CR5","doi-asserted-by":"publisher","first-page":"902","DOI":"10.1097\/00000658-197011000-00017","volume":"172","author":"A Breslow","year":"1970","unstructured":"Breslow, A. Thickness, cross-sectional areas and depth of invasion in the prognosis of cutaneous melanoma. Ann. Surg. 172(5), 902\u2013908. https:\/\/doi.org\/10.1097\/00000658-197011000-00017 (1970).","journal-title":"Ann. Surg."},{"issue":"13","key":"31526_CR6","doi-asserted-by":"publisher","first-page":"11683","DOI":"10.18632\/oncotarget.3549","volume":"6","author":"J Nsengimana","year":"2015","unstructured":"Nsengimana, J. et al. Independent replication of a melanoma subtype gene signature and evaluation of its prognostic value and biological correlates in a population cohort. Oncotarget 6(13), 11683 (2015).","journal-title":"Oncotarget"},{"key":"31526_CR7","unstructured":"The Royal College of Pathologists. Meeting pathology demand Histopathology workforce census. Published online 2018. https:\/\/www.rcpath.org\/uploads\/assets\/952a934d-2ec3-48c9-a8e6e00fcdca700f\/meeting-pathology-demand-histopathology-workforce-census-2018.pdf"},{"issue":"7639","key":"31526_CR8","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1038\/nature21056","volume":"542","author":"A Esteva","year":"2017","unstructured":"Esteva, A. et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639), 115\u2013118. https:\/\/doi.org\/10.1038\/nature21056 (2017).","journal-title":"Nature"},{"issue":"12","key":"31526_CR9","doi-asserted-by":"publisher","first-page":"1636","DOI":"10.1097\/PAS.0000000000001151","volume":"42","author":"DF Steiner","year":"2018","unstructured":"Steiner, D. F. et al. Impact of deep learning assistance on the histopathologic review of lymph nodes for metastatic breast cancer. Am. J. Surg. Pathol. 42(12), 1636\u20131646. https:\/\/doi.org\/10.1097\/PAS.0000000000001151 (2018).","journal-title":"Am. J. Surg. Pathol."},{"key":"31526_CR10","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW54120.2021.00076","author":"S Sankarapandian","year":"2021","unstructured":"Sankarapandian, S. et al. A pathology deep learning system capable of triage of melanoma specimens utilizing dermatopathologist consensus as ground truth*. Proc. IEEE Int. Conf. Comput. Vis. https:\/\/doi.org\/10.1109\/ICCVW54120.2021.00076 (2021).","journal-title":"Proc. IEEE Int. Conf. Comput. Vis."},{"issue":"149","key":"31526_CR11","doi-asserted-by":"publisher","first-page":"94","DOI":"10.1016\/j.ejca.2021.02.032","volume":"2021","author":"J Hohn","year":"1990","unstructured":"Hohn, J. et al. Combining CNN-based histologic whole slide image analysis and patient data to improve skin cancer classification. Eur. J. Cancer Oxf. Engl. 2021(149), 94\u2013101. https:\/\/doi.org\/10.1016\/j.ejca.2021.02.032 (1990).","journal-title":"Eur. J. Cancer Oxf. Engl."},{"key":"31526_CR12","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3389\/fonc.2020.01559","volume":"10","author":"F De Logu","year":"2020","unstructured":"De Logu, F. et al. Recognition of cutaneous melanoma on digitized histopathological slides via artificial intelligence algorithm. Front. Oncol. 10, 1\u20138. https:\/\/doi.org\/10.3389\/fonc.2020.01559 (2020).","journal-title":"Front. Oncol."},{"issue":"4","key":"31526_CR13","doi-asserted-by":"publisher","first-page":"388","DOI":"10.1001\/jama.2017.19163","volume":"319","author":"MDF McInnes","year":"2018","unstructured":"McInnes, M. D. F. et al. Preferred reporting items for a systematic review and meta-analysis of diagnostic test accuracy studies the PRISMA-DTA statement. JAMA J. Am. Med. Assoc. 319(4), 388\u2013396. https:\/\/doi.org\/10.1001\/jama.2017.19163 (2018).","journal-title":"JAMA J. Am. Med. Assoc."},{"issue":"6","key":"31526_CR14","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1136\/bmjopen-2020-047709","volume":"11","author":"V Sounderajah","year":"2021","unstructured":"Sounderajah, V. et al. Developing a reporting guideline for artificial intelligence-centred diagnostic test accuracy studies: The STARD-AI protocol. BMJ Open 11(6), 1\u20137. https:\/\/doi.org\/10.1136\/bmjopen-2020-047709 (2021).","journal-title":"BMJ Open"},{"key":"31526_CR15","doi-asserted-by":"publisher","DOI":"10.1002\/jrsm.1411","author":"LA McGuinness","year":"2020","unstructured":"McGuinness, L. A. & Higgins, J. P. T. Risk-of-bias visualization (robvis): An R package and Shiny web app for visualizing risk-of-bias assessments. Res. Synth. Methods https:\/\/doi.org\/10.1002\/jrsm.1411 (2020).","journal-title":"Res. Synth. Methods"},{"issue":"1","key":"31526_CR16","doi-asserted-by":"publisher","first-page":"34","DOI":"10.1002\/jrsm.1439","volume":"12","author":"A Patel","year":"2021","unstructured":"Patel, A., Cooper, N., Freeman, S. & Sutton, A. Graphical enhancements to summary receiver operating characteristic plots to facilitate the analysis and reporting of meta-analysis of diagnostic test accuracy data. Res. Synth. Methods 12(1), 34\u201344. https:\/\/doi.org\/10.1002\/jrsm.1439 (2021).","journal-title":"Res. Synth. Methods"},{"issue":"1","key":"31526_CR17","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12874-019-0724-x","volume":"19","author":"SC Freeman","year":"2019","unstructured":"Freeman, S. C. et al. Development of an interactive web-based tool to conduct and interrogate meta-analysis of diagnostic test accuracy studies: MetaDTA. BMC Med. Res. Methodol 19(1), 1\u201311. https:\/\/doi.org\/10.1186\/s12874-019-0724-x (2019).","journal-title":"BMC Med. Res. Methodol"},{"issue":"2","key":"31526_CR18","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1002\/cl2.1230","volume":"18","author":"NR Haddaway","year":"2022","unstructured":"Haddaway, N. R., Page, M. J., Pritchard, C. C. & McGuinness, L. A. PRISMA2020: An R package and Shiny app for producing PRISMA 2020-compliant flow diagrams, with interactivity for optimised digital transparency and open synthesis. Campbell Syst. Rev. 18(2), 1\u201312. https:\/\/doi.org\/10.1002\/cl2.1230 (2022).","journal-title":"Campbell Syst. Rev."},{"key":"31526_CR19","unstructured":"Macaskill P, Gatsonis C, Deeks J, Harbord R, Takwoingi Y. Chapter 10 Analysing and Presenting Results. Version 1. The Cochrane Collaboration; 2010:79. http:\/\/srdta.cochrane.org\/."},{"issue":"9","key":"31526_CR20","doi-asserted-by":"publisher","first-page":"882","DOI":"10.1016\/j.jclinepi.2005.01.016","volume":"58","author":"J Deeks","year":"2005","unstructured":"Deeks, J., Macaskill, P. & Irwig, L. The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. J. Clin. Epidemiol. 58(9), 882\u2013893 (2005).","journal-title":"J. Clin. Epidemiol."},{"key":"31526_CR21","doi-asserted-by":"publisher","first-page":"586292","DOI":"10.3389\/fonc.2020.586292","volume":"10","author":"K Zormpas-Petridis","year":"2020","unstructured":"Zormpas-Petridis, K., Noguera, R. D. K. I., Roxanis, I., Jamin, Y. & Yuan, Y. SuperHistopath: A deep learning pipeline for mapping tumor heterogeneity on low-resolution whole-slide digital histopathology images. Front. Oncol. 10, 586292\u2013586292. https:\/\/doi.org\/10.3389\/fonc.2020.586292 (2020).","journal-title":"Front. Oncol."},{"key":"31526_CR22","doi-asserted-by":"publisher","first-page":"91","DOI":"10.1016\/j.ejca.2019.06.012","volume":"118","author":"A Hekler","year":"2019","unstructured":"Hekler, A. et al. Deep learning outperformed 11 pathologists in the classification of histopathological melanoma images. Eur. J. Cancer 118, 91\u201396. https:\/\/doi.org\/10.1016\/j.ejca.2019.06.012 (2019).","journal-title":"Eur. J. Cancer"},{"key":"31526_CR23","doi-asserted-by":"publisher","first-page":"130A","DOI":"10.1038\/labinvest.2016.165","volume":"97","author":"J Henriet","year":"2017","unstructured":"Henriet, J., Monnin, C., Clerc, J., Morello, B. & Zehrouni, N. 508 diagnosis of spitz tumor using artificial neural networks. Lab. Invest. 97, 130A-130A. https:\/\/doi.org\/10.1038\/labinvest.2016.165 (2017).","journal-title":"Lab. Invest."},{"key":"31526_CR24","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0062070","author":"E Rexhepaj","year":"2013","unstructured":"Rexhepaj, E. et al. A texture based pattern recognition approach to distinguish melanoma from non-melanoma cells in histopathological tissue microarray sections. PLoS ONE https:\/\/doi.org\/10.1371\/journal.pone.0062070 (2013).","journal-title":"PLoS ONE"},{"key":"31526_CR25","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1016\/j.compmedimag.2019.01.006","volume":"73","author":"S Alheejaw","year":"2019","unstructured":"Alheejaw, S., Xu, H., Berendt, R., Jha, N. & Mandal, M. Novel lymph node segmentation and proliferation index measurement for skin melanoma biopsy images. Comput. Med. Imaging Graph Off. J. Comput. Med. Imaging Soc. 73, 19\u201329. https:\/\/doi.org\/10.1016\/j.compmedimag.2019.01.006 (2019).","journal-title":"Comput. Med. Imaging Graph Off. J. Comput. Med. Imaging Soc."},{"key":"31526_CR26","doi-asserted-by":"publisher","first-page":"101659","DOI":"10.1016\/j.tice.2021.101659","volume":"73","author":"S Alheejawi","year":"2021","unstructured":"Alheejawi, S., Berendt, R., Jha, N., SP, M. & Mandal, M. Detection of malignant melanoma in H&E-stained images using deep learning techniques. Tissue Cell. 73, 101659. https:\/\/doi.org\/10.1016\/j.tice.2021.101659 (2021).","journal-title":"Tissue Cell."},{"issue":"8","key":"31526_CR27","doi-asserted-by":"publisher","first-page":"2738","DOI":"10.1016\/j.patcog.2015.02.023","volume":"48","author":"C Lu","year":"2015","unstructured":"Lu, C. & Mandal, M. Automated analysis and diagnosis of skin melanoma on whole slide histopathological images. Pattern Recognit. 48(8), 2738\u20132750. https:\/\/doi.org\/10.1016\/j.patcog.2015.02.023 (2015).","journal-title":"Pattern Recognit."},{"key":"31526_CR28","first-page":"1","volume":"34","author":"C Lu","year":"2012","unstructured":"Lu, C., Mahmood, M., Jha, N. & Mandal, M. A robust automatic nuclei segmentation technique for quantitative histopathological image analysis. Anal. Quant. Cytopathol. Histopathol. 34, 1\u201313 (2012).","journal-title":"Anal. Quant. Cytopathol. Histopathol."},{"key":"31526_CR29","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1016\/j.jid.2018.03.303","volume":"138","author":"Y Ota","year":"2018","unstructured":"Ota, Y. et al. Deep ackerman a novel deep learning method to develop dermatopathology diagnosis by artificial intelligence. J. Invest. Dermatol. 138, 5 (2018).","journal-title":"J. Invest. Dermatol."},{"key":"31526_CR30","doi-asserted-by":"publisher","DOI":"10.1155\/2021\/5972962","author":"T Li","year":"2021","unstructured":"Li, T. et al. Automated diagnosis and localization of melanoma from skin histopathology slides using deep learning: A multicenter study. J. Healthc. Eng. https:\/\/doi.org\/10.1155\/2021\/5972962 (2021).","journal-title":"J. Healthc. Eng."},{"key":"31526_CR31","doi-asserted-by":"publisher","DOI":"10.1016\/j.jid.2019.03.904","author":"F Li","year":"2019","unstructured":"Li, F. et al. 828 Dermatopathologist-level classification of skin cancer with deep neural networks at multi-magnification. J. Invest. Dermatol. https:\/\/doi.org\/10.1016\/j.jid.2019.03.904 (2019).","journal-title":"J. Invest. Dermatol."},{"key":"31526_CR32","doi-asserted-by":"publisher","DOI":"10.1136\/bjophthalmol-2018-313706","author":"L Wang","year":"2019","unstructured":"Wang, L. et al. Automated identification of malignancy in whole-slide pathological images: Identification of eyelid malignant melanoma in gigapixel pathological slides using deep learning. Br J. Ophthalmol. https:\/\/doi.org\/10.1136\/bjophthalmol-2018-313706 (2019).","journal-title":"Br J. Ophthalmol."},{"key":"31526_CR33","doi-asserted-by":"publisher","DOI":"10.1016\/j.jid.2020.05.047","author":"P Xie","year":"2020","unstructured":"Xie, P. et al. Predicting metaststic melanoma from melanoma pathologica images using a convolutional neural network: A multicentre study. J. Invest. Dermatol. https:\/\/doi.org\/10.1016\/j.jid.2020.05.047 (2020).","journal-title":"J. Invest. Dermatol."},{"key":"31526_CR34","unstructured":"Reinke A, Eisenmann M, Tizabi MD, et al. (2021)Common limitations of image processing metrics: A picture story. Pub. Online 1\u201311."},{"key":"31526_CR35","unstructured":"The Cochrane Collaboration. The Cochrane Handbook for DTA Reviews.; (2016)."}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-023-31526-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-023-31526-7","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-023-31526-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,3,23]],"date-time":"2023-03-23T12:09:34Z","timestamp":1679573374000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-023-31526-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,23]]},"references-count":35,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["31526"],"URL":"https:\/\/doi.org\/10.1038\/s41598-023-31526-7","relation":{},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,23]]},"assertion":[{"value":"19 October 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 March 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 March 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"We declare the following interests: Author DT is Director of National Pathology Imaging Co-operative (NPIC). Author DM is a director of HeteroGenius Limited. Author RW is a Doctoral Research Fellow funded by the National Institute for Health Research. Author EC is a Clinical Research Training Fellow funded by the Medical Research Council. Author JNB has no conflicts of interest to declare.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"4774"}}