{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,1]],"date-time":"2026-06-01T23:32:21Z","timestamp":1780356741245,"version":"3.54.1"},"reference-count":41,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2022,10,27]],"date-time":"2022-10-27T00:00:00Z","timestamp":1666828800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,10,27]],"date-time":"2022-10-27T00:00:00Z","timestamp":1666828800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100002341","name":"Academy of Finland","doi-asserted-by":"crossref","award":["Projects TensorML#311277"],"award-info":[{"award-number":["Projects TensorML#311277"]}],"id":[{"id":"10.13039\/501100002341","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Neural Process Lett"],"published-print":{"date-parts":[[2023,8]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Early diagnosis plays a key role in prevention and treatment of skin cancer. Several machine learning techniques for accurate detection of skin cancer from medical images have been reported. Many of these techniques are based on pre-trained convolutional neural networks (CNNs), which enable training the models based on limited amounts of training data. However, the classification accuracy of these models still tends to be severely limited by the scarcity of representative images from malignant tumours. We propose a novel ensemble-based convolutional neural network (CNN) architecture where multiple CNN models, some of which are pre-trained and some are trained only on the data at hand, along with auxiliary data in the form of metadata associated with the input images, are combined using a meta-learner. The proposed approach improves the model\u2019s ability to handle limited and imbalanced data. We demonstrate the benefits of the proposed technique using a dataset with 33,126 dermoscopic images from 2056 patients. We evaluate the performance of the proposed technique in terms of the F1-measure, area under the ROC curve (AUC-ROC), and area under the PR-curve (AUC-PR), and compare it with that of seven different benchmark methods, including two recent CNN-based techniques. The proposed technique compares favourably in terms of all the evaluation metrics.<\/jats:p>","DOI":"10.1007\/s11063-022-11049-4","type":"journal-article","created":{"date-parts":[[2022,10,27]],"date-time":"2022-10-27T15:23:35Z","timestamp":1666884215000},"page":"4461-4479","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":72,"title":["Transfer Learning with Ensembles of Deep Neural Networks for Skin Cancer Detection in Imbalanced Data Sets"],"prefix":"10.1007","volume":"55","author":[{"given":"Aqsa Saeed","family":"Qureshi","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Teemu","family":"Roos","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,10,27]]},"reference":[{"issue":"3","key":"11049_CR1","doi-asserted-by":"publisher","first-page":"583","DOI":"10.1016\/S0733-8635(18)30064-0","volume":"13","author":"BK Armstrong","year":"1995","unstructured":"Armstrong BK, Kricker A (1995) Skin cancer. Dermatol Clin 13(3):583\u2013594. https:\/\/doi.org\/10.1016\/S0733-8635(18)30064-0","journal-title":"Dermatol Clin"},{"issue":"1","key":"11049_CR2","doi-asserted-by":"publisher","first-page":"8","DOI":"10.1016\/j.canlet.2014.11.001","volume":"357","author":"MF Sim\u00f5es","year":"2015","unstructured":"Sim\u00f5es MF, Sousa JS, Pais AC (2015) Skin cancer and new treatment perspectives: a review. Cancer Lett 357(1):8\u201342. https:\/\/doi.org\/10.1016\/j.canlet.2014.11.001","journal-title":"Cancer Lett"},{"issue":"6","key":"11049_CR3","doi-asserted-by":"publisher","first-page":"394","DOI":"10.3322\/caac.21609","volume":"68","author":"F Bray","year":"2018","unstructured":"Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A (2018) Global cancer statistics 2018: Globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA A Cancer J Clin 68(6):394\u2013424. https:\/\/doi.org\/10.3322\/caac.21609","journal-title":"CA A Cancer J Clin"},{"issue":"2","key":"11049_CR4","doi-asserted-by":"publisher","first-page":"283","DOI":"10.1046\/j.1365-2133.1998.02075.x","volume":"138","author":"C Morton","year":"1998","unstructured":"Morton C, Mackie R (1998) Clinical accuracy of the diagnosis of cutaneous malignant melanoma. Br J Dermatol 138(2):283\u2013287. https:\/\/doi.org\/10.1046\/j.1365-2133.1998.02075.x","journal-title":"Br J Dermatol"},{"key":"11049_CR5","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1016\/j.det.2011.08.002","volume":"30","author":"KJ Buster","year":"2012","unstructured":"Buster KJ, Stevens EI, Elmets CA (2012) Dermatologic health disparities. Dermatol Clin 30:53. https:\/\/doi.org\/10.1016\/j.det.2011.08.002","journal-title":"Dermatol Clin"},{"issue":"8","key":"11049_CR6","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10916-019-1400-8","volume":"43","author":"A Murugan","year":"2019","unstructured":"Murugan A, Nair SAH, Kumar KS (2019) Detection of skin cancer using SVM, random forest and kNN classifiers. J Med Syst 43(8):1\u20139. https:\/\/doi.org\/10.1007\/s10916-019-1400-8","journal-title":"J Med Syst"},{"key":"11049_CR7","doi-asserted-by":"publisher","unstructured":"Ballerini L, Fisher RB, Aldridge B, Rees J (2013) A color and texture based hierarchical K-NN approach to the classification of non-melanoma skin lesions. In: Color medical image analysis, pp 63\u201386. https:\/\/doi.org\/10.1007\/978-94-007-5389-1_4","DOI":"10.1007\/978-94-007-5389-1_4"},{"key":"11049_CR8","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2020.101915","volume":"68","author":"SM Thomas","year":"2021","unstructured":"Thomas SM, Lefevre JG, Baxter G, Hamilton NA (2021) Interpretable deep learning systems for multi-class segmentation and classification of non-melanoma skin cancer. Med Image Anal 68:101915. https:\/\/doi.org\/10.1016\/j.media.2020.101915","journal-title":"Med Image Anal"},{"key":"11049_CR9","doi-asserted-by":"publisher","unstructured":"Lau HT, Al-Jumaily A (2009) Automatically early detection of skin cancer: study based on neural network classification. In: 2009 international conference of soft computing and pattern recognition. IEEE, pp 375\u2013380. https:\/\/doi.org\/10.1109\/socpar.2009.80","DOI":"10.1109\/socpar.2009.80"},{"issue":"39","key":"11049_CR10","doi-asserted-by":"publisher","first-page":"28477","DOI":"10.47611\/harp.95","volume":"79","author":"SS Chaturvedi","year":"2020","unstructured":"Chaturvedi SS, Tembhurne JV, Diwan T (2020) A multi-class skin cancer classification using deep convolutional neural networks. Multimed Tools Appl 79(39):28477\u201328498. https:\/\/doi.org\/10.47611\/harp.95","journal-title":"Multimed Tools Appl"},{"issue":"8","key":"11049_CR11","doi-asserted-by":"publisher","first-page":"1123","DOI":"10.3390\/biom10081123","volume":"10","author":"S Jinnai","year":"2020","unstructured":"Jinnai S, Yamazaki N, Hirano Y, Sugawara Y, Ohe Y, Hamamoto R (2020) The development of a skin cancer classification system for pigmented skin lesions using deep learning. Biomolecules 10(8):1123. https:\/\/doi.org\/10.3390\/biom10081123","journal-title":"Biomolecules"},{"issue":"1","key":"11049_CR12","doi-asserted-by":"publisher","first-page":"339","DOI":"10.1002\/jemt.23908","volume":"85","author":"M Nawaz","year":"2022","unstructured":"Nawaz M, Mehmood Z, Nazir T, Naqvi RA, Rehman A, Iqbal M, Saba T (2022) Skin cancer detection from dermoscopic images using deep learning and fuzzy k-means clustering. Microsc Res Tech 85(1):339\u2013351. https:\/\/doi.org\/10.1002\/jemt.23908","journal-title":"Microsc Res Tech"},{"issue":"3","key":"11049_CR13","doi-asserted-by":"publisher","first-page":"365","DOI":"10.1007\/s13555-020-00372-0","volume":"10","author":"S Chan","year":"2020","unstructured":"Chan S, Reddy V, Myers B, Thibodeaux Q, Brownstone N, Liao W (2020) Machine learning in dermatology: current applications, opportunities, and limitations. Dermatol Ther 10(3):365\u2013386. https:\/\/doi.org\/10.1007\/s13555-020-00372-0","journal-title":"Dermatol Ther"},{"issue":"5","key":"11049_CR14","doi-asserted-by":"publisher","first-page":"1325","DOI":"10.1007\/s10278-020-00371-9","volume":"33","author":"KM Hosny","year":"2020","unstructured":"Hosny KM, Kassem MA, Fouad MM (2020) Classification of skin lesions into seven classes using transfer learning with AlexNet. J Digit Imaging 33(5):1325\u20131334. https:\/\/doi.org\/10.1007\/s10278-020-00371-9","journal-title":"J Digit Imaging"},{"key":"11049_CR15","volume-title":"Advances in Neural Information Processing Systems","author":"A Krizhevsky","year":"2012","unstructured":"Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ (eds) Advances in Neural Information Processing Systems. Morgan Kaufmann Publishers Inc., San Francisco"},{"issue":"8","key":"11049_CR16","doi-asserted-by":"publisher","first-page":"9909","DOI":"10.1007\/s11042-018-5714-1","volume":"77","author":"U-O Dorj","year":"2018","unstructured":"Dorj U-O, Lee K-K, Choi J-Y, Lee M (2018) The skin cancer classification using deep convolutional neural network. Multimed Tools Appl 77(8):9909\u20139924. https:\/\/doi.org\/10.1007\/s11042-018-5714-1","journal-title":"Multimed Tools Appl"},{"key":"11049_CR17","doi-asserted-by":"publisher","first-page":"67","DOI":"10.1016\/j.imu.2018.06.006","volume":"12","author":"S Guo","year":"2018","unstructured":"Guo S, Yang Z (2018) Multi-Channel-ResNet: an integration framework towards skin lesion analysis. Inform Med Unlocked 12:67\u201374. https:\/\/doi.org\/10.1016\/j.imu.2018.06.006","journal-title":"Inform Med Unlocked"},{"key":"11049_CR18","doi-asserted-by":"publisher","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770\u2013778. https:\/\/doi.org\/10.1109\/cvpr.2016.90","DOI":"10.1109\/cvpr.2016.90"},{"issue":"2","key":"11049_CR19","doi-asserted-by":"publisher","first-page":"556","DOI":"10.3390\/s18020556","volume":"18","author":"Y Li","year":"2018","unstructured":"Li Y, Shen L (2018) Skin lesion analysis towards melanoma detection using deep learning network. Sensors 18(2):556. https:\/\/doi.org\/10.3390\/s18020556","journal-title":"Sensors"},{"issue":"6","key":"11049_CR20","doi-asserted-by":"publisher","first-page":"891","DOI":"10.1111\/srt.12891","volume":"26","author":"G Hirano","year":"2020","unstructured":"Hirano G, Nemoto M, Kimura Y, Kiyohara Y, Koga H, Yamazaki N, Christensen G, Ingvar C, Nielsen K, Nakamura A et al (2020) Automatic diagnosis of melanoma using hyperspectral data and GoogLeNet. Skin Res Technol 26(6):891\u2013897. https:\/\/doi.org\/10.1111\/srt.12891","journal-title":"Skin Res Technol"},{"key":"11049_CR21","doi-asserted-by":"publisher","unstructured":"Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: 2016 IEEE conference on computer vision and pattern recognition, CVPR 2016, pp 2818\u20132826. https:\/\/doi.org\/10.1109\/CVPR.2016.308","DOI":"10.1109\/CVPR.2016.308"},{"key":"11049_CR22","doi-asserted-by":"publisher","first-page":"114822","DOI":"10.1109\/access.2020.3003890","volume":"8","author":"MA Kassem","year":"2020","unstructured":"Kassem MA, Hosny KM, Fouad MM (2020) Skin lesions classification into eight classes for ISIC 2019 using deep convolutional neural network and transfer learning. IEEE Access 8:114822\u2013114832. https:\/\/doi.org\/10.1109\/access.2020.3003890","journal-title":"IEEE Access"},{"issue":"7639","key":"11049_CR23","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1038\/nature21056","volume":"542","author":"A Esteva","year":"2017","unstructured":"Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Thrun S (2017) Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639):115\u2013118. https:\/\/doi.org\/10.1038\/nature21056","journal-title":"Nature"},{"key":"11049_CR24","doi-asserted-by":"publisher","unstructured":"Dietterich TG et al (2002) Ensemble learning. In: The handbook of brain theory and neural networks, vol 2, pp 110\u2013125. https:\/\/doi.org\/10.7551\/mitpress\/3413.003.0009","DOI":"10.7551\/mitpress\/3413.003.0009"},{"key":"11049_CR25","doi-asserted-by":"publisher","unstructured":"Polikar R (2012) Ensemble learning. In: Ensemble machine learning, pp 1\u201334. https:\/\/doi.org\/10.1007\/978-1-4419-9326-7_1","DOI":"10.1007\/978-1-4419-9326-7_1"},{"key":"11049_CR26","doi-asserted-by":"publisher","unstructured":"Ain QU, Al-Sahaf H, Xue B, Zhang M (2020) A genetic programming approach to feature construction for ensemble learning in skin cancer detection. In: Proceedings of the 2020 genetic and evolutionary computation conference, pp 1186\u20131194. https:\/\/doi.org\/10.1145\/3377930.3390228","DOI":"10.1145\/3377930.3390228"},{"key":"11049_CR27","doi-asserted-by":"publisher","unstructured":"Mahbod A, Schaefer G, Wang C, Ecker R, Ellinge I (2019) Skin lesion classification using hybrid deep neural networks. In: ICASSP 2019\u20132019 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 1229\u20131233. https:\/\/doi.org\/10.1109\/icassp.2019.8683352","DOI":"10.1109\/icassp.2019.8683352"},{"key":"11049_CR28","unstructured":"Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: Bengio Y, LeCun Y (eds) 3rd international conference on learning representations (ICLR)"},{"issue":"9","key":"11049_CR29","doi-asserted-by":"publisher","first-page":"1263","DOI":"10.1109\/TKDE.2008.239","volume":"21","author":"H He","year":"2009","unstructured":"He H, Garcia E (2009) Learning from imbalanced data. IEEE Trans Knowl Data Eng 21(9):1263\u20131284. https:\/\/doi.org\/10.1109\/TKDE.2008.239","journal-title":"IEEE Trans Knowl Data Eng"},{"key":"11049_CR30","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2020.105568","volume":"195","author":"Z Qin","year":"2020","unstructured":"Qin Z, Liu Z, Zhu P, Xue Y (2020) A GAN-based image synthesis method for skin lesion classification. Comput Methods Programs Biomed 195:105568. https:\/\/doi.org\/10.1016\/j.cmpb.2020.105568","journal-title":"Comput Methods Programs Biomed"},{"issue":"13","key":"11049_CR31","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6560\/ab86d3","volume":"65","author":"H Zunair","year":"2020","unstructured":"Zunair H, Hamza AB (2020) Melanoma detection using adversarial training and deep transfer learning. Phys Med Biol 65(13):135005. https:\/\/doi.org\/10.1088\/1361-6560\/ab86d3","journal-title":"Phys Med Biol"},{"key":"11049_CR32","doi-asserted-by":"publisher","unstructured":"Zhu J-Y, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE international conference on computer vision, pp 2223\u20132232. https:\/\/doi.org\/10.1109\/iccv.2017.244","DOI":"10.1109\/iccv.2017.244"},{"issue":"1","key":"11049_CR33","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41597-021-00815-z","volume":"8","author":"V Rotemberg","year":"2021","unstructured":"Rotemberg V, Kurtansky N, Betz-Stablein B, Caffery L, Chousakos E, Codella N, Combalia M, Dusza S, Guitera P, Gutman D et al (2021) A patient-centric dataset of images and metadata for identifying melanomas using clinical context. Sci Data 8(1):1\u20138. https:\/\/doi.org\/10.1038\/s41597-021-00815-z","journal-title":"Sci Data"},{"key":"11049_CR34","doi-asserted-by":"publisher","unstructured":"Torrey L, Shavlik J (2010) Transfer learning. In: Handbook of research on machine learning applications and trends algorithms, methods, and techniques, pp 242\u2013264. https:\/\/doi.org\/10.4018\/978-1-60566-766-9.ch011","DOI":"10.4018\/978-1-60566-766-9.ch011"},{"issue":"10","key":"11049_CR35","doi-asserted-by":"publisher","first-page":"1345","DOI":"10.1109\/tkde.2009.191","volume":"22","author":"SJ Pan","year":"2009","unstructured":"Pan SJ, Yang Q (2009) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345\u20131359. https:\/\/doi.org\/10.1109\/tkde.2009.191","journal-title":"IEEE Trans Knowl Data Eng"},{"key":"11049_CR36","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine Learning in Python. J Mach Learn Res 12:2825\u20132830","journal-title":"J Mach Learn Res"},{"key":"11049_CR37","doi-asserted-by":"publisher","unstructured":"Davis J, Goadrich M (2006) The relationship between precision\u2013recall and ROC curves. In: Proceedings of the 23rd international conference on machine learning, pp 233\u2013240. https:\/\/doi.org\/10.1145\/1143844.1143874","DOI":"10.1145\/1143844.1143874"},{"key":"11049_CR38","doi-asserted-by":"publisher","first-page":"0118432","DOI":"10.1371\/journal.pone.0118432","volume":"10","author":"T Saito","year":"2015","unstructured":"Saito T, Rehmsmeier M (2015) The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS ONE 10:0118432. https:\/\/doi.org\/10.1371\/journal.pone.0118432","journal-title":"PLoS ONE"},{"key":"11049_CR39","doi-asserted-by":"publisher","first-page":"189","DOI":"10.1080\/0952813X.2012.680252","volume":"25","author":"D Berrar","year":"2013","unstructured":"Berrar D, Lozano JA (2013) Significance tests or confidence intervals: which are preferable for the comparison of classifiers? J Exp Theor Artif Intell 25:189\u2013206. https:\/\/doi.org\/10.1080\/0952813X.2012.680252","journal-title":"J Exp Theor Artif Intell"},{"key":"11049_CR40","doi-asserted-by":"publisher","first-page":"1132","DOI":"10.1038\/s41592-021-01256-7","volume":"18","author":"BJ Heil","year":"2021","unstructured":"Heil BJ, Hoffman MM, Markowetz F, Lee S-I, Greene CS, Hicks SC (2021) Reproducibility standards for machine learning in the life sciences. Nat Methods 18:1132\u20131135. https:\/\/doi.org\/10.1038\/s41592-021-01256-7","journal-title":"Nat Methods"},{"issue":"1","key":"11049_CR41","doi-asserted-by":"publisher","first-page":"1465","DOI":"10.1002\/wics.1465","volume":"12","author":"TH Johansen","year":"2020","unstructured":"Johansen TH, M\u00f8llersen K, Ortega S, Fabelo H, Garcia A, Callico GM, Godtliebsen F (2020) Recent advances in hyperspectral imaging for melanoma detection. WIREs Comput Stat 12(1):1465. https:\/\/doi.org\/10.1002\/wics.1465","journal-title":"WIREs Comput Stat"}],"container-title":["Neural Processing Letters"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11063-022-11049-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11063-022-11049-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11063-022-11049-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,7,31]],"date-time":"2023-07-31T16:46:12Z","timestamp":1690821972000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11063-022-11049-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,27]]},"references-count":41,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,8]]}},"alternative-id":["11049"],"URL":"https:\/\/doi.org\/10.1007\/s11063-022-11049-4","relation":{},"ISSN":["1370-4621","1573-773X"],"issn-type":[{"value":"1370-4621","type":"print"},{"value":"1573-773X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,10,27]]},"assertion":[{"value":"26 September 2022","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 October 2022","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}