{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,29]],"date-time":"2026-07-29T05:04:14Z","timestamp":1785301454912,"version":"3.55.0"},"reference-count":47,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,7,29]],"date-time":"2026-07-29T00:00:00Z","timestamp":1785283200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Med."],"abstract":"<jats:sec>\n                    <jats:title>Introduction<\/jats:title>\n                    <jats:p>Accurate and early classification of skin lesions is essential for the early detection of disease and improved clinical outcomes. However, automated multiclass classification is still hindered by severe class imbalance, high inter-class visual similarity, and the comparatively under-explored use of complementary clinical metadata relative to image-only pipelines, despite growing interest in such metadata fusion. In line with the growing demand for artificial intelligence (AI) tools that integrate heterogeneous data in clinical scenarios, this study presents a multimodal machine learning-based computer-aided diagnosis (ML-CAD) framework that fuses dermoscopic images with patient metadata.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>The framework follows a five-phase pipeline. A structured multimodal balancing strategy combines class-wise Synthetic Minority Oversampling Technique for Nominal and Continuous features (SMOTENC) for metadata with controlled image augmentation. This is followed by cross-modal alignment to preserve clinical consistency. A fine-tuned Vision Transformer (ViT-B\/16) at reduced input resolution performs feature extraction and late fusion with encoded metadata, yielding an 800-dimensional multimodal representation. Individual and stacked ensemble classifiers are subsequently trained on the fused features, with Bayesian optimization based on Gaussian process surrogates used for hyperparameter tuning. Gradient-weighted Class Activation Mapping (Grad-CAM) provides a visual interpretation across all seven lesion categories.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Experiments on the HAM10000 dataset demonstrate that the proposed SVM+KNN stacking ensemble achieved an accuracy of 98.55% and a ROC-AUC of 99.88%, achieving competitive accuracy among recently reported methods under broadly similar balanced settings.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Discussion<\/jats:title>\n                    <jats:p>The findings underscore the necessity of an AI framework that integrates imaging and clinical data to provide interpretable clinical decision support. This framework represents a step toward providing clinicians with multimodal, interpretable decision support. We emphasize, however, that external and prospective clinical validation remain necessary before translation into routine dermatological practice.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fmed.2026.1871689","type":"journal-article","created":{"date-parts":[[2026,7,29]],"date-time":"2026-07-29T04:49:52Z","timestamp":1785300592000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Interpretable multimodal fusion for skin lesion classification using dermoscopic images and patient metadata"],"prefix":"10.3389","volume":"13","author":[{"given":"Emrullah","family":"Sonu\u00e7","sequence":"first","affiliation":[{"name":"Computational Optimisation and Learning Lab, School of Computer Science, University of Nottingham","place":["Nottingham, United Kingdom"]},{"name":"Department of Computer Engineering, Karabuk University","place":["Karab\u00fck, T\u00fcrkiye"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qusay","family":"Saihood","sequence":"additional","affiliation":[{"name":"Department of Computer Engineering, Karabuk University","place":["Karab\u00fck, T\u00fcrkiye"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yusuf","family":"Yarg\u0131 Baydilli","sequence":"additional","affiliation":[{"name":"Department of Computer Engineering, Hakkari University","place":["Hakkari, T\u00fcrkiye"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ender","family":"\u00d6zcan","sequence":"additional","affiliation":[{"name":"Computational Optimisation and Learning Lab, School of Computer Science, University of Nottingham","place":["Nottingham, United Kingdom"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,7,29]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"e2795","DOI":"10.7717\/peerj-cs.2795","article-title":"Enhancing skin lesion classification: a CNN approach with human baseline comparison","volume":"11","author":"Ajabani","year":"2025","journal-title":"PeerJ Comput Sci"},{"key":"B2","doi-asserted-by":"publisher","first-page":"14","DOI":"10.1038\/s41598-024-81961-3","article-title":"Decoding skin cancer classification: perspectives, insights, and advances through researchers' lens","volume":"12","author":"Ray","year":"2024","journal-title":"Sci Rep"},{"key":"B3","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1111\/jdv.20368","article-title":"Skin cancer in Europe today and challenges for tomorrow","volume":"2","author":"Brochez","year":"2025","journal-title":"J Eur Acad Dermatol Venereol"},{"key":"B4","doi-asserted-by":"publisher","first-page":"5479","DOI":"10.3390\/ijerph18105479","article-title":"Skin cancer detection: A review using deep learning techniques","volume":"18","author":"Dildar","year":"2021","journal-title":"Int J Environ Res Public Health"},{"key":"B5","doi-asserted-by":"publisher","first-page":"4519","DOI":"10.52711\/0974-360X.2022.00758","article-title":"Skin cancer detection using deep learning","volume":"15","author":"Rajarajeswari","year":"2022","journal-title":"Res J Pharm Technol"},{"key":"B6","doi-asserted-by":"publisher","first-page":"364","DOI":"10.1016\/j.neucom.2021.08.096","article-title":"Skin disease diagnosis with deep learning: a review","volume":"464","author":"Li","year":"2021","journal-title":"Neurocomputing"},{"key":"B7","doi-asserted-by":"publisher","first-page":"9335","DOI":"10.1007\/s11063-023-11204-5","article-title":"A novel vision transformer model for skin cancer classification","volume":"55","author":"Yang","year":"2023","journal-title":"Neural Proc Lett"},{"key":"B8","doi-asserted-by":"publisher","first-page":"3022192","DOI":"10.1155\/2024\/3022192","article-title":"Skin cancer segmentation and classification using vision transformer for automatic analysis in dermatoscopy-based noninvasive digital system","volume":"2024","author":"Himel","year":"2024","journal-title":"Int J Biomed Imag"},{"key":"B9","doi-asserted-by":"publisher","first-page":"97","DOI":"10.3390\/bioengineering9030097","article-title":"Machine learning and deep learning algorithms for skin cancer classification from dermoscopic images","volume":"9","author":"Bechelli","year":"2022","journal-title":"Bioengineering"},{"key":"B10","doi-asserted-by":"publisher","first-page":"111","DOI":"10.1007\/s12626-025-00181-x","article-title":"An advanced deep learning framework for skin cancer classification","volume":"19","author":"Khan","year":"2025","journal-title":"Rev Socionetw Strat"},{"key":"B11","doi-asserted-by":"publisher","first-page":"2179","DOI":"10.3390\/cancers15072179","article-title":"DSCC_Net: multi-classification deep learning models for diagnosing of skin cancer using dermoscopic images","volume":"15","author":"Tahir","year":"2023","journal-title":"Cancers"},{"key":"B12","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/GCITC60406.2023.10425889","article-title":"Skin cancer detection using the HAM10000 dataset: a comparative study of machine learning models","author":"Velaga","year":"2023","journal-title":"2023 global conference on information technologies and communications (GCITC)"},{"key":"B13","doi-asserted-by":"publisher","first-page":"1681542","DOI":"10.3389\/fmed.2025.1724427","article-title":"An explainable hybrid deep learning framework for precise skin lesion segmentation and multi-class classification","volume":"12","author":"Fiaz","year":"2025","journal-title":"Front Med"},{"key":"B14","doi-asserted-by":"publisher","first-page":"18134","DOI":"10.1038\/s41598-022-22644-9","article-title":"Skin lesion classification of dermoscopic images using machine learning and convolutional neural network","volume":"12","author":"Shetty","year":"2022","journal-title":"Sci Rep"},{"key":"B15","doi-asserted-by":"publisher","first-page":"100036","DOI":"10.1016\/j.mlwa.2021.100036","article-title":"An enhanced technique of skin cancer classification using deep convolutional neural network with transfer learning models","volume":"5","author":"Ali","year":"2021","journal-title":"Mach Learn Applic"},{"key":"B16","doi-asserted-by":"publisher","first-page":"108929","DOI":"10.1016\/j.bspc.2025.108929","article-title":"Cutting-edge CNN-based skin cancer detection with batch normalization and advanced imbalance learning for superior medical image classification","volume":"113","author":"Govindu","year":"2026","journal-title":"Biomed Signal Process Control"},{"key":"B17","doi-asserted-by":"publisher","first-page":"2481","DOI":"10.3390\/healthcare10122481","article-title":"Melanoma detection using deep learning-based classifications","volume":"10","author":"Alwakid","year":"2022","journal-title":"Healthcare"},{"key":"B18","doi-asserted-by":"publisher","first-page":"3","DOI":"10.61356\/SMIJ.2025.10507","article-title":"A robust skin cancer classification using deep learning","volume":"10","author":"Abbas","year":"2025","journal-title":"Sustain Mach Intell J"},{"key":"B19","doi-asserted-by":"publisher","first-page":"32853","DOI":"10.1021\/acsomega.4c03640","article-title":"Machine learning approaches for skin neoplasm diagnosis","volume":"9","author":"Asaduzzaman","year":"2024","journal-title":"ACS Omega"},{"key":"B20","doi-asserted-by":"publisher","first-page":"797","DOI":"10.1002\/jemt.24736","article-title":"Early-stage melanoma cancer diagnosis framework for imbalanced data from dermoscopic images","volume":"88","author":"Khan","year":"2025","journal-title":"Microsc Res Tech"},{"key":"B21","doi-asserted-by":"publisher","first-page":"124","DOI":"10.1007\/s42979-024-03581-y","article-title":"Enhanced skin cancer diagnosis via deep convolutional neural networks with ensemble learning","volume":"6","author":"Khan","year":"2025","journal-title":"SN Comput Sci"},{"key":"B22","doi-asserted-by":"publisher","first-page":"1290","DOI":"10.1038\/s41598-024-84864-5","article-title":"Boosting skin cancer diagnosis accuracy with ensemble approach","volume":"15","author":"Natha","year":"2025","journal-title":"Sci Rep"},{"key":"B23","doi-asserted-by":"publisher","first-page":"4008","DOI":"10.3390\/s22114008","article-title":"An effective skin cancer classification mechanism via medical vision transformer","volume":"22","author":"Aladhadh","year":"2022","journal-title":"Sensors"},{"key":"B24","doi-asserted-by":"publisher","first-page":"105939","DOI":"10.1016\/j.compbiomed.2022.105939","article-title":"An improved transformer network for skin cancer classification","volume":"149","author":"Xin","year":"2022","journal-title":"Comput Biol Med"},{"key":"B25","doi-asserted-by":"publisher","first-page":"41405","DOI":"10.1007\/s11042-025-20797-z","article-title":"LesionAid: vision transformers-based skin lesion generation and classification-a practical review","volume":"84","author":"Rao","year":"2025","journal-title":"Multimed Tools Appl"},{"key":"B26","doi-asserted-by":"publisher","first-page":"102307","DOI":"10.1016\/j.media.2021.102307","article-title":"FusionM4Net: a multi-stage multi-modal learning algorithm for multi-label skin lesion classification","volume":"76","author":"Tang","year":"2022","journal-title":"Med Image Anal"},{"key":"B27","doi-asserted-by":"publisher","first-page":"37775","DOI":"10.1038\/s41598-025-14839-7","article-title":"Multimodal dual-stage feature refinement for robust skin lesion classification","volume":"15","author":"Khurshid","year":"2025","journal-title":"Sci Rep"},{"key":"B28","doi-asserted-by":"publisher","first-page":"45660","DOI":"10.1038\/s41598-025-30534-z","article-title":"Multimodal deep learning ensemble framework for skin cancer detection","volume":"15","author":"Saeed","year":"2025","journal-title":"Sci Rep"},{"key":"B29","doi-asserted-by":"publisher","first-page":"102588","DOI":"10.1016\/j.compmedimag.2025.102588","article-title":"multimodal skin lesion classification through cross-attention fusion and collaborative edge computing","volume":"124","author":"Tran-Van","year":"2025","journal-title":"Computer Med Imag Graph"},{"key":"B30","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/sdata.2018.161","article-title":"The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions","volume":"5","author":"Tschandl","year":"2018","journal-title":"Sci Data"},{"key":"B31","doi-asserted-by":"publisher","first-page":"321","DOI":"10.1613\/jair.953","article-title":"SMOTE synthetic minority over-sampling technique","volume":"16","author":"Chawla","year":"2002","journal-title":"J Artif Intell Res"},{"key":"B32","doi-asserted-by":"publisher","first-page":"55","DOI":"10.33480\/jitk.v11i1.6676","article-title":"Enhancing machine learning algorithm performance for PCOS diagnosis using SMOTENC on imbalanced data","volume":"11","author":"Dewi","year":"2025","journal-title":"JITK"},{"key":"B33","doi-asserted-by":"publisher","first-page":"125","DOI":"10.3390\/info11020125","article-title":"Albumentations: fast and flexible image augmentations","volume":"11","author":"Buslaev","year":"2020","journal-title":"Information"},{"key":"B34","article-title":"An image is worth 16x16 words: transformers for image recognition at scale","author":"Dosovitskiy","year":"2020","journal-title":"arXiv preprint arXiv:2010.11929"},{"key":"B35","doi-asserted-by":"publisher","first-page":"273","DOI":"10.1007\/BF00994018","article-title":"Support-vector networks","volume":"20","author":"Cortes","year":"1995","journal-title":"Mach Learn"},{"key":"B36","volume-title":"Classification and Regression Trees","author":"Breiman","year":"1984"},{"key":"B37","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1109\/TIT.1967.1053964","article-title":"Nearest neighbor pattern classification","volume":"13","author":"Cover","year":"1967","journal-title":"IEEE Trans Inf Theory"},{"key":"B38","doi-asserted-by":"publisher","first-page":"215","DOI":"10.1111\/j.2517-6161.1958.tb00292.x","article-title":"The regression analysis of binary sequences","volume":"20","author":"Cox","year":"1958","journal-title":"J R Stat Soc Series B"},{"key":"B39","doi-asserted-by":"publisher","first-page":"722","DOI":"10.55730\/1300-0632.4013","article-title":"A practical framework for early detection of diabetes using ensemble machine learning models","volume":"31","author":"Saihood","year":"2023","journal-title":"Turkish J Electr Eng Comput Sci"},{"key":"B40","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach Learn"},{"key":"B41","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1007\/s10994-006-6226-1","article-title":"Extremely randomized trees","volume":"63","author":"Geurts","year":"2006","journal-title":"Mach Learn"},{"key":"B42","doi-asserted-by":"publisher","first-page":"785","DOI":"10.1145\/2939672.2939785","article-title":"Xgboost: a scalable tree boosting system","author":"Chen","year":"2016","journal-title":"Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining"},{"key":"B43","first-page":"3146","article-title":"LightGBM: a highly efficient gradient boosting decision tree","author":"Ke","year":"2017","journal-title":"Advances in neural information processing systems"},{"key":"B44","doi-asserted-by":"publisher","first-page":"241","DOI":"10.1016\/S0893-6080(05)80023-1","article-title":"Stacked generalization","volume":"5","author":"Wolpert","year":"1992","journal-title":"Neural Netw"},{"key":"B45","doi-asserted-by":"publisher","first-page":"217","DOI":"10.1007\/s12530-020-09345-2","article-title":"Automatic tuning of hyperparameters using Bayesian optimization","volume":"12","author":"Victoria","year":"2021","journal-title":"Evolv Syst"},{"key":"B46","doi-asserted-by":"publisher","first-page":"388","DOI":"10.9734\/jerr\/2024\/v26i61188","article-title":"Hyperparameter tuning in machine learning: a comprehensive review","volume":"26","author":"Ilemobayo","year":"2024","journal-title":"J Eng Res Rep"},{"key":"B47","doi-asserted-by":"publisher","first-page":"618","DOI":"10.1109\/ICCV.2017.74","article-title":"Grad-CAM: visual explanations from deep networks via gradient-based localization","author":"Selvaraju","year":"2017","journal-title":"Proceedings of the IEEE international conference on computer vision (ICCV)"}],"container-title":["Frontiers in Medicine"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2026.1871689\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,29]],"date-time":"2026-07-29T04:49:59Z","timestamp":1785300599000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2026.1871689\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,29]]},"references-count":47,"alternative-id":["10.3389\/fmed.2026.1871689"],"URL":"https:\/\/doi.org\/10.3389\/fmed.2026.1871689","relation":{},"ISSN":["2296-858X"],"issn-type":[{"value":"2296-858X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,29]]},"article-number":"1871689"}}