{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,13]],"date-time":"2026-06-13T14:32:24Z","timestamp":1781361144735,"version":"3.54.1"},"reference-count":50,"publisher":"MDPI AG","issue":"17","license":[{"start":{"date-parts":[[2021,8,30]],"date-time":"2021-08-30T00:00:00Z","timestamp":1630281600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100007835","name":"Politechnika \u015al\u0105ska","doi-asserted-by":"publisher","award":["07\/010\/RGH20\/1004"],"award-info":[{"award-number":["07\/010\/RGH20\/1004"]}],"id":[{"id":"10.13039\/501100007835","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100007835","name":"Politechnika \u015al\u0105ska","doi-asserted-by":"publisher","award":["BK-296\/RIB1\/2021"],"award-info":[{"award-number":["BK-296\/RIB1\/2021"]}],"id":[{"id":"10.13039\/501100007835","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This study presents the first application of convolutional neural networks to high-frequency ultrasound skin image classification. This type of imaging opens up new opportunities in dermatology, showing inflammatory diseases such as atopic dermatitis, psoriasis, or skin lesions. We collected a database of 631 images with healthy skin and different skin pathologies to train and assess all stages of the methodology. The proposed framework starts with the segmentation of the epidermal layer using a DeepLab v3+ model with a pre-trained Xception backbone. We employ transfer learning to train the segmentation model for two purposes: to extract the region of interest for classification and to prepare the skin layer map for classification confidence estimation. For classification, we train five models in different input data modes and data augmentation setups. We also introduce a classification confidence level to evaluate the deep model\u2019s reliability. The measure combines our skin layer map with the heatmap produced by the Grad-CAM technique designed to indicate image regions used by the deep model to make a classification decision. Moreover, we propose a multicriteria model evaluation measure to select the optimal model in terms of classification accuracy, confidence, and test dataset size. The experiments described in the paper show that the DenseNet-201 model fed with the extracted region of interest produces the most reliable and accurate results.<\/jats:p>","DOI":"10.3390\/s21175846","type":"journal-article","created":{"date-parts":[[2021,8,31]],"date-time":"2021-08-31T22:58:15Z","timestamp":1630450695000},"page":"5846","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":19,"title":["Deep Learning-Based High-Frequency Ultrasound Skin Image Classification with Multicriteria Model Evaluation"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6237-1793","authenticated-orcid":false,"given":"Joanna","family":"Czajkowska","sequence":"first","affiliation":[{"name":"Faculty of Biomedical Engineering, Silesian University of Technology, 41-800 Zabrze, Poland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3941-4927","authenticated-orcid":false,"given":"Pawel","family":"Badura","sequence":"additional","affiliation":[{"name":"Faculty of Biomedical Engineering, Silesian University of Technology, 41-800 Zabrze, Poland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2558-608X","authenticated-orcid":false,"given":"Szymon","family":"Korzekwa","sequence":"additional","affiliation":[{"name":"Department of Temporomandibular Disorders, Division of Prosthodontics, Poznan University of Medical Sciences, 60-512 Pozna\u0144, Poland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0806-6760","authenticated-orcid":false,"given":"Anna","family":"P\u0142atkowska-Szczerek","sequence":"additional","affiliation":[{"name":"Anclara sp. z o.o., 02-624 Warszawa, Poland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7875-7383","authenticated-orcid":false,"given":"Monika","family":"S\u0142owi\u0144ska","sequence":"additional","affiliation":[{"name":"Department of Dermatology, Military Institute of Medicine, 01-755 Warszawa, Poland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,8,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1016\/j.eng.2018.11.020","article-title":"Deep Learning in Medical Ultrasound Analysis: A Review","volume":"5","author":"Liu","year":"2019","journal-title":"Engineering"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1186\/s13089-021-00222-w","article-title":"High-frequency ultrasound in clinical dermatology: A review","volume":"13","author":"Levy","year":"2021","journal-title":"Ultrasound J."},{"key":"ref_3","first-page":"237","article-title":"Application of high frequency ultrasound in dermatology","volume":"26","author":"Bhatta","year":"2018","journal-title":"Discov. Med."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"28360","DOI":"10.1038\/srep28360","article-title":"Ultrahigh Frequency (100 MHz\u2013300 MHz) Ultrasonic Transducers for Optical Resolution Medical Imagining","volume":"6","author":"Fei","year":"2016","journal-title":"Sci. Rep."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"262","DOI":"10.1590\/abd1806-4841.20164446","article-title":"High frequency ultrasound with color Doppler in dermatology","volume":"91","author":"Carvalho","year":"2016","journal-title":"An. Bras. Dermatol."},{"key":"ref_6","first-page":"112","article-title":"Nevus sebaceus of Jadassohn\u2014High frequency ultrasound imaging and videodermoscopy examination. Case presentation","volume":"94","author":"Bezugly","year":"2021","journal-title":"Med. Pharm. Rep."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"106412","DOI":"10.1016\/j.ultras.2021.106412","article-title":"Deep learning approach to skin layers segmentation in inflammatory dermatoses","volume":"114","author":"Czajkowska","year":"2021","journal-title":"Ultrasonics"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"513","DOI":"10.1007\/s40257-020-00517-z","article-title":"A Review of Noninvasive Techniques for Skin Cancer Detection in Dermatology","volume":"21","author":"Heibel","year":"2020","journal-title":"Am. J. Clin. Dermatol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"478","DOI":"10.1016\/j.jaad.2011.12.016","article-title":"Ultrasound in dermatology: Principles and applications","volume":"67","author":"Kleinerman","year":"2012","journal-title":"J. Am. Acad. Dermatol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1111\/srt.12153","article-title":"Monitoring of therapy in atopic dermatitis\u2014Observations with the use of high-frequency ultrasonography","volume":"21","author":"Silny","year":"2015","journal-title":"Skin Res. Technol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1186\/s13075-020-02285-x","article-title":"Ultrasound assessment of skin thickness and stiffness: The correlation with histology and clinical score in systemic sclerosis","volume":"22","author":"Chen","year":"2020","journal-title":"Arthritis Res. Ther."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1509","DOI":"10.1109\/TMI.2012.2190617","article-title":"Segmentation of Skin Lesions in 2-D and 3-D Ultrasound Images Using a Spatially Coherent Generalized Rayleigh Mixture Model","volume":"31","author":"Pereyra","year":"2012","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2166","DOI":"10.1016\/j.ultrasmedbio.2013.04.006","article-title":"Automated Skin Segmentation in Ultrasonic Evaluation of Skin Toxicity in Breast Cancer Radiotherapy","volume":"39","author":"Gao","year":"2013","journal-title":"Ultrasound Med. Biol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"302","DOI":"10.1016\/j.compbiomed.2017.06.012","article-title":"Improved boundary segmentation of skin lesions in high-frequency 3D ultrasound","volume":"87","author":"Sciolla","year":"2017","journal-title":"Comput. Biol. Med."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"101676","DOI":"10.1016\/j.compmedimag.2019.101676","article-title":"Computer Aided Diagnosis of Atopic Dermatitis","volume":"79","author":"Czajkowska","year":"2020","journal-title":"Comput. Med. Imaging Graph."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"106268","DOI":"10.1016\/j.ultras.2020.106268","article-title":"Automated seeding for ultrasound skin lesion segmentation","volume":"110","author":"Szalai","year":"2021","journal-title":"Ultrasonics"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"582","DOI":"10.1007\/s10278-019-00227-x","article-title":"Deep Learning Techniques for Medical Image Segmentation: Achievements and Challenges","volume":"32","author":"Hesamian","year":"2019","journal-title":"J. Digit. Imaging"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"3627","DOI":"10.1364\/BOE.8.003627","article-title":"ReLayNet: Retinal layer and fluid segmentation of macular optical coherence tomography using fully convolutional networks","volume":"8","author":"Roy","year":"2017","journal-title":"Biomed. Opt. Express"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"3484","DOI":"10.1364\/BOE.10.003484","article-title":"Segmentation of mouse skin layers in optical coherence tomography image data using deep convolutional neural networks","volume":"10","author":"Kepp","year":"2019","journal-title":"Biomed. Opt. Express"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A.A. (2017, January 4\u20139). Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.","DOI":"10.1609\/aaai.v31i1.11231"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2281","DOI":"10.1109\/TMI.2019.2903562","article-title":"CE-Net: Context Encoder Network for 2D Medical Image Segmentation","volume":"38","author":"Gu","year":"2019","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"220","DOI":"10.3389\/fmed.2020.00220","article-title":"Automatic Segmentation of Epidermis and Hair Follicles in Optical Coherence Tomography Images of Normal Skin by Convolutional Neural Networks","volume":"7","author":"Morales","year":"2020","journal-title":"Front. Med."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"5137904","DOI":"10.1155\/2018\/5137904","article-title":"Machine Learning in Ultrasound Computer-Aided Diagnostic Systems: A Survey","volume":"2018","author":"Huang","year":"2018","journal-title":"BioMed Res. Int."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"713","DOI":"10.21037\/atm.2020.02.44","article-title":"A review of the application of deep learning in medical image classification and segmentation","volume":"8","author":"Cai","year":"2020","journal-title":"Ann. Transl. Med."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"7714","DOI":"10.1088\/1361-6560\/aa82ec","article-title":"A deep learning framework for supporting the classification of breast lesions in ultrasound images","volume":"62","author":"Han","year":"2017","journal-title":"Phys. Med. Biol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"477","DOI":"10.1007\/s10278-017-9997-y","article-title":"Thyroid Nodule Classification in Ultrasound Images by Fine-Tuning Deep Convolutional Neural Network","volume":"30","author":"Chi","year":"2017","journal-title":"J. Digit. Imaging"},{"key":"ref_27","first-page":"5804","article-title":"Liver fibrosis classification based on transfer learning and FCNet for ultrasound images","volume":"5","author":"Meng","year":"2017","journal-title":"IEEE Access"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"10200","DOI":"10.1038\/s41598-020-67076-5","article-title":"Evaluation of deep convolutional neural networks for automatic classification of common maternal fetal ultrasound planes","volume":"10","author":"Eixarch","year":"2020","journal-title":"Sci. Rep."},{"key":"ref_29","unstructured":"Karimi, D., Warfield, S.K., and Gholipour, A. (2020, May 30). Critical Assessment of Transfer Learning for Medical Image Segmentation with Fully Convolutional Neural Networks. Available online: https:\/\/arxiv.org\/abs\/2006.00356."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1018","DOI":"10.1109\/TMI.2014.2366792","article-title":"Transfer Learning Improves Supervised Image Segmentation Across Imaging Protocols","volume":"34","author":"Ikram","year":"2015","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"104115","DOI":"10.1016\/j.compbiomed.2020.104115","article-title":"A scoping review of transfer learning research on medical image analysis using ImageNet","volume":"128","author":"Morid","year":"2021","journal-title":"Comput. Biol. Med."},{"key":"ref_32","unstructured":"(2021, April 08). ImageNet. Available online: http:\/\/www.image-net.org."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Ayana, G., Dese, K., and woon Choe, S. (2021). Transfer Learning in Breast Cancer Diagnoses via Ultrasound Imaging. Cancers, 13.","DOI":"10.3390\/cancers13040738"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Kim, I., Rajaraman, S., and Antani, S. (2019). Visual Interpretation of Convolutional Neural Network Predictions in Classifying Medical Image Modalities. Diagnostics, 9.","DOI":"10.3390\/diagnostics9020038"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017, January 22\u201329). Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.74"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., and Torralba, A. (2016, January 27\u201330). Learning Deep Features for Discriminative Localization. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.319"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"106412","DOI":"10.1016\/j.ultras.2021.106412","article-title":"Data for: Deep Learning Approach to Skin Layers Segmentation in Inflammatory Dermatoses","volume":"114","author":"Czajkowska","year":"2021","journal-title":"Ultrasonics"},{"key":"ref_38","unstructured":"Huang, G., Liu, Z., van der Maaten, L., and Weinberger, K.Q. (2018, January 28). Densely Connected Convolutional Networks. Available online: https:\/\/arxiv.org\/abs\/1608.06993."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2014, September 17). Going Deeper with Convolutions. Available online: https:\/\/arxiv.org\/abs\/1409.4842.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.C. (2019, March 21). MobileNetV2: Inverted Residuals and Linear Bottlenecks. Available online: https:\/\/arxiv.org\/abs\/1801.04381.","DOI":"10.1109\/CVPR.2018.00474"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_42","unstructured":"Chen, L.C., Papandreou, G., Schroff, F., and Adam, H. (2017, December 05). Rethinking Atrous Convolution for Semantic Image Segmentation. Available online: https:\/\/arxiv.org\/abs\/1706.05587."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H. (2018, August 22). Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. Available online: https:\/\/arxiv.org\/abs\/1802.02611.","DOI":"10.1007\/978-3-030-01234-2_49"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Chollet, F. (2017, January 21\u201326). Xception: Deep Learning with Depthwise Separable Convolutions. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.195"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Li, F.F. (2009, January 20\u201325). Imagenet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Sudre, C.H., Li, W., Vercauteren, T., Ourselin, S., and Cardoso, M.J. (2017). Generalised Dice Overlap as a Deep Learning Loss Function for Highly Unbalanced Segmentations. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Springer International Publishing.","DOI":"10.1007\/978-3-319-67558-9_28"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"6423","DOI":"10.1016\/j.vaccine.2007.05.046","article-title":"Echographic measurement of skin thickness in adults by high frequency ultrasound to assess the appropriate microneedle length for intradermal delivery of vaccines","volume":"25","author":"Laurent","year":"2007","journal-title":"Vaccine"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"56","DOI":"10.4103\/tjps.TJPS_2_17","article-title":"Measurement of epidermis, dermis, and total skin thicknesses from six different body regions with a new ethical histometric technique","volume":"26","author":"Oltulu","year":"2018","journal-title":"Turk. J. Plast. Surg."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"297","DOI":"10.2307\/1932409","article-title":"Measures of the Amount of Ecologic Association Between Species","volume":"26","author":"Dice","year":"1945","journal-title":"Ecology"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1016\/j.compbiomed.2018.10.029","article-title":"Joint segmentation and characterization of the dermis in 50 MHz ultrasound 2D and 3D images of the skin","volume":"103","author":"Sciolla","year":"2018","journal-title":"Comput. Biol. Med."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/17\/5846\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:55:56Z","timestamp":1760165756000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/17\/5846"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,30]]},"references-count":50,"journal-issue":{"issue":"17","published-online":{"date-parts":[[2021,9]]}},"alternative-id":["s21175846"],"URL":"https:\/\/doi.org\/10.3390\/s21175846","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,8,30]]}}}