{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,26]],"date-time":"2025-12-26T23:10:39Z","timestamp":1766790639987,"version":"3.40.3"},"publisher-location":"Cham","reference-count":19,"publisher":"Springer International Publishing","isbn-type":[{"type":"print","value":"9783030904388"},{"type":"electronic","value":"9783030904395"}],"license":[{"start":{"date-parts":[[2021,1,1]],"date-time":"2021-01-01T00:00:00Z","timestamp":1609459200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,1,1]],"date-time":"2022-01-01T00:00:00Z","timestamp":1640995200000},"content-version":"vor","delay-in-days":365,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Skin cancer is one of the most common types of cancer and, with its increasing incidence, accurate early diagnosis is crucial to improve prognosis of patients. In the process of visual inspection, dermatologists follow specific dermoscopic algorithms and identify important features to provide a diagnosis. This process can be automated as such characteristics can be extracted by computer vision techniques. Although deep neural networks can extract useful features from digital images for skin lesion classification, performance can be improved by providing additional information. The extracted pseudo-features can be used as input (multimodal) or output (multi-tasking) to train a robust deep learning model. This work investigates the multimodal and multi-tasking techniques for more efficient training, given the single optimization of several related tasks in the latter, and generation of better diagnosis predictions. Additionally, the role of lesion segmentation is also studied. Results show that multi-tasking improves learning of beneficial features which lead to better predictions, and pseudo-features inspired by the ABCD rule provide readily available helpful information about the skin lesion.<\/jats:p>","DOI":"10.1007\/978-3-030-90439-5_3","type":"book-chapter","created":{"date-parts":[[2021,12,2]],"date-time":"2021-12-02T14:13:49Z","timestamp":1638454429000},"page":"27-38","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Multimodal Multi-tasking for\u00a0Skin Lesion Classification Using Deep Neural Networks"],"prefix":"10.1007","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6936-3286","authenticated-orcid":false,"given":"Rafaela","family":"Carvalho","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7588-8927","authenticated-orcid":false,"given":"Jo\u00e3o","family":"Pedrosa","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2047-2254","authenticated-orcid":false,"given":"Tudor","family":"Nedelcu","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,1,1]]},"reference":[{"issue":"2","key":"3_CR1","doi-asserted-by":"publisher","first-page":"148","DOI":"10.1016\/j.compmedimag.2008.11.002","volume":"33","author":"ME Celebi","year":"2009","unstructured":"Celebi, M.E., Iyatomi, H., Schaefer, G., Stoecker, W.V.: Lesion border detection in dermoscopy images. Comput. Med. Imaging Graph. 33(2), 148\u2013153 (2009)","journal-title":"Comput. Med. Imaging Graph."},{"issue":"6","key":"3_CR2","doi-asserted-by":"publisher","first-page":"362","DOI":"10.1016\/j.compmedimag.2007.01.003","volume":"31","author":"ME Celebi","year":"2007","unstructured":"Celebi, M.E., et al.: A methodological approach to the classification of dermoscopy images. Comput. Med. Imaging Graph. 31(6), 362\u2013373 (2007)","journal-title":"Comput. Med. Imaging Graph."},{"key":"3_CR3","doi-asserted-by":"crossref","unstructured":"Codella, N.C., et al.: Skin lesion analysis toward melanoma detection: a challenge at the 2017 international symposium on biomedical imaging (isbi), hosted by the international skin imaging collaboration (isic). In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pp. 168\u2013172. IEEE (2018)","DOI":"10.1109\/ISBI.2018.8363547"},{"key":"3_CR4","doi-asserted-by":"crossref","unstructured":"Esteva, A., et al.: Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639), 115\u2013118 (2017)","DOI":"10.1038\/nature21056"},{"key":"3_CR5","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770\u2013778 (2016)","DOI":"10.1109\/CVPR.2016.90"},{"key":"3_CR6","doi-asserted-by":"crossref","unstructured":"Iyatomi, H., Oka, H., Celebi, M.E., Tanaka, M., Ogawa, K.: Parameterization of dermoscopic findings for the internet-based melanoma screening system. In: 2007 IEEE Symposium on Computational Intelligence in Image and Signal Processing, pp. 189\u2013193. IEEE (2007)","DOI":"10.1109\/CIISP.2007.369315"},{"issue":"3","key":"3_CR7","doi-asserted-by":"publisher","first-page":"240","DOI":"10.1016\/S0738-081X(02)00236-5","volume":"20","author":"RH Johr","year":"2002","unstructured":"Johr, R.H.: Dermoscopy: alternative melanocytic algorithms\u2013the abcd rule of dermatoscopy, menzies scoring method, and 7-point checklist. Clin. Dermatol. 20(3), 240\u2013247 (2002)","journal-title":"Clin. Dermatol."},{"issue":"2","key":"3_CR8","doi-asserted-by":"publisher","first-page":"538","DOI":"10.1109\/JBHI.2018.2824327","volume":"23","author":"J Kawahara","year":"2018","unstructured":"Kawahara, J., Daneshvar, S., Argenziano, G., Hamarneh, G.: Seven-point checklist and skin lesion classification using multitask multimodal neural nets. IEEE J. Biomed. Health Inform. 23(2), 538\u2013546 (2018)","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"3_CR9","unstructured":"Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)"},{"key":"3_CR10","unstructured":"Liao, H., Luo, J.: A deep multi-task learning approach to skin lesion classification. arXiv preprint arXiv:1812.03527 (2018)"},{"issue":"4","key":"3_CR11","doi-asserted-by":"publisher","first-page":"552","DOI":"10.1016\/j.jaad.2012.08.012","volume":"68","author":"MA Pizzichetta","year":"2013","unstructured":"Pizzichetta, M.A., et al.: Negative pigment network: an additional dermoscopic feature for the diagnosis of melanoma. J. Am. Acad. Dermatol. 68(4), 552\u2013559 (2013)","journal-title":"J. Am. Acad. Dermatol."},{"issue":"3","key":"3_CR12","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","volume":"115","author":"O Russakovsky","year":"2015","unstructured":"Russakovsky, O., et al.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211\u2013252 (2015)","journal-title":"Int. J. Comput. Vis."},{"key":"3_CR13","doi-asserted-by":"crossref","unstructured":"Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-cam: Visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 618\u2013626 (2017)","DOI":"10.1109\/ICCV.2017.74"},{"key":"3_CR14","doi-asserted-by":"crossref","unstructured":"Siegel, R.L., Miller, K.D., Jemal, A.: Cancer statistics, 2019. CA: Canc. J. Clin. 69(1), 7\u201334 (2019)","DOI":"10.3322\/caac.21551"},{"issue":"10","key":"3_CR15","doi-asserted-by":"publisher","first-page":"1222","DOI":"10.1111\/j.1468-3083.2010.03920.x","volume":"25","author":"S Stricklin","year":"2011","unstructured":"Stricklin, S., Stoecker, W., Oliviero, M., Rabinovitz, H., Mahajan, S.: Cloudy and starry milia-like cysts: how well do they distinguish seborrheic keratoses from malignant melanomas? J. Euro. Acad. Dermatol. Venereology 25(10), 1222\u20131224 (2011)","journal-title":"J. Euro. Acad. Dermatol. Venereology"},{"key":"3_CR16","doi-asserted-by":"crossref","unstructured":"Sung, H., et al.: Global cancer statistics 2020: globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA: Canc. J. Clin. 71(3), 209\u2013249 (2021)","DOI":"10.3322\/caac.21660"},{"key":"3_CR17","doi-asserted-by":"crossref","unstructured":"Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1\u20139 (2015)","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"3_CR18","unstructured":"Tan, M., Le, Q.: Efficientnet: rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105\u20136114. PMLR (2019)"},{"issue":"11","key":"3_CR19","doi-asserted-by":"publisher","first-page":"1261","DOI":"10.1111\/exd.13777","volume":"27","author":"J Yap","year":"2018","unstructured":"Yap, J., Yolland, W., Tschandl, P.: Multimodal skin lesion classification using deep learning. Exp. Dermatol. 27(11), 1261\u20131267 (2018)","journal-title":"Exp. Dermatol."}],"container-title":["Lecture Notes in Computer Science","Advances in Visual Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/978-3-030-90439-5_3","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,12,2]],"date-time":"2021-12-02T14:20:57Z","timestamp":1638454857000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/978-3-030-90439-5_3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021]]},"ISBN":["9783030904388","9783030904395"],"references-count":19,"URL":"https:\/\/doi.org\/10.1007\/978-3-030-90439-5_3","relation":{},"ISSN":["0302-9743","1611-3349"],"issn-type":[{"type":"print","value":"0302-9743"},{"type":"electronic","value":"1611-3349"}],"subject":[],"published":{"date-parts":[[2021]]},"assertion":[{"value":"1 January 2022","order":1,"name":"first_online","label":"First Online","group":{"name":"ChapterHistory","label":"Chapter History"}},{"value":"ISVC","order":1,"name":"conference_acronym","label":"Conference Acronym","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"International Symposium on Visual Computing","order":2,"name":"conference_name","label":"Conference Name","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"2021","order":5,"name":"conference_year","label":"Conference Year","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"4 October 2021","order":7,"name":"conference_start_date","label":"Conference Start Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"6 October 2021","order":8,"name":"conference_end_date","label":"Conference End Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"isvc2021","order":10,"name":"conference_id","label":"Conference ID","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"http:\/\/www.isvc.net\/?utm_source=researchbib","order":11,"name":"conference_url","label":"Conference URL","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Double-blind","order":1,"name":"type","label":"Type","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"Easychair","order":2,"name":"conference_management_system","label":"Conference Management System","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"135","order":3,"name":"number_of_submissions_sent_for_review","label":"Number of Submissions Sent for Review","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"48","order":4,"name":"number_of_full_papers_accepted","label":"Number of Full Papers Accepted","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"0","order":5,"name":"number_of_short_papers_accepted","label":"Number of Short Papers Accepted","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"36% - The value is computed by the equation \"Number of Full Papers Accepted \/ Number of Submissions Sent for Review * 100\" and then rounded to a whole number.","order":6,"name":"acceptance_rate_of_full_papers","label":"Acceptance Rate of Full Papers","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"3","order":7,"name":"average_number_of_reviews_per_paper","label":"Average Number of Reviews per Paper","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"2.5","order":8,"name":"average_number_of_papers_per_reviewer","label":"Average Number of Papers per Reviewer","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"No","order":9,"name":"external_reviewers_involved","label":"External Reviewers Involved","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}}]}}