{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,16]],"date-time":"2026-09-16T01:14:44Z","timestamp":1789521284459,"version":"build-2803163510"},"update-to":[{"DOI":"10.12688\/f1000research.169102.1","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2026,2,9]],"date-time":"2026-02-09T00:00:00Z","timestamp":1770595200000}}],"reference-count":43,"publisher":"F1000 Research Ltd","license":[{"start":{"date-parts":[[2026,2,9]],"date-time":"2026-02-09T00:00:00Z","timestamp":1770595200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["f1000research.com"],"crossmark-restriction":false},"short-container-title":["F1000Res"],"abstract":"<ns3:p>Background Atopic dermatitis (AD) is a prevalent, chronic inflammatory skin disease with diverse clinical presentations, often overlapping with other dermatoses. Its diagnosis remains largely dependent on clinical expertise, leading to variability and limited diagnostic accuracy, particularly among general practitioners. This study aimed to develop and evaluate a multimodal artificial intelligence (AI) model that integrates lesion image analysis and structured anamnesis to improve AD diagnosis. Methods This diagnostic study was conducted in two phases: Phase 1 used retrospective data from 2021\u20132024, and Phase 2 involved prospective external validation from multiple hospitals in 2025. Patients with AD or related skin conditions were included, with diagnoses based on AAD 2014 criteria. Multimodal fusion combined ResNet50-extracted image features and MPNet-based anamnesis text features using a late fusion model. This approach mimics clinical reasoning by integrating visual and contextual clinical information to classify cases as AD or non-AD. Results and Discussion The multimodal AI model integrating ResNet50 (image) and MPNet (anamnesis) achieved 98.28% accuracy in classifying AD vs non-AD, outperforming image-or text-only models. It offers clinical advantages by mimicking physician reasoning, improving diagnostic consistency, reducing subjectivity, and enabling mass triage. However, real-world generalizability remains a challenge due to limited training diversity, potential language constraints (Bahasa Indonesia), and narrow differential diagnoses. External validation and explainable AI (XAI) are critical for broader application. Despite limitations, the model aligns with emerging literature, showing multimodal AI can approach or surpass expert-level performance in dermatological diagnosis when rigorously validated. Conclusions The multimodal ResNet50-MPNet model shows near-perfect accuracy in diagnosing AD by mimicking clinician reasoning. It offers consistent, holistic assessment but requires external validation and improved interpretability for clinical adoption. Continued AI-clinician collaboration is vital to translating this promising technology into real-world dermatological care.<\/ns3:p>","DOI":"10.12688\/f1000research.169102.2","type":"journal-article","created":{"date-parts":[[2026,2,9]],"date-time":"2026-02-09T13:44:08Z","timestamp":1770644648000},"page":"952","update-policy":"https:\/\/doi.org\/10.12688\/f1000research.crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Multimodal Machine Learning Approach for Diagnosing Atopic Dermatitis"],"prefix":"10.12688","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0009-0002-6325-0843","authenticated-orcid":false,"given":"Alida","family":"Widiawaty","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wresti","family":"Indriatmi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wisnu","family":"Jatmiko","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Endi","family":"Novianto","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aria","family":"Kekalih","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hendra","family":"Gunawan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pramudita","family":"Satria Palar","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Muhammad","family":"Febrian Rachmadi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-8530-662X","authenticated-orcid":false,"given":"Sherly","family":"Dermawan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tengku","family":"Laras Malahayati","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alif","family":"Wicaksana Ramadhan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"2560","published-online":{"date-parts":[[2026,2,9]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","first-page":"337-352","DOI":"10.1111\/cea.14276","article-title":"Trends in eczema prevalence in children and adolescents: A Global Asthma Network Phase I Study.","volume":"53","author":"S Langan","year":"2023 Mar","journal-title":"Clin Exp Allergy."},{"key":"ref2","doi-asserted-by":"publisher","first-page":"936","DOI":"10.3390\/life11090936","article-title":"The Epidemiology and Global Burden of Atopic Dermatitis: A Narrative Review.","volume":"11","author":"H Hadi","year":"2021 Sept 9","journal-title":"Life (Basel)."},{"key":"ref3","doi-asserted-by":"publisher","first-page":"98-108","DOI":"10.1016\/j.jdin.2020.10.013","article-title":"The global burden of skin cancer: A longitudinal analysis from the Global Burden of Disease Study, 1990-2017.","volume":"2","author":"K Urban","year":"2021 Mar","journal-title":"JAAD Int."},{"key":"ref4","doi-asserted-by":"publisher","first-page":"1057","DOI":"10.3390\/children8111057","article-title":"Skin Disease in Children: Effects on Quality of Life, Stigmatization, Bullying, and Suicide Risk in Pediatric Acne, Atopic Dermatitis, and Psoriasis Patients.","volume":"8","author":"K Kelly","year":"2021 Nov 16","journal-title":"Children (Basel)."},{"key":"ref5","doi-asserted-by":"publisher","first-page":"1161-1187","DOI":"10.1007\/s10787-025-01642-z","article-title":"Atopic dermatitis: a comprehensive updated review of this intriguing disease with futuristic insights.","volume":"33","author":"H Abdel-Mageed","year":"2025 Mar 1","journal-title":"Inflammopharmacol."},{"key":"ref6","doi-asserted-by":"publisher","first-page":"191-196","DOI":"10.2174\/1573396317666210901124211","article-title":"Quality of Life for Children with Allergic Skin Diseases.","volume":"18","author":"K Hon","year":"2022","journal-title":"Curr. Pediatr. Rev."},{"key":"ref7","doi-asserted-by":"publisher","first-page":"587-603","DOI":"10.5863\/1551-6776-29.6.587","article-title":"Atopic Dermatitis: A Review of Diagnosis and Treatment.","volume":"29","author":"J Jeskey","year":"2024 Dec","journal-title":"J. Pediatr. Pharmacol. Ther."},{"key":"ref8","doi-asserted-by":"publisher","first-page":"24-40.e4","DOI":"10.1016\/j.jaci.2025.02.029","article-title":"Endotypes of atopic dermatitis.","volume":"156","author":"N Fyhrquist","year":"2025 July","journal-title":"J. Allergy Clin. Immunol."},{"key":"ref9","doi-asserted-by":"publisher","first-page":"895-915","DOI":"10.1016\/j.abd.2024.06.001","article-title":"Update on the pathogenesis of atopic dermatitis.","volume":"99","author":"P Criado","year":"2024","journal-title":"An. Bras. Dermatol."},{"key":"ref10","doi-asserted-by":"publisher","first-page":"116-132","DOI":"10.1016\/j.jaad.2014.03.023","article-title":"Guidelines of care for the management of atopic dermatitis: section 2. Management and treatment of atopic dermatitis with topical therapies.","volume":"71","author":"L Eichenfield","year":"2014 July","journal-title":"J. Am. Acad. Dermatol."},{"key":"ref11","doi-asserted-by":"publisher","first-page":"698-704","DOI":"10.1111\/jdv.17929","article-title":"Prevalence and characteristics of atopic dermatitis among young adult females and males-report from the Swedish population-based study BAMSE.","volume":"36","author":"E Johansson","year":"2022 May","journal-title":"J. Eur. Acad. Dermatol. Venereol."},{"key":"ref12","doi-asserted-by":"publisher","first-page":"e15901","DOI":"10.1111\/dth.15901","article-title":"Children atopic dermatitis: Diagnosis, mimics, overlaps, and therapeutic implication.","volume":"35","author":"M Napolitano","year":"2022 Dec","journal-title":"Dermatol. Ther."},{"key":"ref13","doi-asserted-by":"publisher","first-page":"2484665","DOI":"10.1080\/07853890.2025.2484665","article-title":"Advancements in artificial intelligence for atopic dermatitis: diagnosis, treatment, and patient management.","volume":"57","author":"F Cao","year":"2025 Dec","journal-title":"Ann. Med."},{"key":"ref14","doi-asserted-by":"publisher","first-page":"230-234","DOI":"10.1111\/j.1440-0960.2005.00189.x","article-title":"Assessing diagnostic skill in dermatology: a comparison between general practitioners and dermatologists.","volume":"46","author":"H Tran","year":"2005 Nov","journal-title":"Australas. J. Dermatol."},{"key":"ref15","doi-asserted-by":"publisher","first-page":"1708-1710.e1","DOI":"10.1016\/j.jaci.2017.04.052","article-title":"Relationship between EASI and SCORAD severity assessments for atopic dermatitis.","volume":"140","author":"R Chopra","year":"2017 Dec","journal-title":"J. Allergy Clin. Immunol."},{"key":"ref16","doi-asserted-by":"publisher","first-page":"365-386","DOI":"10.1007\/s13555-020-00372-0","article-title":"Machine Learning in Dermatology: Current Applications, Opportunities, and Limitations.","volume":"10","author":"S Chan","year":"2020 June","journal-title":"Dermatol Ther (Heidelb)."},{"key":"ref17","doi-asserted-by":"publisher","first-page":"106712","DOI":"10.1016\/j.compbiomed.2023.106712","article-title":"TFormer: A throughout fusion transformer for multi-modal skin lesion diagnosis.","volume":"157","author":"Y Zhang","year":"2023 May","journal-title":"Comput. Biol. Med."},{"key":"ref18","doi-asserted-by":"publisher","first-page":"4293","DOI":"10.1038\/s41598-023-31340-1","article-title":"Exploring the potential of artificial intelligence in improving skin lesion diagnosis in primary care.","volume":"13","author":"A Escal\u00e9-Besa","year":"2023 Mar 15","journal-title":"Sci. Rep."},{"key":"ref19","doi-asserted-by":"publisher","first-page":"581","DOI":"10.21037\/atm.2020.04.39","article-title":"A deep learning, image based approach for automated diagnosis for inflammatory skin diseases.","volume":"8","author":"H Wu","year":"2020 May","journal-title":"Ann Transl Med."},{"key":"ref20","doi-asserted-by":"publisher","first-page":"51-55","DOI":"10.1016\/j.ifacol.2022.06.008","article-title":"Diagnosis of Atopic dermatitis Using Artificial Neural Network.","volume":"55","author":"A Dautovic","year":"2022","journal-title":"IFAC PapersOnLine."},{"key":"ref21","doi-asserted-by":"publisher","first-page":"107413","DOI":"10.1016\/j.compbiomed.2023.107413","article-title":"Artificial intelligence-assisted dermatology diagnosis: From unimodal to multimodal.","volume":"165","author":"N Luo","year":"2023 Oct","journal-title":"Comput. Biol. Med."},{"key":"ref22","doi-asserted-by":"publisher","first-page":"282","DOI":"10.3390\/bioengineering12030282","article-title":"Deep multi-modal skin-imaging based information-switching network for skinlesion recognition.","volume":"12","author":"Y Yu","year":"2025","journal-title":"Bioengineering."},{"key":"ref23","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-025-20951-7","article-title":"MiSC: A hybrid multi-modal deep learning approach for accurate skin cancer detection.","author":"E Mohamed","year":"2025 June 4 cited 2025 Aug 15","journal-title":"Multimed. Tools Appl."},{"key":"ref24","doi-asserted-by":"publisher","first-page":"e0287786","DOI":"10.1371\/journal.pone.0287786","article-title":"Multi-modal medical image classification using deep residual network and genetic algorithm.","volume":"18","author":"M Abid","year":"2023","journal-title":"PLoS One."},{"key":"ref25","doi-asserted-by":"publisher","first-page":"201","DOI":"10.1186\/s12880-024-01356-8","article-title":"Enhanced skin cancer diagnosis using optimized CNN architecture and checkpoints for automated dermatological lesion classification.","volume":"24","author":"M Musthafa","year":"2024 Aug 2","journal-title":"BMC Med. Imaging."},{"key":"ref27","doi-asserted-by":"publisher","first-page":"1115","DOI":"10.3390\/diagnostics15091115","article-title":"A Comparison of Skin Lesions\u2019 Diagnoses Between AI-Based Image Classification, an Expert Dermatologist, and a Non-Expert.","volume":"15","author":"L Mevorach","year":"2025 Apr 28","journal-title":"Diagnostics (Basel)."},{"key":"ref28","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1038\/s41746-024-01103-x","article-title":"A systematic review and meta-analysis of artificial intelligence versus clinicians for skin cancer diagnosis.","volume":"7","author":"M Salinas","year":"2024 May 14","journal-title":"npj Digit Med."},{"key":"ref29","doi-asserted-by":"publisher","first-page":"e511","DOI":"10.52225\/narra.v3i3.511","article-title":"Evaluation of atopic dermatitis severity using artificial intelligence.","volume":"3","author":"A Maulana","year":"2023 Dec","journal-title":"Narra J."},{"key":"ref30","article-title":"Comparisons between a Large Language Model-based Real-Time Compound Diagnostic Medical AI Interface and Physicians for Common Internal Medicine Cases using Simulated Patients.","author":"H Park","year":"2025 cited 2025 June 19","journal-title":"arXiv."},{"key":"ref31","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1038\/s41746-021-00549-7","article-title":"Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review.","volume":"5","author":"A Hond","year":"2022 Jan 10","journal-title":"npj Digit Med."},{"key":"ref32","doi-asserted-by":"publisher","first-page":"2637-2651","DOI":"10.1007\/s13555-022-00833-8","article-title":"Artificial Intelligence in Dermatology: Challenges and Perspectives.","volume":"12","author":"K Liopyris","year":"2022 Oct 28","journal-title":"Dermatol Ther (Heidelb)."},{"key":"ref33","doi-asserted-by":"publisher","first-page":"19-20","DOI":"10.1001\/jama.2017.19198","article-title":"What This Computer Needs Is a Physician: Humanism and Artificial Intelligence.","volume":"319","author":"A Verghese","year":"2018 Jan 2","journal-title":"JAMA."},{"key":"ref34","doi-asserted-by":"publisher","first-page":"1-14","DOI":"10.18608\/jla.2023.8199","article-title":"Designing Visual Learning Analytics for Supporting Equity in STEM Classrooms.","author":"A Raza","year":"2024 cited 2025 June 19","journal-title":"arXiv.org."},{"key":"ref35","first-page":"1-12","article-title":"A multimodal vision foundation model for clinical dermatology.","author":"S Yan","year":"2025 June 6","journal-title":"Nat. Med."},{"key":"ref36","doi-asserted-by":"publisher","first-page":"5649","DOI":"10.1038\/s41467-024-50043-3","article-title":"Pre-trained multimodal large language model enhances dermatological diagnosis using SkinGPT-4.","volume":"15","author":"J Zhou","year":"2024 July 5","journal-title":"Nat. Commun."},{"key":"ref37","article-title":"MTEB: Massive Text Embedding Benchmark.","author":"N Muennighoff","year":"2023 cited 2025 June 19","journal-title":"arXiv."},{"key":"ref38","article-title":"Towards Realization of Augmented Intelligence in Dermatology: Advances and Future Directions.","author":"R Daneshjou","year":"2021 cited 2025 June 19","journal-title":"arXiv.org."},{"key":"ref39","doi-asserted-by":"publisher","first-page":"205","DOI":"10.1038\/s41746-024-01208-3","article-title":"Evaluating multimodal AI in medical diagnostics.","volume":"7","author":"R Kaczmarczyk","year":"2024 Aug 7","journal-title":"npj Digit Med."},{"key":"ref40","article-title":"Multimodal Machine Learning Approach for Diagnosing Atopic Dermatitis.","year":"2025 cited 2025 Aug 17","journal-title":"figshare"},{"key":"ref41","article-title":"AlidaWidiawaty\/multimodal-skin-lesion-classification.","year":"2025 cited 2025 Aug 29"},{"key":"ref42","article-title":"AlidaWidiawaty\/multimodal-dermatitis-classification-anamnesys.","year":"2025 cited 2025 Aug 29"},{"key":"ref43","article-title":"AlidaWidiawaty\/multimodal-skin-lesion-classification: v1.0.0.","year":"2025 cited 2025 Aug 29","journal-title":"Zenodo."},{"key":"ref44","article-title":"AlidaWidiawaty\/multimodal-dermatitis-classification-anamnesys: v1.0.0.","year":"2025 cited 2025 Aug 29","journal-title":"Zenodo."}],"container-title":["F1000Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/f1000research.com\/articles\/14-952\/v2\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/f1000research.com\/articles\/14-952\/v2\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/f1000research.com\/articles\/14-952\/v2\/iparadigms","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,17]],"date-time":"2026-02-17T03:14:10Z","timestamp":1771298050000},"score":1,"resource":{"primary":{"URL":"https:\/\/f1000research.com\/articles\/14-952\/v2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,9]]},"references-count":43,"URL":"https:\/\/doi.org\/10.12688\/f1000research.169102.2","relation":{"has-version":[{"id-type":"doi","id":"10.12688\/f1000research.169102.1","asserted-by":"subject"}],"has-review":[{"id-type":"doi","id":"10.5256\/f1000research.195755.r456887","asserted-by":"subject"},{"id-type":"doi","id":"10.5256\/f1000research.195755.r456887","asserted-by":"object"}]},"ISSN":["2046-1402"],"issn-type":[{"value":"2046-1402","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,9]]},"assertion":[{"value":"Indexed","URL":"https:\/\/f1000research.com\/articles\/14-952\/v2#article-reports","order":0,"name":"referee-status","label":"Referee status","group":{"name":"current-referee-status","label":"Current Referee Status"}},{"value":"10.5256\/f1000research.186388.r420623, Sebastian Sitaru, Technical University of Munich, Munich, Munich, 29 Oct 2025, version 1, 1 approved, 1 approved with reservations","URL":"https:\/\/f1000research.com\/articles\/14-952\/v1#referee-response-420623","order":0,"name":"referee-response-420623","label":"Referee Report","group":{"name":"article-reports","label":"Article Reports"}},{"value":"<b>Alida Widiawaty<\/b>; \n<i>Posted: 25 Jan 2026<\/i>; Dear Sebastian Sitaru Technical University of Munich, Munich, Germany  We thank the reviewer for the thorough evaluation of our manuscript and for the constructive comments that have substantially improved the clarity, transparency, and methodological rigor of the study. All points raised have been carefully addressed, and the manuscript has been revised accordingly.  \n<b>1. Clarification of annotation scope <\/b>  Reviewer comment: In the methods it says, &quot;Two board-certified dermatologists independently annotated all images&quot;. Do you mean all cases, so image + text? Or did they just annotate the images alone? Response: Thank you for this insightful comment. We have clarified this point in the manuscript to state that two board-certified dermatologists independently annotated both the clinical images and the corresponding text data for all cases.   \n<b>2. Clarification of patients' skin tone<\/b>  Reviewer comment: Was the skin tone of the patients recorded? Even if not, I think this point deserves a mention in the discussion Response:&nbsp; Thank you for this comment. Skin tone was not explicitly recorded as a standalone variable in this study. However, the study population predominantly consisted of Indonesian patients, who generally fall within Fitzpatrick skin phototypes III\u2013IV. Within this relatively homogeneous skin tone range, no systematic variation in clinical presentation or image-based model performance attributable to skin tone was observed. Therefore, skin tone was not considered a confounding factor influencing the study outcomes. Nonetheless, we acknowledge in the Discussion that external validation in cohorts with a broader range of Fitzpatrick skin types would be valuable to confirm the model\u2019s robustness across more diverse populations.   We have revised the manuscript in accordance with all the reviewers\u2019 comments. Thank you once again for your constructive guidance. Please let us know if any further clarification or revisions are required.  Warm regards, Alida Widiawaty Faculty of Medicine, Universitas Riau, Pekanbaru, 28133, Indonesia The Doctoral Program in Medical Sciences, Faculty of Medicine Universitas Indonesia, Jakarta, 10430, Indonesia","URL":"https:\/\/f1000research.com\/articles\/14-952\/v2#referee-comment-15338","order":1,"name":"referee-comment-15338","label":"Referee Comment","group":{"name":"article-reports","label":"Article Reports"}},{"value":"10.5256\/f1000research.195755.r456887, Tomasz Krzywicki, University of Warmia and Mazury in Olsztyn, Olsztyn, Warmian-Masurian Voivodeship, 17 Feb 2026, version 2, indexed","URL":"https:\/\/f1000research.com\/articles\/14-952\/v2#referee-response-456887","order":2,"name":"referee-response-456887","label":"Referee Report","group":{"name":"article-reports","label":"Article Reports"}},{"value":"This study was supported by the Indonesian Education Scholarship, the Center for Higher Education Funding and Assessment, and the Indonesian Endowment Fund for Education. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.","order":3,"name":"grant-information","label":"Grant Information"},{"value":"This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.","order":0,"name":"copyright-info","label":"Copyright"}]}}